Optical flow speed estimation method and system based on image scale invariance

Abstract

The invention discloses an optical flow speed estimation method and system based on image scale invariance, and relates to the technical field of image processing, and the method comprises the steps: collecting image data through a camera, carrying out the preprocessing, constructing a pyramid according to the preprocessed image data, and generating a descriptor; calculating a matching error based on the generated descriptor, constructing an optical flow estimation model through the matching error, and then outputting a preliminary result; according to the method, the image pyramid structure and the multi-layer descriptor generation technology are combined, the problems that an existing optical flow estimation method is insufficient in precision and poor in robustness in a dynamic scene are solved, in the generation process of the pyramid structure, the robustness of the optical flow estimation method is improved, and the robustness of the optical flow estimation method is improved. According to the method, a weighting method based on feature complexity and edge response intensity is adopted, the image feature extraction precision under multiple scales is effectively improved, and accurate estimation of the optical flow speed on different scales is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an optical flow velocity estimation method and system based on image scale invariance. Background Art

[0002] As one of the basic problems in computer vision, optical flow estimation has been widely used in many fields such as object tracking, motion analysis, autonomous driving, and robot navigation. Accurate estimation of optical flow velocity depends on extracting motion information from continuous image sequences, which is usually achieved by analyzing the intensity changes of image pixels. However, existing optical flow estimation methods usually have the following problems: On the one hand, most traditional methods rely on the brightness consistency assumption, which often leads to large estimation errors in the case of poor image quality or large illumination changes; on the other hand, since most traditional methods fail to fully consider the multi-scale characteristics of images, they cannot fully capture motion information when dealing with objects of different sizes and speeds, resulting in low accuracy and poor robustness. Summary of the invention

[0003] In view of the above existing problems, the present invention is proposed.

[0004] Therefore, the present invention provides an optical flow velocity estimation method and system based on image scale invariance to solve the problems that most of the existing traditional methods rely on the brightness consistency assumption, which often leads to large estimation errors and fails to fully consider the multi-scale characteristics of the image when the image quality is poor or the illumination changes greatly. As a result, they cannot fully capture the motion information when processing the movement of objects of different sizes and speeds, and have low accuracy and poor robustness.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides an optical flow velocity estimation method based on image scale invariance, which comprises: The image data is collected by the camera for preprocessing, and a pyramid is constructed and a descriptor is generated based on the preprocessed image data; Calculate the matching error based on the generated descriptors, and use the matching error to build an optical flow estimation model and output preliminary results; Based on the output preliminary results, the residual is calculated and then fused and optimized to obtain the final result: The final results are stored and displayed through a visual interface.

[0006] As a preferred solution of the optical flow velocity estimation method based on image scale invariance of the present invention, the preprocessing of image data collected by a camera includes: Capture image data by using a high frame rate camera and record the image resolution and size; Use histogram equalization technology to enhance the contrast of the acquired image data; The enhanced image is decomposed into sub-images of different frequency bands by two-dimensional wavelet transform, and the high-frequency components in the sub-images are removed by soft threshold method. The sub-images after high-frequency components are removed are reconstructed into denoised images by wavelet reconstruction method. The denoised image is enhanced using the Laplace method to obtain an enhanced image.

[0007] As a preferred solution of the optical flow velocity estimation method based on image scale invariance of the present invention, the step of constructing a pyramid based on preprocessed image data and generating a descriptor comprises: The obtained enhanced image is used as the initial layer of the pyramid, and the maximum number of layers of the pyramid is calculated according to the recorded image resolution: The enhanced image of the pyramid level l-1 Divide into m×n areas by grid; Using the sliding window technique, the local variance within each grid is calculated; Based on the obtained local variance, the blur intensity of the enhanced image at the current pyramid level is calculated: , In the formula, represents the blur intensity of the enhanced image at the lth level of the pyramid, Indicates the basic blur strength, represents the variance within the i-th and j-th grids, Represents the maximum value among all grid variances; According to the obtained blur intensity, the enhanced image of the current pyramid layer is Gaussian filtered using the Gaussian filtering method to obtain a blurred image. : Use the downsampling method to downsample the blurred image and generate the next layer of blurred image : Based on the generated blurred image, the Sobel operator method is used to calculate the gradient and amplitude, and then the feature complexity of the blurred image of the pyramid layer l is further calculated. : Based on the gradient and amplitude, the edge response intensity of the blurred image at the lth layer of the pyramid is further calculated. : Using a normalization method to normalize the obtained feature complexity and edge response strength, to obtain normalized feature complexity and edge response strength; The normalized feature complexity and edge response strength are weighted and summed using the weighted summation method to obtain the final feature complexity of the current pyramid layer; The final feature complexity is accumulated layer by layer, and the cumulative summation formula is used to sum it up to get the cumulative feature complexity: Set the target threshold to , the cumulative feature complexity and the threshold Compare, when the cumulative feature complexity is greater than or equal to the threshold When , stop generating new pyramid levels and generate pyramid images; When the cumulative feature complexity is less than the threshold When , the pyramid level is increased, and the feature complexity is calculated and added to the cumulative feature complexity; Based on the pyramid image, the Gaussian difference method is used to detect the response value of each layer of pixels in the pyramid image: , In the formula, Indicates that the pixel is at scale The response value under Represents the pyramid image at scale The fuzzy response under, k represents the scale increment factor; According to the obtained pixel response value, the non-maximum suppression method is used to select local extreme points, and the selected local extreme points are used as potential feature points; Taking the potential feature point as the center, select a neighborhood M of g pixels within a radius, and divide the neighborhood M into z×z sub-regions; According to the obtained potential feature points, the Sobel operator method is used to calculate the gradient direction and amplitude of the pixels in the neighborhood of the potential feature points; According to the gradient direction of each pixel in the neighborhood, the amplitude is weighted and accumulated in the corresponding sub-region direction: , In the formula, represents the weighted histogram of sub-region directions, represents a sub-region of a pixel, represents the gradient magnitude of pixels in the neighborhood, and represents the center coordinates of potential feature points, represents the standard deviation of the weight distribution; In the sub-region direction, the Haar wavelet transform is used to find the direction with the largest amplitude, and the direction with the largest amplitude is taken as the main direction of the potential feature point; According to the main direction of the potential feature points obtained, all pixels in the neighborhood are rotated and normalized using the rotation normalization method; The rotated and normalized neighborhood is divided into z×z sub-regions, and the gradient histogram of z square directions is generated in each sub-region to obtain Dimensional feature descriptor.

[0008] As a preferred solution of the optical flow velocity estimation method based on image scale invariance of the present invention, the method of calculating the matching error based on the generated descriptor and constructing the optical flow estimation model through the matching error and then outputting the preliminary result comprises: based on Dimensional feature descriptor definition feature descriptor matching error : Construct an optical flow estimation model based on the matching error and define the objective function: , , In the formula, It means optimizing the horizontal component u and vertical component v of the optical flow field and finding u and v that minimize the objective function value. represents the Gaussian smoothing kernel, Represents a pixel, Indicates the time change of pixel intensity between the current frame and the next frame, and Respectively represent the spatial gradients of the pyramid image in the horizontal direction s and the vertical direction o, represents the weight coefficient of the regularization term, represents the square of the feature descriptor error; The Adam optimizer is used to iteratively optimize the parameters of the optical flow estimation model. During the training process, the average endpoint error is calculated by layer-by-layer propagation. During the continuous iteration process, when the loss of the optical flow estimation model no longer decreases significantly, the iteration is stopped and the final optical flow estimation model is output to complete the optimization. The pyramid image is input into the optical flow estimation model, and the optical flow estimation model outputs the horizontal and vertical components of the optical flow.

[0009] As a preferred solution of the optical flow velocity estimation method based on image scale invariance of the present invention, wherein: the residual is calculated based on the output preliminary result, and then fused and optimized to obtain the final result, including: Use linear interpolation to temporally align the horizontal and vertical components of optical flow and feature matching errors; Based on the aligned horizontal and vertical components of the optical flow, the Horn-Schunck method is used to calculate and obtain the optical flow residual; Based on the aligned feature matching error, the residual calculation method is used to calculate the feature matching residual; Image processing technology is used to extract high-frequency features in the optical flow residual for compensation, and the compensated high-frequency features are superimposed with the optical flow residual to obtain the compensated optical flow residual: In the formula, represents the compensated optical flow residual, represents the optical flow residual, represents the compensation coefficient, Represents the extracted high-frequency features; The local variance of the image area is obtained by using the sliding window technique. The dynamic uncertainty in the compensated optical flow residual is quantified according to the obtained local variance, and then the dynamic weights of the compensated optical flow residual and the feature matching residual are calculated: , , In the formula, represents the dynamic weight for compensating the optical flow residual, represents the dynamic weight of feature matching residual, represents the regularization parameter, represents the mean of the compensated optical flow residual in the local window, Represents the mean of feature matching residuals within the local window; The compensated optical flow residual and the feature matching residual are normalized by using a normalization method to obtain the normalized compensated optical flow residual and the feature matching residual; Based on the normalized compensated optical flow residual and feature matching residual, the residuals are weighted fused using a weighted fusion method to obtain a fused residual; A residual optimization model is constructed based on the normalized compensated optical flow residual and feature matching residual, and the objective function is defined: , In the formula, represents the objective function of optical flow residual optimization, T represents the number of time frames, represents the set of pixels in the optimization area, Represents the normalized compensated optical flow residual; The inverse variance method is used for weight allocation, and the gradient descent method is used for parameter optimization. During the iteration process, when the loss no longer decreases significantly, the iteration is stopped and convergence is completed, and the final residual optimization model is output to complete the optimization; The fused residual is input into the residual optimization model, and the optimized residual is output through the residual optimization model to obtain the optimized optical flow field.

[0010] As a preferred solution of the optical flow velocity estimation method based on image scale invariance of the present invention, wherein: storing the final result includes: Based on the obtained optimized optical flow field, a serialization method is used to convert the optimized optical flow field into a binary format file; According to the converted binary format file, Python is used to compress the binary format file, and the compressed file is stored in the database.

[0011] As a preferred solution of the optical flow velocity estimation method based on image scale invariance of the present invention, the display through a visual interface includes: Based on the stored binary format files, use the zipfile tool to decompress the compressed files; Read the decompressed file using the binary reading method and draw an arrow graph using the quiver function; The arrows are color-coded based on their length: longer arrows indicate stronger movements and are colored red, while shorter arrows indicate weaker movements and are colored blue.

[0012] In a second aspect, the present invention provides an optical flow velocity estimation system based on image scale invariance, comprising: The acquisition and generation module is used to preprocess the acquired image data and generate descriptors after building a pyramid; The error and output module is used to calculate the matching error of the descriptor, build the optical flow estimation module and output the preliminary results; The fusion and optimization module is used to calculate the residuals based on the preliminary results and to fuse and optimize the residuals; The storage and display module is used to store data and perform visual display.

[0013] The beneficial effects of the present invention are as follows: the present invention solves the problems of insufficient accuracy and poor robustness of existing optical flow estimation methods in dynamic scenes by combining an image pyramid structure with a multi-layer descriptor generation technology. In the process of generating the pyramid structure, the present invention adopts a weighted method based on feature complexity and edge response strength, which effectively improves the accuracy of image feature extraction at multiple scales and ensures accurate estimation of optical flow speed at different scales. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0015] Figure 1 This is a flow chart of the optical flow velocity estimation method based on image scale invariance in the present invention.

[0016] Figure 2It is a structural schematic diagram of the optical flow velocity estimation system based on image scale invariance in the present invention. DETAILED DESCRIPTION

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0018] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0019] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0020] Example, see Figure 1 and Figure 2 This embodiment provides an optical flow velocity estimation method based on image scale invariance, comprising the following steps: S1, collect image data through the camera for preprocessing, build a pyramid based on the preprocessed image data and generate a descriptor; Specifically, the preprocessing of image data collected by the camera includes: Capture image data by using a high frame rate camera and record the image resolution and size; Use histogram equalization technology to enhance the contrast of the acquired image data; The enhanced image is decomposed into sub-images of different frequency bands by two-dimensional wavelet transform, and the high-frequency components in the sub-images are removed by soft threshold method. The sub-images after high-frequency components are removed are reconstructed into denoised images by wavelet reconstruction method. The denoised image is enhanced using the Laplace method to obtain an enhanced image.

[0021] By using a high frame rate camera, we ensure that high-quality image data in dynamic scenes is captured, providing accurate input for subsequent processing. Histogram equalization is used to enhance image contrast and improve detail performance. Two-dimensional wavelet transform combined with soft threshold denoising removes high-frequency noise while retaining image details to avoid information loss. Wavelet reconstruction effectively restores the denoised image information and ensures image quality. The Laplace method further enhances the edges and details of the image to ensure image clarity.

[0022] Further, constructing a pyramid and generating a descriptor based on the preprocessed image data includes: The obtained enhanced image is used as the initial layer of the pyramid, and the maximum number of layers of the pyramid is calculated according to the recorded image resolution: , Where L represents the maximum number of pyramid layers, H represents the height of the input image, and W represents the width of the input image; The enhanced image of the pyramid level l-1 Divide into m×n areas by grid; Using the sliding window technique, the local variance within each grid is calculated; Based on the obtained local variance, the blur intensity of the enhanced image at the current pyramid level is calculated: , In the formula, represents the blur intensity of the enhanced image at the lth level of the pyramid, Indicates the basic blur strength, represents the variance within the i-th and j-th grids, Represents the maximum value among all grid variances; The basic blur intensity can be set by combining the domain knowledge of optical image processing ; According to the obtained blur intensity, the enhanced image of the current pyramid layer is Gaussian filtered using the Gaussian filtering method to obtain a blurred image: , In the formula, represents the blurred image of the lth level of the pyramid, represents the Gaussian filter kernel, represents the enhanced image of the l-1th layer of the pyramid, represents the convolution operator; The Gaussian filter kernel can be set according to the needs ; The blurred image is downsampled using the downsampling method, and the next layer of blurred image is generated: , In the formula, represents the next layer of blurred image, represents the downsampling operation of the blurred image; Based on the generated blurred image, the Sobel operator method is used for calculation, and after obtaining the gradient and amplitude, the feature complexity of the blurred image of the lth layer of the pyramid is further calculated: , In the formula, represents the feature complexity of the blurred image at the lth level of the pyramid, represents the gradient of the blurred image at the lth level of the pyramid in the horizontal direction s, Represents the gradient of the blurred image at the lth layer of the pyramid in the vertical direction o, Represents the total number of pixels of the blurred image at layer l; Based on the gradient and amplitude, the edge response intensity of the blurred image at the lth layer of the pyramid is further calculated: , In the formula, Represents the edge response strength of the blurred image at the lth level of the pyramid; Using a normalization method to normalize the obtained feature complexity and edge response strength, to obtain normalized feature complexity and edge response strength; The normalized feature complexity and edge response strength are weighted and summed using the weighted summation method to obtain the final feature complexity of the current pyramid layer; The final feature complexity is accumulated layer by layer, and the cumulative summation formula is used to sum it up to get the cumulative feature complexity: The target threshold is set according to the needs and relevant experience. , the cumulative feature complexity and the threshold Compare, when the cumulative feature complexity is greater than or equal to the threshold When , stop generating new pyramid levels and generate pyramid images; When the cumulative feature complexity is less than the threshold When , the pyramid level is increased, and the feature complexity is calculated and added to the cumulative feature complexity; Based on the pyramid image, the Gaussian difference method is used to detect the response value of each layer of pixels in the pyramid image: , In the formula, Indicates that the pixel is at scale The response value under Represents the pyramid image at scale The fuzzy response under, k represents the scale increment factor; The scale increment factor k can be determined by referring to the typical value in the SIFT algorithm; According to the obtained pixel response value, the non-maximum suppression method is used to select local extreme points, and the selected local extreme points are used as potential feature points; Taking the potential feature point as the center, select a neighborhood M of g pixels within a radius, and divide the neighborhood M into z×z sub-regions; According to the obtained potential feature points, the Sobel operator method is used to calculate the gradient direction and amplitude of the pixels in the neighborhood of the potential feature points; According to the gradient direction of each pixel in the neighborhood, the amplitude is weighted and accumulated in the corresponding sub-region direction: , In the formula, represents the weighted histogram of sub-region directions, represents a sub-region of a pixel, represents the gradient magnitude of pixels in the neighborhood, and represents the center coordinates of potential feature points, represents the standard deviation of the weight distribution; In the sub-region direction, the Haar wavelet transform is used to find the direction with the largest amplitude, and the direction with the largest amplitude is taken as the main direction of the potential feature point; According to the main direction of the potential feature points obtained, all pixels in the neighborhood are rotated and normalized using the rotation normalization method; The rotated and normalized neighborhood is divided into z×z sub-regions, and the gradient histogram of z square directions is generated in each sub-region to obtain Dimensional feature descriptor.

[0023] The generation of a pyramid structure and calculation of descriptors enable the present invention to extract high-quality feature points at different scales, and the multi-level characteristics of the pyramid structure effectively cope with changes in images at different resolutions, viewing angles and sizes, so that the present invention has strong scale invariance, and by processing images at multiple levels, the present invention can extract rich feature information from image details at different levels, providing a more comprehensive and accurate image representation for subsequent feature matching and optical flow estimation, and accurately detecting and optimizing the blur degree of the image by calculating the local variance and blur intensity. At each layer of the pyramid, the local variance is used to evaluate the clarity of the image, and the Gaussian filtering method is used to blur the image based on the calculation of the blur intensity. The processing method of this step ensures the clarity and detail retention of the image at different levels, effectively reduces the risk of mismatching due to image blur, and improves the subsequent The accuracy of feature point extraction. In terms of feature point detection, the present invention adopts non-maximum suppression technology, which can accurately select local maxima as feature points. This technology greatly reduces the interference of noise points and pseudo feature points, and improves the stability and accuracy of feature points. By combining different scales and fuzzy image features in the pyramid structure, non-maximum suppression can further optimize the selection of feature points and improve the reliability of the present invention in complex scenes. Secondly, the present invention calculates the gradient direction and amplitude of potential feature points, and determines the main direction based on the Haar wavelet transform to generate a rotation-invariant dimensional feature descriptor. The main direction calculation and rotation normalization processing of the feature points ensure the stability of the feature point descriptor at different rotation angles, thereby improving the feature matching accuracy at different viewing angles. This step enables the present invention to maintain a high matching effect and image stability under the conditions of rotation, scale change and different viewing angles.

[0024] S2, calculate the matching error based on the generated descriptor, and output the preliminary result after constructing the optical flow estimation model through the matching error; Specifically, the matching error is calculated based on the generated descriptors, and the optical flow estimation model is constructed through the matching error to output preliminary results including: based on Dimensional feature descriptor defines feature descriptor matching error: , In the formula, represents the feature matching error, and denote the i-th and j-th feature descriptors respectively, represents the total dimension of the feature descriptor, and denote the z-th eigenvalues ​​of the i-th and j-th elements respectively; Construct an optical flow estimation model based on the matching error and define the objective function: , , In the formula, It means optimizing the horizontal component u and vertical component v of the optical flow field and finding u and v that minimize the objective function value. represents the Gaussian smoothing kernel, Represents a pixel, Indicates the time change of pixel intensity between the current frame and the next frame, and Respectively represent the spatial gradients of the pyramid image in the horizontal direction s and the vertical direction o, represents the brightness consistency constraint of the optical flow, represents the weight coefficient of the regularization term, represents the square of the feature descriptor error; The Gaussian smoothing kernel can be obtained by empirical or adaptive methods. ; The Adam optimizer is used to iteratively optimize the parameters of the optical flow estimation model. During the training process, the average endpoint error is calculated by layer-by-layer propagation. During the continuous iteration process, when the loss of the optical flow estimation model no longer decreases significantly, the iteration is stopped and the final optical flow estimation model is output to complete the optimization. The pyramid image is input into the optical flow estimation model, and the optical flow estimation model outputs the horizontal and vertical components of the optical flow.

[0025] By utilizing the matching error of feature descriptors, the similarity between different feature points in the image can be accurately evaluated, thereby improving the accuracy of feature matching, especially in complex scenes, and maintaining high robustness. Secondly, the optical flow estimation model effectively constrains the smoothness and consistency of the optical flow field by introducing Gaussian smoothing kernels and brightness consistency constraints, avoiding mismatching and unreasonable optical flow changes, and ensuring more accurate motion estimation. The use of pyramid image structure for multi-scale processing enables the optical flow estimation to be gradually refined at different resolutions, improving computational efficiency and accuracy. The Adam optimizer is used to optimize the parameters of the optical flow estimation model, further improving the stability and convergence speed of the training process, and ensuring accurate optical flow estimation results.

[0026] S3. Calculate the residual based on the output preliminary results, perform fusion and optimization, and get the final result: Specifically, the residual is calculated based on the output preliminary results, and then fusion and optimization are performed to obtain the final results including: Use linear interpolation to temporally align the horizontal and vertical components of optical flow and feature matching errors; Based on the aligned horizontal and vertical components of the optical flow, the Horn-Schunck method is used to calculate and obtain the optical flow residual; Based on the aligned feature matching error, the residual calculation method is used to calculate the feature matching residual; Image processing technology is used to extract high-frequency features in the optical flow residual for compensation, and the compensated high-frequency features are superimposed with the optical flow residual to obtain the compensated optical flow residual: In the formula, represents the compensated optical flow residual, represents the optical flow residual, represents the compensation coefficient, Represents the extracted high-frequency features; The local variance of the image area is obtained by using the sliding window technique. The dynamic uncertainty in the compensated optical flow residual is quantified according to the obtained local variance, and then the dynamic weights of the compensated optical flow residual and the feature matching residual are calculated: , , In the formula, represents the dynamic weight for compensating the optical flow residual, represents the dynamic weight of feature matching residual, represents the regularization parameter, represents the mean of the compensated optical flow residual in the local window, Represents the mean of feature matching residuals within the local window; The compensated optical flow residual and the feature matching residual are normalized by using a normalization method to obtain the normalized compensated optical flow residual and the feature matching residual; Based on the normalized compensated optical flow residual and feature matching residual, the residuals are weighted fused using a weighted fusion method to obtain a fused residual; A residual optimization model is constructed based on the normalized compensated optical flow residual and feature matching residual, and the objective function is defined: , In the formula, represents the objective function of optical flow residual optimization, T represents the number of time frames, represents the set of pixels in the optimization area, Represents the normalized compensated optical flow residual; The inverse variance method is used for weight allocation, and the gradient descent method is used for parameter optimization. During the iteration process, when the loss no longer decreases significantly, the iteration is stopped and convergence is completed, and the final residual optimization model is output to complete the optimization; The fused residual is input into the residual optimization model, and the optimized residual is output through the residual optimization model to obtain the optimized optical flow field.

[0027] By using time alignment, residual compensation, and dynamic weighting technology when processing image sequences, the consistency of data between different frames is effectively guaranteed, and the results of optical flow estimation are optimized. The horizontal and vertical components of the optical flow and the feature matching errors are time-aligned through linear interpolation, and the optical flow residual is calculated using the Horn-Schunck method to accurately extract motion information. The subsequent extraction and compensation of high-frequency features help to improve the estimation accuracy of detail areas. The dynamic uncertainty is quantified through the sliding window technology, and the residuals are fused using dynamic weights to ensure that the optical flow estimation accuracy in different areas is optimized. By constructing a residual optimization model and minimizing the loss function through the gradient descent method, the optical flow field is accurately optimized. Finally, after multiple optimization steps, the optical flow field obtained can more accurately reflect the motion information of objects in the image sequence, especially in high-dynamic scenes.

[0028] S4, storing the final result and displaying it through a visual interface; Specifically, storing the final result includes: Based on the obtained optimized optical flow field, a serialization method is used to convert the optimized optical flow field into a binary format file; According to the converted binary format file, Python is used to compress the binary format file, and the compressed file is stored in the database.

[0029] The optimized optical flow field data is converted into a compact binary format through serialization method for easy storage and transmission. Then, compression technology is used to further reduce the storage space requirements and improve the data transmission and loading speed. The compressed data can effectively support the efficient storage of large-scale image data, which is particularly suitable for data-intensive applications that require fast processing and access. Finally, the compressed optical flow data is stored in the database, making data management more efficient and supporting fast query and operation.

[0030] Furthermore, the display through the visual interface includes: Based on the stored binary format files, use the zipfile tool to decompress the compressed files; Read the decompressed file using the binary reading method and draw an arrow graph using the quiver function; The arrows are color-coded based on their length: longer arrows indicate stronger movements and are colored red, while shorter arrows indicate weaker movements and are colored blue.

[0031] The zipfile tool is used to compress and store optical flow data to save storage space, and binary format files are used to efficiently transmit and read data. During the display process, the quiver function is used to draw arrow diagrams to intuitively display the direction and intensity of movement, and the movement intensity is corresponded to the arrow length through color coding. Red indicates strong movement and blue indicates weak movement, which enhances the visual effect. In addition, this step has high real-time performance and can quickly reflect the dynamic changes in the optical flow field.

[0032] This embodiment also provides an optical flow velocity estimation system based on image scale invariance, including: The acquisition and generation module is used to preprocess the acquired image data and generate descriptors after building a pyramid; The error and output module is used to calculate the matching error of the descriptor, build the optical flow estimation module and output the preliminary results; The fusion and optimization module is used to calculate the residuals based on the preliminary results and to fuse and optimize the residuals; The storage and display module is used to store data and perform visual display.

[0033] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for estimating optical flow velocity based on image scale invariance, characterized in that: include: The image data is collected by the camera for preprocessing, and a pyramid is constructed and a descriptor is generated based on the preprocessed image data; Calculate the matching error based on the generated descriptors, and use the matching error to build an optical flow estimation model and output preliminary results; Based on the output preliminary results, the residual is calculated and then fused and optimized to obtain the final result: The final results are stored and displayed through a visual interface.

2. The optical flow velocity estimation method based on image scale invariance according to claim 1, characterized in that: The preprocessing of the image data collected by the camera includes: Capture image data by using a high frame rate camera and record the image resolution and size; Use histogram equalization technology to enhance the contrast of the acquired image data; The enhanced image is decomposed into sub-images of different frequency bands by two-dimensional wavelet transform, and the high-frequency components in the sub-images are removed by soft threshold method. The sub-images after high-frequency components are removed are reconstructed into denoised images by wavelet reconstruction method. The denoised image is enhanced using the Laplace method to obtain an enhanced image.

3. The optical flow velocity estimation method based on image scale invariance according to claim 2, characterized in that: The step of constructing a pyramid and generating a descriptor according to the preprocessed image data comprises: The obtained enhanced image is used as the initial layer of the pyramid, and the maximum number of layers of the pyramid is calculated according to the recorded image resolution: The enhanced image of the pyramid level l-1 Divide into m×n areas by grid; Using the sliding window technique, the local variance within each grid is calculated; Based on the obtained local variance, the blur intensity of the enhanced image at the current pyramid level is calculated: ， In the formula, represents the blur intensity of the enhanced image at the lth level of the pyramid, Indicates the basic blur strength, represents the variance within the i-th and j-th grids, Represents the maximum value among all grid variances; According to the obtained blur intensity, the enhanced image of the current pyramid layer is Gaussian filtered using the Gaussian filtering method to obtain a blurred image. : Use the downsampling method to downsample the blurred image and generate the next layer of blurred image : Based on the generated blurred image, the Sobel operator method is used to calculate the gradient and amplitude, and then the feature complexity of the blurred image of the pyramid layer l is further calculated. : Based on the gradient and amplitude, the edge response intensity of the blurred image at the lth layer of the pyramid is further calculated. : Using a normalization method to normalize the obtained feature complexity and edge response strength, to obtain normalized feature complexity and edge response strength; The normalized feature complexity and edge response strength are weighted summed using the weighted summation method to obtain the final feature complexity of the current pyramid layer; The final feature complexity is accumulated layer by layer, and the cumulative summation formula is used to sum it up to get the cumulative feature complexity: Set the target threshold to , the cumulative feature complexity and the threshold Compare, when the cumulative feature complexity is greater than or equal to the threshold When , stop generating new pyramid levels and generate pyramid images; When the cumulative feature complexity is less than the threshold When , the pyramid level is increased, and the feature complexity is calculated and added to the cumulative feature complexity; Based on the pyramid image, the Gaussian difference method is used to detect the response value of each layer of pixels in the pyramid image. : According to the obtained pixel response value, the non-maximum suppression method is used to select local extreme points, and the selected local extreme points are used as potential feature points; Taking the potential feature point as the center, select a neighborhood M of g pixels within a radius, and divide the neighborhood M into z×z sub-regions; According to the obtained potential feature points, the Sobel operator method is used to calculate the gradient direction and amplitude of the pixels in the neighborhood of the potential feature points; According to the gradient direction of each pixel in the neighborhood, the amplitude is weighted and accumulated in the corresponding sub-region direction: ， In the formula, represents the weighted histogram of sub-region directions, represents a sub-region of a pixel, represents the gradient magnitude of pixels in the neighborhood, and represents the center coordinates of potential feature points, represents the standard deviation of the weight distribution; In the sub-region direction, the Haar wavelet transform is used to find the direction with the largest amplitude, and the direction with the largest amplitude is taken as the main direction of the potential feature point; According to the main direction of the potential feature points obtained, all pixels in the neighborhood are rotated and normalized using the rotation normalization method; The rotated and normalized neighborhood is divided into z×z sub-regions, and the gradient histogram of z square directions is generated in each sub-region to obtain Dimensional feature descriptor.

4. The optical flow velocity estimation method based on image scale invariance according to claim 3, characterized in that: The calculation of the matching error based on the generated descriptor and the output of the preliminary result after constructing the optical flow estimation model through the matching error include: based on Dimensional feature descriptor definition feature descriptor matching error : Construct an optical flow estimation model based on the matching error and define the objective function: ， ， In the formula, It means optimizing the horizontal component u and vertical component v of the optical flow field and finding u and v that minimize the objective function value. represents the Gaussian smoothing kernel, Represents a pixel, Indicates the time change of pixel intensity between the current frame and the next frame, and Respectively represent the spatial gradients of the pyramid image in the horizontal direction s and the vertical direction o, represents the weight coefficient of the regularization term, represents the square of the feature descriptor error; The Adam optimizer is used to iteratively optimize the parameters of the optical flow estimation model. During the training process, the average endpoint error is calculated by layer-by-layer propagation. During the continuous iteration process, when the loss of the optical flow estimation model no longer decreases significantly, the iteration is stopped and the final optical flow estimation model is output to complete the optimization. The pyramid image is input into the optical flow estimation model, and the optical flow estimation model outputs the horizontal and vertical components of the optical flow.

5. The optical flow velocity estimation method based on image scale invariance according to claim 4, characterized in that: The residual is calculated based on the output preliminary result, and then fused and optimized to obtain the final result including: Use linear interpolation to temporally align the horizontal and vertical components of optical flow and feature matching errors; Based on the aligned horizontal and vertical components of the optical flow, the Horn-Schunck method is used to calculate and obtain the optical flow residual; Based on the aligned feature matching error, the residual calculation method is used to calculate the feature matching residual; Image processing technology is used to extract high-frequency features in the optical flow residual for compensation, and the compensated high-frequency features are superimposed with the optical flow residual to obtain the compensated optical flow residual: In the formula, represents the compensated optical flow residual, represents the optical flow residual, represents the compensation coefficient, Represents the extracted high-frequency features; The local variance of the image area is obtained by using the sliding window technique. The dynamic uncertainty in the compensated optical flow residual is quantified according to the obtained local variance, and then the dynamic weights of the compensated optical flow residual and the feature matching residual are calculated: ， , In the formula, represents the dynamic weights for compensating the optical flow residual, represents the dynamic weight of feature matching residual, represents the regularization parameter, represents the mean of the compensated optical flow residual in the local window, Represents the mean of feature matching residuals within the local window; The compensated optical flow residual and the feature matching residual are normalized by using a normalization method to obtain the normalized compensated optical flow residual and the feature matching residual; Based on the normalized compensated optical flow residual and feature matching residual, the residuals are weighted fused using a weighted fusion method to obtain a fused residual; A residual optimization model is constructed based on the normalized compensated optical flow residual and feature matching residual, and the objective function is defined: , In the formula, represents the objective function of optical flow residual optimization, T represents the number of time frames, represents the set of pixels in the optimization area, Represents the normalized compensated optical flow residual; The inverse variance method is used for weight allocation, and the gradient descent method is used for parameter optimization. During the iteration process, when the loss no longer decreases significantly, the iteration is stopped and convergence is completed, and the final residual optimization model is output to complete the optimization; The fused residual is input into the residual optimization model, and the optimized residual is output through the residual optimization model to obtain the optimized optical flow field.

6. The optical flow velocity estimation method based on image scale invariance according to claim 5, characterized in that: The storing of the final result comprises: Based on the obtained optimized optical flow field, a serialization method is used to convert the optimized optical flow field into a binary format file; According to the converted binary format file, Python is used to compress the binary format file, and the compressed file is stored in the database.

7. The optical flow velocity estimation method based on image scale invariance according to claim 6, characterized in that: The display through the visual interface includes: Based on the stored binary format files, use the zipfile tool to decompress the compressed files; Read the decompressed file using the binary reading method and draw an arrow graph using the quiver function; The arrows are color-coded based on their length: longer arrows indicate stronger movements and are colored red, while shorter arrows indicate weaker movements and are colored blue.

8. An optical flow velocity estimation system based on image scale invariance, based on the optical flow velocity estimation method based on image scale invariance according to any one of claims 1 to 7, characterized in that: include: The acquisition and generation module is used to preprocess the acquired image data and generate descriptors after building a pyramid; The error and output module is used to calculate the matching error of the descriptor, then build the optical flow estimation module and output the preliminary results; The fusion and optimization module is used to calculate the residuals based on the preliminary results and to fuse and optimize the residuals; The storage and display module is used to store data and perform visual display.

Images