Curvature enhanced large displacement image based variational optical flow method for motion estimation

Abstract

The application provides a curvature-enhanced large displacement image variational optical flow method for motion estimation, relates to the field of image processing, and aims to depict the complexity of local structures of an image by introducing the curvature of the image contour line, and to restrict the adaptive weighting adjustment of the brightness invariable constraint and the gradient invariable constraint on the basis of the curvature in the data item level of the variational model of the optical flow, so as to inhibit the interference of unreliable matching pairs on the optical flow estimation in the complex structure area, improve the robustness of the optical flow estimation under the scenes of illumination change, complex texture and large displacement, and guarantee the numerical stability and convergence of the model in the multi-scale calculation process.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and specifically relates to a curvature-enhanced variational optical flow method for large displacement images oriented towards motion estimation. Background Technology

[0002] Optical flow estimation is a fundamental task in computer vision, used to estimate pixel-level motion information from a continuous sequence of images. Traditional variational optical flow methods typically construct an energy function based on the assumptions of brightness invariance or gradient invariance, and introduce regularization terms to smooth the optical flow field. However, when dealing with scenes with large displacement motion, complex texture structures, or significant changes in illumination, traditional methods are prone to mismatches in structurally complex regions, leading to a decrease in the accuracy of optical flow estimation.

[0003] Existing methods often employ layer-by-layer optimization using image pyramid structures or introduce local feature enhancement constraints, but they still do not fully consider the impact of local image geometry on matching reliability. Curvature, as a feature describing the local geometric complexity of an image, can be used to distinguish between flat regions and structurally complex regions, but currently no method has systematically incorporated it into the weighting mechanism of optical flow data terms to adaptively adjust the constraint strength of different regions. To address the above issues, this invention proposes introducing the curvature of image contour lines to characterize the local structural complexity of an image, and based on this curvature, performing constrained adaptive weighting adjustment of brightness-invariant constraints and gradient-invariant constraints at the data term level of the optical flow variational model. This suppresses the interference of unreliable matching in complex structural regions on optical flow estimation, improves the robustness of optical flow estimation under illumination changes, complex textures, and large displacement scenarios, and ensures the numerical stability and convergence of the model during multi-scale computation. Summary of the Invention

[0004] The purpose of this invention is to provide a curvature-enhanced variational optical flow method for large displacement images oriented towards motion estimation. This method characterizes the local structural complexity of an image by introducing the curvature of image contour lines, and then adaptively adjusts the brightness-invariant and gradient-invariant constraints at the data term level of the optical flow variational model based on this curvature. This suppresses the interference of unreliable matching in complex structural regions on optical flow estimation, improves the robustness of optical flow estimation under illumination changes, complex textures, and large displacement scenes, and ensures the numerical stability and convergence of the model during multi-scale computation.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a curvature-enhanced large displacement image variational optical flow method for motion estimation, comprising the following steps.

[0006] S1. Obtain two adjacent input images and construct an image pyramid for the two adjacent input images.

[0007] S2. Select the current pyramid layer for optical flow estimation in the image pyramid, initialize the optical flow field and optical flow increment of this layer, and construct a variational optical flow energy model that includes brightness invariant constraints, gradient invariant constraints, and curvature weighting factors. .

[0008] S3, in the variational optical flow energy model To determine the position of the data item coefficients, an adaptive weight adjustment mechanism based on image curvature is introduced. The image curvature characterizes the complexity of the local structure of the image, and the brightness-invariant constraint term and the gradient-invariant constraint term are adjusted with differentiated weights.

[0009] S4. Construct a curvature weight function that is monotonically decreasing and bounded with respect to the curvature of the image, and introduce an adaptive adjustment factor to dynamically constrain and adjust the curvature weight.

[0010] S5. For the nonlinear variational optical flow model after introducing curvature weighting, the alternating direction multiplier method is used to perform variable splitting and co-optimization on the data terms and the total variational regularization terms to achieve model solution.

[0011] S6. After obtaining the optical flow update result at the current pyramid layer, the optical flow update result is propagated to the higher resolution layer for interpolation, and steps S2 to S5 are repeated until the highest resolution layer completes the calculation.

[0012] S7. After completing the incremental iterative update of optical flow at the highest resolution layer, the final optical flow field is output to characterize the pixel-level motion information between two adjacent input images.

[0013] Preferably, in step S1, two adjacent input frames are acquired. and The two adjacent input frames are preprocessed to reduce the impact of noise and illumination variations on optical flow estimation. Then, an image pyramid is constructed consisting of N layers of images at different resolutions, with the lowest resolution image at the top and the highest resolution image at the bottom. Initialize the horizontal component of the optical flow field corresponding to the current pyramid level. and vertical components and the corresponding optical flow increment and After completing the optical flow estimation of the initial optical flow calculation layer, the optical flow field obtained from this layer is used as the initial value for optical flow calculation of the next higher resolution image. The optical flow is then estimated layer by layer in the order from low resolution layer to high resolution layer, so as to realize the step-by-step estimation of large displacement optical flow.

[0014] Preferably, in step S2, a variational optical flow energy model incorporating brightness-invariant constraints and gradient-invariant constraints is constructed. The specific steps are as follows:

[0015] S21. Initialize the optical flow field of the two input images at the current image pyramid level to obtain the horizontal component of optical flow. and vertical components and the corresponding optical flow increment and Partial derivatives are calculated for the two input images of the current pyramid layer, where the first-order spatial partial derivative of the input image in the horizontal direction is calculated based on the brightness invariance constraint. First-order spatial partial derivative in the vertical direction And the partial derivative of brightness change along the time axis Based on gradient invariance constraints, the spatial gradient of the input image is calculated to obtain the first... Gradient partial derivative of the input frame image in the horizontal direction Gradient partial derivative in the vertical direction And the corresponding gradient change term over time. ,in =1 and 2 represent the first and second frame input images, respectively;

[0016] S22. Simultaneously introduce brightness invariance constraints and gradient invariance constraints into the current pyramid layer to construct a variational optical flow energy function that includes a curvature weighting factor. It is used to measure the motion consistency between adjacent image pixels, and its functional form is defined as: ,in, This represents the gradient vector of the brightness of the input image in space. This represents the magnitude of the gradient vector. Describing the divergence operator This represents the curvature information of the contour lines in the image. This is a weighting function constructed based on image curvature, used to differentially weight and adjust the brightness-invariant constraint term and the gradient-invariant constraint term according to the complexity of the local structure of the image; Indicates the image domain. These are the weighting coefficients of the total variation regularization term; based on this, a hybrid data term containing brightness-invariant and gradient-invariant constraints is constructed. , represented as The hybrid data items are used to enhance the robustness of optical flow matching under varying illumination and complex texture conditions.

[0017] Preferably, in step S2, the optical flow field of two adjacent input images is initialized at the current image pyramid layer, and the spatial partial derivative, temporal partial derivative, and gradient change term of the input image are calculated based on the brightness invariant constraint and the gradient invariant constraint, respectively. On this basis, the brightness invariant constraint and the gradient invariant constraint are introduced simultaneously to construct a variational optical flow energy model containing the image curvature weight factor, which is used to measure the motion consistency between adjacent image pixels. By introducing curvature information into the weighted adjustment process of the data terms, the model can adaptively adjust the influence of the brightness constraint and the gradient constraint according to the complexity of the local structure of the image, thereby improving the robustness and stability of optical flow estimation under illumination changes and complex texture conditions.

[0018] Preferably, in step S3, the introduction of an adaptive weight adjustment mechanism based on image curvature into the data terms of the variational optical flow energy model specifically includes:

[0019] S31. Construct an adaptive weighting function based on image curvature. ,in , is used to represent the curvature information of the brightness contour lines of the input image, and to characterize the geometric complexity of the local structure of the image;

[0020] S32, the adaptive weight function As a whole coefficient, multiplied by the data item First, the brightness-invariant constraint term and the gradient-invariant constraint term are differentiated and weighted to obtain the improved variational optical flow energy model: When the local curvature of the image is large, the curvature weighting function A smaller value is chosen to reduce the impact of data terms on optical flow estimation in complex structural regions; when the local curvature of the image is small, the curvature weighting function... A larger value is chosen to enhance the constraint of data terms on optical flow estimation in structurally flat regions.

[0021] Preferably, in step S3, by introducing an adaptive weighting function based on image curvature, the data items are adaptively adjusted according to the complexity of the local structure of the image, thereby improving the robustness of optical flow estimation and maintaining the stability of the image in flat regions.

[0022] Preferably, in step S4, a curvature weight function that is monotonically decreasing and bounded with respect to image curvature is constructed, and an adaptive adjustment factor is introduced to dynamically constrain and adjust the curvature weight, specifically including:

[0023] S41. Calculate the image curvature based on the input image at the current pyramid level. The image curvature is defined as: , where ε is a positive constant to prevent the denominator from being zero;

[0024] S42, Based on the image curvature Construct curvature weight functions that act on the brightness-invariant constraint term and the gradient-invariant constraint term, respectively. and The curvature weighting function is a function that is monotonically decreasing with respect to the absolute value of curvature and has bounded values. Its mathematical model is expressed as follows: , And satisfy 0 < ≤1, 0< ≤1; where, and These are the curvature adjustment coefficients corresponding to the brightness-invariant constraint term and the gradient-invariant constraint term, respectively.

[0025] S43. Introduce an adaptive factor into the curvature adjustment coefficient. The curvature adjustment coefficient is dynamically adjusted according to the image curvature, and its mathematical model is expressed as follows: , ,in and These are the initial adjustment parameters. This is the curvature adaptive control factor; based on this, the curvature weighting function is introduced into the brightness-invariant constraint term and the gradient-invariant constraint term respectively, and the original mixed data term is weighted and corrected to obtain the curvature-enhanced data term. This allows the weights of data items to be dynamically adjusted according to the local curvature of the image. The mathematical model is as follows: .

[0026] Preferably, in step S4, the curvature of the brightness contour lines of the input image is calculated, and a curvature weight function with monotonically decreasing curvature and bounded values ​​is constructed based on the absolute value of curvature. This allows the weights of the brightness-invariant constraint term and the gradient-invariant constraint term to adaptively change with the complexity of the local geometric structure of the image. Simultaneously, an adaptive curvature control factor is introduced into the curvature adjustment coefficient, enabling smooth and continuous dynamic adjustment of the curvature weight within different curvature intervals. This avoids abrupt changes or abnormal amplification of the weight in high curvature or noisy regions, thereby effectively suppressing the interference of unreliable data terms on optical flow estimation in complex structural regions and enhancing the constraint ability of data terms on optical flow estimation in structurally flat regions. Through the combination of the above curvature weight constraint and adaptive adjustment mechanism, the data terms with enhanced curvature maintain numerical stability while ensuring robustness, providing reliable data consistency constraints for the efficient optimization of the subsequent variational optical flow model.

[0027] Preferably, in step S5, for the nonlinear variational optical flow energy model after introducing curvature adaptive weighting, the alternating direction multiplier method is used to solve the energy model. Specifically, this includes: in the variational optical flow energy model, splitting the gradient constraint in the total variational regularization term and introducing auxiliary variables. and its corresponding Lagrange multipliers And construct the following constraint relationships: ,in The index represents the optical flow component. Based on this, the data terms for curvature enhancement and the total variation regularization terms are combined to construct an augmented Lagrange energy function, the mathematical model of which is: ;in For total variation regularization weight parameters, To constrain the penalty parameters, an alternating optimization strategy is used to sequentially optimize the optical flow increment variable. Auxiliary variables and Lagrange multipliers Iterative updates are performed to achieve coordinated optimization among curvature-enhancing data terms, total variational regularization terms, and constraints; wherein the update method of the Lagrange multipliers satisfies the following relationship: The auxiliary variable The update is performed using the soft threshold shrinkage operator, and the update method satisfies the following relationship: The optical flow increment variable By obtaining the first-order optimality condition of the augmented Lagrange energy function, the solution formula for the unknowns is as follows: The algorithm is updated iteratively to achieve synergistic optimization among optical flow increment variables, regularization constraints, and Lagrange multipliers.

[0028] Preferably, in step S5, the nonlinear variational optical flow energy model after introducing curvature adaptive weighting is solved using the alternating direction multiplier method, and the gradient constraint in the total variational regularization term is split into variables. This decomposes the originally coupled non-smooth optimization problem into multiple subproblems that can be solved alternately, which helps to reduce the difficulty of solving the problem and improve computational efficiency. By introducing auxiliary variables and Lagrange multipliers, the optical flow increment update, regularization term constraint, and constraint penalty are decoupled, so that the curvature-enhanced data term and the total variational regularization term can be co-optimized during the iteration process. Among them, the auxiliary variable is updated using a soft threshold shrinkage operator, which effectively maintains the edge structure of the optical flow field and suppresses noise interference. The iterative update method of the optical flow increment variable satisfying the Euler-Lagrange optimality condition ensures the numerical stability and convergence of the overall optimization process, thereby achieving efficient co-optimization between the curvature-enhanced data term, the total variational regularization term, and the constraint conditions.

[0029] Preferably, in step S6, the optical flow increment and Based on the Euler-Lagrange optimality condition, gradient descent is used for iterative updates, and the update method satisfies the following relationship: ,in, and Representing the spatial coordinates of the image respectively and The partial derivative operator, To constrain the penalty parameters, As an auxiliary variable for the horizontal optical flow gradient, As an auxiliary variable for vertical optical flow, The Lagrange multipliers corresponding to the horizontal optical flow constraint. The Lagrange multiplier corresponding to the vertical optical flow constraint; the parameters The preset iteration step size parameter, For the first This process involves multiple iterations to ensure the numerical stability and convergence of the optical flow incremental update process.

[0030] Preferably, in step S6, based on satisfying the Eulerian-Lagrange optimality condition, the optical flow increment is iteratively updated using gradient descent. By simultaneously introducing the curvature-enhanced data term gradient, Lagrange multiplier term, and constraint penalty term during the update process, the optical flow increment update can satisfy the total variational regularization constraint while ensuring data consistency. In particular, by setting reasonable iteration step size parameters and constraint penalty parameters, the update amplitude of the optical flow increment is effectively controlled to avoid numerical oscillation and divergence problems, thereby improving the numerical stability and convergence of the optical flow increment iterative update process, and enabling the optical flow estimation results to maintain a good balance between smoothness and fine structure in complex structures and large displacement scenarios.

[0031] Preferably, in step S7, at the highest resolution image pyramid layer Obtain the corresponding final optical flow increment and The final optical flow increment is then compared with the corresponding horizontal optical flow component of that layer. and vertical components Superposition and updating are performed to obtain the horizontal component of the final optical flow field. and vertical components The update relation satisfies: The final optical flow field is used as the output of the large displacement variational optical flow calculation method based on curvature enhancement to characterize the pixel-level motion information between two adjacent input images.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows.

[0033] This invention introduces an adaptive weight adjustment mechanism based on image curvature into a multi-scale variational optical flow framework. This effectively integrates local geometric information of the image into the optical flow data term modeling process, allowing the constraint strength of brightness-invariant and gradient-invariant constraints to be dynamically adjusted according to the complexity of the image structure. This suppresses the interference of unreliable matching on optical flow estimation in complex textures and high curvature regions, and enhances the constraint ability of data terms on optical flow estimation in structurally flat regions. Simultaneously, by combining the stepwise estimation strategy of the image pyramid with the efficient solution method of the alternating direction multiplier method, the stability, robustness, and convergence performance of optical flow estimation in large displacement scenarios are effectively improved. Therefore, this invention can obtain more accurate, smooth, and structurally consistent pixel-level motion estimation results under conditions of illumination variation, noise interference, and complex structures, and has good engineering application value. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall process of a curvature-enhanced large displacement image variational optical flow calculation method for motion estimation provided by the present invention.

[0035] Figure 2 This is a schematic diagram of the multi-scale optical flow estimation process of the image pyramid in this invention.

[0036] Figure 3 This is a schematic diagram of the curvature-enhanced large displacement variational optical flow energy model in this invention.

[0037] Figure 4 This is a schematic diagram of the adaptive weight adjustment module based on image curvature in this invention.

[0038] Figure 5 This is a schematic diagram of the curvature enhancement variational optical flow solving module based on the alternating direction multiplier method in this invention.

[0039] Figure 6 This is a schematic diagram of the incremental iterative update of optical flow and the final optical flow field generation process in this invention. Detailed Implementation

[0040] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0041] Please see Figures 1 to 6This invention provides a curvature-enhanced large displacement image variational optical flow method for motion estimation. The overall process includes steps such as image pyramid construction, curvature-enhanced variational optical flow modeling, multi-scale iterative solution, and final optical flow field output, which is used to estimate pixel-level large displacement motion between two adjacent input images.

[0042] Please see Figure 1 As shown in the embodiment of this application, a curvature-enhanced large displacement image variational optical flow method for motion estimation is presented.

[0043] S1. Obtain two adjacent input images and construct an image pyramid for the two adjacent input images.

[0044] Furthermore, in this embodiment, two adjacent input frames are acquired by an in-vehicle camera, which is fixedly mounted on the windshield of the vehicle and acquires a continuous video sequence of the road scene. The two adjacent input frames are respectively denoted as... and The two adjacent input frames are preprocessed to reduce the impact of noise and illumination variations on optical flow estimation. Then, an image pyramid is constructed consisting of N layers of images at different resolutions, where N is set to 4. The lowest resolution image is located at the top of the pyramid, and the highest resolution image is located at the bottom. The lowest resolution layer... Initialize the horizontal component of the optical flow field corresponding to the current pyramid level. and vertical components and the corresponding optical flow increment and .

[0045] Furthermore, such as Figure 2 As shown, after completing the optical flow estimation of the initial optical flow calculation layer, the optical flow field obtained by this layer is used as the initial value for optical flow calculation of the next higher resolution image, and optical flow estimation is performed layer by layer in the order from low resolution layer to high resolution layer to realize the step-by-step estimation of large displacement optical flow.

[0046] S2. Select the current pyramid layer used for optical flow estimation in the image pyramid, initialize the optical flow field of the layer and the corresponding optical flow increment, and construct a variational optical flow energy model that includes both brightness invariant constraints and gradient invariant constraints.

[0047] Furthermore, a variational optical flow energy model incorporating brightness-invariant and gradient-invariant constraints is constructed. The specific steps are as follows.

[0048] S21. Initialize the optical flow field of the two input images at the current image pyramid level to obtain the horizontal component of the optical flow. and vertical components and the corresponding optical flow increment and Partial derivatives are calculated for the two input images of the current pyramid layer, where the first-order spatial partial derivative of the input image in the horizontal direction is calculated based on the brightness invariance constraint. First-order spatial partial derivative in the vertical direction And the partial derivative of brightness change along the time axis Based on gradient invariance constraints, the spatial gradient of the input image is calculated to obtain the first... Gradient partial derivative of the input frame image in the horizontal direction Gradient partial derivative in the vertical direction And the corresponding gradient change term over time. ,in =1 and 2 represent the first and second frame input images, respectively.

[0049] S22. Simultaneously introduce brightness invariance constraints and gradient invariance constraints into the current pyramid layer to construct a variational optical flow energy function that includes a curvature weighting factor. It is used to measure the motion consistency between adjacent image pixels, and its functional form is defined as: ,in, This represents the gradient vector of the brightness of the input image in space. This represents the magnitude of the gradient vector. Denotes the divergence operator, This represents the curvature information of the contour lines in the image. This is a weighting function constructed based on image curvature, used to differentially weight and adjust the brightness-invariant constraint term and the gradient-invariant constraint term according to the complexity of the local structure of the image; Indicates the image domain. The weighting coefficients of the total variation regularization term, parameters Set to 0.15; based on this, construct a hybrid data term that includes brightness-invariant constraints and gradient-invariant constraints. , represented as The hybrid data items are used to enhance the robustness of optical flow matching under varying lighting conditions and complex textures.

[0050] S3, in the variational optical flow energy model The coefficient positions of the data items are determined by introducing an adaptive weight adjustment mechanism based on image curvature.

[0051] Furthermore, such as Figure 3 As shown, in step S3, the introduction of an adaptive weight adjustment mechanism based on image curvature into the data terms of the variational optical flow energy model specifically includes the following:

[0052] S31. Construct an adaptive weighting function based on image curvature. ,in It is used to represent the curvature information of the brightness contour lines of the input image, and to characterize the geometric complexity of the local structure of the image.

[0053] S32, the adaptive weight function As a whole coefficient, multiplied by the data item First, the brightness-invariant constraint term and the gradient-invariant constraint term are differentiated and weighted to obtain the improved variational optical flow energy model: When the local curvature of the image is large, the curvature weighting function A smaller value is chosen to reduce the impact of data terms on optical flow estimation in complex structural regions; when the local curvature of the image is small, the curvature weighting function... A larger value is chosen to enhance the constraint of data terms on optical flow estimation in structurally flat regions.

[0054] S4. By constructing a weight function with monotonically decreasing curvature and bounded values ​​for the image, and introducing an adaptive adjustment factor, the curvature weight is dynamically constrained and adjusted.

[0055] Furthermore, such as Figure 4 As shown, in step S4, a curvature weight function with monotonically decreasing and bounded curvature of the image is constructed, and an adaptive adjustment factor is introduced to dynamically adjust the curvature weight, specifically including:

[0056] S41. Calculate the image curvature based on the input image at the current pyramid level. The image curvature is defined as: Where ε is a positive constant to prevent the denominator from being zero, set to . This is to prevent numerical instability when the gradient magnitude approaches zero.

[0057] S42, Based on the image curvature Construct curvature weight functions that act on the brightness-invariant constraint term and the gradient-invariant constraint term, respectively. and The curvature weighting function is a function that is monotonically decreasing with respect to the absolute value of curvature and has bounded values. Its mathematical model is expressed as follows: , And satisfy 0 < ≤1, 0< ≤1; where, and These are the curvature adjustment coefficients corresponding to the brightness-invariant constraint term and the gradient-invariant constraint term, respectively.

[0058] S43. Introduce an adaptive factor into the curvature adjustment coefficient. The curvature adjustment coefficient is dynamically adjusted according to the image curvature, and its mathematical model is expressed as follows: , ,in and These are the initial adjustment parameters. For curvature adaptive control factor, The value is set to 0.5. Based on this, the curvature weighting function is introduced into the brightness invariant constraint term and the gradient invariant constraint term respectively to perform weighted correction on the original mixed data term, thereby obtaining the curvature-enhanced data term. This allows the weights of data items to be dynamically adjusted according to the local curvature of the image. The mathematical model is as follows: .

[0059] S5. For the nonlinear variational optical flow energy model after introducing curvature adaptive weighting, the present invention uses the alternating direction multiplier method to solve the energy model.

[0060] Furthermore, such as Figure 5 As shown, in step S5, for the nonlinear variational optical flow energy model after introducing curvature adaptive weighting, the alternating direction multiplier method is used to solve the energy model. Specifically, this includes: in the variational optical flow energy model, splitting the gradient constraint in the total variational regularization term and introducing auxiliary variables. and its corresponding Lagrange multipliers And construct the following constraint relationships: Where i represents the optical flow component index, and based on this, the curvature-enhancing data term and the total variation regularization term are jointly constructed into an augmented Lagrange energy function, the mathematical model of which is: ,in, For total variation regularization weight parameters, To constrain the penalty parameters, Set to 1, with a maximum number of iterations of 100. When the change in optical flow between two consecutive iterations is less than 1... Stop iteration when the time is right; use an alternating optimization strategy to sequentially optimize the optical flow increment variable. Auxiliary variables and Lagrange multipliers Iterative updates are performed to achieve coordinated optimization among curvature-enhancing data terms, total variational regularization terms, and constraints; wherein the update method of the Lagrange multipliers satisfies the following relationship: The auxiliary variable The update is performed using the soft threshold shrinkage operator, and the update method satisfies the following relationship: The optical flow increment variable By obtaining the first-order optimality condition of the augmented Lagrange energy function, the solution formula for the unknowns is as follows: The algorithm is updated iteratively to achieve synergistic optimization among optical flow increment variables, regularization constraints, and Lagrange multipliers.

[0061] S6. After obtaining the optical flow update result at the current pyramid layer, the optical flow estimation result of this layer is propagated to the image at a higher resolution layer through interpolation and used as the initial value for the optical flow estimation of this high-resolution layer.

[0062] Further, in step S6, the optical flow increment and Based on the Euler-Lagrange optimality condition, gradient descent is used for iterative updates, and the update method satisfies the following relationship: ,in, and Representing the spatial coordinates of the image respectively and The partial derivative operator, To constrain the penalty parameters, As an auxiliary variable for the horizontal optical flow gradient, As an auxiliary variable for vertical optical flow, The Lagrange multipliers corresponding to the horizontal optical flow constraint. The Lagrange multiplier corresponding to the vertical optical flow constraint; the parameters In this embodiment, the preset iteration step size parameter is used. Take 0.2, For the first This process involves multiple iterations to ensure the numerical stability and convergence of the optical flow incremental update process.

[0063] S7, at the highest resolution image pyramid layer Obtain the corresponding final optical flow increment and The final optical flow increment is then compared with the corresponding horizontal optical flow component of that layer. and vertical components Superposition and updating are performed to obtain the horizontal component of the final optical flow field. and vertical components The update relation satisfies: The final optical flow field is used as the output of the large displacement variational optical flow calculation method based on curvature enhancement to characterize the pixel-level motion information between two adjacent input images and describe the corresponding motion displacement of pixels in the first image in the second image.

[0064] Furthermore, the final optical flow field is visualized and output using HSV color encoding, where the optical flow direction is mapped to hue H and the optical flow amplitude is mapped to luminance V. An optical flow visualization image file is generated and saved in PNG format to display the motion direction and speed distribution of vehicles and pedestrians in road scenes.

[0065] Furthermore, such as Figure 6 As shown, in step S7, the final optical flow field integrates the multi-scale pyramid layer-by-layer estimation results and the adjustment effect of the curvature adaptive weighting mechanism at each scale. It is used to characterize the pixel-level motion information between two adjacent input images and serves as the output result of the curvature-enhanced large displacement image variational optical flow method for motion estimation described in this invention.

[0066] Furthermore, this embodiment uses Python to implement the curvature-enhanced large-displacement variational optical flow calculation method. The operating environment is Windows 11, the development tool is PyCharm, the Python version is 3.8, the image processing library is OpenCV, and the matrix operation library is NumPy. Further, during program execution, the program reads the input video sequence frame image files, sequentially performs image preprocessing, image pyramid construction, curvature calculation, ADMM solution, and multi-scale iterative propagation, and outputs the final optical flow field and visualization results. During program execution, it reads two adjacent input frames from the video sequence or image sequence as input data for optical flow estimation, sequentially performs preprocessing operations such as grayscale conversion and filtering / denoising, and constructs a multi-layer image pyramid; then, starting from the lowest resolution layer, ... Each pyramid layer calculates the image partial derivatives and gradient change terms required for brightness-invariant and gradient-invariant constraints. It then calculates the image curvature and constructs an adaptive curvature weight function, introducing these weights into the data terms of the variational optical flow energy model. Next, it iteratively solves the energy model using the alternating direction multiplier method, alternately updating the optical flow increment, auxiliary variables, and Lagrange multipliers. After completing the solution for the current layer, the optical flow results are interpolated and propagated to higher resolution layers until the highest resolution layer completes the calculation. The program outputs the final optical flow field result at the highest resolution layer and saves the optical flow field as a two-dimensional vector field. Simultaneously, it generates an optical flow visualization image to display the pixel-level motion direction and amplitude distribution between adjacent frames, thereby estimating and displaying the motion information of vehicles and pedestrians in road scenes.

[0067] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these modifications and improvements all fall within the protection scope of the present invention.

Claims

1. A curvature-enhanced variational optical flow method for large displacement images oriented towards motion estimation, characterized in that, Includes the following steps: S1. Obtain two adjacent input images and construct an image pyramid for the two adjacent input images; S2. Select the current pyramid layer for optical flow estimation in the image pyramid, initialize the optical flow field and optical flow increment of this layer, and construct a variational optical flow energy model that includes brightness invariant constraints, gradient invariant constraints, and curvature weighting factors. ; S3, in the variational optical flow energy model The data item coefficients are located, and an adaptive weight adjustment mechanism based on image curvature is introduced. The image curvature characterizes the complexity of the local structure of the image, and the brightness invariant constraint term and the gradient invariant constraint term are adjusted differentially by weighting. S4. Construct a curvature weight function that is monotonically decreasing and bounded with respect to the curvature of the image, and introduce an adaptive adjustment factor to dynamically constrain and adjust the curvature weight. S5. For the nonlinear variational optical flow model after introducing curvature weighting, the alternating direction multiplier method is used to perform variable splitting and co-optimization on the data terms and the total variational regularization terms to achieve model solution. S6. After obtaining the optical flow update result at the current pyramid layer, the optical flow update result is propagated to the higher resolution layer for interpolation, and steps S2 to S5 are repeated until the highest resolution layer completes the calculation. S7. After completing the incremental iterative update of optical flow at the highest resolution layer, the final optical flow field is output to characterize the pixel-level motion information between two adjacent input images.

2. The curvature-enhanced variational optical flow method for large displacement images oriented towards motion estimation according to claim 1, characterized in that, In step S1, two adjacent input frames are acquired. and The two adjacent input images are preprocessed to construct an image pyramid consisting of N layers of images with different resolutions, where the lowest resolution image is located at the top of the pyramid and the highest resolution image is located at the bottom of the pyramid. In the pyramid level Initialize the horizontal component of the optical flow field corresponding to the current pyramid level. and vertical components and the corresponding optical flow increment and After completing the optical flow estimation of the initial optical flow calculation layer, the optical flow field obtained from this layer is used as the initial value for optical flow calculation of the next higher resolution image. The optical flow is then estimated layer by layer in the order from low resolution layer to high resolution layer, so as to realize the step-by-step estimation of large displacement optical flow.

3. The curvature-enhanced variational optical flow method for large displacement images oriented towards motion estimation according to claim 2, characterized in that, In step S2, a variational optical flow energy model is constructed that simultaneously includes brightness invariance constraints, gradient invariance constraints, and curvature weighting factors. The specific steps are as follows: S21. Initialize the optical flow field of the two input images at the current image pyramid level to obtain the horizontal component of optical flow. and vertical components and the corresponding optical flow increment and ; Partial derivatives are calculated for the two input images of the current pyramid layer. Based on the brightness invariance constraint, the first-order spatial partial derivative of the input image in the horizontal direction is calculated. First-order spatial partial derivative in the vertical direction And the partial derivative of brightness change along the time axis. ; Based on the gradient invariance constraint, the spatial gradient of the input image is calculated to obtain the first... Gradient partial derivative of the input frame image in the horizontal direction Gradient partial derivative in the vertical direction And the corresponding gradient change term over time. ,in =1 and 2 represent the first and second frame input images, respectively; S22. Introduce brightness-invariant and gradient-invariant constraints into the current pyramid layer, and construct a variational optical flow energy function that includes a curvature weighting factor. It is used to measure the motion consistency between adjacent image pixels, and its functional form is defined as: ,in, This represents the gradient vector of the brightness of the input image in space. This represents the magnitude of the gradient vector. Denotes the divergence operator, This represents the curvature information of the contour lines in the image. This is a weighting function constructed based on image curvature, used to differentially weight and adjust the brightness-invariant constraint term and the gradient-invariant constraint term according to the complexity of the local structure of the image; Indicates the image domain. These are the weight coefficients of the total variation regularization term. Based on this, a hybrid data term containing brightness-invariant and gradient-invariant constraints is constructed. , represented as This hybrid data item is used to enhance the robustness of optical flow matching under varying lighting conditions and complex textures.

4. The curvature-enhanced variational optical flow method for large displacement images oriented towards motion estimation according to claim 3, characterized in that, In step S3, an adaptive weight adjustment mechanism based on image curvature is introduced into the data terms of the variational optical flow energy model, specifically including: S31. Construct an adaptive weighting function based on image curvature. ,in , is used to represent the curvature information of the brightness contour lines of the input image, and to characterize the geometric complexity of the local structure of the image; S32, Adjust the curvature adaptive weighting function As a whole coefficient, multiplied by the data item First, the brightness-invariant constraint term and the gradient-invariant constraint term are differentiated and weighted to obtain the improved variational optical flow energy model: When the local curvature of the image is large, the adaptive weighting function A smaller value is chosen to reduce the impact of data terms on optical flow estimation in complex structural regions; when the local curvature of the image is small, the adaptive weighting function... A larger value is chosen to enhance the constraint of data terms on optical flow estimation in structurally flat regions.

5. The curvature-enhanced variational optical flow method for large displacement images oriented towards motion estimation according to claim 4, characterized in that, In step S4, a curvature weight function with monotonically decreasing and bounded curvature of the image is constructed, and an adaptive adjustment factor is introduced to dynamically constrain and adjust the curvature weight, specifically including: S41. Calculate the image curvature based on the input image at the current pyramid level. The image curvature is defined as: , where ε is a positive constant to prevent the denominator from being zero; S42, Based on the image curvature Construct curvature weight functions that act on the brightness-invariant constraint term and the gradient-invariant constraint term, respectively. and The curvature weighting function is a function that is monotonically decreasing with respect to the absolute value of curvature and has bounded values. Its mathematical model is expressed as follows: , And satisfying 0 < ≤1, 0< ≤1; where, and These are the curvature adjustment coefficients corresponding to the brightness-invariant constraint term and the gradient-invariant constraint term, respectively. S43. Introduce an adaptive factor into the curvature adjustment coefficient. The curvature adjustment coefficient is dynamically adjusted according to the image curvature, and its mathematical model is expressed as follows: , ,in, and Let η be the initial adjustment parameter and η be the curvature adaptive control factor. Based on this, the curvature weighting function is introduced into the brightness-invariant constraint term and the gradient-invariant constraint term respectively to perform weighted correction on the original mixed data term, thereby obtaining the curvature-enhanced data term. This allows the weights of data items to be dynamically adjusted according to the local curvature of the image. The mathematical model is as follows: .

6. The curvature-enhanced variational optical flow method for large displacement images oriented towards motion estimation according to claim 5, characterized in that, In step S5, for the nonlinear variational optical flow energy model after introducing curvature adaptive weighting, the alternating direction multiplier method is used to solve the energy model. Specifically, this includes: in the variational optical flow energy model, splitting the gradient constraint in the total variational regularization term and introducing auxiliary variables. and its corresponding Lagrange multipliers And construct the following constraint relationships: ,in The index represents the optical flow component. Based on this, the data terms for curvature enhancement and the total variation regularization terms are combined to construct an augmented Lagrange energy function, the mathematical model of which is: ,in, For total variation regularization weight parameters, To constrain the penalty parameters, an alternating optimization strategy is used to sequentially optimize the optical flow increment variable. Auxiliary variables and Lagrange multipliers Iterative updates are performed to achieve coordinated optimization among curvature-enhanced data terms, total variational regularization terms, and constraints.

7. The curvature-enhanced variational optical flow method for large displacement images oriented towards motion estimation according to claim 6, characterized in that, In step S6, the optical flow increment and Based on the Euler-Lagrange optimality condition, gradient descent is used for iterative updates, and the update method satisfies the following relationship: ,in, and Representing the spatial coordinates of the image respectively and The partial derivative operator, To constrain the penalty parameters, As an auxiliary variable for the horizontal optical flow gradient, As an auxiliary variable for vertical optical flow, The Lagrange multipliers corresponding to the horizontal optical flow constraint. The Lagrange multiplier corresponding to the vertical optical flow constraint; the parameters The preset iteration step size parameter, For the first This process involves multiple iterations to ensure the numerical stability and convergence of the optical flow incremental update process.

8. The curvature-enhanced variational optical flow method for large displacement images oriented towards motion estimation according to claim 7, characterized in that, In step S7, at the highest resolution image pyramid layer Obtain the corresponding final optical flow increment and The final optical flow increment is then compared with the corresponding horizontal optical flow component of that layer. and vertical components Superposition and updating are performed to obtain the horizontal component of the final optical flow field. and vertical components The update relation satisfies: The final optical flow field is used as the output of the large displacement variational optical flow calculation method based on curvature enhancement to characterize the pixel-level motion information between two adjacent input images.

Images