Unsupervised optical flow estimation method and system for different exposure low dynamic range images

Abstract

The application discloses an unsupervised optical flow estimation method and system for different exposure low dynamic range images, and relates to the technical field of optical flow estimation. First, based on an intensity mapping function (IMF), the brightness of low dynamic range images with different exposures is normalized. Then, based on a RAFT algorithm, the low dynamic range images after brightness normalization are subjected to preliminary optical flow estimation. Based on the result of the preliminary optical flow estimation, the RAFT algorithm is trained by using an unsupervised learning method. Finally, the low dynamic range images after brightness normalization are subjected to final optical flow estimation by using the trained RAFT algorithm. The application can be applied to images with different exposures and images under more complex lighting conditions, and the best optical flow estimation result can be achieved, which is more efficient and more robust than existing methods.

Description

Technical Field

[0001] This invention relates to the field of optical flow estimation technology, and more specifically to an unsupervised optical flow estimation method and system for low dynamic range images with different exposures. Background Technology

[0002] Optical flow records the pixel displacements (or correspondences) between two images and can be used for motion estimation, semantic understanding, and image alignment. Improvements in optical flow directly benefit downstream tasks such as visual odometry, object tracking, and HDR imaging.

[0003] Photometric consistency is a crucial assumption in optical flow estimation. It assumes that corresponding pixels between two images have the same intensity. Recent deep learning-based optical flow methods enforce this assumption by constructing a correlation quantity that computes the dot product between the deep feature representations of the images. The correlation quantity has proven to be an efficient component and has been widely used in existing neural networks. For unsupervised learning of optical flow, photometric consistency becomes even more critical when true optical flow labels are unavailable during training. Unsupervised methods are typically formalized as minimizing the photometric error between two images aligned by optical flow. Since, by assumption, the appearance of objects does not change as they move, a smaller error indicates more accurate optical flow.

[0004] When applied to visual odometry and object tracking, optical flow can be easily estimated using readily available methods. However, estimating optical flow becomes difficult for HDR imaging. HDR is a popular mode for mobile photography on smartphones and digital cameras. Multiple LDR images with different exposures are captured sequentially and then merged together to generate an HDR image. Notably, if there is camera or object motion in the input image, the HDR image may exhibit blurring or ghosting artifacts. This problem can be mitigated by optical flow, which aligns the input LDR image with a selected reference image. Therefore, synchronized LDR images can be treated as images of a static scene, and they can be merged into the HDR image or into the enhanced LDR image. However, due to the different exposures, there are significant pixel intensity variations between LDR images with different exposures. The photometric consistency assumption becomes untenable, making optical flow estimation extremely difficult.

[0005] To mitigate this problem, existing methods typically apply gamma correction to normalize the brightness of LDR images before estimating optical flow. However, these methods have two limitations: a) the gamma function can only be used to normalize images captured with different exposures using a special camera response function (CRF); b) optical flow is estimated using conventional methods, which are often computationally expensive.

[0006] Therefore, how to overcome the above-mentioned technical defects is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides an unsupervised optical flow estimation method and system for low dynamic range images with different exposures.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] An unsupervised optical flow estimation method for low dynamic range images with different exposures includes the following steps:

[0010] Step 1: Based on the Intensity Mapping Function (IMF), normalize the brightness of low dynamic range images with different exposures;

[0011] Step 2: Based on the RAFT algorithm, perform preliminary optical flow estimation on the low dynamic range image after brightness normalization;

[0012] Step 3: Based on the preliminary optical flow estimation results, the RAFT algorithm is trained using unsupervised learning.

[0013] Step 4: Use the trained RAFT algorithm to perform final optical flow estimation on the low dynamic range image after brightness normalization.

[0014] Optionally, in step 1, the specific steps for normalizing the brightness of low dynamic range images with different exposures based on the intensity mapping function (IMF) are as follows:

[0015] Select a pair of low dynamic range images Z1 and Z2;

[0016] Calculate the intensity mapping function (IMF) from image Z1 to image Z2, denoted as Λ. 1→2 (z);

[0017] Using Λ 1→2 (z) Normalize the brightness of image Z1 to the brightness of image Z2 to obtain Λ 1→2 (Z1).

[0018] Optionally, in step 1, the standard for brightness normalization is to normalize the brightness of the input low dynamic range images with different exposures to the brightness of the image with the highest brightness.

[0019] Optionally, in step 2, the method for preliminary optical flow estimation of the low dynamic range image after brightness normalization based on the RAFT algorithm is as follows:

[0020] Λ 1→2 Using (Z1) and Z2 as inputs, calculate the optical flow from Z2 to Z1.

[0021] O2→1 =F(Z2, Λ) 1→2 (Z1); θ);

[0022] Among them, O 2→1 Let Z represent the optical flow from Z2 to Z1, and θ represent the trainable parameters of the RAFT algorithm.

[0023] Optionally, in step 3, the method for training the RAFT algorithm using unsupervised learning based on the preliminary optical flow estimation results is as follows:

[0024] Calculate the photometric consistency term L photo ;

[0025] Calculate the smoothing term L smooth ;

[0026] The photometric consistency term L photo Smoothing term L smooth Together they constitute learning objective L;

[0027] Determine the optimal parameters of the RAFT algorithm based on the learning objective L. Implement training of the RAFT algorithm.

[0028] Optional, optimal parameters for:

[0029]

[0030] Where λ is the equilibrium photometric uniformity term L photo Smoothing term L smooth Hyperparameters.

[0031] An unsupervised optical flow estimation system for low dynamic range images with different exposures includes:

[0032] The brightness normalization module is used to normalize the brightness of low dynamic range images with different exposures based on the intensity mapping function (IMF).

[0033] The preliminary optical flow estimation module is used to perform preliminary optical flow estimation on low dynamic range images after brightness normalization based on the RAFT algorithm.

[0034] An unsupervised learning module is used to train the RAFT algorithm using an unsupervised learning method based on the results of the preliminary optical flow estimation.

[0035] The final optical flow estimation module is used to perform final optical flow estimation on the low dynamic range image after brightness normalization using the RAFT algorithm after training.

[0036] As can be seen from the above technical solution, this invention provides an unsupervised optical flow estimation method and system for low dynamic range images with different exposures by fusing data-driven networks (RAFT) and model-based algorithms (IMF). Compared with the prior art, it has the following beneficial effects:

[0037] (1) Compared with gamma correction, IMF uses the correspondence between the histograms of two images to identify brightness changes. These two images do not need to be aligned. Furthermore, IMF considers all possible factors that may affect brightness. Therefore, this invention can be used not only to study images with different exposures, but also to study images under more complex lighting conditions.

[0038] (2) Using IMF-based unsupervised learning objectives to train the optical flow network, and adaptively selecting pixels from IMF-normalized or non-normalized images to calculate photometric errors, can be extended to other cases with large brightness variations to help the network solve the problem of inconsistent brightness.

[0039] (3) The method of the present invention achieves the best optical flow estimation results, and all LDR images are merged into HDR images or enhanced LDR images, which is more efficient and robust than existing methods.

[0040] (4) The brightness of the input LDR image is normalized by using the Intensity Mapping Function (IMF), which avoids introducing matching ambiguity into the correlation of RAFT and preserves the photometric consistency between images with different exposures. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the method steps of the present invention;

[0043] Figure 2 Visualizations of optical flow estimated by different ablation methods; the first column is the overlay image of the source image and the reference image, the second column is the result image of ablation method 2, the third column is the result image of ablation method 4, the fourth column is the result image of ablation method 5, the fifth column is the result image of ablation method 6, and the sixth column is the actual optical flow image.

[0044] Figure 3 Here is a visualization example of the variables in equation (13); the first column is the Z2 image, and the second column is the Λ image. 1→2 (Z 1→2The image is shown in the first column, the third column is the M2 image, and the fourth column is the Λ image. 2→1 (Z2) Image, fifth column is Z 1→2 The image, the sixth column is the M1 image;

[0045] Figure 4 Visualizations of optical flow supervised by different smoothing weights;

[0046] Figure 5 A schematic diagram illustrating the performance of methods with different thresholds T;

[0047] Figure 6 Visualizations of optical flow estimated by different algorithms; the first column is the overlapping image of the source image and the reference image, the second column is the result image of optical flow estimated by UnOpticalFlow, the third column is the result image of optical flow estimated by UFlow, the fourth column is the result image of optical flow estimated by UPFlow, the fifth column is the result image of optical flow estimated by the method of this invention, and the sixth column is the real optical flow image;

[0048] Figure 7 This is a schematic diagram of the system modules of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] This invention discloses an unsupervised optical flow estimation method for low dynamic range images with different exposures. See [link to relevant documentation]. Figure 1 This includes the following steps:

[0051] Step 1: Based on the Intensity Mapping Function (IMF), normalize the brightness of low dynamic range images with different exposures. The specific steps are as follows:

[0052] Step 1.1: Select a pair of low dynamic range images Z1 and Z2. In this embodiment, Z1 represents the source image and Z2 represents the reference image. Make the exposure of Z2 greater than that of Z1. Therefore, it is preferable to normalize the brightness of image Z1 to the brightness of image Z2.

[0053] It is worth noting that in other embodiments, the brightness of image Z2 can also be normalized to the brightness of image Z1, that is, the brightness of the high-exposure image can be normalized to the brightness of the low-exposure image.

[0054] Step 1.2: Calculate the intensity mapping function (IMF) from image Z1 to image Z2.

[0055] With Λ 1→2 (z) and Λ 2→1 (z) denotes the intensity mapping function IMF from image Z1 to image Z2 and the intensity mapping function IMF from image Z2 to image Z1, where z represents the intensity of the image pixel, ranging from 0 to 255 for an 8-bit LDR image.

[0056] Λ 1→2 (z) and Λ 2→1 (z) are all non-decreasing functions, and in this embodiment, they can be estimated using the Weighted Histogram Average (WHA) algorithm. For example, the WHA algorithm is used to estimate Λ. 1→2 The method for (z) is:

[0057] Let Ω i (z) represents the image Z. i The set of pixels, z represents the intensity of an image pixel, Ω i (z) represents Ω i The basis of (z). Cumulative histogram C i (z) is defined as:

[0058]

[0059] Furthermore, define a non-decreasing mapping function ψ 1→2 (z) is:

[0060] C2(ψ 1→2 (z)-1)≤C i (z)≤C2(ψ 1→2 (z)) (2)

[0061] Furthermore, ψ 1→2 (-1) is set to 0. ψ 1→2 (z) A histogram bin-level correspondence was established between two images with different exposures. C2(ψ) 1→2 (z)-1) and C2(ψ) 1→2 (z)) corresponds to the first and last bin of binΩ1(z), respectively.

[0062] Then we can calculate Λ 1→2 (z):

[0063]

[0064] in, This represents the size of the subbin (or bin) corresponding to binΩ1(k) in the image Z2.

[0065] There are two possible scenarios:

[0066] Case 1: ψ 1→2 (z-1)<ψ 1→2 (z),

[0067]

[0068] Case 2: ψ 1→2 (z-1)=ψ 1→2 (z),

[0069] This represents the segment in the cumulative histogram of image Z2 that maps to the z-th histogram bin of image Z1, i.e.:

[0070]

[0071] This embodiment uses the WHA algorithm to estimate IMFΛ. 1→2 (z) It is more robust and more accurate than existing technologies.

[0072] Step 1.3, using Λ 1→2 (z) Normalize the brightness of image Z1 to the brightness of image Z2 to obtain a new image Λ with normalized brightness. 1→2 (Z1).

[0073] Step 2: Based on the RAFT algorithm, perform preliminary optical flow estimation on the brightness-normalized low dynamic range image. The specific steps are as follows:

[0074] Λ 1→2 Using (Z1) and Z2 as inputs, calculate the optical flow from Z2 to Z1.

[0075] O 2→1 =F(Z2, Λ) 1→2 (Z1);θ)(7)

[0076] Among them, O 2→1 Let Z represent the optical flow from Z2 to Z1, and θ represent the trainable parameters of the RAFT algorithm.

[0077] Step 3: Based on the preliminary optical flow estimation results, the RAFT algorithm is trained using unsupervised learning. The specific steps are as follows:

[0078] Step 3.1: Calculate the photometric consistency term L photo Photometric uniformity term L photo Used to measure reference image Z2 and reconstructed image Z 1→2 The distance, the reconstructed image Z 1→2 =W(Z1, O) 2→1 W represents the grid-sample-based warp function, with the warp direction opposite to the optical flow direction.

[0079] Photometric uniformity term L photo The calculation formula is:

[0080]

[0081] Where T is a preset threshold used to ignore outliers;

[0082]

[0083] Where α can be set to 0.85, and SSIM represents the structural similarity index;

[0084] The calculation method is as follows:

[0085] The calculation method is as follows:

[0086] p is the number of image pixels, and the function ω(z) is...

[0087] Furthermore, two 0-1 masks can be defined, M1 = (ω(Z)). 1→2 (p))≤ω(Z2(p)))andM2=(ω(Z 1→2 (p))>ω(Z2(p))), where 0 represents false and 1 represents true, then:

[0088]

[0089] Step 3.2: Calculate the smoothing term L smooth :

[0090]

[0091] in and This represents the image gradient along the horizontal and vertical directions.

[0092] Step 3.3, the photometric consistency item L photo Smoothing term L smooth Together they constitute learning objective L;

[0093] Step 3.4: Determine the optimal parameters of the RAFT algorithm based on the learning objective L. Implement training of the RAFT algorithm. Optimal parameters. for:

[0094]

[0095] Where λ is the equilibrium photometric uniformity term L photoSmoothing term L smooth Hyperparameters.

[0096] Step 4: Use the trained RAFT algorithm to perform final optical flow estimation on the low dynamic range image after brightness normalization.

[0097] The experimental data from specific embodiments are listed below to verify the performance of the above method.

[0098] I. Obtaining Datasets and Determining Evaluation Metrics

[0099] The training dataset consists of the training sets for Kal17 and Pra19, and the real-world dataset for Chen21. Each image sequence in these datasets comprises images with alternating exposures. However, some sequences in Kal17 and Pra19 contain targets with large displacements, which negatively impact unsupervised optical flow learning. Therefore, these sequences were removed, resulting in final training datasets for Kal17, Pra19, and Chen21 containing 60, 147, and 284 sequences, respectively. For testing, this embodiment uses the test sets for Kal17 (i.e., Kal17(PAP) and Kal17(EXT)), Pra19, Sen12, and Tursun16.

[0100] The actual optical flow is first generated by FullFlow and then optimized by MirrorFlow. The total time required to acquire a single real label is approximately 40-60 minutes. In this embodiment, Endpoint Error (EPE) is used as the evaluation metric, which represents the average L2 distance between the predicted and actual optical flows. Furthermore, this embodiment uses s 0-10 s 10-40 and s 40+ The EPE is used to represent the true optical flow amplitude at 0-10, 10-40, and 40+ pixels, which correspond to the errors at near, medium, and far distances, respectively.

[0101] II. Experimental Conditions

[0102] This embodiment uses Python and PyTorch to implement the algorithm. Experiments were conducted on a 4029GP-TRT server with an NVIDIA TITAN RTX GPU and 24GB of memory. The number of RAFT iterations was set to 12 during training and 32 during testing. During training, images were randomly cropped to a size of 384 x 704. Random flipping, random scaling, and color jittering were used for data augmentation. The optimizer was AdamW, and gradients were clipped to the range [-1, 1]. The learning rate scheduler was OneCycle.

[0103] III. Ablation Study Results

[0104] The ablation study aimed to investigate the effectiveness of the proposed unsupervised learning objective function. The batch size was set to 3, with each batch including bidirectional optical flow loss, requiring approximately 20GB of GPU memory. The initial learning rate was set to 2.5e-4. The total number of training iterations reached 100k, and training on a single GPU took almost a day. All ablation methods and their results are listed in Table 1. The experimental conditions are listed on the left, and each condition is explained below:

[0105] IMF: √ indicates that the input image pair is normalized by IMF, where the brightness is normalized to the brightness of the brighter image in the pair. × indicates that brightness normalization is not used.

[0106] MAE / IMAE: These represent the penalty weights for the objective functions MAE and IMAE, respectively. MAE indicates that the L1 distance is the distance between the reference image and the reconstructed image measured at brighter levels. IMAE indicates the proposed IMF-based MAE, which selectively measures the error between images at either brighter or darker levels.

[0107] Smooth: Represents the penalty weight of the smoothing loss term, i.e., the hyperparameter λ.

[0108] SSIM / ISSIM: These represent the penalty weights for the target SSIM and ISSIM, respectively. SSIM and ISSIM correspond to MAE and IMAE mentioned above, but they measure the SSIM distance.

[0109] Table 1

[0110]

[0111] The results of the first and second ablation operations show that adding a smoothing term significantly reduces EPE, meaning that pixels within local areas tend to have the same motion. Compared to ablation method 2, method 3 removes IMF normalization, resulting in a performance decrease. This is because brightness variations can introduce ambiguity into the correlation quantities in RAFT, where corresponding pixels may not have high correlation.

[0112] Comparing ablations 2 and 4, and ablations 5 and 6, it is clear that the proposed objective (13) does indeed have a positive impact on unsupervised optical flow learning.

[0113] Poorly exposed pixels that are normalized to brightness are not included in the calculation of L1 and SSIM distances. Therefore, the proposed objective provides a better image similarity measurement and has a stronger ability to facilitate RAFT in determining pixel correspondences between source and reference images. It can also be observed that ablation #6 further enhances ablation #5 in the medium (s10-40) and long (s40+) distance ranges.

[0114] The visualization results of ablation methods 2, 4, 5, and 6 are as follows: Figure 2 As shown, the proposed target-supervised RAFT can generate consistent optical flow in local regions, especially in background areas where pixels are more likely to be poorly exposed. Furthermore, as visualized in ablation scans 2 and 5, and 4 and 6, SSIM is an important supervisory signal in optical flow learning.

[0115] To further demonstrate the effectiveness of the proposed objective, this embodiment visualizes the variables in equation (13) during training, such as... Figure 3 As shown in the image in the first row, Z 1→2 The overexposed area in Λ 1→2 (Z 1→2 Color distortion occurs in elements such as walls, windows, and railings, marked by red dashed boxes. This distortion is seen in the calculation of Z2 and Λ. 1→2 (Z 1→2 When there is a photometric error between Λ, these areas are precisely masked by M2 to avoid mismatching image similarity metrics. On the other hand, underexposed areas in Z2 are in Λ 2→1 (Z2) is filled with noise, such as the shirt marked with a blue dashed box. The color of the black shirt is also mistakenly changed to red. Similarly, in calculating Λ 2→1 (Z2) and Z 1→2 When there is an error between these areas, they are masked by M1. Several other examples are visualized in rows two through four; readers can zoom in for a better examination. In general, after IMF normalization, areas with high exposure tend to exhibit color distortion, while areas with low exposure tend to produce noise. Both should be discarded because they do not adhere to photometric consistency.

[0116] IV. Selection of Hyperparameters

[0117] 1) Smoothing Weight λ: Ablation methods 7 and 8 in Table 1 change the penalty weight of the smoothing term. While increasing the smoothness weight helps reduce the overall EPE, it significantly increases the EPE at long distances. Furthermore, excessively large weights can smooth out the boundaries of moving objects, such as... Figure 4 As shown. Considering both qualitative and quantitative optical flow results, the final smoothing weight λ was chosen to be 0.01.

[0118] 2) Threshold T: The experimental conditions were kept the same as ablation method 6 in Table 1, and the threshold T was continuously changed to observe performance changes. It is worth noting that the method with T=1 is equivalent to the original ablation method 6. Experimental results with T ranging from 1 to 0.05 are plotted on... Figure 5As can be seen, when T equals 0.15, EPE reaches the lowest value on both test datasets (2.86 for Kal17(PAP) and 3.09 for Kal17(EXT)). A smaller T helps to exclude difficult cases in optical flow learning, such as large displacements, occlusions, and appearance changes. The final T value was chosen to be 0.15 because values ​​smaller than it degrade performance.

[0119] V. Comparison of the Method of the Invention with the Prior Art

[0120] The proposed method was compared with several state-of-the-art unsupervised optical flow methods. The batch size was set to 12. The initial learning rate was set to 4e-4. λ, α, and T were set to 0.01, 0.85, and 0.15, respectively.

[0121] Experimental results are shown in Table 2. The proposed method achieves state-of-the-art EPE on datasets Kal17(PAP), Kal17(EXT), Pra19, and Sen12. Results for s10+ show that it is also competitive in estimating optical flow with large displacements. Notably, the test images in Tursun16 typically have large displacements, where our method achieves the second-best result.

[0122] Table 2

[0123]

[0124] BackToBasic performs the worst because it only includes a smoothing term and a photometric term. EPIFlow adds epipolar constraints for low-texture regions. UnOpticalFlow estimates occluded regions using bidirectional optical flow. ARFlow utilizes data augmentation to address appearance variations during unsupervised learning. However, these methods do not address the core problem of estimating optical flow for images with different exposures. This invention uses a photometric consistency term based on the IMF to eliminate matching ambiguities caused by different exposures. Benefiting from this task-specific learning objective function, the method of this invention achieves better performance, outperforming the state-of-the-art methods UFlow and UPFlow. Visualization results are shown below. Figure 6 As shown, the method of the present invention produces optical flow with clear boundaries and a clean background.

[0125] Photometric error can also be used as a metric for evaluating the quality of predicted optical flow. In the Pra19 testsplit, image Z2 is provided in addition to images Z1 and Z2. 1 Z2 1 It is aligned with Z2, while Z2 1 The exposure is the same as Z1. Therefore, we can calculate Z. 1→2 and Z2 1The photometric error between the two is used to evaluate the predicted optical flow O. 2→1 The quality of the optical flow was assessed. Peak signal-to-noise ratio (PSNR), SSIM, and mean square error (MSE) were used as metrics. The final results are reported in Table 3. The method of this invention still achieved the best results across all metrics, demonstrating the reliability of the predicted optical flow.

[0126] Table 3

[0127]

[0128] In other embodiments, the optical flow estimation method described above can also be applied to problems such as ghosting removal.

[0129] In another embodiment, an unsupervised optical flow estimation system for low dynamic range images with different exposures is also disclosed, see [link to relevant documentation]. Figure 7 ,include:

[0130] The brightness normalization module is used to normalize the brightness of low dynamic range images with different exposures based on the intensity mapping function (IMF).

[0131] The preliminary optical flow estimation module is used to perform preliminary optical flow estimation on low dynamic range images after brightness normalization based on the RAFT algorithm.

[0132] An unsupervised learning module is used to train the RAFT algorithm using an unsupervised learning method based on the results of the preliminary optical flow estimation.

[0133] The final optical flow estimation module is used to perform final optical flow estimation on the low dynamic range image after brightness normalization using the RAFT algorithm after training.

[0134] The system modules disclosed in the embodiments are described simply because they correspond to the methods disclosed in the embodiments. For relevant details, please refer to the method section.

[0135] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0136] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An unsupervised optical flow estimation method for low dynamic range images with different exposures, characterized in that, Includes the following steps: Step 1: Based on the Intensity Mapping Function (IMF), normalize the brightness of low dynamic range images with different exposures; Step 2: Based on the RAFT algorithm, perform preliminary optical flow estimation on the low dynamic range image after brightness normalization; Step 3: Based on the preliminary optical flow estimation results, train the RAFT algorithm using unsupervised learning; the specific steps are as follows: Step 3.1: Calculate the photometric consistency term Lphoto. The photometric consistency term Lphoto is used to measure the reference image Z2 and the reconstructed image. The distance, the reconstructed image W represents a warp function based on grid-sample. Z 1 represents the source image. O 2→1 Indicates from Z 2 to Z The optical flow of 1, with the warp direction opposite to the optical flow direction; The formula for calculating the photometric uniformity term Lphoto is: Where T is a preset threshold used to ignore outliers; in, It can be set to 0.85, where SSIM represents the structural similarity index; The calculation method is as follows: The calculation method is as follows: Λ 1→2 (z) and Λ 2→1 (z) represents the image Z 1 to image Z The intensity mapping function IMF and from the image Z 2 to image Z The intensity mapping function IMF of 1; For image pixels, the function for Furthermore, two 0-1 masks are defined. and Where 0 represents false and 1 represents true, then: Step 3.2, Calculate the smoothing term Lsmooth: in x and y represents the image gradient along the horizontal and vertical directions; Step 3.3: The photometric consistency term Lphoto and the smoothing term Lsmooth together constitute the learning objective L; Step 3.4: Determine the optimal parameters of the RAFT algorithm based on the learning objective L. This enables training of the RAFT algorithm; optimal parameters. for: Where λ is a hyperparameter that balances the photometric uniformity term Lphoto and the smoothing term Lsmooth; Step 4: Use the trained RAFT algorithm to perform final optical flow estimation on the low dynamic range image after brightness normalization.

2. The unsupervised optical flow estimation method for low dynamic range images with different exposures according to claim 1, characterized in that, In step 1, the specific steps for brightness normalization of low dynamic range images with different exposures based on the intensity mapping function (IMF) are as follows: Select a pair of low dynamic range images Z 1 and image Z 2; Calculation from image Z 1 to image Z The intensity mapping function IMF of 2 is Λ 1→2 (z); Using Λ 1→2 (z) will display the image Z The brightness of 1 is normalized to the image. Z The brightness of 2 is used to obtain Λ 1→2 ( Z 1).

3. The unsupervised optical flow estimation method for low dynamic range images with different exposures according to claim 1, characterized in that, In step 1, the standard for brightness normalization is to normalize the brightness of the input low dynamic range images with different exposures to the brightness of the image with the highest brightness.

4. The unsupervised optical flow estimation method for low dynamic range images with different exposures according to claim 2, characterized in that, In step 2, the method for preliminary optical flow estimation of the low dynamic range image after brightness normalization based on the RAFT algorithm is as follows: Λ 1→2 ( Z 1) and Z 2 as input, calculate from Z 2 to Z Optical flow of 1 ； in, O 2→1 Indicates from Z2 to Z Optical flow of 1 θ This represents the trainable parameters of the RAFT algorithm.

5. An unsupervised optical flow estimation system for low dynamic range images with different exposures, characterized in that, include: The brightness normalization module is used to normalize the brightness of low dynamic range images with different exposures based on the intensity mapping function (IMF). The preliminary optical flow estimation module is used to perform preliminary optical flow estimation on low dynamic range images after brightness normalization based on the RAFT algorithm. The unsupervised learning module is used to train the RAFT algorithm using unsupervised learning methods based on the results of the preliminary optical flow estimation; the specific steps are as follows: Step 3.1: Calculate the photometric consistency term Lphoto. The photometric consistency term Lphoto is used to measure the reference image Z2 and the reconstructed image. The distance, the reconstructed image W represents a warp function based on grid-sample. Z 1 represents the source image. O 2→1 Indicates from Z 2 to Z The optical flow of 1, with the warp direction opposite to the optical flow direction; The formula for calculating the photometric uniformity term Lphoto is: Where T is a preset threshold used to ignore outliers; in, It can be set to 0.85, where SSIM represents the structural similarity index; The calculation method is as follows: The calculation method is as follows: Λ 1→2 (z) and Λ 2→1 (z) represents the image Z 1 to image Z The intensity mapping function IMF and from the image Z 2 to image Z The intensity mapping function IMF of 1; For image pixels, the function for Furthermore, two 0-1 masks are defined. and Where 0 represents false and 1 represents true, then: Step 3.2, Calculate the smoothing term Lsmooth: in x and y represents the image gradient along the horizontal and vertical directions; Step 3.3: The photometric consistency term Lphoto and the smoothing term Lsmooth together constitute the learning objective L; Step 3.4: Determine the optimal parameters of the RAFT algorithm based on the learning objective L. This enables training of the RAFT algorithm; optimal parameters. for: Where λ is a hyperparameter that balances the photometric uniformity term Lphoto and the smoothing term Lsmooth; The final optical flow estimation module is used to perform final optical flow estimation on the low dynamic range image after brightness normalization using the RAFT algorithm after training.

Images