An unsupervised mismatch detection method based on reinforcement learning

Abstract

The present application relates to a kind of based on reinforcement learning's unsupervised mismatch detection method, the present application first inputs the matching point without label to mismatch detection network, training detection network, by multiple iterations to obtain best network parameter, and using the final network model of saving best network parameter to the matching point set to be detected for detection, by generating N extract subset and evaluating its corresponding model, finally the maximum consistent set of model is regarded as correct matching point pair set, to complete mismatch detection and elimination.The present application has higher mismatch detection efficiency, precision and stability, and on the one hand, it overcomes the sample label problem;On the other hand, it is not limited by mismatch rate, and a large number of correct matching points can be obtained with fewer sampling times;And the method is a kind of unsupervised learning framework, which can be compatible with other classification networks, to solve the mismatch detection problem.

Description

Technical Field

[0001] This invention relates to the field of photogrammetry and remote sensing, and specifically to an unsupervised mismatch detection method based on reinforcement learning. Background Technology

[0002] Obtaining reliable matching points between image pairs is a core issue in photogrammetry and remote sensing. However, geometric and radiometric distortions between images lead to a significant number of false matches, reducing the reliability of image matching results and consequently decreasing the accuracy of subsequent measurement tasks, such as basic matrix estimation and airborne triangulation. Therefore, false match detection is a necessary step to ensure correct matching and improve matching accuracy. This involves employing false match detection methods to remove as many false matches as possible, thereby improving the quality of image matching results.

[0003] Based on the method of finding the optimal transformation model, existing image mismatch point detection methods can be roughly divided into two categories: methods based on manual design and methods based on deep learning.

[0004] (1) Methods based on manual design

[0005] Hand-designed methods primarily rely on the global or local invariance between images to construct corresponding transformation models to constrain matching points, thereby achieving false match detection. Random sample consensus (RANSAC) is one of the most representative hand-designed methods, constructing a maximum consensus set by iteratively sampling to obtain the optimal global transformation model. Many RANSAC-based methods focus on accurately estimating the probability distribution of correct matches and reducing the number of samples. However, as the false match rate increases, the number of samples also increases. When the false match rate exceeds a certain threshold, RANSAC-based methods cannot obtain the correct maximum consensus set within a certain number of samples.

[0006] Another manual design approach is primarily based on the assumption of local invariance: due to physical constraints, the local structure between images is generally invariant; therefore, a set of functions can be used to approximate the transformation model between locally matching points. Local Preserving Match (LPM) and Grid-Based Statistical Motion (GMS) are two typical methods based on local invariance: LPM relaxes and approximates the local transformation model through regularized Euclidean distance; GMS uses the number of neighboring matching points to distinguish between correct and incorrect matches. However, these methods based on local invariance fail when the number of matching points is small or incorrect matches are not randomly distributed.

[0007] (2) Deep learning-based methods

[0008] Machine learning-based methods utilize deep neural networks (DNNs) to learn and mine the inherent geometric information (global or local invariance) between matching points, which is more conducive to obtaining better transformation models for mismatch detection. These methods generally outperform hand-designed methods. The mismatch detection problem is essentially a binary classification problem, so deep neural networks for point cloud segmentation (such as PointNet) can be used to solve it. LFGC (Learning to Find Good Correspondences) and NM-Net (Neighbors Mining Network) improve PointNet by embedding geometric constraints into the loss function. However, most current deep learning-based methods are supervised learning methods, requiring large amounts of labeled datasets for supervised learning and network training. Labeling data is time-consuming and labor-intensive, and mislabeling is difficult to avoid; supervised learning methods are also susceptible to the negative impact of mislabeling.

[0009] Training DNNs in an unsupervised manner can address the aforementioned issues of labeled data and mislabeled data. ULCM (unsupervised learning of consensus maximization) is an unsupervised learning method that trains the LFGC network by maximizing the number of correct matches. Due to its non-smooth loss function, ULCM is difficult to train. NGRANSAC (Neural-guided RANSAC), on the other hand, trains the network by maximizing the expected number of correct matches, and can use a set of matches with arbitrary mismatch rates as the training set. However, NGRANSAC still requires a supervised pre-training process to ensure and accelerate convergence. Therefore, NGRANSAC is also susceptible to the effects of mislabeled data.

[0010] Analysis of existing mismatch detection methods reveals that:

[0011] (1) The detection efficiency of manually designed methods needs to be improved, and the stability is poor. RANSAC and its improved methods based on global invariance require more sampling times when the false match rate is high, resulting in low efficiency; after the false match rate exceeds a certain threshold, it is impossible to obtain a correct match through sampling, resulting in poor stability. Methods based on local invariance cannot handle situations where the number of matching points is small or the false matches are not randomly distributed, thus leading to low accuracy and poor stability.

[0012] (2) Supervised learning methods require a large amount of labeled data, while existing unsupervised methods are either difficult to train or require pre-training using supervised learning methods. Supervised learning methods require supervised learning based on a large amount of labeled datasets; however, labeling data is time-consuming and labor-intensive, and mislabeling is difficult to avoid, which can easily lead to negative impacts. Unsupervised learning methods such as ULCM must be trained on datasets with a constant mismatch rate, thus limiting their application scope; NGRANSAC still requires a supervised pre-training process to ensure and accelerate convergence, and is also adversely affected by mislabeling.

[0013] (3) Poor compatibility. Most existing algorithms are dedicated algorithms and cannot be extended. Summary of the Invention

[0014] To address the shortcomings of existing mismatch detection methods, this invention proposes an unsupervised mismatch detection method based on reinforcement learning. On the one hand, it overcomes the sample labeling problem; on the other hand, it is not limited by the mismatch rate and can obtain a large number of correct matching points with fewer samplings; furthermore, this method is an unsupervised learning framework, compatible with other classification networks, and can solve the mismatch detection problem.

[0015] To solve the above technical problems, the present invention adopts the following technical solution:

[0016] An unsupervised mismatch detection method based on reinforcement learning includes the following steps:

[0017] Step 1: Input the matching point set s into the detection network π. The detection network π generates a relative matching probability θ for each matching point pair, θ = π(s; ω), where ω is the network parameter of π.

[0018] Step 2: Initialize a classification distribution using the relative matching probability θ, and sample N times from s according to this classification distribution to obtain N subsets of matching points (i.e., the minimum set of matching points that can be used to calculate the geometric transformation model between matching points). For any subset Ω of matching points... k Calculate its corresponding geometric transformation model h k ;

[0019] Step 3: Using the formula The updated network parameters ω' are calculated, where η represents the preset learning rate. in It can be automatically determined using tools in the deep learning library;

[0020] and

[0021] Where, r k Represents the geometric transformation model h kThe rating, I(y) represents the j-th matching point pair in the k-th sample set. j ) indicates solving The index number in set s, yes The corresponding relative matching probability;

[0022] After calculating ω', increment the iteration count by 1, and determine whether the iteration termination condition is met. If yes, proceed to the next step; otherwise, proceed to step 1 and update the network parameter ω in step 1 to ω'.

[0023] Step 4: Save the network parameters obtained from the last iteration to obtain the final optimized network;

[0024] Step 5: Input the set of matching points t to be detected into the final optimized network to obtain the relative matching probability θ of the matching points. t Using θ t Initialize a classification distribution, and perform M repeated samplings from t according to the distribution to obtain M matching point subsets and calculate the model corresponding to each matching point subset;

[0025] Step 6: Calculate the consistency set (i.e. the set of matching points compatible with the model) for each model. The consistency set containing the most matching points is taken as the set of correct matching point pairs corresponding to the set of matching points to be detected.

[0026] Furthermore, the method for scoring the geometric transformation model corresponding to the sampled matching point subset includes the following steps:

[0027] Step 1: Calculate the corresponding geometric transformation model based on the subset of matching points;

[0028] Step 2: Calculate the error of each pair of matching points under the constraints of the model using the geometric transformation model from Step 1, and determine whether the error is less than a predefined threshold. If so, add the matching point to set C; otherwise, do not add it. Repeat the above process until all matching points in the matching point set have been detected. Set C is the consistent set of the geometric transformation model.

[0029] Step 3: The number of elements in the consensus set C is the score of the geometric transformation model.

[0030] Furthermore, the iteration termination condition is that the number of iterations reaches a preset maximum number of iterations.

[0031] Furthermore, in step 1, the number of matching point pairs in the matching point subset is the same.

[0032] Furthermore, in step 4, the number of matching point pairs in the matching point subset is the same.

[0033] Furthermore, M = N.

[0034] Furthermore, the sampling methods in steps 1 and 4 are both Monte Carlo sampling methods.

[0035] Furthermore, in step 2 It can be automatically calculated using deep learning libraries such as PyTorch or TensorFlow.

[0036] Compared with the prior art, the present invention, by adopting the above technical solution, has the following advantages:

[0037] This invention overcomes the sample labeling problem and is not limited by the mismatch rate, allowing for the acquisition of a large number of correct matching points with fewer sampling attempts. Furthermore, this method is an unsupervised learning framework, compatible with other classification networks, and solves the mismatch detection problem. This invention implements the unsupervised mismatch detection method ULMR based on reinforcement learning. ULMR can learn the optimal detection network from an unlabeled set of matching points. For image pairs with a mismatch rate exceeding 80%, ULMR still achieves high-precision detection results and exhibits high stability. Compared to the classic RANSAC method, it significantly reduces the number of sampling attempts; compared to deep learning methods such as NM-Net and ULCM, it does not require labeling the initial set of matching points. Overall, ULMR has higher mismatch detection efficiency, accuracy, and stability. Moreover, ULMR is a general unsupervised learning framework for mismatch detection, capable of combining with different classification networks such as PointNet, NM-Net, and LFGC-Net to obtain high-quality detection results, demonstrating excellent compatibility.

[0038] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the process of the present invention;

[0040] Figure 2 This is a schematic diagram of the false match detection results of the test experiment, where the white dashed lines connect the false matches and the gray solid lines connect the correct matches;

[0041] Figure 3 This is a schematic diagram of the quantitative detection results of the test experiment;

[0042] Figure 4 This is a schematic diagram of the detection results from the application experiment;

[0043] Figure 5 A schematic diagram showing the changes in ULMR and RANSAC with the number of samplings;

[0044] Figure 6This is a schematic diagram showing the mismatch detection results of the three integrated methods. Detailed Implementation

[0045] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0046] This invention is based on reinforcement learning and aims to maximize the expected consistent set. It mainly includes the following four parts: (1) objective function; (2) Monte Carlo sampling to obtain a subset of matching points; (3) scoring of the sample set; and (4) network framework.

[0047] (1) Objective function

[0048] The training process for mismatch detection based on reinforcement learning is as follows: Input the set of matching points s into DNN π; the network π outputs a policy θ, that is, π generates a relative matching probability (RMP, relative matching probability) for each pair of matching points, and θ = π(s; ω), where ω is the network parameter of π; take action a based on policy θ using the Monte Carlo method, that is, sample a subset Ω from the set of matching point pairs s according to a certain probability distribution, and generate the geometric model corresponding to the subset Ω; score the geometric model corresponding to the sampled set obtained by the above state-action (a|s) to obtain a score r.

[0049] The optimal network parameters ω can be achieved by maximizing the expected score of the state-action (a|s) interval. Solve for the desired score. for:

[0050]

[0051] In the formula, p(a|s;θ) is the probability distribution function of state-action (a|s), and its parameter is θ; (a|s)~p(a|s;θ) means that state-action (a|s) is obtained by sampling with probability p(a|s;θ); r(a|s) is the score of state-action (a|s).

[0052] To maximize the expected score in equation (1) We can calculate first. The gradient with respect to θ (called the policy gradient):

[0053]

[0054] Where a,p θ , r are abbreviations for state-action (a|s), probability distribution p(a|s; θ), and rating r(a|s), respectively; a~p θ Indicates a probability p θExtract a state-action (a|s).

[0055] Calculate the policy gradient Then, the chain rule can be used to calculate the gradient.

[0056]

[0057] Since θ = π(s; ω) is a definite function of ω, therefore It can be automatically calculated using deep learning libraries such as PyTorch or TensorFlow.

[0058] Finally, the DNN parameters can be continuously optimized during training iterations:

[0059]

[0060] Where i represents the i-th iteration, and η represents the learning rate. express About ω (i) The gradient.

[0061] Policy gradient in equation (2) This is crucial for learning mismatch detection. The following will detail the calculation process of the policy gradient: 1) Calculate the probability distribution p θ and with probability p θ 1) Obtain a subset of matching points through Monte Carlo sampling; 2) Calculate the score r of the sampled subset of matching points, that is, use the average value to approximate the expected score.

[0062] (2) Monte Carlo sample set

[0063] As described above, state-action (a|s) is action a that samples a subset Ω from the set of matching points s, where all elements of the subset Ω are sampled with probability p(a|s; θ). The problem of obtaining a subset of matching points based on probability p(a|s; θ) can be solved using Monte Carlo sampling. Figure 1 As shown, the classification network π outputs a relative matching probability RMP (i.e., θ) for each pair of matching points in set s, where θ can be initialized with a classification distribution:

[0064] Cat(y i |θ)=θ i (5)

[0065] In the formula, y i It is the i-th matching point pair in set s, θ = (θ 1 ,θ 2 ,…,θ n ), θ iIt is the i-th element in θ. Since the sampling process generally satisfies the independent and identically distributed assumption, the probability p of obtaining the sample set Ω is... θ for:

[0066]

[0067] Where m is the number of elements in the sampled matching subset Ω, and y j It is a matching point pair in the subset Ω, I(y) j ) indicates solving for y j The index number in set s.

[0068] Given the probability distribution p θ Then, Monte Carlo sampling can be used to obtain the matching point subset Ω, and the sampling process is shown in Algorithm 1.

[0069]

[0070] (3) Scoring of the sample set

[0071] After obtaining a subset Ω through Monte Carlo sampling, firstly, a geometric transformation model is calculated based on the sampled subset Ω; then, the error of each pair of matching points under the constraints of this model is calculated, and it is determined whether the error is less than a predefined threshold. If it is less than the threshold, it is added to the consensus set C corresponding to the geometric model; finally, the number of elements in the consensus set C is the state-action (a|s) score.

[0072] r=|C| (7)

[0073] In the formula, |·| represents the number of elements in the set.

[0074] If the number of samplings is large enough, the expectation in equation (2) can be approximated by the average value. That is, if algorithm 1 is run N times to generate N subsets, and the mean of the N sampled sets is used to approximate the expectation, then equation (2) can be transformed into:

[0075]

[0076] In the formula, r k This represents the score of the geometric model corresponding to the k-th sample set. This represents the j-th matching point pair in the k-th sample set. Indicates the solution The index number in set s, yes The relative matching probability RMP.

[0077] After solving equation (8), the optimal parameters ω of the classification network π can be obtained through equations (3) and (4). Finally, the optimized network π is used to identify correct and incorrect matches.

[0078] (4) ULMR network framework

[0079] The ULMR network π adds a softmax operation to the NM-Net network framework to learn the RMP for each matching point pair. According to equations (8), (3), and (4), the optimal parameters ω of NM-Net can be obtained through training iterations. The training process of ULMR is as follows... Figure 1 As shown in (a), based on the optimized NM-Net, Algorithm 1 is repeated N times to generate N sample sets and evaluate their corresponding geometric models. Finally, the largest consistent set with the most elements is taken as the set of correctly matched point pairs, thus completing the mismatch detection and removal. The prediction process of ULMR is as follows: Figure 1 As shown in (b).

[0080] The ULMR training settings are as follows: the number of samplings N in equation (8) is 100, the geometric transformation model is an epipolar constraint error model, and the epipolar error threshold in the consistent set solution process is 3.0 pixels; the training batch size is 32, the training epoch is 60; the optimizer is SGD, the learning rate decay algorithm is cosine annealing, and the initial learning rate is 0.01.

[0081] Three experiments were conducted: (1) a test experiment to verify the generalization ability of the invention; (2) an application experiment to verify the actual effect of the invention; and (3) an ablation experiment to test the efficiency and compatibility of the invention. Five mismatch detection methods were selected for comparison: hand-designed methods (RANSAC and GMS), supervised learning methods (LFGC and NM-Net), and unsupervised learning methods (ULCM).

[0082] The training image sets were brown_bm_1 from the SUN3D database and st_peters_square from the Yahoo YFCC100M database. For the supervised methods LFGC and NM-Net, firstly, the SIFT method was used to extract 2000 feature points for each image pair and perform matching; then, the structure-of-motion (COMO) method was used to accurately calculate the fundamental matrix; finally, matches with an epipolar error of less than 3.0 pixels were marked as correct matches, and the rest were marked as incorrect matches. For the unsupervised method ULCM, the mismatch rate of matched point pairs in different training batches was calculated. For the ULMR of this invention, no labels were used for correct and incorrect matches; training was performed only using the matched SIFT feature points.

[0083] (1) Test Experiment

[0084] The test experiment selected six image pairs, and the initial matching point set was obtained using the SIFT method. The initial matching point count for each of the six image pairs was 2000, and the average false match rate was higher than 80%. The false match detection results of ULMR are as follows: Figure 2-3 As shown.

[0085] The algorithm's performance is evaluated using the number of correct matches, the number of incorrect matches, and the accuracy (the ratio of correct matches to total matches). Figure 3 and Figure 4 Intuitively, compared to the five comparison methods, ULMR achieves better accuracy, fewer false matches, a more stable number of correct matches, and the correct matches are evenly distributed. Firstly, ULMR has the highest positional accuracy (positional accuracy is the symmetric polar error), such as... Figure 3 As shown in Figure (1), its average accuracy is close to 95.0%, and the lowest accuracy is even higher than 88.3%. Secondly, ULMR has the fewest RFMs, with an average number of mismatches of about 8 and a maximum number of mismatches of less than 15 (e.g., Figure 3 (See Figure 2). Third, although ULMR did not obtain the most correct matching points, the number of its correct matching points was greater than 50 (e.g., Figure 3 As shown in Figure (3), these correctly matched points are sufficient for applications such as airborne triangulation and self-localization; the correctly matched points of ULMR are more evenly distributed (e.g., Figure 3 As shown in the figure, this is beneficial for improving positioning accuracy. Therefore, the ULMR algorithm in this invention has better generalization ability and can obtain more matching points with higher accuracy.

[0086] (2) Application Experiment

[0087] The application experiment used the Reichstag dataset containing 1174 image pairs. First, the SIFT method was used to extract the initial matching point set between image pairs. Then, ULMR and the five comparison methods mentioned above were used to detect and remove mismatches from the initial matching point set, yielding the final matching results. Finally, for each image pair after mismatch detection, the 8-point algorithm was used to estimate its fundamental matrix, and its positional accuracy was calculated to evaluate the matching results. Here, average positional accuracy, median positional accuracy, and maximum positional accuracy were used to comprehensively evaluate the mismatch detection effect, and the results are as follows: Figure 4 As shown.

[0088] Depend on Figure 4It can be seen that for 1174 image pairs, ULMR achieved the highest quality mismatch detection results, significantly outperforming the other five comparison methods. First, ULMR exhibits the best stability; as shown in Figures 5(a) and (b), ULMR has the lowest box height, indicating minimal fluctuations in its average and median positional accuracy. Second, ULMR demonstrates the highest accuracy; as shown in Figures 5(a) and (b), ULMR has the lowest box height, indicating minimal fluctuations in its average and median positional accuracy. Figure 4 As shown in the left and middle figures, its average positional accuracy and median positional accuracy are both higher than all the comparison methods, while its maximum positional accuracy is lower than that of the comparison methods. Third, ULMR obtains the best quality of remaining matches; for example... Figure 4 As shown in the right figure, the average maximum positional accuracy is 3.69 pixels, and the median maximum positional accuracy is 3.86 pixels, which are the lowest among the several comparison methods. This demonstrates that the ULMR algorithm in this invention exhibits better stability and higher accuracy when dealing with high false-match rate data.

[0089] (3) Ablation test

[0090] The ablation experiments also used the Reichstag dataset containing 1174 image pairs to test the effect of ULMR sampling counts and the compatibility of the ULMR framework with other DNNs. Here, accuracy and recall (the ratio of correctly identified points to the total number of points) were used to quantitatively evaluate the mismatch detection results, and the classic RANSAC was used as a comparison method. The experimental results for ULMR sampling counts are as follows: Figure 5 As shown.

[0091] like Figure 5 As shown, when the number of MC samplings is 100, ULMR performs best overall, with the highest accuracy (approximately 0.90) and a relatively high match recall (approximately 0.30). RANSAC, on the other hand, performs best with 1500 samplings, achieving an accuracy of approximately 0.87 and a match recall of approximately 0.30. Therefore, ULMR achieves better performance with fewer samplings while ensuring high-quality detection results.

[0092] In the experiments described above, the classification network in the ULMR framework was NM-Net. The DNNs in the ULMR framework can be set to NM-Net, LFGC-Net, and PointNet respectively, resulting in three ensemble methods: ULMR+NM-Net, ULMR+LFGC-Net, and ULMR+PointNet. Accuracy, correct match recall (the ratio of correctly identified correct matches to all correct matches), and incorrect match recall (the ratio of correctly identified incorrect matches to all incorrect matches) were used to test the compatibility of the ULMR framework with other classification networks. The experimental results are as follows: Figure 6 As shown.

[0093] Depend on Figure 6 It can be seen that the three ensemble methods based on the ULMR framework can achieve good accuracy, correct match recall, and mismatch recall, indicating that the ULMR framework has high compatibility with commonly used mismatch deep learning networks (DNNs).

[0094] As can be seen from the above three experiments, namely the test experiment, application experiment and ablation experiment, the ULMR proposed in this invention has better stability, higher accuracy and efficiency, and better compatibility.

[0095] The above description provides examples of the preferred embodiments of the present invention. Parts not detailed herein are common knowledge to those skilled in the art. The scope of protection of the present invention is determined by the claims. Any equivalent modifications based on the technical teachings of the present invention are also within the scope of protection of the present invention.

Claims

1. A reinforcement learning based unsupervised mismatch detection method, characterized in that, The method comprises the following steps: Step 1, input the matching point set s into the detection network π, the detection network π generates an output relative matching probability θ for each matching point pair, θ = π(s; ω), ω is the network parameter of π; Step 2, initialize a classification distribution with the relative matching probability θ, and sample N times from s according to the classification distribution to obtain N matching point subsets, for any matching point subset Ω k , calculate its corresponding geometric transformation model h k ; Step 3, the updated network parameters ω' are calculated by the formula wherein η represents a preset learning rate, wherein automatically solved by a tool in a deep learning library wherein r k denotes the score of the geometric transformation model h k denotes the jth matching point pair of the kth matching sub-set, denotes the solution of the index number in the set s, is the corresponding relative matching probability;​ After ω' is calculated, the iteration number is increased by 1, and it is judged whether the iteration end condition is met, if yes, the next step is entered, if no, step 1 is entered, and the network parameter ω in step 1 is updated to ω'; Step 4, save the network parameter obtained in the last iteration to obtain the final optimized network; Step 5, input the to-be-detected matching point set t into the final optimized network to obtain the relative matching probability θ of the matching points; initialize a classification distribution by using θ, and repeatedly sample M times from t according to the distribution to obtain M matching point subsets and calculate the model corresponding to each matching point subset; Step 6, count the consistent set corresponding to each model, and take the consistent set containing the most matching points as the correct matching point pair set corresponding to the to-be-detected matching point set.

2. The reinforcement learning based unsupervised mismatch detection method of claim 1, wherein, The method for scoring the geometric transformation model corresponding to the matching point subset comprises the following steps: Step 1, calculate the geometric transformation model corresponding to the matching point subset based on the matching point subset; Step 2, calculate the error of each pair of matching points under the constraint of the geometric transformation model in step 1, and judge whether the error is less than a predefined threshold; if yes, the matching point is added to set C, otherwise not; repeat the above process until all matching points in the matching point set are detected; set C is the consistent set of the geometric transformation model; Step 3, the number of elements in the consistent set C is the score of the geometric transformation model.

3. The reinforcement learning based unsupervised mismatch detection method of claim 1, wherein, The iteration end condition is that the iteration number reaches a preset maximum iteration number.

4. The reinforcement learning based unsupervised mismatch detection method of claim 1, wherein, The number of matching point pairs in each matching point subset in step 1 is the same.

5. The reinforcement learning based unsupervised mismatch detection method of claim 1, wherein, The number of matching point pairs in each sampling subset in step 4 is the same.

6. The reinforcement learning based unsupervised mismatch detection method of claim 1, wherein, M = N.

7. The reinforcement learning based unsupervised mismatch detection method of claim 1, wherein, The sampling method in steps 1 and 4 is the Monte Carlo sampling method.

8. The reinforcement learning based unsupervised mismatch detection method of claim 1, wherein, The step 2 in Can be automatically derived by pytorch or tensorflow in deep learning library.

Images