Data sampling and abnormal point removing method and system based on reinforcement learning
Through data sampling and outlier removal methods based on reinforcement learning, the performance limitations of traditional image matching methods in large-scale data sets and complex environments are solved, the accuracy and efficiency of image matching are improved, and the dependence on labeled data is reduced.
Patent Information
- Application Number
- CN202510692191.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-27
- Publication Date
- 2025-09-05
AI Technical Summary
Traditional image matching methods have limited performance when processing large-scale data sets or complex environments, and it is difficult to effectively remove outliers, which affects matching accuracy and efficiency.
A reinforcement learning-based data sampling and outlier removal method is adopted. Through multimodal feature extraction, reinforcement learning agent construction, feature point relationship graph construction and multi-stage post-processing, the graph convolutional network and attention mechanism are used to optimize the matching strategy, and the strategy is dynamically adjusted to adapt to different data distributions and noise levels.
It improves the performance of image matching and data utilization efficiency, reduces the dependence on large amounts of labeled data, enhances the ability to identify outliers, and reduces the occurrence of mismatches.
Smart Images
Figure CN120599296A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image matching, and in particular relates to a data sampling and outlier removal method and system based on reinforcement learning. Background Art
[0002] Image matching is a key branch of computer vision, involving the problem of finding correspondences between different images. This technology plays a key role in many applications, such as object recognition, 3D reconstruction, image stitching, and robotic navigation. With the rapid development of computer vision, the research and application of image matching technology has also continued to advance. In particular, driven by deep learning, the accuracy and efficiency of image matching have been significantly improved.
[0003] Traditional image matching methods are mainly based on feature extraction and feature description. These methods generally include feature detection, feature description, feature matching, and geometric consistency verification. These methods work well when processing small-scale datasets, but their performance is often limited when processing large-scale datasets or complex environments.
[0004] Therefore, how to provide a data sampling and outlier removal method and system based on reinforcement learning, so that by interacting with the environment, the reinforcement learning model can learn to select the most representative strategy in specific situations, thereby improving data utilization efficiency and image matching performance, has become a technical problem that needs to be solved urgently. Summary of the Invention
[0005] The embodiment of the present invention provides a data sampling and outlier removal method and system based on reinforcement learning. By interacting with the environment, the reinforcement learning model can learn to select the most representative strategy in specific situations, thereby improving data utilization efficiency and image matching performance.
[0006] In one embodiment of the present invention, a method for data sampling and outlier removal based on reinforcement learning is provided, comprising:
[0007] S101, multimodal feature extraction, constructing correspondence features, extracting key points and their descriptors from the two input images, generating an initial correspondence set and performing coordinate normalization processing to obtain robust basic features;
[0008] S102: Construct a reinforcement learning agent, and train the agent through interactive simulation to learn optimal strategies for data sampling and outlier removal, wherein: the state space includes the characteristics of the current dataset, the number of identified outliers, and the sampling strategy; the action space includes the selection of the sampling strategy and the outlier removal method; and the reward function dynamically adjusts the strategy based on the correct or incorrect matching results;
[0009] S103. Construct a feature point relationship graph, using nodes as matching points. Edge weights are calculated jointly by feature similarity and spatial distance. Node representations are iteratively updated using a graph convolutional network (GCN). The association between key nodes is strengthened through an attention mechanism. Based on the clustering results and confidence thresholds of the node representations, normal matching points and outliers are dynamically segmented.
[0010] S104, multi-stage post-processing and image alignment, multi-scale geometric verification of the screened high-confidence matching points, use the improved PROSAC algorithm to eliminate local inconsistent points, estimate the affine transformation matrix based on the optimal matching point set, and introduce illumination invariance constraints to optimize image alignment accuracy.
[0011] Furthermore, a reinforcement learning agent is constructed and trained through interactive simulation of the environment to learn the optimal strategy for data sampling and outlier removal, including:
[0012] The Q-learning reinforcement learning algorithm is used, and its update formula is:
[0013] Q(s,a)←Q(s,a)+α[r+γmax a′ Q(s′,a′)-Q(s,a)]
[0014] Among them, α is the learning rate; γ is the discount factor; r is the immediate reward; s' is the next state.
[0015] The agent learns interactively in a simulated environment, which includes the generation of data sets and the simulation of outliers. The agent selects an action based on the current state and observes the results.
[0016] Reward=Reward Function(s,a)
[0017] Among them, the Reward Function is the reward calculated based on the behavior and results of the agent.
[0018] The agent implements different data sampling and outlier removal strategies to accumulate experience.
[0019] Furthermore, the method comprises:
[0020] Based on the behavior of the agent, corresponding rewards or penalties are given; reward function: when the agent successfully identifies and matches the correct point, it is given a positive reward; when the agent incorrectly matches the point or fails to identify the outlier point, it is given a negative penalty;
[0021]
[0022] Continuously adjust the agent's behavior optimization strategy based on reward and penalty signals to perform policy updates:
[0023] π(s)←π(s)+β[Q(s,a)-π(s)]
[0024] Where β is the policy update rate.
[0025] Furthermore, a feature point relationship graph is constructed, with nodes as matching points. Edge weights are calculated jointly by feature similarity and spatial distance. Graph convolutional networks (GCNs) are used to iteratively update node representations, and the association between key nodes is strengthened through the attention mechanism, including:
[0026] Using the extracted feature descriptors, we search for matching keypoint pairs between the two images, and use the RANSAC algorithm to verify the matching points and identify and remove outliers caused by incorrect matching:
[0027] M=RANSAC(S)
[0028] Where M is a set of consistent matching point pairs screened by the RANSAC algorithm;
[0029] Based on the image matching process derived from continuous learning by the intelligent agent, convolutional neural network and graph neural network modules are added. MLP is used as the decision layer and the output of CNN and GNN is combined to determine which points are outliers.
[0030] Use CNN to extract deep features from the image, and then train a CNN model to identify outliers. This is achieved by training a binary classification model, which learns to distinguish between normal points and outliers:
[0031] F=CNN(I)
[0032] P=σ(W f F+b f )
[0033] Among them, F represents the extracted features, I represents the input image; P is the predicted outlier probability, W f and b f are the weights and biases of the model, and σ is the sigmoid activation function.
[0034] Furthermore, the method comprises:
[0035] Construct a graph where nodes represent feature points and edges represent relationships between feature points. Use graph convolutional neural networks to process graph structure data, learn node representations, and use the node representations learned by GNN to identify outliers.
[0036] G=(V,E)
[0037]
[0038] O=GNN(G)
[0039] Among them, V is the node set, E is the edge set, H (l) is the node feature of the lth layer, A is the adjacency matrix, D is the degree matrix, W (l) is the weight matrix, O is the output node representation for outlier detection.
[0040] Use GNN to optimize matching points, improve matching accuracy by learning the spatial relationship between matching points, and use MLP to classify matching points to distinguish correct matches from incorrect matches:
[0041] H′=GNN(G′)
[0042] C=MLP(H′)
[0043] Among them, G′ is a graph constructed based on matching points, H′ is the representation of optimized matching points, and C is the classification result, indicating the correctness of the matching points.
[0044] Furthermore, in the method, outlier removal includes:
[0045] Maximize the cumulative reward to learn the optimal strategy to eliminate abnormal matching points and screen out high-confidence matching points. The outlier removal strategy is:
[0046]
[0047] in, is the average of all Q values, and N is the number of samples.
[0048] Furthermore, multi-stage post-processing and image alignment are performed, including:
[0049] Estimate the collective transformation between images based on matching points and apply the collective transformation to align the images:
[0050] Transformation=Estimate Transformation(High Confidence Matches)
[0051] I aligned =ApplyTransformation(I2, Transformation).
[0052] Furthermore, key points and their descriptors are extracted from the two input images, including:
[0053] By including at least feature detection algorithms SIFT and SuperPoint, key points and their descriptors for identifying similar areas between images are extracted from the two input images I and I'.
[0054] I keypoints =FeatureDetector(I,I′)
[0055] Use the nearest neighbor matching strategy to generate the initial correspondence set S and normalize the coordinates of the matching pairs to ensure that the feature descriptors have the possibility between different images and different batches.
[0056] S=Nearest Neighbor Matching(I keypoints1 , I keypoints2 )
[0057] I resized =Resize(I,tar get_size)
[0058] F=BN(IN(S))
[0059] Among them, N represents normalization, BN represents batch normalization, and F represents the robust basic features obtained after normalization.
[0060] In another embodiment of the present invention, a data sampling and outlier removal system based on reinforcement learning is based on any one of the above methods for data sampling and outlier removal based on reinforcement learning, characterized in that the detection system comprises: a feature extraction module, a strategy optimization engine, a graph neural network processor, and a multi-stage verification unit;
[0061] The feature extraction module is used for multimodal feature extraction, constructing correspondence features, extracting key points and their descriptors from the two input images, generating an initial correspondence set and performing coordinate normalization processing to obtain robust basic features;
[0062] The strategy optimization engine is used to construct a reinforcement learning agent, and interactively train the agent through a simulated environment to enable it to learn the optimal strategy for data sampling and outlier removal, wherein: the state space includes the characteristics of the current data set, the number of identified outliers, and the sampling strategy; the action space includes the selection of the sampling strategy and the outlier removal method; and the reward function dynamically adjusts the strategy based on the correct or incorrect matching results;
[0063] The graph neural network processor is used to construct a feature point relationship graph, using nodes as matching points. The edge weights are calculated by jointly calculating feature similarity and spatial distance. The node representation is iteratively updated using a graph convolutional network (GCN). The association between key nodes is strengthened through an attention mechanism. Based on the clustering results and confidence thresholds of the node representations, normal matching points and outliers are dynamically segmented.
[0064] The multi-stage verification unit is used for multi-stage post-processing and image alignment, performs multi-scale geometric verification on the screened high-confidence matching points, uses an improved PROSAC algorithm to eliminate local inconsistent points, estimates the affine transformation matrix based on the optimal matching point set, and introduces illumination invariance constraints to optimize image alignment accuracy.
[0065] In another embodiment of the present invention, the system includes: a processor and a memory, the memory storing a computer program, and when the computer program is executed by the processor, implementing any of the above-mentioned methods for data sampling and outlier removal based on reinforcement learning.
[0066] The beneficial effects brought about by the present invention are as follows:
[0067] As can be seen from the above scheme, the embodiment of the present invention provides a data sampling and outlier removal method and system based on reinforcement learning, including: multimodal feature extraction, construction of corresponding relationship features, extraction of key points and their descriptors from two input images, generation of an initial corresponding relationship set and coordinate normalization processing; construction of a reinforcement learning agent, training the agent through simulated environment interaction to enable it to learn the optimal strategy for data sampling and outlier removal; construction of a feature point relationship graph, dynamic segmentation of normal matching points and outliers based on the clustering results and confidence thresholds represented by the nodes; multi-stage post-processing and image alignment, multi-scale geometric verification of the screened high-confidence matching points, use of the improved PROSAC algorithm to eliminate local inconsistent points, estimation of the affine transformation matrix based on the optimal matching point set, and introduction of illumination invariance constraints to optimize image alignment accuracy. The technical solution of the present invention, through interaction with the environment, the reinforcement learning model can learn to select the most representative strategy in a specific situation, thereby improving data utilization efficiency and image matching performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] Figure 1 This is a flow chart of a data sampling and outlier removal method based on reinforcement learning according to an embodiment of the present invention. DETAILED DESCRIPTION
[0069] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.
[0070] like Figure 1 As shown, Figure 1 This is a flow chart of a model fusion method based on federated learning in an embodiment of the present invention.
[0071] Figure 1 In [1], a data sampling and outlier removal method based on reinforcement learning includes:
[0072] S101, multimodal feature extraction, constructing correspondence features, extracting key points and their descriptors from the two input images, generating an initial correspondence set and performing coordinate normalization processing to obtain robust basic features;
[0073] S102: Construct a reinforcement learning agent, and train the agent through interactive simulation to learn optimal strategies for data sampling and outlier removal, wherein: the state space includes the characteristics of the current dataset, the number of identified outliers, and the sampling strategy; the action space includes the selection of the sampling strategy and the outlier removal method; and the reward function dynamically adjusts the strategy based on the correct or incorrect matching results;
[0074] S103. Construct a feature point relationship graph, using nodes as matching points. Edge weights are calculated jointly by feature similarity and spatial distance. Node representations are iteratively updated using a graph convolutional network (GCN). The association between key nodes is strengthened through an attention mechanism. Based on the clustering results and confidence thresholds of the node representations, normal matching points and outliers are dynamically segmented.
[0075] S104, multi-stage post-processing and image alignment, multi-scale geometric verification of the screened high-confidence matching points, use the improved PROSAC algorithm to eliminate local inconsistent points, estimate the affine transformation matrix based on the optimal matching point set, and introduce illumination invariance constraints to optimize image alignment accuracy.
[0076] In an embodiment of the present invention, a data sampling and outlier removal method based on reinforcement learning is provided. Reinforcement learning can make effective decisions in a dynamic environment, making the data sampling process more intelligent and. Through interaction with the environment, the reinforcement learning model can learn to select the most representative strategy in a specific situation, thereby improving data utilization efficiency and image matching performance. When dealing with the problem of removing outliers, the strategy can be dynamically adjusted to adapt to different data distributions and noise levels, which can not only improve the model's ability to identify outliers, but also reduce incorrect matches caused by outliers. In addition, the reinforcement learning-based method can reduce dependence on large amounts of labeled data because it can learn from interactions with the environment, rather than relying solely on static data sets.
[0077] In one embodiment of the present invention, a reinforcement learning agent is constructed that can optimize data sampling and outlier removal strategies through reinforcement learning. The agent's state space, action space, and reward function are as follows: The state space S includes the eigenvalues of the current dataset, the number of identified outliers, the current sampling strategy, and other information. The action space A includes the selection of different sampling strategies and outlier removal methods. The reward function R(s, a) adjusts the agent's strategy based on the results of its behavior. For example, positive rewards are given for successfully identifying and matching the correct points, while negative penalties are given for incorrect matching or failure to identify outliers.
[0078] The agent is trained through interactive simulation to learn optimal strategies for data sampling and outlier removal, including:
[0079] The Q-learning reinforcement learning algorithm is used, and its update formula is:
[0080] Q(s,a)←Q(s,a)+α[r+γmax a′ Q(s′,a′)-Q(s,a)]
[0081] Among them, α is the learning rate; γ is the discount factor; r is the immediate reward; s' is the next state.
[0082] The agent learns interactively in a simulated environment, which includes the generation of data sets and the simulation of outliers. The agent selects an action based on the current state and observes the results, i.e., the new state and reward. For example:
[0083] Reward=Reward Function(s,a)
[0084] Among them, the Reward Function is the reward calculated based on the behavior and results of the agent.
[0085] The agent implements different data sampling and outlier removal strategies to accumulate experience.
[0086] In one embodiment of the present invention, the method, data sampling and outlier removal, comprises:
[0087] Based on the results of the agent's behavior, corresponding rewards or penalties are given; reward function: when the agent successfully identifies and matches the correct points, positive rewards are given; when the agent incorrectly matches points or fails to identify outliers, negative penalties are given; this can encourage the agent to learn how to identify matching points and outliers more accurately.
[0088]
[0089] Continuously adjust the agent's behavior optimization strategy based on reward and penalty signals to perform policy updates:
[0090] π(s)←π(s)+β[Q(s,a)-π(s)]
[0091] Among them, β is the policy update rate, and the value of β is usually between 0.01 and 0.1.
[0092] In one embodiment of the present invention, an agent uses an optimal strategy learned by the agent to guide image matching. The agent uses extracted feature descriptors to find matching keypoint pairs between two images. This can be achieved by calculating the distance between the descriptors (e.g., Euclidean distance). Because the agent has optimized the matching strategy, it is better able to select the optimal matching algorithm or adjust matching parameters.
[0093] Construct a feature point relationship graph, using nodes as matching points. Edge weights are calculated jointly by feature similarity and spatial distance. Graph Convolutional Network (GCN) is used to iteratively update node representations. The attention mechanism is used to strengthen the association between key nodes, including:
[0094] Using the extracted feature descriptors, we search for matching key point pairs between the two images. To ensure the geometric consistency between the matching point pairs, we use the RANSAC algorithm to verify the matching point pairs and identify and remove outliers caused by incorrect matching:
[0095] M=RANSAC(S)
[0096] Where M is a set of consistent matching point pairs screened by the RANSAC algorithm;
[0097] Based on the image matching process derived from continuous learning by the intelligent agent, convolutional neural network and graph neural network modules are added. MLP is used as the decision layer and the output of CNN and GNN is combined to determine which points are outliers.
[0098] Use CNN to extract deep features from images, and then train a CNN model to identify outliers. This is achieved by training a binary classification model, where the model learns to distinguish between normal points and outliers.
[0099] F=CNN(I)
[0100] P=σ(W f F+b f )
[0101] Among them, F represents the extracted features, I represents the input image; P is the predicted outlier probability, W f and b g are the weights and biases of the model, and σ is the sigmoid activation function.
[0102] In one embodiment of the present invention, the method includes:
[0103] Construct a graph where nodes represent feature points and edges represent relationships between feature points. Use graph convolutional neural networks to process graph structure data, learn node representations, and use the node representations learned by GNN to identify outliers.
[0104] G=(V,E)
[0105]
[0106] O=GNN(G)
[0107] Among them, V is the node set, E is the edge set, H (l) is the node feature of the lth layer, A is the adjacency matrix, D is the degree matrix, W (l) is the weight matrix, O is the output node representation for outlier detection.
[0108] Use GNN to optimize matching points, improve matching accuracy by learning the spatial relationship between matching points, and use MLP to classify matching points to distinguish correct matches from incorrect matches:
[0109] H′=GNN(G′)
[0110] C=MLP(H′)
[0111] Among them, G′ is a graph constructed based on matching points, H′ is the representation of optimized matching points, and C is the classification result, indicating the correctness of the matching points.
[0112] In one embodiment of the present invention, in the method, outlier removal includes:
[0113] Maximize the cumulative reward to learn the optimal strategy to eliminate abnormal matching points and screen out high-confidence matching points. The outlier removal strategy is:
[0114]
[0115] in, is the average of all Q values, and N is the number of samples.
[0116] Furthermore, multi-stage post-processing and image alignment are performed, including:
[0117] Estimate the collective transformation between images based on matching points and apply the collective transformation to align the images:
[0118] Transformation=Estimate Transformation(High Confidence Matches)
[0119] I aligned =ApplyTransformation(I2, Transformation).
[0120] In one embodiment of the present invention, extracting key points and their descriptors from two input images includes:
[0121] By including at least the feature detection algorithms SIFT and SuperPoint, key points and their descriptors for identifying similar areas between the two input images I and I' are extracted:
[0122] I keypoints =FeatureDetector(I,I′)
[0123] The nearest neighbor matching strategy is used to generate the initial correspondence set S, and the coordinates of the matching pairs are normalized, including instance normalization and batch normalization, to ensure that the feature descriptors are possible between different images and different batches, thereby improving the generalization ability of the model.
[0124] S=Nearest Neighbor Matching(I keypoints1 , I keypoints2 )
[0125] I resized =Resize(I,target_size)
[0126] F=BN(IN(S))
[0127] Among them, N represents normalization, BN represents batch normalization, and F represents the robust basic features obtained after normalization.
[0128] In another embodiment of the present invention, a data sampling and outlier removal system based on reinforcement learning is based on any one of the above methods for data sampling and outlier removal based on reinforcement learning, characterized in that the detection system comprises: a feature extraction module, a strategy optimization engine, a graph neural network processor, and a multi-stage verification unit;
[0129] The feature extraction module is used for multimodal feature extraction, constructing correspondence features, extracting key points and their descriptors from the two input images, generating an initial correspondence set and performing coordinate normalization processing to obtain robust basic features;
[0130] The strategy optimization engine is used to construct a reinforcement learning agent, and interactively train the agent through a simulated environment to enable it to learn the optimal strategy for data sampling and outlier removal, wherein: the state space includes the characteristics of the current data set, the number of identified outliers, and the sampling strategy; the action space includes the selection of the sampling strategy and the outlier removal method; and the reward function dynamically adjusts the strategy based on the correct or incorrect matching results;
[0131] The graph neural network processor is used to construct a feature point relationship graph, using nodes as matching points. The edge weights are calculated by jointly calculating feature similarity and spatial distance. The node representation is iteratively updated using a graph convolutional network (GCN). The association between key nodes is strengthened through an attention mechanism. Based on the clustering results and confidence thresholds of the node representations, normal matching points and outliers are dynamically segmented.
[0132] The multi-stage verification unit is used for multi-stage post-processing and image alignment, performs multi-scale geometric verification on the screened high-confidence matching points, uses an improved PROSAC algorithm to eliminate local inconsistent points, estimates the affine transformation matrix based on the optimal matching point set, and introduces illumination invariance constraints to optimize image alignment accuracy.
[0133] In another embodiment of the present invention, the system includes: a processor and a memory, the memory storing a computer program, and when the computer program is executed by the processor, implementing any of the above-mentioned methods for data sampling and outlier removal based on reinforcement learning.
[0134] In one embodiment of the present invention, a data sampling and outlier removal method and system based on reinforcement learning are provided, including: multimodal feature extraction, construction of correspondence features, extraction of key points and their descriptors from two input images, generation of an initial correspondence set and coordinate normalization processing; construction of a reinforcement learning intelligent agent, interactive training of the intelligent agent through a simulated environment, so that it learns the optimal strategy for data sampling and outlier removal; construction of a feature point relationship graph, dynamic segmentation of normal matching points and outlier points based on the clustering results and confidence thresholds represented by the nodes; multi-stage post-processing and image alignment, multi-scale geometric verification of the screened high-confidence matching points, use of an improved PROSAC algorithm to eliminate local inconsistent points, estimation of the affine transformation matrix based on the optimal matching point set, and introduction of illumination invariance constraints to optimize image alignment accuracy.
[0135] The technical solution of the present invention, through interaction with the environment, the reinforcement learning model can learn to select the most representative strategy in a specific situation, thereby improving data utilization efficiency and image matching performance.
[0136] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A data sampling and outlier removal method based on reinforcement learning, characterized in that: The method comprises: S101, multimodal feature extraction, constructing correspondence features, extracting key points and their descriptors from the two input images, generating an initial correspondence set and performing coordinate normalization processing to obtain robust basic features; S102: Construct a reinforcement learning agent, and train the agent through interactive simulation to learn optimal strategies for data sampling and outlier removal, wherein: the state space includes the characteristics of the current dataset, the number of identified outliers, and the sampling strategy; the action space includes the selection of the sampling strategy and the outlier removal method; and the reward function dynamically adjusts the strategy based on the correct or incorrect matching results; S103. Construct a feature point relationship graph, using nodes as matching points. Edge weights are calculated jointly by feature similarity and spatial distance. Node representations are iteratively updated using a graph convolutional network (GCN). The association between key nodes is strengthened through an attention mechanism. Based on the clustering results and confidence thresholds of the node representations, normal matching points and outliers are dynamically segmented. S104, multi-stage post-processing and image alignment, multi-scale geometric verification of the screened high-confidence matching points, use the improved PROSAC algorithm to eliminate local inconsistent points, estimate the affine transformation matrix based on the optimal matching point set, and introduce illumination invariance constraints to optimize image alignment accuracy.
2. The data sampling and outlier removal method based on reinforcement learning according to claim 1, characterized in that: Construct a reinforcement learning agent and train it through interactive simulation to learn the optimal strategy for data sampling and outlier removal, including: The Q-learning reinforcement learning algorithm is used, and its update formula is: Q(s,a)←Q(s,a)+α[r+γmax a′ Q(s′,a′)-Q(s,a)] Among them, α is the learning rate; γ is the discount factor; r is the immediate reward; s' is the next state. The agent learns interactively in a simulated environment, which includes the generation of data sets and the simulation of outliers. The agent selects an action based on the current state and observes the results. Reward=Reward Function(s,a) Among them, the Reward Function is the reward calculated based on the behavior and results of the agent. The agent implements different data sampling and outlier removal strategies to accumulate experience.
3. The data sampling and outlier removal method based on reinforcement learning according to claim 2, characterized in that: The method comprises: Based on the behavior of the agent, corresponding rewards or penalties are given; reward function: when the agent successfully identifies and matches the correct point, it is given a positive reward; when the agent incorrectly matches the point or fails to identify the outlier point, it is given a negative penalty; Continuously adjust the agent's behavior optimization strategy based on reward and penalty signals to perform policy updates: π(s)←π(s)+β[Q(s,a)-π(s)] Where β is the policy update rate.
4. The data sampling and outlier removal method based on reinforcement learning according to claim 1, characterized in that: Construct a feature point relationship graph, using nodes as matching points. Edge weights are calculated jointly by feature similarity and spatial distance. Graph Convolutional Network (GCN) is used to iteratively update node representations. The attention mechanism is used to strengthen the association between key nodes, including: Using the extracted feature descriptors, we search for matching keypoint pairs between the two images, and use the RANSAC algorithm to verify the matching points and identify and remove outliers caused by incorrect matching: M=RANSAC(S) Where M is a set of consistent matching point pairs screened by the RANSAC algorithm; Based on the image matching process derived from continuous learning by the intelligent agent, convolutional neural network and graph neural network modules are added. MLP is used as the decision layer and the output of CNN and GNN is combined to determine which points are outliers. Use CNN to extract deep features from the image, and then train a CNN model to identify outliers. This is achieved by training a binary classification model, which learns to distinguish between normal points and outliers: F=CNN(I) P=σ(W f F+b f ) Among them, F represents the extracted features, I represents the input image; P is the predicted outlier probability, W f and b f are the weights and biases of the model, and σ is the sigmoid activation function.
5. The data sampling and outlier removal method based on reinforcement learning according to claim 4, characterized in that: The method comprises: Construct a graph where nodes represent feature points and edges represent relationships between feature points. Use graph convolutional neural networks to process graph structure data, learn node representations, and use the node representations learned by GNN to identify outliers. G=(V,E) O=GNN(G) Among them, V is the node set, E is the edge set, H (l) is the node feature of the lth layer, A is the adjacency matrix, D is the degree matrix, W (l) is the weight matrix, O is the output node representation for outlier detection. Use GNN to optimize matching points, improve matching accuracy by learning the spatial relationship between matching points, and use MLP to classify matching points to distinguish correct matches from incorrect matches: H′=GNN(G′) C=MLP(H′) Among them, G′ is a graph constructed based on matching points, H′ is the representation of optimized matching points, and C is the classification result, indicating the correctness of the matching points.
6. The data sampling and outlier removal method based on reinforcement learning according to claim 5, characterized in that: In the method, outlier removal includes: Maximize the cumulative reward to learn the optimal strategy to eliminate abnormal matching points and screen out high-confidence matching points. The outlier removal strategy is: in, is the average of all Q values, and N is the number of samples.
7. The data sampling and outlier removal method based on reinforcement learning according to claim 1, characterized in that: Multi-stage post-processing and image alignment, including: Estimate the collective transformation between images based on matching points and apply the collective transformation to align the images: Transformation=Estimate Transformation(High Confidence Matches) I aligned =ApplyTransformation(I2,Transformation)。 8. The data sampling and outlier removal method based on reinforcement learning according to claim 1, characterized in that: Extract key points and their descriptors from the two input images, including: By including at least feature detection algorithms SIFT and SuperPoint, key points and their descriptors for identifying similar areas between images are extracted from the two input images I and I'. I keypoints =FeatureDetector(I,I′) Use the nearest neighbor matching strategy to generate the initial correspondence set S and normalize the coordinates of the matching pairs to ensure that the feature descriptors have the possibility between different images and different batches. S=Nearest Neighbor Matching(I keypoints1 ,I keypoints2 ) I resized =Resize(I,target_size) F=BN(IN(S)) Among them, N represents normalization, BN represents batch normalization, and F represents the robust basic features obtained after normalization.
9. A data sampling and outlier removal system based on reinforcement learning, based on the data sampling and outlier removal method based on reinforcement learning according to any one of claims 1 to 8, characterized in that: The detection system includes: a feature extraction module, a strategy optimization engine, a graph neural network processor and a multi-stage verification unit; The feature extraction module is used for multimodal feature extraction, constructing correspondence features, extracting key points and their descriptors from the two input images, generating an initial correspondence set and performing coordinate normalization processing to obtain robust basic features; The strategy optimization engine is used to construct a reinforcement learning agent, and interactively train the agent through a simulated environment to enable it to learn the optimal strategy for data sampling and outlier removal, wherein: the state space includes the characteristics of the current data set, the number of identified outliers, and the sampling strategy; the action space includes the selection of the sampling strategy and the outlier removal method; and the reward function dynamically adjusts the strategy based on the correct or incorrect matching results; The graph neural network processor is used to construct a feature point relationship graph, using nodes as matching points. The edge weights are calculated by jointly calculating feature similarity and spatial distance. The node representation is iteratively updated using a graph convolutional network (GCN). The association between key nodes is strengthened through an attention mechanism. Based on the clustering results and confidence thresholds of the node representations, normal matching points and outliers are dynamically segmented. The multi-stage verification unit is used for multi-stage post-processing and image alignment, performs multi-scale geometric verification on the screened high-confidence matching points, uses an improved PROSAC algorithm to eliminate local inconsistent points, estimates the affine transformation matrix based on the optimal matching point set, and introduces illumination invariance constraints to optimize image alignment accuracy.
10. A data sampling and outlier removal system based on reinforcement learning, characterized in that: The system includes: a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the data sampling and outlier removal method based on reinforcement learning according to any one of claims 1 to 8.