A reinforcement learning-based image comparison and recognition method and device

Abstract

This invention belongs to the field of intelligent agent visual navigation technology. It discloses an image comparison and recognition method and device based on reinforcement learning, including the following steps: (1) designing the target recognition process as an MDP problem; (2) designing a domain randomization method that integrates the reservoir sampling method to sample multiple feature values ​​after randomization of the environmental feature domain; (3) gradually increasing environmental variable factors and task difficulty; (4) using a reward function to evaluate the strategy generated by the agent; (5) designing an experience replay mechanism and setting an experience buffer; (6) using a random sampling method for actions to enable the agent to interact with the environment to obtain a large amount of initial experience, while using a network loss experience replay mechanism to filter the experience in the experience pool for learning; (7) dynamically adjusting the exploratory desire coefficient in the agent's learning process, thereby realizing image comparison and recognition. This invention improves the generalization ability of target recognition.

Description

Technical Field

[0001] This invention belongs to the field of intelligent agent visual navigation technology, and more specifically, relates to an image comparison and recognition method and device based on reinforcement learning. Background Technology

[0002] In visual navigation systems for intelligent agents, environmental perception and visual target detection are crucial components. Image comparison and recognition involves extracting features from a target image and environmental images containing the target. By comparing the similarity of features between the two images, the target can be identified and located. Theoretically, this method can recognize any given image, which is of great significance for enabling autonomous visual navigation of robots when the specific coordinates of the target are unknown. Deep learning methods are commonly used for target recognition and localization in intelligent agent visual navigation. Currently, the most successful recognition algorithms include template matching, YOLO, R-CNN, and Triplet. Triplet, in particular, achieves target image recognition by self-calibrating a large number of positive and negative samples and comparing them with the target image.

[0003] Currently, with increasingly demanding requirements for generalization performance and operating environments in agent-based visual navigation methods, traditional deep learning methods are struggling to meet these needs. Therefore, improvements to target recognition methods have garnered significant attention from researchers. These improvements primarily focus on network structure and dataset creation, reducing training difficulty and time by simplifying the network structure and creating dedicated training sets tailored to specific operating environments, thus enhancing the method's environmental adaptability. However, current methods are still constrained by the need for large datasets, making it difficult to meet the demands for visual recognition generalization capabilities across different scenarios. Therefore, for agent-based visual navigation methods, it is necessary to explore how to combine unsupervised reinforcement learning methods with target detection tasks and achieve target recognition across different scenarios, thereby reducing deployment costs and improving the stability and generalization of agent-based visual navigation. Summary of the Invention

[0004] To address the aforementioned shortcomings or improvement needs of existing technologies, this invention provides an image comparison and recognition method and device based on reinforcement learning. It leverages the powerful exploratory capabilities of unsupervised reinforcement learning to improve the generalization ability of target recognition. Simultaneously, it utilizes curriculum learning and domain randomization methods to progressively adjust the training environment of the agent, thereby enhancing the stability and generalization of the target recognition method.

[0005] To achieve the above objectives, according to one aspect of the present invention, an image comparison and recognition method based on reinforcement learning is provided, the method comprising the following steps:

[0006] (1) Based on the reinforcement learning model, the target recognition task is modeled by Markov decision to design the target recognition process as an MDP problem.

[0007] (2) Build a simulation environment, design a domain randomization method that integrates the reservoir sampling method, and use the error information in the training process to determine whether to apply domain randomization to adjust the training environment. At the same time, use the reservoir sampling method to sample and obtain multiple feature values ​​after the environmental feature domain randomization.

[0008] (3) Adopt a learning evaluation mechanism designed using a curriculum learning model to gradually increase environmental variables and task difficulty;

[0009] (4) The reward function based on the potential field idea is used to evaluate the policy generated by the agent and optimize the policy during the training process;

[0010] (5) An experience replay mechanism is designed based on the training requirements of reinforcement learning, and an experience cache is set up. The experience cache is used to store the agent's observation space, action space, reward value, state space and policy function during the training process.

[0011] (6) The warm-up mechanism is used to initialize the experience buffer, and the random sampling method of actions is used to enable the agent to interact with the environment to obtain a large amount of initial experience. At the same time, the network loss experience replay mechanism is used to filter the experience in the experience pool for learning.

[0012] (7) Based on the internal exploratory desire mechanism of the reinforcement learning SAC algorithm, the butterfly optimization algorithm and the stage learning evaluation mechanism, the exploratory desire coefficient of the agent in the learning process is dynamically adjusted, thereby realizing image comparison and recognition.

[0013] Furthermore, the MDP problem involves designing the state space, observation space, action space, and objective function.

[0014] Furthermore, the state space is objectively all the characteristics of an intelligent agent in its environment;

[0015] The description of the environment perceived by an agent is defined as the agent's observation space, denoted by O. In target comparison and recognition modeling, O is represented as:

[0016] O={P env ,P tar}

[0017] Among them, P env and P tar These represent the environment image and the target image, respectively.

[0018] Furthermore, the set of actions that an agent can take is called the agent's action space, denoted by A, where,

[0019] A={(x,y)|(X tar ,Y tar = (W·x, H·y), x, y ∈ (0, 1)}

[0020] In the formula, (x,y) are the direct output of the neural network, which are two numbers with values ​​ranging from (0,1); the width and height (W,H) of the original image are mapped to the center coordinates of the object in the actual image.

[0021] The formula for the target policy that the agent needs to learn, for the target comparison and recognition task, is expressed as follows:

[0022]

[0023]

[0024] in, The policy function is the agent's target policy. The action sampled from the policy function, when executed, allows the agent to approach the target while avoiding collisions with the target and other objects. t It is the action result obtained from the sampling policy function; (x t ,y t () represents the position coordinates of the agent at time t. d represents the coordinates of the obstacle's center. s Within the safe range of the obstacle, The initial distance between the agent and the target.

[0025] Furthermore, the simulation environment includes multiple obstacles, various targets as objects to be identified, and a mobile intelligent agent. The simulation environment must include domain randomization functionality, and the expression for domain randomization is:

[0026]

[0027] Among them, f i For each feature domain of the randomization, the parameter values ​​of the environmental factors represented by each feature domain are i = 1, 2, 3, ..., n. The domain randomization method in the training process is similar to the data augmentation method in deep learning. By randomizing the target and environmental factors, combined with the agent's observation space design, the agent can learn an image comparison and recognition ability after comparing the target image and the environmental image.

[0028] Furthermore, for the parameter values ​​of each environmental factor domain, a reservoir sampling method is used for sampling; the parameter domain values ​​of each dimension are used as the data stream of the reservoir sampling, and a set of parameter values ​​of each dimension are randomly sampled. Each set of parameter values ​​contains k values, which are used as the training environment parameter values ​​for the subsequent k training rounds.

[0029] Furthermore, learning metrics are set for each stage in conjunction with the policy function. Once the agent achieves the learning metrics for each stage, it proceeds to the next stage. The formula for the learning metrics is:

[0030]

[0031] Where loss(i) is the loss at time i, l th It is the set loss threshold; G tar (i) indicates whether the task is completed in round i, and its expression is:

[0032]

[0033] When the average loss over the last 100 training rounds is less than a preset threshold and the task success rate over the last 100 training rounds is greater than or equal to 0.95, it indicates that this stage of learning is complete and the next stage of training begins. After entering the next stage of training, new domains are added to the mentioned domain randomization method until all preset domains in the simulation environment have been added.

[0034] Furthermore, the reward function based on the potential field method is:

[0035]

[0036] Where r represents the action evaluation function, r d Defined as a task completion evaluation function based on the potential field principle, its formula is as follows:

[0037]

[0038] Where σ is the completion coefficient, which is a constant; r d The square of the nearest distance to the target in a training round The closer to the target, the higher the completion rate of the surface task. However, if the task is not completed, a certain amount of evaluation correction is required. The dataset used for training the agent is obtained by the agent through interaction with the environment in the constructed simulation environment.

[0039] The present invention also provides an image comparison and recognition system based on reinforcement learning. The system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the image comparison and recognition method based on reinforcement learning as described above.

[0040] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the reinforcement learning-based image comparison and recognition method as described above.

[0041] In summary, compared with the prior art, the image comparison and recognition method and device based on reinforcement learning provided by the present invention have the following advantages:

[0042] 1. This invention uses a domain randomization method in a simulation environment to randomly reset conditions such as agent pose, target pose, and interference in the environment in the scene, generating a large amount of discrete learning experience, which enhances the training data of the agent. After comparing the given target image with the environment image, the agent outputs the position of the target in the environment, and can learn the ability to identify targets by comparing images, thus improving the generalization ability of target recognition.

[0043] 2. This invention introduces the concept of course learning into the learning process. By setting course learning evaluation indicators, the learning difficulty of the agent is controlled, so that the agent's learning strategy has a strong adaptability to changes in objective environmental factors (such as ambient lighting, environmental background texture, etc.), thus ensuring the stability of the method.

[0044] 3. In the original entropy-based reinforcement learning, this invention introduces entropy regularization sparsity optimized by the butterfly algorithm. The value of the action evaluation function is used as the optimization index. The policy entropy of the agent is adjusted in real time during the training process, which ensures that the agent can generate successful experience during training and guarantees the success rate of the algorithm recognition. Attached Figure Description

[0045] Figure 1 This is a flowchart of an image comparison and recognition method based on reinforcement learning provided by the present invention;

[0046] Figure 2 This is a training diagram of an image comparison and recognition method based on reinforcement learning provided by the present invention;

[0047] Figure 3 This is a schematic diagram of an environmental observation image. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0049] Please see Figure 1 and Figure 2 This invention provides an image comparison and recognition method based on reinforcement learning, which mainly includes the following steps:

[0050] Step 1: Based on the reinforcement learning model, the target recognition task is modeled using Markov decision-making to design the target recognition process as an MDP problem.

[0051] Based on the reinforcement learning model, target recognition is modeled using Markov decision-making, and the target recognition process is designed as an MDP problem. The MDP problem mainly designs basic elements such as state space, observation space, action space, and objective function. Therefore, the task is designed according to the requirements of target recognition.

[0052] Among them, the state space is all the characteristics of an intelligent agent in the environment, such as position, velocity, sensor information, etc., and is generally represented by S.

[0053] Observation Space: Due to the limitations of the agent's sensors, the environment it perceives differs from the actual environment it is in. Therefore, the description of the perceived environment by the agent is defined as the agent's observation space, denoted by O. In target comparison and recognition modeling, O is represented as:

[0054] O={P env ,P tar}

[0055] Among them, P env and P tar These represent the environment image and the target image, respectively.

[0056] Action space: The set of actions that an agent can take is called the agent's action space, denoted by A. Specifically, the formula for A is as follows:

[0057] A={(x,y)|(X tar ,Y tar = (W·x, H·y), x, y ∈ (0, 1)}

[0058] Here, (x, y) are the direct outputs of the neural network, consisting of two numbers ranging from (0, 1), which are mapped from the width and height (W, H) of the original image to the center coordinates of the object in the actual image. When comparing and recognizing the target, the defined action space is the pixel center of the target in the environmental image, i.e., the target recognition result.

[0059] Policy function: A formula representing the target policy that the agent needs to learn. For target comparison and recognition tasks, the policy function is expressed as:

[0060]

[0061]

[0062] in, The policy function is the agent's target policy. The action sampled from the policy function, when executed, allows the agent to approach the target while avoiding collisions with the target and other objects. t It is the action result obtained from the sampling policy function; (x t ,y t () represents the position coordinates of the agent at time t. d represents the coordinates of the obstacle's center. s Within the safe range of the obstacle, The initial distance between the agent and the target.

[0063] Step 2: Build a simulation environment, design a domain randomization method that integrates the reservoir sampling method, and use the error information during the training process to determine whether to apply domain randomization to adjust the training environment. At the same time, use the reservoir sampling method to sample and obtain multiple feature values ​​of the environmental feature domain after randomization.

[0064] Based on the aforementioned MDP model, a simulation environment that meets the model's requirements is constructed. This environment includes numerous obstacles, various targets as objects to be identified, and a mobile intelligent agent. Furthermore, the simulation environment must include domain randomization functionality, the expression for which is:

[0065]

[0066] Among them, f i (i = 1, 2, 3, ..., n) represents the feature domains of the domain randomization. For the parameter values ​​of environmental factors represented by each feature domain (such as ambient light intensity, scene texture, object color, target type, etc.), the domain randomization method in the training process is similar to the data augmentation method in deep learning. By randomizing the target and environmental factors, combined with the intelligent agent observation space design described in step one, the intelligent agent learns an image comparison and recognition ability after comparing the target image and the environmental image.

[0067] To improve sampling efficiency, this implementation uses a reservoir sampling method for the parameter values ​​of each environmental factor domain. The reservoir sampling method can be used to extract data from a given, very long, or unknown data stream at a certain proportion, ensuring that each data point has an equal probability of being selected. The parameter domain values ​​of each dimension are used as the data stream for reservoir sampling. A set of parameter values ​​for each dimension is randomly sampled, with each set containing k values, which are then used as the training environment parameter values ​​for the subsequent k training iterations.

[0068] Step 3: Use a learning evaluation mechanism designed with a course learning model to gradually increase environmental variables and task difficulty.

[0069] The learning metric for each stage is set using the policy function. Once the agent achieves the learning metric for each stage, it moves on to the next stage. The formula for the learning metric is as follows:

[0070]

[0071] Where loss(i) is the loss at time i, l th It is the set loss threshold; G tar (i) indicates whether the task is completed in round i, and its expression is:

[0072]

[0073] When the average loss over the last 100 training rounds is less than a preset threshold and the task success rate over the last 100 training rounds is greater than or equal to 0.95, this stage of learning is considered complete, and the next stage of training begins. It's important to note that the task success rate is not set to 100% to prevent the agent's target recognition network from overfitting during the previous learning stage, which could negatively impact subsequent learning. After entering the next stage of training, new domains are added to the aforementioned domain randomization method until all preset domains in the simulation environment have been added.

[0074] Step four: Use a reward function based on the potential field concept to evaluate the policy generated by the agent and optimize the policy during the training process.

[0075] The reward function based on the potential field method is designed as follows:

[0076]

[0077] Where r represents the action evaluation function, r d Defined as a task completion evaluation function based on the potential field principle, its formula is as follows:

[0078]

[0079] Where σ is the completion coefficient, which is a constant; r d The square of the nearest distance to the target in a training round Inversely proportional, the closer to the target, the higher the apparent task completion rate; however, if the task is not completed, a certain amount of evaluation and correction is required.

[0080] Step 5: Design an experience replay mechanism based on the training requirements of reinforcement learning, and set up an experience cache. The experience cache is used to store the agent's observation space, action space, reward value, state space and policy function during the training process.

[0081] Based on the training requirements of reinforcement learning, an experience caching mechanism is set up. The dataset used for training the agent is not pre-created and labeled by humans, but is obtained by the agent through interaction with the environment in the simulation environment built in step two. The expression of a dataset (or an experience) learned at time t is as follows:

[0082] exp t =(P env,t ,P tar,t ,a t ,P env,t+1 ,r t ,l t )

[0083] Among them, P env,t Let a be the environmental image observed by the agent at time t. t In the observation state (P) env,t ,P tar,t The actions P is output by a neural network and executed by an agent. env,t+1 r represents the observation result after the agent performs an action. t To evaluate the value of the neural network output action by the evaluation function, l t This represents the loss value of the neural network at that moment.

[0084] Step six: Initialize the experience buffer using a warm-up mechanism, use random sampling of actions to enable the agent to interact with the environment to obtain a large amount of initial experience, and use a network loss experience replay mechanism to filter the experience in the experience pool for learning.

[0085] Since the experience cache is created at the initial stage of training and there is no prior experience, a warm-up mechanism is used. Random action sampling allows the agent to move randomly in the environment, generating a certain amount of initial experience for neural network training. Simultaneously, a priority experience replay mechanism is designed. When sampling experience, based on previous random sampling, the replay is performed according to the l-value of each experience. t Sampling weights are generated based on experience, and sampling is performed according to the weight values.

[0086] The sumtree structure is used to store the weights of each experience. The weights of each experience are used as child nodes of the sumtree structure, and parent nodes are generated layer by layer. During sampling, the data to be sampled is randomly generated. Through the layer-by-layer sampling of the sumtree, the experience with the larger the weight has a greater probability of being sampled. In this way, the experience with the larger loss value is selected for network learning.

[0087] Step 7: Based on the internal exploratory desire mechanism of the reinforcement learning SAC algorithm, the butterfly optimization algorithm, and the stage learning evaluation mechanism, the exploratory desire coefficient of the agent in the learning process is dynamically adjusted to achieve image comparison and recognition.

[0088] The optimal policy function π of the SAC algorithm * The formula is as follows:

[0089]

[0090] Wherein, π(·|P t H(π(·|P)) is the policy function fitted by the neural network, which is a probability distribution of actions that the agent can perform. t ) represents the entropy value of the probability distribution; γ represents the discount coefficient; α represents the entropy regularization coefficient. α can be used to control the ratio between entropy and the reward function. The larger α is, the more important the policy entropy in the policy function becomes. Since the agent's policy entropy is essentially the agent's exploration desire, α can be used to control the agent's exploration desire when searching the solution space, enabling it to search for the correct solution in a huge solution space.

[0091] Preferably, the reinforcement network model uses the SAC algorithm as the basic algorithm, such as Figure 3 As shown, this includes two Value networks, two Critic networks, and one Actor network. The Actor network incorporates an attention mechanism to accelerate policy learning and improve network performance. The loss functions for each network are as follows, starting with the loss function for the Value network:

[0092]

[0093]

[0094] Among them, v i This is the output value of the Value network, representing the Value network's estimate of the environmental state. i These are the network parameters; The target value updated by the Value network is obtained based on the action evaluation function in step four and the temporal difference principle. γ is the discount factor in the temporal difference principle, and P... i B is the environmental state of the agent at time i; B is the number of experiences randomly drawn from the experience pool during training.

[0095] The loss function of the Critic network is as follows:

[0096]

[0097]

[0098] Wherein, π(a′) t |s t ;θ) represents the policy function fitted by the Actor network, which is essentially a probability distribution of a set of actions, a′ t Using π(a′) t |s t The action function obtained by sampling θ) and lnπ(a′ t |s t ;θ) represents the policy entropy of the policy function, which is H(π(·|P) in the optimal policy function. t Mathematically, α is optimized in real time according to the optimization algorithm in step six. By evaluating the agent's exploration and utilization through two networks, Critic and Value, the agent can improve the task success rate while acquiring a large amount of useful data.

[0099] The loss function of the Actor network is as follows:

[0100]

[0101] Among them, a t This refers to the actual actions output by the Actor network, which updates itself based on the evaluation given by the Critic network. The relationships between the three networks are shown in the appendix. Figure 3 As shown.

[0102] Preferably, to better regulate the relationship between "exploration" and "exploitation" in reinforcement learning, a butterfly optimization algorithm is designed to adaptively adjust α during the training process. The butterfly optimization algorithm (BOA) is a natural heuristic algorithm proposed by Arora based on the foraging process of butterflies (BOA). The formula is as follows:

[0103]

[0104] f i =c·r t β

[0105] Among them, f i The fragrance factor given to the i-th butterfly is related to the perception factor c and the action evaluation value r. t It is related to the parameter β; rand represents a random number in the interval (0,1), α *This represents the optimal solution so far. Butterflies use their sensory organs to locate food sources. In this algorithm, it is assumed that each butterfly produces a scent of a certain intensity, which spreads and is perceived by other butterflies in the area. The scent released by each butterfly is related to its fitness. This means that when a butterfly moves, its fitness will also change. Utilizing the global and local search characteristics of the butterfly algorithm, it can continuously search for the optimal α value during agent training.

[0106] During training, since the learning objective of reinforcement learning is to obtain the maximum environmental reward, and the action evaluation function in step four is set based on the distance to the target, all the agent's strategies are aimed at getting closer to the target. For the agent, target information is input to the neural network simultaneously as an image and an environmental image. Therefore, the neural network guides the agent to the target by outputting the target position in the environmental image. The neural network takes two images as input and outputs the pixel center coordinates of the object in the second image within the first image. Subsequently, the agent calculates the target's orientation relative to itself based on the pixel coordinates output by the network and the camera intrinsic parameter matrix, and finally moves to the target according to the calculated orientation. In this embodiment, the agent's actions are only used to verify the correctness of the neural network's output, and the trained network can be directly deployed to other application scenarios.

[0107] The present invention also provides an image comparison and recognition system based on reinforcement learning. The system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the image comparison and recognition method based on reinforcement learning as described above.

[0108] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the reinforcement learning-based image comparison and recognition method as described above.

[0109] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for image comparison recognition based on reinforcement learning, characterized in that, The method includes the following steps: (1) Based on the reinforcement learning model, the target recognition task is modeled by Markov decision to design the target recognition process as an MDP problem; (2) Build a simulation environment, design a domain randomization method that integrates the reservoir sampling method, and use the error information in the training process to determine whether to apply domain randomization to adjust the training environment. At the same time, use the reservoir sampling method to sample and obtain multiple feature values ​​after the environmental feature domain randomization. (3) Adopt a learning evaluation mechanism designed using a curriculum learning model to gradually increase environmental variables and task difficulty; (4) The reward function based on the potential field idea is used to evaluate the policy generated by the agent and optimize the policy during the training process; (5) Design an experience replay mechanism based on the training requirements of reinforcement learning, and set up an experience cache area, which is used to store the agent's observation space, action space, reward value, state space and policy function during the training process; (6) The warm-up mechanism is used to initialize the experience buffer, and the random sampling method of actions is used to enable the agent to interact with the environment to obtain a large amount of initial experience. At the same time, the network loss experience replay mechanism is used to filter the experience in the experience pool for learning. (7) Based on the internal exploratory desire mechanism of the reinforcement learning SAC algorithm, the butterfly optimization algorithm and the stage learning evaluation mechanism, the exploratory desire coefficient of the agent in the learning process is dynamically adjusted, thereby realizing image comparison and recognition; The state space is the objective representation of all the characteristics of an intelligent agent in its environment; The description of the environment perceived by the agent is defined as the observation space of the agent, denoted as In target alignment identification modeling, is denoted as: wherein, and respectively represent an environmental image and a target image; The set of actions that an agent can take is the action space of the agent, denoted by A, where A = {a1, a2,..., an} and n is the number of actions available to the agent. where, In the formula, It is the direct output of the neural network, consisting of two numbers ranging from (0,1); through the width and height of the original image ( W , H This is mapped to the center coordinates of the object in the actual image; The formula for the target policy that the agent needs to learn, for the target comparison and recognition task, is expressed as follows: in, The policy function is the agent's target policy. The action sampled by the policy function is the action that allows the agent to approach the target while avoiding collisions with the target and other objects. It is the action result obtained from the sampling strategy function; , )express t The position coordinates of the agent at any given time, , () represents the coordinates of the obstacle's center. Within the safe range of the obstacle, The initial distance between the agent and the target.

2. The image comparison and recognition method based on reinforcement learning as described in claim 1, characterized in that: The MDP problem involves designing the state space, observation space, action space, and objective function.

3. The image comparison and recognition method based on reinforcement learning as described in claim 2, characterized in that: The simulation environment includes multiple obstacles, various targets as objects to be identified, and a mobile intelligent agent. The simulation environment must include domain randomization functionality, and the expression for domain randomization is: in, For each feature domain of the randomized domain, the parameter values ​​of the environmental factors represented by each feature domain are... The domain randomization method during training is similar to the data augmentation method in deep learning. By randomizing the target and environmental factors and combining them with the agent's observation space design, the agent learns an image comparison and recognition ability after comparing the target image with the environmental image.

4. The image comparison and recognition method based on reinforcement learning as described in claim 1, characterized in that: For the parameter values ​​of each environmental factor domain, a reservoir sampling method is used for sampling; the parameter domain values ​​of each dimension are used as the data stream for reservoir sampling, and a set of parameter values ​​for each dimension are randomly sampled. Each set of parameter values ​​contains... k These values ​​will be used for subsequent training. k The training environment parameter values ​​for each round.

5. The image comparison and recognition method based on reinforcement learning as described in claim 2, characterized in that: The learning metric for each stage is set using the policy function. Once the agent achieves the learning metric for each stage, it moves on to the next stage. The formula for the learning metric is: in, For the first i The loss of time, It is the set loss threshold; Indicates the first i The expression for whether a round has completed the task is: ； When the average loss over the last 100 training rounds is less than a preset threshold and the task success rate over the last 100 training rounds is greater than or equal to 0.95, it indicates that this stage of learning is complete and the next stage of training begins. After entering the next stage of training, new domains are added to the mentioned domain randomization method until all preset domains in the simulation environment have been added.

6. The image comparison and recognition method based on reinforcement learning as described in claim 5, characterized in that: The reward function based on the potential field method is: in, This represents the action evaluation function. Defined as a task completion evaluation function based on the potential field principle, its formula is as follows: in, σ The completion coefficient is a constant. The square of the nearest distance to the target in a training round The closer to the target, the higher the completion rate of the surface task. However, if the task is not completed, an evaluation correction amount needs to be given. The dataset used for training the agent is obtained by the agent through interaction with the environment in the constructed simulation environment.

7. An image comparison and recognition system based on reinforcement learning, characterized in that: The system includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it performs the image comparison and recognition method based on reinforcement learning as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the reinforcement learning-based image comparison and recognition method according to any one of claims 1-6.

Images