Intelligent decision-making method and device for motion of lunar miniature robot

Abstract

The invention provides an intelligent decision-making method and device for motion of a lunar miniature robot, and relates to the reinforcement learning technology, and the method comprises the steps: obtaining a current image shot by the lunar miniature robot and a target image meeting the shooting requirement; inputting the current image and the target image into a trained intelligent decision model, and outputting an action decision for enabling the robot to move to a position where an image most similar to the target image is shot; wherein the intelligent decision-making model is obtained by training a preset action decision-making network through a training sample set, the training sample set comprises a training target image and a training initial image, and the training target image is obtained by shooting a to-be-shot target at any first position by the robot; the training initial image is shot by the robot at any second position in a circular area with the first position as the circle center. The scheme can be used in a satellite-borne computing environment in which computing resources are seriously limited, so that the lunar miniature robot can automatically make decisions.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to an intelligent decision-making method and device for the movement of a tiny lunar robot. Background Art

[0002] When using a tiny lunar robot to take photos of a cooperative target in an extraterrestrial exploration scenario, in order to take a photo of the cooperative target in the middle area of ​​the image with a suitable composition in a short time, it is necessary to adjust the robot's position and posture based on visual information, and then use the camera fixed on the robot to take the photo.

[0003] In the related technology, the traditional visual servoing method requires manual feature selection and huge amount of calculation, which makes it difficult to adapt to the satellite computing environment where computing resources are severely limited, and thus causes the tiny robots on the lunar surface to be unable to automatically adjust their shooting position and posture according to the image information.

[0004] Based on this, there is an urgent need for an intelligent decision-making method and device for the movement of lunar micro-robots to solve the above-mentioned technical problems. Summary of the invention

[0005] The embodiment of the present invention provides an intelligent decision-making method and device for the movement of a lunar micro-robot, which can automatically make movement decisions based on visual image information.

[0006] In a first aspect, an embodiment of the present invention provides an intelligent decision-making method for the movement of a lunar micro-robot, comprising:

[0007] Obtain the current image taken by the lunar microrobot and the target image that meets the shooting requirements;

[0008] The current image and the target image are input into a trained intelligent decision-making model, and an action decision is outputted to move the robot to a position where an image most similar to the target image is taken; wherein the intelligent decision-making model is obtained by training a preset action decision network with a training sample set, and the training sample set includes a training target image and a training initial image, wherein the training target image is obtained by the robot photographing the target at any first position, and the training initial image is obtained by the robot photographing any second position within a circular area with the first position as the center.

[0009] In a second aspect, an embodiment of the present invention further provides an intelligent decision-making device for the movement of a lunar micro-robot, comprising:

[0010] An acquisition module is used to acquire the current image taken by the lunar micro-robot and the target image that meets the shooting requirements;

[0011] A decision module is used to input the current image and the target image into a trained intelligent decision model, and output an action decision to move the robot to a position to shoot an image most similar to the target image; wherein the intelligent decision model is obtained by training a preset action decision network with a training sample set, and the training sample set includes a training target image and an initial training image, the training target image is obtained by the robot shooting a target at any first position, and the initial training image is obtained by the robot shooting at any second position within a circular area with the first position as the center.

[0012] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0013] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0014] The embodiment of the present invention provides an intelligent decision-making method and device for the movement of a tiny robot on the lunar surface. The method uses a deep reinforcement learning method to construct an intelligent decision-making network model. The constructed neural network is trained based on a training set composed of visual image features, and finally the trained neural network can autonomously plan the movement of the robot in an uncertain environment. The method does not require accurate modeling of the environment and can be used in unknown extraterrestrial environments. It has a small amount of calculation and can be used in a satellite computing environment with severely limited computing resources, so that the tiny robot on the lunar surface can make decisions automatically without manual control. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 This is a flow chart of an intelligent decision-making method for the movement of a lunar micro-robot provided by an embodiment of the present invention;

[0017] Figure 2 It is a schematic diagram of a network structure built by a multi-layer perceptron provided in one embodiment of the present invention;

[0018] Figure 3 is a hardware architecture diagram of an electronic device provided by an embodiment of the present invention;

[0019] Figure 4 It is a structural diagram of an intelligent decision-making device for the movement of a lunar micro-robot provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0021] As mentioned earlier, in order to ensure that the shooting effect of the tiny robot on the lunar surface meets the mission requirements, it is necessary to continuously adjust the robot's shooting position and posture according to the visual information of the image. The traditional visual servo method requires manual feature selection and has a huge amount of calculation, making it difficult to use in extraterrestrial exploration environments.

[0022] Based on this, the idea of ​​the present invention is to adopt the method of deep reinforcement learning to construct an intelligent decision-making network model, and carry out learning training on the constructed neural network so that the trained neural network can autonomously adapt to the unknown surface environment and autonomously plan the robot's movement movements in an uncertain environment.

[0023] The specific implementation of the above concept is described below.

[0024] Please refer to Figure 1 The embodiment of the present invention provides an intelligent decision-making method for the movement of a lunar micro-robot, the method comprising:

[0025] Step 100, obtaining the current image taken by the lunar surface micro-robot and the target image that meets the shooting requirements;

[0026] Step 102, input the current image and the target image into a trained intelligent decision model, and output an action decision that causes the robot to move to a position to capture an image that is most similar to the target image; wherein the intelligent decision model is obtained by training a preset action decision network with a training sample set, and the training sample set includes a training target image and a training initial image, the training target image is obtained by the robot capturing the target at any first position, and the training initial image is obtained by the robot capturing the target at any second position within a circular area with the first position as the center.

[0027] In the embodiment of the present invention, a deep reinforcement learning method is used to construct an intelligent decision-making network model, and the constructed neural network is trained according to a training set composed of visual image features, so that the trained neural network can autonomously plan the robot's movement in an uncertain environment. This method does not require accurate modeling of the environment and can be used in unknown extraterrestrial environments. It has a small amount of calculation and can be used in satellite computing environments with severely limited computing resources, so that the lunar microrobot can make decisions automatically without manual control.

[0028] Described below Figure 1 How the various steps are performed.

[0029] First, with respect to step 100, the current image taken by the lunar micro-robot and the target image that meets the shooting requirements are obtained.

[0030] The images in the embodiments of the present invention are all taken by a camera installed on the lunar micro-robot. The camera can take RGB images, thereby facilitating subsequent intelligent decision-making based on the visual information in the image.

[0031] Specifically, at any position in the robot's workspace, use the camera installed on the robot to take an image of the landing assembly within the camera's field of view, and use it as the target image. The landing assembly includes three typical components: the lander, the ascender, and the national flag. The current image is any image taken by the lunar microrobot in the workspace.

[0032] Then, for step 102, the current image and the target image are input into a trained intelligent decision model, and an action decision is outputted to move the robot to a position where an image most similar to the target image is captured.

[0033] In an embodiment of the present invention, the intelligent decision-making model is obtained by training a preset action decision network through a training sample set, wherein the training sample set includes a training target image and a training initial image, wherein the training target image is obtained by the robot photographing the target at any first position, and the training initial image is obtained by the robot photographing at any second position within a circular area with the first position as the center.

[0034] Specifically, the images in the training sample set were all taken by the lunar micro-robot in a simulated scene. The simulated scene is composed of the lunar micro-robot, the landing assembly and the lunar surface. A camera is installed on the lunar micro-robot, and the landing assembly includes an ascender, a lander, a sailboard, a national flag and a robotic arm. Installation parameters include: the camera's field of view is 86°×57°, and the size of the image taken by the camera is 320×200; the camera is fixed in front of the robot, and the height from the bottom of the robot is 10 cm.

[0035] Furthermore, a camera is used to capture a training target image in the constructed simulation scene, that is, at any first position in the robot's workspace, a camera installed on the robot is used to capture an image of the assembly within the camera's field of view, which is used as the target image.

[0036] The working space in this embodiment is set as: a circular area with a radius of 5m to 30m and the body coordinate origin of the upper assembly as the center; in addition, in order to ensure the training effect of the subsequent intelligent decision-making model, this embodiment ensures that the upper assembly is within the camera field of view in the following way: the line connecting the body coordinate origin of the upper assembly and the body coordinate origin of the robot is 0°, and the robot's posture change range is set to -25° to 25°, and random values ​​are taken within this range.

[0037] In other words, the angle between the camera direction of the lunar microrobot and the connecting line ranges from -25° to 25°. This ensures image diversity while also ensuring that the training target image always contains the target to be photographed, thereby avoiding invalid training.

[0038] Furthermore, the initial training image is obtained by shooting on the basis of the target training image. The shooting position of the initial training image is the position where the target image is shot as the target position. The maximum distance d between the initial position and the target position of the robot is set. A circular area is formed with the target position as the center and the maximum distance d as the radius. The robot is moved to any second position of the circular area. This position is used as the initial position of the robot. The image is shot using a camera installed on the robot and used as the initial training image. For example, with the target position as the origin, the distance between the initial position and the target position is ±(1-3) m, and the range of the rotation angle r is (0-360°), that is, the image used as the initial training image can be shot from any angle, even if it does not contain the target to be shot, it can also be used as the initial training image.

[0039] In this embodiment, the process of training to obtain an intelligent decision-making model includes the following steps: setting up a reinforcement learning environment for training the intelligent decision-making model; the reinforcement learning environment includes a state space, an action space, a reward function and a termination condition; building the action decision network and the value network based on a multi-layer perceptron model; using a reinforcement learning algorithm to train the action decision network and the value network with a training sample set as input to obtain the intelligent decision-making model.

[0040] Specifically, the first image feature of the training target image and the second image feature of the training initial image are combined to establish the state space, which is 18-dimensional and includes image features of three typical components, namely, the lander, the ascender, and the national flag, in the training target image and the training initial image.

[0041] Image features are obtained by detecting two types of images. The detection results include the coordinates of the upper left vertex of the rectangular detection box of the typical component in the image pixel coordinate system, as well as the width and height of the rectangular detection box. Then, based on the detection results, the pixel coordinates of the center point of the rectangular detection box are calculated:

[0042]

[0043] Among them, (x in ,y in ) is the pixel coordinate of the center point of the nth rectangular detection box in the ith image; (x lin ,y lin ) is the pixel coordinate of the upper left vertex of the nth rectangular detection box; w in is the width of the rectangular detection frame; h in is the height of the rectangular detection box;

[0044] According to the detection results, the total number of pixels in the rectangular detection frame is calculated:

[0045] p in =h in w in

[0046] Among them, p in is the total number of pixels in the nth rectangular detection box;

[0047] According to the center pixel coordinates and the total number of pixels, the current image features are determined as: [x i1 ,y i1 ,p i1 ,…,x in ,y in ,p in ].

[0048] For the combination with only three typical parts, the image feature obtained is [x i1 ,y i1 ,p i1 ,…,x i3 ,y i3 ,p i3 ], and the synthetic feature vector contains the features of an initial training image and a target training image, so the 18-dimensional synthetic feature vector is obtained by reordering the diagonals in the state space: xyp = (x0, y0, p0, x1, y1, p1, …, x5, y5, p5).

[0049] Furthermore, the action space is established according to the movement mode of the lunar micro-robot; wherein the action space includes all one-dimensional discrete actions of the robot and decision values ​​used to characterize the one-dimensional discrete actions. For example, the one-dimensional actions include forward movement, backward movement, left front movement, right front movement, left rear movement and right rear movement, and the decision values ​​are between 0 and 5, which are respectively represented as 0: forward, 1: backward, 2: left front, 3: left rear, 4: right front, 5: right rear.

[0050] Furthermore, the reward function is used to evaluate the reward score of the action output by the action decision network. The reward function includes visibility reward, task completion reward, total step reward and single-step similarity reward, where:

[0051] The visibility reward is used to characterize whether the latest image captured after moving according to the action output by the action decision network contains the target to be photographed.

[0052] For example, the visibility reward value can be:

[0053]

[0054] The method for determining whether the photographed cooperative target is in the image is as follows: the number of pixels p of component i of the photographed cooperative target in the image is calculated. i If the number of pixels of all components is less than 30, it is determined that the latest image does not contain the target to be photographed, otherwise it is determined that the latest image contains the target to be photographed.

[0055] The task completion reward is used to indicate whether the latest image meets the shooting requirements, that is, the maximum number of steps in a set round of training step max Rewards for completing the smart photo-taking task within the time limit;

[0056] For example, the value of the task completion reward can be:

[0057]

[0058] That is to say, if the robot successfully adjusts the photographing position to the same as the photographing position of the training target image and takes a picture of the training target image in one round of training, the robot is considered to have completed the intelligent photographing task, and the task completion reward can be calculated.

[0059] The total step reward is used to represent the running step reward in one round of training;

[0060] For example, the value of the total step reward can be:

[0061]

[0062] Among them, steptotal Indicates the total number of steps the robot takes to complete the smart photo taking task.

[0063] The single-step similarity reward is used to characterize whether the current image taken after completing an action is more similar to the training target image than the previous image;

[0064] For example, the value of the single-step similarity reward can be:

[0065]

[0066] In the embodiment of the present invention, both the action decision network and the value network adopt Figure 2 The multi-layer perceptron (MLP) shown in the figure can be used to build the action decision network. The action decision network can continuously output the action decision value that makes the robot move according to the difference between the initial training image and the training target image, thereby helping the robot to move and take photos similar to the training target image. According to the score of each action step calculated by the reward function, the corresponding reward curve is constructed, and whether the reward curve meets the convergence condition is used as the termination condition to determine whether the training of the intelligent decision-making model is completed.

[0067] After completing the construction of the action decision network and the value network, the reinforcement learning method SAC (soft actor-critic) can be used to train them to obtain an intelligent decision model. The training process includes: inputting the initial training image and the training target image into the action decision network, and outputting the action decision value; adjusting the position and posture of the lunar micro-robot according to the action decision value, and shooting the target to be shot after adjustment to obtain the latest image; using the reward function to calculate the reward score of the action corresponding to the latest image, and judging whether the latest image meets the shooting task requirements; if so, end this round of training, otherwise repeatedly output the action decision value to adjust the robot until the image meets the requirements or the number of training steps reaches the maximum value; establish a reward curve based on all the reward scores, and when the reward curve meets the preset convergence conditions, end all training and output the intelligent decision model.

[0068] For example, at the beginning of the first round of training, the action decision network trains the initial image and the training target image to determine the decision action output in the first step:

[0069] For example, if it is a forward action, after the robot completes the action, it takes a new image at the new position, and then uses the reward function to compare the new image with the training target image, and outputs the reward score for the first step of the action:

[0070] For example, if the target to be photographed is in the image, the smart photography task has not been completed, and the latest image after the first step is completed is more similar to the training target image, substituting it into the above reward function can get a reward score of 5 points for this step.

[0071] And so on, until the number of steps reaches the maximum number of steps. For example, a round of training executes a maximum of 50 steps. If the smart photo-taking task is still not completed after the 50th step, this round of training is completed and the next round of training begins; if the smart photo-taking task is completed at the 20th step, this round of training ends early.

[0072] It is worth noting that in the initial stage of training, the action decision value output by the action decision network is arbitrary, that is, the robot may move in any direction. As the number of training times increases, the action decisions made by the network will gradually tend towards the optimal action. In addition, at the beginning of each round of training, the training target image and the initial training image need to be updated, that is, a new set of images are replaced for training. The upper limit of the number of training steps in each round can be adjusted according to actual needs, and the number of training rounds is determined according to the convergence degree of the reward curve. When the reward curve meets the convergence conditions, all training is terminated to obtain an intelligent decision-making model.

[0073] Finally, the current image taken by the lunar micro-robot and the target image that meets the shooting requirements are input into the intelligent decision-making model. The robot moves in the direction corresponding to the decision action output by the model to realize the autonomous movement of the lunar micro-robot.

[0074] For example, for a straight-line control instruction, the trajectory length is the straight-line length of the robot's movement. For a turning control instruction, the trajectory length is the arc length of the robot's movement. In this embodiment, the trajectory length of the robot's movement is 30 cm. Noise is added to it during training. The actual movement trajectory length of the robot in the simulation environment is (25 to 35) cm.

[0075] like Figure 3 , Figure 4 As shown, the embodiment of the present invention provides an intelligent decision-making device for the movement of a lunar micro-robot. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, Figure 3 As shown, it is a hardware architecture diagram of an electronic device in which an intelligent decision-making device for the movement of a lunar micro-robot provided by an embodiment of the present invention is located. Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown, the electronic device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 4As shown, as a device in a logical sense, the CPU of the electronic device in which it is located reads the corresponding computer program in the non-volatile memory into the memory and runs it. This embodiment provides an intelligent decision-making device for the movement of a small robot on the lunar surface, including:

[0076] An acquisition module 400 is used to acquire the current image taken by the lunar surface micro-robot and a target image that meets the shooting requirements;

[0077] The decision module 402 is used to input the current image and the target image into the trained intelligent decision model, and output an action decision to move the robot to the position of shooting an image most similar to the target image; wherein the intelligent decision model is obtained by training a preset action decision network with a training sample set, and the training sample set includes a training target image and a training initial image, the training target image is obtained by the robot shooting the target at any first position, and the training initial image is obtained by the robot shooting at any second position within a circular area with the first position as the center.

[0078] In an embodiment of the present invention, when the decision module 402 trains the action decision network to obtain an intelligent decision model, it is specifically used to perform the following operations: setting a reinforcement learning environment for intelligent decision model training; the reinforcement learning environment includes a state space, an action space, a reward function and a termination condition; building an action decision network and a value network based on a multi-layer perceptron model; using a reinforcement learning algorithm to train the action decision network and the value network that have input a training sample set to obtain an intelligent decision model.

[0079] In the embodiment of the present invention, the state space is set by: merging the first image feature of the training target image and the second image feature of the training initial image to establish the state space;

[0080] In an embodiment of the present invention, the action space is set in the following manner: the action space is established according to the movement mode of the lunar micro-robot; wherein the action space includes all one-dimensional discrete actions of the robot and decision values ​​used to characterize the one-dimensional discrete actions, and the one-dimensional discrete actions include forward movement, backward movement, left front movement, right front movement, left rear movement and right rear movement.

[0081] In an embodiment of the present invention, the reward function includes a visibility reward, a task completion reward, a total step reward and a single-step similarity reward, wherein: the visibility reward is used to characterize whether the latest image taken after the robot moves according to the action decision value contains the target to be photographed; the task completion reward is used to characterize whether the latest image meets the shooting requirements; the total step reward is used to characterize the running step reward in a round of training; the single-step similarity reward is used to characterize whether the current image taken after completing an action is more similar to the training target image than the previous image.

[0082] In an embodiment of the present invention, whether the latest image contains the target to be photographed is determined by: calculating the number of pixels of the components of the target to be photographed in the latest image; judging whether the calculation result is less than a preset detection threshold, if so, determining that the latest image does not contain the target to be photographed, otherwise determining that the latest image contains the target to be photographed.

[0083] In an embodiment of the present invention, a reinforcement learning algorithm is used to train an action decision network and a value network that have input a training sample set to obtain an intelligent decision model, including: inputting an initial training image and a training target image into an action decision network, and outputting an action decision value; adjusting the position and posture of the lunar micro-robot according to the action decision value, and photographing the target to be photographed after the adjustment to obtain the latest image; using a reward function to calculate a reward score for the action corresponding to the latest image, and judging whether the latest image meets the requirements of the shooting task; if so, ending the round of training, otherwise repeatedly outputting the action decision value to adjust the robot until the image meets the requirements or the number of training steps reaches a maximum value; establishing a reward curve according to all reward scores, and when the reward curve meets the preset convergence conditions, ending all training and outputting the intelligent decision model.

[0084] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the intelligent decision-making device for the movement of a small robot on the lunar surface. In other embodiments of the present invention, the intelligent decision-making device for the movement of a small robot on the lunar surface may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0085] The information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For the specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0086] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, an intelligent decision-making method for the movement of a lunar micro-robot in any embodiment of the present invention is implemented.

[0087] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes an intelligent decision-making method for the movement of a lunar micro-robot in any embodiment of the present invention.

[0088] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.

[0089] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0090] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0091] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0092] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0093] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical factors in the process, method, article or device including the elements.

[0094] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent decision-making method for the movement of a lunar micro-robot, characterized in that: include: Obtain the current image taken by the lunar microrobot and the target image that meets the shooting requirements; The current image and the target image are input into a trained intelligent decision-making model, and an action decision is outputted to move the robot to a position where an image most similar to the target image is taken; wherein the intelligent decision-making model is obtained by training a preset action decision network with a training sample set, and the training sample set includes a training target image and a training initial image, wherein the training target image is obtained by the robot photographing the target at any first position, and the training initial image is obtained by the robot photographing any second position within a circular area with the first position as the center.

2. The method according to claim 1, characterized in that The intelligent decision-making model is trained in the following way: Setting a reinforcement learning environment for intelligent decision-making model training; the reinforcement learning environment includes a state space, an action space, a reward function and a termination condition; Building the action decision network and value network based on a multi-layer perceptron model; The action decision network and the value network input with the training sample set are trained by using a reinforcement learning algorithm to obtain the intelligent decision model.

3. The method according to claim 2, characterized in that The state space is set up as follows: The state space is established by combining the first image feature of the training target image and the second image feature of the training initial image.

4. The method according to claim 2, characterized in that: The action space is set up in the following manner: the action space is established according to the movement mode of the lunar micro-robot; wherein the action space includes all one-dimensional discrete actions of the robot and action decision values ​​used to characterize the one-dimensional discrete actions, and the one-dimensional discrete actions include forward movement, backward movement, left front movement, right front movement, left rear movement and right rear movement.

5. The method according to claim 2, characterized in that: The reward function includes visibility reward, task completion reward, total step reward and single-step similarity reward, where: The visibility reward is used to indicate whether the latest image captured by the robot after it moves according to the action decision value contains the target to be captured; The task completion reward is used to indicate whether the latest image meets the shooting requirements; The total step reward is used to represent the running step reward in one round of training; The single-step similarity reward is used to represent whether the current image taken after completing an action is more similar to the training target image than the previous image.

6. The method according to claim 5, characterized in that Whether the latest image contains the target to be photographed is determined by: Calculate the number of pixels of the component of the target to be photographed in the latest image; It is determined whether the calculation result is less than a preset detection threshold, if so, it is determined that the latest image does not contain the target to be photographed, otherwise it is determined that the latest image contains the target to be photographed.

7. The method according to claim 2, characterized in that The method of using a reinforcement learning algorithm to train the action decision network and the value network input with the training sample set to obtain the intelligent decision model includes: Inputting the combined feature vector of the training initial image and the training target image into the action decision network, and outputting an action decision value; Adjust the position and posture of the lunar surface micro-robot according to the action decision value, and shoot the target to be shot after the adjustment to obtain the latest image; The reward function is used to calculate the reward score of the action corresponding to the latest image, and it is determined whether the latest image meets the shooting task requirements; if so, a round of training is terminated, otherwise the action decision value is repeatedly output to adjust the robot until the image meets the requirements or the number of training steps reaches the maximum value; A reward curve is established according to all the reward scores. When the reward curve meets a preset convergence condition, all training is terminated and the intelligent decision-making model is output.

8. An intelligent decision-making device for the movement of a lunar micro-robot, characterized in that: include: An acquisition module is used to acquire the current image taken by the lunar micro-robot and the target image that meets the shooting requirements; A decision module is used to input the current image and the target image into a trained intelligent decision model, and output an action decision to move the robot to a position to shoot an image most similar to the target image; wherein the intelligent decision model is obtained by training a preset action decision network with a training sample set, and the training sample set includes a training target image and an initial training image, the training target image is obtained by the robot shooting a target at any first position, and the initial training image is obtained by the robot shooting at any second position within a circular area with the first position as the center.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images