Robot motion control method and device, electronic equipment and readable storage medium

Abstract

The invention provides a robot motion control method and device, electronic equipment and a readable storage medium, and relates to the technical field of robots. The method comprises the steps of obtaining a first environment image currently collected by a target robot and a current first joint position state of the target robot; performing environment feature extraction on the first environment image by using a target feature extraction network based on a channel attention mechanism to obtain a first environment feature; the first environment features and the first joint position states are input into a prediction network, initial motion control signals are obtained, and the prediction network is obtained through training according to the sample environment features, the sample joint position states and the sample motion control signals corresponding to the moments; and controlling the target robot to move according to the initial motion control signal. Thus, the important features in the visual input are accurately extracted, and then the signal for performing motion control on the robot can be generated in combination with the joint position state, so that the situation that the motion of the robot is not matched with the actual environment is reduced.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a robot motion control method, device, electronic device and readable storage medium. Background Art

[0002] At present, the control of robots is generally achieved through program instructions for indicating fixed actions. However, since the robot's actions are fixed, this may cause the robot's actions to not match the actual environment, resulting in poor robot movement effects. Summary of the invention

[0003] The embodiments of the present application provide a robot motion control method, device, electronic device and readable storage medium, which can accurately extract important features in visual input, and then generate signals for motion control of the robot in combination with joint position states, thereby reducing the situation where the robot's movements do not match the actual environment.

[0004] The embodiments of the present application can be implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a robot motion control method, the method comprising:

[0006] Obtaining a first environment image currently collected by the target robot and a current first joint position state of the target robot;

[0007] Using a pre-trained target feature extraction network based on a channel attention mechanism to extract environmental features from the first environmental image, to obtain a first environmental feature;

[0008] Inputting the first environmental feature and the first joint position state into a prediction network to obtain an initial motion control signal, wherein the prediction network is trained according to sample environmental features, sample joint position states, and sample motion control signals corresponding to each of multiple moments, the sample environmental features and joint position states are used as samples, and the sample motion control signals are used as labels corresponding to the samples;

[0009] The target robot is controlled to move according to the initial motion control signal.

[0010] In a second aspect, an embodiment of the present application provides a robot motion control device, the device comprising:

[0011] An acquisition module, used to obtain a first environment image currently acquired by the target robot and a current first joint position state of the target robot;

[0012] A processing module, configured to extract environmental features from the first environmental image using a pre-trained target feature extraction network based on a channel attention mechanism to obtain a first environmental feature;

[0013] The processing module is further used to input the first environmental feature and the first joint position state into a prediction network to obtain an initial motion control signal, wherein the prediction network is trained according to sample environmental features, sample joint position states and sample motion control signals corresponding to each of multiple moments, the sample environmental features and joint position states are used as samples, and the sample motion control signals are used as labels corresponding to the samples;

[0014] A control module is used to control the movement of the target robot according to the initial movement control signal.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement the robot motion control method described in the aforementioned embodiment.

[0016] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the robot motion control method as described in the aforementioned embodiment is implemented.

[0017] The robot motion control method, device, electronic device and readable storage medium provided by the embodiment of the present application, first, obtain the first environment image currently obtained by the target robot and the current first joint position state of the target robot; then, use the target feature extraction network based on the pre-trained channel attention mechanism to extract the environment features of the first environment image to obtain the first environment feature; then, input the above-mentioned first environment feature and the first joint position state into the prediction network to obtain the initial motion control signal, the above-mentioned prediction network is trained according to the sample environment features, sample joint position states and sample motion control signals corresponding to each other at multiple moments, the sample environment features and joint position states are used as samples, and the sample motion control signals are used as labels corresponding to the samples; finally, the target robot is controlled to move according to the initial motion control signal. In this way, the target feature extraction network can be used to accurately extract important features in the visual input, and then the signal for motion control of the robot is generated through the prediction network in combination with the joint position state, thereby reducing the situation where the robot's action does not match the actual environment, and it is convenient to drive the robot to perform precise motion tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A block diagram of an electronic device provided in an embodiment of the present application;

[0020] Figure 2 One of the flow charts of the robot motion control method provided in the embodiment of the present application;

[0021] Figure 3 A schematic diagram of the principle of the robot motion control method provided in the embodiment of the present application;

[0022] Figure 4 A second flow chart of the robot motion control method provided in an embodiment of the present application;

[0023] Figure 5 One of the block diagrams of the robot motion control device provided in the embodiment of the present application;

[0024] Figure 6 The second block diagram of the robot motion control device provided in the embodiment of the present application.

[0025] Icon: 100 - electronic device; 110 - memory; 120 - processor; 130 - communication unit; 200 - robot motion control device; 201 - training module; 210 - acquisition module; 220 - processing module; 230 - control module. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0027] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0028] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0029] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0030] Please refer to Figure 1 , Figure 1 A block diagram of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may be, but is not limited to, a server, a robot, etc. The electronic device 100 includes a memory 110, a processor 120, and a communication unit 130. The memory 110, the processor 120, and the communication unit 130 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines.

[0031] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0032] The processor 120 is used to read / write data or programs stored in the memory 110 and execute corresponding functions. For example, the memory 110 stores a robot motion control device 200, and the robot motion control device 200 includes at least one software function module that can be stored in the memory 110 in the form of software or firmware. The processor 120 executes various functional applications and data processing by running software programs and modules stored in the memory 110, such as the robot motion control device 200 in the embodiment of the present application, that is, realizing the robot motion control method in the embodiment of the present application.

[0033] The communication unit 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through a network, and to send and receive data through the network.

[0034] It should be understood that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0035] Please refer to Figure 2 , Figure 2 This is one of the flow diagrams of the robot motion control method provided in the embodiment of the present application. The method is applied to the above-mentioned electronic device 100. The specific flow of the robot motion control method is described in detail below. In this embodiment, the method may include steps S110 to S140.

[0036] Step S110, obtaining a first environment image currently collected by the target robot and a current first joint position state of the target robot.

[0037] In the present embodiment, the target robot is the robot targeted by the current robot motion control. The target robot may include a camera, and the specific installation position and the number of cameras may be determined in combination with actual needs, and are not specifically limited herein. The image currently captured by the camera may be used as the first environmental image currently captured by the target robot. The current joint position state of the target robot may also be obtained as the first joint position state. The first joint position state is used to indicate the current position of each joint of the target robot (i.e., the angle of the motor of each joint). The first joint position state may include the current position of each joint. The first environmental image and the first joint position state are used to describe the environment at a moment and the joint state of the target robot.

[0038] Step S120, using a pre-trained target feature extraction network based on a channel attention mechanism to extract environmental features from the first environmental image to obtain a first environmental feature.

[0039] Step S130, inputting the first environmental feature and the first joint position state into a prediction network to obtain an initial motion control signal.

[0040] In this embodiment, the inventor of the present application has found through research that if the first environment image and the first joint position state are directly input into the prediction network to predict the control signal, the subtle features in the complex environment may not be accurately extracted for prediction, which may result in poor predicted control signals. The channel attention mechanism, by adjusting the importance of each channel, improves the ability to focus on key features, enabling the network to better process the channel-level information of the visual input.

[0041] Based on the above considerations, in this embodiment, when the first environment image is obtained, the first environment image is input into a target feature extraction network of a pre-trained channel attention mechanism, and the output result of the target feature extraction network based on the first environment image is used as the first environment feature. The target feature extraction network can be specifically determined in combination with actual needs and is not specifically limited here.

[0042] When the first environmental feature is obtained, the first environmental feature and the first joint position state can be input into the prediction network together, and the initial motion control signal can be obtained from the output result of the prediction network for the first environmental feature and the first joint position state. The prediction network is trained based on the sample environmental features, sample joint position states and sample motion control signals corresponding to each of the multiple moments, with the sample environmental features and joint position states as samples and the sample motion control signals as labels corresponding to the samples; that is, the prediction network is trained based on the sample environmental features, sample joint position states and sample motion control signals corresponding to each of the multiple moments, with the sample environmental features and joint position states as samples and the sample motion control signals as labels corresponding to the samples.

[0043] Step S140, controlling the movement of the target robot according to the initial movement control signal.

[0044] In this embodiment, when the initial motion control signal is obtained, the initial motion control signal can be directly used as a signal used directly during control to control the movement of the target robot so that the joint position state of the target robot at the next moment is the joint position state represented by the initial motion control signal. The initial motion control signal can also be adjusted, and the adjusted signal can be used as a signal used directly during control to control the movement of the target robot so that the joint position state of the target robot at the next moment is the joint position state represented by the adjusted initial motion control signal. The specific method of controlling the movement of the target robot according to the initial motion control signal can be determined in combination with actual needs and is not specifically limited here.

[0045] Optionally, as a possible implementation, the head and / or wrist of the target robot is provided with a camera, and optionally, the wrist camera obtains an image only from one perspective, or can obtain an image from two different perspectives (i.e., the wrist camera can obtain two images at the same time). Correspondingly, the image currently obtained by the robot head camera of the target robot and / or the image currently obtained by the robot wrist camera can be used as the first environment image currently collected by the target robot, that is, the first environment image includes the image data obtained by the head camera and / or the image data obtained by the wrist camera. It can be understood that the type of image data corresponding to the sample environment features used in the training process of obtaining the prediction network is the same as the type of image data of the first environment image, for example, all are image data obtained by the head camera, or all are image data obtained by the wrist camera, or all include image data obtained by the head camera and image data obtained by the wrist camera. In this way, it is convenient to use the image data obtained by the head camera and / or the image data obtained by the wrist camera to control the movement of the mechanical arm of the target robot.

[0046] The current positions of the joints of the target robot can also be obtained in any manner to obtain the first joint position state. It is understandable that if the target robot includes only one joint, the first joint position state only includes the joint position of the joint; if the target robot includes multiple joints, the first joint position state includes the joint positions of the multiple joints. The first environment image and the first joint position state are used to describe the state of the current environment and the target robot. If it is desired to control the movement of the mechanical arm of the target robot, the first joint position state may include the joint positions of the joints of the mechanical arm.

[0047] Optionally, the target feature extraction network is a SEnet network or an ECA-Net network.

[0048] In an exemplary implementation, Figure 3 As shown, the target feature extraction network is a SEnet network, that is, a Senet encoder. The Senet encoder uses a channel attention mechanism to extract the input image data x (that is, the first environment image, that is, Figure 3 The image data in the image is encoded to generate a feature vector U of the image (i.e., the first environmental feature mentioned above), which can capture important visual information in the environment. SEnet adaptively enhances the weights of important channels through the two steps of "Squeeze" and "Excitation", and extracts key visual features through "Re-weighting". Among them, the Squeeze operation compresses the spatial information of channel features into a global description; the Excitation operation enhances important channel features through adaptive weight adjustment; the Re-weighting operation uses the learned weights to re-weight the feature map to obtain a new feature map. The encoding process of SEnet can be expressed as: U = f tr (x), where f tr It represents the environmental feature U extracted by SEnet encoder.

[0049] In this embodiment, the prediction network can be specifically determined in combination with actual needs. As a possible implementation, the prediction network is a generative model, and the specific generative model of the prediction network can be determined in combination with actual needs. The inventors of the present application have found through research that the conditional variational autoencoder (CVAE) can generate output data that meets specific needs based on input conditional information, and is very suitable for robot state estimation and control instruction generation tasks. Therefore, the prediction network can be CVAE.

[0050] In this embodiment, the prediction network may include an encoder and a decoder, and the first environmental feature and the first joint position state may be input into the encoder so that the encoder can generate latent variables based on the first environmental feature and the first joint position state; thereafter, the above-mentioned latent variables may be input into the decoder so that the decoder can decode them based on the generated latent variables to generate the initial motion control signal.

[0051] As a way to achieve this, Figure 3 As shown, the prediction network is a CVAE network, which transforms the first environmental feature U and the first joint position state y (i.e. Figure 3The joint(p) in ( ) is input into the CVAE encoder to generate a latent variable z. The potential energy z can be regarded as an abstract representation of the first environment image and the first joint position state. The process can be expressed by the following formula: z~q(z|U,y)=N(μ(U,y),σ 2 (U,y)), where μ(U,y) represents the mean calculated by the encoder (i.e., the mean of the latent variable), σ 2 (U, y) represents the variance calculated by the encoder (i.e., the variance of the latent variable), μ(U, y) and σ 2 (U, y) reflects the potential distribution of visual features and joint position states. After that, the latent variable z is decoded by CVAE decoder to generate the initial motion control signal Both CVAE encoder and CVAE decoder are transform networks.

[0052] Optionally, after obtaining the initial motion control signal, in order to avoid jitter, a target motion control signal can be calculated through smoothing based on multiple historical motion control signals and the initial motion control signal, and then the target robot movement is controlled according to the target motion control signal. The joint position state of the target robot at the next moment is the state indicated by the target motion control signal. In this way, the generated initial motion control signal can be optimized to obtain the target motion control signal to ensure the motion accuracy and stability of the target robot. Afterwards, the target robot can be driven to perform actions according to the target motion control signal corresponding to the current environment and joint position state.

[0053] Alternatively, if Figure 3 As shown, a moving average filter can be used for smoothing to obtain a target motion control signal. When a moving average filter is used for smoothing, the most recent preset number of historical motion control signals and the initial motion control signal can be obtained, and the average value of the joint positions indicated by the above motion control signals is used as the joint position indicated by the target motion control signal.

[0054] In this embodiment, the image data currently acquired by the camera can be encoded by SEnet, and important features in the image can be extracted, and input into the CVAE model together with the current robot joint position state to generate a potential representation; then, the CVAE model decodes the potential representation to generate a preliminary motion control signal, and after further processing, generates the final joint control signal, thereby driving the robot to perform precise motion tasks. The above method is a visual motion control method based on conditional variational autoencoder (CVAE) and channel attention mechanism (SEnet). Combined with the channel attention mechanism of SEnet and the generation ability of CVAE, it is expected to improve the accuracy and adaptability of the robot's visual perception and motion control method in complex environments, and can improve the robot's motion control accuracy in complex environments.

[0055] The above-mentioned visual motion control method based on conditional variational autoencoder (CVAE) and channel attention mechanism (SEnet) can generate high-precision control instructions by enhancing the extraction and analysis of important features of image channels and combining robot joint state data. It is suitable for robot motion control in complex environments.

[0056] Please refer to Figure 4 , Figure 4 The second flowchart of the robot motion control method provided in the embodiment of the present application. In this embodiment, before step S110, the method may further include steps S101 to S102.

[0057] Step S101, obtaining multiple pieces of sample data.

[0058] Step S102: training the initial model according to the plurality of sample data to obtain a control strategy generation model.

[0059] In this embodiment, a large amount of original image data and the corresponding original joint position states taken by the robot head camera can be collected, and then the above data are preprocessed to obtain multiple sample data. Among them, each sample data includes a sample environment image corresponding to a moment, a sample joint position state and a sample motion control signal. That is, a sample data is used to represent a second environment image, a second joint position state, and a motion control signal of a robot at moment T. The second environment image at moment T is used to describe the environment at moment T, the second joint position state at moment T is used to describe the robot state at moment T, and the motion control signal at moment T is the motion control signal to be executed by the robot at moment T. The above data can be used to train the initial model to obtain a control strategy generation model to ensure that the control strategy generation model can generate high-precision motion control signals in different environments. During the training process, the second environment image and the second joint position state in a sample data are samples, and the sample motion control signal in the sample data is the label corresponding to the sample. The control strategy generation model includes the target feature extraction network and the prediction network. That is, the control strategy generation model including the target feature extraction network and the prediction network is obtained by training the above sample data in an end-to-end manner.

[0060] In this embodiment, the target feature extraction network is a SEnet network, which can be trained to extract important features in an image. SEnet compresses the spatial information of each channel into a global feature description through a "compression" operation, and then dynamically adjusts the weights according to the importance of different channels through an "excitation" operation to optimize the feature extraction capability. Finally, the feature vector U output by SEnet is a compressed feature representation of the image.

[0061] In this embodiment, the prediction network is a CVAE model. The CVAE model generates latent variables z by learning the joint distribution of visual features U and joint position states y, and generates motion control signals through the decoder. The training goal of CVAE is to minimize the reconstruction loss and KL divergence loss. The sum of the above two losses is as follows:

[0062] Γ CVAE =E q(z|U,y) [logp(y|z)]-D KL ([q(z|U,y)||p(z)])

[0063] Among them, the reconstruction loss is used to ensure that the joint control signal generated by the CVAE decoder is as close to the real joint control signal as possible; the KL divergence loss is used to minimize the difference between the latent variable distribution q(z|U,y) and the prior distribution p(z).

[0064] During the training process, the reconstruction loss and the KL divergence loss may be calculated, and the current initial model may be adjusted according to the obtained reconstruction loss and the KL divergence loss to obtain the control strategy generation model.

[0065] Joint training can be performed based on the above sample data. By optimizing the parameters of the SEnet and CVAE models, the models can generate high-precision control signals in different environments. SEnet is responsible for extracting important features from image data, while the CVAE model generates motion control signals to ensure that the robot can make the best decision based on the current environment and joint status. Multiple sets of experiments can also be used to verify the effectiveness and robustness of the method in complex environments. The accuracy and performance of the model can be evaluated by measuring the error (such as mean square error) between the generated control signal and the real signal.

[0066] In order to execute the corresponding steps in the above embodiments and various possible methods, a method for implementing a robot motion control device 200 is given below. Optionally, the robot motion control device 200 can adopt the above Figure 1 The device structure of the electronic device 100 is shown. Figure 5 , Figure 5 This is one of the block diagrams of the robot motion control device 200 provided in the embodiment of the present application. It should be noted that the basic principle and technical effects of the robot motion control device 200 provided in this embodiment are the same as those of the above embodiments. For the sake of brief description, for parts not mentioned in this embodiment, reference can be made to the corresponding contents in the above embodiments. In this embodiment, the robot motion control device 200 may include: an acquisition module 210, a processing module 220 and a control module 230.

[0067] The acquisition module 210 is used to obtain the first environment image currently acquired by the target robot and the current first joint position state of the target robot.

[0068] The processing module 220 is used to extract environmental features from the first environmental image using a pre-trained target feature extraction network based on a channel attention mechanism to obtain a first environmental feature.

[0069] The processing module 220 is further used to input the first environmental feature and the first joint position state into a prediction network to obtain an initial motion control signal. The prediction network is trained based on sample environmental features, sample joint position states, and sample motion control signals corresponding to multiple moments, with the sample environmental features and joint position states serving as samples and the sample motion control signals serving as labels corresponding to the samples.

[0070] The control module 230 is used to control the movement of the target robot according to the initial movement control signal.

[0071] Please refer to Figure 6 , Figure 6 The second block diagram of the robot motion control device 200 provided in the embodiment of the present application. In this embodiment, the robot motion control device 200 may further include a training module 201 .

[0072] The training module 201 is used to: obtain multiple sample data, wherein each sample data includes a sample environment image, a sample joint position state and a sample motion control signal corresponding to a moment; train the initial model according to the multiple sample data to obtain a control strategy generation model, wherein the control strategy generation model includes the target feature extraction network and the prediction network.

[0073] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory 110 shown in the figure may be fixed in the operating system (OS) of the electronic device 100 and may be Figure 1 Meanwhile, the data and program codes required for executing the above modules may be stored in the memory 110.

[0074] An embodiment of the present application also provides a readable storage medium having a computer program stored thereon, and the computer program implements the robot motion control method when executed by a processor.

[0075] In summary, the embodiment of the present application provides a robot motion control method, device, electronic device and readable storage medium. First, the first environment image currently obtained by the target robot and the current first joint position state of the target robot are obtained; then, the first environment image is extracted by using the pre-trained target feature extraction network based on the channel attention mechanism to obtain the first environment feature; then, the first environment feature and the first joint position state are input into the prediction network to obtain the initial motion control signal, and the prediction network is trained according to the sample environment features, sample joint position states and sample motion control signals corresponding to each of the multiple moments, the sample environment features and joint position states are used as samples, and the sample motion control signals are used as labels corresponding to the samples; finally, the target robot is controlled to move according to the initial motion control signal. In this way, the target feature extraction network can be used to accurately extract important features in the visual input, and then the signal for motion control of the robot is generated through the prediction network in combination with the joint position state, thereby improving the adaptability of the robot's visual perception and motion control strategy in complex environments, reducing the situation where the robot's actions do not match the actual environment, and facilitating the driving of the robot to perform precise motion tasks.

[0076] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0077] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0078] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0079] The above description is only an optional embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A robot motion control method, characterized in that: The method comprises: Obtaining a first environment image currently collected by the target robot and a current first joint position state of the target robot; Using a pre-trained target feature extraction network based on a channel attention mechanism to extract environmental features from the first environmental image, to obtain a first environmental feature; Inputting the first environmental feature and the first joint position state into a prediction network to obtain an initial motion control signal, wherein the prediction network is trained according to sample environmental features, sample joint position states, and sample motion control signals corresponding to each of multiple moments, the sample environmental features and joint position states are used as samples, and the sample motion control signals are used as labels corresponding to the samples; The target robot is controlled to move according to the initial motion control signal.

2. The method according to claim 1, characterized in that: The prediction network includes an encoder and a decoder, and the first environmental feature and the first joint position state are input into the prediction network to obtain an initial motion control signal, including: generating a latent variable based on the first environmental feature and the first joint position state using the encoder; The initial motion control signal is generated based on the generated latent variable using the decoder.

3. The method according to claim 1, characterized in that The step of controlling the target robot to move according to the initial motion control signal comprises: According to the plurality of historical motion control signals and the initial motion control signal, a target motion control signal is calculated by smoothing; The target robot is controlled to move according to the target motion control signal, wherein the joint position state of the target robot at the next moment is the state indicated by the target motion control signal.

4. The method according to claim 1, characterized in that: The method further comprises: Obtaining a plurality of sample data, wherein each sample data includes a sample environment image corresponding to a moment, a sample joint position state, and a sample motion control signal; The initial model is trained according to the plurality of sample data to obtain a control strategy generation model, wherein the control strategy generation model includes the target feature extraction network and the prediction network.

5. The method according to claim 4, characterized in that The prediction network is a conditional variational autoencoder, and the initial model is trained according to the plurality of sample data to obtain a control strategy generation model, including: During the training process, the reconstruction loss and the KL divergence loss are calculated, and the current initial model is adjusted according to the obtained reconstruction loss and the KL divergence loss to obtain the control strategy generation model.

6. The method according to claim 1, characterized in that The prediction network is a conditional variational autoencoder, and / or the target feature extraction network is a SEnet network or an ECA-Net network.

7. The method according to any one of claims 1 to 6, characterized in that: The first environment image and the image corresponding to the sample environment feature both include an image obtained by a robot head camera and / or an image obtained by a robot wrist camera.

8. A robot motion control device, characterized in that: The device comprises: An acquisition module, used to obtain a first environment image currently acquired by the target robot and a current first joint position state of the target robot; A processing module, configured to extract environmental features from the first environmental image using a pre-trained target feature extraction network based on a channel attention mechanism to obtain a first environmental feature; The processing module is further used to input the first environmental feature and the first joint position state into a prediction network to obtain an initial motion control signal, wherein the prediction network is trained according to sample environmental features, sample joint position states and sample motion control signals corresponding to each of multiple moments, the sample environmental features and joint position states are used as samples, and the sample motion control signals are used as labels corresponding to the samples; A control module is used to control the movement of the target robot according to the initial movement control signal.

9. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement the robot motion control method described in any one of claims 1-7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the robot motion control method according to any one of claims 1 to 7 is implemented.

Images