Robot control method, device, electronic device, computer-readable storage medium, and computer program product

Abstract

The present application provides a robot control method, device, electronic device, computer-readable storage medium and computer program product; the method includes: obtaining a first joint state of a robot's manipulator arm at a first moment, and an environmental image collected by the robot for the current environment at the first moment; determining a first joint feature between the environmental image and the first joint state, and predicting the joint state of the manipulator arm at a second moment based on the first joint feature to obtain a second joint state; determining antagonistic feedback details, and adjusting the state of the second joint state based on the antagonistic feedback details to obtain a third joint state; when the second moment is reached, controlling the manipulator arm to swing to the third joint state. Through the present application, the control accuracy of the robot can be improved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a robot control method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of robotics, vision-guided motion control has gained widespread application in robotic automation control systems. In particular, control systems that use robotic cameras to provide visual input enable the robot to autonomously perceive and interact with its environment. However, traditional vision-based motion control methods often require extensive testing and tuning in specific environments. Limited by the computing power of the controller and the generalizability of the algorithm, they often lack the precision to control the robot's manipulator arm in novel scenarios. Summary of the Invention

[0003] Embodiments of the present application provide a robot control method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the control accuracy of the robot.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] An embodiment of the present application provides a robot control method, the method comprising:

[0006] Acquire a first joint state of a robot's robotic arm at a first moment, and an environmental image collected by the robot for the current environment at the first moment; determine a first joint feature between the environmental image and the first joint state, and predict the joint state of the robotic arm at the second moment based on the first joint feature to obtain a second joint state; determine adversarial feedback details, and adjust the state of the second joint state based on the adversarial feedback details to obtain a third joint state; when the second moment is reached, control the robotic arm to swing to the third joint state.

[0007] An embodiment of the present application provides a robot control device, comprising:

[0008] an acquisition module, configured to acquire a first joint state of a robotic arm of the robot at a first moment, and an environmental image collected by the robot for a current environment at the first moment;

[0009] a prediction module, configured to determine a first joint feature between the environment image and the first joint state, and predict the joint state of the robotic arm at a second moment based on the first joint feature to obtain a second joint state;

[0010] an adjustment module, configured to determine antagonistic feedback details, and adjust the second joint state based on the antagonistic feedback details to obtain a third joint state;

[0011] The control module is used to control the robotic arm to swing to the third joint state when the second moment is reached.

[0012] In the above scheme, the prediction module is also used to extract features from the environmental image to obtain image features, and to extract features from the first joint state to obtain joint state features of the first joint state; map the image features and the joint state features to a latent space to obtain a first mapping feature of the image features in the latent space, a second mapping feature of the joint state features in the latent space, and an intrinsic correlation between the image features and the joint state features; based on the intrinsic correlation, perform feature fusion on the first mapping features and the second mapping features to obtain a first joint feature between the environmental image and the first joint state.

[0013] In the above scheme, the prediction module is also used to map the first joint feature from the latent space to the original space where the first joint state is located, and obtain a third mapping feature of the first joint feature in the original space; based on the third mapping feature, the joint state of the robotic arm at the second moment is predicted to obtain the second joint state.

[0014] In the above scheme, the adjustment module is also used to refine the joint state characteristics of the second joint state to obtain the refined joint state characteristics of the second joint state; add the antagonistic feedback details to the refined joint state characteristics to obtain the enhanced joint state characteristics; perform feature adjustment on the enhanced joint state characteristics to obtain the adjusted joint state characteristics, and generate the third joint state based on the adjusted joint state characteristics.

[0015] In the above scheme, the adjustment module is also used to obtain the first historical joint state of the robot's robotic arm at a first historical moment, the second historical joint state of the robotic arm at a second historical moment, and the historical environment image collected by the robot for the historical environment at the first historical moment; determine the second joint feature between the historical environment image and the first historical joint state, and predict the joint state of the robotic arm at the second historical moment based on the second joint feature to obtain a first predicted joint state; perform adversarial learning based on the first predicted joint state and the second historical joint state to obtain the adversarial feedback details.

[0016] In the above scheme, the adjustment module is also used to perform adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details; perform state adjustment on the first predicted joint state based on the first adversarial feedback details to obtain first adjusted joint state; perform adversarial learning based on the first adjusted joint state and the second historical joint state to obtain second adversarial feedback details; perform state adjustment on the first adjusted joint state based on the second adversarial feedback details to obtain second adjusted joint state, and when the error between the second adjusted joint state and the second historical joint state is lower than a preset error, the second adversarial feedback details are determined as the adversarial feedback details.

[0017] In the above scheme, the adjustment module is also used to classify the first predicted joint state to obtain a first category score of the first predicted joint state, and classify the second historical joint state to obtain a second category score of the second historical joint state; determine the first loss value of the adversarial learning based on the first category score and the second category score; determine the state difference between the first predicted joint state and the second historical joint state based on the first loss value, and obtain the first adversarial feedback details based on the state difference.

[0018] In the above scheme, the adjustment module is also used to determine a second loss and a third loss based on the first predicted joint state and the second historical joint state, wherein the second loss is used to characterize the similarity between the first predicted joint state and the second historical joint state, and the third loss is used to characterize the difference between the data space corresponding to the first predicted joint state and the data space corresponding to the second historical joint state; based on the second loss and the third loss, adversarial learning is performed on the first predicted joint state and the second historical joint state.

[0019] An embodiment of the present application provides an electronic device, comprising:

[0020] a memory for storing computer-executable instructions or computer programs;

[0021] The processor is used to implement the robot control method provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0022] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the robot control method provided in the embodiment of the present application when executed by a processor.

[0023] An embodiment of the present application provides a computer program product, including a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a processor, the robot control method provided in the embodiment of the present application is implemented.

[0024] The embodiments of the present application have the following beneficial effects:

[0025] In the above manner, by obtaining the first joint state of the robot at the first moment and the environmental image captured at the first moment, the first joint feature between the first joint state and the environmental image is determined to learn the joint information of the two, thereby making an initial prediction of the joint state of the robotic arm at the second moment based on the first joint feature to obtain the second joint state, and then, adjusting the state of the second joint state based on the adversarial feedback details determined by the adversarial learning (that is, a secondary prediction) to obtain a more accurate third joint state, thereby controlling the robot's robotic arm to swing based on the third joint state. This method obtains the correlation between the environmental image and the first joint state through the first joint feature, and further adjusts the state through the adversarial feedback details, which can be better generalized to the data of the robot in different scenarios, thereby improving the accuracy of the prediction of the future joint state of the robot's robotic arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic diagram of the structure of the robot control system provided by the embodiment of the present application;

[0027] Figure 2 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0028] Figure 3A 1 is a flow chart of a robot control method provided in an embodiment of the present application;

[0029] Figure 3B is a flowchart of a method for determining a first joint feature provided in an embodiment of the present application;

[0030] Figure 3C is a flow chart of a method for determining a third joint state provided in an embodiment of the present application;

[0031] Figure 3D is a flowchart of a method for determining countermeasure feedback details provided in an embodiment of the present application;

[0032] Figure 4 is a schematic diagram of a robot provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of the structure of the robot control model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0035] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0037] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or portion of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal. It can be implemented in whole or in part using software, hardware (such as processing circuits or memory), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the functionality of the module or unit.

[0038] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0039] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0040] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0041] 1) Latent space refers to the potential representation of the data learned by the encoder. This latent space is a mapping of high-dimensional data into a low-dimensional space. It captures the essential characteristics of the data, allowing the data to be effectively represented and manipulated in the low-dimensional space. The latent space provides a low-dimensional representation of the data, which helps reduce the complexity of the data and makes it easier to process. Each point in the latent space corresponds to a latent variable of the data, which is usually assumed to follow a certain probability distribution, such as a Gaussian distribution. : The latent space is usually continuous, which means that close points are also close in the latent space, which helps to generate coherent data samples.

[0042] 2) The original space refers to the space where the input data exists in the variational autoencoder. This space contains all possible input data points, which are used to learn the parameters of the encoder and decoder during the training process.

[0043] Embodiments of the present application provide a robot control method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the control accuracy of the robot.

[0044] See also Figure 1 , Figure 1 This is a schematic diagram of the architecture of the robot control system 100 provided in an embodiment of the present application. In order to support a robot control application, the terminal 401 is connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two. Here, the terminal 401 can be a terminal set on the robot, or a terminal that communicates with the robot in real time.

[0045] The terminal 401 is configured to send a robot control request to the server 200 in response to a control instruction for the robot. The server 200 is configured to obtain a first joint state of the robot's mechanical arm at a first moment, and an environmental image captured by the robot for the current environment at the first moment; determine a first joint feature between the environmental image and the first joint state, and predict the joint state of the mechanical arm at a second moment based on the first joint feature to obtain a second joint state; determine adversarial feedback details, and adjust the second joint state based on the adversarial feedback details to obtain a third joint state; the server 200 may send the third joint state to the terminal 401, so that the terminal 401 controls the mechanical arm to swing to the third joint state when the second moment arrives.

[0046] In some embodiments, the server 200 may directly send the third joint state obtained in response to the robot control request to the terminal 401 ; the terminal 401 may also actively obtain the third joint state from the server 200 .

[0047] In some embodiments, terminal 401 can be implemented as various types of terminals such as a robot terminal, a laptop computer, a tablet computer, a desktop computer, a set-top box, a smart phone, a smart speaker, a smart watch, a smart TV, a car terminal, etc., and can also be implemented as a server.

[0048] In some embodiments, server 200 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal and the server may be connected directly or indirectly via wired or wireless communication, which is not limited in the embodiments of the present application.

[0049] See also Figure 2 , Figure 2 is a structural diagram of an electronic device 400 provided in an embodiment of the present application, Figure 2 The electronic device 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the electronic device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0050] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0051] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0052] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0053] Memory 450 includes volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. Nonvolatile memory may be read-only memory (ROM), and volatile memory may be random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0054] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0055] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0056] A network communication module 452 is configured to communicate with other electronic devices via one or more (wired or wireless) network interfaces 420 . Exemplary network interfaces 420 include Bluetooth, Wi-Fi, and USB.

[0057] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripheral devices and displaying content and information);

[0058] The input processing module 454 is configured to detect one or more user inputs or interactions from one of the one or more input devices 432 and to translate the detected inputs or interactions.

[0059] In some embodiments, the robot control device provided in the embodiments of the present application can be implemented in a software manner. Figure 2A robot control device 455 stored in memory 450 is shown. This device can be software in the form of a program or plug-in, and includes the following software modules: an acquisition module 4551, a prediction module 4552, an adjustment module 4553, and a control module 4554. These modules are logical and can be arbitrarily combined or further separated according to the functions they implement. The functions of each module will be described below.

[0060] In other embodiments, the robot control device provided in the embodiments of the present application can be implemented in hardware. As an example, the robot control device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the robot control method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs) or other electronic components.

[0061] The following describes the robot control method provided by the embodiments of the present application with reference to the accompanying drawings. As previously mentioned, the electronic device implementing the robot control method of the embodiments of the present application can be a terminal 401, a server 200, or a combination of the two. Therefore, the execution entity of each step will not be repeatedly described below.

[0062] The robot control method of the embodiment of the present application is described by taking the execution subject as terminal 401 as an example. Figure 3A , Figure 3A This is a flow chart of the robot control method provided in the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained.

[0063] In step 101 , a first joint state of a robot's robotic arm at a first moment and an environment image collected by the robot for the current environment at the first moment are obtained.

[0064] In the embodiments of the present application, the robot has a movable robotic arm. The robot may be an industrial robot, used to perform industrial tasks such as handling, assembly, packaging, welding, and sorting through its robotic arm. The robot may also be a service robot, used for medical assistance (e.g., surgical assistance, rehabilitation assistance, etc.), household tasks (e.g., cleaning, nursing, etc.), and catering services (e.g., food preparation, delivery, etc.). The robot may also be any other robot capable of utilizing robotic arm functions, such as a detection robot, a rescue robot, or a bionic robot, without specific limitation herein. It will be understood that the robotic arms of different types of robots are designed with different degrees of freedom, ranging from a simple two degrees of freedom to highly complex seven or more degrees of freedom, enabling the robot to perform simple repetitive tasks or highly complex and precise operations.

[0065] Robots can also be equipped with cameras or other visual sensors to capture visual information from their surroundings (e.g., through images). Robots can analyze these images to identify and perceive specific objects, landmarks, obstacles, and other information in the environment.

[0066] Below, see Figure 4 , Figure 4 This is a schematic diagram of a robot provided in an embodiment of the present application. Taking a sorting robot as an example, the sorting robot is equipped with a head camera 110 and a wrist camera 120. The head camera 110 and the wrist camera 120 can collect environmental images of the robot's current environment. The robotic arm of the sorting robot has movable shoulder joints 130, elbow joints 140 and wrist joints 150. The sorting robot can control the position of each joint to control the robotic arm to perform the task of sorting items.

[0067] Each joint node of the robot is equipped with sensors (such as position sensors, velocity sensors, force sensors, accelerometers, etc.), and the robot can use these sensors to collect the joint status of each joint of the robotic arm.

[0068] Here, the joint state of a robot generally refers to the motion state and position information of the joints of the robotic arm at a specific moment. As an example, the joint state may include the following information: the specific position of the joint in space, which may be an angle (for a rotary joint) or a linear displacement (for a linear joint); speed, used to describe the speed of joint movement, usually expressed in degrees per second (for a rotary joint) or millimeters per second (for a linear joint); acceleration, indicating how fast the joint speed changes, usually expressed in degrees per second squared or millimeters per second squared; force or torque, used to reflect the magnitude of the force or torque applied or received by the joint; joint angle, for a rotary joint, this is the rotation angle of the joint axis relative to a reference position, such as the absolute rotation angle of a motor that controls the rotation of the joint; joint displacement, for a linear joint, this is the displacement of the joint along its axis; joint posture, describing the posture of the joint relative to a reference position, including a combination of angle and displacement; electrical parameters, including motor current, voltage and other information, which can be used to monitor the working status of the motor; encoder reading, the position feedback provided by the encoder, used to precisely control the joint position.

[0069] In actual implementation, when controlling a robot to perform a task, the robot's first joint state at a first moment and the environmental image captured by the robot's camera at the first moment of the current environment can be obtained. Here, the first moment can be understood as the current moment; the first joint state can include joint state information of one or more joints of the robot arm, and the first joint state information at least includes joint position information, such as joint angles, joint displacements, and other position information; the environmental image is an image of the direction the robot's camera is facing. For example, the head camera of a sorting robot can capture an image of the object to be sorted that it is facing.

[0070] During actual implementation, a trained robot control model can be configured in the robot's terminal 401. By inputting the environmental image captured by the robot's camera and the first joint state captured by the sensor into the robot control model, the robot control model adopts the robot control method of the embodiment of the present application to determine the third joint state for controlling the movement of the robot arm, thereby realizing control of the robot arm.

[0071] As an example, Figure 5 For a schematic diagram of the structure of the robot control model provided in the embodiment of the present application, see Figure 5 The robot control model can be composed of a variational autoencoder (VAE) and a generative adversarial network (GAN), where the variational autoencoder includes an encoder and a decoder, and the generative adversarial network includes a generator and a discriminator.

[0072] As an example, the model architecture of a robot control model can be built using a variational autoencoder and a generative adversarial network. The encoder and decoder in the variational autoencoder and the generator and discriminator in the generative adversarial network can be built based on a deep learning network, such as the Transformer network structure.

[0073] Continue reading Figure 3A , continue with the above step 101 for explanation.

[0074] In step 102 , a first joint feature between the environment image and the first joint state is determined, and the joint state of the robot arm at a second moment is predicted based on the first joint feature to obtain a second joint state.

[0075] Here, the second moment is later than the first moment, and the first moment is the current moment. The second moment is the next moment later than the current moment determined based on the control frequency of the robotic arm. Taking the control frequency of the robotic arm that controls the robotic arm once per second when the robot is working as an example, if the first moment is the current moment XX:XX:12, the second moment can be XX:XX:13.

[0076] In some embodiments, combined Figure 5 The robot control model shown in Figure 3B , “determining the first joint feature between the environment image and the first joint state” in step 102 can be implemented through steps 1021 to 1023.

[0077] In step 1021 , feature extraction is performed on the environment image to obtain image features, and feature extraction is performed on the first joint state to obtain joint state features of the first joint state.

[0078] In actual implementation, after obtaining the environment image and the first joint state at the first moment, the environment image and the first joint state can be used as inputs of the encoder in the variational autoencoder, and the encoder can be used to extract features of the environment image and the first joint state respectively.

[0079] For example, after receiving an environment image as input, the encoder of a variational autoencoder extracts features from the environment image through a series of convolutional layers built on a Transformer network. These convolutional layers capture local features and spatial hierarchical structure in the environment image. Image features can include: visual features. Basic visual features such as texture, color, shape, and edges of the environment image are extracted by the encoder. These visual features help the encoder understand the objects, scene, and background in the image; spatial relationships. These spatial relationships between objects in the environment image, such as position, orientation, and distance, help the robot control model understand the overall layout of the environment image; contextual information in the environment image, such as the interaction between objects and the surrounding environment. If an environment image contains a robotic arm object, the encoder will note the relationship between the robotic arm object and surrounding tools or objects; dynamic information. If the environment image is a sequence and the sequence is continuous, the encoder will extract dynamic changes between the environment images, such as the robot's motion trajectory or the position changes of objects; semantic information. Semantic information in the environment image, such as object categories, behaviors, and events, is also extracted by the encoder, especially if this information is relevant to the robot's task.

[0080] For example, after receiving the first joint state as input, the encoder of the variational autoencoder can extract joint state features from the first joint state through a series of fully connected layers or recurrent layers constructed by the Transformer network. When extracting features from the first joint state, the encoder typically focuses on key information that reflects joint motion and behavior, such as position information. This information enables the robot control model to better understand the current state of the manipulator and predict future behavior. Joint state features may include: position features, such as joint angle and joint displacement. For a rotary joint, the joint angle is the angle of the joint relative to a reference position or the absolute rotation angle of the motor controlling the joint. For a linear joint, the joint displacement is the displacement of the joint along its axis; velocity features, such as angular velocity and linear velocity. For a rotary joint, angular velocity is the magnitude of the joint angular velocity. For a linear joint, linear velocity is the speed of the joint displacement; acceleration features, such as angular acceleration and linear acceleration. For a rotary joint, angular acceleration is the magnitude of the joint angular acceleration. For a linear joint, linear acceleration is the acceleration of the joint displacement; force or torque features, such as joint force and joint torque. The joint force reflects the magnitude of the force applied by the joint, and the joint torque reflects the magnitude of the torque applied by the joint; dynamic features, such as motion smoothness and motion stability. Motion smoothness can reflect whether the joint motion is smooth and can be measured by the rate of change of acceleration, while motion stability can reflect whether the joint motion remains on the desired trajectory; time features, such as the periodic characteristics of joint motion, such as the period of swinging motion, time series features, etc. During feature extraction, derived features are also considered, such as those obtained through differentiation and integration operations, as well as those derived through statistical analysis (e.g., mean, standard deviation, variance, etc.). These features can be used in a variety of control strategies, including model predictive control, feedback control, adaptive control, and intelligent control (e.g., deep learning and reinforcement learning). The choice of joint state features depends on the specific control task, the design of the robot arm, and the required control accuracy. Feature selection is typically an iterative process that requires adjustment and optimization based on the control performance.

[0081] In step 1022, the image features and the joint state features are mapped to the latent space to obtain the first mapping features of the image features in the latent space, the second mapping features of the joint state features in the latent space, and the intrinsic correlation between the image features and the joint state features.

[0082] In actual implementation, after receiving the environment image and the first joint state, the encoder of the variational autoencoder will perform feature extraction on the environment image and the first joint state respectively through step 1021. The format of the obtained joint state features is compatible with the image features, for example, both are feature vectors of the same dimension or a matrix of a fixed size. Then, the integrated joint state features and image features are fed into different neural network layers of the encoder to perform feature mapping on the image features and the joint state features.

[0083] Here, the latent space refers to the potential representation of the data learned by the encoder network of the variational autoencoder. This latent space is a mapping of high-dimensional data into a low-dimensional space, which can capture the essential characteristics of the data and enable the data to be effectively represented and manipulated in the low-dimensional space. Specifically, the latent space provides a low-dimensional representation of the data, which can reduce the complexity of the data and make it easier to process. Each point in the latent space corresponds to a latent variable of the data, which is assumed to follow certain probability distributions. In addition, the latent space is usually continuous, which means that similar points are also close in the latent space, which helps to generate coherent data.

[0084] In actual implementation, in order to extract related information from the environment image and the first joint state and capture significant and useful information in the two types of data, the neural network layer of the encoder can map the image features and the joint state features to the latent space, and obtain the first mapping features of the image features in the latent space and the second mapping features of the joint state features in the latent space. The intrinsic correlation between the image features and the joint state features is obtained through the performance of the first mapping features in the latent space and the performance of the second mapping features in the latent space.

[0085] Here, the intrinsic association may include: spatial correspondence, the correspondence between the spatial position in the environmental image and the first joint state, for example, a specific area in the environmental image may correspond to a specific joint movement of the robotic arm; temporal relationship, if the input data contains time series information, the encoder can extract the temporal relationship between the environmental image and the first joint state, for example, a change in the first joint state may indicate an imminent change in certain features in the environmental image; motion trajectory relationship, the encoder can extract the relationship between the motion trajectory of the robotic arm and the motion trajectory of the object in the image; posture and perspective relationship, the encoder can extract how the posture change of the robotic arm affects the camera perspective, and how this change is reflected in the environmental image; feature mapping relationship, the encoder can map features in the environmental image to adjustments to the first joint state in order to achieve specific task goals.

[0086] In the above manner, the variational autoencoder of the robot control model learns the data distribution of the environment image and the first joint state in the latent space, which can make the model better generalize to new data and improve the generalization ability of the model.

[0087] In step 1023 , based on the intrinsic association, feature fusion is performed on the first mapping feature and the second mapping feature to obtain a first joint feature between the environment image and the first joint state.

[0088] In actual implementation, the encoder of the variational autoencoder can combine the first mapping feature and the second mapping feature based on the intrinsic correlation through a shared layer or a specific fusion layer, and output a latent vector. This latent vector is a low-dimensional representation of the intrinsic correlation between the environment image and the first joint state in the latent space. The latent vector can contain joint information of the environment image and the first key state. Here, the latent vector is the first joint feature, which is the comprehensive representation obtained by the encoder that can simultaneously characterize the environment image and the first joint state. Through feature fusion, the robot control model can understand the interaction between the environment image and the first joint state based on the first joint feature.

[0089] In the above manner, the encoder of the variational autoencoder processes the features of the environment image and the first joint state, so that the robot control model can learn the correlation between the visual features of the environment image and the first joint state. For example, in the robot sorting task, the direct relationship between the object position in the environment image and the joint of the robotic arm is obtained, and the encoder can reveal the dependence of certain features in the environment image on the first joint state. For example, the movement of a joint of the robotic arm may cause a change in the camera perspective, thereby affecting the input of the environment image; the encoder can also enable the robot control model to learn how the environment image and the first joint state change synergistically through the first joint feature.

[0090] The description will continue with the above step 102.

[0091] In some embodiments, "predicting the joint state of the robotic arm at the second moment based on the first joint feature to obtain the second joint state" in step 102 can be achieved in the following way: mapping the first joint feature from the latent space to the original space where the first joint state is located to obtain the third mapping feature of the first joint feature in the original space; predicting the joint state of the robotic arm at the second moment based on the third mapping feature to obtain the second joint state.

[0092] In actual implementation, the decoder of the variational autoencoder is responsible for mapping features in the latent space back to the original space, generating data similar to the original input.

[0093] Here, the original space refers to the space in which the input data of the variational autoencoder (here, the first joint state) exists. The original space contains all possible input data points; the original space is defined by the dimensionality of the input data; the data type in the original space depends on the type of the input data; the data points in the original space follow a certain distribution, which is usually unknown. One of the goals of the variational autoencoder is to approximate this distribution through learning; the data in the original space is usually represented as high-dimensional vectors that contain all the characteristics of the data.

[0094] In actual implementation, the decoder receives the first joint feature from the encoder's latent space and maps the first joint feature in the latent space back to the original space of the first joint state. This means converting the latent space representation (the first joint feature) into features (the third mapping feature) with the same dimensions as the original data (the joint state features of the first joint state). This mapping process can be implemented through a fully connected layer or a convolutional layer. The third mapping feature is then used to predict the joint state of the robot at the second moment, obtaining the second joint state.

[0095] Here, the second joint state is the result of the initial prediction made by the variational autoencoder for the position of the robot arm at the second moment.

[0096] Continue reading Figure 3A , continue with the above step 102 for description.

[0097] In step 103 , the antagonistic feedback details are determined, and the second joint state is adjusted based on the antagonistic feedback details to obtain a third joint state.

[0098] In actual implementation, the generator of the generative adversarial network can generate a more refined joint state from the second joint state output by the variational autoencoder based on the adversarial feedback details, resulting in a third joint state. Here, the adversarial feedback details come from the data distribution learned by the generator or the feedback obtained from the discriminator during training of the robot control model.

[0099] In some embodiments, see Figure 3C , “adjusting the second joint state based on the antagonistic feedback details to obtain the third joint state” in step 103 can be implemented through steps 1031 to 1033.

[0100] In step 1031 , the joint state feature of the second joint state is refined to obtain the refined joint state feature of the second joint state.

[0101] In actual implementation, after the generator receives the preliminary predicted second joint state, it can identify the joint state characteristics of the second joint state, and enhance some key features in the joint state characteristics of the second joint state to realize the process of feature refinement of the joint state characteristics of the second joint state, and obtain the refined joint state characteristics of the second joint state.

[0102] In step 1032, the adversarial feedback details are added to the refined joint state features to obtain enhanced joint state features.

[0103] In actual implementation, before adding details to the refined joint state features, the generator can also use convolutional layers or fully connected layers to map the refined joint state features to a higher dimensional space, and perform scaling, translation or other nonlinear transformations to produce more accurate joint position information.

[0104] In actual implementation, the generator can add adversarial feedback details to the refined joint state features on the basis of the refined joint state features to achieve feature enhancement and obtain enhanced joint state features.

[0105] In step 1033, feature adjustment is performed on the enhanced joint state feature to obtain an adjusted joint state feature, and a third joint state is generated based on the adjusted joint state feature.

[0106] In actual implementation, the generator can also perform feature adjustments on the enhanced joint state features, that is, optimize the state representation embodied by the enhanced joint state features to make them more consistent with the distribution of actual joint states. The adjusted joint state features obtained after feature adjustment can be understood as the joint state features obtained by the generator based on the second joint state through secondary prediction. That is, by decoding the enhanced joint state features, the third joint state predicted by the generator can be obtained. Through the generator's in-depth processing of the second joint state, the obtained third joint state can be more accurate, reliable, and practical.

[0107] During the above processing, the generator usually introduces noise to enhance the diversity of generation, thereby exploring and obtaining richer joint state information.

[0108] Continue reading Figure 3A , continue with the above step 103 for explanation.

[0109] In step 104 , when the second moment arrives, the robotic arm is controlled to swing to the third joint state.

[0110] In actual implementation, the third joint state is the predicted state of the robot's robotic arm at the second moment. Therefore, after obtaining the third joint state, the swing path of the robotic arm from the first moment to the second moment can be planned according to the state difference between the third joint state and the first joint state (for example, the rotation angle of the motor that controls the joint, etc.). Then, the robotic arm is controlled to swing based on the planned swing path, so that the robotic arm can be in the third joint state when it arrives at the second moment.

[0111] In the above manner, by obtaining the first joint state of the robot at the first moment and the environmental image captured at the first moment, the first joint feature between the first joint state and the environmental image is determined to learn the joint information of the two, thereby making an initial prediction of the joint state of the robotic arm at the second moment based on the first joint feature to obtain the second joint state, and then, adjusting the state of the second joint state based on the adversarial feedback details determined by the adversarial learning (that is, a secondary prediction) to obtain a more accurate third joint state, thereby controlling the robot's robotic arm to swing based on the third joint state. This method obtains the correlation between the environmental image and the first joint state through the first joint feature, and further adjusts the state through the adversarial feedback details, which can be better generalized to the data of the robot in different scenarios, thereby improving the accuracy of the prediction of the future joint state of the robot's robotic arm.

[0112] Continue reading Figure 3A , continue to explain the "determining the details of the adversarial feedback" in the above step 103.

[0113] In some embodiments, the details of the adversarial feedback in step 103 may be obtained during the training of the robot control model, see Figure 3D , determining the adversarial feedback details can be achieved through steps 1034 to 1036.

[0114] In step 1034, a first historical joint state of the robot's manipulator arm at a first historical moment, a second historical joint state of the manipulator arm at a second historical moment, and a historical environment image collected by the robot for the historical environment at the first historical moment are obtained.

[0115] In actual implementation, firstly, training samples used for training the robot control model are prepared. Here, the training samples can use the historical joint states and historical environment images of the robot in the historical period.

[0116] As an example, a set of training samples used to train a robot control model may include: a first historical joint state of the robot's manipulator arm at a first historical moment, a second historical joint state at a second historical moment, and historical environment images captured by the robot's head and wrist cameras at the first historical moment. Here, the second historical moment is later than the first historical moment. During training, the robot control model can predict the manipulator arm's joint state at the second historical moment based on the first historical joint state and the historical environment images. The model is then trained based on the actual state at the second historical moment (i.e., the second historical joint state) and the predicted joint state.

[0117] In step 1035 , a second joint feature between the historical environment image and the first historical joint state is determined, and the joint state of the robotic arm at the second historical moment is predicted based on the second joint feature to obtain a first predicted joint state.

[0118] In actual implementation, see Figure 5 , the historical environment image and the first historical joint state are used as the input of the encoder in the variational autoencoder, and the encoder extracts features of the historical environment image and the first historical joint state respectively to obtain historical image features and first historical joint state features. Then, the historical image features and the first historical joint state features are mapped to the latent space to obtain the first historical mapping features of the historical image features in the latent space, the second historical mapping features of the first historical joint state features in the latent space, and the intrinsic correlation between the historical image features and the first historical joint state features. Based on the intrinsic correlation, the first historical mapping features and the second historical mapping features are feature fused to obtain the second joint feature between the historical environment image and the first historical joint state.

[0119] In actual implementation, when the encoder combines the first historical mapping feature and the second historical mapping feature based on the intrinsic correlation, it outputs the second joint feature by learning the potential distribution between the two.

[0120] As an example, the encoder can obtain the second joint feature by the following formula:

[0121]

[0122] in, Represents historical environmental images; Indicates the first historical joint state; represents the second joint feature (i.e., latent vector); It describes the encoder converting the historical environment image and the first historical joint state Compressed into a low-dimensional latent vector ; Representing a computational history environment image and the first historical joint state The mean between the features of Representing a computational history environment image and the first historical joint state The variance between the features.

[0123] In the above formula, The process of description is mainly to explore and express the image of the historical environment and the first historical joint state The potential relationship between and the first historical joint state As input, the encoder tries to find the intrinsic relationship between them and compress this information into a low-dimensional space, which can help the model understand the interaction between the two, for example, how the position of the robotic arm affects or is affected by certain features in the image. represents a point in the latent space that captures the input data (historical environment image and the first historical joint state ), in the variational autoencoder, the mean can be regarded as the central position of the latent variable, that is, it represents the most likely latent representation of the historical environment image and the first historical joint state. The uncertainty or range of the latent variable describes the diversity of the input data or the dispersion of the latent representations. In practice, large variance indicates that the encoder has high uncertainty about the given input, either because the input data itself has large variations or because there are multiple possible representations in the latent space.

[0124] In general, the encoder captures the input data (historical environment images) by calculating the mean and variance. and the first historical joint state ) and use this to express the complex relationships between them. The mean and variance together define a probability distribution, and the latent vector z is sampled from this distribution for subsequent decoding and generation processes.

[0125] Subsequently, the decoder of the variational autoencoder is responsible for mapping the features in the latent space back to the original space, generating data similar to the original input.

[0126] As an example, the decoder receives a point in the latent space from the encoder (i.e., the latent vector z), maps the point in the latent space back to the original space (the data space where the first historical joint state is located), and then generates a probability distribution that can generate new data (the first predicted joint state).

[0127] As an example, the first predicted joint state can be obtained by the following formula:

[0128]

[0129] in, represents the first predicted joint state, Representation based on the potential vector z and the first historical joint state Determined probability distribution.

[0130] The description will continue with the above step 1035.

[0131] In step 1036 , adversarial learning is performed based on the first predicted joint state and the second historical joint state to obtain adversarial feedback details.

[0132] In some embodiments, step 1036 can be implemented by: performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details; performing state adjustment on the first predicted joint state based on the first adversarial feedback details to obtain a first adjusted joint state; performing adversarial learning based on the first adjusted joint state and the second historical joint state to obtain second adversarial feedback details; performing state adjustment on the first adjusted joint state based on the second adversarial feedback details to obtain a second adjusted joint state, and when the error between the second adjusted joint state and the second historical joint state is lower than a preset error, determining the second adversarial feedback details as the adversarial feedback details.

[0133] In actual implementation, adversarial learning in a generative adversarial network is a game between a generator and a discriminator. The goal of this game is for the generator to learn to generate more realistic joint states, while the discriminator learns to distinguish between joint states generated by the generator and real joint states. Specifically, the generator receives the first predicted joint states predicted by the decoder and attempts to generate joint states with a data distribution similar to that of the real data (the second historical joint states). The generator's training goal is to deceive the discriminator, making it unable to distinguish between the generated data and the real data. The discriminator receives the joint states generated by the generator and the real data (the second historical joint states), classifies the generated data and the real data, and determines the score for the second historical joint states classified as real and the score for the generated joint states classified as non-real. The discriminator's training goal is to improve its ability to distinguish between real data and generated data.

[0134] In actual implementation, the training process of the adversarial learning network is an iterative process, which may include the following processes: initializing the parameters of the generator and the discriminator; under the condition of fixing the discriminator, obtaining the first adversarial feedback details and feeding them back to the generator through the discriminator's judgment on the first predicted joint state and the second historical joint state; updating the parameters of the generator based on the judgment result of the discriminator, and the generator with updated parameters may adjust the state of the first predicted joint state based on the first adversarial feedback details to obtain the first adjusted joint state, and give the first adjusted joint state to the discriminator; under the condition of fixing the generator, updating the parameters of the discriminator according to the generation result of the generator, and making the discriminator make authenticity judgment on the first adjusted joint state and the second historical joint state, and obtaining the second adversarial feedback details according to the judgment; then fixing the discriminator, updating the parameters of the generator, and making the generator adjust the state of the first adjusted joint state based on the second adversarial feedback details to obtain the second adjusted joint state; repeating the above process until the error between the second adjusted joint state and the second historical joint state is lower than the preset error, and determining the second adversarial feedback details as the adversarial feedback details. As iterative training progresses, the confrontation between the generator and the discriminator reaches a dynamic balance. The joint states generated by the generator become more and more realistic, while the discriminator becomes more and more adept at distinguishing between real and generated data.

[0135] Here, when training a generative adversarial network, adversarial feedback details refer to the specific information about the interaction between the generator and the discriminator during training. These feedback details are crucial for understanding the training dynamics of the adversarial network and improving model performance.

[0136] In some embodiments, "performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details" can be achieved in the following ways: classifying the first predicted joint state to obtain a first category score for the first predicted joint state, and classifying the second historical joint state to obtain a second category score for the second historical joint state; determining a first loss value for adversarial learning based on the first category score and the second category score; determining a state difference between the first predicted joint state and the second historical joint state based on the first loss value, and obtaining first adversarial feedback details based on the state difference.

[0137] In practice, adversarial feedback details refer to the specific information about the interaction between the generator and the discriminator during training. During training, the generator receives feedback from the discriminator to understand how similar its generated samples are to real samples. The discriminator usually determines the details of the adversarial feedback by generating loss.

[0138] As an example, after receiving the first predicted joint state, the discriminator will classify the first predicted joint state and the second historical joint state, that is, classifying the first predicted joint state as non-real and classifying the second historical joint state as real. The purpose of classification is to distinguish between real data and generated data. When performing classification, the discriminator calculates the category score of the data classification. For example, classifying the first predicted joint state can obtain the first category score of the first predicted joint state, and classifying the second historical joint state can obtain the second category score of the second historical joint state.

[0139] The first loss value of adversarial learning is calculated using the following formula:

[0140]

[0141] in, Indicates the second category score; Indicates the first category score; represents the first loss value; Represents the expected output of the discriminator for the real data (the second historical joint state), which measures the discriminator's prediction quality for the second historical joint state. The larger the value of , the more reliable the discriminator is in classifying the second historical joint state; represents the expected output of the discriminator for the generated data (the first predicted joint state), which measures the accuracy of the discriminator's judgment on the first predicted joint state. Ideally, the discriminator should correctly classify the first predicted joint state as non-real, that is, The value of tends to 0, so The larger it is, the more accurate the discriminator is in identifying the first predicted joint state.

[0142] Then, the first loss value of adversarial learning is determined by the sum of the first category score and the second category score. ; Based on the first loss value, the state difference between the first predicted joint state and the second historical joint state is determined, and the first adversarial feedback details are obtained based on the state difference, that is, the discriminator can obtain information such as parameters for gradient update as the first adversarial feedback details based on the first loss value.

[0143] In some embodiments, "performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details" can also be achieved in the following way: based on the first predicted joint state and the second historical joint state, determining the second loss and the third loss, wherein the second loss is used to characterize the degree of similarity between the first predicted joint state and the second historical joint state, and the third loss is used to characterize the degree of difference between the data space corresponding to the first predicted joint state and the data space corresponding to the second historical joint state; based on the second loss and the third loss, performing adversarial learning on the first predicted joint state and the second historical joint state.

[0144] In actual implementation, after predicting the first predicted joint state, the variational autoencoder also calculates its prediction loss, which includes a second loss and a third loss. The second loss is the variational autoencoder's reconstruction loss, which is used to characterize the similarity between the first predicted joint state and the second historical joint state. The third loss is the variational autoencoder's KL divergence loss, which is used to characterize the difference between the data space corresponding to the first predicted joint state and the data space corresponding to the second historical joint state.

[0145] As an example, the prediction loss can be calculated by the following formula:

[0146]

[0147] in, represents the prediction loss; represents the first predicted joint state; represents the potential distribution of the latent vector z; represents the prior distribution of the latent vector z; represents the reconstruction loss; Represents KL divergence loss, which can measure the difference between the potential distribution and the prior distribution. The smaller the loss value, the closer the learned potential distribution is to the set prior distribution.

[0148] In actual implementation, the second loss, the third loss and the first loss can be combined to reversely adjust the parameters of the robot control model, that is, to train the robot control model.

[0149] As an example, the fusion of losses can be achieved by the following formula:

[0150]

[0151] in, represents the total loss of the robot control model; represents the prediction loss (i.e., the sum of the second loss and the third loss); Indicates the first loss; It is a hyperparameter that weighs the two loss terms to balance the generation quality and reconstruction accuracy of the model.

[0152] In actual implementation, the total loss obtained can be used to reversely adjust the relevant parameters of the entire model. Then, based on the above training process, the robot control model is iteratively trained again using other training samples until the total loss meets the preset loss value or the number of iterations reaches the preset number. The iterative training is ended to obtain a trained robot control model.

[0153] By combining a generative adversarial network and a variational autoencoder in a robot control model, the variational autoencoder can be used to generate a latent vector that conforms to a probability distribution, learn the correlation between the environmental image and the joint state, and improve the accuracy of predicting future joint states. Furthermore, by learning the data distribution in the latent space, the model can better generalize to new data. Adversarial learning through the generative adversarial network can improve the accuracy of the generated joint states. By combining these two network structures, the robot can regenerate a more precise and robust control strategy for visual motion control tasks. Once the robot control model is trained, the inference process is very fast, making it suitable for real-time robot control and for robot motion control tasks in various complex environments.

[0154] In a specific embodiment, the robot control system of the embodiment of the present application may include the following modules:

[0155] Input module, used for visual information data and joint state input of the robotic arm.

[0156] Specifically, the robot's head camera and wrist camera can capture environmental images in real time, allowing the robot to perceive the surrounding environmental conditions; the sensors at the joints can capture the first joint status of the corresponding joints in real time as state feedback for the robot's motion control.

[0157] Model processing module, the robot control model carried by the robot includes variational autoencoder and adversarial learning network.

[0158] Specifically, the variational autoencoder's encoder receives the environment image x and the first joint state y from the input module and encodes them into a latent vector z in the latent space. The decoder decodes the latent vector z to generate preliminary robotic arm joint control instructions (i.e., the second joint state mentioned above). The generator of the generative adversarial network receives the preliminary robotic arm joint control instructions (i.e., the second joint state mentioned above) and generates more refined joint control instructions (i.e., the third joint state mentioned above).

[0159] The control module is used to control the corresponding joints of the robot's mechanical arm to perform corresponding operations according to the generated joint control instructions (i.e. the above-mentioned third joint state).

[0160] In another specific embodiment, the training process of the robot control model may include the following process:

[0161] First, data preparation involves collecting a large number of training samples, including the robot's historical environment images and the corresponding historical joint states.

[0162] Then, the variational autoencoder is trained to learn the latent vector z between the input historical environment image and the historical joint state by minimizing the reconstruction loss and the KL divergence loss. The calculation process of the latent vector z and the two losses can be referred to in the relevant embodiments above and will not be repeated here.

[0163] Furthermore, the goal of training the adversarial learning network is to optimize the joint states generated by the generator to make them closer to the true joint states. The generator receives the joint control instructions initially generated by the decoder (i.e., the first predicted joint state) as input and generates refined joint control instructions (the second predicted joint state). The discriminator simultaneously receives the true joint state and the joint control instructions generated by the generator and outputs two classification values, representing the probabilities of the true joint state and the joint control instructions generated by the generator, respectively. The first loss is calculated based on the first loss calculation formula in the above-mentioned related embodiments.

[0164] Finally, the reconstruction loss, KL divergence loss and the first loss are combined to obtain the total loss, and the parameters of the robot control model are adjusted based on the total loss.

[0165] It should be noted that during the training process, the parameters of the variational autoencoder and the adversarial learning network can be alternately optimized so that the generator can generate more realistic joint control instructions.

[0166] The following continues to describe the exemplary structure of the robot control device 455 provided in the embodiment of the present application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the robot control device 455 of the memory 450 may include:

[0167] The acquisition module 4551 is used to acquire a first joint state of the robot's mechanical arm at a first moment, and an environment image collected by the robot for the current environment at the first moment.

[0168] The prediction module 4552 is used to determine a first joint feature between the environment image and the first joint state, and predict the joint state of the robot arm at the second moment based on the first joint feature to obtain a second joint state.

[0169] The adjustment module 4553 is used to determine the antagonistic feedback details and adjust the second joint state based on the antagonistic feedback details to obtain the third joint state.

[0170] The control module 4554 is used to control the robotic arm to swing to the third joint state when the second moment arrives.

[0171] In some embodiments, the prediction module 4552 is also used to extract features from the environmental image to obtain image features, and to extract features from the first joint state to obtain joint state features of the first joint state; map the image features and the joint state features to the latent space to obtain first mapping features of the image features in the latent space, second mapping features of the joint state features in the latent space, and the intrinsic correlation between the image features and the joint state features; based on the intrinsic correlation, perform feature fusion on the first mapping features and the second mapping features to obtain a first joint feature between the environmental image and the first joint state.

[0172] In some embodiments, the prediction module 4552 is also used to map the first joint feature from the latent space to the original space where the first joint state is located, to obtain a third mapping feature of the first joint feature in the original space; and predict the joint state of the robotic arm at the second moment based on the third mapping feature to obtain the second joint state.

[0173] In some embodiments, the adjustment module 4553 is also used to refine the joint state characteristics of the second joint state to obtain the refined joint state characteristics of the second joint state; add the antagonistic feedback details to the refined joint state characteristics to obtain the enhanced joint state characteristics; perform feature adjustment on the enhanced joint state characteristics to obtain the adjusted joint state characteristics, and generate the third joint state based on the adjusted joint state characteristics.

[0174] In some embodiments, the adjustment module 4553 is also used to obtain the first historical joint state of the robot's robotic arm at a first historical moment, the second historical joint state of the robotic arm at a second historical moment, and the historical environment image collected by the robot for the historical environment at the first historical moment; determine the second joint feature between the historical environment image and the first historical joint state, and predict the joint state of the robotic arm at the second historical moment based on the second joint feature to obtain a first predicted joint state; perform adversarial learning based on the first predicted joint state and the second historical joint state to obtain adversarial feedback details.

[0175] In some embodiments, the adjustment module 4553 is also used to perform adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details; perform state adjustment on the first predicted joint state based on the first adversarial feedback details to obtain a first adjusted joint state; perform adversarial learning based on the first adjusted joint state and the second historical joint state to obtain second adversarial feedback details; perform state adjustment on the first adjusted joint state based on the second adversarial feedback details to obtain a second adjusted joint state, and when the error between the second adjusted joint state and the second historical joint state is lower than a preset error, the second adversarial feedback details are determined as the adversarial feedback details.

[0176] In some embodiments, the adjustment module 4553 is also used to classify the first predicted joint state to obtain a first category score of the first predicted joint state, and classify the second historical joint state to obtain a second category score of the second historical joint state; determine the first loss value of adversarial learning based on the first category score and the second category score; determine the state difference between the first predicted joint state and the second historical joint state based on the first loss value, and obtain the first adversarial feedback details based on the state difference.

[0177] In some embodiments, the adjustment module 4553 is also used to determine a second loss and a third loss based on the first predicted joint state and the second historical joint state, wherein the second loss is used to characterize the degree of similarity between the first predicted joint state and the second historical joint state, and the third loss is used to characterize the degree of difference between the data space corresponding to the first predicted joint state and the data space corresponding to the second historical joint state; based on the second loss and the third loss, adversarial learning is performed on the first predicted joint state and the second historical joint state.

[0178] An embodiment of the present application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the robot control method described above in the embodiment of the present application.

[0179] The embodiment of the present application provides a computer-readable storage medium in which computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will be caused to execute the robot control method provided by the embodiment of the present application, for example, Figure 3A The robot control method is shown.

[0180] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0181] In some embodiments, computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0182] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0183] By way of example, computer-executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed across multiple sites and interconnected by a communication network.

[0184] To summarize, through the embodiments of the present application, by obtaining the first joint state of the robot at the first moment and the environmental image captured at the first moment, the first joint feature between the first joint state and the environmental image is determined to learn the joint information of the two, thereby making an initial prediction of the joint state of the robotic arm at the second moment based on the first joint feature to obtain the second joint state, and then, adjusting the state of the second joint state based on the adversarial feedback details determined by adversarial learning (that is, secondary prediction) to obtain a more accurate third joint state, thereby controlling the robot's robotic arm to swing based on the third joint state. This method obtains the correlation between the environmental image and the first joint state through the first joint feature, and further adjusts the state through the adversarial feedback details, which can be better generalized to the data of the robot in different scenarios, thereby improving the accuracy of the prediction of the future joint state of the robot's robotic arm.

[0185] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A robot control method, characterized in that: The method comprises: Acquire a first joint state of a robotic arm of the robot at a first moment, and an environment image collected by the robot for a current environment at the first moment; Mapping the image features of the environment image and the joint state features of the first joint state into a latent space to obtain a first mapping feature of the image features in the latent space, a second mapping feature of the joint state features in the latent space, and an intrinsic association between the image features and the joint state features; Based on the intrinsic association, feature fusion is performed on the first mapping feature and the second mapping feature to obtain a first joint feature between the environment image and the first joint state, and the joint state of the robotic arm at a second moment is predicted based on the first joint feature to obtain a second joint state; Determining antagonistic feedback details, and adjusting the second joint state based on the antagonistic feedback details to obtain a third joint state; When the second moment arrives, the robotic arm is controlled to swing to the third joint state.

2. The method according to claim 1, characterized in that The predicting the joint state of the robot arm at the second moment based on the first joint feature to obtain a second joint state includes: Mapping the first joint feature from the latent space to the original space where the first joint state is located to obtain a third mapping feature of the first joint feature in the original space; The joint state of the robotic arm at the second moment is predicted based on the third mapping feature to obtain a second joint state.

3. The method according to claim 1, characterized in that The step of adjusting the second joint state based on the antagonistic feedback details to obtain a third joint state includes: Refining the joint state feature of the second joint state to obtain a refined joint state feature of the second joint state; adding the antagonistic feedback details to the refined joint state feature to obtain an enhanced joint state feature; The enhanced joint state feature is subjected to feature adjustment to obtain an adjusted joint state feature, and the third joint state is generated based on the adjusted joint state feature.

4. The method according to claim 1, wherein The determining of the adversarial feedback details includes: Acquire a first historical joint state of a robotic arm of the robot at a first historical moment, a second historical joint state of the robotic arm at a second historical moment, and a historical environment image collected by the robot for the historical environment at the first historical moment; determining a second joint feature between the historical environment image and the first historical joint state, and predicting the joint state of the robotic arm at the second historical moment based on the second joint feature to obtain a first predicted joint state; Adversarial learning is performed based on the first predicted joint state and the second historical joint state to obtain the adversarial feedback details.

5. The method according to claim 4, characterized in that The performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain the adversarial feedback details includes: performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details; and performing state adjustment on the first predicted joint state based on the first adversarial feedback details to obtain a first adjusted joint state. performing adversarial learning based on the first adjusted joint state and the second historical joint state to obtain second adversarial feedback details; The first adjusted joint state is adjusted based on the second antagonistic feedback detail to obtain a second adjusted joint state, and when the error between the second adjusted joint state and the second historical joint state is lower than a preset error, the second antagonistic feedback detail is determined as the antagonistic feedback detail.

6. The method according to claim 5, characterized in that The performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details includes: Classifying the first predicted joint state to obtain a first category score of the first predicted joint state, and classifying the second historical joint state to obtain a second category score of the second historical joint state; Determining a first loss value for the adversarial learning based on the first category score and the second category score; A state difference between the first predicted joint state and the second historical joint state is determined based on the first loss value, and the first adversarial feedback details are obtained based on the state difference.

7. The method according to claim 5, characterized in that The performing adversarial learning based on the first predicted joint state and the second historical joint state includes: Determining a second loss and a third loss based on the first predicted joint state and the second historical joint state, wherein the second loss is used to characterize the similarity between the first predicted joint state and the second historical joint state, and the third loss is used to characterize the difference between the data space corresponding to the first predicted joint state and the data space corresponding to the second historical joint state; Based on the second loss and the third loss, adversarial learning is performed on the first predicted joint state and the second historical joint state.

8. A robot control device, characterized in that: The device comprises: an acquisition module, configured to acquire a first joint state of a robotic arm of the robot at a first moment, and an environmental image collected by the robot for a current environment at the first moment; a prediction module, configured to map the image features of the environment image and the joint state features of the first joint state into a latent space, obtain a first mapping feature of the image feature in the latent space, a second mapping feature of the joint state feature in the latent space, and an intrinsic association between the image feature and the joint state feature; based on the intrinsic association, perform feature fusion on the first mapping feature and the second mapping feature to obtain a first joint feature between the environment image and the first joint state; and predict the joint state of the robotic arm at a second moment based on the first joint feature to obtain a second joint state; an adjustment module, configured to determine antagonistic feedback details, and adjust the second joint state based on the antagonistic feedback details to obtain a third joint state; The control module is used to control the robotic arm to swing to the third joint state when the second moment is reached.

9. An electronic device, characterized in that: The electronic device comprises: a memory for storing computer-executable instructions or computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing computer-executable instructions or computer programs stored in the memory.

10. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images