Robot control method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides a robot control method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining a first joint state of a mechanical arm of a robot at a first moment and an environment image collected by the robot for a current environment at the first moment; determining a first joint feature between the environment image and the first joint state, and predicting a joint state of the mechanical arm at a second moment based on the first joint feature to obtain a second joint state; confrontation feedback details are determined, state adjustment is conducted on the second joint state based on the confrontation feedback details, and a third joint state is obtained; and when the second moment arrives, the mechanical arm is controlled to swing to the third joint state. According to the invention, the control accuracy of the robot can be improved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a robot control method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of robotics technology, vision-guided motion control has been widely used in robotic automation control systems. In particular, control systems that use robot cameras to provide visual input can enable the robot to autonomously perceive and interact with the environment. However, traditional visual motion control methods usually require a lot of testing and adjustments in specific environments. Due to the limitations of the controller's computing power and the generalization of the algorithm, the robot's arm control accuracy is poor when dealing with new scenarios. Summary of the invention

[0003] The embodiments of the present application provide a robot control method, device, electronic device, computer-readable storage medium and computer program product, which can improve the control accuracy of the robot.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a robot control method, the method comprising:

[0006] Acquire a first joint state of a robot's mechanical arm at a first moment, and an environmental image collected by the robot for the current environment at the first moment; determine a first joint feature between the environmental image and the first joint state, and predict the joint state of the mechanical arm at the second moment based on the first joint feature to obtain a second joint state; determine adversarial feedback details, and adjust the state of the second joint state based on the adversarial feedback details to obtain a third joint state; when the second moment is reached, control the mechanical arm to swing to the third joint state.

[0007] The present application provides a robot control device, comprising:

[0008] An acquisition module, used to acquire a first joint state of a robot's mechanical arm at a first moment, and an environment image collected by the robot for a current environment at the first moment;

[0009] A prediction module, used for determining a first joint feature between the environment image and the first joint state, and predicting the joint state of the robot arm at a second moment based on the first joint feature to obtain a second joint state;

[0010] an adjustment module, configured to determine the details of the antagonistic feedback, and adjust the second joint state based on the details of the antagonistic feedback to obtain a third joint state;

[0011] The control module is used to control the robotic arm to swing to the third joint state when the second moment is reached.

[0012] In the above scheme, the prediction module is also used to extract features from the environment image to obtain image features, and to extract features from the first joint state to obtain joint state features of the first joint state; map the image features and the joint state features to a latent space to obtain a first mapping feature of the image features in the latent space, a second mapping feature of the joint state features in the latent space, and an intrinsic correlation between the image features and the joint state features; based on the intrinsic correlation, perform feature fusion on the first mapping features and the second mapping features to obtain a first joint feature between the environment image and the first joint state.

[0013] In the above scheme, the prediction module is also used to map the first joint feature from the latent space to the original space where the first joint state is located, and obtain a third mapping feature of the first joint feature in the original space; based on the third mapping feature, the joint state of the robotic arm at the second moment is predicted to obtain the second joint state.

[0014] In the above scheme, the adjustment module is also used to refine the joint state characteristics of the second joint state to obtain the refined joint state characteristics of the second joint state; add the antagonistic feedback details to the refined joint state characteristics to obtain the enhanced joint state characteristics; perform feature adjustment on the enhanced joint state characteristics to obtain the adjusted joint state characteristics, and generate the third joint state based on the adjusted joint state characteristics.

[0015] In the above scheme, the adjustment module is also used to obtain the first historical joint state of the robot's mechanical arm at a first historical moment, the second historical joint state of the mechanical arm at a second historical moment, and the historical environment image collected by the robot for the historical environment at the first historical moment; determine the second joint feature between the historical environment image and the first historical joint state, and predict the joint state of the mechanical arm at the second historical moment based on the second joint feature to obtain a first predicted joint state; perform adversarial learning based on the first predicted joint state and the second historical joint state to obtain the adversarial feedback details.

[0016] In the above scheme, the adjustment module is also used to perform adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details; perform state adjustment on the first predicted joint state based on the first adversarial feedback details to obtain first adjusted joint state; perform adversarial learning based on the first adjusted joint state and the second historical joint state to obtain second adversarial feedback details; perform state adjustment on the first adjusted joint state based on the second adversarial feedback details to obtain second adjusted joint state, and when the error between the second adjusted joint state and the second historical joint state is lower than a preset error, determine the second adversarial feedback details as the adversarial feedback details.

[0017] In the above scheme, the adjustment module is also used to classify the first predicted joint state to obtain a first category score of the first predicted joint state, and classify the second historical joint state to obtain a second category score of the second historical joint state; determine the first loss value of the adversarial learning based on the first category score and the second category score; determine the state difference between the first predicted joint state and the second historical joint state based on the first loss value, and obtain the first adversarial feedback details based on the state difference.

[0018] In the above scheme, the adjustment module is also used to determine a second loss and a third loss based on the first predicted joint state and the second historical joint state, wherein the second loss is used to characterize the similarity between the first predicted joint state and the second historical joint state, and the third loss is used to characterize the difference between the data space corresponding to the first predicted joint state and the data space corresponding to the second historical joint state; based on the second loss and the third loss, adversarial learning is performed on the first predicted joint state and the second historical joint state.

[0019] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0020] A memory for storing computer executable instructions or computer programs;

[0021] The processor is used to implement the robot control method provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0022] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the robot control method provided in the embodiment of the present application when executed by a processor.

[0023] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the robot control method provided in the embodiment of the present application is implemented.

[0024] The embodiments of the present application have the following beneficial effects:

[0025] Through the above method, by obtaining the first joint state of the robot at the first moment and the environmental image collected at the first moment, the first joint feature between the first joint state and the environmental image is determined to learn the joint information of the two, so as to make an initial prediction of the joint state of the robotic arm at the second moment based on the first joint feature to obtain the second joint state, and then, adjust the state of the second joint state based on the adversarial feedback details determined by adversarial learning (that is, secondary prediction) to obtain a more accurate third joint state, so as to control the robot's robotic arm to swing based on the third joint state. This method obtains the correlation between the environmental image and the first joint state through the first joint feature, and further adjusts the state through the adversarial feedback details, which can be better generalized to the data of the robot in different scenarios, thereby improving the accuracy of predicting the future joint state of the robot's robotic arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a structural diagram of the robot control system architecture provided by an embodiment of the present application;

[0027] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0028] Figure 3A is a flow chart of a robot control method provided in an embodiment of the present application;

[0029] Figure 3B is a flowchart of a method for determining a first joint feature provided in an embodiment of the present application;

[0030] Figure 3C is a flow chart of a method for determining a third joint state provided in an embodiment of the present application;

[0031] Figure 3D is a flowchart of a method for determining countermeasure feedback details provided in an embodiment of the present application;

[0032] Figure 4 is a schematic diagram of a robot provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of the structure of the robot control model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0035] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0037] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0038] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0039] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0040] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0041] 1) Latent space refers to the potential representation of data learned by the encoder. This latent space is a mapping of high-dimensional data in a low-dimensional space, which captures the essential characteristics of the data and enables the data to be effectively represented and operated in a low-dimensional space. The latent space provides a low-dimensional representation of the data, which helps reduce the complexity of the data and makes the data easier to process. Each point in the latent space corresponds to a latent variable of the data, which is usually assumed to follow a certain probability distribution, such as a Gaussian distribution. : The latent space is usually continuous, which means that close points are also close in the latent space, which helps to generate coherent data samples.

[0042] 2) Original space refers to the space where the input data exists in the variational autoencoder. This space contains all possible input data points, which are used to learn the parameters of the encoder and decoder during the training process.

[0043] Embodiments of the present application provide a robot control method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the control accuracy of the robot.

[0044] See also Figure 1 , Figure 1 It is a schematic diagram of the architecture of the robot control system 100 provided in an embodiment of the present application. In order to support a robot control application, the terminal 401 is connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two. Here, the terminal 401 can be a terminal set on the robot, or a terminal that communicates with the robot in real time.

[0045] The terminal 401 is used to respond to the control instruction for the robot and send a robot control request to the server 200. The server 200 is used to obtain the first joint state of the robot's mechanical arm at the first moment, and the environmental image collected by the robot for the current environment at the first moment; determine the first joint feature between the environmental image and the first joint state, and predict the joint state of the mechanical arm at the second moment based on the first joint feature to obtain the second joint state; determine the adversarial feedback details, and adjust the state of the second joint state based on the adversarial feedback details to obtain the third joint state; the server 200 can send the third joint state to the terminal 401, so that the terminal 401 controls the mechanical arm to swing to the third joint state when the second moment arrives.

[0046] In some embodiments, the server 200 may directly send the third joint state obtained in response to the robot control request to the terminal 401 ; the terminal 401 may also actively obtain the third joint state from the server 200 .

[0047] In some embodiments, terminal 401 can be implemented as various types of terminals such as a robot terminal, a laptop computer, a tablet computer, a desktop computer, a set-top box, a smart phone, a smart speaker, a smart watch, a smart TV, a car terminal, etc., and can also be implemented as a server.

[0048] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0049] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device 400 provided in an embodiment of the present application, Figure 2 The electronic device 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .

[0050] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0051] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0052] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0053] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0054] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0055] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0056] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;

[0057] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0058] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0059] In some embodiments, the robot control device provided in the embodiments of the present application can be implemented in a software manner. Figure 2The robot control device 455 stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, including the following software modules: acquisition module 4551, prediction module 4552, adjustment module 4553, control module 4554, these modules are logical, so according to the functions implemented, they can be arbitrarily combined or further split. The functions of each module will be explained below.

[0060] In other embodiments, the robot control device provided in the embodiments of the present application can be implemented in hardware. As an example, the robot control device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the robot control method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other electronic components.

[0061] The robot control method provided by the embodiment of the present application will be described below in conjunction with the accompanying drawings. As mentioned above, the electronic device that implements the robot control method of the embodiment of the present application can be a terminal 401, a server 200, or a combination of the two. Therefore, the execution subject of each step will not be repeatedly described below.

[0062] The robot control method of the embodiment of the present application is described by taking the execution subject as the terminal 401 as an example. Figure 3A , Figure 3A is a flow chart of the robot control method provided in the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained.

[0063] In step 101, a first joint state of a robot's mechanical arm at a first moment and an environment image collected by the robot for a current environment at the first moment are obtained.

[0064] In the embodiment of the present application, the robot has a movable mechanical arm. The type of robot may be an industrial robot, which is used to perform industrial tasks such as handling, assembly, packaging, welding, and sorting through a mechanical arm; the type of robot may also be a service robot, which is used for medical assistance (such as surgical assistance, rehabilitation assistance, etc.), household tasks (such as cleaning, nursing, etc.), and catering services (such as food preparation, meal delivery, etc.); the type of robot may also be any other robot that can use the function of a mechanical arm, such as a detection robot, a rescue robot, a bionic robot, etc., which is not specifically limited here. It is understandable that the mechanical arms of different types of robots have different degrees of freedom in design, ranging from simple two degrees of freedom to highly complex seven or more degrees of freedom, which enable the robot to perform simple repetitive tasks or highly complex and precise operations.

[0065] The robot can also be equipped with a camera or other visual sensor to capture visual information in the surrounding environment (e.g., obtain visual information through images). The robot can identify and perceive specific objects, landmarks, obstacles, and other information in the environment by analyzing the collected images.

[0066] Next, see Figure 4 , Figure 4 1 is a schematic diagram of a robot provided in an embodiment of the present application, and a sorting robot is used for explanation. The sorting robot is equipped with a head camera 110 and a wrist camera 120. The head camera 110 and the wrist camera 120 can collect environmental images of the current environment in which the robot is located. The robotic arm of the sorting robot has movable shoulder joints 130, elbow joints 140 and wrist joints 150. The sorting robot can control the position of each joint to control the robotic arm to perform the task of sorting items.

[0067] Sensors (such as position sensors, velocity sensors, force sensors, accelerometers, etc.) are configured at each joint node of the robot, and the robot can use these sensors to collect the joint status of each joint of the robotic arm.

[0068] Here, the joint state of a robot generally refers to the motion state and position information of the joints of the robotic arm at a specific moment. As an example, the joint state may include the following information: the specific position of the joint in space, which may be an angle (for a rotary joint) or a linear displacement (for a linear joint); speed, which is used to describe the speed of joint movement, and is usually expressed in degrees per second (for a rotary joint) or millimeters per second (for a linear joint); acceleration, which indicates how fast the joint speed changes, and is usually expressed in degrees per second squared or millimeters per second squared; force or torque, which is used to reflect the magnitude of the force or torque applied to or received by the joint; joint angle, which is the rotation angle of the joint axis relative to a reference position for a rotary joint, such as the absolute rotation angle of a motor that controls the rotation of the joint; joint displacement, which is the displacement of the joint along its axis for a linear joint; joint posture, which describes the posture of the joint relative to a reference position, and includes a combination of angles and displacements; electrical parameters, which include information such as motor current and voltage, which can be used to monitor the working status of the motor; encoder readings, which are position feedback provided by the encoder and are used to precisely control the joint position.

[0069] In actual implementation, when it is necessary to control the robot to perform a task, the first joint state of the robot's mechanical arm at the first moment and the environmental image captured by the robot's camera for the current environment at the first moment can be obtained. Here, the first moment can be understood as the current moment; the first joint state can include joint state information of one or more joints of the mechanical arm, and the first joint state information at least includes position information of the joint, such as joint angle, joint displacement and other position information; the environmental image is an image of the direction in which the robot's camera is facing, for example, the head camera of the sorting robot can capture an image including the object to be sorted in the direction in which it is facing.

[0070] In actual implementation, a trained robot control model can be configured in the robot's terminal 401. By inputting the environmental image captured by the robot's camera and the first joint state captured by the sensor into the robot control model, the robot control model can be used to adopt the robot control method of the embodiment of the present application to determine the third joint state for controlling the movement of the robot arm, thereby achieving control of the robot arm.

[0071] As an example, Figure 5 For a schematic diagram of the structure of the robot control model provided in the embodiment of the present application, see Figure 5 The robot control model can be composed of a variational autoencoder (VAE) and a generative adversarial network (GAN), wherein the variational autoencoder includes an encoder and a decoder, and the generative adversarial network includes a generator and a discriminator.

[0072] As an example, the model architecture of the robot control model can be built using a variational autoencoder and a generative adversarial network. The encoder and decoder in the variational autoencoder and the generator and discriminator in the generative adversarial network can be built based on a deep learning network, such as the Transformer network structure.

[0073] Continue reading Figure 3A , and the description will continue with the above step 101.

[0074] In step 102, a first joint feature between the environment image and the first joint state is determined, and the joint state of the robot arm at a second moment is predicted based on the first joint feature to obtain a second joint state.

[0075] Here, the second moment is later than the first moment, and the first moment is the current moment. The second moment is the next moment later than the current moment determined based on the control frequency of the robotic arm. Taking the control frequency of the robotic arm once per second when the robot is working as an example, if the first moment is the current moment XX:XX:12, the second moment can be XX:XX:13.

[0076] In some embodiments, in combination Figure 5 The robot control model shown in Figure 3B , “determining the first joint feature between the environment image and the first joint state” in step 102 can be implemented through steps 1021 to 1023.

[0077] In step 1021, feature extraction is performed on the environment image to obtain image features, and feature extraction is performed on the first joint state to obtain joint state features of the first joint state.

[0078] In actual implementation, after obtaining the environment image and the first joint state at the first moment, the environment image and the first joint state can be used as inputs of the encoder in the variational autoencoder, and the encoder can be used to extract features of the environment image and the first joint state respectively.

[0079] As an example, after the encoder of the variational autoencoder receives the environment image as input, it can extract the features of the environment image through a series of convolutional layers built on the Transformer network. These convolutional layers can capture the local features and spatial hierarchical structures in the environment image. Image features can include: visual features. The basic visual features of the environment image, such as texture, color, shape, and edge, will be extracted by the encoder. These visual features can help the encoder understand the objects, scenes, and backgrounds in the image; spatial relationships. The spatial relationships between objects in the environment image, such as position, direction, and distance, can help the robot control model understand the overall layout of the environment image; contextual information. The contextual information in the environment image, such as the interaction between the object and the surrounding environment, if there is a robotic arm object in an environment image, the encoder will notice the relationship between the robotic arm object and the surrounding tools or objects; dynamic information. If the environment image is a sequence and the sequence is continuous, the encoder will extract the dynamic changes between the environment images, such as the trajectory of the robotic arm or the position change of the object; semantic information. The semantic information in the environment image, such as the category, behavior, and event of the object, will also be extracted by the encoder, especially if this information is related to the task of the robotic arm.

[0080] As an example, after receiving the first joint state as input, the encoder of the variational autoencoder can extract the joint state features in the first joint state through a series of fully connected layers or recurrent layers constructed by the Transformer network. When extracting features from the first joint state, the encoder usually focuses on key information that can reflect joint movement and behavior, such as position information, which enables the robot control model to better understand the current state of the robot arm and predict future behavior. Joint state features may include: position features, such as joint angle and joint displacement. For a rotary joint, the joint angle is the angle of the joint relative to a reference position or the absolute rotation angle of the motor controlling the joint. For a linear joint, the joint displacement is the displacement of the joint along its axis; velocity features, such as angular velocity and linear velocity. For a rotary joint, the angular velocity is the magnitude of the joint angular velocity, and for a linear joint, the linear velocity is the velocity of the joint displacement; acceleration features, such as angular acceleration and linear acceleration. For a rotary joint, the angular acceleration is the magnitude of the joint angular acceleration, and for a linear joint, the linear acceleration is the acceleration of the joint displacement; force or torque features, such as joint force and joint torque. The joint force reflects the magnitude of the force applied by the joint, and the joint torque reflects the magnitude of the torque applied by the joint; dynamic features, such as motion smoothness and motion stability. Motion smoothness can reflect whether the joint motion is smooth, which can be measured by the rate of change of acceleration, while motion stability can reflect whether the joint motion remains on the desired trajectory; time features, such as the periodic characteristics of joint motion, such as the period of swinging motion, time series features, etc. In the feature extraction process, some derived features are also considered, such as features obtained by differential and integral operations, and features obtained by statistical analysis (such as mean, standard deviation, variance, etc.). These features can be used in a variety of control strategies, including model predictive control, feedback control, adaptive control and intelligent control (such as deep learning, reinforcement learning, etc.), and which joint state features are selected depends on the specific control task, the design of the robot arm and the required control accuracy. Usually, feature selection is an iterative process that needs to be adjusted and optimized according to the control effect.

[0081] In step 1022, the image features and the joint state features are mapped to the latent space to obtain the first mapping features of the image features in the latent space, the second mapping features of the joint state features in the latent space, and the intrinsic relationship between the image features and the joint state features.

[0082] In actual implementation, after receiving the environment image and the first joint state, the encoder of the variational autoencoder will extract features of the environment image and the first joint state respectively by way of step 1021, and the format of the obtained joint state features is a format compatible with the image features, for example, both are feature vectors of the same dimension or matrices of a fixed size. Then, the joint state features and image features in the integrated format will be sent to different neural network layers of the encoder to perform feature mapping on the image features and the joint state features.

[0083] Here, latent space refers to the potential representation of data learned by the encoder network of the variational autoencoder. This latent space is a mapping of high-dimensional data in low-dimensional space, which can capture the essential characteristics of the data and enable the data to be effectively represented and operated in low-dimensional space. Specifically, the latent space provides a low-dimensional representation of the data, which can reduce the complexity of the data and make the data easier to process. Each point in the latent space corresponds to a latent variable of the data, which is assumed to follow certain probability distributions. In addition, the latent space is usually continuous, which means that close points are also close in the latent space, which helps to generate coherent data.

[0084] In actual implementation, in order to extract related information from the environment image and the first joint state and capture significant and useful information in the two types of data, the neural network layer of the encoder can map the image features and the joint state features to the latent space to obtain the first mapping features of the image features in the latent space and the second mapping features of the joint state features in the latent space. The intrinsic correlation between the image features and the joint state features can be obtained through the representation of the first mapping features in the latent space and the representation of the second mapping features in the latent space.

[0085] Here, the intrinsic association may include: spatial correspondence, the correspondence between the spatial position in the environment image and the first joint state, for example, a specific area in the environment image may correspond to a specific joint movement of the robotic arm; temporal relationship, if the input data contains time series information, the encoder can extract the temporal relationship between the environment image and the first joint state, for example, a change in the first joint state may indicate an upcoming change in certain features in the environment image; motion trajectory relationship, the encoder can extract the relationship between the motion trajectory of the robotic arm and the motion trajectory of the object in the image; posture and perspective relationship, the encoder can extract how the posture change of the robotic arm affects the camera perspective, and how this change is reflected in the environment image; feature mapping relationship, the encoder can map features in the environment image to adjustments to the first joint state in order to achieve specific task goals.

[0086] In the above manner, the variational autoencoder of the robot control model learns the data distribution of the environment image and the first joint state in the latent space, which can make the model better generalize to new data and improve the generalization ability of the model.

[0087] In step 1023, based on the intrinsic association, the first mapping feature and the second mapping feature are fused to obtain a first joint feature between the environment image and the first joint state.

[0088] In actual implementation, the encoder of the variational autoencoder can combine the first mapping feature and the second mapping feature based on the intrinsic correlation through a shared layer or a specific fusion layer, and output a latent vector. This latent vector is a low-dimensional representation of the intrinsic correlation between the environment image and the first joint state in the latent space. The latent vector can contain the joint information of the environment image and the first key state. Here, the latent vector is the first joint feature, and the first joint feature is the comprehensive representation obtained by the encoder that can simultaneously characterize the environment image and the first joint state. Through feature fusion, the robot control model can understand the interaction between the environment image and the first joint state based on the first joint feature.

[0089] In the above manner, by processing the features of the environment image and the first joint state through the encoder of the variational autoencoder, the robot control model can learn the correlation between the visual features of the environment image and the first joint state. For example, in the robot sorting task, the direct relationship between the object position in the environment image and the joint of the robotic arm is obtained, and the encoder can reveal the dependence of certain features in the environment image on the first joint state. For example, the movement of a joint of the robotic arm may cause a change in the camera's perspective, thereby affecting the input of the environment image; the encoder can also enable the robot control model to learn how the environment image and the first joint state change synergistically through the first joint feature.

[0090] The description will be continued with the above step 102.

[0091] In some embodiments, "predicting the joint state of the robotic arm at the second moment based on the first joint feature to obtain the second joint state" in step 102 can be achieved in the following way: mapping the first joint feature from the latent space to the original space where the first joint state is located, and obtaining the third mapping feature of the first joint feature in the original space; predicting the joint state of the robotic arm at the second moment based on the third mapping feature to obtain the second joint state.

[0092] In actual implementation, the decoder of the variational autoencoder is responsible for mapping the features in the latent space back to the original space, generating data similar to the original input.

[0093] Here, the original space refers to the space where the input data of the variational autoencoder (here refers to the first joint state) exists. The original space contains all possible input data points; the original space is defined by the dimension of the input data; the data type in the original space depends on the type of the input data; the data points in the original space follow a certain distribution, which is usually unknown, and one of the goals of the variational autoencoder is to approximate this distribution through learning; the data in the original space is usually represented as high-dimensional vectors, which contain all the characteristics of the data.

[0094] In actual implementation, the decoder receives the first joint feature of the latent space from the encoder, maps the features of the first joint feature in the latent space back to the original space where the first joint state is located, that is, converts the representation of the latent space (the first joint feature) into a feature (the third mapping feature) with the same dimension as the original data (the joint state feature of the first joint state). This mapping process can be achieved through a fully connected layer or a convolutional layer. Then, the joint state of the robot at the second moment is predicted through the third mapping feature to obtain the second joint state.

[0095] Here, the second joint state is the result of the initial prediction made by the variational autoencoder for the position of the robot arm at the second moment.

[0096] Continue reading Figure 3A , and continue with the above step 102 for explanation.

[0097] In step 103, the antagonistic feedback details are determined, and the state of the second joint is adjusted based on the antagonistic feedback details to obtain a third joint state.

[0098] In actual implementation, the generator of the generative adversarial network can generate a more refined joint state from the second joint state output by the variational autoencoder based on the adversarial feedback details to obtain the third joint state. Here, the adversarial feedback details come from the data distribution learned by the generator or the feedback obtained from the discriminator during the training of the robot control model.

[0099] In some embodiments, see Figure 3C , "adjusting the second joint state based on the confrontation feedback details to obtain the third joint state" in step 103 can be implemented through steps 1031 to 1033.

[0100] In step 1031, the joint state features of the second joint state are refined to obtain refined joint state features of the second joint state.

[0101] In actual implementation, after the generator receives the preliminary predicted second joint state, it can identify the joint state characteristics of the second joint state, and enhance some key features in the joint state characteristics of the second joint state to realize the process of feature refinement of the joint state characteristics of the second joint state, and obtain the refined joint state characteristics of the second joint state.

[0102] In step 1032, the adversarial feedback details are added to the refined joint state features to obtain enhanced joint state features.

[0103] In actual implementation, before adding details to the refined joint state features, the generator can also use a convolutional layer or a fully connected layer to map the refined joint state features to a higher dimensional space, and perform scaling, translation or other nonlinear transformations to produce more accurate joint position information.

[0104] In actual implementation, the generator can add adversarial feedback details to the refined joint state features on the basis of the refined joint state features to achieve feature enhancement and obtain enhanced joint state features.

[0105] In step 1033, feature adjustment is performed on the enhanced joint state feature to obtain an adjusted joint state feature, and a third joint state is generated based on the adjusted joint state feature.

[0106] In actual implementation, the generator can also adjust the features of the enhanced joint state features, that is, optimize the state representation embodied by the enhanced joint state features to make it more consistent with the distribution of the actual joint state. The adjusted joint state features obtained after feature adjustment can be understood as the features of the joint state obtained by the generator based on the second joint state through secondary prediction, that is, by decoding the enhanced joint state features, the third joint state predicted by the generator can be obtained. Through the deep processing of the second joint state by the generator, the third joint state obtained can be more accurate, reliable and practical.

[0107] During the above processing, the generator usually introduces noise to enhance the diversity of generation, thereby exploring and obtaining richer joint state information.

[0108] Continue reading Figure 3A , and continue with the above step 103 for explanation.

[0109] In step 104, when the second moment arrives, the robot arm is controlled to swing to the third joint state.

[0110] In actual implementation, the third joint state is the predicted state of the robot's robotic arm at the second moment. Therefore, after obtaining the third joint state, the swing path of the robotic arm from the first moment to the second moment can be planned according to the state difference between the third joint state and the first joint state (for example, the rotation angle of the motor controlling the joint, etc.), and then, the robotic arm is controlled to swing based on the planned swing path, so that the robotic arm can be in the third joint state when it arrives at the second moment.

[0111] Through the above method, by obtaining the first joint state of the robot at the first moment and the environmental image collected at the first moment, the first joint feature between the first joint state and the environmental image is determined to learn the joint information of the two, so as to make an initial prediction of the joint state of the robotic arm at the second moment based on the first joint feature to obtain the second joint state, and then, adjust the state of the second joint state based on the adversarial feedback details determined by adversarial learning (that is, secondary prediction) to obtain a more accurate third joint state, so as to control the robot's robotic arm to swing based on the third joint state. This method obtains the correlation between the environmental image and the first joint state through the first joint feature, and further adjusts the state through the adversarial feedback details, which can be better generalized to the data of the robot in different scenarios, thereby improving the accuracy of predicting the future joint state of the robot's robotic arm.

[0112] Continue reading Figure 3A , continue to explain the "determining the adversarial feedback details" in the above step 103.

[0113] In some embodiments, the details of the adversarial feedback in step 103 may be obtained during the training of the robot control model, see Figure 3D , determining the details of the adversarial feedback can be achieved through steps 1034 to 1036.

[0114] In step 1034, a first historical joint state of the robot's mechanical arm at a first historical moment, a second historical joint state of the mechanical arm at a second historical moment, and a historical environment image collected by the robot for the historical environment at the first historical moment are obtained.

[0115] In actual implementation, firstly, training samples used for training the robot control model are prepared. Here, the training samples may use the historical joint states and historical environment images of the robot in a historical period.

[0116] As an example, a set of training samples for training a robot control model may include: a first historical joint state of a robot's mechanical arm corresponding to a first historical moment, a second historical joint state corresponding to a second historical moment, and a historical environment image captured by the robot's head camera and wrist camera for the historical environment at the first historical moment. Here, the second historical moment is later than the first historical moment. During training, the robot control model may predict the joint state of the mechanical arm at the second historical moment based on the first historical joint state and the historical environment image, and train the model based on the actual state at the second historical moment (i.e., the second historical joint state) and the predicted joint state.

[0117] In step 1035, a second joint feature between the historical environment image and the first historical joint state is determined, and the joint state of the robot arm at the second historical moment is predicted based on the second joint feature to obtain a first predicted joint state.

[0118] In actual implementation, see Figure 5 , the historical environment image and the first historical joint state are used as the input of the encoder in the variational autoencoder, and the encoder extracts features of the historical environment image and the first historical joint state respectively to obtain historical image features and first historical joint state features, and then maps the historical image features and the first historical joint state features to the latent space to obtain the first historical mapping features of the historical image features in the latent space, the second historical mapping features of the first historical joint state features in the latent space, and the intrinsic correlation between the historical image features and the first historical joint state features, and based on the intrinsic correlation, the first historical mapping features and the second historical mapping features are feature fused to obtain the second joint feature between the historical environment image and the first historical joint state.

[0119] In actual implementation, when the encoder combines the first historical mapping feature and the second historical mapping feature based on the intrinsic correlation, it outputs the second joint feature by learning the potential distribution between the two.

[0120] As an example, the encoder may obtain the second joint feature by the following formula:

[0121] z~q(z|x,y)=N(μ(μ,y),σ 2 (x,y)

[0122] Where x represents the historical environment image; q represents the first historical joint state; z represents the second joint feature (i.e., latent vector); z~q(z|x,y) describes that the encoder compresses the historical environment image x and the first historical joint state y into a low-dimensional latent vector z; μ(x,y) represents the mean between the features of the historical environment image x and the first historical joint state y; σ2 (x,y) represents the calculation of the variance between the features of the historical environment image x and the first historical joint state y.

[0123] In the above formula, the process described by z~q(z|x,y) is mainly to explore and express the potential relationship between the historical environment image x and the first historical joint state y. By taking the historical environment image x and the first historical joint state y as input, the encoder tries to find the intrinsic relationship between them and compress this information into a low-dimensional space, which can help the model understand the interaction between the two, for example, how the position of the robotic arm affects or is affected by certain features in the image. The mean μ(x,y) represents a point in the latent space, which captures the central trend or typical characteristics of the input data (historical environment image x and the first historical joint state y). In the variational autoencoder, the mean can be regarded as the central position of the latent variable, that is, it represents the most likely latent representation of the historical environment image and the first historical joint state. Variance σ 2 (x,y) represents the uncertainty or range of the latent variable, which describes the diversity of the input data or the dispersion of the latent representation. In practical applications, large variance indicates that the encoder has high uncertainty about the given input, either because the input data itself has large variations or because there are multiple possible representations in the latent space.

[0124] In general, the encoder captures the statistical characteristics of the input data (historical environment image x and the first historical joint state y) by calculating the mean and variance, and uses this to express the complex relationship between them. The mean and variance together define a probability distribution, and the latent vector z is sampled from this distribution for subsequent decoding and generation processes.

[0125] Subsequently, the decoder of the variational autoencoder is responsible for mapping the features in the latent space back to the original space, generating data similar to the original input.

[0126] As an example, the decoder receives a point in the latent space from the encoder (i.e., the latent vector z), maps the point in the latent space back to the original space (the data space where the first historical joint state is located), and then generates a probability distribution that can generate new data (the first predicted joint state).

[0127] As an example, the first predicted joint state can be obtained by the following formula:

[0128]

[0129] in, represents the first predicted joint state, and p(y|z) represents the probability distribution determined based on the potential vector z and the first historical joint state y.

[0130] The description continues with the above step 1035.

[0131] In step 1036, adversarial learning is performed based on the first predicted joint state and the second historical joint state to obtain adversarial feedback details.

[0132] In some embodiments, step 1036 can be implemented in the following manner: performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details; performing state adjustment on the first predicted joint state based on the first adversarial feedback details to obtain a first adjusted joint state; performing adversarial learning based on the first adjusted joint state and the second historical joint state to obtain second adversarial feedback details; performing state adjustment on the first adjusted joint state based on the second adversarial feedback details to obtain a second adjusted joint state, and when the error between the second adjusted joint state and the second historical joint state is lower than a preset error, determining the second adversarial feedback details as the adversarial feedback details.

[0133] In actual implementation, the adversarial learning of the generative adversarial network is a game process between the generator and the discriminator. The goal of this game is that the generator learns to generate more realistic joint states, while the discriminator learns to distinguish between the joint states generated by the generator and the real joint states. Specifically, the generator receives the first predicted joint state predicted by the decoder, and tries to generate joint states with data distribution similar to the real data (the second historical joint state). The training goal of the generator is to deceive the discriminator so that it cannot distinguish between the generated data and the real data; the discriminator receives the joint state generated by the generator and the real data (the second historical joint state), classifies the generated data and the real data, and judges the score of the second historical joint state classified as real and the score of the generated joint state classified as non-real. The training goal of the discriminator is to improve the ability to distinguish between real data and generated data.

[0134] In actual implementation, the training process of the adversarial learning network is an iterative process, which may include the following processes: initializing the parameters of the generator and the discriminator; under the condition of a fixed discriminator, obtaining the first adversarial feedback details and feeding them back to the generator through the judgment of the discriminator on the first predicted joint state and the second historical joint state; updating the parameters of the generator based on the judgment result of the discriminator, and the generator with updated parameters may adjust the state of the first predicted joint state based on the first adversarial feedback details to obtain the first adjusted joint state, and give the first adjusted joint state to the discriminator; under the condition of a fixed generator, updating the parameters of the discriminator according to the generation result of the generator, and making the discriminator make authenticity judgment on the first adjusted joint state and the second historical joint state, and obtaining the second adversarial feedback details according to the judgment; fixing the discriminator again, updating the parameters of the generator, and making the generator adjust the state of the first adjusted joint state based on the second adversarial feedback details to obtain the second adjusted joint state; repeating the above process iteratively until the error between the second adjusted joint state and the second historical joint state is lower than the preset error, and the second adversarial feedback details are determined as the adversarial feedback details. As iterative training progresses, the confrontation between the generator and the discriminator reaches a dynamic balance, the joint states generated by the generator become more and more realistic, and the discriminator becomes more and more adept at distinguishing between real and generated data.

[0135] Here, when training a generative adversarial network, adversarial feedback details refer to the specific information about the interaction between the generator and the discriminator during the training process. These feedback details are crucial to understanding the training dynamics of the adversarial generative network and improving model performance.

[0136] In some embodiments, "performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details" can be achieved in the following ways: classifying the first predicted joint state to obtain a first category score for the first predicted joint state, and classifying the second historical joint state to obtain a second category score for the second historical joint state; determining a first loss value for adversarial learning based on the first category score and the second category score; determining a state difference between the first predicted joint state and the second historical joint state based on the first loss value, and obtaining first adversarial feedback details based on the state difference.

[0137] In actual implementation, adversarial feedback details refer to the specific information of the interaction between the generator and the discriminator during the training process. During the training process, the generator will receive feedback from the discriminator to understand how similar its generated samples are to real samples. When the discriminator generates adversarial feedback details, it is usually determined by the loss situation.

[0138] As an example, after receiving the first predicted joint state, the discriminator will classify the first predicted joint state and the second historical joint state, that is, classify the first predicted joint state as unreal and classify the second historical joint state as real. The purpose of classification is to distinguish between real data and generated data. When the discriminator performs classification, it calculates the category score of the data classification. For example, classifying the first predicted joint state can obtain the first category score of the first predicted joint state, and classifying the second historical joint state can obtain the second category score of the second historical joint state.

[0139] The first loss value of adversarial learning is calculated by the following formula:

[0140] Γ GAN =E[logD(y)]+E[log(1-D(y′))]

[0141] Where D(y) represents the score of the second category; D(y′) represents the score of the first category; Γ GAN represents the first loss value; E[logD(y)] represents the expected output of the discriminator for the real data (the second historical joint state), which measures the prediction quality of the discriminator for the second historical joint state. The larger the value of logD(y), the more reliable the discriminator's classification of the second historical joint state; E[log (1-D(y′))] represents the expected output of the discriminator for the generated data (the first predicted joint state), which measures the accuracy of the discriminator's judgment on the first predicted joint state. Ideally, the discriminator should correctly classify the first predicted joint state as non-real, that is, the value of D(y′) tends to 0. Therefore, the larger the log(1-D(y′)), the more accurate the discriminator's recognition of the first predicted joint state.

[0142] Then, the first loss value Γ of adversarial learning is determined by the sum of the first category score and the second category score. GAN ; Based on the first loss value, determine the state difference between the first predicted joint state and the second historical joint state, and obtain the first adversarial feedback details based on the state difference, that is, the discriminator can obtain information such as parameters used for gradient update according to the first loss value as the first adversarial feedback details.

[0143] In some embodiments, "performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details" can also be achieved in the following way: based on the first predicted joint state and the second historical joint state, determine the second loss and the third loss, wherein the second loss is used to characterize the similarity between the first predicted joint state and the second historical joint state, and the third loss is used to characterize the difference between the data space corresponding to the first predicted joint state and the data space corresponding to the second historical joint state; based on the second loss and the third loss, perform adversarial learning on the first predicted joint state and the second historical joint state.

[0144] In actual implementation, after the variational autoencoder predicts the first predicted joint state, it will also calculate its prediction loss, which includes the second loss and the third loss. Among them, the second loss is the reconstruction loss of the variational autoencoder, which is used to characterize the similarity between the first predicted joint state and the second historical joint state; the third loss is the KL divergence loss of the variational autoencoder, which is used to characterize the difference between the data space corresponding to the first predicted joint state and the data space corresponding to the second historical joint state.

[0145] As an example, the prediction loss can be calculated by the following formula:

[0146] μ VAE =E q(z|x,y) [logp(y|z)]-D KL (q(z|x,y)||p(z))

[0147] Among them, Γ VAE represents the prediction loss; p(y|z) represents the first predicted joint state; q(z|x,y) represents the potential distribution of the potential vector z; p(z) represents the prior distribution of the potential vector z; E q(z|x,y) [logp(y|z)] represents the reconstruction loss; D KL (q(z|x,y)||p(z)) represents the KL divergence loss, which can measure the difference between the potential distribution and the prior distribution. The smaller the loss value, the closer the learned potential distribution is to the set prior distribution.

[0148] In actual implementation, the second loss, the third loss and the first loss can be combined to reversely adjust the parameters of the robot control model, that is, to train the robot control model.

[0149] As an example, the fusion of losses can be achieved by the following formula:

[0150] Γ=Γ VAE +λΓ GAN

[0151] Where Γ represents the total loss of the robot control model; Γ VAE represents the prediction loss (i.e., the sum of the second loss and the third loss); Γ GAN Represents the first loss; λ is a hyperparameter that weighs the two loss terms and is used to balance the generation quality and reconstruction accuracy of the model.

[0152] In actual implementation, the total loss obtained can be used to reversely adjust the relevant parameters of the entire model. Then, based on the above training process, the robot control model is iteratively trained again using other training samples until the total loss meets the preset loss value or the number of iterations reaches the preset number. The iterative training is terminated to obtain a trained robot control model.

[0153] In the above way, by combining the generative adversarial network and the variational autoencoder in the robot control model, the variational autoencoder can be used to generate a latent vector that conforms to the probability distribution, learn the correlation between the environmental image and the joint state, and improve the prediction accuracy of the future joint state. By learning the data distribution of the latent space, the model can be better generalized to new data; and the adversarial learning of the generative adversarial network can improve the accuracy of the generated joint state. By combining these two network structures, the robot can regenerate a more accurate and robust control strategy in visual motion control tasks, and once the robot control model is trained, the reasoning process is very fast, which is suitable for real-time control of robots and robot motion control tasks in various complex environments.

[0154] In a specific embodiment, the robot control system of the embodiment of the present application may include the following modules:

[0155] Input module, used for visual information data and joint state input of the robotic arm.

[0156] Specifically, the robot's head camera and wrist camera can collect environmental images in real time, which is used for the robot to perceive the surrounding environmental conditions; the sensors at the joints can collect the first joint states of the corresponding joints in real time as state feedback for the robot's motion control.

[0157] Model processing module, the robot control model carried by the robot includes variational autoencoder and adversarial learning network.

[0158] Specifically, the encoder of the variational autoencoder can receive the environment image x and the first joint state y input by the input module, and encode them into a latent vector z in the latent space; the decoder can decode and generate preliminary robot arm joint control instructions (i.e., the second joint state mentioned above) based on the latent vector z. The generator of the generative adversarial network receives preliminary robot arm joint control instructions (i.e., the second joint state mentioned above) and generates more refined joint control instructions (i.e., the third joint state mentioned above).

[0159] The control module is used to control the corresponding joints of the robot's mechanical arm to perform corresponding operations according to the generated joint control instructions (i.e. the above-mentioned third joint state).

[0160] In another specific embodiment, the training process of the robot control model may include the following process:

[0161] First, data preparation involves collecting a large number of training samples, including the robot’s historical environment images and the corresponding historical joint states.

[0162] Then, the variational autoencoder is trained to learn the potential vector z between the input historical environment image and the historical joint state by minimizing the reconstruction loss and the KL divergence loss. The calculation process of the potential vector z and the calculation process of the two losses can refer to the above-mentioned related embodiments, which will not be repeated here.

[0163] Furthermore, for the training of the adversarial learning network, the goal is to optimize the joint state generated by the generator to make it closer to the real joint state. The generator receives the joint control instruction (i.e., the first predicted joint state) initially generated by the decoder as input, and generates a refined joint control instruction (the second predicted joint state). The discriminator receives the real joint state and the joint control instruction generated by the generator at the same time, and outputs two classification values, which respectively represent the probabilities of the real joint state and the joint control instruction generated by the generator. Based on the first loss calculation formula in the above-mentioned related embodiment, the first loss is calculated.

[0164] Finally, the reconstruction loss, KL divergence loss and the first loss are combined to obtain the total loss, and the parameters of the robot control model are adjusted based on the total loss.

[0165] It should be noted that during the training process, the parameters of the variational autoencoder and the adversarial learning network can be alternately optimized so that the generator can generate more realistic joint control instructions.

[0166] The following is a description of an exemplary structure of a robot control device 455 provided in an embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the robot control device 455 of the memory 450 may include:

[0167] The acquisition module 4551 is used to acquire the first joint state of the robot's mechanical arm at the first moment, and the environment image collected by the robot for the current environment at the first moment.

[0168] The prediction module 4552 is used to determine a first joint feature between the environment image and the first joint state, and predict the joint state of the robot arm at the second moment based on the first joint feature to obtain a second joint state.

[0169] The adjustment module 4553 is used to determine the details of the antagonistic feedback, and adjust the second joint state based on the details of the antagonistic feedback to obtain the third joint state.

[0170] The control module 4554 is used to control the robot arm to swing to the third joint state when the second moment arrives.

[0171] In some embodiments, the prediction module 4552 is also used to extract features from the environment image to obtain image features, and to extract features from the first joint state to obtain joint state features of the first joint state; map the image features and the joint state features to the latent space to obtain first mapping features of the image features in the latent space, second mapping features of the joint state features in the latent space, and the intrinsic correlation between the image features and the joint state features; based on the intrinsic correlation, perform feature fusion on the first mapping features and the second mapping features to obtain a first joint feature between the environment image and the first joint state.

[0172] In some embodiments, the prediction module 4552 is also used to map the first joint feature from the latent space to the original space where the first joint state is located, and obtain a third mapping feature of the first joint feature in the original space; based on the third mapping feature, the joint state of the robotic arm at the second moment is predicted to obtain the second joint state.

[0173] In some embodiments, the adjustment module 4553 is also used to refine the joint state characteristics of the second joint state to obtain the refined joint state characteristics of the second joint state; add the antagonistic feedback details to the refined joint state characteristics to obtain the enhanced joint state characteristics; perform feature adjustment on the enhanced joint state characteristics to obtain the adjusted joint state characteristics, and generate the third joint state based on the adjusted joint state characteristics.

[0174] In some embodiments, the adjustment module 4553 is also used to obtain a first historical joint state of the robot's mechanical arm at a first historical moment, a second historical joint state of the mechanical arm at a second historical moment, and a historical environment image collected by the robot for the historical environment at the first historical moment; determine a second joint feature between the historical environment image and the first historical joint state, and predict the joint state of the mechanical arm at the second historical moment based on the second joint feature to obtain a first predicted joint state; perform adversarial learning based on the first predicted joint state and the second historical joint state to obtain adversarial feedback details.

[0175] In some embodiments, the adjustment module 4553 is also used to perform adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details; perform state adjustment on the first predicted joint state based on the first adversarial feedback details to obtain a first adjusted joint state; perform adversarial learning based on the first adjusted joint state and the second historical joint state to obtain second adversarial feedback details; perform state adjustment on the first adjusted joint state based on the second adversarial feedback details to obtain a second adjusted joint state, and when the error between the second adjusted joint state and the second historical joint state is lower than a preset error, the second adversarial feedback details are determined as adversarial feedback details.

[0176] In some embodiments, the adjustment module 4553 is also used to classify the first predicted joint state to obtain a first category score of the first predicted joint state, and classify the second historical joint state to obtain a second category score of the second historical joint state; determine a first loss value of adversarial learning based on the first category score and the second category score; determine the state difference between the first predicted joint state and the second historical joint state based on the first loss value, and obtain first adversarial feedback details based on the state difference.

[0177] In some embodiments, the adjustment module 4553 is also used to determine a second loss and a third loss based on the first predicted joint state and the second historical joint state, wherein the second loss is used to characterize the similarity between the first predicted joint state and the second historical joint state, and the third loss is used to characterize the difference between the data space corresponding to the first predicted joint state and the data space corresponding to the second historical joint state; based on the second loss and the third loss, adversarial learning is performed on the first predicted joint state and the second historical joint state.

[0178] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the robot control method described in the embodiment of the present application.

[0179] The present application embodiment provides a computer-readable storage medium, in which computer executable instructions or computer programs are stored. When the computer executable instructions or computer programs are executed by a processor, the processor will be caused to execute the robot control method provided by the present application embodiment, for example, Figure 3A The robot control method is shown.

[0180] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0181] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0182] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0183] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0184] To summarize, through the embodiments of the present application, by obtaining the first joint state of the robot at the first moment and the environmental image collected at the first moment, the first joint feature between the first joint state and the environmental image is determined to learn the joint information of the two, so that the joint state of the robotic arm at the second moment is initially predicted based on the first joint feature to obtain the second joint state, and then, the state of the second joint state is adjusted based on the adversarial feedback details determined by adversarial learning (that is, secondary prediction) to obtain a more accurate third joint state, so as to control the robot's robotic arm to swing based on the third joint state. This method obtains the correlation between the environmental image and the first joint state through the first joint feature, and further adjusts the state through the adversarial feedback details, which can be better generalized to the data of the robot in different scenarios, thereby improving the accuracy of predicting the future joint state of the robot's robotic arm.

[0185] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A robot control method, characterized in that: The method comprises: Acquire a first joint state of a robot's mechanical arm at a first moment, and an environment image collected by the robot for a current environment at the first moment; Determine a first joint feature between the environment image and the first joint state, and predict the joint state of the robot arm at a second moment based on the first joint feature to obtain a second joint state; Determining antagonistic feedback details, and adjusting the second joint state based on the antagonistic feedback details to obtain a third joint state; When the second moment is reached, the robotic arm is controlled to swing to the third joint state.

2. The method according to claim 1, characterized in that The determining of a first joint feature between the environment image and the first joint state comprises: Performing feature extraction on the environment image to obtain image features, and performing feature extraction on the first joint state to obtain joint state features of the first joint state; Mapping the image feature and the joint state feature to a latent space to obtain a first mapping feature of the image feature in the latent space, a second mapping feature of the joint state feature in the latent space, and an intrinsic association between the image feature and the joint state feature; Based on the intrinsic association, feature fusion is performed on the first mapping feature and the second mapping feature to obtain a first joint feature between the environment image and the first joint state.

3. The method according to claim 2, characterized in that The predicting the joint state of the robot arm at the second moment based on the first joint feature to obtain a second joint state includes: Mapping the first joint feature from the latent space to the original space where the first joint state is located to obtain a third mapping feature of the first joint feature in the original space; The joint state of the robot arm at the second moment is predicted based on the third mapping feature to obtain a second joint state.

4. The method according to claim 1, characterized in that The step of adjusting the second joint state based on the antagonistic feedback details to obtain a third joint state includes: Refining the joint state feature of the second joint state to obtain a refined joint state feature of the second joint state; adding the antagonistic feedback details to the refined joint state feature to obtain an enhanced joint state feature; The enhanced joint state feature is subjected to feature adjustment to obtain an adjusted joint state feature, and the third joint state is generated based on the adjusted joint state feature.

5. The method according to claim 1, characterized in that The determining of the adversarial feedback details includes: Acquire a first historical joint state of a mechanical arm of the robot at a first historical moment, a second historical joint state of the mechanical arm at a second historical moment, and a historical environment image collected by the robot for the historical environment at the first historical moment; Determine a second joint feature between the historical environment image and the first historical joint state, and predict the joint state of the robot arm at the second historical moment based on the second joint feature to obtain a first predicted joint state; Adversarial learning is performed based on the first predicted joint state and the second historical joint state to obtain the adversarial feedback details.

6. The method according to claim 5, characterized in that The performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain the adversarial feedback details includes: Performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details; performing state adjustment on the first predicted joint state based on the first adversarial feedback details to obtain a first adjusted joint state; Performing adversarial learning based on the first adjusted joint state and the second historical joint state to obtain second adversarial feedback details; The first adjusted joint state is adjusted based on the second antagonistic feedback details to obtain a second adjusted joint state, and when the error between the second adjusted joint state and the second historical joint state is lower than a preset error, the second antagonistic feedback details are determined as the antagonistic feedback details.

7. The method according to claim 6, characterized in that The performing adversarial learning based on the first predicted joint state and the second historical joint state to obtain first adversarial feedback details includes: Classifying the first predicted joint state to obtain a first category score of the first predicted joint state, and classifying the second historical joint state to obtain a second category score of the second historical joint state; Determining a first loss value for the adversarial learning based on the first category score and the second category score; A state difference between the first predicted joint state and the second historical joint state is determined based on the first loss value, and the first adversarial feedback details are obtained based on the state difference.

8. The method according to claim 6, characterized in that The performing adversarial learning based on the first predicted joint state and the second historical joint state includes: Based on the first predicted joint state and the second historical joint state, determining a second loss and a third loss, wherein the second loss is used to characterize the similarity between the first predicted joint state and the second historical joint state, and the third loss is used to characterize the difference between the data space corresponding to the first predicted joint state and the data space corresponding to the second historical joint state; Based on the second loss and the third loss, adversarial learning is performed on the first predicted joint state and the second historical joint state.

9. A robot control device, characterized in that: The device comprises: An acquisition module, used to acquire a first joint state of a mechanical arm of the robot at a first moment, and an environment image collected by the robot for a current environment at the first moment; A prediction module, used for determining a first joint feature between the environment image and the first joint state, and predicting the joint state of the robot arm at a second moment based on the first joint feature to obtain a second joint state; an adjustment module, configured to determine the details of the antagonistic feedback, and adjust the second joint state based on the details of the antagonistic feedback to obtain a third joint state; The control module is used to control the robotic arm to swing to the third joint state when the second moment is reached.

10. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; A processor, configured to implement the method according to any one of claims 1 to 8 when executing computer executable instructions or computer programs stored in the memory.

11. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer programs are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising computer executable instructions or a computer program, characterized in that: When the computer executable instructions or computer programs are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images