Robot control methods, devices and electronic equipment

Abstract

This application discloses a robotic arm control method, device, and electronic device. The method includes: acquiring a first task of the robotic arm; determining a first neural network based on the first task; reconstructing the first task using the first neural network to obtain a control strategy for the reconstructed first task, the control strategy including a series of basic control instructions; acquiring control parameters for controlling the robotic arm based on the control strategy; and controlling the robotic arm to perform control actions according to the control parameters, the control parameters including parameters corresponding to the basic control instructions. Because a neural network model is used, self-learning of the robotic arm control can be achieved, meeting the flexible and varied control needs of the robotic arm and enabling accurate control. Furthermore, since the control is performed by a server, the computational power requirements of the robotic arm are reduced, making it suitable for a wider range of scenarios.

Description

Technical Field

[0001] This application relates to the field of robotic arm technology, and particularly to a robotic arm control method, device, and electronic device. Background Technology

[0002] There are various control methods for robotic arms, such as traditional PID control (proportional, integral, and derivative control) and fuzzy control. PID control is most suitable when a system and the controlled object are not fully understood, or when system parameters cannot be obtained through effective measurement methods. In practice, PI and PD control also exist. A PID controller calculates the control quantity based on the system error using proportional, integral, and derivative functions. In fuzzy control, the input quantity is fuzzified into a fuzzy variable. The fuzzy variable is then used to infer a fuzzy output through fuzzy rules. After defuzzification, a clear output quantity is obtained for control.

[0003] However, many external factors affect the ability of intelligent robot manipulators to achieve good control. With the increasing intelligence of robots and the development of artificial intelligence, intelligent robots can no longer meet the needs of flexible, ever-changing, and self-learning operations through traditional control methods, and can no longer accurately perform corresponding control. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the related art. To this end, this application proposes a robot control method, device, and electronic device, which can realize flexible and versatile self-learning control of the robot.

[0005] In a first aspect, this application provides a robotic arm control method, the method comprising:

[0006] Obtain the first task of the robotic arm, and determine the first neural network based on the first task;

[0007] The first task is reconstructed using the first neural network to obtain the control strategy of the reconstructed first task, and the control strategy includes a series of basic control instructions.

[0008] Based on the control strategy, control parameters for controlling the robotic arm are obtained, and the robotic arm is controlled to perform control actions according to the control parameters. The control parameters include parameters corresponding to the basic control commands.

[0009] In some embodiments, the first task of acquiring the robotic arm includes:

[0010] Obtain the first task and the corresponding environmental state information, wherein the environmental state information includes the surrounding environment information and the manipulator's own state information in the simulation environment;

[0011] The process of reconstructing the first task using a pre-trained first neural network to obtain a control strategy for the reconstructed first task includes:

[0012] The surrounding environment information, the self-state information, and the first task are input into the first neural network to obtain the reconstructed control strategy of the first task. The control strategy includes the basic control instructions and the scene control instructions.

[0013] In some embodiments, obtaining control parameters for controlling the robotic arm based on the control strategy, and controlling the robotic arm to perform control actions according to the control parameters, includes:

[0014] Based on the control strategy, control parameters and scene parameters for controlling the robotic arm are obtained, wherein the scene parameters are the parameters corresponding to the scene control commands.

[0015] The control parameters and / or the scene parameters are reduced to obtain simplified control parameters and / or scene parameters, and the robot arm is controlled to perform control actions according to the simplified control parameters and / or scene parameters.

[0016] In some embodiments, each of the control parameters and / or the scene parameters corresponds to a basic action, and controlling the robotic arm to perform control actions based on the simplified control parameters and / or scene parameters includes:

[0017] The basic movements of the robotic arm and the execution order of each basic movement are obtained based on the simplified control parameters and / or the scene parameters, and the robotic arm is controlled to execute the basic movements sequentially according to the execution order.

[0018] In some embodiments, the method further includes:

[0019] Based on the reinforcement learning algorithm, the first neural network is trained using the control strategy to obtain feedback parameters, and the first neural network is adjusted according to the feedback parameters to obtain an updated first neural network.

[0020] Secondly, this application provides a robotic arm control device, which includes:

[0021] The acquisition module is used to acquire the first task of the robotic arm and determine the first neural network based on the first task;

[0022] The reconstruction module is used to reconstruct the first task using the first neural network to obtain the control strategy of the reconstructed first task, wherein the control strategy includes a series of basic control instructions.

[0023] The control module is used to obtain control parameters for controlling the robot arm based on the control strategy, and to control the robot arm to perform control actions according to the control parameters. The control parameters include parameters corresponding to the basic control commands.

[0024] In some embodiments, the acquisition module is further configured to acquire the first task and the environmental state information corresponding to the first task, wherein the environmental state information includes the surrounding environment information and the manipulator's own state information in the simulation environment;

[0025] The reconstruction module is further configured to input the surrounding environment information, the self-state information, and the first task into the first neural network to obtain the reconstructed control strategy of the first task, wherein the control strategy includes the basic control instructions and the scene control instructions.

[0026] In some embodiments, the control module is further configured to acquire control parameters and scene parameters for controlling the robotic arm based on the control strategy, wherein the scene parameters are parameters corresponding to the scene control command; reduce the control parameters and / or the scene parameters to obtain simplified control parameters and / or the scene parameters, and control the robotic arm to perform control actions according to the simplified control parameters and / or the scene parameters.

[0027] In some embodiments, the control module is further configured to obtain the basic movements of the robotic arm and the execution order of each basic movement based on the simplified control parameters and / or the scene parameters, and control the robotic arm to execute the basic movements sequentially according to the execution order.

[0028] In some embodiments, the apparatus further includes:

[0029] The model training module, based on the reinforcement learning algorithm, uses the control strategy to train the first neural network, obtains feedback parameters, and adjusts the first neural network according to the feedback parameters to obtain an updated first neural network.

[0030] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the robotic arm control method described in the first aspect above.

[0031] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robotic arm control method as described in the first aspect above.

[0032] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the robotic arm control method as described in the first aspect.

[0033] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the robotic arm control method as described in the first aspect above.

[0034] The above-described one or more technical solutions in the embodiments of this application have at least the following technical effects:

[0035] The robotic arm control method, device, and electronic device provided in this application obtain the first task of the robotic arm through a server, select and determine a first neural network based on the first task, and reconstruct the first task using the first neural network to obtain a control strategy for the reconstructed first task. The control strategy includes a series of basic control instructions, and control parameters for controlling the robotic arm are obtained based on the series of basic control instructions. Finally, the robotic arm is controlled to perform control actions according to the control parameters to complete the corresponding movement. Since a neural network model is used, self-learning of robotic arm control can be achieved, meeting the flexible and ever-changing control needs of the robotic arm, and accurately making corresponding control. Furthermore, since it is completed by the server, the computing power requirements of the robotic arm can be reduced, making it suitable for a wider range of scenarios.

[0036] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0037] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0038] Figure 1 This is a schematic diagram of a robotic arm control frame provided in an embodiment of this application;

[0039] Figure 2 This is a flowchart illustrating the robotic arm control method provided in an embodiment of this application;

[0040] Figure 3 This is a schematic diagram of the structure of the robotic arm control device provided in the embodiments of this application;

[0041] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0043] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0044] First, the overall workflow of the robotic arm is described; please refer to [link / reference]. Figure 1 , Figure 1 The diagram illustrates one possible structure of a robotic arm control framework. The control of the robotic arm can be based on artificial intelligence (AI). This AI can be installed in the cloud, such as hundreds or thousands of robots being controlled by a cloud server; alternatively, it can be installed on the robot's internal brain or processor, such as the robotic arm's motion control being controlled by a robotic chip; furthermore, the robotic arm can also be controlled by a control chip embedded in the arm itself. Generally, the robot's control processor needs to handle the control of the entire robot body, and the computing power allocated to the robotic arm is limited; the processing power of the robotic arm's control processor is weaker than that of the robot's processor.

[0045] Artificial intelligence originates from data. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." Basic data, combined with the aggregation of external data, such as multimodal data from vision, hearing, and touch, can provide sufficient data support to help achieve the control of robotic arms.

[0046] The robotic arm control method, device, and electronic device provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios. The robotic arm control method can be applied to a terminal, and can be executed by hardware or software in the terminal.

[0047] like Figure 2As shown, this robotic arm control method, applied to a server, includes the following steps:

[0048] S100, obtain the first task of the robotic arm, and determine the first neural network based on the first task.

[0049] It is understood that the first task in this embodiment can be acquired by the robotic arm and then sent to the server, or it can be acquired directly by the server. Specifically, the robotic arm / server receives the first task input by the operator, that is, the first task can be input through the receiving interface of the hardware device; it can also be a pre-input task, from which the robotic arm / server can select the first task; or it can be a pre-set task generation rule, so that the robotic arm can generate the first task according to the generation rule. In some embodiments, the generation rule can learn to form increasingly difficult tasks. For example, if the initial task is for the robotic arm to clench its fist, the generated first task, which is more difficult than the initial task, could be for the robotic arm to switch between clenching and opening its fist, switching every 3 seconds; another example is that the generated first task, which is even more difficult, could be for the robotic arm to pick up a cup from the table. In some embodiments, the first task can be a control command, which can be an action command such as picking up, lifting, dragging, or pulling.

[0050] It should be noted that the robotic arm in the embodiments of this application can be an intelligent robot.

[0051] S200, the first task is reconstructed using the first neural network to obtain the control strategy of the reconstructed first task, the control strategy including a series of basic control instructions.

[0052] In some embodiments, after obtaining a first task, the server determines a first neural network based on the task, and then uses the first neural network to reconstruct the first task to obtain the control strategy corresponding to the reconstructed first task. It can be understood that the first neural network can be interpreted as retrieving the path and result for completing the first task from a knowledge base or skill base, and breaking it down into basic control instructions, thereby forming the control strategy for the first task. Each skill in the skill base or knowledge base can be specifically represented as a neural network or as an operational rule. For example, a specific skill in the knowledge base or skill base could be picking up an empty cup with one hand, picking up a watermelon with both hands, or picking up an object weighing more than 10 kg with both hands.

[0053] In some embodiments, the first neural network can be pre-trained using its skill base or knowledge base for different tasks. It should be noted that before utilizing the first neural network, a step of acquiring the first neural network may be included. That is, the server may store multiple neural networks and skill bases. The first neural network can then be a neural network trained based on the simulation environment corresponding to the second task. In other words, the first neural network can be a mature neural network from at least one pre-trained neural network. Correspondingly, the server can determine the skill base corresponding to the first neural network as the skill base. More specifically, the operator can select the first neural network from at least one pre-trained neural network, and the server can then acquire the first neural network selected by a person skilled in the art; alternatively, the server can autonomously select the first neural network from at least one pre-trained neural network, wherein the semantic information of the first task is similar to the semantic information of the second task. Specifically, the similarity of semantic information between the first task and the second task can refer to several factors. First, the semantic information of the first task can be obtained using a neural network and compared to determine similarity. Second, the constraints obtained from decomposing the first and second tasks can be similar. For example, if the result of decomposing the first task is to lift something, and the constraint of decomposing the second task is to lift something that is particularly heavy or light, then the semantic information of the first task can be considered similar to that of the second task. Third, the operating environments of the first and second tasks can be similar. For example, if the first task is to lift something, and the second task is to support something, then the semantic information of the first task can be considered similar to that of the second task. Of course, other methods can also be used to determine the similarity of semantic information between the first and second tasks. The examples here are only for the convenience of understanding this solution and do not exhaustively list all implementation methods.

[0054] In some embodiments, after determining the first task and the neural network type of the first neural network, the server may initialize a first neural network and initially train a skill library based on the simulation environment corresponding to the first task using a reinforcement learning algorithm. In another implementation, after determining the first task and the neural network type of the first neural network, the server may initialize a first neural network, and then a person skilled in the art may configure at least one skill in the skill library according to the first task. Since the skills in the skill library can be expanded in subsequent steps, the number of skills in the skill library does not need to be particularly large. The skill library contains some basic action components of the embodied robot, such as drinking water, carrying, and walking.

[0055] In some embodiments, step S100 includes: acquiring the first task and the environmental state information corresponding to the first task, wherein the environmental state information includes the surrounding environment information and the manipulator's own state information in the simulation environment.

[0056] In this embodiment, the server inputs environmental state information related to the first task into the first neural network to obtain the skills selected by the first neural network from the skill library. Specifically, the environmental state information includes the surrounding environment information of the robot in the simulation environment corresponding to the first task and the robot's own state information. Specifically, it may include map information around the intelligent robot, interaction information of the intelligent robot, movement information of adjacent intelligent robots, etc. As an example, if this embodiment of the application is applied to the field of embodied robot intelligent robotic arms, the environmental state information may include positioning information, map information, interaction information, task information, operation information, etc.

[0057] Step S200 includes: inputting the surrounding environment information, the self-state information, and the first task into the first neural network to obtain the reconstructed control strategy of the first task, wherein the control strategy includes the basic control instructions and the scene control instructions.

[0058] In this embodiment, in addition to considering the instruction information of the first task itself, the surrounding environment information and the robot's own state information are also fully considered. The control strategy reconstructed by considering this information is more in line with the robot's operation, achieving accurate motion control. For example, if the first task is to reach a certain destination, without considering the surrounding environment information, the robot would move directly in a straight line. This way of moving might cause the robot to encounter obstacles and be unable to move. The control strategy that considers the surrounding environment information will avoid obstacles and use a straight-line + turning instruction to control the robot to complete the action. Considering the robot's own state information can avoid conflicts between two tasks. For example, if the robot is currently performing a task, it needs to wait for that task to be completed before executing the first task.

[0059] In some embodiments, the environmental state information can be in the form of images, sequence data, or other data formats. When inputting this environmental state information into the first neural network, the type of the first neural network can be determined according to the data type of the input data. For example, if the input data is image data, the first neural network can be a convolutional neural network (CNN); if the input data is sequence data, the first neural network can be a recurrent neural network (RNN), etc. Other cases will not be listed here.

[0060] In some embodiments, when the first neural network outputs a control strategy, it predicts the skill to be selected based on the input first task and the corresponding environmental state information, and outputs the sequence number of the corresponding skill. The server then selects the appropriate skill from the skill library based on these output sequence numbers. It is understood that a unique sequence number can be pre-configured for each skill in the skill library. Each skill can be understood as the most basic motion control command when the robotic arm performs an action, such as forward, backward, left turn, right turn, etc.

[0061] It is understandable that the process of the first neural network reconstructing the first task is equivalent to decomposing and recombining the first task to obtain a series of basic control instructions.

[0062] Understandably, during the process of reconstructing the first task, the first neural network will confirm the expected processing result. If it finds that the first task is impossible to complete during reconstruction, it will stop outputting the specific action. For example, if the maximum lifting weight of the robotic arm is 50kg, but the acquired environmental information shows that the weight to be lifted is 100kg, then it exceeds the robotic arm's capability. The first neural network needs to output information to the robot or robotic arm, stop the task and the command action, and issue a corresponding verbal prompt.

[0063] S300, based on the control strategy, obtain the control parameters for controlling the robot arm, so as to control the robot arm to perform control actions according to the control parameters.

[0064] In some embodiments, step S300 includes: obtaining the basic actions for controlling the robotic arm and the execution order of each basic action according to the control parameters, and controlling the robotic arm to execute each basic action sequentially according to the execution order.

[0065] Once the control strategy of the robotic arm is obtained, it is equivalent to obtaining a series of basic control instructions for the robotic arm. Each control instruction corresponds to a control parameter. By adjusting the control parameters of the robotic arm, the robotic arm can execute each basic action in sequence according to the execution order, so as to realize the set path and / or action operation.

[0066] Furthermore, in some embodiments, the basic movements of the robotic arm and the execution order of each basic movement are obtained based on the control parameters and / or the scene parameters, and the robotic arm is controlled to execute the basic movements sequentially according to the execution order to complete the corresponding operation. Specifically, when controlling the robotic arm to execute the basic movements sequentially according to the execution order, performance parameters such as time, computing power, and resources used, as well as the energy, force, angle, speed, and arm / finger degrees of freedom used in the basic movements, are estimated, and the robotic arm is controlled according to the estimated performance parameters.

[0067] In some embodiments, step S300 includes: obtaining control parameters and scene parameters for controlling the robotic arm based on the control strategy, wherein the scene parameters are parameters corresponding to the scene control command; reducing the control parameters to obtain simplified control parameters; and controlling the robotic arm to perform control actions according to the simplified control parameters.

[0068] In some embodiments, step S300 further includes: each of the control parameters and / or the scene parameters corresponds to a basic action; obtaining the basic action of the robot and the execution order of each basic action according to the simplified control parameters and / or the scene parameters; and controlling the robot to execute the basic actions sequentially according to the execution order.

[0069] Specifically, when outputting the control strategy, the robot's degrees of freedom need to be considered. That is, when determining the control strategy, the relationship between the degrees of freedom required to complete the first task and the robot's own degrees of freedom is assessed. If the degrees of freedom required to complete the first task are greater than the robot's own degrees of freedom, the control strategy is further simplified; if the degrees of freedom required are less than the robot's own degrees of freedom, the control strategy for the first task is executed. When the degrees of freedom required are less than the robot's own degrees of freedom, it indicates that the degrees of freedom for the first task are redundant, representing a better control strategy that can be better learned by the neural network.

[0070] After receiving the control strategy, the server can further simplify the control parameters and / or scene parameters to control the robotic arm's movements with the simplest possible parameters, avoiding overly complex and unnecessary motions. Scene parameters can be understood as the scene elements of the primary task. The fewer the scene elements, the clearer the control strategy based on the simplified scene elements, thus enabling better control of the robotic arm. Specifically, the scene elements for reconstructing the primary task include the objective, result, process, control method, and precautions. For example, in the action of drinking water, the objective is to drink water, the result is to drink water, the process is to lift the cup to drink, the control method is to lift the cup, and precautions include whether there is water in the cup and what material the cup is made of. As mentioned earlier, the drinking action is stored in the skill library. If the reconstructed primary task has many scene parameters, these parameters are simplified. The principle of simplification is to remove useless and redundant parameters, such as the color of the cup and the liquid, retaining only key parameters such as the cup's position, size, and mass. If the number of key parameters is the same as or slightly greater than the number of parameters for the basic actions in the skill library, simplification is unnecessary. Of course, parameters can also be reduced. If there are actions such as raising a cup and drinking water, they can be directly simplified to drinking water.

[0071] It should be noted that the server can be a cloud server, which controls the robotic arm through communication.

[0072] The robotic arm control method provided in this application obtains the first task of the robotic arm through a server, selects and determines a first neural network based on the first task, and reconstructs the first task using the first neural network to obtain a control strategy for the reconstructed first task. The control strategy includes a series of basic control instructions. Based on the series of basic control instructions, control parameters for controlling the robotic arm are obtained. Finally, the robotic arm is controlled to perform control actions according to the control parameters to complete the corresponding movement. Since this method utilizes a neural network model, it can achieve self-learning of robotic arm control, meet the flexible and varied control needs of the robotic arm, and accurately perform corresponding control. Furthermore, since this method is completed by the server, it can also reduce the computing power requirements of the robotic arm and adapt to a wider range of scenarios.

[0073] In some embodiments, based on a reinforcement learning algorithm, the control strategy is used to train a neural network model, obtain feedback parameters, and adjust the feedback parameters to obtain a trained second neural network.

[0074] By using reinforcement learning algorithms, the control strategy is memorized and trained to enable rapid control execution when encountering similar scenarios again.

[0075] It should be noted that when the server controls the robotic arm's movements based on the simulation environment corresponding to the first task, it will first complete the first task using at least one skill selected by the first neural network. After the server controls the intelligent robotic arm to perform the first task, it will acquire the data (such as control parameters and scene parameters) of the intelligent robotic arm's performance in performing the first task, and use reinforcement learning algorithms to retrain the parameters of the first neural network to obtain an updated first neural network.

[0076] More specifically, the server inputs environmental state information into the first neural network to obtain the skill selected by the first neural network. The environmental state information includes the environmental information around the intelligent robotic arm in the simulation environment corresponding to the first task and the intelligent robotic arm's own state information. Through the skill selected by the first neural network to execute the first task, basic control commands are obtained. Then, the intelligent robotic arm can be controlled in the simulator to perform operations corresponding to the basic control commands. During execution, the server performs an execution state acquisition operation for the skill selected by the first neural network every preset time interval until the execution state of the skill selected by the first neural network is "execution completed". The server obtains data generated during the intelligent robotic arm's execution of the operation corresponding to the control commands. The data includes one or more of the following: the intelligent robotic arm's operation path, operation speed, or operation destination. Based on the data, the server updates the parameters of the first neural network using a reinforcement learning algorithm. The concepts of intelligent robotic arm, preset time interval, and execution state have been described in detail above and will not be repeated here. In this embodiment, the server determines whether the skill selected by the first neural network has finished executing by acquiring the execution status of the skill selected by the first neural network every preset time interval. In this way, the server can iteratively update the new skill strategy and the parameters of the new skill in a timely manner based on the operation behavior information of the intelligent robotic arm, which is beneficial to improving the accuracy of the training process.

[0077] The robotic arm control method provided in this application can be executed by a robotic arm control device. This application uses the example of a robotic arm control device executing the robotic arm control method to illustrate the robotic arm control device provided in this application.

[0078] This application also provides a robotic arm control device, such as... Figure 3 As shown, the robotic arm control device includes: an acquisition module 100, used to acquire the first task of the robotic arm and determine a first neural network based on the first task;

[0079] The reconstruction module 200 is used to reconstruct the first task using the first neural network to obtain the control strategy of the reconstructed first task, the control strategy including a series of basic control instructions.

[0080] The control module 300 is used to obtain control parameters for controlling the robot arm based on the control strategy, and to control the robot arm to perform control actions according to the control parameters. The control parameters include parameters corresponding to the basic control instructions.

[0081] The robotic arm control device provided in this application embodiment obtains the first task of the robotic arm through a server, selects and determines a first neural network based on the first task, and reconstructs the first task using the first neural network to obtain a control strategy for the reconstructed first task. The control strategy includes a series of basic control instructions, and control parameters for controlling the robotic arm are obtained based on the series of basic control instructions. Finally, the robotic arm is controlled to perform control actions according to the control parameters to complete the corresponding movement. Since the device utilizes a neural network model, it can achieve self-learning of robotic arm control, meet the flexible and ever-changing control needs of the robotic arm, and accurately perform corresponding control. Furthermore, since the device is completed by a server, it can also reduce the computing power requirements of the robotic arm and adapt to a wider range of scenarios.

[0082] In some embodiments, the acquisition module is further configured to acquire the first task and the environmental state information corresponding to the first task, wherein the environmental state information includes the surrounding environment information and the manipulator's own state information in the simulation environment;

[0083] The reconstruction module is further configured to input the surrounding environment information, the self-state information, and the first task into the first neural network to obtain the reconstructed control strategy of the first task, wherein the control strategy includes the basic control instructions and the scene control instructions.

[0084] In some embodiments, the control module is further configured to acquire control parameters and scene parameters for controlling the robotic arm based on the control strategy, wherein the scene parameters are parameters corresponding to the scene control command; reduce the control parameters and / or the scene parameters to obtain simplified control parameters and / or the scene parameters, and control the robotic arm to perform control actions according to the simplified control parameters and / or the scene parameters.

[0085] In some embodiments, each of the control parameters and / or the scene parameters corresponds to a basic action. The control module is further configured to obtain the basic actions of the robot and the execution order of each basic action according to the simplified control parameters and / or the scene parameters, and control the robot to execute the basic actions sequentially according to the execution order.

[0086] In some embodiments, the model training module trains the first neural network based on a reinforcement learning algorithm using the control strategy, obtains feedback parameters, and adjusts the first neural network according to the feedback parameters to obtain an updated first neural network.

[0087] The robotic arm control device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific type of device.

[0088] The robotic arm control device in this application embodiment can be a device with an operating system. This operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit it.

[0089] The robotic arm control device provided in this application embodiment can achieve... Figure 2 To avoid repetition, the various processes implemented in the method embodiments shown will not be described again here.

[0090] The robotic arm control method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the control method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The control method provided in this application embodiment is described below using an electronic device as the execution subject.

[0091] In some embodiments, such as Figure 4 As shown, this application embodiment also provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the program is executed by the processor 801, it implements the various processes of the above-described robotic arm control method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0092] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0093] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described robotic arm control method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0094] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0095] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described robotic arm control method.

[0096] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0097] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described robotic arm control method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0098] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0099] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0100] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0101] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0102] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0103] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for controlling a robotic arm, characterized in that, Applied to a server, the method includes: Obtain the first task of the robotic arm, and determine the first neural network based on the first task; The first task is reconstructed using the first neural network to obtain the control strategy of the reconstructed first task, and the control strategy includes a series of basic control instructions. Based on the control strategy, control parameters for controlling the robotic arm are obtained, and the robotic arm is controlled to perform control actions according to the control parameters. The control parameters include parameters corresponding to the basic control commands. The first task of acquiring the robotic arm includes: Obtain the first task and the corresponding environmental state information, wherein the environmental state information includes the surrounding environment information and the manipulator's own state information in the simulation environment; The process of reconstructing the first task using a pre-trained first neural network to obtain a control strategy for the reconstructed first task includes: The surrounding environment information, the self-state information, and the first task are input into the first neural network to obtain the reconstructed control strategy of the first task. The control strategy includes the basic control instructions and the scene control instructions. The step of obtaining control parameters for controlling the robotic arm based on the control strategy, and controlling the robotic arm to perform control actions according to the control parameters, includes: Based on the control strategy, control parameters and scene parameters for controlling the robotic arm are obtained, wherein the scene parameters are the parameters corresponding to the scene control commands. The control parameters and / or the scene parameters are reduced to obtain simplified control parameters and / or scene parameters, and the robot arm is controlled to perform control actions according to the simplified control parameters and / or scene parameters.

2. The method according to claim 1, characterized in that, Each of the control parameters and / or the scene parameters corresponds to a basic action. Controlling the robotic arm to perform control actions based on the simplified control parameters and / or scene parameters includes: The basic movements of the robotic arm and the execution order of each basic movement are obtained based on the simplified control parameters and / or the scene parameters, and the robotic arm is controlled to execute the basic movements sequentially according to the execution order.

3. The method according to any one of claims 1-2, characterized in that, The method further includes: Based on the reinforcement learning algorithm, the first neural network is trained using the control strategy to obtain feedback parameters, and the first neural network is adjusted according to the feedback parameters to obtain an updated first neural network.

4. A robotic arm control device, characterized in that, Applied to a server, the device includes: The acquisition module is used to acquire the first task of the robotic arm and determine the first neural network based on the first task; The reconstruction module is used to reconstruct the first task using the first neural network to obtain the control strategy of the reconstructed first task, wherein the control strategy includes a series of basic control instructions. The control module is used to acquire control parameters for controlling the robotic arm based on the control strategy, and to control the robotic arm to perform control actions according to the control parameters. The control parameters include parameters corresponding to the basic control instructions. The acquisition module is further configured to acquire the first task and the environmental state information corresponding to the first task, wherein the environmental state information includes the surrounding environment information and the manipulator's own state information in the simulation environment; The reconstruction module is further configured to input the surrounding environment information, the self-state information, and the first task into the first neural network to obtain the control strategy of the reconstructed first task, wherein the control strategy includes the basic control instructions and the scene control instructions. The control module is further configured to acquire control parameters and scene parameters for controlling the robotic arm based on the control strategy, wherein the scene parameters are parameters corresponding to the scene control command; to reduce the control parameters and / or the scene parameters to obtain simplified control parameters and / or the scene parameters, and to control the robotic arm to perform control actions according to the simplified control parameters and / or the scene parameters.

5. The apparatus according to claim 4, characterized in that, The device further includes: The model training module, based on the reinforcement learning algorithm, uses the control strategy to train the first neural network, obtains feedback parameters, and adjusts the first neural network according to the feedback parameters to obtain an updated first neural network.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the robotic arm control method as described in any one of claims 1-3.

Images