A model training method and related equipment

Abstract

This application discloses a model training method and related equipment, which can effectively reduce the time required for model training and improve the efficiency of model training. The method includes: when training a model to be trained is required, firstly, obtaining the description information of the target object that the robot is about to manipulate, and the actual description information of the manipulation task that the robot needs to complete on the target object, and inputting this information into the model to be trained. Next, the model to be trained can perform a series of processes on the description information of the target object and the actual description information of the manipulation task to obtain the predicted action for the robot to complete the manipulation task. Then, the robot can be controlled to perform the predicted action on the target object to obtain the predicted description information of the manipulation task. Finally, based on the predicted description information and the actual description information of the manipulation task, the model to be trained can be trained to obtain the target model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a model training method and related equipment. Background Technology

[0002] Driven by the convergence and innovation of various technologies such as information technology, materials technology, and sensor technology, robots are becoming increasingly intelligent and flexible. Robot control models are an important component of robot control systems. Based on the control task, the robot control model can generate corresponding actions, thereby controlling the robot to execute those actions and complete the control task.

[0003] Currently, when training a robot control model, information about the target object and the control task for that object is first obtained, and this information is then input into the model to be trained. The model then processes this information to obtain predicted actions for completing the control task. Since the actual actions used to complete the control task can be obtained in advance, the model can be trained based on both the predicted and actual actions to obtain the robot control model.

[0004] However, the actual actions used to complete the manipulation tasks often need to be designed by operators according to the required manipulation tasks, and the robot must be instructed to perform these actual actions on the target object in advance so that the robot can correctly complete the manipulation tasks. Therefore, collecting actual actions consumes a significant amount of time, resulting in low model training efficiency. Summary of the Invention

[0005] This application provides a model training method and related equipment, which can effectively reduce the time required for model training and improve the efficiency of model training.

[0006] A first aspect of this application provides a model training method, the method comprising:

[0007] When it is necessary to train the model to be trained, the description information of the target object collected by the robot and the real description information of the control task for the target object can be obtained from the user device. The real description information of the control task can also be called the single-modal demonstration information or multi-modal demonstration information designed by the user for the control task through the user device.

[0008] After obtaining the description information of the target object and the actual description information of the control task, the description information of the target object and the actual description information of the control task can be input into the model to be trained. The model to be trained will then perform a series of processing on the description information of the target object and the actual description information of the control task to obtain the predicted actions for the robot to complete the control task.

[0009] After obtaining the predicted actions to complete the manipulation task, the robot can be controlled to execute the predicted actions on the target object, thereby changing the state of both the robot and the target object. Thus, the robot completes the manipulation task on the target object. The user equipment can then observe the states of the robot and the target object to generate predicted description information for the manipulation task. Therefore, predicted description information for the manipulation task can be obtained from the user equipment. This predicted description information can also be referred to as the single-modal or multi-modal observation information of the manipulation task collected by the user through the user equipment.

[0010] After obtaining the predicted description information of the manipulation task, the model to be trained can be trained using the predicted description information and the actual description information of the manipulation task, thereby obtaining the target model.

[0011] As can be seen from the above method, when training a model, the following steps can be taken: First, obtain the description information of the target object that the robot is about to manipulate, as well as the actual description information of the control task the robot needs to complete on the target object. Then, input the description information of the target object and the actual description information of the control task into the model to be trained. Next, the model to be trained can perform a series of processes on the description information of the target object and the actual description information of the control task to obtain the predicted actions for the robot to complete the control task. Then, control the robot to execute the predicted actions on the target object to obtain the predicted description information of the control task. Finally, based on the predicted description information and the actual description information of the control task, train the model to be trained to obtain the target model, which is the trained robot control model. In the aforementioned process, after the model to be trained obtains the predicted action based on the real description information of the manipulation task (i.e., the demonstration information of the manipulation task), the robot can be instructed to perform the predicted action on the target object, thereby observing the state of the target object and the state of the robot to obtain the predicted description information of the manipulation task (i.e., the observation information of the manipulation task). In this way, the real description information of the manipulation task can be directly used as the label of the predicted description information of the manipulation task, and the model to be trained can be trained in a self-supervised training manner to obtain the target model. It is not necessary to use robots to collect training samples on a large scale in advance, which can effectively reduce the time required for model training and improve the efficiency of model training.

[0012] In one possible implementation, the process of processing the description information of the target object and the real description information of the manipulation task using the model to be trained to obtain the predicted action for completing the manipulation task includes: extracting features from the description information of the target object and the real description information of the manipulation task using the model to be trained to obtain features of the target object and features of the manipulation task; and generating the predicted action for completing the manipulation task based on the features of the target object and the features of the manipulation task using the model to be trained. In the aforementioned implementation, after inputting the description information of the target object and the real description information of the manipulation task into the model to be trained, the model to be trained can perform feature extraction on the description information of the target object (e.g., convolution operation, pooling operation, fully connected operation, etc.) to obtain the features of the target object. At the same time, the model to be trained can also perform feature extraction on the real description information of the manipulation task (e.g., convolution operation, pooling operation, fully connected operation, etc.) to obtain the features of the manipulation task. After obtaining the features of the target object and the features of the manipulation task, the model to be trained can further process the features of the target object and the features of the manipulation task to accurately generate the predicted action for completing the manipulation task.

[0013] In one possible implementation, generating a predicted action for completing the manipulation task based on the features of the target object and the features of the manipulation task by the model under test includes: fusing the features of the target object and the features of the manipulation task using the model under test to obtain a feature fusion result; compressing the feature fusion result using the model under test to obtain multiple distinct sub-actions; and superimposing the multiple sub-actions using the model under test to obtain the predicted action for completing the manipulation task. In the aforementioned implementation, after obtaining the features of the target object and the features of the manipulation task, the model under test can first fuse the features of the target object and the features of the manipulation task into a whole to obtain a feature fusion result, which is used to indicate the model under test's comprehensive understanding of the target object and the manipulation task. After obtaining the feature fusion result, the model under test can compress the feature fusion result to predict multiple sub-actions. After obtaining multiple sub-actions, the model under test can superimpose the multiple sub-actions to accurately obtain the predicted action for completing the manipulation task.

[0014] In one possible implementation, training the model to be trained based on the predicted and true descriptions of the manipulation task to obtain the target model includes: calculating a target loss based on the predicted and true descriptions of the manipulation task, whereby the target loss indicates the difference between the predicted and true descriptions of the manipulation task; and updating the parameters of the model to be trained based on the target loss until the model training conditions are met, thus obtaining the target model. In the aforementioned implementation, after obtaining the predicted descriptions of the manipulation task, a preset loss function is used to calculate the predicted and true descriptions of the manipulation task to obtain the target loss, which indicates the difference between the predicted and true descriptions of the manipulation task. After obtaining the target loss, the parameters of the model to be trained can be updated based on the target loss to obtain the updated model to be trained. Then, the updated model to be trained continues to be trained until the model training conditions are met, thus obtaining the target model. As can be seen, this implementation method constructs a target loss for updating model parameters by detecting the consistency between the demonstration information and the observation information of the manipulation task, thereby enabling self-supervised training. The resulting target model has better performance, that is, better robot manipulation capabilities.

[0015] In one possible implementation, the descriptive information of the target object includes at least one of the following: two-dimensional data of the target object and three-dimensional data of the target object, such as images, videos, point cloud data, etc. of the target object.

[0016] In one possible implementation, the realistic description information of the manipulation task includes at least one of the following: realistic description information of the manipulation task in text form, realistic description information of the manipulation task in image form, realistic description information of the manipulation task in audio form, and realistic description information of the manipulation task in instruction form. In the aforementioned implementation, the realistic description information of the manipulation task can be unimodal or multimodal demonstration information. Therefore, this information can be presented as text, images, audio, and instructions input by the user through a user device to describe the manipulation task. This allows the model to more efficiently and accurately understand and learn new skills used by the user to manipulate the target object using the robot, thus resulting in a target model with superior performance.

[0017] In one possible implementation, the predicted description information for the manipulation task includes at least one of the following: predicted description information in text form, predicted description information in image form, predicted description information in audio form, and predicted description information in instruction form. In the aforementioned implementation, the actual description information for the manipulation task can be unimodal or multimodal observation information. Therefore, this information can be presented as text, images, audio, and instructions collected by the user device to describe the manipulation task. This allows the model to more efficiently and accurately understand and learn new skills used by the user to manipulate the target object using the robot, thus resulting in a target model with superior performance.

[0018] A second aspect of this application provides a robot manipulation method, which includes: acquiring description information of a target object and description information of a manipulation task for the target object; processing the description information of the target object and the description information of the manipulation task through a target model to obtain a predicted action for completing the manipulation task; and executing the predicted action on the target object.

[0019] A third aspect of this application provides a model training apparatus, comprising: an acquisition module for acquiring descriptive information of a target object and real descriptive information of a manipulation task for the target object; a processing module for processing the descriptive information of the target object and the real descriptive information of the manipulation task using a model to be trained to obtain a predicted action for completing the manipulation task; an execution module for executing the predicted action on the target object to obtain the predicted descriptive information of the manipulation task; and a training module for training the model to be trained based on the predicted descriptive information of the manipulation task and the real descriptive information of the manipulation task to obtain a target model.

[0020] As can be seen from the above device, when training a model to be trained is required, the description information of the target object that the robot is about to operate on, and the actual description information of the control task that the robot needs to complete on the target object, can be obtained first. This description information and the actual description information of the control task are then input into the model to be trained. Next, the model to be trained can perform a series of processes on the description information of the target object and the actual description information of the control task to obtain the predicted actions for the robot to complete the control task. Then, the robot can be controlled to execute the predicted actions on the target object to obtain the predicted description information of the control task. Finally, based on the predicted description information and the actual description information of the control task, the model to be trained can be trained to obtain the target model, which is the trained robot control model. In the aforementioned process, after the model to be trained obtains the predicted action based on the real description information of the manipulation task (i.e., the demonstration information of the manipulation task), the robot can be instructed to perform the predicted action on the target object, thereby observing the state of the target object and the state of the robot to obtain the predicted description information of the manipulation task (i.e., the observation information of the manipulation task). In this way, the real description information of the manipulation task can be directly used as the label of the predicted description information of the manipulation task, and the model to be trained can be trained in a self-supervised training manner to obtain the target model. It is not necessary to use robots to collect training samples on a large scale in advance, which can effectively reduce the time required for model training and improve the efficiency of model training.

[0021] In one possible implementation, the processing module is configured to: extract features from the description information of the target object and the real description information of the manipulation task using the model to be trained, to obtain the features of the target object and the features of the manipulation task; and generate a predicted action for completing the manipulation task based on the features of the target object and the features of the manipulation task using the model to be trained.

[0022] In one possible implementation, the processing module is configured to: fuse the features of the target object and the features of the manipulation task using the model to be trained to obtain a feature fusion result; compress the feature fusion result using the model to be trained to obtain multiple distinct sub-actions; and superimpose the multiple sub-actions using the model to be trained to obtain a predicted action for completing the manipulation task.

[0023] In one possible implementation, the training module is used to: calculate the target loss by combining the predicted description information of the manipulation task with the actual description information of the manipulation task, the target loss being used to indicate the difference between the predicted description information of the manipulation task and the actual description information of the manipulation task; and update the parameters of the model to be trained based on the target loss until the model training conditions are met, thereby obtaining the target model.

[0024] In one possible implementation, the descriptive information of the target object includes at least one of the following: two-dimensional data of the target object and three-dimensional data of the target object.

[0025] In one possible implementation, the real description information of the control task includes at least one of the following: real description information of the control task in text form, real description information of the control task in image form, real description information of the control task in audio form, and real description information of the control task in instruction form.

[0026] In one possible implementation, the predictive description information for the manipulation task includes at least one of the following: predictive description information for the manipulation task in text form, predictive description information for the manipulation task in image form, predictive description information for the manipulation task in audio form, and predictive description information for the manipulation task in instruction form.

[0027] A fourth aspect of this application provides a robot control device, comprising: an acquisition module for acquiring description information of a target object and description information of a control task for the target object; a processing module for processing the description information of the target object and the description information of the control task through a target model to obtain a predicted action for completing the control task; and an execution module for executing the predicted action on the target object.

[0028] A fifth aspect of this application provides a model training apparatus, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the model training apparatus performs the method described in the first aspect or any possible implementation thereof.

[0029] A sixth aspect of this application provides a robot control device, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code, wherein when the code is executed, the robot control device performs the method described in the second aspect or any possible implementation thereof.

[0030] A seventh aspect of this application provides a circuit system including a processing circuit configured to perform the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0031] An eighth aspect of this application provides a chip system including a processor for calling a computer program or computer instructions stored in a memory to cause the processor to perform the method as described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0032] In one possible implementation, the processor is coupled to the memory via an interface.

[0033] In one possible implementation, the chip system also includes a memory that stores computer programs or computer instructions.

[0034] A ninth aspect of this application provides a computer storage medium storing a computer program that, when executed by a computer, causes the computer to perform the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0035] A tenth aspect of this application provides a computer program product storing instructions that, when executed by a computer, cause the computer to perform the method as described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0036] In this embodiment, when training the model to be trained is required, the description information of the target object that the robot is about to operate on, and the actual description information of the control task that the robot needs to complete on the target object, can be obtained first. The description information of the target object and the actual description information of the control task are then input into the model to be trained. Next, the model to be trained can perform a series of processes on the description information of the target object and the actual description information of the control task to obtain the predicted action for the robot to complete the control task. Then, the robot can be controlled to perform the predicted action on the target object to obtain the predicted description information of the control task. Finally, based on the predicted description information and the actual description information of the control task, the model to be trained can be trained to obtain the target model, which is the trained robot control model. In the aforementioned process, after the model to be trained obtains the predicted action based on the real description information of the manipulation task (i.e., the demonstration information of the manipulation task), the robot can be instructed to perform the predicted action on the target object, thereby observing the state of the target object and the state of the robot to obtain the predicted description information of the manipulation task (i.e., the observation information of the manipulation task). In this way, the real description information of the manipulation task can be directly used as the label of the predicted description information of the manipulation task, and the model to be trained can be trained in a self-supervised training manner to obtain the target model. It is not necessary to use robots to collect training samples on a large scale in advance, which can effectively reduce the time required for model training and improve the efficiency of model training. Attached Figure Description

[0037] Figure 1 A structural diagram illustrating the main framework of artificial intelligence;

[0038] Figure 2a A schematic diagram of the robot control system provided in the embodiments of this application;

[0039] Figure 2b This is another structural schematic diagram of the robot control system provided in the embodiments of this application;

[0040] Figure 2c A schematic diagram of the robot control related equipment provided in the embodiments of this application;

[0041] Figure 3 A schematic diagram of the system 100 architecture provided in the embodiments of this application;

[0042] Figure 4 A schematic diagram of the structure of the model to be trained provided in an embodiment of this application;

[0043] Figure 5 A schematic flowchart of the model training method provided in the embodiments of this application;

[0044] Figure 6 A schematic diagram illustrating an application example of the model training method provided in this application embodiment;

[0045] Figure 7 A schematic diagram of the action instruction generation model provided in the embodiments of this application;

[0046] Figure 8 This is a schematic diagram illustrating another application example of the model training method provided in the embodiments of this application;

[0047] Figure 9 A schematic diagram illustrating an application scenario of the target model provided in this application embodiment;

[0048] Figure 10 A flowchart illustrating the robot control method provided in this application embodiment;

[0049] Figure 11 A schematic diagram of the structure of the model training apparatus provided in the embodiments of this application;

[0050] Figure 12 A schematic diagram of the robot control device provided in the embodiments of this application;

[0051] Figure 13 A schematic diagram of the structure of the execution device provided in the embodiments of this application;

[0052] Figure 14 A schematic diagram of the structure of the training device provided in the embodiments of this application;

[0053] Figure 15 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0054] This application provides a model training method and related equipment, which can effectively reduce the time required for model training and improve the efficiency of model training.

[0055] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0056] Driven by the convergence and innovation of various technologies such as information technology, materials technology, and sensor technology, robots are becoming increasingly intelligent and flexible. Robot control models are an important component of robot control systems. Based on the control task, the robot control model can generate corresponding actions, thereby controlling the robot to execute those actions and complete the control task.

[0057] Currently, when training a robot control model, information about the target object to be controlled by the robot and the control task for that target object are first obtained, and this information is then input into the model to be trained. Next, the model processes this information to obtain predicted actions for completing the control task. Since the actual actions used to complete the control task can be obtained in advance, the parameters of the model can be updated based on the predicted and actual actions, resulting in an updated model. By repeating this process, the robot control model can be trained.

[0058] However, the actual actions used to complete the manipulation task often need to be designed by the operator according to the required manipulation task, and the robot needs to be instructed to perform these actual actions on the target object in advance so that the robot can correctly complete the manipulation task (predicted actions may not necessarily enable the robot to complete the manipulation task correctly). Therefore, collecting actual actions requires a significant amount of time, which leads to low model training efficiency.

[0059] To address the aforementioned problems, embodiments of this application provide a model training method and a robot manipulation method, which can be implemented in conjunction with artificial intelligence (AI) technology. AI technology is a discipline that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence. AI technology achieves optimal results by perceiving the environment, acquiring knowledge, and using that knowledge. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. Using artificial intelligence for data processing is a common application of AI.

[0060] First, the overall workflow of the artificial intelligence system is described; please refer to [link / reference]. Figure 1 , Figure 1 This is a structural diagram illustrating the main framework of artificial intelligence. The following explanation of the AI ​​framework is based on two dimensions: the "Intelligent Information Chain" (horizontal axis) and the "IT Value Chain" (vertical axis). The "Intelligent Information Chain" reflects a series of processes from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." The "IT Value Chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence and information (provided and processed by technology) to the industrial ecosystem of the system.

[0061] (1) Infrastructure

[0062] Infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. This communication occurs through sensors; computing power is provided by intelligent chips (hardware acceleration chips such as CPUs, NPUs, GPUs, ASICs, and FPGAs); and the basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0063] (2) Data

[0064] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0065] (3) Data processing

[0066] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0067] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0068] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0069] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0070] (4) General ability

[0071] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0072] (5) Smart Products and Industry Applications

[0073] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They are the encapsulation of overall artificial intelligence solutions, productizing intelligent information decision-making and realizing practical applications. Their application areas mainly include: intelligent terminals, intelligent transportation, intelligent healthcare, autonomous driving, smart cities, etc.

[0074] The following section introduces several application scenarios for this application.

[0075] Figure 2a This is a schematic diagram of a robot control system provided in an embodiment of this application. The robot control system includes user equipment and data processing equipment. The user equipment includes smart terminals such as mobile phones, personal computers, or information processing centers. The user equipment is the initiator of robot control requests; typically, the user initiates the request through the user equipment.

[0076] The aforementioned data processing equipment can be cloud servers, network servers, application servers, management servers, or other devices or servers with robot control capabilities. The data processing equipment receives robot control requests from intelligent terminals through an interactive interface, and then performs robot control through machine learning, deep learning, search, reasoning, and decision-making methods via a storage device that stores the data and a processor for robot control. The storage device in the data processing equipment can be a general term, including local storage and a database storing historical data. The database can be located on the data processing equipment or on other network servers.

[0077] exist Figure 2a In the robot control system shown, the user equipment can receive user instructions. For example, the user equipment can acquire the description information of the target object to be controlled and the description information of the control task for the target object, input / selected by the user. Then, it sends a request to the data processing device, causing the data processing device to perform a series of processes on the information from the user equipment, thereby obtaining a predicted action to complete the control task. For instance, the user equipment can acquire the description information of the target object and the description information of the control task input by the user. Then, the user equipment can send a robot control request to the data processing device, causing the data processing device to perform a series of processes on the description information of the target object and the description information of the control task based on the robot control request, thereby obtaining a predicted action to complete the control task, and controlling the robot to execute the predicted action on the target object, thus completing the control task.

[0078] exist Figure 2a In this process, the data processing device can execute the robot control method of the embodiments of this application.

[0079] Figure 2b This is another structural schematic diagram of the robot control system provided in the embodiments of this application. Figure 2b In this context, the user equipment (UE) directly functions as a data processing device. This UE can directly acquire input from the user and process it directly through its own hardware. The specific process is similar to... Figure 2a Similar to the description above, it will not be repeated here.

[0080] exist Figure 2b In the robot control system shown, the user equipment can receive user instructions. For example, the user equipment can obtain the description information of the target object and the description information of the control task for the target object input by the user. Then, the user equipment can perform a series of processing on the description information of the target object and the description information of the control task to obtain the predicted action for completing the control task, and control the robot to execute the predicted action on the target object to complete the control task.

[0081] exist Figure 2b In this application, the user equipment itself can execute the robot control method of the embodiments of this application.

[0082] Figure 2c This is a schematic diagram of a robot control device provided in an embodiment of this application.

[0083] The above Figure 2a and Figure 2b The user equipment in the context can specifically be Figure 2c Local device 301 or local device 302 in the system. Figure 2a The data processing equipment in the middle can specifically be Figure 2c The execution device 210 in the process includes a data storage system 250 that can store the data to be processed by the execution device 210. The data storage system 250 can be integrated into the execution device 210 or set up in the cloud or on other network servers.

[0084] Figure 2a and Figure 2b The processor in the system can perform data training / machine learning / deep learning through neural network models or other models (e.g., support vector machine-based models), and use the data to finally train or learn the model to perform robot control applications on information, thereby obtaining the corresponding processing results.

[0085] Figure 3 A schematic diagram of the system 100 architecture provided in this application embodiment, in Figure 3 In the process, the execution device 110 is configured with an input / output (I / O) interface 112 for data interaction with external devices. Users can input data to the I / O interface 112 through the client device 140. The input data in this embodiment may include various scheduled tasks, callable resources, and other parameters.

[0086] During the preprocessing of input data by the execution device 110, or during the calculation module 111 of the execution device 110 performing calculations and other related processing (such as implementing the function of the target model in this application), the execution device 110 may call data, code, etc. in the data storage system 150 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 150.

[0087] Finally, I / O interface 112 returns the processing result to client device 140, thereby providing it to the user.

[0088] It is worth noting that the training device 120 can generate corresponding target models / rules based on different training data for different objectives or tasks. These corresponding target models / rules can be used to achieve the aforementioned objectives or complete the aforementioned manipulation tasks, thereby providing the user with the required results. The training data can be stored in the database 130 and originates from training samples collected by the data acquisition device 160.

[0089] exist Figure 3 In the scenario shown, the user can manually provide input data, which can be done through the interface provided by I / O interface 112. Alternatively, the client device 140 can automatically send input data to I / O interface 112. If user authorization is required for the client device 140 to automatically send input data, the user can set the corresponding permissions in the client device 140. The user can view the output results of the execution device 110 on the client device 140, which can be presented in various forms such as display, sound, or animation. The client device 140 can also act as a data acquisition terminal, collecting the input data and output results of the input I / O interface 112 as new sample data and storing them in the database 130. Alternatively, data can be collected directly from the I / O interface 112 without going through the client device 140, using the input data and output results of the input I / O interface 112 as new sample data and storing them in the database 130.

[0090] It is worth noting that, Figure 3 This is merely a schematic diagram of a system architecture provided in an embodiment of this application. The positional relationships between the devices, components, modules, etc., shown in the diagram do not constitute any limitation. For example, in Figure 3 In this context, the data storage system 150 is an external memory relative to the execution device 110. However, in other cases, the data storage system 150 can also be placed within the execution device 110. For example... Figure 3 As shown, a neural network can be trained using training device 120.

[0091] This application also provides a chip including a neural network processor (NPU). This chip can be configured as follows: Figure 3 The execution device 110 shown is used to perform the calculations of the calculation module 111. This chip can also be located in, for example... Figure 3 The training device 120 shown is used to complete the training work of the training device 120 and output the target model / rules.

[0092] The Neural Processing Unit (NPU) is a coprocessor mounted on the main central processing unit (CPU) (host CPU), where tasks are assigned by the CPU. The core of the NPU is the computation circuitry, which is controlled by a controller to retrieve data from memory (weight memory or input memory) and perform calculations.

[0093] In some implementations, the arithmetic circuitry includes multiple process engines (PEs). In some implementations, the arithmetic circuitry is a two-dimensional pulsating array. The arithmetic circuitry can also be a one-dimensional pulsating array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuitry is a general-purpose matrix processor.

[0094] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory and caches it in each PE (Process Equipment) of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory and performs matrix operations with matrix B. The partial or final result of the obtained matrix is ​​stored in the accumulator.

[0095] Vector computation units can further process the output of computational circuits, such as vector multiplication, vector addition, exponentiation, logarithmic operations, size comparisons, etc. For example, vector computation units can be used for computation in non-convolutional / non-FC layers of neural networks, such as pooling, batch normalization, and local response normalization.

[0096] In some implementations, the vector computation unit can store the processed output vector into a unified buffer. For example, the vector computation unit can apply a nonlinear function to the output of the arithmetic circuit, such as a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit generates normalized values, merged values, or both. In some implementations, the processed output vector can be used as activation input to the arithmetic circuit, for example, for use in subsequent layers of a neural network.

[0097] The unified memory is used to store input data and output data.

[0098] The weight data is directly transferred from the external memory to the input memory and / or unified memory, stored in the weight memory, and stored in the unified memory to the external memory through the direct memory access controller (DMAC).

[0099] The bus interface unit (BIU) is used to enable interaction between the main CPU, DMAC, and instruction fetch memory via a bus.

[0100] The instruction fetch buffer, connected to the controller, is used to store the instructions used by the controller.

[0101] The controller is used to invoke instructions cached in the memory to control the operation of the computing accelerator.

[0102] Generally, the unified memory, input memory, weight memory, and instruction fetch memory are all on-chip memories, while external memory is memory outside the NPU. This external memory can be double data rate synchronous dynamic random access memory (DDRSDRAM), high bandwidth memory (HBM), or other readable and writable memories.

[0103] Since the embodiments of this application involve a large number of neural network applications, for ease of understanding, the relevant terms and concepts such as neural networks involved in the embodiments of this application will be introduced below.

[0104] (1) Neural Network

[0105] A neural network can be composed of neural units, which can be operational units that take xs and an intercept of 1 as inputs, and whose output can be:

[0106]

[0107] Where s = 1, 2, ..., n, where n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of this activation function can be used as the input of the next convolutional layer. The activation function can be the sigmoid function. A neural network is a network formed by connecting many of the above-mentioned individual neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.

[0108] The work of each layer in a neural network can be described by the mathematical expression y = a(Wx + b). From a physical perspective, the work of each layer in a neural network can be understood as transforming the input space (the set of input vectors) to the output space (i.e., from the row space to the column space of a matrix) through five operations on the input space. These five operations include: 1. Dimensionality increase / decrease; 2. Magnification / scaling; 3. Rotation; 4. Translation; 5. "Bending". Operations 1, 2, and 3 are performed by Wx, operation 4 by +b, and operation 5 by a(). The term "space" is used here because the objects being classified are not individual things, but a class of things, and space refers to the set of all individuals of this class of things. Here, W is the weight vector, and each value in this vector represents the weight value of a neuron in that layer of the neural network. This vector W determines the spatial transformation from the input space to the output space mentioned above; that is, the weights W of each layer control how the space is transformed. The purpose of training a neural network is to ultimately obtain the weight matrix of all layers of the trained neural network (a weight matrix formed by the vectors W of many layers). Therefore, the training process of a neural network is essentially about learning how to control the transformation space, and more specifically, learning the weight matrix.

[0109] Because we want the output of the neural network to be as close as possible to the actual predicted value, we can compare the current network's prediction with the desired target value, and then update the weight vector of each layer of the neural network based on the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring the parameters of each layer in the neural network). For example, if the network's prediction is too high, the weight vector is adjusted to make it predict lower, and this adjustment is continued until the neural network can predict the actual target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value," which is the loss function or objective function. These are important equations used to measure the difference between the predicted value and the target value. Taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so training the neural network becomes the process of minimizing this loss as much as possible.

[0110] (2) Backpropagation algorithm

[0111] Neural networks can employ backpropagation (BP) to correct the parameters of the initial neural network model during training, thereby reducing the reconstruction error loss. Specifically, forward propagation of the input signal to the output generates error loss; this error loss information is then propagated back to update the parameters of the initial neural network model, leading to convergence of the error loss. The backpropagation algorithm is an error-loss-driven backpropagation process aimed at obtaining the optimal parameters of the neural network model, such as the weight matrix.

[0112] (3) Robot manipulation

[0113] By generating motion commands and sending them to the robot, the robot can perform corresponding actions to change the state of the environment or objects. The process of generating motion commands can be implemented on the user side (e.g., user devices, etc.) or on a remote side (e.g., cloud servers, etc.), both of which act as the controller of the robot.

[0114] The method provided in this application is described below from the perspectives of neural network training and neural network application.

[0115] The model training method provided in this application involves data sequence processing and can be applied to data training, machine learning, deep learning, and other methods. It performs symbolic and formal intelligent information modeling, extraction, preprocessing, and training on training data (e.g., the description information of the target object and the real description information of the manipulation task in the model training method provided in this application), ultimately obtaining a trained neural network (e.g., the target model in the model training method provided in this application). Furthermore, the robot manipulation method provided in this application can utilize the trained neural network to input input data (e.g., the description information of the target object and the description information of the manipulation task in the robot manipulation method provided in this application) into the trained neural network to obtain output data (e.g., the predicted actions used to complete the manipulation task in the robot manipulation method provided in this application). It should be noted that the model training method and the robot manipulation method provided in this application are inventions based on the same concept and can be understood as two parts of a system or two stages of an overall process: such as the model training stage and the model application stage.

[0116] The model training method provided in this application can be used to train a model to be trained, such as... Figure 4 As shown ( Figure 4This is a schematic diagram of the structure of the model to be trained provided in an embodiment of this application. The model to be trained (the robot control model to be trained) includes three trained neural network models and one neural network model to be trained. The first trained neural network model is a trained visual basic model, the second trained neural network model is a trained 3D basic model, the third trained neural network model is a trained multimodal basic model, and the neural network model to be trained is a motion command generation model to be trained. The input terminals of the trained visual basic model, the trained 3D basic model, and the trained multimodal basic model serve as the input terminals of the entire model to be trained. The output terminal of the trained visual basic model is connected to the first input terminal of the motion command generation model to be trained, the output terminal of the trained 3D basic model is connected to the second input terminal of the motion command generation model to be trained, the output terminal of the trained multimodal basic model is connected to the third input terminal of the motion command generation model to be trained, and the output terminal of the motion command generation model to be trained serves as the output terminal of the entire model to be trained.

[0117] To further understand the training process for the model to be trained, the following section combines... Figure 5 The training process will be described in more detail. Figure 5 This is a flowchart illustrating a model training method provided in an embodiment of this application. It should be noted that the model training method provided in this application can be implemented using a training device. The training samples and other information required by the training device can be collected by a user device in conjunction with a robot, which will not be elaborated further below. Figure 5 As shown, the method includes:

[0118] 501. Obtain the description information of the target object, as well as the actual description information of the manipulation task targeting the target object.

[0119] In this embodiment, when it is necessary to train the model to be trained, a batch of training samples can be collected first. The training samples can be collected in the following ways: (1) The user equipment controls the robot to move to the vicinity of the target object to be operated, and then controls the robot to collect information on the target object so that the robot collects and returns the description information of the target object to the user equipment. Therefore, the description information of the target object input by the user equipment can be received. (2) The user equipment receives the real description information of the manipulation task for the target object. The manipulation task for the target object is the manipulation task that the robot needs to complete on the target object.

[0120] Specifically, the descriptive information of the target object (e.g., an environment or an object in that environment) may include at least one of the following: (1) two-dimensional data of the target object, such as images of the target object, videos of the target object, and two-dimensional key points of the target object. (2) three-dimensional data of the target object, such as point cloud data of the target object, and three-dimensional key points of the target object.

[0121] More specifically, the actual description information of the control task can be either a single-modal demonstration of the control task input by the user device or a multi-modal demonstration of the control task input by the user device. The actual description information of the control task can include at least one of the following: (1) A textual description of the control task, for example, the control task is presented as text input by the user device, which includes the following text: "Grab the object in the upper left corner of the desktop," etc. (2) An image-based description of the control task, for example, the control task is presented as an image input by the user device, which includes the following: "The desktop and multiple objects on the desktop, wherein the object in the upper left corner of the desktop is marked with a special symbol," etc. (3) An audio-based description of the control task, for example, the control task is presented as audio input by the user device, which, after playback, includes the following: "Pick up the object located in the upper left corner of the desktop," etc. (4) The actual description of the control task in the form of an instruction. For example, the control task is presented as an instruction input by the user device. Suppose the user device displays an interface to the user, and the content of the interface shows the desktop and multiple objects on the desktop. The instruction is the touch or click operation performed by the user on the object in the upper left corner of the interface, etc.

[0122] 502. The model to be trained processes the description information of the target object and the real description information of the control task to obtain the predicted action for completing the control task.

[0123] After obtaining the description information of the target object and the actual description information of the control task, the description information of the target object and the actual description information of the control task can be input into the model to be trained. The model to be trained will then perform a series of processing on the description information of the target object and the actual description information of the control task to obtain the predicted actions for the robot to complete the control task.

[0124] Specifically, the model to be trained can obtain the predicted actions used to complete the manipulation task in the following ways:

[0125] (1) After inputting the description information of the target object and the real description information of the manipulation task into the model to be trained, the trained visual basic model and the trained 3D basic model can extract features from the description information of the target object to obtain the features of the target object, and send the features of the target object to the action command generation model to be trained. At the same time, the trained multimodal basic model can extract features from the real description information of the manipulation task to obtain the features of the manipulation task, and send the features of the manipulation task to the action command generation model to be trained.

[0126] For example, such as Figure 6 As shown ( Figure 6 (This is a schematic diagram of an application example of the model training method provided in the embodiments of this application). Suppose that a model needs to be trained to generate action instructions. First, the robot can obtain two-dimensional data and three-dimensional data of the desktop, as well as multimodal demonstration information (including text, images, audio and instructions used to describe the control task) input by the user. The control task is to grasp the object in the upper left corner of the desktop.

[0127] Therefore, 2D data of the desktop can be input into a trained visual baseline model to obtain the visual features of the desktop, which are then sent to the motion command generation model to be trained. Similarly, 3D data of the desktop can be input into a trained 3D baseline model to obtain the 3D features of the desktop, which are then sent to the motion command generation model to be trained. Likewise, multimodal demonstration information of the manipulation task can be input into a trained multimodal baseline model to obtain the features of the manipulation task, which are then sent to the motion command generation model to be trained.

[0128] (2) After obtaining the features of the target object and the features of the control task, the action instruction generation model to be trained can further process the features of the target object and the features of the control task to generate predicted actions for completing the control task.

[0129] As in the example above, after obtaining the visual features of the desktop, the 3D features of the desktop, and the features of the control task, the action command generation model to be trained can process the visual features of the desktop, the 3D features of the desktop, and the features of the control task to obtain the predicted actions for the robot to complete the control task.

[0130] More specifically, the action instruction generation model to be trained can obtain predicted actions for completing the manipulation task in the following ways:

[0131] (1) After obtaining the features of the target object and the features of the control task, the action command generation model to be trained can first perform feature fusion on the features of the target object and the features of the control task to obtain the feature fusion result (which can be regarded as a whole feature). The feature fusion result is used to indicate the comprehensive understanding of the target object and the control task by the action command generation model to be trained.

[0132] (2) After obtaining the feature fusion result, the action command generation model to be trained can compress (predict) the feature fusion result to obtain multiple sub-actions that are different from each other.

[0133] (3) After obtaining multiple sub-actions, the action instruction generation model to be trained can superimpose the multiple sub-actions to obtain the predicted action used to complete the control task.

[0134] As in the example above, such as Figure 7 As shown ( Figure 7 This is a schematic diagram of the action instruction generation model provided in the embodiments of this application. Figure 7 Is Figure 6 Based on the data obtained from the desktop (drawn), the motion command generation model to be trained includes a feature fusion module, a compression module, and an overlay module. After obtaining the visual features, 3D features, and features of the control task, the feature fusion module processes these features to obtain a feature fusion result, which is then sent to the compression module. The compression module compresses the feature fusion result to obtain multiple basic actions (sub-actions), including rotation of XX degrees and translation of YY meters, and sends these to the overlay module. Finally, the overlay module combines these basic actions into a final predicted action, such as rotating the robot by XX degrees and translating it by YY meters.

[0135] 503. Perform predictive actions on the target object to obtain predictive description information of the control task.

[0136] After obtaining the predicted actions to complete the manipulation task, the robot can be controlled to execute the predicted actions on the target object, thereby changing the state of both the robot and the target object. At this point, the robot has completed the manipulation task on the target object (although the robot may not have accurately completed the task). The user equipment can then observe the state of the robot and the target object from a third-party perspective to generate a predicted description of the manipulation task based on these states. Therefore, it can receive the predicted description of the manipulation task input by the user equipment.

[0137] Specifically, the predictive description information of the control task can be either single-modal observation information about the control task input by the user device or multi-modal observation information about the control task input by the user device. Therefore, the predictive description information of the control task can include at least one of the following: (1) Predictive description information in text form, for example, the control task is presented as text input by the user device, which includes the following text: After the robot performs the predicted action facing the table, the robot's manipulator just picked up the object in the middle of the table, etc. (2) Predictive description information in image form, for example, the control task is presented as an image input by the user device, which includes the following: The positions of each joint of the robot after performing the predicted action are marked with special symbols, indicating that the robot's manipulator picked up the object in the middle of the table, etc. (3) Predictive description information in audio form, for example, the control task is presented as audio input by the user device, which, after playback, includes the following: After the robot performs the predicted action facing the table, the robot's manipulator picked up the object located in the middle of the table, etc. (4) Predictive description information in the form of instructions for the control task. For example, the control task is presented as a certain instruction input by the user device. Suppose the user device displays an interface to the user, and the content of the interface shows the robot, the desktop and multiple objects on the desktop. The instruction is the touch or click operation performed by the user on the interface on the robot after the predicted action is executed, etc.

[0138] 504. Based on the predicted description information of the manipulation task and the real description information of the manipulation task, the model to be trained is trained to obtain the target model.

[0139] After obtaining the predicted description information of the control task, the predicted description information and the actual description information of the control task can be used to train the model to be trained, thereby obtaining the target model (the trained robot control model).

[0140] Specifically, the target model can be trained in the following ways:

[0141] (1) After obtaining the predicted description information of the control task, the predicted description information and the actual description information of the control task are calculated by a preset loss function to obtain the target loss. The target loss is used to indicate the difference between the predicted description information and the actual description information of the control task.

[0142] (2) After obtaining the target loss, the parameters of the action command generation model to be trained can be updated based on the target loss to obtain the action command generation model with updated parameters. Then, the next batch of training samples is used to continue training the action command generation model with updated parameters until the model training conditions are met (e.g., the target loss converges), thus obtaining the trained action command generation model. Then, the trained visual basic model, the trained 3D basic model, the trained multimodal basic model, and the trained action command generation model constitute the target model.

[0143] To further understand the model training method provided in the embodiments of this application, the method will be further described below with reference to a specific application example. For example... Figure 8 As shown ( Figure 8 (This is a schematic diagram illustrating another application example of the model training method provided in the embodiments of this application), the application example includes:

[0144] (1) Acquire multimodal demonstration information about the control task input by the user through the user device and environmental data collected by the robot;

[0145] (2) The multimodal basic model extracts the features of the control task based on the multimodal demonstration information of the control task, while the visual basic model and the 3D basic model extract the features of the environment based on the environmental data.

[0146] (3) The motion instruction generation model generates predicted actions based on the characteristics of the control task and the characteristics of the environment, and the robot executes the predicted actions.

[0147] (4) Users observe the state of the robot from a third-person perspective through user devices, that is, the position of each joint of the robot, in order to generate multimodal observation information for the control task.

[0148] (5) Obtain multimodal observation information of the user-input control task, train the action command generation model, and obtain the trained action command generation model. Then, the multimodal basic model, the visual basic model, the 3D basic model and the trained action command generation model constitute the target model, which is the trained robot control model.

[0149] In addition, such as Figure 9 As shown ( Figure 9(A schematic diagram of the application scenario of the target model provided in the embodiments of this application). The target model trained in the embodiments of this application can be applied to various scenarios such as controlling home service robots, maintenance robots, inspection robots, and warehouse robots, thereby improving human-machine collaboration efficiency, saving labor costs, and enhancing user experience. For example, in the home robot scenario, it can assist humans or automatically complete daily tedious housework. As another example, in the inspection robot scenario, it can automatically perform inspection tasks, reduce the intensity of personnel inspection, and improve efficiency and safety. In addition, unmanned operation tasks in other industries include: (1) Supply chain: equipment asset inventory; (2) Computer room: replacing hard drives, plugging and unplugging network cables, replacing single boards, and picking and placing accessories; (3) Base station construction: installation of baseband control unit (BBU) / single board, installation of active antenna unit (AAU) antenna, and plugging and unplugging cables; (4) Manufacturing: opening and closing test of folding equipment; (5) Power: button operation of maintenance robot; (6) Energy: plugging and unplugging operation of charging pile of charging robot, etc.

[0150] In this embodiment, when training the model to be trained is required, the description information of the target object that the robot is about to operate on, and the actual description information of the control task that the robot needs to complete on the target object, can be obtained first. The description information of the target object and the actual description information of the control task are then input into the model to be trained. Next, the model to be trained can perform a series of processes on the description information of the target object and the actual description information of the control task to obtain the predicted action for the robot to complete the control task. Then, the robot can be controlled to perform the predicted action on the target object to obtain the predicted description information of the control task. Finally, based on the predicted description information and the actual description information of the control task, the model to be trained can be trained to obtain the target model, which is the trained robot control model. In the aforementioned process, after the model to be trained obtains the predicted action based on the real description information of the manipulation task (i.e., the demonstration information of the manipulation task), the robot can be instructed to perform the predicted action on the target object, thereby observing the state of the target object and the state of the robot to obtain the predicted description information of the manipulation task (i.e., the observation information of the manipulation task). In this way, the real description information of the manipulation task can be directly used as the label of the predicted description information of the manipulation task, and the model to be trained can be trained in a self-supervised training manner to obtain the target model. It is not necessary to use robots to collect training samples on a large scale in advance, which can effectively reduce the time required for model training and improve the efficiency of model training.

[0151] Furthermore, in this embodiment, the input to the model to be trained is multimodal demonstration information (real description information) about the manipulation task. This enables the model to more efficiently and accurately understand and learn new skills of the user in manipulating the target object using the robot, thus the trained target model has better performance.

[0152] The above is a detailed description of the model training method provided in the embodiments of this application. The robot control method provided in the embodiments of this application will be described below. Figure 10 A flowchart illustrating the robot control method provided in this application embodiment is shown below. Figure 10 As shown, the method includes:

[0153] 1001. Obtain the description information of the target object and the description information of the manipulation tasks targeting the target object.

[0154] In this embodiment, when it is necessary to control the robot to operate on a target object in order to complete the manipulation task, the description information of the target object and the description information of the manipulation task for the target object can be obtained first.

[0155] 1002. The target model is used to process the description information of the target object and the description information of the control task to obtain the predicted action for completing the control task.

[0156] After obtaining the description information of the target object and the description information of the control task, the description information of the target object and the description information of the control task can be input into... Figure 5 The target model obtained in the embodiment shown is used to process the description information of the target object and the description information of the control task through a series of processes, thereby obtaining the predicted action for completing the control task.

[0157] 1003. Perform the predicted action on the target object.

[0158] Once the predicted actions for completing the manipulation task are obtained, the robot can be controlled to perform the predicted actions on the target object, thereby successfully completing the manipulation task.

[0159] The above is a detailed description of the model training method and robot control method provided in the embodiments of this application. The model training device and robot control device provided in the embodiments of this application will be described below. Figure 11 A schematic diagram of the model training apparatus provided in the embodiments of this application is shown below. Figure 11 As shown, the device includes:

[0160] The acquisition module 1101 is used to acquire the description information of the target object and the actual description information of the manipulation task for the target object;

[0161] The processing module 1102 is used to process the description information of the target object and the real description information of the control task through the model to be trained, so as to obtain the predicted action for completing the control task.

[0162] The execution module 1103 is used to perform predictive actions on the target object in order to obtain predictive description information of the control task;

[0163] Training module 1104 is used to train the model to be trained based on the predicted description information of the manipulation task and the real description information of the manipulation task, so as to obtain the target model.

[0164] In this embodiment, when training the model to be trained is required, the description information of the target object that the robot is about to operate on, and the actual description information of the control task that the robot needs to complete on the target object, can be obtained first. The description information of the target object and the actual description information of the control task are then input into the model to be trained. Next, the model to be trained can perform a series of processes on the description information of the target object and the actual description information of the control task to obtain the predicted action for the robot to complete the control task. Then, the robot can be controlled to perform the predicted action on the target object to obtain the predicted description information of the control task. Finally, based on the predicted description information and the actual description information of the control task, the model to be trained can be trained to obtain the target model, which is the trained robot control model. In the aforementioned process, after the model to be trained obtains the predicted action based on the real description information of the manipulation task (i.e., the demonstration information of the manipulation task), the robot can be instructed to perform the predicted action on the target object. This allows for the observation of the target object's state and the robot's state to obtain the predicted description information of the manipulation task (i.e., the observation information of the manipulation task). In this way, the real description information of the manipulation task can be directly used as the label for the predicted description information of the manipulation task, enabling self-supervised training of the model to obtain the target model. This eliminates the need for large-scale pre-collection of training samples using robots, effectively reducing the time required for model training and improving its efficiency.

[0165] In one possible implementation, the processing module is configured to: extract features from the description information of the target object and the real description information of the manipulation task using the model to be trained, to obtain the features of the target object and the features of the manipulation task; and generate a predicted action for completing the manipulation task based on the features of the target object and the features of the manipulation task using the model to be trained.

[0166] In one possible implementation, the processing module is configured to: fuse the features of the target object and the features of the manipulation task using the model to be trained to obtain a feature fusion result; compress the feature fusion result using the model to be trained to obtain multiple distinct sub-actions; and superimpose the multiple sub-actions using the model to be trained to obtain a predicted action for completing the manipulation task.

[0167] In one possible implementation, the training module is used to: calculate the target loss by combining the predicted description information of the manipulation task with the actual description information of the manipulation task, the target loss being used to indicate the difference between the predicted description information of the manipulation task and the actual description information of the manipulation task; and update the parameters of the model to be trained based on the target loss until the model training conditions are met, thereby obtaining the target model.

[0168] In one possible implementation, the descriptive information of the target object includes at least one of the following: two-dimensional data of the target object and three-dimensional data of the target object.

[0169] In one possible implementation, the real description information of the control task includes at least one of the following: real description information of the control task in text form, real description information of the control task in image form, real description information of the control task in audio form, and real description information of the control task in instruction form.

[0170] In one possible implementation, the predictive description information for the manipulation task includes at least one of the following: predictive description information for the manipulation task in text form, predictive description information for the manipulation task in image form, predictive description information for the manipulation task in audio form, and predictive description information for the manipulation task in instruction form.

[0171] Figure 12 A schematic diagram of the robot control device provided in the embodiments of this application is shown below. Figure 12 As shown, the device includes:

[0172] The acquisition module 1201 is used to acquire description information of the target object and description information of the manipulation task for the target object.

[0173] The processing module 1202 is used to process the description information of the target object and the description information of the control task through the target model, so as to obtain the predicted action for completing the control task.

[0174] Execution module 1203 is used to perform prediction actions on the target object.

[0175] It should be noted that the information interaction and execution process between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of this application, and the resulting technical effects are the same as those of the method embodiment of this application. For details, please refer to the description in the method embodiment shown above in the embodiment of this application, and it will not be repeated here.

[0176] This application also relates to an execution device. Figure 13 This is a schematic diagram of the execution device provided in an embodiment of this application. Figure 13 As shown, the execution device 1300 can specifically manifest as a mobile phone, tablet, laptop, smart wearable device, server, etc., and is not limited here. Among them, the execution device 1300 may deploy... Figure 12 The robot control device described in the corresponding embodiment is used to achieve Figure 10 The corresponding embodiment describes the robot control function. Specifically, the execution device 1300 includes: a receiver 1301, a transmitter 1302, a processor 1303, and a memory 1304 (wherein the execution device 1300 may have one or more processors 1303). Figure 13 (Taking a processor as an example), processor 1303 may include application processor 13031 and communication processor 13032. In some embodiments of this application, receiver 1301, transmitter 1302, processor 1303 and memory 1304 may be connected via bus or other means.

[0177] Memory 1304 may include read-only memory and random access memory, and provides instructions and data to processor 1303. A portion of memory 1304 may also include non-volatile random access memory (NVRAM). Memory 1304 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0178] Processor 1303 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together through a bus system, which may include not only the data bus, but also power buses, control buses, and status signal buses. However, for clarity, all buses are referred to as the bus system in the diagram.

[0179] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1303. The processor 1303 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 1303 or by instructions in software form. The processor 1303 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and may further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1303 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 1304. Processor 1303 reads the information in memory 1304 and, in conjunction with its hardware, completes the steps of the above method.

[0180] Receiver 1301 can be used to receive input digital or character information, and to generate signal inputs related to the settings and function control of the execution device. Transmitter 1302 can be used to output digital or character information through the first interface; transmitter 1302 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; transmitter 1302 may also include a display device such as a display screen.

[0181] In one embodiment of this application, the processor 1303 is used to... Figure 10 The robot is controlled to perform actions on the target object according to the target model in the corresponding embodiment in order to complete the manipulation task.

[0182] This application also relates to a training device. Figure 14 This is a schematic diagram of the structure of a training device provided in an embodiment of this application. Figure 14As shown, the training device 1400 is implemented by one or more servers. The training device 1400 can vary significantly due to differences in configuration or performance, and may include one or more central processing units (CPUs) 1414 (e.g., one or more processors) and memory 1432, and one or more storage media 1430 (e.g., one or more mass storage devices) for storing application programs 1442 or data 1444. The memory 1432 and storage media 1430 can be temporary or persistent storage. The program stored in the storage media 1430 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the training device. Furthermore, the CPU 1414 may be configured to communicate with the storage media 1430 and execute the series of instruction operations in the storage media 1430 on the training device 1400.

[0183] The training device 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input / output interfaces 1458; or, one or more operating systems 1441, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0184] Specifically, the training equipment can perform Figure 5 The model training method in the corresponding embodiment is used to obtain the target model.

[0185] This application also relates to a computer storage medium storing a program for signal processing, which, when run on a computer, causes the computer to perform steps as performed by the aforementioned execution device, or causes the computer to perform steps as performed by the aforementioned training device.

[0186] This application also relates to a computer program product that stores instructions that, when executed by a computer, cause the computer to perform steps as performed by the aforementioned execution device, or to perform steps as performed by the aforementioned training device.

[0187] The execution device, training device, or terminal device provided in this application embodiment can specifically be a chip. The chip includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip within the execution device to execute the data processing method described in the above embodiments, or to cause the chip within the training device to execute the data processing method described in the above embodiments. Optionally, the storage unit can be a storage unit within the chip, such as a register or cache. Alternatively, the storage unit can be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0188] For details, please refer to Figure 15 , Figure 15 This is a schematic diagram of the chip provided in an embodiment of this application. The chip can be represented as a neural network processor (NPU) 1500. The NPU 1500 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core part of the NPU is the arithmetic circuit 1503, which is controlled by the controller 1504 to extract matrix data from the memory and perform multiplication operations.

[0189] In some implementations, the arithmetic circuit 1503 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 1503 is a two-dimensional pulsating array. The arithmetic circuit 1503 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1503 is a general-purpose matrix processor.

[0190] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 1502 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 1501 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 1508.

[0191] Unified memory 1506 is used to store input and output data. Weight data is directly transferred to weight memory 1502 via Direct Memory Access Controller (DMAC) 1505. Input data is also transferred to unified memory 1506 via DMAC.

[0192] BIU stands for Bus Interface Unit, which is used for interaction between the AXI bus and the DMAC and the Instruction Fetch Buffer (IFB) 1509.

[0193] The Bus Interface Unit (BIU) 1513 is used by the instruction fetch memory 1509 to fetch instructions from external memory, and also by the memory access controller 1505 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0194] The DMAC is mainly used to move input data from external memory DDR to unified memory 1506, or to weight data to weight memory 1502, or to input data to input memory 1501.

[0195] The vector computation unit 1507 includes multiple processing units that further process the output of the computation circuit 1503 when necessary, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is mainly used for computation in non-convolutional / fully connected layers of neural networks, such as Batch Normalization, pixel-level summation, and upsampling of the predicted label plane.

[0196] In some implementations, the vector computation unit 1507 can store the processed output vector in the unified memory 1506. For example, the vector computation unit 1507 can apply a linear function, or a nonlinear function, to the output of the computation circuit 1503, such as linearly interpolating the predicted label plane extracted from the convolutional layer, or, for example, accumulating a vector of values ​​to generate activation values. In some implementations, the vector computation unit 1507 generates normalized values, pixel-level summed values, or both. In some implementations, the processed output vector can be used as an activation input to the computation circuit 1503, for example, for use in subsequent layers of the neural network.

[0197] The instruction fetch buffer 1509 connected to the controller 1504 is used to store the instructions used by the controller 1504;

[0198] Unified memory 1506, input memory 1501, weighted memory 1502, and instruction fetch memory 1509 are all on-chip memories. External memory is proprietary to this NPU hardware architecture.

[0199] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of the above program.

[0200] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0201] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0202] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0203] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A model training method, characterized in that, The method includes: Obtain descriptive information about the target object, as well as actual descriptive information about the manipulation tasks targeting the target object; The model to be trained processes the description information of the target object and the real description information of the control task to obtain the predicted action for completing the control task. The prediction action is performed on the target object to obtain the prediction description information of the manipulation task; Based on the predicted description information of the control task and the actual description information of the control task, the model to be trained is trained to obtain the target model. The step of processing the description information of the target object and the real description information of the manipulation task through the model to be trained to obtain the predicted action for completing the manipulation task includes: The features of the target object and the features of the control task are obtained by extracting features from the description information of the target object and the real description information of the control task through the model to be trained. The model to be trained generates a predicted action for completing the control task based on the features of the target object and the features of the control task.

2. The method according to claim 1, characterized in that, The step of generating a predicted action for completing the manipulation task based on the features of the target object and the features of the manipulation task using the model to be trained includes: The feature fusion result is obtained by fusing the features of the target object and the features of the manipulation task using the model to be trained. The feature fusion result is compressed using the model to be trained to obtain multiple distinct sub-actions; The multiple sub-actions are superimposed using the model to be trained to obtain the predicted action used to complete the control task.

3. The method according to claim 1 or 2, characterized in that, The process of training the model to be trained based on the predicted description information and the actual description information of the control task to obtain the target model includes: The predicted description information and the actual description information of the control task are calculated to obtain the target loss, which is used to indicate the difference between the predicted description information and the actual description information of the control task. The parameters of the model to be trained are updated based on the target loss until the model training conditions are met, thus obtaining the target model.

4. The method according to claim 1 or 2, characterized in that, The description information of the target object includes at least one of the following: two-dimensional data of the target object and three-dimensional data of the target object.

5. The method according to claim 1 or 2, characterized in that, The true description information of the control task includes at least one of the following: the true description information of the control task in text form, the true description information of the control task in image form, the true description information of the control task in audio form, and the true description information of the control task in instruction form.

6. The method according to claim 1 or 2, characterized in that, The predicted description information of the control task includes at least one of the following: predicted description information of the control task in text form, predicted description information of the control task in image form, predicted description information of the control task in audio form, and predicted description information of the control task in instruction form.

7. A model training device, characterized in that, The device includes: The acquisition module is used to acquire description information of the target object, as well as actual description information of the manipulation task for the target object; The processing module is used to process the description information of the target object and the real description information of the control task through the model to be trained, so as to obtain the predicted action for completing the control task. An execution module is used to perform the prediction action on the target object to obtain the prediction description information of the manipulation task; The training module is used to train the model to be trained based on the predicted description information of the control task and the actual description information of the control task, so as to obtain the target model. The processing module is used for: The features of the target object and the features of the control task are obtained by extracting features from the description information of the target object and the real description information of the control task through the model to be trained. The model to be trained generates a predicted action for completing the control task based on the features of the target object and the features of the control task.

8. The apparatus according to claim 7, characterized in that, The processing module is used for: The feature fusion result is obtained by fusing the features of the target object and the features of the manipulation task using the model to be trained. The feature fusion result is compressed using the model to be trained to obtain multiple distinct sub-actions; The multiple sub-actions are superimposed using the model to be trained to obtain the predicted action used to complete the control task.

9. The apparatus according to claim 7 or 8, characterized in that, The training module is used for: The predicted description information and the actual description information of the control task are calculated to obtain the target loss, which is used to indicate the difference between the predicted description information and the actual description information of the control task. The parameters of the model to be trained are updated based on the target loss until the model training conditions are met, thus obtaining the target model.

10. The apparatus according to claim 7 or 8, characterized in that, The description information of the target object includes at least one of the following: two-dimensional data of the target object and three-dimensional data of the target object.

11. The apparatus according to claim 7 or 8, characterized in that, The true description information of the control task includes at least one of the following: the true description information of the control task in text form, the true description information of the control task in image form, the true description information of the control task in audio form, and the true description information of the control task in instruction form.

12. The apparatus according to claim 7 or 8, characterized in that, The predicted description information of the control task includes at least one of the following: predicted description information of the control task in text form, predicted description information of the control task in image form, predicted description information of the control task in audio form, and predicted description information of the control task in instruction form.

13. A model training device, characterized in that, The device includes a memory and a processor; the memory stores code, and the processor is configured to execute the code, wherein when the code is executed, the model training device performs the method as described in any one of claims 1 to 6.

14. A computer storage medium, characterized in that, The computer storage medium stores one or more instructions that, when executed by one or more computers, cause the one or more computers to perform the method of any one of claims 1 to 6.

15. A computer program product, characterized in that, The computer program product stores instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1 to 6.

Images