Industrial scene humanoid robot task execution method and related device

Abstract

The application discloses an industrial scene humanoid robot task execution method and related equipment, and belongs to the technical field of industrial robots. The method comprises the following steps: in response to a task issuing request, obtaining a task instruction to be executed; extracting a complexity parameter of the task instruction, and determining a target task processing model according to the complexity parameter; decomposing the task instruction into a subtask sequence by using the target task processing model, and generating an execution parameter for each subtask; controlling the humanoid robot to execute the task based on the subtask sequence and the execution parameter, and outputting a result; wherein the complexity parameter comprises the number of task steps and the accuracy requirement level; and the target task processing model comprises a multi-modal fusion execution model and a single-process execution model. By extracting the number of task steps and the accuracy requirement level, the optimal processing model is dynamically matched, the task execution efficiency and accuracy of multiple humanoid robots are effectively improved, and the use demand in the industrial scene is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial robots, and in particular to an industrial scene humanoid robot task execution method and related equipment. BACKGROUND

[0002] Currently, in industrial automation production, industrial humanoid robots need to complete diversified tasks such as carrying, assembling, and detecting.

[0003] In related technologies, the task execution mode of the robot is usually a fixed process: that is, according to a preset program to execute actions to process a single type or a fixed complexity task. However, in actual applications, it is found that the adaptability and execution accuracy of the traditional robot task method in the industrial scene are limited.

[0004] To sum up, the technical problems existing in related technologies need to be improved. SUMMARY

[0005] The embodiments of the present application provide an industrial scene humanoid robot task execution method and related equipment, which can effectively improve the task execution efficiency and accuracy of the humanoid robot and meet the use requirements in the industrial scene.

[0006] In one aspect, the embodiments of the present application provide an industrial scene humanoid robot task execution method, which comprises the following steps:

[0007] In response to a task issuing request, a task instruction to be executed is acquired;

[0008] A complexity parameter of the task instruction is extracted, and a target task processing model is determined according to the complexity parameter;

[0009] The task instruction is decomposed into a subtask sequence by using the target task processing model, and an execution parameter is generated for each subtask;

[0010] Based on the subtask sequence and the execution parameter of each subtask, the humanoid robot is controlled to execute the task, and a task result is output;

[0011] The complexity parameter includes the number of task steps and the accuracy requirement level; and the target task processing model includes a multi-modal fusion execution model and a single-process execution model.

[0012] Optionally, based on the subtask sequence and the execution parameter of each subtask, the humanoid robot is controlled to execute the task, and a task result is output, which comprises:

[0013] Based on the subtask sequence and the execution parameter of each subtask, the humanoid robot is controlled to execute the task, and environment feedback data in the execution task process is acquired in real time;

[0014] based on the environmental feedback data, dynamically adjusting the subtask sequence and the execution parameter of each subtask;

[0015] based on the updated subtask sequence and the execution parameter of each subtask, adjusting the humanoid robot to execute the task and outputting a task result.

[0016] Optionally, the complexity parameter of the task instruction is extracted, and a target task processing model is determined according to the complexity parameter, including:

[0017] The task instruction is semantically parsed to extract a task target and associated device information, and then the number of task steps of the task instruction is determined;

[0018] Based on a preset industrial scene database, the precision level corresponding to the task target is matched and determined, so as to determine the precision requirement level of the task instruction;

[0019] According to the number of task steps and the precision requirement level of the task instruction, a target task processing model is determined.

[0020] Optionally, the target task processing model is determined according to the number of task steps and the precision requirement level of the task instruction, including:

[0021] In the case where the number of task steps is greater than or equal to a first preset threshold or the precision requirement level is greater than or equal to a second preset threshold, the target task processing model is determined as the multi-modal fusion execution model;

[0022] In the case where the number of task steps is less than the first preset threshold and the precision requirement level is less than the second preset threshold, the target task processing model is determined as the single-process execution model.

[0023] Optionally, the dynamic adjustment of the subtask sequence and the execution parameter of each subtask based on the environmental feedback data includes:

[0024] An environmental interference coefficient, a subtask execution error, and a working state of an associated device are obtained;

[0025] The execution parameter is dynamically adjusted based on the environmental interference coefficient and the subtask execution error;

[0026] The subtask sequence is dynamically adjusted based on the working state of the associated device.

[0027] Optionally, the environmental interference coefficient is obtained, including:

[0028] An environmental interference vibration frequency, an illumination intensity deviation, and an electromagnetic interference value are obtained;

[0029] The environmental interference coefficient is calculated based on the environmental interference vibration frequency, the illumination intensity deviation, the electromagnetic interference value, and an environmental reference value.

[0030] In another aspect, an embodiment of the present application provides an industrial scene humanoid robot task execution device, the device comprising:

[0031] An instruction receiving module is configured to obtain a task instruction to be executed in response to a task issuing request.

[0032] A model determining module is configured to extract a complexity parameter of the task instruction, and determine a target task processing model according to the complexity parameter.

[0033] A task generating module is configured to decompose the task instruction into a subtask sequence by using the target task processing model, and generate an execution parameter for each subtask.

[0034] A task execution module is configured to control a humanoid robot to execute a task based on the subtask sequence and the execution parameter of each subtask, and output a task result.

[0035] The complexity parameter comprises a number of task steps and an accuracy requirement level, and the target task processing model comprises a multi-modal fusion execution model and a single-process execution model.

[0036] In another aspect, an embodiment of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the above method when executing the computer program.

[0037] In another aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above method.

[0038] In another aspect, an embodiment of the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the above method.

[0039] The embodiment of the present application extracts the number of steps and the accuracy requirement level of the task instruction, dynamically matches the optimal processing model, effectively improves the task execution efficiency and accuracy of multiple humanoid robots, and meets the use requirements in the industrial scene. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is an implementation environment schematic diagram of an industrial scene humanoid robot task execution method provided by an embodiment of the present application;

[0041] Figure 2is a flowchart of a method for task execution of a humanoid robot in an industrial scene according to an embodiment of the present application;

[0042] Figure 3 is a flowchart of a method for dynamically updating execution parameters according to an embodiment of the present application;

[0043] Figure 4 is a flowchart of a method for adjusting control of a humanoid robot to perform a task according to an embodiment of the present application;

[0044] Figure 5 is a structural diagram of a device for task execution of a humanoid robot in an industrial scene according to an embodiment of the present application;

[0045] Figure 6 is a hardware structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application. When the following description refers to the accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with embodiments of the present application. They are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0047] It can be understood that the terms "first", "second", and the like used herein can be used to describe various concepts in this document, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0048] The terms "at least one", "multiple", "each", "any", and the like used herein include one, two or more than two, multiple includes two or more than two, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0050] Currently, in industrial automation production, industrial humanoid robots need to complete diversified tasks such as carrying, assembling, and detecting.

[0051] In the related art, the task execution mode of the robot is usually a fixed flow: that is, the action is executed according to a preset program, and a single type or fixed complexity task is processed. However, it is found in actual application that the adaptability and execution precision of the traditional robot task method in the industrial scene are limited.

[0052] Therefore, in the embodiments of the present application, an industrial scene humanoid robot task execution method and related equipment are provided, which dynamically match the optimal processing model by extracting the number of steps and the accuracy requirement level of the task instruction, effectively improve the task execution efficiency and precision of multiple humanoid robots, and meet the use requirements in the industrial scene.

[0053] It should be noted that in each specific embodiment of the present application, when it is necessary to process data related to the identity or characteristics of the user, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate permission or separate consent of the user will be obtained through a pop-up window or a jump to a confirmation page, and after obtaining the separate permission or separate consent of the user, the necessary user-related data for the normal operation of the embodiments of the present application will be obtained.

[0054] The specific embodiments of the embodiments of the present application will be described in detail below with reference to the accompanying drawings. First, an industrial scene humanoid robot task execution method provided in the embodiments of the present application is described with reference to the accompanying drawings.

[0055] Please refer to Figure 1 , Figure 1 is an implementation environment schematic diagram of an industrial scene humanoid robot task execution method provided in the embodiments of the present application. In this implementation environment, the main hardware and software subjects involved include a terminal processor 110 and a server 120.

[0056] Specifically, the terminal processor 110 can be installed with a related industrial scene humanoid robot task execution method control program, and the server 120 is a background server of the control program. The terminal processor 110 and the background server 120 are in communication connection. The industrial scene humanoid robot task execution method provided in the embodiments of the present application can be executed on the terminal processor 110 side.

[0057] The server 120 can be a stand-alone physical server, a server cluster composed of multiple physical servers, or a distributed system, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms.

[0058] In addition, the server 120 can also be a node server in a blockchain network.

[0059] The terminal processor 110 and the server 120 can establish a communication connection through a wireless network. The wireless network uses standard communication technology and / or protocols, and the network can be set as the Internet, or any other network, for example, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile or wireless network, a private network or a virtual private network, or any combination thereof. In addition, the above-mentioned software and hardware subjects can use the same communication connection method or different communication connection methods, and the application does not make specific limitations.

[0060] Of course, it can be understood that Figure 1 The implementation environment in the above Figure 1 The application does not make specific limitations.

[0061] As Figure 2 As Figure 2 is a flowchart of an industrial scene humanoid robot task execution method provided by the embodiments of the present application, and specifically includes but is not limited to steps S100 to S400.

[0062] Step S100, in response to a task issuing request, obtaining a task instruction to be executed.

[0063] In the embodiments of the present application, the execution subject can be set as a control unit of a humanoid robot, which can be integrated in the humanoid robot body or an edge computing server, and is connected to the sensors (such as visual sensors, force feedback sensors, and environment sensors, etc.) integrated in the industrial humanoid robot body and the execution mechanism (such as joint motors and end effectors, etc.) of the humanoid robot through a communication module.

[0064] When the control unit responds to and receives a task issuing request sent by a remote control terminal or an operator, the control unit can identify and extract the task issuing request, obtain a task instruction to be executed, such as an assembly, spot welding, bolt tightening, and other types of industrial scene task instructions.

[0065] Step S200, extracting a complexity parameter of the task instruction, and determining a target task processing model according to the complexity parameter; wherein the complexity parameter includes a task step number and an accuracy requirement level; and the target task processing model includes a multi-modal fusion execution model and a single-process execution model.

[0066] In the embodiments of the present application, by identifying and analyzing the obtained task instruction to be executed, the complexity parameter of the task instruction is extracted, so as to determine the task complexity corresponding to the task instruction according to the task step number and the accuracy requirement level of the task instruction, and match the optimal target task processing model, that is, when the task complexity is low, the target task processing model can be determined as a single-process execution model, and when the task complexity is high, the steps are many and the accuracy requirement is high, the target task processing model can be determined as a multi-modal fusion execution model.

[0067] Optionally, as an optional implementation, the extracting the complexity parameter of the task instruction and determining the target task processing model according to the complexity parameter includes:

[0068] performing semantic analysis on the task instruction, extracting a task target and associated device information, and then determining a task step number of the task instruction;

[0069] determining an accuracy level corresponding to the task target based on a preset industrial scene database, so as to determine an accuracy requirement level of the task instruction;

[0070] determining a target task processing model according to the task step number and the accuracy requirement level of the task instruction.

[0071] In the embodiments of the present application, by performing semantic identification and analysis on the obtained task instruction, for example, by using a pre-trained natural language processing model, the task target and associated device information of the task instruction are extracted, and the task step number of the task instruction is determined.

[0072] Exemplarily, the control unit receives a task instruction sent by the host computer through an industrial bus or wireless communication, the instruction format is structured text, for example, "grab the PCB board on the assembly line (model PCB-001), detect the pin pitch (accuracy level 2), and send to the assembly station", then the NLP model can identify that the number of task steps is 3 steps (grabbing, detecting, and carrying), and based on the preset industrial scene database, the accuracy level corresponding to the task target is matched and determined, so as to determine the accuracy requirement level of the task instruction. Taking the above example of grabbing the PCB board and detecting and carrying as an example, the standard accuracy level corresponding to the PCB board pin detection can be retrieved from the industrial database, which is level 2, so as to determine that the accuracy requirement level of the task instruction is level 2.

[0073] Optionally, the NLP model can use a BERT model trained based on an industrial task corpus, and is trained by containing multiple industrial task instructions as a data set.

[0074] It can be understood that the industrial scene database can be pre-set by constructing the industrial scene database by taking historical task work orders as a data source, and annotating the accuracy requirements of the task instructions of different steps, so as to complete the pre-set construction of the industrial scene database, and can be updated and optimized in real time in later applications, so as to adapt to the use requirements in different industrial scenes.

[0075] Further, according to the number of task steps and the accuracy requirement level of the determined task instruction, a more suitable target task processing model is matched and determined.

[0076] Therefore, compared with the prior art, the task execution mode of the robot is usually a fixed process: that is, the action is executed according to the preset program, and only a single type or fixed complexity task can be processed. For example, a traditional assembly robot can only complete bolt tightening according to a fixed path, when the task includes multiple steps (such as first grabbing a workpiece, then detecting the size, and finally assembling), the program needs to be manually switched; and in a high-precision requirement scene (such as electronic component welding), the fixed execution model cannot cope with the precision deviation caused by environmental interference (such as vibration and light change). In addition, the task processing model does not match the complexity, and the single-process execution of high-complexity tasks leads to low efficiency, and the use of complex models for low-complexity tasks causes resource waste.

[0077] The present application dynamically matches the complexity parameter with the task processing model, optimizes the resource configuration, and effectively improves the task execution efficiency and accuracy of the humanoid robot.

[0078] In practical applications, the target task processing model is determined according to the number of task steps and the accuracy requirement level of the task instruction, including:

[0079] In a case where the number of task steps is greater than or equal to a first preset threshold or the accuracy requirement level is greater than or equal to a second preset threshold, the target task processing model is determined as the multi-modal fusion execution model.

[0080] In a case where the number of task steps is less than the first preset threshold and the accuracy requirement level is less than the second preset threshold, the target task processing model is determined as the single-process execution model.

[0081] In the embodiments of the present application, by threshold comparison judgment on the number of task steps and the accuracy requirement level, a corresponding target task processing model can be matched and determined.

[0082] Table 1: Complexity parameter and target task processing model mapping table

[0083]

[0084] Exemplarily, referring to Table 1, Table 1 is a complexity parameter and target task processing model mapping table. The first preset threshold can be set as 3 steps and the second preset threshold can be set as 3 levels. When the number of task steps in the extracted complexity parameter is greater than or equal to 3 steps or the accuracy requirement level is greater than or equal to 3 levels, the target task processing model can be determined as the multi-modal fusion execution model. When the number of task steps in the complexity parameter is less than 3 steps and the accuracy requirement level is less than 3 levels, the target task processing model can be determined as the single-process execution model. In addition, when the number of task steps in the complexity parameter is equal to 2 steps and the accuracy requirement level is equal to 2 levels, the target task processing model can be determined as the multi-modal fusion execution model to cope with relatively complex industrial scene task requirements.

[0085] In step S300, the task instruction is decomposed into a sub-task sequence by using the target task processing model, and execution parameters are generated for each sub-task.

[0086] In the embodiments of the present application, by selecting the multi-modal fusion execution model and the single-process execution model to meet task instructions of different complexity, model resources can be effectively allocated.

[0087] The execution logic of the multi-modal fusion execution model can be set to obtain workpiece three-dimensional coordinate data through a visual sensor, and generate a joint angle sequence by using a kinematics inverse solution algorithm, different execution parameters are set for different sub-tasks, and the edge computing unit can be used to process the dependency relationship among the sub-tasks in parallel, for example, when the completion degree of sub-task A is greater than or equal to 90%, the preprocessing of sub-task B is started, so as to connect different sub-tasks. Further, the execution logic of the single-process execution model can be set to call a standard process template in a preset action library, and calculate a template matching degree, when the template matching degree is greater than a rated percentage value, the template is directly reused, otherwise, the execution parameter can be further adjusted.

[0088] It can be understood that the preset action library can be pre-constructed according to the execution parameters of the historical work orders. Taking the "bolt tightening task" as an example, when constructing the entry of the task instruction of this type, the task can be first disassembled, and the "bolt tightening" can be disassembled into sub-actions: positioning, aligning, pre-tightening, and final tightening, and the humanoid robot can be used to execute the task several times, and the key parameters (such as joint angle sequence, end effector trajectory, and tightening torque curve) are recorded and stored as a template. After the corresponding label is labeled for the template, the template is stored in the preset action library, so as to be searched according to the task type and accuracy level in the future.

[0089] Further, the PID control algorithm can be used to maintain the execution accuracy. In actual application, the proportional coefficient Kp can be set to 5+(accuracy requirement level-1)×2, the integral coefficient Ki can be set to 0.1, and the differential coefficient Kd can be set to 0.5.

[0090] In step S400, the humanoid robot is controlled to execute the task based on the sub-task sequence and the execution parameters of each sub-task, and a task result is output.

[0091] In the embodiments of the present application, the humanoid robot executes each sub-task according to the disassembled sub-task sequence, controls each end effector according to the execution parameters of each sub-task, and finally outputs a task result.

[0092] Therefore, the present application can avoid problems such as resource waste or insufficient accuracy by dynamically matching the most suitable model according to the complexity parameter. Low-complexity tasks can simplify the process by using the single-process execution model, and reduce the calculation and response delay. High-complexity tasks can improve the task execution efficiency and accuracy of the humanoid robot through multi-modal cooperation.

[0093] It can be understood that the present application can control the terminal to simultaneously load and run the control program related to the industrial scene humanoid robot task execution method in multiple humanoid robots in an industrial scene, so as to reasonably arrange the task execution sequence among the multiple humanoid robots.

[0094] In practical applications, the humanoid robot is controlled to execute the task based on the subtask sequence and the execution parameter of each subtask, and a task result is outputted, including:

[0095] The humanoid robot is controlled to execute the task based on the subtask sequence and the execution parameter of each subtask, and environment feedback data in the process of executing the task is acquired in real time;

[0096] The subtask sequence and the execution parameter of each subtask are dynamically adjusted based on the environment feedback data;

[0097] The humanoid robot is controlled to execute the task based on the updated subtask sequence and the execution parameter of each subtask, and a task result is outputted.

[0098] In the embodiments of the present application, please refer to Figure 3 , Figure 3 is a flowchart of dynamically updating the execution parameter provided in the embodiments of the present application. In the process of controlling the humanoid robot to execute the task, the subtask sequence and the execution parameter of each subtask can also be corrected in real time according to the environment feedback data, so as to reduce the error of each subtask.

[0099] Specifically, first, the humanoid robot is controlled to execute the task according to the subtask sequence and the initial execution parameter of each subtask, and environment feedback data is collected in real time. The error in the process of executing the task is determined through the environment feedback data, and a correction amount is calculated. The execution parameter is corrected by using the calculation result of the correction amount, so as to continue to execute each subtask by using the corrected execution parameter. In addition, the subtask sequence can also be adjusted in real time based on the environment feedback data, so as to improve the use efficiency of each device.

[0100] Therefore, the execution parameter of the subtask is continuously optimized relying on real-time feedback, which can significantly improve the task execution efficiency and task accuracy of multiple humanoid robots.

[0101] In practical applications, the subtask sequence and the execution parameter of each subtask are dynamically adjusted based on the environment feedback data, including:

[0102] An environment interference coefficient, a subtask execution error, and a working state of an associated device are acquired.

[0103] The execution parameter is dynamically adjusted based on the environment interference coefficient and the subtask execution error.

[0104] The subtask sequence is dynamically adjusted based on the working state of the associated device.

[0105] In the embodiments of the present application, the environmental interference coefficient, the subtask execution error and the working state of the associated device can be obtained as the basis for dynamically adjusting the subtask sequence and the execution parameters of each subtask.

[0106] By way of example, reference is made to Figure 4 , Figure 4 is a flowchart of adjusting the control of the humanoid robot to perform a task provided by the embodiments of the present application. The environmental interference coefficient can be calculated from the data collected by the environmental sensor, for example, according to the collected environmental interference vibration frequency, illumination intensity deviation and electromagnetic interference value, combined with the environmental reference value, to calculate the environmental interference coefficient. The calculation formula can be expressed as: environmental interference coefficient = (environmental interference vibration frequency x 0.3 + illumination intensity deviation x 0.2 + electromagnetic interference value x 0.5) / environmental reference value.

[0107] Further, the subtask execution error can be detected by the visual sensor or the force feedback sensor, so as to combine the environmental interference coefficient and the subtask execution error, and perform data fusion processing in the edge computing layer to trigger the process of dynamically adjusting the execution parameters, and adjust the execution parameters of the subtask according to the real-time optimization algorithm.

[0108] In addition, the executable state of different subtasks can also be determined according to the working state of the associated device, for example, the subtask sequence is S1, S2, S3, and it is detected that the working state of the associated device of S2 is in a busy state and the working state of S3 is in an idle state. The subtask sequence can be adjusted to S1, S3, S2 in priority, so as to improve the task execution efficiency of the humanoid robot. For example, it is detected that the assembly station is currently idle, the priority of the S3 assembly task is improved, and the image analysis process of S2 is performed in parallel (i.e. directly moving after grabbing, and synchronously performing pin detection data processing)

[0109] Finally, based on the updated subtask sequence and the execution parameters of each subtask, the humanoid robot is adjusted to perform a task, and in the execution control layer, the joint controller and each end effector are adjusted to adjust the path planning.

[0110] In actual application, when the multi-modal fusion execution model generates a joint angle sequence using the kinematics inverse solution algorithm, a joint angle correction value can be calculated based on the environmental interference coefficient, and the calculation formula is joint angle correction value = basic angle value + (environmental interference coefficient x 0.1 x dynamic compensation coefficient). The basic angle value can be preset according to the specific scene use requirement, and the dynamic compensation coefficient can be set according to the subtask type, for example, the value is set to 1.2 for a grabbing task and 1.8 for an assembly task.

[0111] Further, when the calculation result of the environmental interference coefficient is greater than a preset rated value, a redundant action verification mechanism can also be started, and the sensor sampling frequency is raised to 50Hz. When the subtask execution error is greater than a preset rated value, a secondary path planning can be started, and the calculation formula of the path correction amount can be expressed as path correction amount = subtask execution error value x 1.2 + environmental interference coefficient x 0.5. In addition, for general action parameters and general parameters, a correction amount can also be calculated based on the environmental interference coefficient, so as to dynamically adjust the execution parameters of each subtask. For example, the corrected general parameter = initial parameter x (1 + environmental interference coefficient x 0.05 - subtask completion degree x 0.03), wherein the subtask completion degree can be quantified by the executed time of the subtask, or the subtask completion degree can be quantified by the number of completed nodes of the subtask.

[0112] In actual application, when it is detected that the execution error is greater than a preset threshold and the duration meets the triggering condition, the task is triggered to pause execution and an alarm signal is sent, so as to facilitate the relevant operating personnel to timely maintain and debug.

[0113] Further, the multi-modal fusion execution model can adopt a neural network model, train by constructing a data set containing multiple industrial task samples, and the labeled parameters include environmental interference coefficient, optimal execution model and error rate. In the training process, the reinforcement learning algorithm is used, the reward function is "shortest execution time" and "minimum error", the model selection threshold is iteratively optimized, and the mapping relationship between the environmental interference coefficient and the correction amount is updated according to the data of each cumulative 100 task executions, so that the calculation result of the parameter correction amount is more accurate.

[0114] Therefore, the application dynamically matches the task processing model by the complexity parameter, optimizes the resource configuration, adjusts the execution parameter combined with the real-time collected environmental feedback data, improves the anti-interference ability, dynamically adjusts the subtask sequence, improves the task execution efficiency of the whole humanoid robot, can effectively shorten the multi-step task execution time, reduce the execution error of high-precision tasks, and effectively meet the use demand in industrial scene.

[0115] Please refer to Figure 5 , Figure 5 is a structural schematic diagram of an industrial scene humanoid robot task execution device provided by an embodiment of the application. The embodiment of the application also provides an industrial scene humanoid robot task execution device, which can implement the above-mentioned industrial scene humanoid robot task execution method. The device comprises:

[0116] The instruction receiving module 510 is configured to acquire the task instruction to be executed in response to a task issuing request.

[0117] The model determining module 520 is configured to extract a complexity parameter of the task instruction, and determine a target task processing model according to the complexity parameter.

[0118] The task generating module 530 is configured to decompose the task instruction into a subtask sequence by using the target task processing model, and generate an execution parameter for each subtask.

[0119] The task executing module 540 is configured to control the humanoid robot to execute a task based on the subtask sequence and the execution parameter of each subtask, and output a task result.

[0120] The complexity parameter includes a number of task steps and an accuracy requirement level, and the target task processing model includes a multi-modal fusion execution model and a single-process execution model.

[0121] It can be understood that the contents in the above method embodiments are all applicable to the device embodiments, the device embodiments specifically implement the functions of the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0122] Please refer to Figure 6 , Figure 6 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application, and the electronic device includes:

[0123] The processor 601 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0124] The memory 602 can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory). The memory 602 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 602 and are called and executed by the processor 601 to implement the above-mentioned method of the embodiments of the present application.

[0125] The input / output interface 603 is configured to realize information input and output.

[0126] The communication interface 604 is configured to realize the communication interaction between the device and other devices, and can realize the communication through a wired manner (for example, a USB, a network cable or the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth or the like).

[0127] The bus 605 is configured to transmit information between various components (for example, the processor 601, the memory 602, the input / output interface 603 and the communication interface 604) of the device.

[0128] The processor 601, the memory 602, the input / output interface 603 and the communication interface 604 are communicatively connected to each other in the device through the bus 605.

[0129] The embodiment of the application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the method.

[0130] It can be understood that the contents in the above method embodiments are applicable to the storage medium embodiment, the storage medium embodiment specifically realizes the functions of the above method embodiments, and the beneficial effects achieved by the storage medium embodiment are the same as the beneficial effects achieved by the above method embodiments.

[0131] The embodiment of the application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to realize the method.

[0132] It can be understood that the contents in the above method embodiments are applicable to the program product embodiment, the program product embodiment specifically realizes the functions of the above method embodiments, and the beneficial effects achieved by the program product embodiment are the same as the beneficial effects achieved by the above method embodiments.

[0133] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Embodiments of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0134] The industrial scene humanoid robot task execution method and related device provided by the embodiment of the application can effectively improve the task execution efficiency and precision of multiple humanoid robots by extracting the number of steps of the task instruction and the precision requirement level, dynamically matching the optimal processing model, and meeting the use requirements in the industrial scene.

[0135] The embodiments described in the specification are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0136] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than those shown in the figures, or combine certain steps, or different steps.

[0137] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0138] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the function modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0139] The terms "first", "second", "third", "fourth" and the like (if any) in the specification of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0140] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B, and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, and c can be single or multiple.

[0141] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0142] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0143] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0144] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0145] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. An industrial scene humanoid robot task execution method, characterized by, The method comprises the following steps: in response to a task issuing request, obtaining a task instruction to be executed; extracting a complexity parameter of the task instruction, and determining a target task processing model according to the complexity parameter; using the target task processing model, decomposing the task instruction into a subtask sequence, and generating an execution parameter for each subtask; based on the subtask sequence and the execution parameter of each subtask, controlling the humanoid robot to execute the task and outputting a task result; wherein the complexity parameter comprises the number of task steps and the accuracy requirement level; the target task processing model comprises a multi-modal fusion execution model and a single-process execution model; the extraction of the complexity parameter of the task instruction and the determination of the target task processing model according to the complexity parameter comprise: performing semantic analysis on the task instruction to extract the task target and associated device information, and then determining the number of task steps of the task instruction; based on a preset industrial scene database, matching to determine the accuracy level corresponding to the task target, thereby determining the accuracy requirement level of the task instruction; determining the target task processing model according to the number of task steps and the accuracy requirement level of the task instruction; the determination of the target task processing model according to the number of task steps and the accuracy requirement level of the task instruction comprises: in the case where the number of task steps is greater than or equal to a first preset threshold or the accuracy requirement level is greater than or equal to a second preset threshold, determining the target task processing model as the multi-modal fusion execution model; in the case where the number of task steps is less than the first preset threshold and the accuracy requirement level is less than the second preset threshold, determining the target task processing model as the single-process execution model.

2. The method of claim 1, wherein, the control of the humanoid robot to execute the task and the output of the task result based on the subtask sequence and the execution parameter of each subtask comprise: based on the subtask sequence and the execution parameter of each subtask, controlling the humanoid robot to execute the task, and obtaining environmental feedback data in real time during the execution of the task; based on the environmental feedback data, dynamically adjusting the subtask sequence and the execution parameter of each subtask; based on the updated subtask sequence and the execution parameter of each subtask, adjusting the control of the humanoid robot to execute the task, and outputting the task result.

3. The method of claim 2, wherein, the dynamic adjustment of the subtask sequence and the execution parameter of each subtask based on the environmental feedback data comprises: obtaining an environmental interference coefficient, a subtask execution error, and a working state of an associated device; based on the environmental interference coefficient and the subtask execution error, dynamically adjusting the execution parameter; based on the working state of the associated device, dynamically adjusting the subtask sequence.

4. The method of claim 3, wherein, the obtaining of the environmental interference coefficient comprises: obtaining an environmental interference vibration frequency, an illumination intensity deviation, and an electromagnetic interference value; based on the environmental interference vibration frequency, the illumination intensity deviation, the electromagnetic interference value, and an environmental reference value, calculating the environmental interference coefficient.

5. An industrial scene humanoid robot task execution apparatus characterized by comprising: the device comprises: an instruction receiving module for obtaining a task instruction to be executed in response to a task issuing request; A model determining module is configured to extract a complexity parameter of the task instruction, and determine a target task processing model according to the complexity parameter; A task generating module is configured to decompose the task instruction into a sub-task sequence by using the target task processing model, and generate an execution parameter for each sub-task; A task executing module is configured to control the humanoid robot to execute a task based on the sub-task sequence and the execution parameter of each sub-task, and output a task result; The complexity parameter includes a task step number and an accuracy requirement level, and the target task processing model includes a multi-modal fusion execution model and a single-process execution model; The extraction of the complexity parameter of the task instruction and the determination of the target task processing model according to the complexity parameter include: The task instruction is subjected to semantic analysis to extract a task target and associated device information, and then the task step number of the task instruction is determined; An accuracy level corresponding to the task target is matched and determined based on a preset industrial scene database, so as to determine the accuracy requirement level of the task instruction; The target task processing model is determined according to the task step number and the accuracy requirement level of the task instruction; The determination of the target task processing model according to the task step number and the accuracy requirement level of the task instruction includes: In a case where the task step number is greater than or equal to a first preset threshold or the accuracy requirement level is greater than or equal to a second preset threshold, the target task processing model is determined as the multi-modal fusion execution model; In a case where the task step number is less than the first preset threshold and the accuracy requirement level is less than the second preset threshold, the target task processing model is determined as the single-process execution model.

6. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method of any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 6. The computer program is executed by the processor to implement the method of any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 4.

Images