A robot manipulation task control method and system

Abstract

The present invention provides a robot manipulation task control method and system, wherein the method comprises: step 1: obtaining the execution status of the first task currently being executed by the manipulation robot; step 2: based on the execution status, adaptively adjusting the first task to obtain the adjusted second task; step 3: based on the second task, relaying the robot. The robot manipulation task control method and system of the present invention does not require staff to manually monitor the task execution status of the robot when executing the task and manually adjust the currently executed task, thereby reducing labor costs, and is particularly suitable for sites with a large number of robots to avoid adverse consequences caused by untimely adjustments.

Description

Technical Field

[0001] The present invention relates to the field of robot technology, and in particular to a robot manipulation task control method and system. Background Art

[0002] At present, when an automated robot (e.g., a manipulator robot) performs a task (e.g., aluminum shell loading and handling), the task execution status will change and the task needs to be adjusted (e.g., the temperature of the manipulator motor rises and the manipulator's loading frequency needs to be reduced). However, in general, the monitoring of task execution and task adjustment require manual work by staff, which has a high labor cost, especially at sites with many robots. There is also the possibility that untimely adjustments may lead to adverse results.

[0003] Therefore, a solution is urgently needed. Summary of the invention

[0004] One of the purposes of the present invention is to provide a robot manipulation task control method, which does not require staff to manually monitor the task execution status of the robot when performing the task and manually adjust the current task, thereby reducing labor costs. The method is particularly suitable for sites with a large number of robots to avoid adverse consequences caused by untimely adjustments.

[0005] An embodiment of the present invention provides a robot manipulation task control method, comprising:

[0006] Step 1: Obtain the execution status of the first task currently being executed by the manipulation robot;

[0007] Step 2: Based on the execution status, adaptively adjust the first task to obtain an adjusted second task;

[0008] Step 3: Based on the second task, the robot is relay-operated.

[0009] Preferably, step 1: obtaining the execution status of the first task currently being executed by the manipulator robot comprises:

[0010] Acquire first state information of the robot when the robot is controlled to perform the first task, and use the information as the execution status;

[0011] and / or,

[0012] Acquire second state information of a field device operated by the robot when the robot is operated to perform the first task;

[0013] and / or,

[0014] A first staff node corresponding to a preset staff member of the robot is obtained, and the execution status of the first task currently being performed by the robot, which is uploaded by the staff member, is obtained through the first staff node.

[0015] Preferably, in step 2, adaptively adjusting the first task based on the execution status includes:

[0016] Based on a preset first feature extraction template, feature extraction is performed on the execution status to obtain a plurality of first feature values;

[0017] Based on the first eigenvalue, construct a situation description vector of the execution situation;

[0018] Get a preset situation complexity evaluation library;

[0019] Determine the complexity based on the situation complexity evaluation library and the situation description vector;

[0020] If the complexity is less than or equal to a preset complexity threshold, obtaining a preset task adjustment model, inputting the situation description vector and the first task into the task adjustment model, and determining an adjusted second task;

[0021] Otherwise, obtaining an idle second staff node of a preset expert staff corresponding to the robot;

[0022] Count the number of second personnel nodes;

[0023] If the number is 1, the first feature value and the first task are delivered to the second staff node for review by the expert staff;

[0024] Obtaining the adjusted second task replied by the expert personnel through the second personnel node;

[0025] If the number is not 1, build an online decision room;

[0026] Connecting expert personnel to the online decision-making room through the second personnel node;

[0027] Outputting and displaying the first characteristic value and the first task in the online decision room for the expert to view;

[0028] Acquire a first decision record generated by an expert in an online decision room, the first decision record comprising: a voice chat record and a text chat record, or a combination of both;

[0029] When all the experts leave the online decision-making room, the first task is adjusted based on the first decision record to obtain an adjusted second task.

[0030] Preferably, adjusting the first task based on the first decision record includes:

[0031] Obtaining the generation time of the first decision record;

[0032] Based on the time when the records were generated, the first decision records were sorted in chronological order to obtain a decision record sequence;

[0033] Performing semantic extraction on the first decision record to obtain first semantics;

[0034] Traverse the first decision record from the starting point of the decision record sequence to the end point of the sequence;

[0035] During each traversal, a preset decision semantic library is obtained, and a first semantic of a first decision record traversed is matched with a second semantic in the decision semantic library;

[0036] If there is a match, stop traversal and obtain a preset adjustment semantic library corresponding to the second semantics that matches the match;

[0037] Matching the first semantics of the first decision record after the first decision record traversed in the decision record sequence with the third semantics in the adjustment semantic library;

[0038] If none of them matches, obtaining a preset first task adjustment rule corresponding to the second semantics that matches;

[0039] If there is a match, the first decision record corresponding to the first semantics of the match is used as the second decision record;

[0040] Acquire a preset second task adjustment rule corresponding to a third semantics matching the first semantics of the second decision record closest to the sequence end point in the second decision record;

[0041] Eliminate the second decision record from the decision record sequence;

[0042] After the elimination is completed, continue the traversal;

[0043] After the traversal is completed, the first task is adjusted in sequence based on the first task adjustment rule and the second task adjustment rule.

[0044] Preferably, the robot manipulation task control method further includes:

[0045] When the operator performs corresponding operations on the robot based on the subtask that requires manual operation in the second task, the operator is assisted in the operation;

[0046] Among them, the operation assistance to the operator includes:

[0047] Acquire a first picture of the palm direction of the palm of the robot through a first image acquisition device arranged next to the palm of the manipulator of the robot, and output it for display;

[0048] Continuously obtain the starting and ending positions of the palm at the beginning and end of the most recent preset time period;

[0049] Determine a first moving direction based on the starting position and the ending position;

[0050] Acquire device positions and shooting directions of a plurality of second image acquisition devices within a first range preset around the robot;

[0051] Based on a preset second feature extraction template, feature extraction is performed on the end position, the first moving direction, the device position, and the shooting direction to obtain a plurality of second feature values;

[0052] Based on the second eigenvalue, construct a first relationship description vector of the shooting angle relationship between the manipulator and the second image acquisition device;

[0053] Obtain a preset shooting angle relationship suitability evaluation library;

[0054] Determine a suitable value based on a shooting angle relationship suitability evaluation library and a first relationship description vector;

[0055] Acquire a second image of the manipulator through a second image acquisition device corresponding to the maximum fitness value, and output it for display;

[0056] Based on the second image, determining the object positions of a plurality of objects within a second range preset around the manipulator;

[0057] Obtain the object state of the object, the object state includes: static and dynamic;

[0058] When the object state of the object is static, based on a preset third feature extraction template, feature extraction is performed on the end position, the object position and the first moving direction to obtain a plurality of third feature values;

[0059] Based on the third eigenvalue, construct a second relational description vector of the relative positional relationship between the manipulator and the object;

[0060] Obtaining a preset relative position relationship collision risk determination library;

[0061] Determine a first collision risk determination result based on the relative position relationship collision risk determination library and the second relationship description vector;

[0062] When the first collision risk determination result is that there is a collision risk, outputting and displaying the first collision risk determination result;

[0063] When the object state of the object is dynamic, obtaining a second moving direction of the object;

[0064] Based on a preset fourth feature extraction template, feature extraction is performed on the end position, the object position, the first moving direction, and the second moving direction to obtain a plurality of fourth feature values;

[0065] Based on the fourth eigenvalue, construct a third relation description vector of the relative position relationship between the manipulator and the object;

[0066] Determining a second collision risk determination result based on the relative position relationship collision risk determination library and the third relationship description vector;

[0067] When the second collision risk determination result is that there is a collision risk, the second collision risk determination result is output and displayed.

[0068] An embodiment of the present invention provides a robot manipulation task control system, comprising:

[0069] An acquisition module, used for acquiring the execution status of the first task currently being executed by the manipulation robot;

[0070] An adjustment module, used for adaptively adjusting the first task based on the execution status to obtain an adjusted second task;

[0071] The manipulation module is used to perform relay manipulation on the robot based on the second task.

[0072] Preferably, the acquisition module acquires the execution status of the first task currently being executed by the manipulator robot, including:

[0073] Acquire first state information of the robot when the robot is controlled to perform the first task, and use the information as the execution status;

[0074] and / or,

[0075] Acquire second state information of a field device operated by the robot when the robot is operated to perform the first task;

[0076] and / or,

[0077] A first staff node corresponding to a preset staff member of the robot is obtained, and the execution status of the first task currently being performed by the robot, which is uploaded by the staff member, is obtained through the first staff node.

[0078] Preferably, the adjustment module adaptively adjusts the first task based on the execution status, including:

[0079] Based on a preset first feature extraction template, feature extraction is performed on the execution status to obtain a plurality of first feature values;

[0080] Based on the first eigenvalue, construct a situation description vector of the execution situation;

[0081] Get a preset situation complexity evaluation library;

[0082] Determine the complexity based on the situation complexity evaluation library and the situation description vector;

[0083] If the complexity is less than or equal to a preset complexity threshold, obtaining a preset task adjustment model, inputting the situation description vector and the first task into the task adjustment model, and determining an adjusted second task;

[0084] Otherwise, obtaining an idle second staff node of a preset expert staff corresponding to the robot;

[0085] Count the number of second personnel nodes;

[0086] If the number is 1, the first feature value and the first task are delivered to the second staff node for review by the expert staff;

[0087] Obtaining the adjusted second task replied by the expert personnel through the second personnel node;

[0088] If the number is not 1, build an online decision room;

[0089] Connecting expert personnel to the online decision-making room through the second personnel node;

[0090] Outputting and displaying the first characteristic value and the first task in the online decision room for the expert to view;

[0091] Acquire a first decision record generated by an expert in an online decision room, the first decision record comprising: a voice chat record and a text chat record, or a combination of both;

[0092] When all the experts leave the online decision-making room, the first task is adjusted based on the first decision record to obtain an adjusted second task.

[0093] Preferably, the adjustment module adjusts the first task based on the first decision record, including:

[0094] Obtaining the generation time of the first decision record;

[0095] Based on the time when the records were generated, the first decision records were sorted in chronological order to obtain a decision record sequence;

[0096] Performing semantic extraction on the first decision record to obtain first semantics;

[0097] Traverse the first decision record from the starting point of the decision record sequence to the end point of the sequence;

[0098] During each traversal, a preset decision semantic library is obtained, and a first semantic of a first decision record traversed is matched with a second semantic in the decision semantic library;

[0099] If there is a match, stop traversal and obtain a preset adjustment semantic library corresponding to the second semantics that matches the match;

[0100] Matching the first semantics of the first decision record after the first decision record traversed in the decision record sequence with the third semantics in the adjustment semantic library;

[0101] If none of them matches, obtaining a preset first task adjustment rule corresponding to the second semantics that matches;

[0102] If there is a match, the first decision record corresponding to the first semantics of the match is used as the second decision record;

[0103] Acquire a preset second task adjustment rule corresponding to a third semantics matching the first semantics of the second decision record closest to the sequence end point in the second decision record;

[0104] Eliminate the second decision record from the decision record sequence;

[0105] After the elimination is completed, continue the traversal;

[0106] After the traversal is completed, the first task is adjusted in sequence based on the first task adjustment rule and the second task adjustment rule.

[0107] Preferably, the robot manipulation task control system further comprises:

[0108] an auxiliary module, used for assisting the operator in manipulating the robot when the operator is manipulating the robot based on the subtask that requires manual manipulation in the second task;

[0109] Among them, the auxiliary module provides operation assistance to the operator, including:

[0110] Acquire a first picture of the palm direction of the palm of the robot through a first image acquisition device arranged next to the palm of the manipulator of the robot, and output it for display;

[0111] Continuously obtain the starting and ending positions of the palm at the beginning and end of the most recent preset time period;

[0112] Determine a first moving direction based on the starting position and the ending position;

[0113] Acquire device positions and shooting directions of a plurality of second image acquisition devices within a first range preset around the robot;

[0114] Based on a preset second feature extraction template, feature extraction is performed on the end position, the first moving direction, the device position, and the shooting direction to obtain a plurality of second feature values;

[0115] Based on the second eigenvalue, construct a first relationship description vector of the shooting angle relationship between the manipulator and the second image acquisition device;

[0116] Obtain a preset shooting angle relationship suitability evaluation library;

[0117] Determine a suitable value based on a shooting angle relationship suitability evaluation library and a first relationship description vector;

[0118] Acquire a second image of the manipulator through a second image acquisition device corresponding to the maximum fitness value, and output it for display;

[0119] Based on the second image, determining the object positions of a plurality of objects within a second range preset around the manipulator;

[0120] Obtain the object state of the object, the object state includes: static and dynamic;

[0121] When the object state of the object is static, based on a preset third feature extraction template, feature extraction is performed on the end position, the object position and the first moving direction to obtain a plurality of third feature values;

[0122] Based on the third eigenvalue, construct a second relational description vector of the relative positional relationship between the manipulator and the object;

[0123] Obtaining a preset relative position relationship collision risk determination library;

[0124] Determine a first collision risk determination result based on the relative position relationship collision risk determination library and the second relationship description vector;

[0125] When the first collision risk determination result is that there is a collision risk, outputting and displaying the first collision risk determination result;

[0126] When the object state of the object is dynamic, obtaining a second moving direction of the object;

[0127] Based on a preset fourth feature extraction template, feature extraction is performed on the end position, the object position, the first moving direction, and the second moving direction to obtain a plurality of fourth feature values;

[0128] Based on the fourth eigenvalue, construct a third relation description vector of the relative position relationship between the manipulator and the object;

[0129] Determining a second collision risk determination result based on the relative position relationship collision risk determination library and the third relationship description vector;

[0130] When the second collision risk determination result is that there is a collision risk, the second collision risk determination result is output and displayed.

[0131] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0132] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0133] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0134] Figure 1 is a flow chart of a robot manipulation task control method in an embodiment of the present invention;

[0135] Figure 2 Schematic diagram of a robot manipulation task control system in an embodiment of the present invention. DETAILED DESCRIPTION

[0136] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0137] The embodiment of the present invention provides a robot manipulation task control method, such as Figure 1 As shown, including:

[0138] Step 1: Obtain the execution status of the first task currently being executed by the manipulation robot;

[0139] Step 2: Based on the execution status, adaptively adjust the first task to obtain an adjusted second task;

[0140] Step 3: Based on the second task, the robot is relay-operated.

[0141] The working principle and beneficial effects of the above technical solution are:

[0142] When the automated robot performs the first task, the execution status is obtained. The first task may be, for example, to transport the aluminum battery casing on the conveyor belt to the loading port of the washing line at a certain joint movement speed, and input it into the washing line for cleaning. The execution status may be, for example, the operating temperature of the motor that controls the movement of the joints of the robot. Based on the execution status, the first task is adjusted. For example, when the operating temperature of the motor that controls the movement of the joints is high, the joint movement speed is reduced to obtain the second task. Based on the second task, the robot is automatically manipulated in a relay. There is no need for staff to manually monitor the execution of tasks when the robot is performing tasks and to manually adjust the currently executed tasks, which reduces labor costs. It is particularly suitable for sites with a large number of robots to avoid adverse consequences caused by untimely adjustments.

[0143] The embodiment of the present invention provides a robot manipulation task control method, step 1: obtaining the execution status of the first task currently being executed by the manipulation robot, including:

[0144] Acquire first state information of the robot when the robot is controlled to perform the first task, and use the information as the execution status;

[0145] and / or,

[0146] Acquire second state information of a field device operated by the robot when the robot is operated to perform the first task;

[0147] and / or,

[0148] A first staff node corresponding to a preset staff member of the robot is obtained, and the execution status of the first task currently being performed by the robot, which is uploaded by the staff member, is obtained through the first staff node.

[0149] The working principle and beneficial effects of the above technical solution are:

[0150] There are three ways to obtain the execution status: the first is to obtain the first state information when the automated robot performs the first task. The first state information can be, for example, the temperature of the joint motor and the working time. When adjusting the first task, for example, when the temperature of the joint motor is high or the working time is long, the joint movement speed is appropriately reduced. The second is to obtain the second state information of the on-site equipment operated by the robot. For example, when the robot performs the task of transporting the battery aluminum shell to the loading port of the washing line, the on-site equipment is the conveyor belt for conveying the battery aluminum shell and the washing line equipment. The second state information can be the number of battery aluminum shells conveyed by the conveyor belt and the number of battery aluminum shells being cleaned by the washing line. When adjusting the first task, for example, when the number of battery aluminum shells conveyed by the conveyor belt decreases or the number of battery aluminum shells being cleaned by the washing line increases, the joint movement speed is reduced. The third is to introduce the first staff node corresponding to the preset staff of the robot. The first staff node is a network node, which is connected to the intelligent terminal used by the staff. The staff can upload the execution status, for example, when patrolling, there are situations that need to be supplemented or there are temporary new tasks that need to be issued to the robot. It improves the comprehensiveness of execution status acquisition and improves the applicability of the system.

[0151] The embodiment of the present invention provides a robot manipulation task control method. In step 2, adaptively adjusting the first task based on the execution status includes:

[0152] Based on a preset first feature extraction template, feature extraction is performed on the execution status to obtain a plurality of first feature values;

[0153] Based on the first eigenvalue, construct a situation description vector of the execution situation;

[0154] Get a preset situation complexity evaluation library;

[0155] Determine the complexity based on the situation complexity evaluation library and the situation description vector;

[0156] If the complexity is less than or equal to a preset complexity threshold, obtaining a preset task adjustment model, inputting the situation description vector and the first task into the task adjustment model, and determining an adjusted second task;

[0157] Otherwise, obtaining an idle second staff node of a preset expert staff corresponding to the robot;

[0158] Count the number of second personnel nodes;

[0159] If the number is 1, the first feature value and the first task are delivered to the second staff node for review by the expert staff;

[0160] Obtaining the adjusted second task replied by the expert personnel through the second personnel node;

[0161] If the number is not 1, build an online decision room;

[0162] Connecting expert personnel to the online decision-making room through the second personnel node;

[0163] Outputting and displaying the first characteristic value and the first task in the online decision room for the expert to view;

[0164] Acquire a first decision record generated by an expert in an online decision room, the first decision record comprising: a voice chat record and a text chat record, or a combination of both;

[0165] When all the experts leave the online decision-making room, the first task is adjusted based on the first decision record to obtain an adjusted second task.

[0166] The working principle and beneficial effects of the above technical solution are:

[0167] A preset first feature extraction template is introduced to extract features of the execution situation and obtain multiple first feature values. The first feature value can be, for example, the temperature rise or fall speed of the joint motor, the rise or fall speed of the number of battery aluminum shells conveyed by the conveyor belt, and the rise or fall speed of the number of battery aluminum shells being cleaned by the washing line. The first feature extraction template is a template adapted to extract such first feature values, which belongs to the scope of the prior art and will not be elaborated. Based on the first feature value, a situation description vector of the execution situation is constructed. The vector constructed based on the data information and the description vector of the data information are all within the scope of the prior art and will not be elaborated. A preset situation complexity evaluation library is introduced, and the complexity corresponding to different situation description vectors is stored in the situation complexity evaluation library. Generally, the greater the temperature rise or fall speed of the joint motor, the greater the rise or fall speed of the number of battery aluminum shells conveyed by the conveyor belt, and the greater the rise or fall speed of the number of battery aluminum shells being cleaned by the washing line, the more complex the execution situation is, and the greater the complexity corresponding to the constructed situation description vector is. The determination efficiency and determination accuracy of the complexity determination of the execution situation are improved.

[0168] A preset complexity threshold is introduced, and the complexity threshold can be, for example, 75. If the complexity is less than or equal to the complexity threshold, it means that the execution situation is not complicated, and a preset task adjustment model is introduced and handed over to the task adjustment model for processing. The task adjustment model is an artificial intelligence model that uses a large number of records of manual adjustments to tasks based on the task execution situation of the robot to train the neural network model until convergence. The situation description vector and the first task are input into the task adjustment model, and the task adjustment model will adjust the first task according to the situation description vector and output the adjusted second task. There is no need for each adjustment task to be solved by experts, which reduces the utilization of experts' resources and reduces labor costs.

[0169] If the complexity is greater than the complexity threshold, it means that the execution is complicated and it is handed over to experts for processing. Reasonable adjustment of task allocation improves the accuracy of task adjustment.

[0170] When it is handed over to the expert for processing, the preset second staff node is introduced. The second staff node is a network node and is connected to the smart terminal used by the expert. When the number of the second staff node is 1, it means that there is only one idle expert at present. The first feature value and the first task are delivered to the second staff node. The expert can view it through the smart terminal, think about it and reply to the adjusted second task. When the number of the second staff node is not 1, it means that there are multiple idle experts at present. The online decision room can be constructed based on the online conference technology. Through the second staff node, the expert is connected to the online decision room, so that the expert joins the online decision room through the smart terminal. The first feature value and the first task are output and displayed in the online decision room for the expert to view. The first decision record generated by the expert in the online decision room is obtained. The first decision record can be a chat record of the expert discussing how to adjust. Based on the first decision record, the first task is adjusted to obtain the adjusted second task. For example, based on the semantic recognition technology, the adjustment intention is determined and the first task is adjusted. According to the number of idle experts, the expert resources are reasonably utilized to improve the adjustment efficiency of task adjustment.

[0171] An embodiment of the present invention provides a robot manipulation task control method, which adjusts a first task based on a first decision record, including:

[0172] Obtaining the generation time of the first decision record;

[0173] Based on the time when the records were generated, the first decision records were sorted in chronological order to obtain a decision record sequence;

[0174] Performing semantic extraction on the first decision record to obtain first semantics;

[0175] Traverse the first decision record from the starting point of the decision record sequence to the end point of the sequence;

[0176] During each traversal, a preset decision semantic library is obtained, and a first semantic of a first decision record traversed is matched with a second semantic in the decision semantic library;

[0177] If there is a match, stop traversal and obtain a preset adjustment semantic library corresponding to the second semantics that matches the match;

[0178] Matching the first semantics of the first decision record after the first decision record traversed in the decision record sequence with the third semantics in the adjustment semantic library;

[0179] If none of them matches, obtaining a preset first task adjustment rule corresponding to the second semantics that matches;

[0180] If there is a match, the first decision record corresponding to the first semantics of the match is used as the second decision record;

[0181] Acquire a preset second task adjustment rule corresponding to a third semantics matching the first semantics of the second decision record closest to the sequence end point in the second decision record;

[0182] Eliminate the second decision record from the decision record sequence;

[0183] After the elimination is completed, continue the traversal;

[0184] After the traversal is completed, the first task is adjusted in sequence based on the first task adjustment rule and the second task adjustment rule.

[0185] The working principle and beneficial effects of the above technical solution are:

[0186] When experts discuss task adjustment decisions, generally, one expert first proposes a preliminary adjustment plan, and other experts disagree with the preliminary adjustment plan. Therefore, it is necessary to conduct a segmented analysis to determine the final task adjustment plan based on the semantics of the chat records.

[0187] Sort the first decision records in chronological order to obtain a sequence of decision records. Extracting the first semantics of the first decision record can be achieved based on semantic extraction technology. Traverse the first decision record from the start point of the sequence to the end point of the sequence. Introduce a preset decision semantic library, which stores a large number of semantics related to the opinions on the task adjustment plan, such as: "The joint movement speed is reduced by 15%". Each time the traversal is performed, the first semantics of the traversed first decision record is matched with the second semantics in the decision semantic library. If the match is met, it means that an expert has stated the preliminary adjustment plan, and the segmented analysis begins. Using a trigger system, there is no need to analyze each first decision record, which reduces analysis resources and improves analysis efficiency.

[0188] The preset adjustment semantic library corresponding to the matched second semantics is introduced. The adjustment semantic library stores a large number of semantics that disagree with the task adjustment scheme published by others corresponding to the matched second semantics. For example, the matched second semantics is "the joint movement speed is reduced by 15%", and the semantics that disagree are "the joint movement speed is reduced too much" and "the joint movement speed can be reduced by 5%". When performing segmentation analysis, it is divided into two situations: the first one is that other experts have not expressed disagreement with the pre-adjustment scheme. The first semantics of the first decision record after the first decision record traversed in the decision record sequence is matched with the third semantics in the adjustment semantic library. If there is no match, it means that no other expert has expressed disagreement with the pre-adjustment scheme. The preset first task adjustment rule corresponding to the matched second semantics is directly introduced. For example, the matched second semantics is "the joint movement speed is reduced by 15%", and the first task adjustment rule is to reduce the joint movement speed of the robot's manipulator by 15%. The second type is that other experts express disagreement with the pre-adjustment plan; that is, when the first semantics of the first decision record after the first decision record traversed in the decision record sequence is matched with the third semantics in the adjustment semantic library, there is a match, and the first decision record corresponding to the first semantics that matches is used as the second decision record. When other personnel express disagreement with the pre-adjustment plan, they will discuss with the experts who published the pre-adjustment plan. Generally, the discussion result is the last speech, for example: "The joint movement speed is reduced by 15%, "The joint movement speed is reduced by 5%", and "Okay, the joint movement speed is reduced by 5%, and the discussion result is that the joint movement speed is reduced by 5%. Therefore, the preset second task adjustment rule corresponding to the third semantics that matches the first semantics of the second decision record closest to the end of the sequence in the second decision record is obtained. The preset second task adjustment rule corresponding to the third semantics is the same as the preset first task adjustment rule corresponding to the second semantics, and no further description is given. After the traversal is completed, the first task can be adjusted in sequence based on the first task adjustment rule and the second task adjustment rule, and the adjustment decision based on the chat record is completed. There is no need for experts to input adjustment decisions themselves, which improves convenience and is more user-friendly. It is especially suitable for situations where the amount of adjustment is large, improving the timeliness of decision confirmation and adjustment of the first task.

[0189] In addition, after a single traversal determines the adjustment rules for the second task, it means that the experts have completed the decision-making discussion on the pre-adjustment plan. Therefore, before the end of the single traversal, the second decision record is removed from the decision record sequence. After the removal is completed, the traversal continues, so that subsequent traversals do not traverse the discussion on the pre-adjustment plan to avoid meaningless operations.

[0190] An embodiment of the present invention provides a robot manipulation task control method, further comprising:

[0191] When the operator performs corresponding operations on the robot based on the subtask that requires manual operation in the second task, the operator is assisted in the operation;

[0192] Among them, the operation assistance to the operator includes:

[0193] Acquire a first picture of the palm direction of the palm of the robot through a first image acquisition device arranged next to the palm of the manipulator of the robot, and output it for display;

[0194] Continuously obtain the starting and ending positions of the palm at the beginning and end of the most recent preset time period;

[0195] Determine a first moving direction based on the starting position and the ending position;

[0196] Acquire device positions and shooting directions of a plurality of second image acquisition devices within a first range preset around the robot;

[0197] Based on a preset second feature extraction template, feature extraction is performed on the end position, the first moving direction, the device position, and the shooting direction to obtain a plurality of second feature values;

[0198] Based on the second eigenvalue, construct a first relationship description vector of the shooting angle relationship between the manipulator and the second image acquisition device;

[0199] Obtain a preset shooting angle relationship suitability evaluation library;

[0200] Determine a suitable value based on a shooting angle relationship suitability evaluation library and a first relationship description vector;

[0201] Acquire a second image of the manipulator through a second image acquisition device corresponding to the maximum fitness value, and output it for display;

[0202] Based on the second image, determining the object positions of a plurality of objects within a second range preset around the manipulator;

[0203] Obtain the object state of the object, the object state includes: static and dynamic;

[0204] When the object state of the object is static, based on a preset third feature extraction template, feature extraction is performed on the end position, the object position and the first moving direction to obtain a plurality of third feature values;

[0205] Based on the third eigenvalue, construct a second relational description vector of the relative positional relationship between the manipulator and the object;

[0206] Obtaining a preset relative position relationship collision risk determination library;

[0207] Determine a first collision risk determination result based on the relative position relationship collision risk determination library and the second relationship description vector;

[0208] When the first collision risk determination result is that there is a collision risk, outputting and displaying the first collision risk determination result;

[0209] When the object state of the object is dynamic, obtaining a second moving direction of the object;

[0210] Based on a preset fourth feature extraction template, feature extraction is performed on the end position, the object position, the first moving direction, and the second moving direction to obtain a plurality of fourth feature values;

[0211] Based on the fourth eigenvalue, construct a third relation description vector of the relative position relationship between the manipulator and the object;

[0212] Determining a second collision risk determination result based on the relative position relationship collision risk determination library and the third relationship description vector;

[0213] When the second collision risk determination result is that there is a collision risk, the second collision risk determination result is output and displayed.

[0214] The working principle and beneficial effects of the above technical solution are:

[0215] Generally, there may be subtasks in the second task that require manual operation. For example, when a new object needs to be moved temporarily, there is no need to reprogram and manual operation can be performed directly to save time. However, when manual operation is performed, the operator usually operates in the control room and can only see the first-person view of the robot's manipulator. Therefore, operation assistance is required.

[0216] A first image acquisition device is arranged beside the palm of the robot's manipulator, and the first image acquisition device can be a camera, etc. The first image acquisition device acquires a first picture of the palm of the hand, i.e., a first perspective picture of the manipulator, and outputs and displays it for the operator to view.

[0217] Next, it is necessary to obtain the on-site picture of the manipulator's movement for the operator to view, so as to avoid collision accidents during operation. However, the acquisition of the on-site picture needs to be targeted. Continuously obtain the starting position and ending position of the palm at the beginning and end moments of the most recent preset time period, and the most recent preset time period can be the most recent 1 second. Based on the starting position and the ending position, determine the first moving direction. The starting and ending positions determine the moving direction, which belongs to the scope of the prior art and will not be elaborated. Obtain the device position and shooting direction of multiple second image acquisition devices within the first range preset around the robot. The preset first range can be, for example: 3 meters. Introduce a preset second feature extraction template, extract features of the end position, the first moving direction, the device position and the shooting direction, and obtain multiple second feature values. The second feature value can be the distance between the end position and the device position and the angle between the first moving direction and the shooting direction. The second feature extraction template is a template adapted to extract such second feature values, which belongs to the scope of the prior art and will not be elaborated. Based on the second eigenvalue, the first relationship description vector of the shooting angle relationship between the manipulator and the second image acquisition device is constructed. The vector constructed based on the data information and the description vector of the data information belong to the scope of the prior art and will not be elaborated. A preset shooting angle relationship suitability evaluation library is introduced, and the shooting angle relationship suitability evaluation library stores the suitable values ​​corresponding to different first relationship description vectors. Generally, the closer the distance between the end position and the device position, the closer the angle between the first moving direction and the shooting direction is to 180° (when the manipulator moves completely facing the camera, the angle between the first moving direction and the shooting direction is 180°), which means that the closer the camera device is to the manipulator, the better the shooting angle, and the larger the suitable value corresponding to the constructed first relationship description vector. Therefore, the second image of the manipulator is obtained by the second image acquisition device corresponding to the maximum suitable value, and the output is displayed. The scene picture of the manipulator moving is shot at the best angle to ensure that the operator can view the scene picture in the direction of the manipulator moving, which improves the humanization of the assistance and is more intelligent.

[0218] In addition, a collision warning can also be performed. Based on the second image, the object positions of multiple objects within a preset second range around the manipulator are determined. The preset second range can be, for example, 0.5 meters. To obtain the object state of the object, the object type of the object on the second image can be determined based on image recognition technology, and the object state is determined according to the object type. For example, if the object is a railing, it is static, and if the object is another manipulator, it is dynamic. When the object state of the object is static, a preset third feature extraction template is introduced to extract features of the end position, the object position and the first moving direction to obtain multiple third feature values. The third feature value can be the distance between the end position and the object position and whether the first moving direction is toward the object position, etc. The third feature extraction template is a template adapted to extract such third feature values, which belongs to the scope of the prior art and will not be elaborated. Based on the third feature value, a second relationship description vector of the relative position relationship between the manipulator and the object is constructed. The vector constructed based on the data information and the description vector of the data information all belong to the scope of the prior art and will not be elaborated. A preset relative position relationship collision risk determination library is introduced, and the relative position relationship collision risk determination library stores the first collision risk determination results corresponding to different second relationship description vectors. Generally, if the distance between the end position and the object position is less than a certain value and / or the first moving direction is toward the object position, the collision risk determination result is that there is a collision risk. Based on the relative position relationship collision risk determination library and the second relationship description vector, the first collision risk determination result is determined. When the first collision risk determination result is that there is a collision risk, the first collision risk determination result is output and displayed, and a collision warning prompt is given to the operator. When the object state of the object is dynamic, the second moving direction of the object is obtained, and it can be obtained by docking with other manipulators. A preset fourth feature extraction template is introduced, and feature extraction is performed on the end position, the object position, the first moving direction and the second moving direction to obtain multiple fourth feature values. The fourth feature value can be, for example: the distance between the end position and the object position and whether the first moving direction and the second moving direction are relatively equal. The fourth feature extraction template is a template adapted to extract such fourth feature values, which belongs to the scope of the prior art and will not be described in detail. Based on the fourth eigenvalue, a third relationship description vector of the relative position relationship between the manipulator and the object is constructed. The vector constructed based on the data information and the description vector of the data information are all within the scope of the prior art and will not be elaborated on. The relative position relationship collision risk judgment library also stores the second collision risk judgment results corresponding to different third relationship description vectors. Generally, the distance between the end position and the object position is less than a certain value and / or the first moving direction and the second moving direction are relative, and the collision risk judgment result is that there is a collision risk. Based on the relative position relationship collision risk judgment library and the third relationship description vector, the second collision risk judgment result is determined. When the second collision risk judgment result is that there is a collision risk, the second collision risk judgment result is output and displayed, and a collision warning prompt is given to the operator.It improves the comprehensiveness and efficiency of collision warning and the safety of manual operation, and is particularly suitable for situations where robots manipulate dynamic and static objects on site.

[0219] The embodiment of the present invention provides a robot manipulation task control system, such as Figure 2 As shown, including:

[0220] An acquisition module 1 is used to acquire the execution status of the first task currently being executed by the manipulation robot;

[0221] An adjustment module 2, configured to adaptively adjust the first task based on the execution status to obtain an adjusted second task;

[0222] The manipulation module 3 is used to perform relay manipulation on the robot based on the second task.

[0223] The embodiment of the present invention provides a robot manipulation task control system, wherein the acquisition module 1 acquires the execution status of the first task currently being executed by the manipulation robot, including:

[0224] Acquire first state information of the robot when the robot is controlled to perform the first task, and use the information as the execution status;

[0225] and / or,

[0226] Acquire second state information of a field device operated by the robot when the robot is operated to perform the first task;

[0227] and / or,

[0228] A first staff node corresponding to a preset staff member of the robot is obtained, and the execution status of the first task currently being performed by the robot, which is uploaded by the staff member, is obtained through the first staff node.

[0229] The embodiment of the present invention provides a robot manipulation task control system, wherein the adjustment module 2 adaptively adjusts the first task based on the execution status, including:

[0230] Based on a preset first feature extraction template, feature extraction is performed on the execution status to obtain a plurality of first feature values;

[0231] Based on the first eigenvalue, construct a situation description vector of the execution situation;

[0232] Get a preset situation complexity evaluation library;

[0233] Determine the complexity based on the situation complexity evaluation library and the situation description vector;

[0234] If the complexity is less than or equal to a preset complexity threshold, obtaining a preset task adjustment model, inputting the situation description vector and the first task into the task adjustment model, and determining an adjusted second task;

[0235] Otherwise, obtaining an idle second staff node of a preset expert staff corresponding to the robot;

[0236] Count the number of second personnel nodes;

[0237] If the number is 1, the first feature value and the first task are delivered to the second staff node for review by the expert staff;

[0238] Obtaining the adjusted second task replied by the expert personnel through the second personnel node;

[0239] If the number is not 1, build an online decision room;

[0240] Connecting expert personnel to the online decision-making room through the second personnel node;

[0241] Outputting and displaying the first characteristic value and the first task in the online decision room for the expert to view;

[0242] Acquire a first decision record generated by an expert in an online decision room, the first decision record comprising: a voice chat record and a text chat record, or a combination of both;

[0243] When all the experts leave the online decision-making room, the first task is adjusted based on the first decision record to obtain an adjusted second task.

[0244] The embodiment of the present invention provides a robot manipulation task control system, wherein the adjustment module 2 adjusts the first task based on the first decision record, including:

[0245] Obtaining the generation time of the first decision record;

[0246] Based on the time when the records were generated, the first decision records were sorted in chronological order to obtain a decision record sequence;

[0247] Performing semantic extraction on the first decision record to obtain first semantics;

[0248] Traverse the first decision record from the starting point of the decision record sequence to the end point of the sequence;

[0249] During each traversal, a preset decision semantic library is obtained, and a first semantic of a first decision record traversed is matched with a second semantic in the decision semantic library;

[0250] If there is a match, stop traversal and obtain a preset adjustment semantic library corresponding to the second semantics that matches the match;

[0251] Matching the first semantics of the first decision record after the first decision record traversed in the decision record sequence with the third semantics in the adjustment semantic library;

[0252] If none of them matches, obtaining a preset first task adjustment rule corresponding to the second semantics that matches;

[0253] If there is a match, the first decision record corresponding to the first semantics of the match is used as the second decision record;

[0254] Acquire a preset second task adjustment rule corresponding to a third semantics matching the first semantics of the second decision record closest to the sequence end point in the second decision record;

[0255] Eliminate the second decision record from the decision record sequence;

[0256] After the elimination is completed, continue the traversal;

[0257] After the traversal is completed, the first task is adjusted in sequence based on the first task adjustment rule and the second task adjustment rule.

[0258] An embodiment of the present invention provides a robot manipulation task control system, further comprising:

[0259] an auxiliary module, used for assisting the operator in manipulating the robot when the operator is manipulating the robot based on the subtask that requires manual manipulation in the second task;

[0260] Among them, the auxiliary module provides operation assistance to the operator, including:

[0261] Acquire a first picture of the palm direction of the palm of the robot through a first image acquisition device arranged next to the palm of the manipulator of the robot, and output it for display;

[0262] Continuously obtain the starting and ending positions of the palm at the beginning and end of the most recent preset time period;

[0263] Determine a first moving direction based on the starting position and the ending position;

[0264] Acquire device positions and shooting directions of a plurality of second image acquisition devices within a first range preset around the robot;

[0265] Based on a preset second feature extraction template, feature extraction is performed on the end position, the first moving direction, the device position, and the shooting direction to obtain a plurality of second feature values;

[0266] Based on the second eigenvalue, construct a first relationship description vector of the shooting angle relationship between the manipulator and the second image acquisition device;

[0267] Obtain a preset shooting angle relationship suitability evaluation library;

[0268] Determine a suitable value based on a shooting angle relationship suitability evaluation library and a first relationship description vector;

[0269] Acquire a second image of the manipulator through a second image acquisition device corresponding to the maximum fitness value, and output it for display;

[0270] Based on the second image, determining the object positions of a plurality of objects within a second range preset around the manipulator;

[0271] Obtain the object state of the object, the object state includes: static and dynamic;

[0272] When the object state of the object is static, based on a preset third feature extraction template, feature extraction is performed on the end position, the object position and the first moving direction to obtain a plurality of third feature values;

[0273] Based on the third eigenvalue, construct a second relational description vector of the relative positional relationship between the manipulator and the object;

[0274] Obtaining a preset relative position relationship collision risk determination library;

[0275] Determine a first collision risk determination result based on the relative position relationship collision risk determination library and the second relationship description vector;

[0276] When the first collision risk determination result is that there is a collision risk, outputting and displaying the first collision risk determination result;

[0277] When the object state of the object is dynamic, obtaining a second moving direction of the object;

[0278] Based on a preset fourth feature extraction template, feature extraction is performed on the end position, the object position, the first moving direction, and the second moving direction to obtain a plurality of fourth feature values;

[0279] Based on the fourth eigenvalue, construct a third relation description vector of the relative position relationship between the manipulator and the object;

[0280] Determining a second collision risk determination result based on the relative position relationship collision risk determination library and the third relationship description vector;

[0281] When the second collision risk determination result is that there is a collision risk, the second collision risk determination result is output and displayed.

[0282] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A robot manipulation task control method, characterized in that: include: Step 1: Obtain the execution status of the first task currently being executed by the manipulation robot; Step 2: Based on the execution status, adaptively adjust the first task to obtain an adjusted second task; Step 3: Based on the second task, performing relay operation on the robot; In step 2, adaptively adjusting the first task based on the execution status includes: Based on a preset first feature extraction template, feature extraction is performed on the execution status to obtain a plurality of first feature values; constructing a situation description vector of the execution situation based on the first eigenvalue; Get a preset situation complexity evaluation library; Determining complexity based on the situation complexity evaluation library and the situation description vector; If the complexity is less than or equal to a preset complexity threshold, obtaining a preset task adjustment model, inputting the situation description vector and the first task into the task adjustment model, and determining an adjusted second task; Otherwise, obtaining an idle second staff node of a preset expert staff corresponding to the robot; Counting the number of the second personnel nodes; If the number is 1, the first feature value and the first task are delivered to the second staff node for the expert staff to review; Acquire the adjusted second task replied by the expert through the second staff node; If the number is not 1, construct an online decision room; Connecting the expert personnel to the online decision-making room through the second personnel node; Outputting and displaying the first feature value and the first task in the online decision room for the expert to view; Acquire a first decision record generated by the expert in the online decision room, wherein the first decision record includes: a voice chat record and a text chat record, or a combination of the two; When all the experts leave the online decision room, the first task is adjusted based on the first decision record to obtain an adjusted second task.

2. A robot manipulation task control method as claimed in claim 1, characterized in that: The step 1: obtaining the execution status of the first task currently being executed by the manipulator robot, comprises: Acquire first state information of the robot when the robot is controlled to perform a first task, and use the information as the execution status; and / or, Acquire second state information of a field device operated by the robot when the robot is operated to perform a first task; and / or, A first staff node corresponding to a preset staff member of the robot is obtained, and the execution status of the first task currently being performed by the robot uploaded by the staff member is obtained through the first staff node.

3. A robot manipulation task control method as claimed in claim 1, characterized in that: The adjusting the first task based on the first decision record includes: Obtaining the generation time of the first decision record; Based on the generation time of the records, the first decision records are sorted in chronological order to obtain a decision record sequence; Performing semantic extraction on the first decision record to obtain first semantics; Traversing the first decision records in sequence from the starting point of the decision record sequence to the end point of the sequence; During each traversal, a preset decision semantic library is obtained, and the first semantics of the first decision record traversed is matched with the second semantics in the decision semantic library; If there is a match, stop traversing and obtain a preset adjustment semantic library corresponding to the second semantic that matches the match; Matching the first semantics of the first decision record after the first decision record traversed in the decision record sequence with the third semantics in the adjustment semantic library; If no match is found, obtaining a preset first task adjustment rule corresponding to the second semantics that matches; If there is a match, taking the first decision record corresponding to the first semantics that matches the match as the second decision record; Acquire a preset second task adjustment rule corresponding to the third semantics that matches the first semantics of the second decision record closest to the sequence end point in the second decision record; Eliminating the second decision record from the decision record sequence; After the elimination is completed, continue the traversal; After the traversal is completed, the first task is adjusted in sequence based on the first task adjustment rule and the second task adjustment rule.

4. A robot manipulation task control method as claimed in claim 1, characterized in that: Also includes: When the operator performs corresponding operations on the robot based on the subtask requiring manual operation in the second task, providing operation assistance to the operator; The step of providing operating assistance to the operator includes: Acquire a first picture of the palm direction of the manipulator of the robot by a first image acquisition device arranged next to the palm of the manipulator of the robot, and output it for display; Continuously obtaining the starting position and ending position of the palm at the beginning and end moments in a recent preset time period; Determining a first moving direction based on the starting position and the ending position; Acquire device positions and shooting directions of a plurality of second image acquisition devices within a first range preset around the robot; Based on a preset second feature extraction template, feature extraction is performed on the end position, the first moving direction, the device position, and the shooting direction to obtain a plurality of second feature values; Based on the second eigenvalue, construct a first relationship description vector of the shooting angle relationship between the manipulator and the second image acquisition device; Obtain a preset shooting angle relationship suitability evaluation library; Determining a suitable value based on the shooting angle relationship suitability evaluation library and the first relationship description vector; Acquire a second image of the manipulator through the second image acquisition device corresponding to the maximum suitable value, and output it for display; Based on the second image, determining the object positions of a plurality of objects within a second range preset around the manipulator; Acquiring an object state of the object, wherein the object state includes: static state and dynamic state; When the object state of the object is static, based on a preset third feature extraction template, feature extraction is performed on the end position, the object position and the first moving direction to obtain a plurality of third feature values; Based on the third eigenvalue, construct a second relationship description vector of the relative position relationship between the manipulator and the object; Obtaining a preset relative position relationship collision risk determination library; Determining a first collision risk determination result based on the relative position relationship collision risk determination library and the second relationship description vector; When the first collision risk determination result is that there is a collision risk, outputting and displaying the first collision risk determination result; When the object state of the object is dynamic, obtaining a second moving direction of the object; Based on a preset fourth feature extraction template, feature extraction is performed on the end position, the object position, the first moving direction, and the second moving direction to obtain a plurality of fourth feature values; Based on the fourth eigenvalue, construct a third relationship description vector of the relative position relationship between the manipulator and the object; Determining a second collision risk determination result based on the relative position relationship collision risk determination library and the third relationship description vector; When the second collision risk determination result is that there is a collision risk, the second collision risk determination result is output and displayed.

5. A robot manipulation task control system, characterized in that: include: An acquisition module, used for acquiring the execution status of the first task currently being executed by the manipulation robot; An adjustment module, configured to adaptively adjust the first task based on the execution status to obtain an adjusted second task; A manipulation module, used for performing relay manipulation on the robot based on the second task; The adjustment module adaptively adjusts the first task based on the execution status, including: Based on a preset first feature extraction template, feature extraction is performed on the execution status to obtain a plurality of first feature values; constructing a situation description vector of the execution situation based on the first eigenvalue; Get a preset situation complexity evaluation library; Determining complexity based on the situation complexity evaluation library and the situation description vector; If the complexity is less than or equal to a preset complexity threshold, obtaining a preset task adjustment model, inputting the situation description vector and the first task into the task adjustment model, and determining an adjusted second task; Otherwise, obtaining an idle second staff node of a preset expert staff corresponding to the robot; Counting the number of the second personnel nodes; If the number is 1, the first feature value and the first task are delivered to the second staff node for the expert staff to review; Acquire the adjusted second task replied by the expert through the second staff node; If the number is not 1, construct an online decision room; Connecting the expert personnel to the online decision-making room through the second personnel node; Outputting and displaying the first feature value and the first task in the online decision room for the expert to view; Acquire a first decision record generated by the expert in the online decision room, wherein the first decision record includes: a voice chat record and a text chat record, or a combination of the two; When all the experts leave the online decision room, the first task is adjusted based on the first decision record to obtain an adjusted second task.

6. A robot manipulation task control system as claimed in claim 5, characterized in that: The acquisition module acquires the execution status of the first task currently being executed by the manipulation robot, including: Acquire first state information of the robot when the robot is controlled to perform a first task, and use the information as the execution status; and / or, Acquire second state information of a field device operated by the robot when the robot is operated to perform a first task; and / or, A first staff node corresponding to a preset staff member of the robot is obtained, and the execution status of the first task currently being performed by the robot uploaded by the staff member is obtained through the first staff node.

7. A robot manipulation task control system as claimed in claim 5, characterized in that: The adjusting module adjusts the first task based on the first decision record, including: Obtaining the generation time of the first decision record; Based on the generation time of the records, the first decision records are sorted in chronological order to obtain a decision record sequence; Performing semantic extraction on the first decision record to obtain first semantics; Traversing the first decision records in sequence from the starting point of the decision record sequence to the end point of the sequence; During each traversal, a preset decision semantic library is obtained, and the first semantics of the first decision record traversed is matched with the second semantics in the decision semantic library; If there is a match, stop traversing and obtain a preset adjustment semantic library corresponding to the second semantic that matches the match; Matching the first semantics of the first decision record after the first decision record traversed in the decision record sequence with the third semantics in the adjustment semantic library; If no match is found, obtaining a preset first task adjustment rule corresponding to the second semantics that matches; If there is a match, taking the first decision record corresponding to the first semantics that matches the match as the second decision record; Acquire a preset second task adjustment rule corresponding to the third semantics that matches the first semantics of the second decision record closest to the sequence end point in the second decision record; Eliminating the second decision record from the decision record sequence; After the elimination is completed, continue the traversal; After the traversal is completed, the first task is adjusted in sequence based on the first task adjustment rule and the second task adjustment rule.

8. A robot manipulation task control system as claimed in claim 5, characterized in that: Also includes: an auxiliary module, configured to assist the operator in manipulating the robot when the operator is manipulating the robot based on the subtask in the second task that requires manual manipulation; The auxiliary module assists the operator in operation, including: Acquire a first picture of the palm direction of the manipulator of the robot by a first image acquisition device arranged next to the palm of the manipulator of the robot, and output it for display; Continuously obtaining the starting position and ending position of the palm at the beginning and end moments in a recent preset time period; Determining a first moving direction based on the starting position and the ending position; Acquire device positions and shooting directions of a plurality of second image acquisition devices within a first range preset around the robot; Based on a preset second feature extraction template, feature extraction is performed on the end position, the first moving direction, the device position, and the shooting direction to obtain a plurality of second feature values; Based on the second eigenvalue, construct a first relationship description vector of the shooting angle relationship between the manipulator and the second image acquisition device; Obtain a preset shooting angle relationship suitability evaluation library; Determining a suitable value based on the shooting angle relationship suitability evaluation library and the first relationship description vector; Acquire a second image of the manipulator through the second image acquisition device corresponding to the maximum suitable value, and output it for display; Based on the second image, determining the object positions of a plurality of objects within a second range preset around the manipulator; Acquiring an object state of the object, wherein the object state includes: static state and dynamic state; When the object state of the object is static, based on a preset third feature extraction template, feature extraction is performed on the end position, the object position and the first moving direction to obtain a plurality of third feature values; Based on the third eigenvalue, construct a second relationship description vector of the relative position relationship between the manipulator and the object; Obtaining a preset relative position relationship collision risk determination library; Determining a first collision risk determination result based on the relative position relationship collision risk determination library and the second relationship description vector; When the first collision risk determination result is that there is a collision risk, outputting and displaying the first collision risk determination result; When the object state of the object is dynamic, obtaining a second moving direction of the object; Based on a preset fourth feature extraction template, feature extraction is performed on the end position, the object position, the first moving direction, and the second moving direction to obtain a plurality of fourth feature values; Based on the fourth eigenvalue, construct a third relationship description vector of the relative position relationship between the manipulator and the object; Determining a second collision risk determination result based on the relative position relationship collision risk determination library and the third relationship description vector; When the second collision risk determination result is that there is a collision risk, the second collision risk determination result is output and displayed.

Images