Robot task completion method, device and equipment and medium

Abstract

The invention discloses a robot task completion method and device, equipment and a medium, and relates to the field of robotics.The robot task completion method comprises the steps that when a target task is executed, the confidence coefficient of the target task completion success rate is determined, and whether the robot needs to seek external assistance or not is judged according to the size relation between the confidence coefficient and a preset confidence coefficient threshold value; if yes, constructing a space map in the target range, identifying and positioning an object group capable of providing help, determining a plurality of groups of candidate objects capable of seeking help based on the space map and the object group, and determining help strength values corresponding to the candidate objects capable of seeking help; determining a target help-seeking candidate object, interacting with the target help-seeking candidate object to obtain guidance information, and determining a plurality of executable tasks based on the guidance information; and after the plurality of executable tasks are executed, judging whether the target task is completed or not, and if the target task is not completed, skipping to the step of constructing the space map in the target range again until the target task is completed. And robot task completion is ensured.

Description

Technical Field

[0001] The present invention relates to the field of robots, and in particular to a method, device, equipment and medium for completing a robot task. Background Art

[0002] As robots are widely used in industrial manufacturing, service industry, exploration and other fields, they often encounter unexpected situations in complex and changing environments, such as partial sensor failure, position loss or incomplete information, which makes them unable to complete tasks independently. Existing robot navigation and task execution strategies mainly rely on pre-programmed algorithms and sensor data, and lack the ability to actively seek external assistance when encountering difficulties. This makes it difficult for robots to flexibly adapt to environmental changes, affecting the efficiency and success rate of tasks. Therefore, how to improve the efficiency and success rate of task completion is an urgent problem to be solved. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a robot task completion method, device, equipment and medium, so that the robot can seek external assistance when it cannot complete the task, efficiently integrate the obtained help information to complete the task, and improve the efficiency and success rate of task completion. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses a robot task completion method, comprising:

[0005] In the process of executing the target task, determining the confidence level of the target task completion success rate, and judging whether the robot needs to seek external assistance based on the relationship between the confidence level and a preset confidence threshold;

[0006] If yes, construct a spatial map within the target range, identify and locate the group of objects that can provide help, determine a number of groups of candidate objects that can be helped based on the spatial map and the group of objects, and determine the help value corresponding to each candidate object that can be helped;

[0007] Determine a target candidate object for help corresponding to the highest help value, interact with the target candidate object for help to obtain corresponding guidance information, and determine a number of executable tasks based on the guidance information;

[0008] After executing a number of the executable tasks, it is determined whether the target task is completed. If not, the process jumps back to the step of constructing a spatial map within the target range until the target task is completed.

[0009] Optionally, the determination of the confidence level of the target task completion success rate includes:

[0010] Determining the reliability of each sensor data based on the signal strength, signal quality and signal integrity of each sensor of the robot;

[0011] Determining sensor data confidence that characterizes integrity and accuracy of sensor data according to preset sensor weights corresponding to each sensor of the robot and the reliability of the sensor data;

[0012] Determine a battery confidence level for characterizing the health of the battery through the capacity, power loss rate, and voltage of the battery of the robot;

[0013] Determining the ground complexity according to the number of obstacles on the road surface, the degree of terrain undulation, and the width of the driving path in the current environment of the robot;

[0014] Determine an environmental confidence level for characterizing the complexity of the current environment using environmental conditions corresponding to the current environment of the robot and the complexity of the ground; the environmental conditions include light intensity, temperature, humidity and wind speed;

[0015] Determining a target confidence level for characterizing the degree of support of the environmental data for the target task based on a vector corresponding to the environmental data detected by the robot and the environmental data required to complete the target task;

[0016] The confidence of the target task completion success rate is determined based on the sensor data confidence, the battery confidence, the environment confidence, and the target confidence.

[0017] Optionally, the determining the confidence of the target task completion success rate based on the sensor data confidence, the battery confidence, the environment confidence, and the target confidence includes:

[0018] The confidence of the target task completion success rate is determined based on the sensor data confidence, the battery confidence, the environment confidence and the target confidence through a confidence calculation formula; the confidence calculation formula is:

[0019] ;

[0020] in, is the confidence level; is the sensor data confidence; is the battery confidence level; A confidence level for the environment; is the target confidence; is the target threshold; w1, w2, w3, w4 are the preset weight values.

[0021] Optionally, judging whether the robot needs to seek external assistance according to the relationship between the confidence and a preset confidence threshold includes:

[0022] If the confidence level is less than or equal to the preset confidence threshold, it is determined that the robot needs to seek external assistance;

[0023] If the confidence level is greater than the preset confidence threshold, it is determined that the robot does not need to seek external assistance.

[0024] Optionally, the determining a plurality of groups of candidate objects for assistance based on the spatial map and the object group includes:

[0025] Constructing group structure data based on the spatial map and the information corresponding to the object groups; the group structure data is data defining labels of each object group, a surrounding map of each object group, a minimum cost path from the robot to each object group, an age of each object group, and a social role of each object group;

[0026] Several groups of candidate objects for assistance are determined according to the group structure data.

[0027] Optionally, determining the assistance value corresponding to each candidate object for help includes:

[0028] Determine the social attributes of the candidate object for help based on the clothes and facial expressions of the candidate object for help; the social attributes include any one or a combination of children, teenagers, middle-aged people, the elderly, primary and secondary school students, high school students and social service providers;

[0029] Determining the appearance affinity value of the candidate object for help by using the preset scores corresponding to each social attribute;

[0030] Determine the behavior value of the candidate object for help according to the walking speed and body movements of the candidate object for help; the body movements include any one or a combination of empty-handed, carrying items and making phone calls; the carrying items movement includes several types of carrying items movement obtained by dividing based on the estimated weight of the items;

[0031] Determining an action cost value for the robot to reach the location of the candidate object for help based on the distance between the robot and the candidate object for help;

[0032] The help-giving value of each candidate object for help is determined according to the appearance affinity value, the behavior value and the action cost value corresponding to each candidate object for help.

[0033] Optionally, the process of interacting with the target candidate object for help to obtain corresponding guidance information further includes:

[0034] Performing face detection and facial emotion recognition on the target candidate for help, determining the facial expression of the target candidate for help, and determining the score corresponding to the facial expression based on the score corresponding to the predetermined facial expression classification;

[0035] Determining a sentiment score of the semantics of the target help-seeking candidate's speech based on a predetermined semantic sentiment score;

[0036] Determining the emotion score of the body movement of the target candidate object for help by using a predetermined body movement score;

[0037] Determine, based on the score corresponding to the facial expression, the sentiment score of the semantics of the speech, and the sentiment score of the body movement, a sentiment willingness value for representing whether the target candidate for help is willing to provide help;

[0038] If the emotional willingness value is less than or equal to the preset emotional willingness threshold, then stop interacting with the target candidate for help, stay away from the target candidate for help, and re-determine a new candidate for help to interact with, so as to obtain corresponding guidance information;

[0039] If the emotional willingness value is greater than the preset emotional willingness threshold, interaction is performed with the target candidate for help to obtain corresponding guidance information.

[0040] Optionally, after the step of jumping back to the step of constructing the spatial map within the target range, the method further includes:

[0041] If the number of times the operation of re-jumping to the step of building the spatial map within the target range is executed is equal to a preset number threshold, and the target task is not completed, help is sought from the operation and maintenance personnel to complete the target task.

[0042] In a second aspect, the present application discloses a robot task completion device, comprising:

[0043] A judgment module, used to determine the confidence level of the target task completion success rate during the execution of the target task, and to judge whether the robot needs to seek external assistance based on the relationship between the confidence level and a preset confidence threshold;

[0044] a help value determination module, for, if yes, constructing a spatial map within the target range, identifying and locating a group of objects that can provide help, determining a number of groups of candidate objects that can be helped based on the spatial map and the group of objects, and determining a help value corresponding to each candidate object that can be helped;

[0045] An executable task determination module is used to determine a target candidate for help corresponding to the highest help value, interact with the target candidate for help to obtain corresponding guidance information, and determine a number of executable tasks based on the guidance information;

[0046] The jump module is used to determine whether the target task is completed after executing several executable tasks. If not, it will jump back to the step of constructing a spatial map within the target range until the target task is completed.

[0047] In a third aspect, the present application discloses an electronic device, comprising:

[0048] Memory, used to store computer programs;

[0049] The processor is used to execute the computer program to implement the robot task completion method as described above.

[0050] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the robot task completion method as described above is implemented.

[0051] In the present application, when the robot is performing the target task, it first determines the confidence of the target task completion success rate, and judges whether the robot needs to seek external assistance according to the size relationship between the confidence and the preset confidence threshold; if so, it constructs a spatial map within the target range, identifies and locates the object group that can provide help, determines several groups of candidate objects that can be asked for help based on the spatial map and the object group, and determines the assistance value corresponding to each candidate object that can be asked for help; then determines the target candidate object that can be asked for help corresponding to the highest assistance value, interacts with the target candidate object that can be asked for help to obtain corresponding guidance information, and determines several executable tasks based on the guidance information; finally, after executing several executable tasks, it determines whether the target task is completed. If not, it jumps back to the step of constructing the spatial map within the target range until the target task is completed. It can be seen that the present application determines whether the robot can complete the task independently through confidence. If it cannot be completed, it seeks external help, so that the robot can autonomously seek external assistance, efficiently integrate the obtained help information to complete the task, and improve the reliability of task completion. At the same time, in the process of seeking external help, the target candidate for help is determined through the help degree value, the selection efficiency of the help object is optimized, and the cost of seeking help is minimized by evaluating and selecting the most suitable help object, thereby improving the efficiency and success rate of task completion. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0053] Figure 1 A flow chart of a robot task completion method disclosed in this application;

[0054] Figure 2 A schematic diagram of a structural data construction method disclosed in this application;

[0055] Figure 3 A logical diagram of information processing and execution disclosed in this application;

[0056] Figure 4 A flowchart of a specific robot task completion method disclosed in this application;

[0057] Figure 5 This is a schematic diagram of the structure of a robot task completion device disclosed in this application;

[0058] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Existing robot navigation and task execution strategies mainly rely on pre-programmed algorithms and sensor data, and lack the ability to actively seek external assistance when encountering difficulties. This makes it difficult for robots to flexibly adapt to environmental changes, affecting the efficiency and success rate of tasks. In order to solve the above technical problems, the present application discloses a robot task completion method, device, equipment and medium, which enables the robot to seek external assistance when it cannot complete the task, efficiently integrate the obtained help information to complete the task, and improve the efficiency and success rate of task completion.

[0061] See also Figure 1 As shown, an embodiment of the present invention discloses a robot task completion method, comprising:

[0062] Step S11: during the execution of the target task, determine the confidence level of the target task completion success rate, and determine whether the robot needs to seek external assistance based on the relationship between the confidence level and a preset confidence threshold.

[0063] In this embodiment, when the robot is performing a task (for example, moving from location A to location B), it first performs a capability assessment. That is, if it loses its direction or landmarks along the way, it first determines that the current perception information cannot support it to reach its destination, and starts the "autonomous assistance seeking" mode; here, "self-capability assessment" is a confidence calculation logic for the success rate of task completion. When determining the confidence level of the target task completion success rate, the reliability of each sensor data is determined based on the signal strength, signal quality and signal integrity of each sensor of the robot; the sensor data confidence level that characterizes the integrity and accuracy of the sensor data is determined based on the preset sensor weights corresponding to each sensor of the robot and the reliability of the sensor data; the battery confidence level that characterizes the health of the battery is determined by the capacity, power failure rate and voltage of the robot's battery; the ground complexity is determined based on the number of obstacles on the road surface, the degree of terrain undulation and the width of the driving path in the robot's current environment; the environmental confidence level that characterizes the complexity of the current environment is determined using the environmental conditions corresponding to the robot's current environment and the ground complexity; the environmental conditions include light intensity, temperature, humidity and wind speed; the target confidence level that characterizes the degree of support of the environmental data for the target task is determined based on the vector corresponding to the environmental data detected by the robot and the environmental data required to complete the target task; the confidence level of the target task completion success rate is determined based on the sensor data confidence level, battery confidence level, environmental confidence level and target confidence level. Specifically, the confidence calculation formula is:

[0064] ;

[0065] in, is the confidence level; is the confidence of sensor data; is the battery confidence level; Provide confidence for the environment; is the target confidence; is the target threshold (the red line threshold of a single factor, when the confidence of any factor ≤ When the robot starts the "autonomous assistance seeking" mode based on the factor to solve the current factor problem, It can be a unified value, or a red line threshold can be set for each factor); w1, w2, w3, w4 are preset weight values, that is, the weights of each factor, reflecting the degree of influence of these factors on the confidence, and the sum of the weights is 1. In a specific embodiment, for an indoor working environment, w1, w2, w3, w4 can be set to 0.3, 0.2, 0.2, 0.3, respectively. For an outdoor working environment, w1, w2, w3, w4 are set to 0.3, 0.15, 0.15, 0.4, respectively. For weights, the weight parameters can also be dynamically adjusted through an environmental perception model.

[0066] Then, based on the relationship between the confidence level and the preset confidence threshold, it is determined whether the robot needs to seek external assistance. If the confidence level is less than or equal to the preset confidence threshold, it is determined that the robot needs to seek external assistance, that is, it needs to seek external assistance to continue to complete the task. At this time, the robot enters the "autonomous assistance" mode. If the confidence level is greater than the preset confidence threshold, it is determined that the robot does not need to seek external assistance and can complete the task autonomously. In this way, the robot's autonomy and task completion rate are improved. The robot has the ability of "self-assessment of ability" and "autonomous assistance seeking". When the task is difficult, it can make intelligent judgments and seek external help, thereby improving the reliability of task completion.

[0067] Step S12: If yes, construct a spatial map within the target range, identify and locate the group of objects that can provide help, determine several groups of candidate objects for help based on the spatial map and the group of objects, and determine the help value corresponding to each candidate object for help.

[0068] In this embodiment, when the confidence level is less than or equal to the preset confidence threshold, it is determined that the robot needs to seek external assistance, and the robot enters the "autonomous assistance seeking" mode, which specifically includes environmental analysis, assistance evaluation, active assistance based on emotional cognition, information processing and execution, and continuous optimization. First, the robot performs environmental analysis, perceives and understands the surrounding environment, identifies potential groups of help objects, and models the surrounding environment. The specific process of environmental analysis is: the robot collects data about the surrounding environment through the sensor, maps the environment within a range of 20m around it, and obtains a spatial map S within a range of 20m around the robot. map The robot uses visual sensors, computer vision, and deep learning models to identify and locate groups of people who may be able to provide assistance, such as pedestrian recognition, face detection, and age and gender detection. helper , such as student groups, youth groups, and social worker groups. Then, based on the spatial map and the object groups, several groups of candidate objects for help are determined. Specifically, group structure data is constructed based on the information corresponding to the spatial map and the object groups; Figure 2As shown, the group structure data is data that defines the labels of each object group, the surrounding map of each object group, the minimum cost path from the robot to each object group, the age of each object group, and the social role of each object group; based on the group structure data, several groups of candidate objects for help are determined.

[0069] When the robot selects N groups of candidate groups for help SL helper After that, the robot enters the "assistance evaluation" stage and evaluates the candidate group SL helper For each group in the game, the help value of each group is calculated based on comprehensive factors such as people's appearance, actions, and arrival costs. , and then the robot gives priority to the group with the largest help value to seek help. When determining the help value corresponding to each candidate for help, the social attributes of the candidate for help are determined according to the candidate's clothing and facial expressions; social attributes include any one or a combination of children, teenagers, middle-aged people, the elderly, elementary and middle school students, high school students and social service providers; the appearance affinity value of the candidate for help is determined by the preset scores corresponding to each social attribute; the behavior value of the candidate for help is determined according to the walking speed and body movements of the candidate for help; body movements include any one or a combination of empty hands, carrying items and making phone calls; carrying items includes several types of carrying items actions obtained after classification based on the estimated weight of the items; the action cost value of the robot reaching the position of the candidate for help is determined based on the distance between the robot and the candidate for help; the help value of each candidate for help is determined by the appearance affinity value, behavior value and action cost value corresponding to each candidate for help. Specifically, the help evaluation formula is:

[0070] ;

[0071] in, is the power value; is the appearance affinity value; is the behavior value; is the action cost; w1, w2, w3 are the weights of each factor. Here, the weight factors can be set to 0.3, 0.4, and 0.3 respectively. The weight factors can also be set dynamically based on different tasks.

[0072] Next, the determination of the appearance affinity value, behavior value, and action cost value is described in detail. It reflects the appearance affinity of the candidate group, and calculates the social attributes of the candidate group through clothing and facial expressions, and whether they are willing to help strangers. Social attributes refer to the social position and age attributes of the group, including children, teenagers, middle-aged people, the elderly, primary and secondary school students, high school students, social service workers, etc. Different social attribute groups are given different help scores:

[0073] ;

[0074] It is the behavioral factor of the candidate group. It determines whether the candidate group has urgent matters to deal with or is dealing with them through walking speed, body movements, etc. The robot gives priority to asking for help from the candidate group that walks steadily and has no urgent matters to deal with. For walking speed, the optical flow method is used to estimate the walking speed of the group. For body movements , including empty-handed, carrying a handbag, carrying heavy objects, talking on the phone, etc.:

[0075] ;

[0076] Here, you can set the maximum walking speed V max is 2m / s. Walking speed weight w v , body movement weight w c Can be set to 0.6, 0.4:

[0077] ;

[0078] The robot calculates the action cost of reaching the candidate group position by distance, and the robot gives priority to the candidate group with low action cost. The closer the distance, the lower the cost:

[0079] ;

[0080] in, is the action cost; D is the distance between the robot and the candidate object for help; The normalized maximum distance can be set to 20m.

[0081] In this way, the cost for the robot to seek help is optimized. The robot builds an accurate environmental model through technologies such as pan-perception, pan-detection, and pan-prediction of the environment, thereby optimizing the efficiency of selecting the person seeking help.

[0082] Step S13: determining a target candidate object for help corresponding to the highest help value, interacting with the target candidate object for help to obtain corresponding guidance information, and determining a number of executable tasks based on the guidance information.

[0083] In this embodiment, the help value of each group is calculated, all candidate groups are sorted, and then the group with the highest help value is selected for help. Afterwards, based on the help request based on emotional calculation, the robot interacts with the selected help object and tries to obtain the required information from the help object. While communicating with the selected help object, the help object's emotional willingness value is maintained to ensure that the help object is willing to provide help during the entire communication process, so as to reduce the discomfort caused by the robot's autonomous help seeking. The emotional willingness value is calculated based on the facial expression, speech semantics, and body movements of the target help-seeking candidate. Face detection and facial emotion recognition are performed on the target help-seeking candidate to determine the facial expression of the target help-seeking candidate. The score corresponding to the facial expression is determined based on the score corresponding to the predetermined facial expression classification; the emotional score of the speech semantics of the target help-seeking candidate is determined based on the predetermined semantic emotion score; the emotional score of the body movement of the target help-seeking candidate is determined by the predetermined body movement score; the emotional willingness value used to characterize whether the target help-seeking candidate is willing to provide help is determined based on the score corresponding to the facial expression, the emotional score of the speech semantics, and the emotional score of the body movement; if the emotional willingness value is less than or equal to the preset emotional willingness threshold, then stop interacting with the target help-seeking candidate, stay away from the target help-seeking candidate, and re-determine a new help-seeking candidate to interact with in order to obtain corresponding guidance information; if the emotional willingness value is greater than the preset emotional willingness threshold, then interact with the target help-seeking candidate to obtain corresponding guidance information. Among them, the specific calculation formula of the emotional willingness value is:

[0084] ;

[0085] in, is the emotional willingness value; is the score corresponding to the facial expression; Score the semantic sentiment of the utterance; is the emotional score of body movements; w1, w2, w3 are the weights of each factor, which can be set to 0.3, 0.3, 0.4 here.

[0086] Specifically, The emotional willingness of the person being helped is measured by the person's facial expression. The facial detection and facial emotion recognition based on deep learning are used to classify the current facial expression. The expression classification includes 7 categories, namely anger, disgust, fear, happiness, sadness, surprise, and neutral. Different scores are given based on different expressions:

[0087] ;

[0088] It is the sentiment score of the semantics of the discourse, which measures the emotional attitude of the help object in the language expression. The semantic sentiment of the discourse can be analyzed through the natural language processing model:

[0089] ;

[0090] It is the emotional score of body movements, which measures the emotional attitude in the body language of the help object. Body language can be identified by body gesture detection using computer vision:

[0091] ;

[0092] When V emotion ≤T emotion When the robot takes the initiative to thank and stop communicating, and takes the initiative to move away from the person being helped. Here, the emotional willingness threshold T is preset. emotion Can be set to 0.4.

[0093] After the robot has completed the communication with the helper, it needs to process and execute the information, convert the guidance steps provided by the helper into executable task instructions, and execute them. The specific information processing and execution logic is as follows: Figure 3 As shown in the figure, understand the helper's guidance steps and extract key information. Extract core actions, goals, and conditions from the guidance, and identify the main tasks that need to be performed; Task decomposition and standardized instructions. Decompose complex steps into smaller, more specific executable tasks to ensure that each task is independent and clear, and determine the format of task instructions to make the instructions standardized and unified; at the same time, determine whether the standardized instructions are within the robot's capabilities. If not, adjust the tasks to adapt to the robot's functional capabilities; according to the relationship between tasks, add necessary logical judgments and execution sequences to ensure the coherence of the robot's task flow. Then the robot performs several executable tasks based on the complete task list. In this way, the cost of seeking help is minimized by evaluating and selecting the most suitable helper. Enhanced human-computer interaction experience, when the robot actively communicates with humans, it takes human emotional feedback as the interaction policy, and actively terminates communication after humans show negative emotions, which improves the interactive experience and increases humans' willingness to assist robots.

[0094] Step S14: after executing a number of the executable tasks, determine whether the target task is completed; if not, jump back to the step of constructing a spatial map within the target range until the target task is completed.

[0095] In this embodiment, after executing several executable tasks, it is determined whether the target task is completed. If not, it will jump back to the step of constructing a spatial map within the target range. At the same time, after the jump, if the number of operations of re-jumping to the step of constructing a spatial map within the target range is equal to the preset number threshold, and the target task is not completed, the operation and maintenance personnel will be asked for help to complete the target task. Specifically, after the robot completes the subtask list obtained by "autonomous help seeking", if the overall task cannot be completed, the robot will continue to optimize and enter the "autonomous help seeking" mode again. Considering the existing comprehensive limitations of the robot, the number of continuous optimization rounds can be set. In a specific embodiment, it is set to 5 rounds. When 5 rounds of help still do not get complete information to complete the task, the robot stays in place for a period of time and continues to seek help or directly seek help from the operation and maintenance personnel. In this way, by designing a strategy for the robot to continue to ask for help, even when the problem is not solved by asking for help, the robot can still dynamically adjust the strategy, reflecting strong robustness and task adaptability.

[0096] In summary, in the process of executing the target task, the robot in this application first determines the confidence of the target task completion success rate, and judges whether the robot needs to seek external assistance according to the size relationship between the confidence and the preset confidence threshold; if so, then construct a spatial map within the target range, identify and locate the object group that can provide help, determine several groups of candidate objects that can be asked for help based on the spatial map and the object group, and determine the assistance value corresponding to each candidate object that can be asked for help; then determine the target candidate object that can be asked for help corresponding to the highest assistance value, interact with the target candidate object that can be asked for help to obtain corresponding guidance information, and determine several executable tasks based on the guidance information; finally, after executing several executable tasks, determine whether the target task is completed. If not, jump back to the step of constructing the spatial map within the target range until the target task is completed. It can be seen that this application determines whether the robot can complete the task independently through confidence. If it cannot be completed, seek external help, so that the robot can autonomously seek external assistance, efficiently integrate the obtained help information to complete the task, and improve the reliability of task completion. At the same time, in the process of seeking external help, the target candidate for help is determined through the help degree value, the selection efficiency of the help object is optimized, and the cost of seeking help is minimized by evaluating and selecting the most suitable help object, thereby improving the efficiency and success rate of task completion.

[0097] Based on the previous embodiment, it can be seen that the present application determines whether the robot needs to seek external assistance based on the relationship between the confidence level and the preset confidence threshold. Next, the process of determining the confidence level of the target task completion success rate will be described in detail. Figure 4As shown, the embodiment of the present invention discloses a specific robot task completion method, including:

[0098] Step S21: Determine the reliability of each sensor data based on the signal strength, signal quality and signal integrity of each sensor of the robot.

[0099] In this embodiment, the reliability of each sensor of the robot includes the signal strength, signal quality, and signal integrity of the sensor. The signal strength indicates whether the data frame rate of the sensor meets the specification requirements, whether the signal transmission is intermittent due to wiring problems, and whether the frame rate does not meet the specification requirements. Signal quality indicates the signal-to-noise ratio of the signal, whether the signal-to-noise ratio meets the standard requirements, and whether it has significantly decreased. The integrity of the signal indicates whether there is a local loss under the resolution of the signal, and the proportion of the effective range to the total range. The reliability of the sensor under different signal strengths, signal qualities, and signal integrity is different. Different reliability values ​​can be set in advance according to various situations corresponding to the signal strength, signal quality, and signal integrity, and then the sensor reliability can be directly determined according to the values ​​corresponding to different situations.

[0100] Step S22: determining the sensor data confidence level representing the integrity and accuracy of the sensor data according to the preset sensor weights corresponding to the sensors of the robot and the reliability of the sensor data.

[0101] In this embodiment, the sensor data confidence level representing the integrity and accuracy of the sensor data is determined according to the preset sensor weights corresponding to the sensors of the robot and the reliability of the sensor data. Specifically, Reflects the integrity and accuracy of sensor data. It is obtained by the weighted sum of the sensor data quality required to complete the current task. The value range is [0, 1]:

[0102] ;

[0103] in, is the weight of the i-th sensor; is the reliability of the i-th sensor; N is the number of sensors; The confidence level of sensor data is used to characterize the integrity and accuracy of sensor data.

[0104] Step S23: determining a battery confidence level for characterizing the health of the battery through the capacity, power loss rate, and voltage of the battery of the robot.

[0105] In this embodiment, It reflects the health of the battery and is used to reflect the impact of the current battery status on system reliability and task completion capability. , power failure rate ,Voltage Three factors determine:

[0106] ;

[0107] in, It reflects whether the battery can provide enough energy to complete the current task. If the battery capacity meets the expected capacity and has a certain surplus, it means that the battery capacity is sufficient. Otherwise, there is a risk that the battery capacity will not be able to complete the task:

[0108] ;

[0109] in, is the current battery capacity; is the capacity; is the estimated battery capacity.

[0110] The power loss rate describes the speed at which the battery power decreases during use. The standard rate of decrease can be obtained statistically. According to the standard power loss rate, compare and analyze whether the current power loss rate conforms to the standard law. The difference between the two curves reflects the power loss rate. . V norm It refers to the standardized value of the battery's operating voltage. The actual voltage is normalized to the range of 0 and 1 according to the upper and lower limits of the battery's design voltage. c 、w_b d 、w_b v Can be set to 0.4, 0.25, 0.35.

[0111] Step S24, determining the complexity of the ground according to the number of obstacles on the road surface, the degree of terrain undulation, and the width of the driving path in the current environment of the robot, and determining the environmental confidence used to characterize the complexity of the current environment using the environmental conditions corresponding to the current environment of the robot and the complexity of the ground; the environmental conditions include light intensity, temperature, humidity, and wind speed.

[0112] In this embodiment, Reflects the complexity of the robot's current environment, which is determined by the lighting, temperature, humidity, wind, and ground complexity:

[0113] ;

[0114] in, This reflects the impact of current lighting conditions on the robot. Appropriate lighting conditions help sensors and cameras work properly:

[0115] ;

[0116] Where L is the light intensity, [llow , l high ] is the most suitable light condition. lower or higher than l higher Light conditions are considered totally unsuitable.

[0117] f temp It reflects the influence of temperature on the internal components and batteries of the robot. The appropriate temperature range can ensure the normal operation of the robot. The formula can be referred to The formula for hum It reflects the effect of humidity on the working reliability of the robot's electronic components. Too high or too low humidity may cause damage to the internal electronic components. The formula can be referred to The formula for wind It reflects the effect of wind speed on the robot's driving stability. The formula can be found in Considering the influence of various factors on the robot, the weights can be set as w_e1=0.2, w_e2=0.2, w_e3=0.15, w_e4=0.15, w_e5=0.3.

[0118] Then the complexity of the road surface is decomposed into multiple influencing factors: the number of obstacles, terrain undulation, and channel width, and the confidence level of the complexity of the robot's working road surface is calculated:

[0119] ;

[0120] in, is the confidence level of ground complexity; Reflects the confidence level of the number of obstacles on the current road:

[0121] ;

[0122] Where n is the number of obstacles detected; n max The maximum number of obstacles allowed is preset.

[0123] f terrain It reflects the undulation of the working road surface. When the terrain is flat, f terrain =1; when the terrain is very undulating, such as with many slopes or irregular ground, f terrain = 0; for terrain with slight bullying, depending on the slope and irregularity, f terrain It decreases linearly between 0 and 1. width It reflects the width of the path. The wider the path, the higher the score, and the narrower the path, the lower the score:

[0124]

[0125] Where w is the path width; w min is the minimum path width; w max is the maximum path width.

[0126] Step S25: determining a target confidence level for characterizing the degree of support of the environmental data for the target task based on the vector corresponding to the environmental data detected by the robot and the environmental data required to complete the target task.

[0127] In this embodiment, Refers to the degree to which the currently perceived data supports the current task decision. When environmental markers, such as road signs during walking, are lost, or when the tools required for the next task, such as No. 5 screws, cannot be perceived, When it is judged that the current perception data cannot meet the necessary data required for task decision-making, It will be low enough to start the "self-help seeking" mode.

[0128] ;

[0129] Among them, D sensor Denotes the perceived actual dependency vector, D task Represents the dependency vector required for the task. The higher the correlation, the more the perception data matches the task requirements. The larger the dependency vector D required by the task task It is a dependency vector composed of the number of environmental targets required in the current task planning. For example, the current task needs to meet the four conditions of "1. Walk 50 meters, 2. Find the intersection, 3. Detect the traffic light, 4. The traffic light turns red" and then turn right and continue walking. According to the degree of dependence of the conditions, a dependency data vector [1, 3, 1, 1] can be established. The perceived actual dependency vector D sensor It refers to the dependency vector that the robot has detected. For example, if the traffic light is not detected, then the current actual dependency vector D sensor is [1, 3, 0, 0].

[0130] .

[0131] Step S26: Determine the confidence of the target task completion success rate based on the sensor data confidence, the battery confidence, the environment confidence and the target confidence.

[0132] In this embodiment, the confidence calculation formula is:

[0133] ;

[0134] in, is the confidence level; is the confidence of sensor data; is the battery confidence level; Provide confidence for the environment; is the target confidence; is the target threshold (the red line threshold of a single factor, when the confidence of any factor ≤ When the robot starts the "autonomous assistance seeking" mode based on the factor to solve the current factor problem, It can be a unified value, or a red line threshold can be set for each factor); w1, w2, w3, w4 are preset weight values, that is, the weights of each factor, reflecting the degree of influence of these factors on the confidence, and the sum of the weights is 1. In a specific embodiment, for an indoor working environment, w1, w2, w3, w4 can be set to 0.3, 0.2, 0.2, 0.3, respectively. For an outdoor working environment, w1, w2, w3, w4 are set to 0.3, 0.15, 0.15, 0.4, respectively. For weights, the weight parameters can also be dynamically adjusted through an environmental perception model.

[0135] In this way, the present application calculates confidence based on the weights of multiple factors (sensor integrity, environmental complexity, battery health, etc.), dynamically adjusts the decision logic, and can adapt to a variety of scenarios.

[0136] See also Figure 5 As shown, an embodiment of the present invention discloses a robot task completion device, comprising:

[0137] The judgment module 11 is used to determine the confidence level of the target task completion success rate during the execution of the target task, and judge whether the robot needs to seek external assistance according to the relationship between the confidence level and a preset confidence threshold;

[0138] The assistance value determination module 12 is used to construct a spatial map within the target range, identify and locate the object group that can provide assistance, determine a number of groups of candidate objects that can be helped based on the spatial map and the object group, and determine the assistance value corresponding to each candidate object that can be helped;

[0139] An executable task determination module 13 is used to determine a target candidate object for help corresponding to the highest help value, interact with the target candidate object for help to obtain corresponding guidance information, and determine a number of executable tasks based on the guidance information;

[0140] The jump module 14 is used to determine whether the target task is completed after executing several executable tasks. If not, it will jump back to the step of constructing a spatial map within the target range until the target task is completed.

[0141] In summary, in the process of executing the target task, the robot in this application first determines the confidence of the target task completion success rate, and judges whether the robot needs to seek external assistance according to the size relationship between the confidence and the preset confidence threshold; if so, then construct a spatial map within the target range, identify and locate the object group that can provide help, determine several groups of candidate objects that can be asked for help based on the spatial map and the object group, and determine the assistance value corresponding to each candidate object that can be asked for help; then determine the target candidate object that can be asked for help corresponding to the highest assistance value, interact with the target candidate object that can be asked for help to obtain corresponding guidance information, and determine several executable tasks based on the guidance information; finally, after executing several executable tasks, determine whether the target task is completed. If not, jump back to the step of constructing the spatial map within the target range until the target task is completed. It can be seen that this application determines whether the robot can complete the task independently through confidence. If it cannot be completed, seek external help, so that the robot can autonomously seek external assistance, efficiently integrate the obtained help information to complete the task, and improve the reliability of task completion. At the same time, in the process of seeking external help, the target candidate for help is determined through the help degree value, the selection efficiency of the help object is optimized, and the cost of seeking help is minimized by evaluating and selecting the most suitable help object, thereby improving the efficiency and success rate of task completion.

[0142] In some specific embodiments, the judgment module 11 can be specifically used to determine the reliability of each sensor data based on the signal strength, signal quality and signal integrity of each sensor of the robot; determine the sensor data confidence characterizing the integrity and accuracy of the sensor data according to the preset sensor weights corresponding to each sensor of the robot and the reliability of the sensor data; determine the battery confidence characterizing the health of the battery through the capacity, power loss rate and voltage of the battery of the robot; determine the ground complexity according to the number of obstacles on the road surface, the terrain undulation and the width of the driving path in the current environment of the robot; determine the environmental confidence characterizing the complexity of the current environment using the environmental conditions corresponding to the current environment of the robot and the ground complexity; the environmental conditions include light intensity, temperature, humidity and wind speed; determine the target confidence characterizing the degree of support of the environmental data for the target task based on the vector corresponding to the environmental data detected by the robot and the environmental data required to complete the target task; determine the confidence of the success rate of completing the target task based on the sensor data confidence, the battery confidence, the environmental confidence and the target confidence.

[0143] In some specific embodiments, the judgment module 11 can be specifically used to determine the confidence of the target task completion success rate based on the sensor data confidence, the battery confidence, the environment confidence and the target confidence through a confidence calculation formula; the confidence calculation formula is:

[0144] ;

[0145] in, is the confidence level; is the sensor data confidence; is the battery confidence level; A confidence level for the environment; is the target confidence; is the target threshold; w1, w2, w3, w4 are the preset weight values.

[0146] In some specific embodiments, the judgment module 11 can be specifically used to determine that the robot needs to seek external assistance if the confidence is less than or equal to the preset confidence threshold; if the confidence is greater than the preset confidence threshold, then determine that the robot does not need to seek external assistance.

[0147] In some specific embodiments, the assistance value determination module 12 can be specifically used to construct group structure data based on the spatial map and the information corresponding to the object groups; the group structure data is data defining the labels of each of the object groups, the surrounding map of each of the object groups, the minimum cost path from the robot to each of the object groups, the age of each of the object groups, and the social role of each of the object groups; and several groups of candidate objects for assistance are determined based on the group structure data.

[0148] In some specific embodiments, the assistance value determination module 12 can be specifically used to determine the social attributes of the candidate object for help based on the clothing and facial expressions of the candidate object for help; the social attributes include any one or a combination of children, teenagers, middle-aged people, the elderly, primary and secondary school students, high school students and social service providers; the appearance affinity value of the candidate object for help is determined by the preset scores corresponding to each social attribute; the behavior value of the candidate object for help is determined according to the walking speed and body movements of the candidate object for help; the body movements include any one or a combination of empty hands, carrying items and making phone calls; the carrying items action includes several types of carrying items actions obtained after classification based on the estimated weight of the items; the action cost value of the robot reaching the position of the candidate object for help is determined based on the distance between the robot and the candidate object for help; the assistance value of each candidate object for help is determined by the appearance affinity value, the behavior value and the action cost value corresponding to each candidate object for help.

[0149] In some specific embodiments, the device can also be used to perform face detection and facial emotion recognition on the target candidate object for help, determine the facial expression of the target candidate object for help, determine the score corresponding to the facial expression based on the score corresponding to the predetermined facial expression classification; determine the emotional score of the speech semantics of the target candidate object for help based on the predetermined semantic emotional score; determine the emotional score of the body movement of the target candidate object for help by using the predetermined body movement score; determine the emotional willingness value used to characterize whether the target candidate object for help is willing to provide help based on the score corresponding to the facial expression, the emotional score of the speech semantics, and the emotional score of the body movement; if the emotional willingness value is less than or equal to a preset emotional willingness threshold, stop interacting with the target candidate object for help, move away from the target candidate object for help, and re-determine a new candidate object for help to interact with to obtain corresponding guidance information; if the emotional willingness value is greater than the preset emotional willingness threshold, interact with the target candidate object for help to obtain corresponding guidance information.

[0150] In some specific embodiments, the device can also be used to seek help from operation and maintenance personnel to complete the target task if the number of times the operation of re-jumping to the step of building the spatial map within the target range is equal to a preset number threshold and the target task is not completed.

[0151] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0152] Figure 6 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the robot task completion method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0153] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0154] In addition, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0155] The operating system 221 is used to manage and control the hardware devices on the electronic device 20 and the computer program 222, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the robot task completion method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0156] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the robot task completion method disclosed above. The specific steps of the method can refer to the corresponding contents disclosed in the above embodiments, and will not be repeated here.

[0157] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0158] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0159] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0160] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0161] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A robot task completion method, characterized in that: include: In the process of executing the target task, determining the confidence level of the target task completion success rate, and judging whether the robot needs to seek external assistance based on the relationship between the confidence level and a preset confidence threshold; If yes, construct a spatial map within the target range, identify and locate the group of objects that can provide help, determine a number of groups of candidate objects that can be helped based on the spatial map and the group of objects, and determine the help value corresponding to each candidate object that can be helped; Determine a target candidate object for help corresponding to the highest help value, interact with the target candidate object for help to obtain corresponding guidance information, and determine a number of executable tasks based on the guidance information; After executing a number of the executable tasks, it is determined whether the target task is completed. If not, the process jumps back to the step of constructing a spatial map within the target range until the target task is completed.

2. The robot task completion method according to claim 1, characterized in that: The confidence level of determining the success rate of completing the target task includes: Determining the reliability of each sensor data based on the signal strength, signal quality and signal integrity of each sensor of the robot; Determining sensor data confidence that characterizes integrity and accuracy of sensor data according to preset sensor weights corresponding to each sensor of the robot and the reliability of the sensor data; Determine a battery confidence level for characterizing the health of the battery through the capacity, power loss rate, and voltage of the battery of the robot; Determining the ground complexity according to the number of obstacles on the road surface, the degree of terrain undulation, and the width of the driving path in the current environment of the robot; Determine an environmental confidence level for characterizing the complexity of the current environment using environmental conditions corresponding to the current environment of the robot and the complexity of the ground; the environmental conditions include light intensity, temperature, humidity and wind speed; Determining a target confidence level for characterizing the degree of support of the environmental data for the target task based on a vector corresponding to the environmental data detected by the robot and the environmental data required to complete the target task; The confidence of the target task completion success rate is determined based on the sensor data confidence, the battery confidence, the environment confidence, and the target confidence.

3. The robot task completion method according to claim 2, characterized in that: The step of determining the confidence of the target task completion success rate based on the sensor data confidence, the battery confidence, the environment confidence, and the target confidence includes: The confidence of the target task completion success rate is determined based on the sensor data confidence, the battery confidence, the environment confidence and the target confidence through a confidence calculation formula; the confidence calculation formula is: ； in, is the confidence level; is the sensor data confidence; is the battery confidence level; A confidence level for the environment; is the target confidence; is the target threshold; w1, w2, w3, w4 are the preset weight values.

4. The robot task completion method according to claim 1, characterized in that: The determining whether the robot needs to seek external assistance according to the magnitude relationship between the confidence and a preset confidence threshold value includes: If the confidence level is less than or equal to the preset confidence threshold, it is determined that the robot needs to seek external assistance; If the confidence level is greater than the preset confidence threshold, it is determined that the robot does not need to seek external assistance.

5. The robot task completion method according to claim 1, characterized in that: The determining of a plurality of groups of candidate objects for assistance based on the spatial map and the object group comprises: Constructing group structure data based on the spatial map and the information corresponding to the object groups; the group structure data is data defining labels of each object group, a surrounding map of each object group, a minimum cost path from the robot to each object group, an age of each object group, and a social role of each object group; Several groups of candidate objects for assistance are determined according to the group structure data.

6. The robot task completion method according to claim 1, characterized in that: Determining the assistance value corresponding to each candidate object for help includes: Determine the social attributes of the candidate object for help based on the clothes and facial expressions of the candidate object for help; the social attributes include any one or a combination of children, teenagers, middle-aged people, the elderly, primary and secondary school students, high school students and social service providers; Determining the appearance affinity value of the candidate object for help by using the preset scores corresponding to each social attribute; Determine the behavior value of the candidate object for help according to the walking speed and body movements of the candidate object for help; the body movements include any one or a combination of empty-handed, carrying items and making phone calls; the carrying items movement includes several types of carrying items movement obtained by dividing based on the estimated weight of the items; Determining an action cost value for the robot to reach the location of the candidate object for help based on the distance between the robot and the candidate object for help; The help-giving value of each candidate object for help is determined according to the appearance affinity value, the behavior value and the action cost value corresponding to each candidate object for help.

7. The robot task completion method according to claim 1, characterized in that: The process of interacting with the target candidate object for assistance to obtain corresponding guidance information also includes: Performing face detection and facial emotion recognition on the target candidate for help, determining the facial expression of the target candidate for help, and determining the score corresponding to the facial expression based on the score corresponding to the predetermined facial expression classification; Determining a sentiment score of the semantics of the target help-seeking candidate's speech based on a predetermined semantic sentiment score; Determining the emotion score of the body movement of the target candidate object for help by using a predetermined body movement score; Determine, based on the score corresponding to the facial expression, the sentiment score of the semantics of the speech, and the sentiment score of the body movement, a sentiment willingness value for representing whether the target candidate for help is willing to provide help; If the emotional willingness value is less than or equal to the preset emotional willingness threshold, then stop interacting with the target candidate for help, stay away from the target candidate for help, and re-determine a new candidate for help to interact with, so as to obtain corresponding guidance information; If the emotional willingness value is greater than the preset emotional willingness threshold, interaction is performed with the target candidate for help to obtain corresponding guidance information.

8. The robot task completion method according to any one of claims 1 to 7, characterized in that: After the step of jumping back to the step of constructing the spatial map within the target range, the method further includes: If the number of times the operation of re-jumping to the step of building the spatial map within the target range is executed is equal to a preset number threshold, and the target task is not completed, help is sought from the operation and maintenance personnel to complete the target task.

9. A robot task completion device, characterized in that: include: A judgment module, used to determine the confidence level of the target task completion success rate during the execution of the target task, and to judge whether the robot needs to seek external assistance based on the relationship between the confidence level and a preset confidence threshold; a help value determination module, for, if yes, constructing a spatial map within the target range, identifying and locating a group of objects that can provide help, determining a number of groups of candidate objects that can be helped based on the spatial map and the group of objects, and determining a help value corresponding to each candidate object that can be helped; An executable task determination module is used to determine a target candidate for help corresponding to the highest help value, interact with the target candidate for help to obtain corresponding guidance information, and determine a number of executable tasks based on the guidance information; The jump module is used to determine whether the target task is completed after executing several executable tasks. If not, it will jump back to the step of constructing a spatial map within the target range until the target task is completed.

10. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the robot task completion method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the robot task completion method as described in any one of claims 1 to 8 is implemented.

Images