A robot explainable artificial intelligence computing method and system thereof

Abstract

The application provides a robot explainable intelligent computing method and system, which comprises the following steps: S1, collecting action behavior data and tactile representation data of a target object operation task of a robot in a working state; S2, extracting feature information of semantic description of the action behavior data and the tactile representation data, projecting feature information with strong correlation into a shared semantic subspace, and establishing a feature information vector matrix corresponding to the data and the task; S3, combining a symbolic planner with a specific tactile model based on a GEP algorithm, so as to achieve the goal of combining a long-term task structure of the robot with an operation strategy learned from tactile signals; and S4, based on a Bayesian trust model, cognitively fusing and calculating the robot action behavior data and the tactile representation data by using a probability method, so as to predict the next action of the robot. The problems of lack of explainability and transparency of a machine learning model are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning, in particular to a robot explainable intelligent computing method and system thereof. BACKGROUND

[0002] With the in-depth development of the field of machine learning, robots have a major application in education, guidance, medical and various aspects. Nowadays, people not only pay attention to the accuracy and efficiency of the robot in executing tasks, but also hope to understand the reasons for its decision and action, so as to judge whether it is reliable.

[0003] However, as known from CN108881446A "Artificial intelligence platform system based on deep learning", the current deep learning model often does not give exact decision basis and judgment on whether the decision is reliable. Due to the lack of explainable results, users feel confused, which seriously limits its wide application in real tasks. SUMMARY

[0004] Therefore, the present application provides a robot explainable intelligent computing method and system, which solves the problem of trust estimation and lack of explainability of artificial intelligence in machine learning.

[0005] The technical scheme of the present application is implemented as follows: the present application provides a robot explainable intelligent computing method, comprising the following steps,

[0006] S1, collecting action behavior data and tactile representation data of a target object operating a task in a working state of a robot;

[0007] S2, extracting feature information of semantic description of the above two kinds of data, and projecting feature information with strong association into a shared semantic subspace to establish a feature information vector matrix corresponding to the data and the task;

[0008] S3, combining a symbolic planner with a specific tactile model based on a GEP algorithm to achieve the goal of combining long-term task structure of the robot with operation strategy learned from tactile signals;

[0009] S4, based on a Bayesian trust model, cognitively fusing and calculating the action behavior data and the tactile representation data of the robot by using a probability method to predict the next action of the robot.

[0010] On the basis of the above technical scheme, preferably, step S1 specifically comprises the following steps:

[0011] The action behavior data and the tactile representation data include force description data and gesture expression data of the robot.

[0012] On the basis of the above technical scheme, preferably, step S2 specifically comprises the following steps:

[0013] The semantic feature information extracted from the robot force description data includes fist, grasp, touch and pinch.

[0014] On the basis of the above technical scheme, preferably, step S3 specifically comprises the following steps:

[0015] The symbolic planner is used to encode the semantic knowledge of the task execution sequence, and uses a context-independent grammar to represent the task.

[0016] On the basis of the above technical scheme, preferably, step S4 specifically comprises the following steps:

[0017] The Bayesian belief model simulates the relationship between the robot and the interactor, including the relationship between the robot's trust, action and the interactor's action, and the estimation of the true situation;

[0018] By analogy with the particle filtering technology, when encountering unknown interactors, events are dynamically generated to train new Bayesian belief models.

[0019] On the other hand, the present application also provides a robot explainable intelligent computing system, the system comprises:

[0020] A data acquisition module is used to acquire the action behavior data and the tactile representation data of the target object operation task of the robot in the working state;

[0021] A feature extraction module is used to extract the feature information of the semantic description of the above two kinds of data, and project the feature information with strong association into a shared semantic subspace, and establish a feature information vector matrix corresponding to the data and the task;

[0022] An operation strategy module is used to combine the symbolic planner and the specific tactile model based on the GEP algorithm, so as to achieve the goal of combining the long-term task structure of the robot with the operation strategy learned from the tactile signal;

[0023] A belief model module is used to predict the next action of the robot by cognitively fusing and calculating the action behavior data and the tactile representation data of the robot based on the Bayesian belief model and using a probabilistic method.

[0024] Preferably, the data acquisition module is specifically used for:

[0025] The action behavior data and the tactile representation data include the force description data and the gesture expression data of the robot.

[0026] Preferably, the feature extraction module is specifically used for:

[0027] The semantic feature information extracted from the robot force description data includes fist, grasp, touch and pinch.

[0028] Preferably, the operation strategy module is specifically used for:

[0029] The symbolic planner is used to encode semantic knowledge of task execution sequences, using context-independent syntax to represent tasks.

[0030] Preferably, the trust model module is specifically used for:

[0031] The Bayesian trust model simulates the connection between the robot and the interactor, including the connection between the trust of the robot, the action and the action of the interactor, and the estimation of the true situation;

[0032] By using the particle filtering technology, when encountering unknown interactors, events are dynamically generated to train new Bayesian trust models.

[0033] The robot explainable intelligent computing method and system of the present application have the following beneficial effects relative to the prior art:

[0034] (1) By fusing the action behavior data and tactile representation data of the target object operation task of the robot in a specific scene, a self-learning fusion computing system is established by using the Bayesian trust model, thereby providing functional and mechanical explanations, so that the model accuracy and complexity are guaranteed while the machine intelligence has explainability;

[0035] (2) The problem of lack of explainability and transparency of machine learning models is solved, and a certain trust relationship between the user and the decision model is established, and the corresponding explanation is given at the same time of decision-making, so as to obtain the trust and understanding of the user. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0037] Figure 1 The flowchart of the robot explainable intelligent computing method of the present application;

[0038] Figure 2 The Bayesian network overview diagram of the present application;

[0039] Figure 3 The module diagram of the robot explainable intelligent computing system of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0041] Embodiment one

[0042] A robot explainable artificial intelligence computing method is provided, as shown in the figure, comprising the following steps, Figure 1

[0043] S1, collecting action behavior data and tactile representation data of a target object operating task in a working state of a robot;

[0044] S2, extracting feature information described by semantics of the two kinds of data, and projecting feature information having strong association therebetween to a shared semantic subspace, to establish a feature information vector matrix corresponding to the data and the task;

[0045] S3, combining a symbolic planner with a specific tactile model based on a GEP (Gene Expression Programming) algorithm, so as to achieve the goal of combining a long-term task structure of a robot with an operating strategy learned from tactile signals;

[0046] S4, based on a Bayesian belief model, cognitively fusing and calculating the action behavior data and the tactile representation data of the robot by using a probability method, to predict an action to be taken by the robot next time.

[0047] The ability of a human to provide explanation for a decision of a machine is a sign of intelligence, but it is not clear what form of explanation can best promote the trust of a human in a machine.

[0048] The step S1 specifically comprises the following steps:

[0049] Collecting action behavior data of a target object operating task in a working state of a robot and tactile representation data generated by the target object operating task in the scene. For example, collecting force information and description data in a "human" state, i.e. force information in a "robot" state recorded from a sensor (having three-dimensional force data) in an end effector of the robot, or any gesture expression data, etc.

[0050] The step S2 specifically comprises the following steps:

[0051] ​Respectively extract the feature information of the above data semantic description, and project the semantic feature information with strong association in the information set to a shared semantic subspace, and establish the semantic feature vector matrix corresponding to the data driving and task driving, such as the semantic feature information extracted from the robot force description data: clenched fist, grab, touch, pinch, etc.

[0052] The abstract force information and tactile information of the robot are converted into specific data vectors, which is convenient for calculation.

[0053] The step S3 specifically comprises the following steps:

[0054] The symbolic planner is combined with the specific tactile feature set based on the GEP algorithm to achieve the goal of combining the long-term task structure of the robot with the operation strategy learned from the tactile signal. Since the robot target object operation task is a challenging multi-step operation, using symbolic representation is beneficial to capture the necessary constraints of the task. Here, a symbolic action planner will be used. The symbolic action planner is mainly used to encode the semantic knowledge of the task execution sequence, which uses a random context-free grammar to represent the task, where the terminal node (word) is the action and the sentence is the action sequence.

[0055] Specifically, given an action grammar, the planner finds the best action to execute next based on the action history, similar to predicting the next word given a partial sentence. Finally, this action sequence is input into a grammar induction algorithm to parse and predict the robot action sequence. In addition, this planner can also solve the single-sample imitation learning problem.

[0056] In order to combine the long-term task structure guided by the symbolic planner with the operation strategy learned from the tactile signal, the GEP algorithm can be used to combine the symbolic planner with the specific tactile model f, which is as follows:

[0057]

[0058] The formula is a posterior probability that considers the grammar prior and the possibility of tactile signal at the same time. Where G is the action grammar, p is the probability function, f t is the tactile input, a t+1 is the action at time t+1 obtained by the symbolic planner, and finally the best action at time t+1

[0059] The GEP framework is used to search for the most likely example of the next action, and the search process starts from the root node of the prefix tree, which is an empty terminal symbol. When the search reaches the leaf node, the search will terminate, all non-leaf nodes represent terminal symbols (i.e. actions), and the last non-leaf node will be the probability of the next operation to be executed.

[0060] Neural networks and genetic algorithms are both based on sample data and iteratively optimize to obtain results; deep learning is to build neural networks that simulate the human brain to analyze and learn, and learn to obtain solutions (models) for problems in a certain field based on a large amount of sample data, such as image recognition and natural language processing; genetic algorithms are usually used to solve optimization problems, that is, to make the design index reach the optimal value under a series of constraints.

[0061] Specifically, step S4 includes the following steps:

[0062] Based on the Bayesian trust model, the robot's tactile representation data and action behavior data are cognitively fused and calculated using probabilistic methods to predict the robot's next possible action.

[0063] Bayesian networks, also known as trust networks or directed acyclic graph models, are probabilistic graphical models that use a directed acyclic graph to obtain a set of random variables {X1, X2, ..., X...}. n The properties of the n conditional probability distributions of a Bayesian network. Generally, the nodes in the directed acyclic graph of a Bayesian network represent random variables {X1, X2, ..., X...}. n These can be observable variables, latent variables, unknown parameters, etc. An arrow connecting two nodes indicates that the two random variables are causally related or unconditionally independent; if there are no arrows connecting the variables in the nodes, the random variables are said to be conditionally independent.

[0064] Bayesian networks simulating the connections between robots and interactors, such as Figure 2 As shown: Node X R and Y R These represent the robot's trust and action, respectively. Node Y R The posterior distribution of Y determines the operation the machine chooses to perform; I and Y R The connection between them represents the influence of the interactors' opinions on the agent's actions. Then, the robot's action Y... R It is its own trust X R and the actions of the interactor Y I The result of their combined effects. Finally, regarding the true situation X... I The estimation enables the machine to effectively distinguish whether an interactor is trusted.

[0065] The above decision making using past memories in the present and future is an important skill to enhance cognitive processes that will enable the robot to react appropriately to a person it has never met before. The design guidelines followed by this algorithm are: memories fade over time; details fade proportionally to the amount of memory; and shocking events, such as surprises and betrayals, should be harder to forget than ordinary experiences.

[0066] To this end, the particle filter technique widely used in mobile robot localization is used, and when an unknown interactor is encountered, this component dynamically generates a number of events to train a new Bayesian trust model. The formula representing the importance of memory v is:

[0067]

[0068] wherein, is the jth event forming the Bayesian trust model s i is the replay dataset consisting of different events. The meaning of the calculation is the difference between the information amount of each event and the total information entropy of its replay dataset , divided by the discrete time difference related to the memory formation time Δt+1. After calculating the value, it can be projected as the number of repetitions F(v) in the replay dataset, as follows:

[0069]

[0070] That is, by observing the probability distribution of the importance value v of the robot and different interactors, values less than or equal to 0.005 in importance v are discarded (i.e. the memory is forgotten).

[0071] Embodiment Two

[0072] A robot explainable intelligence computing system is provided, as shown in Figure 3 the system comprises,

[0073] a data acquisition module for acquiring motion behavior data and tactile representation data of a target object operating task of a robot in a working state;

[0074] a feature extraction module for extracting feature information of semantic description of the above two kinds of data, and projecting feature information with strong association into a shared semantic subspace, to establish a feature information vector matrix corresponding to the data and the task;

[0075] an operation strategy module for combining a symbolic planner with a specific tactile model based on a GEP algorithm, so as to achieve the goal of combining a long-term task structure of a robot with an operation strategy learned from tactile signals;

[0076] ​Trust model module: based on the Bayesian trust model, through the robot action behavior data and tactile representation data, the probability method is used for cognitive fusion and calculation, so as to predict the next action of the robot.

[0077] Human trust in machines is a sign of intelligence in the ability to provide explanations for decisions, but it is not clear what form of explanation best promotes human trust in machines.

[0078] The data acquisition module is specifically used in the following scenarios:

[0079] The action behavior data of the robot in the working state and the target object operation task and the tactile representation data generated in this scene are collected. For example, collect the strength information and description data in the state of "human", that is, the strength information in the state of "robot" recorded from the sensor (with three-dimensional force data) in the robot end effector, or any gesture expression data, etc.

[0080] The feature extraction module is specifically used in the following scenarios:

[0081] The feature information of the above data semantic description is extracted respectively, and the semantic feature information with strong association in the above information set is projected into a shared semantic subspace, and the semantic feature vector matrix corresponding to data-driven and task-driven is established, such as the semantic feature information extracted from the robot strength description data: clenched fist, grab, touch, pinch, etc.

[0082] The abstract force information and tactile information of the robot are converted into specific data vector matrix for calculation.

[0083] The operation strategy module is specifically used in the following scenarios:

[0084] The symbolic planner is combined with the specific tactile feature set based on the GEP algorithm to complete the goal of combining the long-term task structure of the robot with the operation strategy learned from the tactile signal. Since the robot target object operation task is a challenging multi-step operation, using symbolic representation is beneficial to capture the necessary constraints of the task. Here, a symbolic action planner will be used. The symbolic action planner is mainly used to encode the semantic knowledge of the task execution sequence, which uses a random context-free grammar to represent the task, where the terminal node (word) is the action and the sentence is the action sequence.

[0085] Specifically, given an action grammar, the planner finds the best action to execute next based on the action history, similar to predicting the next word given a part of the sentence. Finally, this action sequence is input into the grammar induction algorithm to parse and predict the robot action sequence. In addition, this planner can also solve the single-sample imitation learning problem.

[0086] To combine the long-term task structure guided by the symbolic planner with the operational strategy learned from the tactile signal, the GEP algorithm can be used to combine the symbolic planner with the concrete tactile model f, which is formulated as follows:

[0087]

[0088] The formula is a posterior probability that considers both the grammatical prior and the possibility of the tactile signal. Where G is the action grammar, p is the probability function, f t is the tactile input, a t+1 is the action at time t+1 obtained by the symbolic planner, and the best action at time t+1 is finally found

[0089] The GEP framework is used to search for the most likely example of the next action, and the search process starts from the root node of the prefix tree, which is an empty terminal symbol. When the search reaches the leaf node, the search terminates, all non-leaf nodes represent terminal symbols (i.e. actions), and the last non-leaf node will be the probability of the next operation performed.

[0090] Both neural networks and genetic algorithms are based on sample data and obtain results through iterative optimization; deep learning is a neural network that simulates the human brain to analyze and learn, and is based on a large amount of sample data to learn and obtain a solution (model) to a problem in a certain field, such as image recognition and natural language processing; genetic algorithms are usually used to solve optimization problems, i.e. to make the design index optimal under a series of limited conditions.

[0091] The trust model module is specifically used in the following scenarios:

[0092] The Bayesian trust model is used to predict the next action of the robot by cognitively fusing and calculating the robot tactile representation data and action behavior data using a probabilistic method.

[0093] A Bayesian network, also known as a belief network or a directed acyclic graph model, is a probabilistic graphical model that learns a set of random variables {X1, X2,..., X n} and their n conditional probability distributions from a directed acyclic graph. Generally speaking, the nodes in the directed acyclic graph of the Bayesian network represent random variables {X1, X2,..., X n} which can be observable variables or hidden variables, unknown parameters, etc. The arrow connecting two nodes represents that the two random variables have a causal relationship or are conditionally independent; if the variables in the node are not connected by an arrow, they are called conditionally independent of each other.

[0094] A Bayesian network modeling the connection between the robot and the interactor is shown in Figure 1 : the nodes X Figure 2 R and Y R represent the robot's trust and action, respectively. The posterior distribution of node Y R determines the operation the robot chooses to perform; the connection between Y I and Y R represents the interactor's opinion's influence on the agent's action. Then, the robot's action Y R is the result of the joint action of its own trust X R and the interactor's action Y I Finally, the estimate of the true situation X I enables the robot to effectively discriminate whether to trust the interactor or not.

[0095] The above described use of past memories to make decisions in the present and future is an important skill to enhance cognitive processes that will enable the robot to react reasonably to a person it has never met before. The design guidelines followed by this algorithm are: memories fade over time; details fade proportionally to the amount of memory; shocking events, such as surprises and betrayals, should be harder to forget than ordinary experiences.

[0096] To this end, the particle filter technique widely used in mobile robot localization is borrowed, and when an unknown interactor is encountered, this component dynamically generates a number of events to train a new Bayesian trust model. The formula that represents the importance of a memory v is:

[0097]

[0098] where, s i is the jth event that forms the Bayesian trust model, and D is the replay dataset composed of different events. The meaning of the computation is the difference between the information content of each event and the total information entropy of its replay dataset divided by the discrete time difference Δt+1 related to the time of memory formation. Once the value v is computed, it can be projected as the number of repetitions F(v) in the replay dataset, as:

[0099]

[0100] i.e. the probability distribution of the importance value v obtained by observing the robot with different interactors, discarding (i.e. forgetting the memory) the values of importance v less than or equal to 0.005.

[0101] ​​The above merely provides the preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A robot interpretable intelligent computing method, characterized in that: The method comprises the following steps, S1, collecting motion behavior data and tactile representation data of the target object operation task of the robot in a working state; S2, extracting semantic description feature information of the motion behavior data and the tactile representation data, and projecting feature information having a strong correlation to a shared semantic subspace to establish a feature information vector matrix corresponding to the data and the task; S3, combining a symbolic planner and a specific tactile model based on a GEP algorithm to achieve the goal of combining a long-term task structure of the robot and an operation strategy learned from tactile signals; S4, predicting the next action of the robot by cognitively fusing and calculating the robot motion behavior data and the tactile representation data based on a Bayesian belief model and using a probability method; The step S4 specifically comprises the following steps: The Bayesian belief model simulates the relationship between the robot and the interactors, including the relationship between the trust of the robot, the action of the robot and the action of the interactor, and the estimation of the real situation; By referring to the particle filtering technology, when encountering unknown interactors, events are dynamically generated to train new Bayesian belief models.

2. The robotic explainable artificial intelligence computing method of claim 1, wherein: The step S1 specifically comprises the following steps: The motion behavior data and the tactile representation data include force description data and gesture expression data of the robot.

3. The robotic explainable artificial intelligence computing method of claim 2, wherein: The step S2 specifically comprises the following steps: The semantic feature information extracted from the force description data of the robot includes fist, grab, touch and pinch.

4. The robotic explainable artificial intelligence computing method of claim 3, wherein: The step S3 specifically comprises the following steps: The symbolic planner is used to encode semantic knowledge of a task execution sequence, and uses a context-independent grammar to represent the task.

5. A robotic explainable artificial intelligence computing system for performing a robotic explainable artificial intelligence computing method according to any one of claims 1-4, characterized in that: The system comprises: a data collection module for collecting motion behavior data and tactile representation data of a target object operation task of the robot in a working state; a feature extraction module for extracting semantic description feature information of the motion behavior data and the tactile representation data, and projecting feature information having a strong correlation to a shared semantic subspace to establish a feature information vector matrix corresponding to the data and the task; an operation strategy module for combining a symbolic planner and a specific tactile model based on a GEP algorithm to achieve the goal of combining a long-term task structure of the robot and an operation strategy learned from tactile signals; a belief model module for predicting the next action of the robot by cognitively fusing and calculating the robot motion behavior data and the tactile representation data based on a Bayesian belief model and using a probability method.

6. The robotic explainable artificial intelligence computing system of claim 5, wherein: The data collection module is specifically configured to: The motion behavior data and the tactile representation data include force description data and gesture expression data of the robot.

7. The robotic explainable artificial intelligence computing system of claim 6, wherein: The feature extraction module is specifically configured to: The semantic feature information extracted from the force description data of the robot includes fist, grab, touch and pinch.

8. The robotic explainable artificial intelligence computing system of claim 7, wherein: The operation strategy module is specifically configured to: The symbolic planner is used to encode semantic knowledge of a task execution sequence, and uses a context-independent grammar to represent the task.

9. The robotic explainable artificial intelligence computing system of claim 8, wherein: The belief model module is specifically configured to: The Bayesian belief model simulates the relationship between the robot and the interactors, including the relationship between the trust of the robot, the action of the robot and the action of the interactor, and the estimation of the real situation; Drawing on particle filtering techniques, a new Bayesian belief model is dynamically generated to train on events when an unknown interlocutor is encountered.

Images