A multi-user communication system based on an AI robot knowledge graph

Abstract

This application relates to the field of robot human-computer interaction and discloses a multi-user communication system based on an AI robot knowledge graph, including a multimodal perception component, a dynamic hypergraph construction module, a semantic potential field arbitration module, an isomorphic pruning verification module, and a semantic motion mapping control module. The method constructs a dynamic hypergraph containing hyperedges by parsing multi-source data, temporarily stores and dynamically completes low-confidence semantics using a time-decaying virtual node mechanism, calculates semantic resistance based on user negative emotions and environmental constraints, and searches for the optimal candidate path using the minimum potential energy principle. It blocks dangerous interactions by extracting local subgraphs and performing isomorphic matching with a taboo graph library. Finally, it establishes a mapping from semantic potential energy to physical space and adjusts joint stiffness and damping parameters in real time based on the semantic potential energy value. This invention effectively solves the problems of intent conflict and context discontinuity in multi-user scenarios, achieving a deep integration of cognitive decision-making and physical compliance control.

Description

Technical Field

[0001] This invention relates to the field of robot human-computer interaction technology, specifically to a multi-user communication system based on AI robot knowledge graph. Background Technology

[0002] Service robots are gradually evolving from single-function automated devices into intelligent terminals with social attributes, and are widely used in scenarios such as home care, public guidance, and business reception. In these unstructured environments, robots need to understand user intentions through multimodal perception methods such as voice and vision, and make corresponding actions or voice responses. The core of human-computer interaction lies in accurately parsing human natural language commands and generating reasonable decision-making logic based on the current environmental state. This requires robots to have robust semantic understanding capabilities and contextual analysis capabilities.

[0003] Existing robot interaction systems typically employ rule-based matching or deep learning-based dialogue management models, combined with traditional knowledge graph techniques to store and retrieve entity relationships. When processing commands from a single user, these systems can utilize sequence models to fill semantic slots and retrieve background knowledge by querying binary relation triples in the knowledge graph, thereby completing a predetermined task flow. For example, a robot can recognize explicit user commands based on a pre-set dialogue tree and invoke the corresponding motion control interface to execute navigation or grasping actions.

[0004] However, existing technical solutions have significant limitations when facing complex scenarios with concurrent interactions among multiple users. Traditional knowledge graph architectures are mainly based on a static binary storage model of "entity-relationship-entity," which makes it difficult to effectively express the diverse coupling relationships involving multiple users, environmental objects, and action commands within a single topological unit. Furthermore, existing systems have high requirements for the completeness and temporal continuity of semantic input, lacking mechanisms for temporarily storing and dynamically completing low-confidence or fragmented commands. When multiple users issue commands simultaneously, or when users exhibit disordered sentence structure, pauses, or missing key semantics during their statements, the system often fails to maintain contextual coherence, making it difficult for the robot to accurately interpret the group's true intentions, resulting in logical gaps in interaction or response failures. Summary of the Invention

[0005] To address the technical problems in existing technologies where robots cannot effectively handle semantic conflicts, contextual fragmentation, and lack of physical safety compliance mechanisms in multi-user concurrent interaction scenarios, this invention proposes a multi-user communication system and method based on AI robot knowledge graphs.

[0006] The first aspect of this invention provides a multi-user communication system based on an AI robot knowledge graph. The system includes an intelligent robot terminal and a control and processing system deployed on the intelligent robot terminal. At the hardware level, the intelligent robot terminal includes a multimodal perception component for collecting environmental data, a motion execution component for performing physical actions, and a computational control center. The computational control center executes computer program instructions stored in memory to construct and run a multi-source semantic parsing module, a dynamic hypergraph construction module, a semantic potential field arbitration module, a homogeneous pruning verification module, and a semantic motion mapping control module.

[0007] The multi-source semantic parsing module is configured to receive environmental data from the multimodal perception component, extract features, and output a structured semantic data stream containing user entity labels, object entity labels, and action primitive labels.

[0008] The dynamic hypergraph construction module is configured to establish a dynamic hypergraph data structure based on the structured semantic data stream. The dynamic hypergraph data structure includes user nodes, object nodes, action primitive nodes, and hyperedges connecting at least two of these nodes. The dynamic hypergraph construction module includes a pending virtual node generation unit, configured to process low-confidence semantic input. When the confidence score of the received semantic input is lower than a preset threshold or is a non-independent modifying semantic, the pending virtual node generation unit creates a pending virtual node data object in the dynamic hypergraph data structure and establishes a temporary hyperedge connecting the pending virtual node data object and the source user node. The pending virtual node data object has a real-time confidence attribute that decreases over time. When the real-time confidence attribute decays to a clearing threshold, the dynamic hypergraph construction module performs a node erasure operation; when matching supplementary semantic data is detected within the monitoring window, the dynamic hypergraph construction module performs a node solidification operation.

[0009] The semantic potential field arbitration module is configured to quantify interaction conflicts and make decisions. First, the semantic potential field arbitration module calculates the semantic resistance value of the hyperedge. The calculation logic is as follows: read the negative sentiment index of the user node associated with the hyperedge, the permission weight value of the user node, and the environmental constraint constant of the environment object node; sum the weighted negative sentiment index and the weighted environmental constraint constant to obtain the semantic resistance value. Subsequently, the semantic potential field arbitration module calculates the total semantic potential value of the connected path. The calculation logic is as follows: accumulate the semantic resistance values ​​of all hyperedges on the connected path, and the information entropy values ​​of all nodes on the connected path. The information entropy value is calculated based on the confidence attribute value of the node, and the information entropy value of the undecided virtual node data object is set to be higher than the information entropy value of the real node. The semantic potential field arbitration module traverses the dynamic hypergraph data structure and searches for the connected path with the smallest total semantic potential value as a candidate execution path. In addition, the semantic potential energy field arbitration module executes deadlock detection logic. If the total semantic potential energy value of the candidate execution path is greater than the preset deadlock determination threshold, a control command is generated pointing to the hovering state node. The hovering state node corresponds to the zero-speed motion command and the maximum damping parameter.

[0010] The isomorphic pruning verification module is configured to perform topology-based security verification. The module calls a taboo graph library stored in memory, which contains several taboo subgraph data templates defining attribute constraints and connection relationships. The module extracts local session subgraphs containing the candidate execution paths, uses a subgraph isomorphic matching algorithm to determine whether the node attribute values ​​of the local session subgraphs satisfy the attribute constraints, and determines whether the topological connection relationships of the local session subgraphs are consistent with the hyperedge set of the taboo subgraph data templates. If they are consistent, a blocking signal is generated. The taboo graph library includes physical security level templates describing physical damage structures and policy compliance level templates describing violations of permissions or spatiotemporal rules.

[0011] The semantic motion mapping control module is configured to convert the verified candidate execution paths into physical control parameters for driving the motion execution component. The semantic motion mapping control module includes a variable stiffness impedance calculation unit, which is configured to receive the minimum total semantic potential energy value and generate joint stiffness parameters and joint damping parameters of the motion execution component based on the minimum total semantic potential energy value. Specifically, the mapping logic is as follows: the joint stiffness parameters are inversely proportional to the minimum total semantic potential energy value, and the joint damping parameters are positively correlated with the minimum total semantic potential energy value. The semantic motion mapping control module sends a hybrid control command to the servo drive controller, including the joint stiffness parameters, the joint damping parameters, and a gravity compensation torque vector, driving the motion execution component to generate low-stiffness and high-viscosity physical motion characteristics when the minimum total semantic potential energy value is high, causing the motion execution component to produce a displacement deviation in the direction of the contact force when subjected to an external contact force vector.

[0012] A second aspect of the present invention provides a multi-user communication method based on an AI robot knowledge graph, wherein the multi-user communication method is executed based on the aforementioned system and includes the following steps: The multimodal perception component is used to collect environmental data and parse it to obtain a structured semantic data stream; Based on the structured semantic data stream, a dynamic hypergraph data structure is constructed. For semantic inputs with insufficient confidence, undecided virtual nodes with time decay characteristics are generated, and the nodes are solidified or erased according to subsequent inputs. Calculate the semantic resistance value of the hyperedge and the information entropy value of the path, search for the candidate execution path with the minimum total semantic potential value based on the preset optimization algorithm, and trigger the hovering state when the total semantic potential value exceeds the deadlock determination threshold. Extract the local session subgraph corresponding to the candidate execution path, perform subgraph isomorphic matching with a preset taboo graph library, and identify and block paths with dangerous topological structures; Based on the minimum total semantic potential value of the candidate execution path, the robot's joint impedance parameters are dynamically modulated to control the robot to exhibit low stiffness and high damping physical compliance characteristics under high semantic conflict conditions.

[0013] This invention provides a multi-user communication system based on an AI robot knowledge graph. It has the following beneficial effects: 1. This invention constructs a dynamic hypergraph data structure containing user, object, and action primitive nodes, and introduces a pending virtual node mechanism with time decay characteristics. It can express multi-user coupling relationships in a single topological unit. By using virtual nodes to temporarily store low-confidence or fragmented voice commands and completing node solidification when supplementing subsequent inputs, it effectively solves the problem of contextual understanding gaps caused by disordered word order or semantic loss in multi-user concurrent interaction, and achieves accurate parsing and dynamic completion of complex intentions of the group.

[0014] 2. This invention quantifies negative emotions and environmental constraints into semantic resistance values ​​by establishing a semantic potential energy field model and isomorphic pruning verification mechanism. Based on the principle of minimum potential energy, it searches for the optimal path and further extracts local conversation subgraphs of candidate paths to compare their topological structure with the taboo graph library to block dangerous interaction patterns. This transforms abstract intention conflicts into a quantifiable mathematical optimization and logical verification process, achieving compliance and safety of robot decision-making in complex social contexts.

[0015] 3. This invention establishes a variable stiffness impedance control mapping relationship from semantic potential energy space to physical joint space. The minimum total semantic potential energy value calculated in the path planning stage is used as an adjustment factor to modulate the dynamic parameters of the motion execution component in real time. When the potential energy value increases due to intensified semantic conflict, the joint stiffness is automatically reduced and the damping coefficient is increased. This makes the robot exhibit low stiffness and high viscosity physical compliance characteristics when there is potential risk. Thus, the risk of hard collision is reduced through physical compliance processing without interrupting the interaction, and the deep integration of cognitive decision-making and motion control is achieved. Attached Figure Description

[0016] Figure 1 This is a system flowchart of the present invention. Detailed Implementation

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example: Please see the appendix Figure 1 This invention provides a multi-user communication system based on an AI robot knowledge graph, including an intelligent robot terminal and a control and processing system deployed on the intelligent robot terminal.

[0019] The intelligent robot terminal includes, at the hardware level, a multimodal perception component, a motion execution component, and a computing control center.

[0020] The multimodal perception component is configured to collect environmental data and interaction data. The multimodal perception component includes a visual acquisition sensor, an audio array sensor, and a force feedback sensor. The visual acquisition sensor is specifically an RGB-D depth camera or LiDAR, used to output visual signals containing environmental depth information and user skeleton information. The audio array sensor is used to acquire multi-channel voice signals and sound source location coordinates. The force feedback sensor is located at the joints or ends of the motion execution component, used to detect contact force values ​​and joint torque values ​​in real time.

[0021] The motion execution component is configured to perform mechanical motion in response to control signals. The motion execution component includes a multi-degree-of-freedom robotic arm joint, an end effector, and a servo drive controller. The multi-degree-of-freedom robotic arm joint is composed of multiple motor modules connected in series or parallel. The servo drive controller is electrically connected to the multi-degree-of-freedom robotic arm joint and is used to adjust the motor current according to input stiffness parameters, damping parameters, and position commands.

[0022] The computing control center includes a central processing unit (CPU), a memory, and a data bus. The memory stores executable computer program instructions and a preset graph database. The CPU is electrically connected to the memory and the data bus, and the CPU invokes the computer program instructions in the memory to execute various steps of the multi-user communication method.

[0023] At the logical function level, when the computer program instructions are executed by the central processing unit, the following processing modules are constructed: a multi-source semantic parsing module, a dynamic hypergraph construction module, a semantic potential field arbitration module, a homogeneous pruning verification module, and a semantic motion mapping control module.

[0024] The multi-source semantic parsing module is communicatively connected to the multimodal perception component and is configured to receive visual and speech signals. The multi-source semantic parsing module includes a feature extraction algorithm for outputting user identification, natural language text vectors, and gesture categories.

[0025] The dynamic hypergraph construction module is connected to the multi-source semantic parsing module and is configured to establish a dynamic hypergraph data structure in the memory. The dynamic hypergraph construction module maps received user identity identifiers and natural language text vectors to nodes and hyperedges in the hypergraph. The dynamic hypergraph construction module includes a pending virtual node generation unit, which is configured to create temporary data objects marked as virtual nodes in the dynamic hypergraph data structure when a semantic fragment with a confidence level lower than a preset threshold is received, and to assign a time decay function to the temporary data objects.

[0026] The semantic potential energy field arbitration module is connected to the dynamic hypergraph construction module and configured to traverse the dynamic hypergraph data structure. The semantic potential energy field arbitration module reads the semantic resistance values ​​on the hyperedges, calculates the total potential energy value from the current state node to the target action node by accumulating the resistance values ​​of all hyperedges on the path and the entropy values ​​of the nodes, and outputs the candidate execution path with the minimum total potential energy value.

[0027] The isomorphic pruning verification module is connected to the semantic potential field arbitration module and is configured to perform topology verification on the candidate execution paths. The isomorphic pruning verification module calls a taboo operation library stored in the memory, which contains several predefined dangerous interaction subgraph data. The isomorphic pruning verification module extracts local subgraphs containing the candidate execution paths, performs subgraph isomorphic matching operations, and outputs a blocking signal if the local subgraph and the dangerous interaction subgraph data structure are identical.

[0028] The semantic motion mapping control module is connected to both the isomorphic pruning verification module and the servo drive controller. The semantic motion mapping control module stores a kinematic mapping matrix from motion nodes to joint angles. The semantic motion mapping control module includes a variable stiffness impedance calculation unit, which is configured to receive the path conflict potential energy value calculated by the semantic potential energy field arbitration module, generate corresponding joint stiffness control commands and damping control commands using an inverse proportional function, and send the joint stiffness control commands and the damping control commands to the servo drive controller.

[0029] This invention provides a multi-user communication method, based on the aforementioned multi-user communication system based on an AI robot knowledge graph, specifically including the following steps: Step S1: Multimodal data acquisition and feature analysis.

[0030] The multi-source semantic parsing module collects visual, speech, and force signals from the environment using a multimodal perception component. Based on a unified timestamp benchmark, the module performs temporal alignment processing on the visual and speech signals. Utilizing feature extraction algorithms, the module parses user identity feature vectors, natural language text vectors, and limb movement skeleton data from the aligned data. The module encapsulates these data into a structured semantic data stream, which includes user entity labels, object entity labels, and action primitive labels.

[0031] Step S2: Dynamic hypergraph model update and virtual node generation.

[0032] The dynamic hypergraph construction module receives the structured semantic data stream output from step S1. Based on the label information in the structured semantic data stream, the module instantiates user nodes, object nodes, and action nodes within the dynamic hypergraph data structure in memory. The module calculates the confidence score of the input semantics in real time. When the confidence score is lower than a preset fuzzy judgment threshold, or when the input semantics are non-independent modifying semantics, the module creates unresolved virtual nodes in the dynamic hypergraph data structure. The module establishes temporary hyperedges connecting the unresolved virtual nodes to associated user nodes and assigns a weight function that decreases over time to the unresolved virtual nodes.

[0033] Step S3: Semantic potential field calculation and path optimization.

[0034] The semantic potential field arbitration module traverses all hyperedges in the dynamic hypergraph data structure. Based on predefined semantic resistance calculation rules, it quantifies the semantic resistance value of each hyperedge. It then searches for all connected paths from the current state node to potential target action nodes. Finally, it sums the semantic resistance values ​​of all hyperedges and the entropy values ​​of the nodes along each connected path to obtain the total semantic potential value for that path. The module compares the total semantic potential values ​​of all connected paths and selects the path with the smallest total semantic potential value as the candidate execution path. If the calculated minimum total semantic potential value exceeds a preset deadlock threshold, the module generates a hovering wait instruction.

[0035] Step S4: Subgraph isomorphism security verification.

[0036] The isomorphic pruning verification module identifies action nodes in candidate execution paths. It extracts a local session subgraph containing the action node and its neighboring nodes from the dynamic hypergraph data structure. The module then calls a pre-defined taboo operation library. Finally, it performs a subgraph isomorphic matching operation to determine if the topology of the local session subgraph is bijectively related to the topology of any taboo subgraph in the taboo operation library. If a bijective relationship exists, the module outputs a blocking signal and terminates the current process; otherwise, it outputs the candidate execution path and its corresponding path conflict potential value.

[0037] Step S5: Semantic-motion mapping and variable stiffness control.

[0038] The semantic motion mapping control module receives the candidate execution paths and path conflict potential energy values ​​output from step S4. Using an inverse kinematics algorithm, the semantic motion mapping control module converts the candidate execution paths into target joint position commands. The variable stiffness impedance calculation unit calculates the control parameters of the servo drive controller based on the path conflict potential energy values, where the path conflict potential energy value is inversely proportional to the joint stiffness parameters and directly proportional to the joint damping parameters. The semantic motion mapping control module sends a hybrid control signal containing the target joint position command, joint stiffness parameters, and joint damping parameters to the servo drive controller, driving the motion execution components to perform physical actions.

[0039] The dynamic hypergraph construction module establishes and maintains a time-varying dynamic hypergraph model in memory. This time-varying dynamic hypergraph model is defined as a binary tuple at the data structure level. ,in This represents the current timestamp of the system's operation. This represents a node dataset that changes dynamically over time. This represents a hyperedge dataset that changes dynamically over time.

[0040] The node dataset The union of four types of entity data subsets is specifically represented by the formula. .in, A subset of user nodes, each user node Internally, it stores a unique identifier and permission level value for the corresponding interactive object; A subset of environment object nodes, each environment object node Internally, it stores the classification labels and three-dimensional spatial coordinate data of the corresponding physical objects; For a subset of action primitive nodes, each action primitive node Internally, it stores atomic control instruction code that can be parsed by motion execution components.

[0041] The node dataset It also includes a subset of undecided virtual nodes. The subset of unresolved virtual nodes This is used to store semantic feature data with a confidence level below a preset threshold. Each undecided virtual node in the subset of undecided virtual nodes is marked as a temporary data object. The temporary data object does not contain a physical entity mapping relationship, and it contains a generation timestamp field and a floating-point value corresponding to the confidence level decay coefficient.

[0042] The hyperedge dataset Composed of several super edges Composition. Each hyperedge It is instantiated in memory as a data object containing a list of node indices. The list of node indices contains the node dataset. The memory address or unique identifier of at least one node in the system.

[0043] Each hyperedge The association has an attribute structure, which at least contains semantic resistance weights. With the generation timestamp The semantic resistance weight This is a floating-point value used to characterize the degree of logical conflict. The dynamic hypergraph construction module updates the semantic resistance weight in real time based on the semantic confidence value and sentiment feature value input from the multi-source semantic parsing module. .

[0044] The dynamic hypergraph construction module performs a topology update operation on the time-varying dynamic hypergraph model according to the system clock. When the multi-source semantic parsing module outputs a new semantic data stream, the dynamic hypergraph construction module updates the node dataset. Write new node data into the hyperedge dataset or write new node data into the hyperedge dataset. New hyperedge data is written to the data. When the effective storage period of a pending virtual node exceeds a preset threshold, the dynamic hypergraph construction module retrieves data from the node dataset. and the aforementioned hyperedge dataset Erase the corresponding node data and associated hyperedge data.

[0045] The dynamic hypergraph construction module handles low-confidence or incomplete semantic input through a pending virtual node mechanism. The specific execution logic is as follows: The dynamic hypergraph construction module receives the semantic data stream output by the multi-source semantic parsing module. The module then reads the confidence score and semantic integrity marker from the semantic data stream.

[0046] When the confidence score is lower than a preset fuzzy judgment threshold, or when the semantic integrity marker indicates that the current semantics are non-independent modifiers, the dynamic hypergraph construction module instantiates a pending virtual node data object in the dynamic hypergraph data structure. The non-independent modifiers include negation instructions, degree adverbs, or interrogative pronouns for missing action objects.

[0047] The dynamic hypergraph construction module generates a temporary hyperedge in the dynamic hypergraph data structure. This temporary hyperedge establishes a topological connection between the pending virtual node data object and the source user node.

[0048] The dynamic hypergraph building module initializes a real-time confidence attribute for undecided virtual node data objects. The real-time confidence attribute The value follows an exponential decay mathematical model, and the specific calculation formula is as follows: in, This represents the initial confidence level value when the pending virtual node data object is generated. This represents the preset time decay constant; This represents the timestamp used to generate the pending virtual node data object. This indicates the current system clock time.

[0049] The dynamic hypergraph construction module continuously monitors for new semantic data input within a preset monitoring time window. When the dynamic hypergraph construction module receives new semantic data, it performs a semantic slot matching operation. The module determines whether the semantic role label of the new semantic data matches the missing semantic slot element of the pending virtual node data object.

[0050] If a semantic role label matches a missing semantic slot, the dynamic hypergraph construction module performs a node solidification operation. This node solidification operation includes merging the feature vector of the new semantic data into the undecided virtual node data object, changing the type label of the undecided virtual node data object to a real node label, and adding the real-time confidence attribute. Locked to the maximum value.

[0051] If the dynamic hypergraph construction module does not detect matching semantic data within the preset monitoring time window, and the real-time confidence attribute... When the value decays to below a preset clearing threshold, the dynamic hypergraph construction module performs a node erasure operation. The node erasure operation includes physically deleting pending virtual node data objects and associated temporary hyperedges from memory.

[0052] The semantic potential field arbitration module uses a graph theory potential field model to quantify the conflict state and path execution cost of multi-user intents. The specific definition and calculation logic are as follows: The semantic potential field arbitration module traverses each hyperedge in the dynamic hypergraph data structure. .

[0053] The semantic potential field arbitration module arbitrates each hyperedge. Calculate a semantic resistance value The semantic resistance value Used for quantizing hyperedges The degree of logical obstruction of the associated interactive events in the current multi-user context.

[0054] The semantic potential field arbitration module calculates the semantic resistance value using the following formula. : In the above formula, Indicates the superedge A user node associated with The negative sentiment index is calculated based on the collected speech intonation and facial expression features. The negative sentiment index range is normalized to a closed interval. . Represents user node The semantic potential field arbitration module reads the permission weight values ​​from the user identity database in the memory. Representing environmental constraint constants, the semantic potential field arbitration module originates from the hyperedge. The environmental constraint constants are read from the attribute fields of the included environmental object nodes. These environmental constraint constants correspond to the preset hazard level of the physical object. This represents the adjustment coefficient for the environmental constraint term.

[0055] The semantic potential field arbitration module further defines the total semantic potential of a connected path. This is applied to a connected path from the current system state node to any target action node. The semantic potential field arbitration module calculates the connectivity path. Total semantic potential Total semantic potential Characterizes the robot's execution of connected paths The total system cost required for the corresponding action sequence.

[0056] The semantic potential field arbitration module calculates the total semantic potential energy using the following formula. : In the above formula, Indicates a connected path Any of the superedges; Indicates a connected path Any node. Represents a node The information entropy value. The semantic potential field arbitration module reads the node. Confidence attribute value And based on the formula or Calculate the information entropy value. When the path is connected... When passing through unresolved virtual nodes, the calculated information entropy value is lower due to the low confidence attribute value of the unresolved virtual nodes. The information entropy value is higher than that of a real node. This represents the weighting coefficient for the resistance term. This represents the weighting coefficient of the entropy term.

[0057] The semantic potential field arbitration module compares the total semantic potential of all candidate connected paths. Identify the total semantic potential energy The smallest optimal path. When connected paths When user nodes on the network exhibit high negative sentiment indices, the connectivity path Cumulative semantic resistance values Increase, leading to total semantic potential Increase, thereby making the connected path It was excluded in the minimum value preferred sorting.

[0058] The semantic potential field arbitration module performs global path search and state decision based on the defined semantic potential field. The specific process is as follows: The semantic potential field arbitration module determines the set of start node and end node in the dynamic hypergraph data structure.

[0059] The semantic potential field arbitration module maps the joint state and task progress of the intelligent robot terminal at the current moment to a state node in a dynamic hypergraph data structure, and marks the state node as the starting node.

[0060] The semantic potential field arbitration module will subset the action primitive nodes. All action primitive nodes are labeled as the set of endpoint nodes.

[0061] The semantic potential field arbitration module runs a graph search algorithm, traversing all potential connected paths from the starting node to any action primitive node in the set of ending nodes. The search cost function of the graph search algorithm is set to the aforementioned total semantic potential. .

[0062] The semantic potential field arbitration module calculates the cumulative potential value of each potential connected path during the traversal process.

[0063] The semantic potential field arbitration module writes all the calculated potential connected paths and their corresponding cumulative potential values ​​into the path candidate list in memory.

[0064] The semantic potential field arbitration module sorts the potential connected paths in the path candidate list from low to high according to their cumulative potential energy values.

[0065] The semantic potential energy field arbitration module selects the path with the smallest cumulative potential energy value as the initial path and marks the cumulative potential energy value of the initial path as the minimum potential energy value. .

[0066] The semantic potential field arbitration module executes deadlock detection logic. The semantic potential field arbitration module reads the preset deadlock determination threshold from memory. .

[0067] The semantic potential field arbitration module will determine the minimum potential energy value. Deadlock detection threshold Perform numerical comparisons.

[0068] If the semantic potential field arbitration module determines the minimum potential energy value Less than or equal to the deadlock threshold The semantic potential field arbitration module confirms the initial path as a candidate execution path and outputs the candidate execution path to the isomorphic pruning verification module.

[0069] If the semantic potential field arbitration module determines the minimum potential energy value Greater than the deadlock threshold The semantic potential field arbitration module determines that the current system is in a multi-user semantic deadlock state. In this state, the semantic potential field arbitration module does not output a path to the action primitive node, but instead generates a control command to the hovering state node. The hovering state node corresponds to the zero-speed motion command and the maximum damping parameter. Simultaneously, the semantic potential field arbitration module sends a user feedback request signal to the audio array sensor or speech synthesis module until the dynamic hypergraph construction module updates the dynamic hypergraph data structure and recalculates the minimum potential energy value. Reduce to the deadlock threshold the following.

[0070] The isomorphic pruning verification module performs security verification based on a taboo image library pre-stored in the memory. The taboo image library is constructed as follows: the isomorphic pruning verification module loads the taboo image library from the memory during the system initialization phase.

[0071] The taboo graph library consists of several independent taboo subgraph data templates. Each taboo subgraph data template is defined in terms of data structure as a static hypergraph model. The taboo subgraph data template is used to describe the topological patterns of interaction relationships that the system pre-determines are not executable.

[0072] The taboo subgraph data template defines node categories with specific attribute characteristics and their connection relationships, rather than targeting a specific single instance.

[0073] The node set of the taboo subgraph data template It contains several node matching rule objects. The node matching rule objects define the logical filtering conditions for the attributes of nodes in the dynamic hypergraph.

[0074] For example, the set of nodes It contains a node matching rule object labeled "vulnerable user class", and the logical filtering conditions of the node matching rule object are set to "user age attribute value is less than a preset security threshold" or "user movement status attribute value is restricted".

[0075] The set of nodes It also includes a node matching rule object marked as "high-risk object class", and the logical filtering conditions of the node matching rule object are set as "object running speed attribute value is greater than zero" or "object surface temperature attribute value is higher than preset burn threshold".

[0076] The set of superedges of the taboo subgraph data template The mandatory connection relationships between the node matching rule objects are defined. Each hyperedge corresponds to a specific action type or spatial location contact relationship.

[0077] For example, in a taboo subgraph data template for high-temperature contact risk, there exists a hyperedge that simultaneously connects the matching rule object of the "vulnerable user class" node, the matching rule object of the "high-risk object class" node, and a "contact action" action primitive node. The hyperedge represents the semantic logical structure of "vulnerable users directly contacting high-risk environmental objects".

[0078] The isomorphic pruning verification module supports multi-dimensional classification and storage of taboo subgraph data templates. The taboo graph library includes taboo subgraph data templates at the physical security level and taboo subgraph data templates at the policy compliance level.

[0079] The taboo subgraph data template at the physical security level is used to describe topologies that may cause hardware damage or personal physical injury; the taboo subgraph data template at the policy compliance level is used to describe topologies that violate preset user permission tables or preset spatiotemporal constraint rules.

[0080] The isomorphic pruning verification module assigns a unique index identifier and a corresponding risk blocking weight value to each taboo subgraph data template. The risk blocking weight value is used to determine the priority level of the signal when the isomorphic pruning verification module generates a blocking signal.

[0081] The isomorphic pruning verification module performs topology-based security verification, and the specific implementation process is as follows: The isomorphic pruning verification module receives candidate execution paths output by the semantic potential field arbitration module. It extracts the target action primitive node from these paths and reads preset search depth parameters from memory. Using the target action primitive node as the topology center, it performs a breadth-first search based on the search depth parameters within the dynamic hypergraph data structure. The module extracts the target action primitive node, its neighboring nodes, and associated hyperedges within the search depth parameter range. It then constructs a local session subgraph data object using the extracted node and hyperedge data. This local session subgraph data object contains the local adjacency matrix information at the current action execution time.

[0082] The isomorphic pruning verification module traverses the taboo graph library in the memory. For each taboo subgraph data template, the isomorphic pruning verification module calls the subgraph isomorphic matching algorithm instruction. The subgraph isomorphic matching algorithm instruction is used to determine whether there is an injective mapping relationship from the node set of the taboo subgraph data template to the node set of the local session subgraph data object.

[0083] When executing the subgraph isomorphic matching algorithm instruction, the isomorphic pruning verification module first performs node attribute compatibility verification. The module compares the node attribute values ​​in the local session subgraph data object with the attribute constraints of the corresponding nodes in the taboo subgraph data template. If the node attribute values ​​in the local session subgraph data object fall within the numerical range defined by the attribute constraints, the isomorphic pruning verification module determines that the node attributes are compatible.

[0084] Under the condition of compatible node attributes, the isomorphic pruning verification module performs a topology consistency check. The isomorphic pruning verification module verifies whether the hyperedge connections between any pair of nodes in the taboo subgraph data template have hyperedge connections of the same type and direction in the corresponding pair of nodes mapped to the local session subgraph data object. If all connections defined in the taboo subgraph data template have corresponding instances in the local session subgraph data object, the isomorphic pruning verification module determines that the topology matching is successful.

[0085] If the isomorphic pruning verification module determines that a local session subgraph data object and any taboo subgraph data template simultaneously satisfy node attribute compatibility and topology matching, the isomorphic pruning verification module generates a highest-priority security blocking signal. The isomorphic pruning verification module overwrites the validity status bit of the candidate execution path to invalid and sends an empty command or interrupt control command to the semantic motion mapping control module. If the isomorphic pruning verification module finds no match after traversing all taboo subgraph data templates, it sends the candidate execution path and its corresponding path conflict potential value to the semantic motion mapping control module.

[0086] The semantic motion mapping control module performs kinematic calculations from action primitive nodes to the physical joint space. The specific implementation process is as follows: The semantic motion mapping control module receives the candidate execution paths output by the isomorphic pruning verification module.

[0087] The semantic motion mapping control module extracts target action primitive nodes from candidate execution paths. Target action primitive node Internally, it stores task space pose parameters, including the target position vector of the end effector 2 in Cartesian coordinates. and target attitude quaternions .

[0088] The semantic motion mapping control module pre-stores the DH parameter table of the intelligent robot terminal. The DH parameter table includes the link length, link offset, joint torsion angle, and initial deviation of the joint angle for the multi-degree-of-freedom robotic arm joints.

[0089] The semantic motion mapping control module uses a numerical iterative inverse kinematics algorithm to process the pose parameters in the task space. The semantic motion mapping control module constructs forward kinematic equations based on the DH parameter table. .

[0090] The semantic motion mapping control module calculates the current joint angle vector. Jacobian matrix under Jacobian matrix It describes the linear mapping relationship between joint space velocity and task space velocity.

[0091] The semantic motion mapping control module performs iterative operations to minimize the error. The semantic motion mapping control module calculates the target position vector. The pose error vector between the current forward kinematics calculation result and the current forward kinematics calculation result .

[0092] The semantic motion mapping control module utilizes the pseudo-inverse matrix of the Jacobian matrix. Update joint angle vector: in, This is the iteration step size coefficient.

[0093] The semantic motion mapping control module repeats the above iterative steps until the pose error vector is obtained. The magnitude is less than the preset convergence threshold, thus obtaining the target joint angle vector. .

[0094] The semantic motion mapping control module establishes a set of action primitive nodes. With the set of joint space states The trajectory interpolation mapping relationship between them.

[0095] The semantic motion mapping control module employs a fifth-order polynomial interpolation algorithm, using the target joint angle vector... As the endpoint boundary condition, generate a continuous joint position control command sequence containing a time series.

[0096] The semantic motion mapping control module transmits a continuous sequence of joint position control commands to the variable stiffness impedance calculation unit.

[0097] In the multi-user communication method provided by this invention, the variable stiffness impedance calculation unit performs joint impedance parameter adjustment based on the path conflict potential energy value output by the semantic potential energy field arbitration module. The specific execution process is as follows: The variable stiffness impedance calculation unit receives the minimum total semantic potential energy value output by the semantic potential energy field arbitration module. .

[0098] The variable stiffness impedance calculation unit simultaneously receives a continuous joint position control command sequence generated by the semantic motion mapping control module. and the target joint angular velocity vector .

[0099] The variable stiffness impedance calculation unit reads the external contact force vector from the force feedback sensor. .

[0100] The variable stiffness impedance calculation unit calculates the joint torque control signal based on a model-based impedance control algorithm. The joint torque control signal The calculation formula is: In the above formula, and These represent the current actual joint angle vector and the current actual joint angular velocity vector of the multi-degree-of-freedom robotic arm joint, respectively. This represents the transpose of the Jacobian matrix of the robotic arm. This represents the gravity compensation torque vector. The variable stiffness impedance calculation unit calculates the gravity compensation torque vector in real time based on the pre-stored link mass parameters and center of mass position parameters.

[0101] Variable stiffness impedance calculation unit constructs stiffness matrix Stiffness matrix The matrix is ​​a diagonal matrix, and the stiffness matrix is... diagonal elements Corresponding to the The dynamic stiffness coefficient of each joint.

[0102] The variable stiffness impedance calculation unit uses an inverse proportional attenuation mapping function to calculate the dynamic stiffness coefficient. : In the above formula, Indicates the first The maximum stiffness constant of each joint is preset in the memory; This represents the stiffness adjustment sensitivity coefficient; This represents the preset system baseline potential energy constant. When the minimum total semantic potential energy value... When increased, the dynamic stiffness coefficient The value decreases.

[0103] Variable stiffness impedance calculation unit constructs damping matrix Damping matrix It is a diagonal matrix, and the damping matrix is... diagonal elements Corresponding to the The dynamic damping coefficient of each joint.

[0104] The variable stiffness impedance calculation unit uses the critical damping superposition mapping function to calculate the dynamic damping coefficient. : In the above formula, Represents the inertia matrix of the robotic arm In the The values ​​of the main diagonal elements at each joint, and the variable stiffness impedance calculation unit based on the current actual joint angle vector. Real-time updates . This represents the preset damping ratio constant. This represents the damping gain coefficient. When the minimum total semantic potential value... When the stiffness is increased, the variable stiffness impedance calculation unit adds an additional damping component while maintaining the critical damping characteristics of the system.

[0105] The variable stiffness impedance calculation unit calculates the stiffness matrix. Damping matrix and gravity compensation torque vector Substitute the values ​​into the aforementioned formula for calculating the joint torque control signal.

[0106] The variable stiffness impedance calculation unit outputs the calculated joint torque control signal to the servo drive controller. .

[0107] When the minimum total semantic potential value When in a high position, the servo drive controller drives the joints of the multi-degree-of-freedom robotic arm to produce low-stiffness and high-viscosity physical motion characteristics, making the joints of the multi-degree-of-freedom robotic arm subject to external contact force vectors. When applied, it produces a displacement deviation in the direction of the contact force.

[0108] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-user communication system based on AI robot knowledge graph, characterized in that, include: Intelligent robot terminal and control and processing system deployed on said intelligent robot terminal; The intelligent robot terminal includes a multimodal perception component, a motion execution component, and a computing control center; the computing control center includes a processor and a memory, and the memory stores computer program instructions; When the processor executes the computer program instructions, it implements the following processing modules: The multi-source semantic parsing module is configured to receive environmental data from the multimodal perception component, extract features, and output a structured semantic data stream; The dynamic hypergraph construction module is configured to build a dynamic hypergraph data structure based on the structured semantic data stream. The dynamic hypergraph data structure includes user nodes, object nodes, action primitive nodes, and hyperedges that connect at least two of the above nodes. The semantic potential field arbitration module is configured to calculate the semantic resistance value of the hyperedge and traverse the dynamic hypergraph data structure to search for the connected path with the minimum total semantic potential value as a candidate execution path. The isomorphic pruning verification module is configured to extract local session subgraphs containing the candidate execution paths and verify the security of the local session subgraphs using a subgraph isomorphic matching algorithm. The semantic motion mapping control module is configured to convert the verified candidate execution paths into physical control parameters that drive the motion execution components.

2. The multi-user communication system based on AI robot knowledge graph according to claim 1, characterized in that, The dynamic hypergraph construction module includes a unit for generating unresolved virtual nodes; The unresolved virtual node generation unit is configured to: when the confidence score of the received semantic input is lower than a preset threshold or is a non-independent modifying semantic, create an unresolved virtual node data object in the dynamic hypergraph data structure, and establish a temporary hyperedge connecting the unresolved virtual node data object and the source user node; The pending virtual node data object is assigned a real-time confidence attribute that decreases over time; when the real-time confidence attribute decays to a clearing threshold, the dynamic hypergraph construction module performs a node erasure operation; when matching supplementary semantic data is detected, the dynamic hypergraph construction module performs a node solidification operation.

3. A multi-user communication system based on an AI robot knowledge graph according to claim 1, characterized in that, The semantic potential energy field arbitration module calculates the semantic resistance value in the following way: Read the negative sentiment index of the user node associated with the hyperedge, the permission weight value of the user node, and the environmental constraint constant of the environment object node; The weighted negative sentiment index and the weighted environmental constraint constant are summed to obtain the semantic resistance value.

4. A multi-user communication system based on an AI robot knowledge graph according to claim 3, characterized in that, The semantic potential energy field arbitration module calculates the total semantic potential energy value of the connected path in the following way: Accumulate the semantic resistance values ​​of all hyperedges on the connected path, and the information entropy values ​​of all nodes on the connected path; The information entropy value is calculated based on the confidence attribute value of the node, and the information entropy value of the undecided virtual node data object is higher than that of the real node.

5. A multi-user communication system based on an AI robot knowledge graph according to claim 1, characterized in that, The isomorphic pruning verification module calls the taboo graph library stored in the memory, which contains several taboo subgraph data templates; The taboo subgraph data template includes node matching rule objects that define attribute constraints, and a set of superedges that define the connection relationships between the node matching rule objects. The isomorphic pruning verification module is configured to: determine whether the node attribute values ​​of the local session subgraph satisfy the attribute constraint conditions, and determine whether the topological connection relationship of the local session subgraph is consistent with the hyperedge set of the taboo subgraph data template; If the determinations are consistent, a blocking signal is generated.

6. A multi-user communication system based on an AI robot knowledge graph according to claim 5, characterized in that, The taboo graph library includes taboo subgraph data templates at the physical security level and taboo subgraph data templates at the policy compliance level; The taboo subgraph data template of the physical security level is used to describe the topology that could lead to hardware damage or physical injury. The taboo subgraph data template of the policy compliance level is used to describe the topology that violates the preset user permission table or the preset spatiotemporal constraint rules.

7. A multi-user communication system based on an AI robot knowledge graph according to claim 1, characterized in that, The semantic motion mapping control module includes a variable stiffness impedance calculation unit. The variable stiffness impedance calculation unit is configured to receive the minimum total semantic potential energy value corresponding to the candidate execution path calculated by the semantic potential energy field arbitration module, and generate the joint stiffness parameters and joint damping parameters of the motion execution component based on the minimum total semantic potential energy value. The joint stiffness parameter is inversely proportional to the minimum total semantic potential energy value, and the joint damping parameter is positively correlated with the minimum total semantic potential energy value.

8. A multi-user communication system based on an AI robot knowledge graph according to claim 7, characterized in that, The variable stiffness impedance calculation unit is configured to send a hybrid control command to the servo drive controller of the motion execution component, which includes the joint stiffness parameters, the joint damping parameters, and the gravity compensation torque vector. The hybrid control command drives the motion execution component to generate low stiffness and high viscosity physical motion characteristics when the minimum total semantic potential energy value is high, so that the motion execution component generates a displacement deviation in accordance with the direction of the contact force when it is subjected to an external contact force vector.

9. A multi-user communication system based on an AI robot knowledge graph according to claim 1, characterized in that, The semantic potential field arbitration module is also configured to execute deadlock detection logic: Compare the total semantic potential value of the candidate execution path with a preset deadlock determination threshold; If the total semantic potential value is greater than the deadlock determination threshold, then a control command is generated pointing to the hovering node. The hovering state node corresponds to the zero-speed motion command and the maximum damping parameter.

10. A multi-user communication system based on an AI robot knowledge graph according to claim 1, characterized in that, The multimodal sensing component includes a visual acquisition sensor that outputs RGB-D depth information, an audio array sensor that acquires the azimuth coordinates of the sound source, and a force feedback sensor that detects the value of the contact force. The motion execution component includes a multi-degree-of-freedom robotic arm joint, an end effector connected to the end of the multi-degree-of-freedom robotic arm joint, and a servo drive controller.

Images