A human-computer interaction method and system based on intelligent sensors

Abstract

The application discloses a man-machine interaction method and system based on intelligent sensors, and belongs to the technical field of robots. The application solves the problem of misoperation caused by abnormal sensor data to a certain extent. Through quality credibility evaluation and a dynamic weight distribution mechanism, the system can still stably operate when part of the sensors fail. Meanwhile, a dynamic safety boundary strategy based on scene semantic understanding enables the robot to autonomously adjust motion constraints according to object attributes and spatial relationships, so that safety and operation efficiency are synergistically optimized in complex tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, and particularly relates to a human-machine interaction method and system based on intelligent sensors. BACKGROUND

[0002] Human-robot collaborative robots are key equipment in the fields of intelligent manufacturing and rehabilitation medicine, and the core is to realize safe, natural and efficient physical interaction. At present, the interaction method based on multi-modal intelligent sensors (such as tactile and electromyographic signals) has become the mainstream of research. These methods usually recognize the interaction intention by collecting the operator's tactile pressure, electromyographic signals and other data, and generate control instructions.

[0003] However, the existing technology has obvious limitations. First, most methods lack real-time evaluation and processing mechanism for sensor data quality. When the sensor is disturbed or partially fails, the system still uses unreliable data for decision-making, leading to intention misjudgment and action out of control. Second, the perception, decision-making and control links are relatively fragmented, and a closed-loop system that dynamically adjusts according to real-time interaction states (such as data reliability and environmental risk) cannot be formed. This makes it difficult to ensure the adaptability, safety and reliability of the robot when it faces unstructured environments or complex tasks.

[0004] Therefore, how to provide a safe and reliable human-machine interaction method has become a research hotspot in the field. SUMMARY

[0005] The embodiments of the present application provide a human-machine interaction method and system based on intelligent sensors, which can improve the reliability of human-machine interaction, and the technical solutions are as follows:

[0006] On the one hand, a human-machine interaction method based on intelligent sensors is provided, and the method comprises:

[0007] In response to human-robot interaction operation of an operator on a target robot, original interaction data between the operator and the target robot and body state sensor data of the target robot are acquired through intelligent sensors, the original interaction data including haptic pressure data and electromyography signal sequences, the haptic pressure data being used to represent a pressure distribution pattern of a contact surface of the human-robot interaction, and the electromyography signal sequences being used to represent a spatiotemporal pattern of muscle activity of the operator; based on the original interaction data, a robot dynamics model of the target robot and the body state sensor data, multi-modal interaction features and an interaction scene semantic graph between the operator and the target robot are determined, the interaction scene semantic graph being a semantic understanding representation of an environmental scene, including object recognition results, spatial relationships between objects and scene function attributes; based on real-time quality reliability of the multi-modal interaction features, the interaction scene semantic graph and the original interaction data, target joint torque instructions and a dynamic safety boundary strategy of the target robot are generated, the dynamic safety boundary strategy being a safety constraint condition that is self-adaptively adjusted with an interaction state, including but not limited to a speed limit, an acceleration limit, a force limit and an obstacle avoidance constraint; based on the target joint torque instructions and the dynamic safety boundary strategy, the target robot is controlled, so that the target robot completes a task indicated by the human-robot interaction operation.

[0008] As can be seen from the above, the human-robot interaction method and system based on intelligent sensors provided in the present application solve the problems of low data reliability and rigid safety strategy in the prior art by generating control instructions in combination with a dynamic safety boundary strategy through real-time evaluation of sensor data quality and fusion of multi-modal features, and have the advantages of improving interaction safety and enhancing environmental adaptability.

[0009] In one aspect, a human-robot interaction system based on intelligent sensors is provided, and the system includes:

[0010] An acquisition module is configured to, in response to human-robot interaction operation of an operator on a target robot, acquire original interaction data between the operator and the target robot and body state sensor data of the target robot through intelligent sensors, the original interaction data including haptic pressure data and electromyography signal sequences, the haptic pressure data being used to represent a pressure distribution pattern of a contact surface of the human-robot interaction, and the electromyography signal sequences being used to represent a spatiotemporal pattern of muscle activity of the operator;

[0011] A determination module is configured to determine multi-modal interaction features and an interaction scene semantic graph between the operator and the target robot based on the original interaction data, a robot dynamics model of the target robot and the body state sensor data;

[0012] The generating module is configured to generate a target joint torque instruction and a dynamic safety boundary strategy of the target robot based on the multi-modal interaction feature, the interaction scene semantic graph, and real-time quality credibility of the original interaction data.

[0013] The control module is configured to control the target robot based on the target joint torque instruction and the dynamic safety boundary strategy, so that the target robot completes the task indicated by the human-robot interaction operation.

[0014] In one aspect, a robot controller is provided, which includes one or more processors and one or more memories, and at least one computer program is stored in the one or more memories, and the computer program is loaded and executed by the one or more processors to implement the intelligent sensor-based human-robot interaction method.

[0015] In one aspect, a computer readable storage medium is provided, and at least one computer program is stored in the computer readable storage medium, and the computer program is loaded and executed by a processor to implement the intelligent sensor-based human-robot interaction method.

[0016] In one aspect, a computer program product or computer program is provided, which includes program code stored in a computer readable storage medium, and a processor of a robot controller reads the program code from the computer readable storage medium, and the processor executes the program code to make the robot controller execute the intelligent sensor-based human-robot interaction method. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a schematic diagram of an implementation environment of an intelligent sensor-based human-robot interaction method provided by the embodiments of the present application;

[0019] Figure 2 is a flowchart of an intelligent sensor-based human-robot interaction method provided by the embodiments of the present application;

[0020] Figure 3 is a partial flowchart of an intelligent sensor-based human-robot interaction method provided by the embodiments of the present application;

[0021] Figure 4is a partial flowchart of another human-computer interaction method based on an intelligent sensor provided by an embodiment of the present application;

[0022] Figure 5 is a partial flowchart of still another human-computer interaction method based on an intelligent sensor provided by an embodiment of the present application;

[0023] Figure 6 is a structural schematic diagram of a human-computer interaction system based on an intelligent sensor provided by an embodiment of the present application;

[0024] Figure 7 is a structural schematic diagram of a robot controller provided by an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0026] In the present application, the terms "first", "second", and the like are used to distinguish the same or similar items with basically the same function and action, and it should be understood that there is no logical or time sequence dependency between "first", "second", and "nth", and the number and execution order are not limited.

[0027] Artificial intelligence (AI) is to use a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain better results.

[0028] Machine learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other subjects. It is a subject that studies how a computer simulates or implements human learning behavior to acquire new knowledge or skills, and reorganizes existing knowledge sub-models to continuously improve its performance. Machine learning is the core of artificial intelligence and the fundamental approach to enabling computers to have intelligence, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0029] Attention weight: can represent the importance of certain data in the training or prediction process, and the importance represents the size of the influence of the input data on the output data. The data with high importance has a higher value of the corresponding attention weight, and the data with low importance has a lower value of the corresponding attention weight. In different scenarios, the importance of data is not the same, and the process of training attention weight of the model is also the process of determining the importance of data.

[0030] Smart sensor: A kind of advanced sensor device that not only has the ability to sense physical signals, but also integrates microprocessors and specific algorithms to preprocess, extract features, diagnose internal states, and even make preliminary decisions on raw sensing signals. Unlike traditional sensors that only output raw signals, smart sensors output information that has been preliminarily processed and has higher-level semantics, and usually have self-calibration, self-compensation, self-diagnosis, and standardized digital interface communication capabilities.

[0031] Haptic force data: A collection of data representing the size and distribution of the contact surface pressure between the robot and the operating object or environment, collected by a distributed haptic sensor array.

[0032] Electromyographic signal sequence: A sequence of bioelectric signals representing the activity of the operator's muscles, collected in chronological order, used to infer the operator's movement intention.

[0033] Robot dynamics model: A physical model that describes the mathematical relationship between the motion of the robot and its joint driving force / torque, including mass, inertia, Coriolis force, gravity, and other parameters.

[0034] Mutual scene semantic graph: A structured representation of environmental knowledge composed of semantic nodes (such as objects, regions) and relationship edges (such as spatial relationships, functional attributes), used by the machine to understand the physical constraints and task context of the interaction scene.

[0035] Target joint torque instruction: The expected torque value calculated by the control algorithm and directly sent to the robot's joint drive, used to accurately control the robot's motion.

[0036] Dynamic safety boundary strategy: A series of safety constraints (such as speed, acceleration, and force upper limits) dynamically adjusted according to the real-time evaluation of the interaction risk level, used to optimize the performance of the robot while ensuring safety.

[0037] Space-time registration: The process of unifying data from different sensors, collected in different time and spatial coordinate systems, to the same time reference and spatial coordinate system.

[0038] Haptic credibility factor: A quantitative indicator used to evaluate the reliability of haptic force data, usually calculated based on the spatial consistency and transient anomalies of the signal.

[0039] Electromyographic credibility factor: A quantitative indicator used to evaluate the quality of the electromyographic signal sequence, usually calculated based on the signal-to-noise ratio and physiological reasonableness of the signal.

[0040] Spatial Distribution Features: Features extracted from tactile pressure data that describe the distribution pattern of pressure on the contact surface, such as pressure center, distribution gradient, symmetry, etc.

[0041] Temporal Sequence Features: Features extracted from sensor data sequences that describe their temporal variation patterns, such as mean, variance, trend, stability, etc.

[0042] Frequency Domain Power Spectrum Features: Features representing the energy distribution of signals in the frequency domain after Fourier transform of time-domain signals like EMG signals.

[0043] Time Domain Amplitude Statistical Features: Statistical features directly extracted from signal waveforms, such as RMS, mean absolute value, peak value, etc.

[0044] Temporal and Spatial Synchronization: The state of making multi-source sensor data have unified timestamps and coordinate references.

[0045] Basic Motion Feature Set: A set of low-level features extracted from robot body state sensor data (such as joint angles, velocities) that describe the robot's own motion state.

[0046] Basic Tactile Feature Set: A set of low-level features directly extracted from raw tactile pressure data, such as pressure values of each sensing unit, rate of change, etc.

[0047] Basic EMG Feature Set: A set of low-level features directly extracted from raw EMG signal sequences, such as signal amplitude, frequency, etc.

[0048] Dynamic Weight Allocation: A strategy that dynamically adjusts the contribution of different data sources according to real-time credibility factors during feature fusion.

[0049] Temporal and Spatial Alignment and Splicing: The process of aligning and merging feature vectors of different modalities in time and space dimensions into a unified feature vector.

[0050] Semantic Analysis: The process of identifying and extracting high-level semantic information (such as object categories, interaction relationships) from low-level perception data using algorithms or models.

[0051] Object Attributes: Intrinsic properties of objects in the interaction scene, such as category, material, size, weight, etc.

[0052] Spatial Relationships: Relative position, orientation, and distance between different objects or entities in the scene.

[0053] Task Context Information: Background information related to the current task, such as task goals, operation steps, historical actions, etc.

[0054] Intent probability distribution: A probability vector whose each element represents the likelihood of the operator currently having a certain pre-set interaction intent.

[0055] Interaction risk level: A quantitative assessment level of the likelihood and severity of potential dangerous events occurring during the current or future period of human-robot interaction.

[0056] Confidence boundary: An uncertainty range added around the predicted trajectory or parameter, used to represent the reliability of the prediction result.

[0057] Standard action precision requirement: The precision standard required for the robot's end effector in position, attitude, or force control to complete a specific type of task.

[0058] Task criticality level: An index that classifies the severity of consequences caused by task execution failure.

[0059] Trajectory deviation tolerance threshold: The maximum deviation limit allowed between the actual motion trajectory of the robot and the expected trajectory during task execution.

[0060] Risk impact weight coefficient: A proportional coefficient used in the risk assessment model to adjust the contribution size of different risk factors to the overall risk.

[0061] Environmental complexity index: A quantitative index that comprehensively represents the difficulty of robot motion caused by the number, distribution, and dynamics of obstacles in the interaction environment.

[0062] Environment-trajectory interaction risk value: A local collision risk quantitative value calculated by comprehensively analyzing the interaction between the predicted motion trajectory and static / dynamic obstacles in the environment.

[0063] Confidence calibration: The process of correcting the reliability of the early calculation results (such as risk probability) using subsequent information (such as data reliability).

[0064] Safety boundary relaxation coefficient: A coefficient used to scale the default safety boundary range, which is negatively related to the interaction risk level, achieving a balance between safety and efficiency.

[0065] Feasibility correction: The process of adjusting the preliminary control instructions generated by the optimization algorithm to meet the physical constraints such as robot dynamics, kinematics, and safety boundaries.

[0066] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0067] Figure 1 is an implementation environment schematic diagram of a human-computer interaction method based on an intelligent sensor provided by an embodiment of the present application, referring to Figure 1 The implementation environment can include a target robot 100, and the target robot 100 includes a robot controller 110.

[0068] The robot controller 110 is connected with the intelligent sensor of the target robot, can acquire data collected by the intelligent sensor, and processes the data, thereby controlling the target robot 100. The robot controller 110 has strong computing power and can quickly process complex data.

[0069] In related technologies, human-robot collaborative robots are widely used in intelligent manufacturing and rehabilitation medical fields, and the core thereof depends on multi-modal intelligent sensors to collect data such as operator's tactile pressure and electromyographic signals to realize interactive control. However, the existing method has the problem of lack of sensor data quality evaluation mechanism, and when the sensor is disturbed, unreliable data is still used for decision-making, which leads to the risk of intention misjudgment. In addition, the traditional safety constraint adopts a fixed threshold mode, which cannot be dynamically adjusted according to the environmental complexity and interaction state, and in unstructured scenes, action delay or excessively high safety redundancy may occur. For example, in assembly work, when the operator's hand sweat causes the tactile sensor to fail, the system cannot identify the data anomaly, and still performs the grabbing action according to the error signal, which may cause damage to the workpiece or injury to the personnel.

[0070] In order to solve the above problems, the inventors observed the defects of the existing system in data quality monitoring and safety strategy linkage. By analyzing typical working conditions such as the appearance of sudden obstacles and the sudden change of operator's intention in industrial scenes, it is found that the traditional method lacks a data reliability evaluation mechanism when the sensor fails, and the safety constraint condition is solidified, which leads to insufficient flexibility of the system. Based on this, a multi-modal data quality real-time evaluation system is established, and the safety boundary strategy is dynamically associated with the interaction risk level. Further, the scene semantic graph is introduced to enhance the environmental context understanding ability, so that the robot can adjust the behavior mode according to the spatial relationship of objects and the task attributes.

[0071] Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps. Therefore, the present application proposes a human-computer interaction method based on an intelligent sensor, taking a robot controller as an example of an execution subject, referring to Figure 2 , which includes the following steps.

[0072] 201、responding to the human-robot interaction operation of the operator on the target robot, acquiring original interaction data between the operator and the target robot and ontology state sensor data of the target robot through an intelligent sensor;

[0073] 202、based on the original interaction data, the robot dynamics model of the target robot and the ontology state sensor data, determining multi-modal interaction features and interaction scene semantic atlas between the operator and the target robot;

[0074] 203、based on the real-time quality credibility of the multi-modal interaction features, the interaction scene semantic atlas and the original interaction data, generating target joint torque instructions and dynamic safety boundary strategies of the target robot;

[0075] 204、based on the target joint torque instructions and the dynamic safety boundary strategies, controlling the target robot to complete the task indicated by the human-robot interaction operation.

[0076] Wherein, the original interaction data includes tactile pressure data and electromyographic signal sequence, the tactile pressure data is used to represent the pressure distribution pattern of the contact surface of human-computer interaction, and the electromyographic signal sequence is used to represent the spatiotemporal pattern of muscle activity of the operator, the tactile pressure data refers to the acquisition of the pressure distribution pattern of the contact surface through a flexible pressure sensor array, which can be realized by a capacitive or piezoresistive sensor, and is used to represent the contact force size and distribution characteristics applied by the operator. The electromyographic signal sequence refers to the time series data of muscle electrical activity collected by a surface electromyographic sensor, which can be realized by a differential electrode and a signal amplifier, and is used to reflect the muscle contraction intensity and timing pattern of the operator. The robot dynamics model refers to a mathematical model describing the relationship between robot motion and force, which can be constructed by Lagrange equation or Newton-Euler equation, and is used to realize the spatiotemporal registration and physical constraint verification of multi-source data. The multi-modal interaction features refer to a composite feature vector that integrates tactile, electromyographic and ontology motion information, which can be generated by spatiotemporal alignment and weight allocation algorithm, and is used to represent the mechanical properties and motion intention of human-computer interaction. The dynamic safety boundary strategy refers to a set of constraint conditions automatically adjusted according to the interaction risk level, which can be realized by fuzzy logic or model predictive control method, and is used to balance the system response speed and safety redundancy.

[0077] Specifically, when the operator performs the assembly operation, the flexible pressure sensor array collects the hand contact pressure distribution in real time, and the surface electromyography sensor synchronously acquires the forearm muscle electrical signal. The robot joint encoder and the inertial measurement unit provide the body motion state data. The haptic data and the electromyography signal are time-stamped aligned through the robot dynamics model, and the haptic reliability factor and the electromyography reliability factor are calculated. When it is detected that the pressure sensor in a certain region abnormally deviates, the haptic reliability factor is reduced, and the system automatically reduces the weight of the data in the region in feature fusion. The fused multi-modal interaction features are input into a scene analysis model to identify semantic information such as workbench workpiece position and clamp orientation, and a scene graph containing object spatial relationship is constructed. According to the real-time quality reliability evaluation result, when the signal-to-noise ratio of the electromyography signal is lower than a threshold value, the system increases the weight proportion of the environmental semantic information in decision-making. Based on the risk level, the joint torque limit threshold is dynamically adjusted, the acceleration constraint is tightened when obstacles are avoided, and the speed limit is relaxed in the free motion stage, so that the balance between safety and efficiency is realized.

[0078] Compared with the related art, the traditional method adopts an architecture of independently processing sensor data, and does not establish a correlation mechanism between data quality and decision logic. The present scheme dynamically adjusts the feature fusion weight through the reliability factor, and can still maintain the intention recognition accuracy when the sensor is partially failed. The existing safety strategy adopts a fixed parameter setting, while the present scheme analyzes the environmental constraint conditions based on the scene semantic graph, so that the safety boundary changes adaptively with the object attributes and spatial relationship. For example, when a fragile workpiece is identified, the system automatically increases the force control accuracy requirement of the end effector, while the traditional method cannot realize such context-related strategy adjustment.

[0079] Through the above technical scheme, the present application solves the problem of misoperation caused by sensor data abnormality to a certain extent, and through the quality reliability evaluation and dynamic weight distribution mechanism, the system can still operate stably when part of the sensor is failed. At the same time, the dynamic safety boundary strategy based on scene semantic understanding enables the robot to autonomously adjust the motion constraint according to the object attributes and spatial relationship, and to realize the cooperative optimization of safety and operation efficiency in complex tasks.

[0080] The present application further proposes the following technical scheme, taking the robot controller as an example of the execution subject, referring to Figure 3 , comprising the following steps.

[0081] 301. Based on the robot dynamics model, the quality of the original interaction data is evaluated and time and space are registered to obtain a time and space synchronized multi-modal data stream and a real-time quality reliability of the original interaction data;

[0082] 302. Based on the multi-modal data stream, the real-time quality reliability, and the body state sensor data, the multi-modal interaction features and the interaction scene semantic graph between the operator and the target robot are determined.

[0083] wherein the real-time quality credibility includes a haptic credibility factor and an electromyography credibility factor, the robot dynamics model refers to a mathematical model describing the relationship between robot motion and force, which can be constructed using Lagrange equation or Newton-Euler equation, and is used to verify whether the sensor data conforms to the physical law. The quality assessment and space-time registration refer to the reliability discrimination and time synchronization processing of multi-source data, which can be realized by Kalman filtering combined with timestamp alignment algorithm, and is used to eliminate sensor noise and transmission delay. The haptic credibility factor refers to an index quantifying the reliability of haptic data, which can be calculated by the matching degree of pressure distribution form and dynamics model prediction value, and is used to dynamically adjust the weight of haptic features. The electromyography credibility factor refers to a parameter evaluating the quality of electromyography signal, which can be obtained by analyzing the correlation between signal signal-to-noise ratio and muscle activation state, and is used to suppress the influence of abnormal electromyography data. The multi-modal data stream refers to a set of heterogeneous sensor data after space-time alignment, which can be generated by interpolation fusion under a unified time reference, and is used to ensure the time sequence consistency of feature extraction.

[0084] The following describes the way of realizing quality assessment and space-time registration by Kalman filtering combined with timestamp alignment algorithm in the above description.

[0085] This process is a collaborative computing process that combines physical model verification and signal processing, aiming to improve the original interaction data from different sensors existing in different space-time references into a multi-modal data stream with unified space-time and controllable quality. The core process can be divided into the following two stages:

[0086] First stage: quality assessment and credibility calculation based on dynamics model.

[0087] The goal of this stage is to give each raw data a quantitative "quality credibility" label.

[0088] Physical consistency verification:

[0089] Process: Use the known robot dynamics model to verify the sensor readings in a physical logic. For example, input the motor current reading of a joint, the joint angle sensor reading and the body inertial measurement unit (IMU) reading into the dynamics model.

[0090] Calculation: The model will calculate a theoretical joint torque or body acceleration, which will be compared with the actual reading of the torque sensor or IMU. If the difference exceeds the reasonable threshold preset according to the model accuracy and sensor noise level, it is determined that the data set has physical inconsistency.

[0091] Output: A physical consistency error score is generated.

[0092] Signal quality analysis:

[0093] Process: Perform signal-level quality assessment on each individual sensor's data stream in parallel.

[0094] Calculation: Kalman-filter-based residual analysis: Perform Kalman filter prediction on the data (e.g., visual marker position), and use the residual between the predicted value and the actual observation as a quality indicator. A sudden increase in residual usually means that the data is disturbed by sudden noise, occlusion, or transmission error.

[0095] Statistical property check: Detect if the signal is saturated (out of range), has a sudden jump (not following the kinematic law), or is stagnant for a long time (sensor failure).

[0096] Output: Produce a set of signal quality indicators (e.g., residual size, jump flag, saturation flag).

[0097] Real-time quality confidence fusion:

[0098] Process: Fuse the above "physical consistency error score" and "signal quality indicators".

[0099] Calculation: Use weighted fusion or fuzzy logic, etc., to output a comprehensive, normalized real-time quality confidence (usually a value between 0 and 1). For example, when the physical consistency error is large and the Kalman filter residual is also large, the data point's confidence will be significantly lowered.

[0100] Second stage: Spatio-temporal registration of multi-modal data.

[0101] The goal of this stage is to unify all data into the same time and spatial coordinate system, solving the "when" and "where" problems.

[0102] Temporal registration:

[0103] Process: Solve the problem of "temporal misalignment" of data caused by different acquisition and transmission delays of different sensors.

[0104] Calculation: Hardware clock synchronization and software timestamp alignment: Synchronize the clocks of each sensor as much as possible at the hardware level, and stamp each data packet with a high-precision timestamp.

[0105] Interpolation based on dynamic model: For data that cannot be strictly synchronized, use the high-quality data stream calculated in the previous step (e.g., joint trajectory smoothed by Kalman filter) as the time reference to perform spatio-temporal interpolation on other data with large delays or lower sampling rates (e.g., camera images triggered asynchronously). That is, according to the dynamic model, the theoretical pose of the robot at the exact moment of camera exposure is calculated, so that the image is associated with the correct robot state.

[0106] Spatial registration:

[0107] Process: Solve the problem of different sensor coordinate systems not being unified.

[0108] Calculation: Calibration parameter application: Apply the sensor inter-transformation matrix obtained through calibration in advance to convert all visual data, force sensation data, etc. to the robot base coordinate system or world coordinate system.

[0109] Model-assisted positioning: Combine the robot's own joint angle readings (calculate pose through forward kinematics) and the dynamics model to provide real-time accurate body state of the robot in the unified coordinate system as the spatial reference for multi-modal data fusion.

[0110] Final output

[0111] After the above two stages of processing, the system outputs: spatiotemporally synchronized multi-modal data stream: all data have unified timestamps and spatial coordinate systems, which can be directly fused and analyzed meaningfully. For example, it can be exactly known that "at t=1.235 seconds, the robot's end in the coordinate system (X, Y, Z) position, the object image seen by the camera at this time, and the contact force (Fx, Fy, Fz, Mx, My, Mz) perceived by the six-axis force sensor are strictly corresponding".

[0112] Real-time quality credibility of original interaction data Each original data point or data frame is accompanied by a credibility score, and subsequent algorithms (such as fusion, recognition, decision-making) can determine the degree of trust in the data based on this score, thereby realizing quality-driven robust interaction.

[0113] Specifically, the robot dynamics model is used to establish physical constraints, and by comparing the difference between the tactile pressure data and the model-predicted contact force distribution, a tactile credibility factor is calculated. At the same time, the frequency domain features of the electromyographic signal sequence are correlated with the muscle activation level calculated by the dynamics model to generate an electromyographic credibility factor. In the spatial registration link, the data of the tactile sensor, electromyographic sensor, and body state sensor are synchronized through timestamp alignment and motion compensation algorithms to form a multi-modal data stream with a unified time reference. Based on the credibility factor, the tactile and electromyographic features are dynamically weighted and fused to generate multi-modal interaction features in combination with the body motion features. The scene analysis model analyzes the object contact mode, motion trajectory, and spatial relationship in the interaction features to construct a graph structure containing semantic nodes and relationship edges.

[0114] Compared with the related art, the existing method usually directly uses original sensor data and lacks a quality evaluation mechanism, and is prone to produce false features when the sensor is disturbed. The scheme verifies the data rationality through a dynamic model and generates a credibility factor, and automatically suppresses the influence of low-quality data in the feature fusion stage. At the same time, the related art usually adopts a multi-modal fusion method with fixed weights, while the scheme dynamically adjusts the feature weight based on real-time credibility, improving the robustness under complex working conditions. In addition, the environmental perception and interaction feature processing of the existing method are independent of each other, and the scheme realizes the joint representation of physical interaction features and environmental semantics through the construction of a semantic atlas.

[0115] Through the above technical scheme, the technical problems of multi-modal sensor data reliability discrimination and time-space synchronization are solved to a certain extent, and robust feature extraction under low-quality data interference is realized. Through the dynamic fusion mechanism of credibility weighting, the dominant role of high-credibility data in decision-making is realized, and the misjudgment caused by sensor failure is avoided. The time-space registration technology eliminates the time misalignment problem of multi-source data, providing time sequence correlation features for subsequent intention recognition. The construction of the semantic atlas converts the physical interaction features into structured environmental semantic information, providing multi-dimensional scene understanding support for safety policy generation.

[0116] The application further proposes a technical scheme for quality evaluation and time-space registration of original interaction data based on a robot dynamics model, including extracting spatial distribution features and time sequence features of tactile pressure data to determine a tactile credibility factor, extracting frequency domain power spectrum features and time domain amplitude statistical features of electromyographic signal sequences to determine an electromyographic credibility factor, and time-space synchronizing tactile pressure data, electromyographic signal sequences and body state sensor data to obtain a multi-modal data stream.

[0117] Among them, the tactile credibility factor refers to a quantitative index for evaluating the quality of tactile data through the rationality of pressure distribution form, which can be specifically realized by combining spatial pressure gradient distribution features with time sequence stability analysis, and is used to detect sensor contact abnormalities or signal distortion. The electromyographic credibility factor refers to a quantitative index for evaluating the effectiveness of electromyographic data through muscle activity signal features, which can be specifically realized by combining frequency band energy ratio analysis with time domain signal variance calculation, and is used to exclude noise interference in electromyographic signal acquisition. Time-space synchronization refers to an operation for eliminating the time delay and spatial coordinate difference of multi-source sensor data, which can be specifically realized by combining timestamp alignment algorithm with robot kinematics inverse solution, and is used to ensure the consistency of multi-modal data in time-space dimensions.

[0118] Specifically, the scheme first performs morphological analysis on the two-dimensional pressure matrix of the tactile pressure data, extracts the spatial gradient features of the pressure distribution to identify contact surface abnormalities, and calculates the autocorrelation coefficient of the time series to judge the signal stability, and comprehensively generates the tactile reliability factor. For electromyographic signals, the energy ratio features of specific frequency bands are extracted by fast Fourier transform, and the time domain amplitude variance is calculated by sliding window statistics, and the electromyographic reliability factor is generated after double verification of signal effectiveness. Finally, the motion state data of the robot body sensor is used as the reference, the time stamp calibration is used to eliminate the sampling rate difference of the sensor, and the robot kinematics model is used to unify the data of each sensor to the same coordinate system, forming a spatiotemporal consistent multi-modal data stream.

[0119] Compared with related technologies, the traditional method only judges the data validity by a single threshold, and cannot distinguish different failure modes of tactile and electromyographic signals. The existing spatiotemporal synchronization technology mainly adopts fixed delay compensation, which is difficult to adapt to coordinate transformation in dynamic robot motion. The scheme solves the key bottleneck in multi-sensor data collaborative processing by designing differentiated quality evaluation methods for tactile and electromyographic signals according to their physical characteristics, and realizing dynamic spatiotemporal registration combined with robot dynamics model.

[0120] Through the above technical solutions, the technical problems of real-time sensor data quality evaluation and spatiotemporal synchronization are solved to a certain extent, the quality of tactile and electromyographic signals is dynamically monitored, and the system misjudgment caused by single sensor failure is avoided. Through the spatiotemporal registration mechanism, the physical consistency of multi-modal data in the robot motion process is realized, which provides a reliable data basis for subsequent interaction decision-making.

[0121] The application further proposes a multi-modal interaction feature and interaction scene semantic graph based on multi-modal data stream, real-time quality reliability and body state sensor data to determine the multi-modal interaction feature and interaction scene semantic graph between the operator and the target robot, including parallel feature extraction of the body state sensor data and the multi-modal data stream to obtain a basic motion feature set, a basic tactile feature set and a basic electromyographic feature set; dynamic weight allocation and feature fusion are performed on the basic tactile feature set and the basic electromyographic feature set based on the tactile reliability factor and the electromyographic reliability factor to obtain a tactile feature vector and an electromyographic feature vector; the tactile feature vector, the electromyographic feature vector and the basic motion feature set are spatiotemporally aligned and spliced to obtain a multi-modal interaction feature; the multi-modal interaction feature is input into a scene analysis model for semantic analysis to obtain target structured data; and an interaction scene semantic graph including semantic nodes and relationship edges is constructed based on the target structured data.

[0122] wherein the parallel feature extraction refers to a process of independently extracting basic features from different modal data, which can be specifically implemented by using a convolutional neural network and a long short-term memory network to process spatial distribution features and time sequence features respectively, thereby avoiding interference between different modal data. The dynamic weight distribution and feature fusion refers to a process of dynamically adjusting the feature fusion weight according to the sensor data quality, which can be specifically implemented by using a credibility factor weighted summation algorithm, thereby improving the feature robustness under low-quality data. The space-time alignment and splicing refers to a process of unifying the space-time reference of multi-modal features, which can be specifically implemented by using a timestamp matching and coordinate system conversion technology, thereby eliminating the feature misplacement caused by the sensor acquisition time delay. The scene analysis model refers to an algorithm model for converting multi-modal features into structured semantic data, which can be specifically implemented by using a graph neural network combined with a knowledge graph embedding technology, thereby realizing deep analysis of environmental semantics. The semantic node and relationship edge refer to graph elements for describing object attributes and spatial relationships, which can be specifically constructed by using entity recognition and relationship extraction algorithms, thereby forming an interpretable scene knowledge expression.

[0123] Specifically, the body state sensor data and the multi-modal data stream are respectively input into three independent feature extraction modules. The motion feature extraction module extracts the robot body motion state through Kalman filtering and covariance analysis. The tactile feature extraction module obtains the contact surface mechanical properties through pressure distribution matrix decomposition and gradient calculation. The electromyographic feature extraction module analyzes the muscle activity pattern through wavelet transform and envelope detection. The tactile credibility factor and the electromyographic credibility factor are input into the adaptive weighted fusion device. When the tactile data credibility is lower than the preset threshold, the weight coefficient is automatically reduced. For example, when the tactile sensor is disturbed by temperature, the credibility factor is reduced to 0.3, and at this time, the electromyographic feature weight is increased to 0.7. The space-time alignment module uses the dynamic time warping algorithm to align the sampling time axes of different sensors, and maps each feature to a unified workspace coordinate system through a coordinate transformation matrix. The graph convolution network in the scene analysis model performs node embedding on the spliced multi-modal features, identifies key objects and their spatial topological relationships by using an attention mechanism, and finally generates a semantic graph containing object material, position coordinates and functional attributes.

[0124] Compared with the related art, the traditional method adopts a fixed weight feature fusion strategy, which cannot guarantee the reliability of the fusion result when the sensor data quality fluctuates, and the environmental semantic modeling is mostly limited to two-dimensional plane relationship. The present scheme solves the feature degradation problem under the condition of sensor failure or interference to a certain extent through a credibility-driven dynamic weight adjustment mechanism, for example, automatically enhancing the tactile feature weight when the electromyographic signal is disturbed by motion artifacts. At the same time, the construction of the three-dimensional semantic graph breaks through the flat representation of spatial relationships in the traditional method, and can accurately express the three-dimensional orientation relationship and functional constraint conditions between objects.

[0125] By the technical solution, the quality adaptive fusion of multi-modal sensor data is realized, and the robustness of the feature extraction process to noise and interference is significantly improved. Meanwhile, through hierarchical semantic analysis and knowledge graph construction, the cognitive understanding ability of the robot to the complex scene is effectively improved, and the environmental semantic support for subsequent safety decision is provided.

[0126] The application further proposes the following technical solutions, taking the robot controller as an example for the execution subject, Figure 4 , comprising the following steps.

[0127] 401. Based on the real-time quality reliability of the multi-modal interaction features, the interaction scene semantic graph and the original interaction data, the intention probability distribution of the operator, the robot expected motion trajectory envelope and the interaction risk level are determined;

[0128] 402. Based on the intention probability distribution, the robot expected motion trajectory envelope, the interaction risk level and the interaction scene semantic graph, the target joint torque instruction and the dynamic safety boundary strategy of the target robot are generated.

[0129] Wherein, the intention probability distribution refers to the diversified probability estimation of the operator's intention, the multi-modal interaction feature refers to a high-dimensional representation vector that fuses tactile, electromyographic and motion features, and specifically can be implemented by extracting feature data with spatio-temporal correlation using a spatio-temporal attention neural network. The feature can reflect the coupling relationship between the operator's action pattern and the robot's motion state. The interaction scene semantic graph refers to graph structure data containing object attributes and spatial relationships, and specifically can be implemented by analyzing semantic information in multi-modal data using a scene analysis model. Its role is to provide environmental constraints for trajectory planning. Real-time quality reliability refers to a quantitative evaluation index of sensor data reliability, and specifically can be implemented by weighted calculation of tactile reliability factor and electromyographic reliability factor, which is used to dynamically adjust the weight distribution of different modal data in the decision-making process. The intention probability distribution refers to the probabilistic description of the operator's potential intention, and specifically can be implemented by using the probability set output by the spatio-temporal attention mechanism combined with the classifier. Its role is to retain the possibility of multiple intentions to cope with data uncertainty. The robot expected motion trajectory envelope refers to a geometric description containing trajectory feasible region and confidence boundary, and specifically can be implemented by superimposed calculation of dynamics model prediction and obstacle space constraint, which is used to limit the physical feasible range of robot motion.

[0130] Specifically, when the operator performs a human-robot collaborative task, the haptic sensor and the electromyography sensor synchronously collect interaction data. The haptic reliability factor and the electromyography reliability factor are calculated by a quality evaluation module, for example, when the electromyography signal is interfered by high-frequency noise, the reliability factor can dynamically reduce its weight. After the multi-modal interaction features are extracted by a spatio-temporal attention network, the intention recognition module generates a probability distribution containing operation categories such as "grab", "translate", and "release". At the same time, the scene semantic graph analyzes the distribution of obstacles and the boundary of the operation area in the environment, for example, identifying the edge coordinates of the workbench and the location of the workpiece stack. The trajectory planning module combines the intention probability distribution and the semantic constraints to generate an expected trajectory envelope, for example, when the "grab" intention probability reaches a threshold, a feasible region containing the gripper opening and closing trajectory and obstacle avoidance path is generated. The safety policy generation module dynamically adjusts the constraint conditions according to the trajectory envelope and the risk level, for example, when approaching the area of precision instruments, the speed limit threshold is automatically reduced.

[0131] Compared with related technologies, the traditional method adopts a fixed threshold safety boundary strategy, which is prone to over-conservatism or constraint failure when the environment changes dynamically. The present scheme analyzes the spatial relationship in real time through the semantic graph, for example, when a moving obstacle is detected to approach, the safety margin of the trajectory envelope can be shrunk in advance. Existing intention recognition systems usually output a single determination result, which is prone to misjudgment when the quality of sensor data fluctuates, while the present scheme retains a multi-element probability distribution, for example, when the electromyography signal is disturbed, the "grab" intention alternative probability can still be maintained through the haptic features, avoiding the mutation of control commands.

[0132] Through the above technical solutions, the present application solves the problem of intention misjudgment caused by fluctuations in the quality of sensor data in traditional human-computer interaction, for example, when the electromyography signal is affected by muscle fatigue, the reliability weight can be reduced to maintain stable intention recognition accuracy. At the same time, the dynamic adaptation of safety constraints is realized, for example, when handling fragile objects, the force limit is automatically enhanced, while in an open area, the acceleration limit is appropriately relaxed when performing rapid movement, thereby improving operation efficiency while ensuring safety. In addition, by fusing scene semantic information, the risk of collision between the robot arm and unknown obstacles can be effectively avoided, for example, when the semantic graph detects a temporarily placed toolbox, the obstacle avoidance constraint condition is updated in time.

[0133] The application further proposes a method for determining the intention probability distribution of an operator, the expected motion trajectory envelope of a robot, and the interaction risk level based on multi-modal interaction features, interaction scene semantic graphs, and real-time quality reliability, including inputting multi-modal interaction features into a spatio-temporal attention neural network to extract high-dimensional interaction features with spatio-temporal correlation, and dynamically adjusting the attention weights of the spatio-temporal attention neural network according to real-time quality reliability; determining the probability of multiple preset intention categories based on high-dimensional interaction features and interaction scene semantic graphs to obtain an intention probability distribution; determining the future motion trajectory of a target robot based on the intention probability distribution and high-dimensional interaction features, and determining the expected motion trajectory envelope of the robot based on the spatial constraint relationship in the interaction scene semantic graph and the future motion trajectory; determining the risk probability of different interaction strategies based on the intention probability distribution, the expected motion trajectory envelope of the robot, and the interaction scene semantic graph, and determining the interaction risk level based on the risk probability of different interaction strategies.

[0134] Among them, the spatio-temporal attention neural network refers to a deep learning model that captures the spatio-temporal correlation of multi-modal data through an attention mechanism, which can be specifically implemented by a convolutional recurrent neural network with a multi-head self-attention structure, and its attention weights are dynamically adjusted according to real-time quality reliability, so that the contribution of low-reliability features in feature extraction is reduced. The intention probability distribution refers to a diversified probability estimate of the operator's intention, which can be specifically implemented by a classifier based on a softmax function to calculate the multi-class probability of high-dimensional interaction features, and the likelihood of different intentions is expressed in the form of a probability set. The expected motion trajectory envelope of the robot refers to the feasible region boundary of the future motion trajectory of the robot, which can be specifically generated by combining the dynamics constraints and environmental spatial relationships through a trajectory optimization algorithm, such as calculating the trajectory confidence interval using a model predictive control algorithm. The interaction risk level refers to a quantitative evaluation result of the safety of the interaction strategy, which can be specifically calculated using a risk weighted evaluation model combined with environmental obstacle distribution, trajectory speed indicators, and task criticality level.

[0135] Specifically, after receiving the multi-modal interaction features, the spatio-temporal attention neural network adjusts the attention weights of each modal feature according to the tactile credibility factor and the electromyography credibility factor. For example, when the electromyography signal credibility decreases, the weight coefficient of the electromyography feature in the attention calculation is reduced. The high-dimensional interaction features and the object spatial relationship in the interaction scene semantic atlas are jointly input into the intention classification module, and the confidence ranking of the possible intentions of the operator is output through the probability distribution. In the generation process of the future motion trajectory, the high-probability intention corresponding candidate trajectory is selected based on the intention probability distribution, and the physical feasibility of the trajectory is verified through the robot dynamics model, for example, whether the joint torque exceeds the motor output limit. The robot expected motion trajectory envelope is obtained by spatially superimposing the verified trajectory and the obstacle position in the scene semantic atlas, calculating the minimum safety distance between each point of the trajectory and the obstacle, and forming a trajectory boundary containing velocity and acceleration constraints. The determination of the interaction risk level is obtained by analyzing the dynamic relationship between the trajectory envelope and the environmental obstacles, for example, when the trajectory envelope overlaps with the predicted path of the dynamic obstacle, the risk probability value is increased, and the task criticality level is combined for weighted comprehensive evaluation.

[0136] Compared with related technologies, the existing method usually adopts a fixed weight feature fusion method to process multi-modal data, which cannot dynamically adjust the feature importance when the sensor data quality fluctuates, resulting in cumulative intention recognition errors. The present scheme dynamically adjusts the attention weight through real-time quality credibility, automatically reduces the influence of electromyography signals when they are disturbed, and improves the robustness of feature extraction. In related technologies, motion trajectory planning often ignores environmental semantic constraints, for example, without considering the influence of obstacle movement trend on trajectory feasibility, while the present scheme provides spatial relationship prior knowledge through the interaction scene semantic atlas, so that the trajectory envelope can adapt to environmental changes. Existing risk assessment is mostly based on a single indicator, for example, only considering the mechanical arm end speed, while the present scheme integrates trajectory dynamic characteristics, environmental obstacle distribution and task criticality to realize multi-dimensional risk coupling evaluation.

[0137] Through the above technical solutions, when the sensor data quality fluctuates, the present application reduces the influence of low-quality data on intention recognition by dynamically adjusting the feature weight, improves the accuracy of intention probability estimation, generates a trajectory envelope that conforms to physical laws and spatial constraints by fusing environmental semantic information, avoids collisions or boundary violations during robot motion, and dynamically evaluates the safety of the interaction strategy through multi-dimensional risk indicators to provide fine-grained basis for subsequent safety strategy adjustment.

[0138] The application further proposes a method for determining a future motion trajectory of a target robot based on an intention probability distribution and high-dimensional interaction features, comprising: determining a plurality of candidate task targets and task target weights based on the intention probability distribution; generating a plurality of predicted motion trajectories based on the high-dimensional interaction features and the task target weights; performing feasibility verification and optimization on the predicted motion trajectories based on dynamic constraints and kinematic constraints to obtain a motion trajectory set; selecting a target trajectory from the set and adding a confidence boundary to obtain a future motion trajectory.

[0139] The intention probability distribution refers to a diversified probability estimate of the operator's intention. Specifically, a spatiotemporal attention neural network can be used to analyze the multi-modal interaction features and output a probability set of different intention categories, which is used to quantify the uncertainty of intention recognition. The task target weight refers to the priority coefficient of the candidate task target in trajectory generation. Specifically, it can be calculated by combining probability normalization and task criticality evaluation, which is used to balance the competition between multiple targets. The dynamic constraint refers to the physical limitations such as joint torque and inertia parameters of the robot. Specifically, a dynamic model can be established by Lagrange equation to verify that the trajectory conforms to the actual driving ability. The kinematic constraint refers to the geometric limitations such as joint angle and speed. Specifically, the workspace boundary can be analyzed by Jacobian matrix to avoid the trajectory exceeding the mechanical structure limit. The confidence boundary refers to the probability envelope range around the target trajectory. Specifically, a Gaussian process regression model can be used to generate it according to the standard deviation of the intention probability distribution, which is used to represent the credible interval of trajectory prediction.

[0140] Specifically, when the intention probability distribution indicates the existence of multiple candidate task targets, the task target weight distribution mechanism is used to associate the operator's behavior patterns implied in the high-dimensional interaction features with each task target, generating predicted motion trajectories with different target orientations. Then, the dynamic constraint verification eliminates trajectory segments that exceed the upper limit of joint torque, and the kinematic constraint optimization adjusts the spatial position of the trajectory points to avoid mechanical interference. Among the trajectory set filtered by the double constraints, the target trajectory with the highest matching degree with the current interaction features is selected, and the confidence boundary width is calculated according to the difference between the maximum probability value and the second maximum probability value in the intention probability distribution. For example, when there is high uncertainty in intention recognition, the confidence boundary is automatically expanded to increase the safety margin, and vice versa, the boundary is contracted to improve tracking accuracy.

[0141] Compared with the related art, the traditional method is usually based on a single intention assumption to generate a fixed trajectory, which is prone to cause trajectory failure when the intention recognition deviates. The scheme generates a safe trajectory by multi-modal expansion and dynamic weight distribution of candidate task targets, compatible with multiple possible intentions in the trajectory generation stage, and then eliminates infeasible schemes through physical constraint verification, and finally superimposes the confidence boundary to form a safe trajectory. In the related art, trajectory optimization mainly focuses on geometric path planning, while the scheme combines intention uncertainty quantification and physical constraint verification to realize full-link reliability guarantee from intention understanding to motion execution.

[0142] Through the above technical solutions, the application can effectively avoid the problem of trajectory planning failure caused by intention misjudgment due to sensor noise or environmental interference. The dual mechanism of multi-candidate target generation and physical constraint verification ensures the feasibility and safety of the trajectory, and the confidence boundary is used to dynamically adjust the tightness of trajectory tracking, which improves the operation fluency on the premise of ensuring the safety of human-machine collaboration.

[0143] The application further proposes a method for determining the risk probability of different interaction strategies based on intention probability distribution, robot expected motion trajectory envelope, and interactive scene semantic graph, including: querying the standard action accuracy requirement and task criticality level corresponding to the candidate intention with the highest probability from the task knowledge base based on the intention probability distribution; extracting the spatial distribution of static obstacles, motion trajectory of dynamic obstacles, and safety operation area boundary in the environment as basic environmental risk factors based on the interactive scene semantic graph; determining the minimum distance between each point of the trajectory and the obstacle, the speed peak at the key point of the trajectory, and the acceleration change rate to obtain the motion trajectory risk index based on the robot expected motion trajectory envelope; determining the risk probability of different interaction strategies based on the standard action accuracy requirement, task criticality level, basic environmental risk factors, and motion trajectory risk index through a risk weighted evaluation model.

[0144] The intention probability distribution refers to a diversified probability estimation of the operator's intention, which can specifically output a probability set by weighting and fusing different modal interaction features using a spatio-temporal attention neural network, to reflect the uncertainty of the operation intention. The task knowledge base refers to a database storing task prior knowledge, which can specifically store the mapping relationship between task types and corresponding action accuracy, criticality level using a graph database, to provide benchmark parameters for risk assessment. The standard action accuracy requirement refers to the operation accuracy threshold required by a specific task type, which can specifically be quantified using a millimeter or micrometer level numerical range, to set the tolerance of trajectory deviation. The basic environmental risk factor refers to a set of environmental elements that affect safety, which can specifically use three-dimensional point cloud data to represent static obstacle distribution and use a Kalman filter to predict the trajectory of dynamic obstacles, to build an environmental risk model. The motion trajectory risk indicator refers to a quantitative parameter reflecting the risk of trajectory execution, which can specifically use the Euclidean distance to calculate the minimum distance between the trajectory point and the obstacle, and use a sliding window to calculate the speed extreme value, to assess the motion control risk.

[0145] Specifically, the method matches the operation intention with the preset task standard through the task knowledge base, establishing a benchmark parameter system for risk assessment. For example, when the assembly task intention is detected, the corresponding high-precision action requirement is automatically called as the evaluation benchmark. The obstacle spatial distribution data is extracted through the interactive scene semantic graph, for example, the fixed device coordinates and moving object trajectories in the work area are identified using a semantic segmentation network, and the environmental risk topology structure is constructed. The kinematic parameters are calculated based on the robot expected trajectory envelope, for example, whether the maximum centripetal acceleration at the trajectory curvature mutation point exceeds the safety threshold is detected. The task criticality, environmental complexity and motion risk indicators are integrated through the risk weighted evaluation model, for example, the criticality weight of the assembly task is set to 0.7 and the carrying task is set to 0.3, and the comprehensive risk value is calculated by combining the obstacle density and speed peak value. The non-accurate parameters are processed using a fuzzy reasoning system, for example, when the trajectory deviation is in a critical state, the membership level of the risk probability is calculated through a membership function.

[0146] Compared with related technologies, the traditional method usually only evaluates the collision risk based on single sensor data, for example, only calculates the straight-line distance between the robot arm end and the obstacle. The present scheme realizes multi-dimensional risk assessment by fusing the prior standards of the task knowledge base and the dynamic characteristics of the environment, for example, considering the task accuracy requirement and the predicted trajectory of the moving obstacle simultaneously when evaluating the risk. Related technologies use fixed threshold to judge the risk level, for example, set a unified safety distance threshold, while the present scheme improves the evaluation adaptability through a dynamic weight distribution mechanism, for example, automatically adjusts the risk tolerance according to the task criticality. The traditional risk assessment model lacks processing of the uncertainty of the operation intention, while the present scheme introduces the intention probability distribution, for example, when multiple intentions coexist, the comprehensive risk value is calculated using a weighted average method, enhancing the system robustness.

[0147] By the technical solution, the present application solves the problem of inaccurate safety strategy caused by single risk factor consideration in related technologies. By integrating the task criticality level and environmental dynamic characteristics, such as automatically increasing the sensitivity to trajectory deviation in a precision assembly scene, the matching of risk assessment and task target is achieved. By quantifying the motion trajectory risk index, such as detecting trajectory acceleration mutation points and correlating environmental obstacle distribution, the prospective prediction of potential collision risk is achieved. By fuzzy reasoning processing multi-source heterogeneous data, such as converting non-velocity peak data into risk membership degree, the decision reliability under complex working conditions is improved. The multi-dimensional risk assessment system thus constructed provides a quantitative basis for dynamically adjusting the safety boundary.

[0148] The present application further proposes to determine the trajectory deviation tolerance threshold based on the standard action precision requirement, to determine the risk influence weight coefficient based on the task criticality level, to determine the environmental complexity index based on the static obstacle spatial distribution and the dynamic obstacle motion trajectory, to determine the environment-trajectory interaction risk value based on the environmental complexity index and the minimum distance, velocity peak and acceleration change rate in the motion trajectory risk index, to input the trajectory deviation tolerance threshold, risk influence weight coefficient and environment-trajectory interaction risk value into the fuzzy reasoning system, to output the initial risk probability through the preset fuzzy rule base for comprehensive risk assessment, and to obtain the risk probability based on the real-time quality confidence.

[0149] The trajectory deviation tolerance threshold refers to the maximum error range allowed for the motion trajectory to deviate from the preset path, which can be specifically implemented by weighted calculation of task precision requirement and robot joint positioning error, and is used to quantify the tolerance degree of trajectory deviation during task execution. The risk influence weight coefficient refers to the amplification factor for adjusting the risk assessment result according to the task importance, which can be specifically obtained by matching the task criticality level with the preset weight mapping table, for example, a higher weight coefficient is given to medical operation tasks. The environmental complexity index refers to the comprehensive evaluation index of obstacle distribution density and dynamic obstacle motion uncertainty, which can be specifically implemented by fusion calculation of obstacle number, moving speed variance and path intersection probability. The environment-trajectory interaction risk value refers to the quantitative representation of the collision probability of robot motion trajectory and environmental elements, which can be specifically calculated by nonlinear combination of the minimum distance, velocity peak and acceleration change rate of trajectory key points and obstacles. The fuzzy rule base refers to the logical judgment set describing the coupling relationship of multiple risk factors, which can be specifically defined in the form of IF-THEN rules to correspond the risk level output of different input variable combinations.

[0150] Specifically, when the robot performs the carrying task, first, the trajectory deviation tolerance threshold is calculated as ±5 mm according to the clamping accuracy requirement, and the high-risk influence weight coefficient is set as 1.8 according to the fragile goods. The environmental complexity index is calculated as 0.75 by the obstacle distribution data obtained by the laser radar, combined with the minimum obstacle distance 0.3 m, the speed peak value 1.2 m / s and the acceleration 2 m / s² in the motion trajectory output by the trajectory planning module, the environment-trajectory interaction risk value is calculated as 0.68. After inputting the above parameters into the fuzzy reasoning system, the fuzzy rule of "high complexity environment + medium risk value" is triggered, and the initial risk probability is output as 65%. At this time, the electromyographic signal credibility factor detects sensor noise interference, and the real-time quality credibility decreases to 0.7. The risk probability is calibrated to 78% by the Bayesian probability correction model, and a more conservative safety boundary strategy is finally generated.

[0151] In some embodiments, the calculation of the environmental complexity index can be divided into two dimensions of static complexity and dynamic complexity, for example, the static obstacle quantity accounts for 60% of the weight, and the dynamic obstacle speed variance accounts for 40% of the weight. The fuzzy rule base can include nine core rules, for example, when the trajectory deviation tolerance threshold is lower than the first threshold and the environment-trajectory interaction risk value is higher than the second threshold, the high-risk level is directly triggered. The confidence calibration process can use linear interpolation, for example, the real-time quality credibility decreases by 0.1, and the risk probability increases by 5 percentage points.

[0152] Compared with related technologies, the traditional risk assessment method usually only considers a single obstacle distance or a preset safety speed threshold, and does not establish a multi-factor coupled evaluation model. The scheme fuses the three-dimensional risk assessment system of task attributes, environmental state and motion characteristics, combines the ability of fuzzy logic to handle nonlinear relationships, and significantly improves the evaluation accuracy in complex dynamic scenes. At the same time, the feedback calibration mechanism of real-time quality credibility is introduced, which effectively overcomes the interference of sensor data fluctuations on the evaluation results.

[0153] Through the above technical solutions, the application can automatically adjust the degree of conservatism of risk assessment when there is interference in the sensor data, accurately identify the motion trajectory segment with high conflict risk, and provide a quantitative basis for dynamic safety boundary strategy. Especially in the scene where multiple obstacles coexist and the task accuracy requirement is high, through the accurate calculation of the environment-trajectory interaction risk value, the potential collision risk can be predicted in advance and the robot motion parameters can be adjusted to avoid the misjudgment problem caused by the failure of local sensors.

[0154] The application further proposes a technical scheme for generating target joint torque instructions and dynamic safety boundary strategies for a target robot based on an intention probability distribution, a robot expected motion trajectory envelope, an interaction risk level, and an interaction scene semantic atlas, including: determining a safety boundary relaxation coefficient negatively correlated with the risk level based on the interaction risk level; constructing a dynamic constraint set based on spatial constraint relationships and object attribute information in the interaction scene semantic atlas, and generating a dynamic safety boundary strategy in combination with the safety boundary relaxation coefficient; generating a preliminary joint torque instruction sequence through a model predictive controller, with the optimization objectives including trajectory tracking accuracy, motion smoothness, and energy consumption; and performing feasibility correction on the preliminary instructions using the dynamic safety boundary strategy, and selecting the first instruction in the corrected sequence as the target joint torque instruction.

[0155] The safety boundary relaxation coefficient is a regulation parameter representing the degree of relaxation of safety constraints, which can be implemented by an exponential decay function to map the risk level to the relaxation coefficient. This parameter is used to automatically reduce the constraint threshold in high-risk scenarios to enhance safety. The dynamic constraint set is a variable physical constraint condition based on environmental semantic understanding, which can be implemented by a three-dimensional space grid method combined with object motion attributes, and is used to convert obstacle distribution in the scene into a robot motion prohibited area. The model predictive controller is a control algorithm based on rolling horizon optimization, which can be implemented by a quadratic programming solver to balance trajectory tracking accuracy and energy economy. The feasibility correction is a constraint compliance verification process for control instructions, which can be implemented by a projection gradient descent method to make the corrected instruction sequence meet the limitation conditions of the dynamic safety boundary.

[0156] Specifically, when generating control instructions, first adjust the strictness of safety constraints according to the real-time calculated interaction risk level. For example, when detecting that the operator's intention is ambiguous or the environment is densely obstructed, the system automatically increases the safety level and tightens the speed limit threshold. Then, based on the semantic atlas, extract the spatial coordinates and motion state of the objects, and construct a safety constraint set containing static obstacle prohibited areas and dynamic obstacle predicted trajectories. After generating preliminary torque instructions that take into account trajectory tracking and energy optimization through the model predictive controller, use the constraint projection algorithm to map the instruction sequence to the feasible region defined by the dynamic safety boundary. Finally, select the first instruction in the corrected sequence as the execution target to form a rolling optimization mechanism for real-time updating of control instructions.

[0157] Compared with the related art, the traditional method adopts a fixed safety threshold, resulting in constraint redundancy in a high-risk scenario or insufficient protection in a low-risk scenario, while the present scheme realizes dynamic constraint adjustment through a risk level-adaptive relaxation coefficient. In the related art, environmental constraints are usually based on preset rules and lack semantic understanding, while the present scheme realizes intelligent construction of constraint conditions through semantic graph analysis of environmental elements. Traditional control methods treat safety strategies as independent modules, resulting in a split between instruction generation and constraint checking, while the present scheme realizes closed-loop fusion of control instructions and safety strategies through feasibility modification.

[0158] Through the above technical solutions, the present application solves the problem of motion out of control caused by the split between control instructions and safety strategies. In the event of ambiguous operator intent or a sudden increase in environmental complexity, the system can automatically tighten safety constraint conditions to avoid collision accidents caused by incorrect instructions. In a low-risk scenario, the constraint conditions are appropriately relaxed to improve the motion efficiency and flexibility of the robot. Through semantic-driven dynamic constraint construction, the safety strategy accurately matches the environmental characteristics, such as automatically increasing the force limit when identifying fragile objects. The combination of model predictive control and real-time modification ensures both forward-looking optimization of instruction generation and immediate compliance checking of safety boundaries.

[0159] The present application further proposes the following technical solutions, taking a robot controller as an example of the execution subject, referring to Figure 5 , comprising the following steps.

[0160] 501. Sending a target joint torque instruction to a joint-level controller of a target robot to drive each joint motor to generate a corresponding output torque;

[0161] 502. During motion execution, real-time acquisition of robot actual joint state data and end effector pose data, and real-time comparison with preset safety boundary conditions in the dynamic safety boundary strategy;

[0162] 503. When a trend of violating safety boundary conditions is monitored in the actual motion state, online modification of the target joint torque instruction based on a preset safety constraint optimization algorithm to generate a modified torque instruction that satisfies all safety boundary conditions;

[0163] 504. Real-time issuance of the modified torque instruction to the joint-level controller to realize safety closed-loop control of the robot, and saving the process data of this interaction for system parameter optimization after completion of the human-robot interaction task.

[0164] Specifically, a distributed joint control system architecture is adopted, each joint is equipped with an independent PID controller or impedance controller, receives the torque command issued by the upper decision system, and controls the joint motor to output accurate torque through high-precision current loop. In specific implementation, the actual joint angle and angular velocity are obtained through the joint encoder, the contact force information is obtained through the six-axis force / torque sensor installed at the end, and the actual pose of the end is calculated by using the forward kinematics of the robot. A real-time optimization algorithm based on quadratic programming is adopted, the safety boundary condition is converted into an inequality constraint, and the modified torque closest to the original command is solved under the condition of meeting all constraint conditions. For example, when the end velocity approaches the preset threshold, the joint torque command is automatically reduced through the optimization algorithm, so that the velocity does not exceed the safety limit. The millisecond-level update of the command is realized through the real-time Ethernet communication protocol (such as EtherCAT), so that the real-time performance of the control system is realized. After the human-computer interaction task is completed, the process data of this interaction is saved for system parameter optimization, including storing complete motion trajectory, torque command sequence, safety boundary trigger record and other data, and optimizing the dynamic model parameters and safety boundary threshold through offline analysis.

[0165] In the precise assembly task, when the robot controller detects that the end effector is about to contact the precise workpiece, the dynamic safety boundary strategy will narrow the position tolerance and the maximum contact force threshold. If it is found through real-time monitoring that the actual contact force tends to exceed the threshold, the safety constraint optimization algorithm will immediately modify the torque command to control the contact force within the safety range. At the same time, the system records the force control data of this contact process for optimizing the impedance control parameters of subsequent similar tasks.

[0166] Compared with traditional position control or fixed gain force control, the embodiment realizes adaptive force control in a dynamic environment through real-time safety monitoring and online optimization and correction. Especially in unstructured task scenarios, it can effectively avoid over-force collision caused by model errors or environmental uncertainties, and significantly improve the safety and accuracy of human-robot collaboration.

[0167] All the optional technical solutions described above can be combined to form optional embodiments of the present application, which will not be described here.

[0168] Figure 6 is a structural schematic diagram of a human-computer interaction system based on an intelligent sensor provided by an embodiment of the present application, referring to Figure 6 The system comprises:

[0169] The acquisition module 601 is configured to acquire original interaction data between the operator and the target robot and ontology state sensor data of the target robot by the intelligent sensor in response to human-robot interaction operation of the operator on the target robot, the original interaction data comprising haptic pressure data and electromyographic signal sequences, the haptic pressure data being used to represent a pressure distribution pattern of a contact surface of the human-robot interaction, and the electromyographic signal sequences being used to represent a spatiotemporal pattern of muscle activity of the operator.

[0170] The determination module 602 is configured to determine a multi-modal interaction feature and an interaction scene semantic atlas between the operator and the target robot based on the original interaction data, a robot dynamics model of the target robot, and the ontology state sensor data.

[0171] The generation module 603 is configured to generate a target joint torque instruction and a dynamic safety boundary strategy of the target robot based on the multi-modal interaction feature, the interaction scene semantic atlas, and real-time quality reliability of the original interaction data.

[0172] The control module 604 is configured to control the target robot based on the target joint torque instruction and the dynamic safety boundary strategy, so that the target robot completes a task indicated by the human-robot interaction operation.

[0173] It should be noted that: the human-robot interaction system based on the intelligent sensor provided in the above embodiment is only exemplified by the division of the above functional modules during model warning, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the robot controller is divided into different functional modules to complete all or part of the functions described above. In addition, the human-robot interaction system based on the intelligent sensor and the human-robot interaction method based on the intelligent sensor provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0174] Figure 7 is a structural schematic diagram of a robot controller provided by the embodiment of the application. The robot controller 700 can have great differences due to different configurations or performances, and can include one or more processors (Central Processing Units, CPU) 701 and one or more memories 702. The one or more memories 702 store at least one computer program, which is loaded and executed by the one or more processors 701 to implement the method provided by each method embodiment described above. Of course, the robot controller 700 can also have a wired or wireless network interface, a keyboard, and an input and output interface, and other components for realizing the functions of the device, which will not be repeated here.

[0175] In an example embodiment, a computer readable storage medium, such as a memory including a computer program, is also provided, the computer program being executable by a processor to implement one of the above-mentioned human-computer interaction methods based on intelligent sensors. For example, the computer readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0176] In an example embodiment, a computer program product or computer program is also provided, the computer program product or computer program including program code stored in a computer readable storage medium, the program code being read by a processor of a robot controller from the computer readable storage medium, the processor executing the program code to cause the robot controller to implement one of the above-mentioned human-computer interaction methods based on intelligent sensors.

[0177] In some embodiments, the computer program related to the embodiments of the present application can be deployed to be executed on a robot controller, or on multiple robot controllers located at one site, or on multiple robot controllers distributed at multiple sites and interconnected through a communication network, which can constitute a blockchain system.

[0178] It can be understood by those skilled in the art that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing related hardware, which can be stored in a computer readable storage medium, such as a Read-Only Memory, a magnetic disk or an optical disk, etc.

[0179] The above is only an optional embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A human-machine interaction method based on intelligent sensors, characterized in that, The method comprises: In response to human-robot interaction operation of an operator on a target robot, acquiring original interaction data between the operator and the target robot and body state sensor data of the target robot through intelligent sensors, the original interaction data comprising haptic pressure data for representing pressure distribution patterns of a contact surface of the human-robot interaction and electromyographic signal sequences for representing spatiotemporal patterns of muscle activity of the operator; Based on the original interaction data, a robot dynamics model of the target robot and the body state sensor data, determining multimodal interaction features between the operator and the target robot and an interaction scene semantic graph, the interaction scene semantic graph being graph structure data containing object attributes and spatial relationships and used to provide environmental constraints for trajectory planning; Based on the multimodal interaction features, the interaction scene semantic graph and real-time quality reliability of the original interaction data, generating target joint torque instructions of the target robot and a dynamic safety boundary strategy, the dynamic safety boundary strategy being a safety constraint condition that is self-adaptively adjusted according to an interaction state, including but not limited to speed limits, acceleration limits, force limits and obstacle avoidance constraints; Based on the target joint torque instructions and the dynamic safety boundary strategy, controlling the target robot to complete a task indicated by the human-robot interaction operation.

2. The method of claim 1, wherein, The determination of the multimodal interaction features between the operator and the target robot and the interaction scene semantic graph based on the original interaction data, the robot dynamics model of the target robot and the body state sensor data comprises: Based on the robot dynamics model, performing quality evaluation and spatiotemporal registration on the original interaction data to obtain spatiotemporally synchronized multimodal data streams and real-time quality reliability of the original interaction data, the real-time quality reliability comprising haptic reliability factors and electromyographic reliability factors; Based on the multimodal data streams, the real-time quality reliability and the body state sensor data, determining the multimodal interaction features between the operator and the target robot and the interaction scene semantic graph.

3. The method of claim 2, wherein, The quality evaluation and spatiotemporal registration on the original interaction data based on the robot dynamics model to obtain spatiotemporally synchronized multimodal data streams and real-time quality reliability of the original interaction data comprises: Extracting spatial distribution features and time sequence features of the haptic pressure data; based on the spatial distribution features and the time sequence features, determining the haptic reliability factors; Extracting frequency domain power spectrum features and time domain amplitude statistical features of the electromyographic signal sequences; based on the frequency domain power spectrum features and the time domain amplitude statistical features, determining the electromyographic reliability factors; Performing spatiotemporal synchronization on the haptic pressure data, the electromyographic signal sequences and the body state sensor data to obtain the multimodal data streams.

4. The method of claim 2, wherein, The multi-modal interaction features and the interaction scene semantic graph between the operator and the target robot are determined based on the multi-modal data stream, the real-time quality credibility and the ontology state sensor data, and the determination includes: Parallel feature extraction is performed on the ontology state sensor data and the multi-modal data stream to obtain a basic motion feature set, a basic haptic feature set and a basic electromyography feature set; Dynamic weight allocation and feature fusion are performed on the basic haptic feature set and the basic electromyography feature set based on the haptic credibility factor and the electromyography credibility factor to obtain a haptic feature vector and an electromyography feature vector; The haptic feature vector, the electromyography feature vector and the basic motion feature set are spatiotemporally aligned and spliced to obtain the multi-modal interaction features; The multi-modal interaction features are input into a scene analysis model, and semantic analysis of the multi-modal interaction features is performed by the scene analysis model to obtain target structured data, the target structured data including object attributes, spatial relationships and task context information; Based on the target structured data, the interaction scene semantic graph including semantic nodes and relationship edges is constructed.

5. The method of claim 1, wherein, The target joint torque instruction and the dynamic safety boundary strategy of the target robot are generated based on the multi-modal interaction features, the interaction scene semantic graph and the real-time quality credibility of the original interaction data, and the generation includes: Based on the multi-modal interaction features, the interaction scene semantic graph and the real-time quality credibility, an intention probability distribution of the operator, a robot expected motion trajectory envelope and an interaction risk level are determined, the intention probability distribution being a diversified probability estimation of the operator's intention; Based on the intention probability distribution, the robot expected motion trajectory envelope, the interaction risk level and the interaction scene semantic graph, the target joint torque instruction and the dynamic safety boundary strategy of the target robot are generated.

6. The method of claim 5, wherein, The intention probability distribution of the operator, the robot expected motion trajectory envelope and the interaction risk level are determined based on the multi-modal interaction features, the interaction scene semantic graph and the real-time quality credibility, and the determination includes: The multi-modal interaction features are input into a spatiotemporal attention neural network to extract high-dimensional interaction features with spatiotemporal correlation, and attention weights of the spatiotemporal attention neural network are dynamically adjusted according to the real-time quality credibility; Based on the high-dimensional interaction features and the interaction scene semantic graph, probabilities of a plurality of preset intention categories are determined to obtain the intention probability distribution; Based on the intention probability distribution and the high-dimensional interaction features, a future motion trajectory of the target robot is determined; based on the spatial constraint relationship in the interaction scene semantic graph and the future motion trajectory, the robot expected motion trajectory envelope is determined; Based on the intention probability distribution, the robot expected motion trajectory envelope and the interaction scene semantic graph, risk probabilities of different interaction strategies are determined, and based on the risk probabilities of different interaction strategies, the interaction risk level is determined.

7. The method of claim 6, wherein, The determining, based on the intention probability distribution and the high-dimensional interaction feature, of the future motion trajectory of the target robot comprises: determining, based on the intention probability distribution, a plurality of candidate task objectives that the target robot needs to perform and a task objective weight of each candidate task objective; generating a plurality of predicted motion trajectories of the target robot based on the high-dimensional interaction feature and the task objective weight; based on the dynamics constraint and the kinematics constraint of the target robot, performing feasibility verification and optimization on the plurality of predicted motion trajectories to obtain a set of motion trajectories that conform to physical laws; selecting a target trajectory from the set of motion trajectories and adding a confidence boundary to the target trajectory according to the uncertainty of the intention probability distribution to obtain the future motion trajectory.

8. The method of claim 6, wherein, The determining, based on the intention probability distribution, the robot expected motion trajectory envelope, and the interaction scene semantic atlas, of the risk probability of different interaction strategies comprises: based on the intention probability distribution, querying, from a task knowledge base, a standard action accuracy requirement and a task criticality level corresponding to a candidate intention with the highest probability; based on the interaction scene semantic atlas, extracting a static obstacle spatial distribution, a dynamic obstacle motion trajectory, and a safe operation region boundary in the environment as basic environmental risk factors; based on the robot expected motion trajectory envelope, determining a minimum distance between each point of the trajectory and an obstacle, a speed peak value at a key point of the trajectory, and an acceleration change rate to obtain a motion trajectory risk indicator; based on the standard action accuracy requirement, the task criticality level, the basic environmental risk factors, and the motion trajectory risk indicator, determining the risk probability of different interaction strategies through a risk weighting evaluation model.

9. The method of claim 8, wherein, The determining, based on the standard action accuracy requirement, the task criticality level, the basic environmental risk factors, and the motion trajectory risk indicator, of the risk probability of different interaction strategies through a risk weighting evaluation model comprises: based on the standard action accuracy requirement, determining a trajectory deviation tolerance threshold; based on the task criticality level, determining a risk influence weight coefficient; based on the static obstacle spatial distribution and the dynamic obstacle motion trajectory in the basic environmental risk factors, determining an environmental complexity indicator; based on the environmental complexity indicator and the minimum distance, the speed peak value, and the acceleration change rate in the motion trajectory risk indicator, determining an environment-trajectory interaction risk value; inputting the trajectory deviation tolerance threshold, the risk influence weight coefficient, and the environment-trajectory interaction risk value into a fuzzy reasoning system to perform comprehensive risk evaluation through a preset fuzzy rule base, and outputting an initial risk probability of different interaction strategies; based on the real-time quality credibility, calibrating the confidence of the initial risk probability to obtain the risk probability of different interaction strategies.

10. A human-machine interaction system based on smart sensors, characterized in that, The system comprises: An acquisition module is configured to acquire original interaction data between an operator and a target robot and body state sensor data of the target robot through intelligent sensors in response to human-robot interaction operation of the operator on the target robot, the original interaction data including haptic pressure data and electromyography signal sequence, the haptic pressure data being used to represent pressure distribution patterns of a contact surface of the human-robot interaction, and the electromyography signal sequence being used to represent spatiotemporal patterns of muscle activities of the operator; A determination module is configured to determine multi-modal interaction features between the operator and the target robot and an interaction scene semantic graph based on the original interaction data, a robot dynamics model of the target robot, and the body state sensor data, the interaction scene semantic graph being graph structure data containing object attributes and spatial relationships and being used to provide environmental constraint conditions for trajectory planning; A generation module is configured to generate a target joint torque instruction of the target robot and a dynamic safety boundary strategy based on the multi-modal interaction features, the interaction scene semantic graph, and real-time quality reliability of the original interaction data, the dynamic safety boundary strategy being a safety constraint condition that is self-adaptively adjusted according to an interaction state, including but not limited to a speed limit, an acceleration limit, a force limit, and an obstacle avoidance constraint; A control module is configured to control the target robot based on the target joint torque instruction and the dynamic safety boundary strategy, so that the target robot completes a task indicated by the human-robot interaction operation.

Images