Task planning system and method for intelligent robot with body based on multi-dimensional situation awareness

Abstract

The invention discloses a system and a method for task planning of an intelligent robot with a body based on multi-dimensional situation awareness, and particularly relates to the technical field of task planning of the intelligent robot with the body, space-time alignment is carried out on asynchronous heterogeneous data generated by a multi-modal sensor channel, and cross-modal space-time features are extracted through a cross-modal feature fusion network; a fusion situation matrix is generated, dynamic causal modeling is used to update association strength among the multi-modal data, an anti-factual reasoning engine is used to identify and trace abnormities, and an abnormities traceability result is output; dynamically adjusting the reliability weight of each sensing channel through a multi-modal credibility evaluation model by using the fusion situation matrix and an abnormal traceability result; and on the basis of the reliability weight, inputting the fusion situation matrix into a real robot dynamic model and a digital twin virtual model, executing collaborative predictive control, and starting an adaptive rule evolution mechanism when a safety score is lower than a threshold value, thereby solving the problem of fusion matrix distortion in dynamic obstacle avoidance and precise grabbing tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of embodied intelligent robot task planning. More specifically, the present invention relates to an embodied intelligent robot task planning system and method based on multi-dimensional situation awareness. Background Art

[0002] Embodied intelligent robots performing tasks in dynamic unstructured environments, such as industrial sorting robotic arms, disaster rescue robots, etc., need to rely on multi-modal sensors such as vision, touch, and lidar to perceive the environmental situation in real time and generate safe and continuous action instructions. However, the existing technologies have significant defects, mainly manifested in the lack of a collaborative processing mechanism for heterogeneous multi-modal signals, resulting in inaccurate perception and decision-making in dynamic physical interaction scenarios. Specifically, the traditional methods use linear interpolation or fixed-delay compensation to align multi-modal data, but they cannot eliminate the spatio-temporal misalignment between high-frequency tactile signals and low-frequency visual data. For example, when the tactile perception has contact residue, the vision has already determined that the obstacle has disappeared, resulting in a millisecond-level spatio-temporal deviation in the fused situation matrix in tasks such as dynamic obstacle avoidance and precise grasping.

[0003] In addition, the static confidence evaluation rules do not integrate physical field parameters and multi-modal semantic associations, and cannot quantify the reliability differences of sensors in dynamic interactions, resulting in the task planner generating oscillating instructions or high-risk actions when signals conflict, such as frequent starts and stops in narrow passages or exceeding the grasping force limit and damaging the target. Such problems are particularly prominent in scenarios with frequent environmental disturbances and significant sensor heterogeneity, seriously restricting the autonomy, efficiency, and safety of robots and constituting a technical bottleneck that urgently needs to be broken through. Summary of the Invention

[0004] To overcome the above-mentioned defects of the prior art, the present invention provides an embodied intelligent robot task planning system and method based on multi-dimensional situation awareness, which solves the problems raised in the above background art by aligning the spatio-temporal misalignment of multi-modal data, improving the accuracy of sensor reliability evaluation in dynamic scenarios, and ensuring the safety of task planning in complex environments.

[0005] To achieve the above object, the present invention provides the following technical solution: An embodied intelligent robot task planning method based on multi-dimensional situation awareness, including the following steps:

[0006] Step 1: Use a spiking neural network to perform spatio-temporal alignment on asynchronous heterogeneous data generated by multi-modal sensor channels, and extract cross-modal spatio-temporal features through a cross-modal feature fusion network to generate a fused situation matrix, which is used as the input for Step 2 and Step 3;

[0007] Step 2: Based on the fused situation matrix generated in Step 1, use dynamic causal modeling to update the association strength between multi-modal data, and identify and trace the source of anomalies through a counterfactual reasoning engine, and output the anomaly tracing result to Step 3;

[0008] Step 3: Using the fused situation matrix generated in Step 1 and the abnormal traceability result output in Step 2, process the dynamic field strength distribution map and real-time physical field parameters through a graph neural network, dynamically adjust the reliability weights of each sensing channel through a multi-modal credibility evaluation model, and output to Step 4;

[0009] Step 4: Based on the reliability weights, input the fused situation matrix into the real robot dynamics model and the digital twin virtual model, execute cooperative predictive control, and start the adaptive rule evolution mechanism when the safety score is lower than the threshold to ensure the safety of the decision-making.

[0010] Preferably, the asynchronous heterogeneous data at least includes asynchronous data streams of a tactile sensing channel, an optical sensing channel, and a three-dimensional ranging sensing channel. The spiking neural network uses a frequency-domain adaptive interpolation algorithm to achieve data synchronization and dynamically adjusts the interpolation accuracy according to the target motion speed; the cross-modal feature fusion network includes a spatio-temporal attention mechanism and a recursive feature extraction unit, establishes the feature correlation between different sensing channels through a cross-attention mechanism, and generates a fused situation matrix containing multi-modal semantic features in combination with temporal dependence modeling. For example, during the operation of the frequency-domain adaptive interpolation algorithm, the interpolation accuracy is dynamically adjusted to match the change in the motion speed of the object in the scene; the self-attention mechanism is used to quantify the correlation between tactile sensor data and visual sensor data, and the spatial topological features of lidar sensor data are fused under the cross-attention mechanism to generate a fused situation matrix to ensure high-precision cross-modal feature extraction in dynamic scene recognition.

[0011] Explanation: The multi-modal sensor channels adopt a distributed array layout, including piezoresistive tactile sensors, depth cameras, and lidar sensors; each sensor achieves time synchronization through a hardware trigger signal, and the trigger signal deviation is less than a preset threshold; the data interface adopts the Robot Operating System protocol, and the message format follows the point cloud data standard;

[0012] Preferably, the asynchronous heterogeneous data includes low-frequency sensor data and high-frequency sensor data. The frequency-domain adaptive interpolation algorithm includes the following steps: performing frequency-domain decomposition on the low-frequency sensor data to extract the main frequency components; encoding the high-frequency sensor data into a pulse sequence, and dynamically setting the pulse firing threshold according to the sensor type, and the firing frequency is non-linearly correlated with the target motion speed; adjusting the interpolation window length based on the motion speed, and using spline interpolation to fill in the missing frames of the low-frequency data; verifying the timestamp alignment error of the multi-channel data, and triggering synchronous calibration when the error exceeds the tolerance.

[0013] Preferably, dynamic causal modeling adopts a Bayesian weight update algorithm to reconstruct the causal relationship between multi-modal sensor data in real time, and triggers anomaly detection when the causal correlation degree between sensor data deviates from the preset dynamic error threshold; for example, when the causal edge weight deviation between tactile sensor data and lidar sensor data exceeds the preset dynamic threshold, anomaly recognition is triggered; the counterfactual reasoning engine simulates the data distribution under abnormal conditions through a generative adversarial network, and outputs an anomaly traceability result containing anomaly type labels and propagation paths in combination with the historical anomaly pattern database.

[0014] Preferably, the graph neural network is based on the dynamic field strength distribution map generated by the simultaneous localization and mapping technology. The multi-modal credibility evaluation model integrates an environmental physical field analysis unit and a sensing error compensation unit, where: the environmental physical field analysis unit constructs a dynamic field strength distribution map through electromagnetic field gradient monitoring and acoustic perturbation detection; for example, fuses the electromagnetic field intensity gradient data and acoustic Doppler effect characteristics monitored in real time to construct a dynamic field strength distribution map; the sensing error compensation unit adopts a multi-physical coupling correction algorithm, generates a sensing data correction matrix based on real-time physical field parameters, and calculates the reliability weights of each sensing channel in combination with the semantic segmentation confidence index.

[0015] Preferably, the safety score is obtained as follows: the digital twin virtual model is constructed through a physics engine, including a state synchronization, environment mapping and delay compensation mechanism, and receives real embodied robot joint state data in real time and generates a three-dimensional occupancy map of the virtual environment; uses a trajectory similarity metric algorithm to calculate the spatial deviation between the real trajectory point sequence and the virtual trajectory point sequence, and generates a trajectory deviation degree through a normalization mapping function; constructs an obstacle occupancy probability model based on the lidar data corrected by multi-physical coupling, fuses the visual sensor confidence and the tactile sensor pressure gradient characteristics for dynamic correction, and calculates the collision risk probability through the three-dimensional risk field superposition; the safety score is generated by weighted calculation of the trajectory deviation degree and the collision risk probability.

[0016] Preferably, the process of calculating the reliability weight includes:

[0017] Let i represent the number index of the sensor channel, and N represent the total number of sensor channels;

[0018] Spatio-temporal consistency quantification: Based on multi-modal consistency test, calculate the spatio-temporal consistency score of the i-th sensing channel where v i is the real-time measurement vector of the i-th sensing channel, is the cross-modal prediction value, the prediction value generated by other sensing channel data through the Transformer-LSTM hybrid network; cos(·) is the vector cosine similarity calculation function, used to quantify the direction consistency between real-time data and the prediction value;

[0019] Environmental interference intensity quantification: Calculate the environmental interference intensity of the i-th sensing channel through the following formula:

[0020] where is the magnitude of the electromagnetic field gradient, obtained by differential calculation of a triaxial magnetic field sensor, and P(f) is the sound pressure power spectral density, extracted by a fast Fourier transform from an acoustic sensor array; fL is the lower integration frequency, representing the lowest effective frequency threshold for the integral calculation of the sound pressure power spectral density, and fH is the upper integration frequency, representing the highest effective frequency threshold for the integral calculation of the sound pressure power spectral density; α and β are sensitivity coefficients related to the sensing type, which are proportionality factors obtained by fitting the interference-error curve in the sensor calibration experiment; for example, in an actual industrial scenario, fL can be set to 50 Hz (to avoid power frequency interference), and fH can be set to 15 kHz (higher than the upper limit of human ear hearing);

[0021] Task relationship metric: According to the safety requirement level L c (dynamically assigned according to the current operation type of the robot, discrete values: {1: non-safety critical, 2: collaborative operation, 3: high-risk operation}) and the operation accuracy requirement R p , with a value range of 0.1 to 1.0, calculate the task criticality factor K c =γL c +(1 - γ)R p ; γ represents the task priority;

[0022] Dynamic weight synthesis: Denote it as Calculate the reliability weight Wi of the i-th sensing channel through a weighted fusion function i , where ε is a small constant to prevent division by zero errors;

[0023]

[0024] where ε is a small positive number, and in the embodiments of the present invention, its value range is 0.01 - 0.0001, used to prevent calculation overflow caused by too small a denominator.

[0025] Preferably, collaborative predictive control calculates the output deviation between the physical model and the digital twin virtual model through a trajectory similarity metric algorithm, and activates safety constraint optimization when the output deviation exceeds the preset trajectory fault tolerance threshold; the adaptive rule evolution mechanism includes rule credibility evaluation and logical structure optimization, verifies the effectiveness of the rules in the simulation environment through a reinforcement learning mechanism, and dynamically updates the constraint conditions and optimization objectives in the decision rule base; the adaptive rule evolution mechanism includes the following steps: constructing an abnormal pattern knowledge graph: based on historical abnormal events of tactile, visual, and lidar sensor data, extracting spatio-temporal correlation features and generating the topological structure of the abnormal propagation path; incremental optimization of the rule base: matching the similarity between the real-time abnormal scenario and the historical knowledge graph through a contrast learning algorithm, dynamically updating the constraint conditions in the decision rule base, and preferentially retaining the rules whose effectiveness has been verified; unsupervised verification mechanism: simulating extreme scenarios in the digital twin virtual model to verify the stability of the newly added rules and removing redundant rules that cause the safety score to decrease; for example, when the similarity with the historical tactile abnormal pattern is detected to exceed 85%, the corresponding decision rule is automatically loaded; adapting the abnormal handling rules of the industrial assembly scenario to the warehousing handling scenario through transfer learning.

[0026] Preferably, the multi-physical coupling correction algorithm includes an interference source localization unit and a dynamic weight redistribution unit; when the detected electromagnetic field strength gradient exceeds the device anti-interference threshold or the acoustic perturbation frequency enters the sensitive frequency band of the sensor, the interference source localization unit is activated for error compensation; the interference source localization unit includes: based on the three-dimensional vector data of the electromagnetic field gradient, establishing a probability model of the spatial distribution of the interference source in combination with the Lorentz force equation, and calculating the coordinates of the maximum likelihood electromagnetic interference source; using the generalized cross-correlation algorithm to process the time-delay estimation of the acoustic sensing array, and constructing a confidence interval for the location of the acoustic interference source through the time difference of arrival of sound waves; the dynamic weight redistribution unit, when the Frobenius norm of the perturbation coefficient matrix composed of the environmental physical field parameters exceeds the adaptive threshold, triggers a weight optimization algorithm based on the Lyapunov stability theory to redistribute the reliability weights of the multi-modal sensor data.

[0027] Preferably, the dynamic evolution of the symbolic rules is based on historical verification data and simulation scenarios. By self-supervised learning to analyze the historical abnormal patterns of tactile sensor data, visual sensor data, and lidar sensor data, the decision rule base is automatically optimized to improve the task planning ability of the robot in unforeseen scenarios.

[0028] Preferably, the multi-physical field coupling compensation mechanism monitors the temperature change rate and electromagnetic interference intensity in real time. When the temperature change rate or electromagnetic interference intensity exceeds the preset corresponding threshold, an online calibration algorithm is triggered to correct the tactile sensor data, visual sensor data, and lidar sensor data, suppressing the influence of environmental noise on the reliability weight distribution and ensuring the accuracy of weight adjustment.

[0029] To achieve the above object, the present invention provides the following technical solutions: An embodied intelligent robot task planning system based on multi-dimensional situation awareness, including:

[0030] A spatio-temporal alignment module, which performs frequency-domain adaptive interpolation on asynchronous heterogeneous data of multi-modal sensor channels (the sensor device includes at least tactile, visual, and lidar) through a spiking neural network, eliminates spatio-temporal misalignment of high-frequency / low-frequency signals, outputs synchronized multi-modal data, and outputs it to the feature fusion module;

[0031] A feature fusion module, which uses a cross-modal feature fusion network (including spatio-temporal attention mechanism and recurrent unit) to extract spatio-temporal correlation features of multi-modal data, generates a fusion situation matrix containing semantic information (including multi-modal features such as tactile pressure gradient, visual target recognition, and lidar spatial topology), and transmits it to the causal modeling module and the trust evaluation module;

[0032] A causal modeling module, based on dynamic causal modeling (Bayesian weight update algorithm), analyzes the causal association strength between multi-modal data in real time, detects sensor data conflicts (such as the contradiction between tactile residue and the disappearance of visual obstacles), outputs an updated sensor causal association map (including abnormal trigger marks), and transmits it to the abnormal traceability module;

[0033] An abnormal traceability module, which simulates the abnormal data distribution through a counterfactual reasoning engine (constructs a counterfactual reasoning engine based on an adversarial network and a historical abnormal database), identifies the abnormal type (such as electromagnetic interference, sensor failure), generates an interpretable abnormal traceability result (including the propagation path and type label), outputs the abnormal type label and the propagation path report, and transmits it to the trust evaluation module;

[0034] A trust evaluation module, which combines the dynamic field strength distribution map (electromagnetic gradient, acoustic perturbation) and the abnormal traceability result, calculates the spatio-temporal consistency score and the environmental interference strength of each sensor channel through a multi-modal credibility evaluation model, dynamically adjusts the reliability weight, outputs a reliability weight matrix of the sensor channel (such as a decrease in tactile weight and an increase in lidar weight), and transmits it to the collaborative control module;

[0035] A collaborative control module, based on the reliability weight and the fusion situation matrix, jointly performs collaborative predictive control (trajectory similarity metric + collision risk calculation) on the real robot dynamics model and the digital twin virtual model, generates a safety score and outputs an action instruction; outputs robot joint control instructions and a safety score, and triggers the rule evolution module when the safety score is lower than the threshold;

[0036] The rule evolution module, through reinforcement learning and the anomaly pattern knowledge graph, verifies and optimizes the decision rule library in the digital twin environment (such as adding obstacle avoidance constraints and adjusting the grasping force threshold), dynamically updates the rules to adapt to extreme scenarios, and outputs the updated safety decision rule library (such as emergency stop rules and dynamic obstacle avoidance strategies). The optimized rule library is fed back to the collaborative control module to form a closed-loop iteration.

[0037] Technical effects and advantages of the present invention:

[0038] (1) The method for embodied intelligent robot task planning based on multi-dimensional situation awareness proposed by the present invention realizes the spatio-temporal alignment of multi-modal data through a spiking neural network and a frequency-domain adaptive interpolation algorithm, and combines the spatio-temporal attention mechanism of a cross-modal feature fusion network to dynamically match the asynchronous data streams of touch, vision, and lidar, eliminating the millisecond-level spatio-temporal deviation between high-frequency touch signals and low-frequency visual data; by adjusting the spline interpolation window with adaptive movement speed, the spatio-temporal synchronization accuracy of multi-modal features is maintained in the object variable-speed scenario, solving the problem of fusion matrix distortion caused by spatio-temporal misalignment in dynamic obstacle avoidance and precise grasping tasks, and enhancing the situation awareness ability in complex scenarios.

[0039] (2) The method for embodied intelligent robot task planning based on multi-dimensional situation awareness proposed by the present invention, through dynamic causal modeling and a multi-modal credibility evaluation model, reconstructs the correlation strength of sensor data in real time and dynamically adjusts the weights by fusing physical field parameters. The counterfactual reasoning engine based on the Bayesian network can trace the abnormal propagation path, and constructs a dynamic field strength distribution map by combining the electromagnetic field gradient and acoustic perturbation parameters. In the sensor conflict scenario, the weight allocation is optimized through the Lyapunov stability theory, effectively solving the problems of over-limit grasping force and inaccurate trajectory planning caused by environmental disturbances in traditional methods. Description of the Drawings

[0040] Figure 1 It is a flowchart of the intelligent robot task planning method of the present invention.

[0041] Figure 2 It is a block diagram of the structure of the robot task planning system of the present invention. Detailed Embodiments

[0042] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully communicated to those skilled in the art.

[0043] At the same time, it should be understood that, for the sake of convenience of description, the sizes of the various parts shown in the drawings are not drawn in actual proportional relationships.

[0044] The following description of at least one exemplary embodiment is merely illustrative and is in no way a limitation on the present application or its application or use.

[0045] Techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification.

[0046] In the prior art, embodied intelligent robots operating in dynamic unstructured environments face challenges in the collaborative processing of multi-modal sensor data; traditional methods rely on fixed-frequency interpolation to achieve data synchronization, making it difficult to eliminate the spatio-temporal misalignment between high-frequency tactile signals and low-frequency visual data. When the robotic arm touches an object, the tactile sensor has detected a pressure change while the visual system has not yet updated the obstacle position information, resulting in a millisecond-level deviation in the fusion situation matrix; static confidence assessment rules do not consider physical field disturbance factors such as electromagnetic interference or temperature changes, and are prone to generating oscillating instructions or high-risk actions when sensor signals conflict, such as the robotic arm repeatedly starting and stopping in a narrow passage or applying a grasping force beyond the target's tolerance range.

[0047] To solve the above problems, regarding the spatio-temporal misalignment problem of multi-modal data, the inventors realized that traditional linear interpolation cannot adapt to scenarios with changing motion speeds, and an interpolation mechanism non-linearly associated with the target motion speed needs to be established; by analyzing the relationship between the pulse firing characteristics of tactile sensors and the object motion speed, an adaptive interpolation method based on frequency-domain decomposition was proposed. In terms of sensor reliability assessment, it was found that the fixed-threshold weighting rule does not fuse environmental physical field parameters, and a dynamic association model of multi-modal data and electromagnetic field gradients needs to be constructed. Combining the virtual-real linkage mechanism, a digital twin virtual model and a real robot collaborative prediction control architecture were designed to achieve dynamic verification of safe decision-making.

[0048] Example 1, referring to Figure 1 the flowchart of the intelligent robot task planning method, the present invention provides a Figure 1 task planning method for an embodied intelligent robot based on multi-dimensional situation awareness as shown in

[0049] Step 1: Use a pulsed neural network to perform spatio-temporal alignment on asynchronous heterogeneous data generated by multi-modal sensor channels, and extract cross-modal spatio-temporal features through a cross-modal feature fusion network to generate a fusion situation matrix, which is used as the input for Step 2 and Step 3;

[0050] Step 2: Based on the fusion situation matrix generated in Step 1, use dynamic causal modeling to update the association strength between multi-modal data, and identify and trace the source of anomalies through a counterfactual reasoning engine, and output the anomaly tracing result to Step 3;

[0051] Step 3: Using the fused situation matrix generated in Step 1 and the abnormal traceability result output in Step 2, process the dynamic field strength distribution map and real-time physical field parameters through a graph neural network, dynamically adjust the reliability weights of each sensing channel through a multi-modal credibility evaluation model, and output to Step 4;

[0052] Step 4: Based on the reliability weights output in Step 3, input the fused situation matrix generated in Step 1 into the real robot dynamics model and the digital twin virtual model, execute cooperative predictive control, and start the adaptive rule evolution mechanism when the safety score is lower than the threshold to ensure the safety of the decision-making.

[0053] In the embodiments of the present invention, it needs to be further explained that the asynchronous heterogeneous data at least includes asynchronous data streams of a tactile sensing channel, an optical sensing channel, and a three-dimensional ranging sensing channel. The spiking neural network uses a frequency-domain adaptive interpolation algorithm to achieve data synchronization, and dynamically adjusts the interpolation accuracy according to the target motion speed; the cross-modal feature fusion network refers to a deep learning architecture that integrates the spatio-temporal features of different modal sensors, including a spatio-temporal attention mechanism and a recursive feature extraction unit, establishes the feature correlation degree between different sensing channels through a cross-attention mechanism, and generates a fused situation matrix containing multi-modal semantic features in combination with temporal dependence modeling. For example, during the operation of the frequency-domain adaptive interpolation algorithm, the interpolation accuracy is dynamically adjusted to match the change in the motion speed of the object in the scene; the self-attention mechanism is used to quantify the correlation between the tactile sensor data and the visual sensor data, and the spatial topological features of the lidar sensor data are fused under the cross-attention mechanism to generate a fused situation matrix, ensuring high-precision cross-modal feature extraction in dynamic scene recognition.

[0054] Explanation: The spiking neural network refers to a neural network that processes asynchronous event data using the pulse transmission mechanism of biological neurons. Specifically, it can be implemented using the third-generation neural network model with time encoding characteristics. Its pulse firing frequency is non-linearly related to the input signal strength, and it is suitable for processing high-frequency pulse signals of tactile sensors.

[0055] Explanation: The multi-modal sensor channels adopt a distributed array layout, including piezoresistive tactile sensors, depth cameras, and lidar sensors; each sensor achieves time synchronization through a hardware trigger signal, and the trigger signal deviation is less than a preset threshold; the data interface uses the robot operating system protocol, and the message format follows the point cloud data standard;

[0056] In the embodiments of the present invention, it needs to be further explained that asynchronous heterogeneous data includes low-frequency sensor data and high-frequency sensor data. The frequency-domain adaptive interpolation algorithm includes the following steps: performing frequency-domain decomposition on the low-frequency sensor data to extract the main frequency components; encoding the high-frequency sensor data into a pulse sequence, where the pulse firing threshold is dynamically set according to the sensor type, and the firing frequency is non-linearly correlated with the target movement speed; adjusting the interpolation window length based on the movement speed, and using spline interpolation to fill in the missing frames of the low-frequency data; verifying the timestamp alignment error of the multi-channel data, and triggering synchronous calibration when the error exceeds the tolerance.

[0057] In the embodiments of the present invention, it needs to be further explained that dynamic causal modeling uses the Bayesian weight update algorithm to reconstruct the causal relationship between multi-modal sensor data in real time. When the causal correlation degree between sensor data deviates from the preset dynamic error threshold, anomaly detection is triggered; for example, when the weight deviation of the causal edge between tactile sensor data and lidar sensor data exceeds the preset dynamic threshold, anomaly recognition is triggered; the counterfactual reasoning engine simulates the data distribution under abnormal conditions through a generative adversarial network, and outputs an anomaly traceability result including anomaly type labels and propagation paths in combination with the historical anomaly pattern database.

[0058] In the embodiments of the present invention, it needs to be further explained that the graph neural network is based on the dynamic field strength distribution map generated by the simultaneous localization and mapping technology. The multi-modal credibility evaluation model integrates an environmental physical field analysis unit and a sensing error compensation unit, where: the environmental physical field analysis unit constructs a dynamic field strength distribution map through electromagnetic field gradient monitoring and acoustic perturbation detection; for example, fusing the real-time monitored electromagnetic field intensity gradient data and acoustic Doppler effect characteristics to construct a dynamic field strength distribution map; the sensing error compensation unit uses a multi-physical coupling correction algorithm to generate a sensing data correction matrix based on real-time physical field parameters, and calculates the reliability weights of each sensing channel in combination with the semantic segmentation confidence index.

[0059] In the embodiments of the present invention, it needs to be further explained that the safety score is obtained in the following way:

[0060] The digital twin virtual model is constructed through a physics engine, including state synchronization, environment mapping, and delay compensation mechanisms, and receives real embodied robot joint state data in real time and generates a three-dimensional occupancy map of the virtual environment;

[0061] The trajectory similarity metric algorithm is used to calculate the spatial deviation between the real trajectory point sequence and the virtual trajectory point sequence, and the trajectory deviation degree is generated through a normalization mapping function;

[0062] Based on the lidar data corrected by multi-physical coupling, an obstacle occupancy probability model is constructed, and it is dynamically corrected by fusing the confidence of the vision sensor and the pressure gradient characteristics of the tactile sensor, and the collision risk probability is calculated through the superposition of the three-dimensional risk field.

[0063] The safety score is generated by weighted calculation of the trajectory deviation degree and the collision risk probability.

[0064] In the embodiments of the present invention, it needs to be further explained that the cooperative predictive control calculates the output deviation between the physical model and the digital twin virtual model through a trajectory similarity measurement algorithm, and activates safety constraint optimization when the output deviation exceeds the preset trajectory fault tolerance threshold; the adaptive rule evolution mechanism includes rule credibility evaluation and logic structure optimization, verifies the effectiveness of the rules in the simulation environment through a reinforcement learning mechanism, and dynamically updates the constraint conditions and optimization objectives in the decision rule library; the adaptive rule evolution mechanism includes the following steps:

[0065] Construct an abnormal pattern knowledge graph: Based on historical abnormal events of tactile, visual, and lidar sensor data, extract spatio-temporal correlation features and generate the topological structure of the abnormal propagation path;

[0066] Incremental optimization of the rule library: Match the similarity between the real-time abnormal scenario and the historical knowledge graph through a contrast learning algorithm, dynamically update the constraint conditions in the decision rule library, and preferentially retain the rules whose effectiveness has been verified;

[0067] Unsupervised verification mechanism: Simulate extreme scenarios in the digital twin virtual model to verify the stability of the newly added rules and eliminate redundant rules that cause the safety score to decrease; for example, when the similarity with the historical tactile abnormal pattern is detected to exceed 85%, automatically load the corresponding decision rule; Adapt the abnormal handling rules of the industrial assembly scenario to the warehousing handling scenario through transfer learning.

[0068] In the embodiments of the present invention, it needs to be further explained that the multi-physical coupling correction algorithm includes an interference source localization unit and a dynamic weight redistribution unit; when it is detected that the electromagnetic field strength gradient exceeds the device anti-interference threshold or the acoustic disturbance frequency enters the sensitive frequency band of the sensor, the interference source localization unit is activated for error compensation; the interference source localization unit includes: based on the three-dimensional vector data of the electromagnetic field gradient, combining the Lorentz force equation to establish a probability model of the spatial distribution of the interference source, and calculating the coordinates of the maximum likelihood electromagnetic interference source; using the generalized cross-correlation algorithm to process the time delay estimation of the acoustic sensing array, and constructing a confidence interval for the location of the acoustic interference source through the time difference of arrival of sound waves; the dynamic weight redistribution unit, when the Frobenius norm of the perturbation coefficient matrix composed of the environmental physical field parameters exceeds the adaptive threshold, triggers a weight optimization algorithm based on the Lyapunov stability theory to redistribute the reliability weights of the multi-modal sensor data. Explanation, the dynamic adjustment of the sensor weights is based on the multi-modal semantic segmentation results and real-time physical field parameters, where the multi-modal semantic segmentation results are generated by the fusion of visual sensor data and lidar sensor data, and the real-time physical field parameters include the electromagnetic field gradient, the sound pressure volatility, and the heat convection intensity; when the Frobenius norm of the perturbation coefficient matrix composed of the real-time physical field parameters exceeds the adaptive threshold of 0.5, a weight redistribution algorithm based on the Lyapunov stability theory is triggered to optimize the weight allocation of the multi-modal sensor data to adapt to complex scenarios.

[0069] In the embodiments of the present invention, it needs to be further explained that the dynamic evolution of the symbol rule is based on historical verification data and simulated scenarios, and automatically optimizes the decision rule library by analyzing the historical abnormal patterns of the tactile sensor data, visual sensor data, and lidar sensor data through self-supervised learning, so as to improve the task planning ability of the robot in unforeseen scenarios.

[0070] Among them, the dynamic evolution of symbol rules means that the decision rule base automatically adjusts its logical structure and constraint conditions according to environmental changes. Specifically, it can be implemented using a graph embedding algorithm based on historical anomaly patterns. By encoding the spatio-temporal correlation features of anomaly events, a rule evolution path is generated to solve the problem that the static rule base cannot adapt to new types of anomalies. Among them, historical verification data refers to a set of sensor data verified in actual scenarios. Specifically, a multi-modal time series database can be used to store historical anomaly event records of tactile, visual, and lidar sensor data, providing reliable data support for rule optimization. Among them, the simulation scenario refers to a virtual environment constructed through a digital twin virtual model, used to verify the effectiveness of newly added rules under extreme conditions. Among them, self-supervised learning refers to a feature extraction method that does not require manual annotation. Specifically, a contrast learning algorithm can be used to perform unsupervised feature alignment on multi-modal sensor data, and cross-modal anomaly correlation mining is achieved by maximizing the similarity of positive sample pairs, used to identify composite anomaly patterns. Among them, historical anomaly patterns refer to the combination of anomaly features that repeatedly appear in sensor data. Specifically, a spatio-temporal graph convolutional network can be used to extract the joint anomaly features of tactile pressure gradient changes and visual semantic segmentation results, used to capture the implicit correlation between sensor data. Among them, automatically optimizing the decision rule base means dynamically updating the logical constraints in the rule base. Specifically, an incremental rule mining algorithm can be used to match the similarity between real-time anomaly scenarios and historical knowledge graphs, and effective rules are selected and redundant rules are eliminated through reinforcement learning, used to improve the adaptability of the rule base.

[0071] Specifically, the spatio-temporal correlation anomalies between tactile pressure gradient features and visual semantic segmentation results are extracted through a self-supervised learning algorithm. For example, when the tactile sensor detects a sudden change in contact surface pressure and the visual sensor does not recognize an obstacle, the system automatically marks it as a potential anomaly event. During the rule base optimization process, a contrast learning algorithm is used to match the similarity between real-time anomaly features and nodes in the historical knowledge graph. When the matching degree exceeds the preset threshold, the rule evolution mechanism is triggered. For example, when a high similarity to historical tactile anomalies is detected, the corresponding force control rules are automatically loaded and the decision logic is adjusted. The optimized rule base is integrated into the original decision-making system through an incremental update mechanism to ensure that the robot can still generate continuous and reliable action instructions when encountering unforeseen sensor conflicts or environmental disturbances.

[0072] Compared with the prior art, traditional methods rely on a fixed rule base to handle sensor anomalies and cannot adapt to new composite anomaly patterns in dynamic environments. For example, the prior art uses manually preset thresholds to judge conflicts between visual and tactile data, which are prone to misjudgment when encountering sudden changes in material stiffness or electromagnetic interference. This solution can automatically mine the implicit associations between multi-modal data through self-supervised learning, and combine with a digital twin simulation verification mechanism to dynamically expand the processing capacity of the rule base. For example, in a disaster rescue scenario, when lidar data is missing due to dust interference, the system can automatically enable redundant decision rules and adjust the motion trajectory planning strategy based on the matching of historical tactile and visual composite anomaly patterns.

[0073] Through the above technical solution, this application realizes the dynamic optimization of the decision rule base and solves the problem of matching failure of static rules in unknown scenarios. By fusing multi-modal historical anomaly features and simulation scenario verification, the recognition accuracy of the robot for composite anomaly patterns is improved. The self-supervised learning mechanism is adopted to avoid the cost of manual annotation and ensure that the rule evolution process can autonomously adapt to environmental changes. For example, in an industrial sorting task, when the robotic arm touches a flexible object, resulting in a conflict between tactile and visual data, the system can quickly adjust the grasping force control rule based on historical similar anomalies to avoid object damage or grasping failure.

[0074] In the embodiments of the present invention, it needs to be further explained that the multi-physical field coupling compensation mechanism monitors the temperature change rate and electromagnetic interference intensity in real time. When the temperature change rate or electromagnetic interference intensity exceeds the preset corresponding threshold, an online calibration algorithm is triggered to correct the data of the tactile sensor, visual sensor, and lidar sensor, suppressing the influence of environmental noise on the reliability weight allocation and ensuring the accuracy of weight adjustment.

[0075] Among them, the temperature change rate refers to the change amplitude of the environmental temperature per unit time. Specifically, it can be realized by a thermocouple array or an infrared temperature sensor for real-time monitoring. Its function is to capture the interference of thermodynamic effects on the pressure gradient characteristics of the tactile sensor. Among them, the electromagnetic interference intensity refers to the degree of interference of the electromagnetic field in the environment on the sensor signal. Specifically, it can be realized by a magnetic field intensity meter or a radio frequency sensor for dynamic measurement. Its function is to identify the disturbance of electromagnetic field anomalies on the lidar point cloud data or the imaging quality of the visual sensor. The online calibration algorithm refers to a compensation method for dynamically correcting sensor data based on physical field parameters. Specifically, a heat conduction model can be used to compensate for the baseline shift of the tactile sensor caused by the temperature gradient, or an electromagnetic shielding coefficient matrix can be used to correct the noise data of the lidar. Its function is to suppress noise propagation from the source of environmental interference and ensure that the multi-modal credibility evaluation model assigns weights based on the corrected data.

[0076] Specifically, when the environmental temperature change rate exceeds a preset threshold (e.g., the baseline drift of the tactile sensor due to thermal expansion of the metal structure) or the electromagnetic interference intensity reaches the device sensitivity threshold (e.g., the increase in lidar point cloud noise near high-voltage equipment), the online calibration algorithm is triggered to dynamically compensate the disturbed sensor data. The tactile sensor data eliminates the baseline shift caused by the temperature gradient through the heat conduction model, the visual sensor data removes image noise through the electromagnetic shielding coefficient matrix, and the lidar data uses the electromagnetic interference correction algorithm based on the Lorentz force equation to repair the point cloud distortion. The calibrated multi-modal data is input into the multi-modal credibility evaluation model, and the reliability weights of each sensing channel are recalculated in combination with the real-time physical field parameters to avoid the problem of inaccurate weight allocation caused by environmental noise.

[0077] Compared with the prior art, the traditional method uses static compensation parameters or single physical field monitoring, which cannot adapt to the multi-physical field coupling interference in a dynamic environment; while this solution realizes the joint correction of cross-modal data under the condition of double-threshold triggering of the temperature change rate and the electromagnetic interference intensity through multi-physical field dynamic monitoring and collaborative compensation, and can effectively solve the problem of weight allocation deviation caused by multi-source interference in a complex environment.

[0078] Embodiment 2, refer to Figure 2 the block diagram of the robot task planning system. The embodiment of the present invention provides the following technical solution: An embodied intelligent robot task planning system based on multi-dimensional situation awareness, including:

[0079] A spatio-temporal alignment module, which performs frequency-domain adaptive interpolation on asynchronous heterogeneous data of multi-modal sensor channels (the sensor device includes at least tactile, visual, and lidar) through a pulsed neural network, eliminates the spatio-temporal misalignment of high-frequency / low-frequency signals, outputs synchronous multi-modal data, and outputs it to the feature fusion module;

[0080] A feature fusion module, which uses a cross-modal feature fusion network (including spatio-temporal attention mechanism and recursive unit) to extract the spatio-temporal correlation features of multi-modal data, generates a fusion situation matrix containing semantic information (including multi-modal features such as tactile pressure gradient, visual target recognition, and lidar spatial topology), and transmits it to the causal modeling module and the credibility evaluation module;

[0081] A causal modeling module, based on dynamic causal modeling (Bayesian weight update algorithm), analyzes the causal association strength between multi-modal data in real time, detects sensor data conflicts (such as the contradiction between tactile residue and the disappearance of visual obstacles), outputs an updated sensor causal association map (including abnormal trigger marks), and transmits it to the abnormal traceability module;

[0082] The anomaly traceability module simulates the anomaly data distribution through a counterfactual reasoning engine (constructed based on an adversarial network and a historical anomaly library), identifies the anomaly types (such as electromagnetic interference, sensor failure), generates interpretable anomaly traceability results (including propagation paths and type labels), outputs the anomaly type labels and propagation path reports, and transmits them to the trustworthiness evaluation module;

[0083] The trustworthiness evaluation module combines the dynamic field strength distribution map (electromagnetic gradient, acoustic perturbation) and the anomaly traceability results, calculates the spatio-temporal consistency scores and environmental interference intensities of each sensor channel through a multi-modal trustworthiness evaluation model, dynamically adjusts the reliability weights, outputs the reliability weight matrix of the sensor channels (such as the tactile weight decreasing and the lidar weight increasing), and transmits it to the collaborative control module;

[0084] The collaborative control module performs collaborative predictive control (trajectory similarity measurement + collision risk calculation) by combining the reliability weights and the fusion situation matrix, jointly using the real robot dynamics model and the digital twin virtual model, generates a safety score and outputs action instructions; outputs the robot joint control instructions and the safety score, and triggers the rule evolution module when the safety score is lower than the threshold;

[0085] The rule evolution module verifies and optimizes the decision rule library (such as adding obstacle avoidance constraints, adjusting the grasping force threshold) in the digital twin environment through reinforcement learning and the anomaly pattern knowledge graph, dynamically updates the rules to adapt to extreme scenarios, outputs the updated safety decision rule library (such as emergency stop rules, dynamic obstacle avoidance strategies), and feeds the optimized rule library back to the collaborative control module to form a closed-loop iteration.

[0086] Finally: The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A task planning method for an embodied intelligent robot based on multi-dimensional situation awareness, characterized in that, It includes the following steps: Step 1: Use a pulsed neural network to perform spatio-temporal alignment on asynchronous heterogeneous data generated by multi-modal sensor channels, implement data synchronization using a frequency-domain adaptive interpolation algorithm, dynamically adjust the interpolation accuracy according to the target motion speed, extract cross-modal spatio-temporal features through a cross-modal feature fusion network, establish the feature correlation degree between different sensing channels through a cross-attention mechanism, and generate a fusion situation matrix containing multi-modal semantic features by combining temporal dependence modeling; Use the self-attention mechanism to quantify the correlation between tactile sensor data and visual sensor data, and fuse the spatial topological features of lidar sensor data under the cross-attention mechanism to generate a fusion situation matrix; Step 2: Based on the fusion situation matrix, use dynamic causal modeling to update the correlation strength between multi-modal data and reconstruct the causal relationship in real time; Identify and trace anomalies based on the anomaly tracing strategy of the generative adversarial network, combine the generative adversarial network to simulate the data distribution under abnormal conditions, and output the anomaly tracing result; Step 3: Use the fusion situation matrix and the anomaly tracing result to process the dynamic field strength distribution map and real-time physical field parameters through a graph neural network, and dynamically adjust the reliability weights of each sensing channel through a multi-modal credibility evaluation model; Step 4: Based on the reliability weights, input the fusion situation matrix into the real robot dynamics model and the digital twin virtual model, perform cooperative predictive control, calculate the spatial deviation amount between the real trajectory point sequence and the virtual trajectory point sequence through the trajectory similarity measurement algorithm, and combine the collision risk probability calculated by the three-dimensional risk field superposition to generate a safety score; And start the adaptive rule evolution mechanism when the safety score is lower than the threshold to ensure the safety of the decision-making.

2. The method for task planning of an embodied intelligent robot based on multi-dimensional situation awareness according to claim 1, wherein The asynchronous heterogeneous data includes low-frequency sensor data and high-frequency sensor data, and the frequency-domain adaptive interpolation algorithm includes the following steps: Perform frequency-domain decomposition on the low-frequency sensor data to extract the main frequency components; Encode the high-frequency sensor data into a pulse sequence, and the pulse firing threshold is dynamically set according to the sensor type, and the firing frequency is non-linearly correlated with the target motion speed; Adjust the interpolation window length based on the motion speed, and use spline interpolation to fill in the missing frames of the low-frequency data; Verify the timestamp alignment error of the multi-channel data, and trigger synchronous calibration when the error exceeds the tolerance.

3. The method for task planning of an embodied intelligent robot based on multi-dimensional situation awareness according to claim 1, characterized in that, The dynamic causal modeling uses the Bayesian weight update algorithm to reconstruct the causal relationship between multi-modal sensor data in real time, and triggers anomaly detection when the causal correlation degree between sensor data deviates from the preset dynamic error threshold; The counterfactual reasoning engine simulates the data distribution under abnormal conditions through the generative adversarial network, and outputs the anomaly tracing result containing the anomaly type label and the propagation path in combination with the historical anomaly pattern database.

4. The method for task planning of an embodied intelligent robot based on multi-dimensional situation awareness according to claim 1, wherein, The graph neural network generates a dynamic field strength distribution map based on the simultaneous localization and mapping technology. The multi-modal credibility evaluation model integrates an environmental physical field analysis unit and a sensing error compensation unit, where: the environmental physical field analysis unit constructs a dynamic field strength distribution map through electromagnetic field gradient monitoring and acoustic disturbance detection; the sensing error compensation unit uses a multi-physical coupling correction algorithm to generate a sensing data correction matrix based on real-time physical field parameters, and calculates the reliability weights of each sensing channel in combination with the semantic segmentation confidence index.

5. The method for task planning of an embodied intelligent robot based on multi-dimensional situation awareness according to claim 1, characterized in that, The method for obtaining the safety score is as follows: The digital twin virtual model is constructed through a physical engine, including a state synchronization, environment mapping, and delay compensation mechanism, and receives real embodied robot joint state data in real time and generates a three-dimensional occupancy map of the virtual environment; The trajectory similarity metric algorithm is used to calculate the spatial deviation between the real trajectory point sequence and the virtual trajectory point sequence, and the trajectory deviation degree is generated through a normalization mapping function; An obstacle occupancy probability model is constructed based on the lidar data after multi-physical coupling correction, and is dynamically corrected by fusing the confidence of the visual sensor and the pressure gradient feature of the tactile sensor, and the collision risk probability is calculated through the superposition of the three-dimensional risk field; The safety score is generated by weighted calculation of the trajectory deviation degree and the collision risk probability.

6. The method for task planning of an embodied intelligent robot based on multi-dimensional situation awareness according to claim 4, wherein, The process of calculating the reliability weight includes: Use i to represent the number index of the sensor channel, and N to represent the total number of sensor channels; Spatio-temporal Consistency Quantification: Based on multimodal consistency checking, calculate the spatio-temporal consistency score of the i-th sensing channel where v i is the real-time measurement vector of the i-th sensing channel, is the cross-modal prediction value, which is the prediction value generated by the Transformer-LSTM hybrid network using data from other sensing channels; cos(·) is the vector cosine similarity calculation function, which is used to quantify the directional consistency between real-time data and prediction values; Quantification of environmental interference intensity: Calculate the environmental interference intensity of the i-th sensing channel through the following formula: wherein, is the magnitude of the electromagnetic field gradient, obtained by differential calculation of a triaxial magnetic field sensor, P(f) is the sound pressure power spectral density, extracted by a fast Fourier transform from an acoustic sensor array; fL is the lower integration limit frequency, fH is the upper integration limit frequency; α and β are sensitivity coefficients related to the sensing type, which are scale factors obtained by fitting the interference-error curve in the sensor calibration experiment; Task relationship metric: Based on the security requirement level L of the current task stage c and the operation precision requirement R p , with a value range of 0.1 to 1.0, calculate the task criticality factor K c =γL c +(1 - γ)R p ; γ represents the task priority Dynamic weight synthesis: Denote as Calculate the reliability weight Wi of the i-th sensing channel through a weighted fusion function i , where ε is a small constant to prevent division-by-zero errors; Where ε is a small positive number, and its value range in the embodiment of the present invention is 0.01 - 0.0001, which is used to prevent calculation overflow caused by too small denominator.

7. The method for task planning of an embodied intelligent robot based on multi-dimensional situation awareness according to claim 5, wherein, The cooperative predictive control calculates the output deviation between the physical model and the digital twin virtual model through the trajectory similarity metric algorithm, and activates the safety constraint optimization when the output deviation exceeds the preset trajectory fault tolerance threshold; The adaptive rule evolution mechanism includes rule credibility evaluation and logical structure optimization, validates the effectiveness of the rules in the simulation environment through a reinforcement learning mechanism, and dynamically updates the constraint conditions and optimization objectives in the decision rule library; The adaptive rule evolution mechanism includes the following steps: Construct an abnormal pattern knowledge graph: Based on historical abnormal events of tactile, visual, and lidar sensor data, extract spatio-temporal correlation features and generate the topological structure of the abnormal propagation path; Incremental optimization of the rule library: Match the similarity between the real-time abnormal scenario and the historical knowledge graph through a contrast learning algorithm, dynamically update the constraint conditions in the decision rule library, and preferentially retain the rules whose effectiveness has been verified; Unsupervised verification mechanism: Simulate extreme scenarios in the digital twin virtual model to verify the stability of the newly added rules, and eliminate redundant rules that cause the safety score to decrease.

8. The method for task planning of an embodied intelligent robot based on multi-dimensional situation awareness according to claim 4, wherein, The multi-physical coupling correction algorithm includes an interference source localization unit and a dynamic weight redistribution unit; when it is detected that the electromagnetic field intensity gradient exceeds the device anti-interference threshold or the acoustic disturbance frequency enters the sensitive frequency band of the sensor, the interference source localization unit is activated for error compensation; The interference source localization unit includes: establishing a probability model of the interference source spatial distribution based on three-dimensional vector data of the electromagnetic field gradient and combining with the Lorentz force equation to calculate the coordinates of the maximum likelihood electromagnetic interference source; using the generalized cross-correlation algorithm to process the time-delay estimation of the acoustic sensing array, and constructing a confidence interval for the location of the acoustic interference source through the time difference of arrival of sound waves. The dynamic weight redistribution unit includes: when the Frobenius norm of the perturbation coefficient matrix composed of environmental physical field parameters exceeds the adaptive threshold, triggering a weight optimization algorithm based on Lyapunov stability theory to redistribute the reliability weights of multi-modal sensor data.

9. The method for task planning of an embodied intelligent robot based on multi-dimensional situation awareness according to claim 1, wherein, The multi-physical field coupling compensation mechanism monitors the temperature change rate and electromagnetic interference intensity in real time. When the temperature change rate or electromagnetic interference intensity exceeds the preset corresponding threshold, it triggers an online calibration algorithm to correct the data of tactile sensors, visual sensors, and lidar sensors, and suppresses the influence of environmental noise on the reliability weight allocation.

10. A task planning system for an embodied intelligent robot based on multi-dimensional situation awareness, which is used to implement the embodied intelligent robot task planning method described in claim 1 above, characterized in that, It includes: The spatio-temporal alignment module adaptively interpolates the asynchronous heterogeneous data of multi-modal sensor channels in the frequency domain through a pulsed neural network, eliminates spatio-temporal misalignment, outputs synchronized multi-modal data, and outputs it to the feature fusion module. The feature fusion module uses a cross-modal feature fusion network to extract the spatio-temporal correlation features of multi-modal data, generates a fusion situation matrix containing semantic information, and transmits it to the causal modeling module and the trust evaluation module. The causal modeling module, based on dynamic causal modeling, analyzes the causal association strength between multi-modal data in real time, detects sensor data conflicts, outputs an updated sensor causal association map, and transmits it to the anomaly tracing module. The anomaly tracing module simulates the abnormal data distribution through a counterfactual reasoning engine, identifies the type of anomaly, generates an interpretable anomaly tracing result, outputs the anomaly type label and the propagation path report, and transmits it to the trust evaluation module. The trust evaluation module combines the dynamic field strength distribution map and the anomaly tracing result, calculates the spatio-temporal consistency score and the environmental interference intensity of each sensor channel through a multi-modal credibility evaluation model, dynamically adjusts the reliability weights, outputs the reliability weight matrix of the sensor channels, and transmits it to the collaborative control module. The collaborative control module, based on the reliability weights and the fusion situation matrix, jointly performs collaborative predictive control with the real robot dynamics model and the digital twin virtual model, generates a safety score and outputs an action instruction. Outputs robot joint control instructions and a safety score. When the safety score is lower than the threshold, it triggers the rule evolution module. The rule evolution module verifies and optimizes the decision rule base in the digital twin environment through reinforcement learning and the anomaly pattern knowledge graph, dynamically updates the rules to adapt to extreme scenarios, outputs the updated safety decision rule base, and feeds the optimized rule base back to the collaborative control module to form a closed-loop iteration.

Images