Unmanned system security monitoring method based on multi-modal perception and autonomous decision discrimination

Abstract

The invention provides an unmanned system security monitoring method based on multi-modal perception and autonomous decision discrimination, belongs to the technical field of artificial intelligence and industrial automation crossing, and aims to solve the problem that a machine learning decision security monitoring system of an existing unmanned system is difficult to quickly perceive environment change and make accurate response. The method solves the problems that in a high-dynamic scene, recognition of sudden risks is lagged, decision adjustment is slow, and the safety and stability of task execution are affected in the prior art, and comprises the steps that S1, key parameters of an unmanned autonomous system are screened; s2, modeling a parameter coupling relation; s3, time sequence driven key parameter extreme value boundary modeling and statistical analysis are carried out; s4, discovering danger cognition; s5, building a real-time prediction framework; s6, constructing a risk assessment model; and S7, determining an optimal safety switching mechanism. The method is suitable for real-time safety control of unmanned equipment such as an unmanned aerial vehicle, an automatic driving vehicle and an industrial robot.

Description

Technical Field

[0001] This invention relates to a safety monitoring method for unmanned systems based on multimodal perception and autonomous decision-making, belonging to the interdisciplinary field of artificial intelligence and industrial automation. Background Technology

[0002] As intelligent unmanned systems evolve towards autonomy and collaboration, machine learning-driven cognitive decision-making mechanisms have become a core supporting technology. However, the contradiction between the autonomous evolutionary characteristics of the system in dynamic environments and its operational safety is becoming increasingly prominent. During the operation of unmanned systems, insufficient dynamic optimization capabilities for operational safety objectives, a lack of operational safety status and hazard recognition and detection capabilities, and a lack of precise control capabilities for operational safety lead to hazard perception and risky decisions by the complex and dynamically evolving machine learning models carried by the unmanned systems, affecting the mission execution capabilities and overall operational safety of the unmanned systems. Furthermore, in complex environments, machine learning systems may develop cognitive errors due to training data bias or adversarial attacks, resulting in incorrect decisions during target recognition or path planning, impacting the execution safety of unmanned aerial vehicle (UAV) missions. Therefore, to ensure the safety of autonomous decision-making systems, a safety monitoring method combining constraint optimization is used to set safety constraints to ensure that decisions made during the actual operation of the unmanned system meet safety requirements.

[0003] Existing machine learning decision-making safety monitoring systems for unmanned systems are insufficient in terms of real-time performance and dynamism. They are unable to quickly perceive environmental changes and make accurate responses, resulting in delayed identification of sudden risks and slow decision-making adjustments in highly dynamic scenarios, which affects the safety and stability of task execution. The specific challenges include: (1) From the modeling perspective, machine learning models often rely on historical data for training. However, when unmanned systems perform tasks, the environment and target state are constantly changing, and traditional static models are difficult to adapt to time dependence. Especially in highly dynamic scenarios, the decision delay of the model may lead to missing the critical control window, which may lead to task failure or safety accidents. At the same time, there is a strong coupling relationship between the perception, prediction, and control modules of unmanned systems. However, due to the randomness of sensor errors, environmental disturbances and data noise, it is difficult to clearly define the decision boundaries of different modules; (2) From the perspective of method establishment, most current unmanned system operation safety monitoring methods only consider the safety constraints of a single task scenario or static environment and rely on models trained with historical data for risk assessment. However, when unmanned systems are in complex and dynamic environments and face sudden risks or drastic environmental changes, existing methods are unable to perceive the global state in real time and quickly adjust decision-making strategies, leading to a decrease in system safety and an increase in erroneous decisions under highly dynamic and uncertain conditions. Summary of the Invention

[0004] This invention addresses the problem that existing machine learning-based decision-making safety monitoring systems for unmanned systems struggle to quickly perceive environmental changes and respond accurately, leading to delayed identification of sudden risks and slow decision-making adjustments in highly dynamic scenarios, thus affecting the safety and stability of task execution. Therefore, this invention proposes a safety monitoring method for unmanned systems based on multimodal perception and autonomous decision-making.

[0005] The technical solution adopted by the present invention to solve the above problems is as follows: The present invention includes the following steps: Step 1: Obtain the original parameter set during the operation of the unmanned autonomous system, and use correlation analysis to analyze the original parameter set. The key parameter set was obtained through filtering. ; Step 2: Based on the key parameter data matrix set Obtain the historical observation data matrix of key parameters Using dynamic Bayesian networks to analyze historical observation data matrices of key parameters The parameter coupling relationship is modeled to obtain the parameter coupling relationship model. ; Step 3: Analyze the key parameter set based on historical observation data. Key parameters in Preliminary limit value statistics were conducted to construct a historical limit data sequence. Combined with parameter coupling relationship model The set of extreme states is expanded to generate a new set of extreme scenario data. ; Step 4: Fit the new extreme scenario data and historical limit data using the generalized extreme value distribution method to obtain the parameters of the generalized extreme value distribution model. Combine the limit probability threshold to determine the limit performance threshold of each key parameter and obtain the set of limit performance thresholds for key parameters. Step 5: Conduct real-time continuous monitoring of key parameters of the unmanned autonomous system, obtain real-time status data of key parameters, and obtain the comprehensive hazard perception index of the unmanned autonomous system based on the real-time status data of key parameters and the performance limit threshold of the corresponding time data. Step 6: Based on the parameter coupling relationship model Real-time predicted values ​​of key parameters are obtained from the comprehensive risk perception index; Step 7: Construct a relative risk intensity index based on the predicted values ​​of key parameters and the corresponding performance limits at each time point, and construct a comprehensive system risk index based on the relative risk intensity index; Step 8: Set the risk tolerance value for the unmanned autonomous system If the risk prediction value The switching decision process is triggered. If the switching trigger condition is met, the alternative mode set is selected. Selecting the optimal control mode It then sends a switching command to the system execution layer, monitors the actual parameter changes, and confirms whether the switching is successfully completed. If a switching failure is detected, it automatically triggers the backoff mode or alarm system.

[0006] Furthermore, step 1 specifically includes: For the original parameter set For each parameter in the equation, calculate its Pearson correlation coefficient with the safety status. According to the Pearson correlation coefficients of all parameters with the safety status Sort the parameters in descending order of their size to obtain the parameter set. Set threshold For the sorted parameter set Truncation yields the set of key parameters. ; Original parameter set The expression is: (1); In formula (1), This represents the total number of data sampling points. The number of parameter types detected; Pearson correlation coefficient The calculation formula is: (2); In formula (2), For the first The sampling time of the first sampling moment Observed values ​​of each parameter, For the first The average of the observed values ​​of each parameter over all time periods. For the first The security label at any given moment is 0 or 1. The mean of the system security status labels, and the Pearson correlation coefficient. The larger the value, the closer it is to the safety of unmanned autonomous systems; Parameter set The expression is: (3); In formula (3), This is represented by the parameter that is most relevant to the safety status. It is the parameter with the second highest correlation to safety status. The parameter with the lowest correlation to safety status; Key parameter set The expression is: (4); In formula (4), These are the key parameters after filtering the parameters in formula (3) based on the threshold.

[0007] Furthermore, in step 2, the historical observation data matrix of key parameters... The expression is: (5); In formula (5), For the first The parameter in the first... The value at time, M The number of key parameters. T The observation time; A dynamic Bayesian network is used to characterize the conditional probabilities and dynamic causal relationships among the parameters in the historical observation data matrix of key parameters, thus obtaining a parameter coupling relationship model. ; Parameter coupling relationship model The calculation formula is: (6); (7); In formulas (6) and (7), For the first Key parameters at future time state, For at any time For the The change in the state of a key parameter directly affects the set of other parameters, namely the parent node. To determine the state value of a certain parameter, For a specific combination of values ​​of the parent node set, Conditions appearing in historical data Number of times, For historical data to appear simultaneously and The number of times.

[0008] Furthermore, step 3 specifically includes: Based on historical observation data Preliminary limit value statistics are performed on each key parameter to construct its historical limit data sequence. Among them, historical extreme data sequences Defined as the set of maximum and minimum values ​​obtained within a certain time period in historical data, using a parameter coupling relationship model. The set of extreme states is expanded to generate a new set of extreme scenario data. ; Historical Limit Data Series The expression is: (8); In formula (8), For the first A time period interval, The total number of time periods into which historical data is divided; New extreme scenario data set The expression is: (9); In formula (9), This refers to a set of limit states that are actively defined through expert experience. For a predefined limit state combination space, This is a new set of extreme scenario data generated by the model.

[0009] Furthermore, step 4 specifically includes: The generalized extreme value distribution method is used to fit new extreme scenario data and historical limit data, and performance limit analysis is performed to obtain the fitting parameters. For the fitted parameters Differentiation yields the parameters of the generalized extreme value distribution model. Set the limit probability threshold Determine the limit performance threshold for each key parameter and output the set of limit performance thresholds for key parameters; Fitting parameters The expression is: (10); In formula (10), Indicates key parameters The fitting parameters are determined using the maximum likelihood estimation method, and their expression is: (11); In formula (11), Key parameters in merged data The One limit value, For parameters The total number of limit value data; The formula for calculating the ultimate performance threshold of each key parameter is as follows: (12); In formula (12), For the first Performance limit thresholds for key parameters; The expression for the set of key parameter performance limit thresholds is: (13).

[0010] Furthermore, step 5 specifically includes: Step 5.1: Using real-time sensors, continuously monitor the key parameters of the unmanned autonomous system in real time and obtain a real-time status dataset of the key parameters. ; Step 5.2: Calculate the hazard level index based on the real-time status data of key parameters and the performance limit threshold of the corresponding time point data. , among which, when When this occurs, it indicates that the corresponding key parameters are approaching or have exceeded the performance limits, and the unmanned autonomous system is at risk of operational danger. When the corresponding key parameters are within a safe range, it indicates that the unmanned autonomous system is operating normally. Step 5.3: Obtain the weight coefficients of the corresponding key parameters based on the analytic hierarchy process (AHP). Weight coefficients based on corresponding key parameters and risk level indicators Calculate the comprehensive risk perception index of unmanned autonomous systems ; Real-time status dataset of key parameters The expression is: (14); In formula (14), Indicates the first These parameters are acquired in real time. The actual value; Risk level indicators The calculation formula is: (15); In formula (15),, For the first Key parameters at real-time monitoring time The level of danger indicators; For the first Key parameters at real-time monitoring time The actual value; The first one calculated from step 5 Key parameters at real-time monitoring time The performance limit threshold; For the first Key parameters at real-time monitoring time The normal reference value is usually taken as the typical value under normal operating conditions. Comprehensive Risk Perception Index The calculation formula is: (16); In formula (16), It is an unmanned autonomous system in real time The comprehensive risk awareness index; It is the first The weighting coefficients of each key parameter need to satisfy... conditions, Obtained through the Analytic Hierarchy Process (AHP).

[0011] Furthermore, step 6 specifically includes: At the current time t, the parameter coupling relationship model is used. Predicting from the next moment By a certain prediction window in the future Values ​​of internal key parameter states When the risk level index When the threshold is exceeded, a predictive conservative adjustment coefficient corresponding to the parameter is introduced. Logarithmic Make corrections and obtain real-time predicted values ​​for the corresponding key parameters; Values ​​of key parameter states The calculation formula is: (17); The formula for calculating the real-time predicted values ​​of the corresponding key parameters is as follows: (18); In formula (18), Let be a constant, representing the th One parameter predicts the conservative adjustment coefficient. For the first The historical prediction error standard deviation of each parameter.

[0012] Furthermore, step 7 specifically includes: At each prediction time For each key parameter Predicted value Expected performance limits The process is performed to obtain the relative risk intensity index. , Based on the relative risk intensity index of each key parameter Construct a comprehensive system risk index; Relative risk intensity index The calculation formula is: (19); In formula (19), This indicates a safety situation; a higher value indicates a higher risk. This indicates that the parameter exceeds the safe range, and the unmanned autonomous system is in danger. The formula for calculating the system's comprehensive risk index is as follows: (20); In formula (20), This represents the total number of key parameters. It is the first The weighting coefficients of each key parameter.

[0013] Furthermore, the optimal control mode in step 8 The expression is: (twenty one); In formula (21), For control mode The unit operating energy consumption characterization parameter; For control mode Parameters characterizing adaptability and security assurance capabilities to current risk scenarios; This is used to balance the relationship between security priorities and resource costs. If the system has higher security requirements, it should be set to... If resources or costs are limited, the requirement should be increased. Weights.

[0014] The beneficial effects of this invention are: 1) This invention, under the premise of accurately predicting the system's performance limits, establishes a quantitative assessment model for system decision-making risks, dynamically integrating safety baseline decisions and machine learning cognitive decisions to enhance the precise control capability of machine learning system operation safety. First, addressing the problem of unclear multi-factor coupling triggering hazard mechanisms in unmanned systems under adversarial operating conditions, this invention clarifies the discovery method for hazard cognition during unmanned system operation by mining the characterization parameters and evolutionary laws of safety during unmanned system operation. Second, addressing the difficulty of existing methods in quantifying decision-making risks in real time and adapting to dynamic environmental changes, this invention simultaneously constructs a risk assessment model based on uncertainty quantification and a dynamic safety decision-making model driven by a real-time decision data chain to predict and control the safety risks of unmanned systems in complex environments in real time. Furthermore, it correlates and integrates the risk assessment results with system execution feedback information to obtain an accurate prediction model of the system's safety risk status. Finally, based on the dynamically integrated safety baseline decision-making framework, real-time risk adjustment and optimization are performed, and intelligent control and safety enhancement are applied to the task execution strategies of unmanned systems in highly dynamic environments to ensure the safety and stability of unmanned systems under complex task conditions.

[0015] 2) This invention constructs a unified framework for monitoring the safety status of unmanned systems. By integrating key parameter screening, coupling relationship modeling, performance limit testing, and hazard recognition mechanisms into a unified framework, dynamic Bayesian networks are introduced into the multi-parameter modeling task of unmanned autonomous systems. A coupling model of key operating parameters for dynamic environments is constructed, which realizes the characterization of the interaction relationships between complex parameters within the system. Furthermore, the adaptive learning of the model structure and parameters is completed through maximum likelihood estimation, significantly improving the system's predictive ability and recognition accuracy for potential risk states.

[0016] 3) This invention addresses the shortcomings and deficiencies of existing unmanned system safety monitoring methods, which rely on single parameter thresholds, have delayed responses, and cannot fully reflect the system state evolution process. Based on the research results of complex system modeling theory, dynamic Bayesian network technology, and multi-parameter fusion prediction and risk assessment, this invention proposes an intelligent operation safety monitoring method driven by parameter coupling modeling and future state prediction, and establishes an operation risk monitoring and control framework that is more forward-looking and intelligent than the traditional rule triggering mechanism.

[0017] 4) Taking UAV simulation as one of the target applications, simulation experiments were conducted under scenarios of varying complexity, and the proposed operational safety monitoring system was used to verify and evaluate the system's safety. The results show that this invention effectively achieves continuous monitoring and dynamic control of the unmanned system's operational status, possessing good practicality and safety assurance capabilities. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the safety monitoring method for unmanned systems based on multimodal perception and autonomous decision-making provided by the present invention. Figure 2 A flowchart illustrating the optimal switching mechanism provided by this invention; Figure 3 This is a schematic diagram of real-time data acquisition for the UAV flight control system provided by the present invention; Figure 4 This invention provides a schematic diagram illustrating the occurrence of dangerous decisions in simulation. Figure 5 A schematic diagram illustrating real-time control of hazard decision-making provided by this invention. Detailed Implementation

[0019] Specific implementation method one: Combining Figure 1 and Figure 2 This implementation method is described as follows: Figure 1 As shown, the steps of the unmanned system safety monitoring method based on multimodal perception and autonomous decision-making in this embodiment include: S1: Screening of key parameters for unmanned autonomous systems; S101: Numerous data parameters are monitored in real-time during the operation of unmanned autonomous systems. To improve model efficiency and reduce system computational complexity, correlation analysis is used to analyze the original parameter set. The set of key parameters that have a significant impact on security was selected from the data. Original parameter set for: (1); In formula (1), This represents the total number of data sampling points. The number of parameter types detected; Correlation analysis was performed on each parameter and the system safety status, which was represented by a binary label: 0 indicates the system is operating safely, and 1 indicates that a danger has occurred. The Pearson correlation coefficient was calculated. (2); In formula (2), For the first The sampling time of the first sampling moment Observed values ​​of each parameter, For the first The average of the observed values ​​of each parameter over all time periods. For the first The security label at any given moment is 0 or 1. The mean of the system security status labels, and the Pearson correlation coefficient. The larger the value, the closer it is to the safety of unmanned autonomous systems; S102: Based on the correlation between the obtained parameters and the safety status Sort the parameters in descending order to obtain the parameter set: (3); In formula (3), This is represented by the parameter that is most relevant to the safety status. It is the parameter with the second highest correlation to safety status. The parameter with the lowest correlation to safety status; S103: By setting a threshold The sorted parameter set is truncated to obtain the key parameter set: (4); In formula (4), These are the key parameters after filtering the parameters in formula (3) based on the threshold.

[0020] S2: Modeling of parameter coupling relationships; S201: The set of key parameters selected using S1 Historical observation data of key parameters were obtained, and the total number of key parameters was set. One, observation duration is At each moment, the historical observation data matrix of key parameters is obtained: (5); In formula (5), For the first The parameter in the first... The value at time, M The number of key parameters. T The observation time; S202: A dynamic Bayesian network is used to represent the conditional probabilities and dynamic causal relationships between parameters, and a parameter coupling expression is constructed. (6); In formula (6), For the first Key parameters at future time state, For at any time For the The set of other parameters that are directly affected by the state change of a key parameter, i.e., the parent node.

[0021] To determine the structure of the dynamic Bayesian, i.e., the set of parent nodes for each parameter. If a certain parameter For parameters If it has a significant impact, then Based on the determined network structure, the conditional probability parameters in the network are estimated using the Maximum Likelihood Estimation (MLE) method, letting... This indicates the state value of a certain parameter. Let represent a specific combination of values ​​for the set of parent nodes. Then the conditional probability of MLE is: (7); In formula (7), To determine the state value of a certain parameter, For a specific combination of values ​​of the parent node set, Conditions appearing in historical data Number of times, For historical data to appear simultaneously and The number of times.

[0022] S3: Time-series driven extreme value boundary modeling and statistical analysis of key parameters; S301: Based on historical observation data Preliminary limit value statistics are performed for each key parameter, that is, for each key parameter... Construct its historical limit data sequence Historical extreme data sequence Defined as the set of maximum and minimum values ​​obtained within a certain time period in historical data: (8); In formula (8), For the first A time period interval, The total number of time periods into which historical data is divided; S302: Due to the limited and potentially incomplete extreme scenarios in historical data, it is necessary to generate more possible extreme operating scenarios, utilizing the parameter coupling relationship model constructed in S2. To supplement insufficient historical data, other key parameters may reach extreme states under these extreme operating conditions. The expansion process is as follows: (9); In formula (9), This refers to a set of limit states that are actively defined through expert experience. For a predefined limit state combination space, This is a new set of extreme scenario data generated by the model.

[0023] S303: Merge historical data and model extension data Then, the generalized extreme value distribution (GEV) method was used for performance limit analysis: (10); In formula (10), Indicates key parameters The fitting parameters are determined using the maximum likelihood estimation method, and their expression is: (11); In formula (11), Key parameters in merged data The One limit value, For parameters The total number of limit value data; S304: After obtaining the GEV model parameters through fitting, select an appropriate limiting probability threshold. Determine the performance limit threshold for each key parameter: (12); In formula (12), For the first The system outputs the performance limit thresholds for each key parameter and the set of performance limit thresholds for each key parameter. (13).

[0024] The set of key parameter performance limit thresholds can provide a quantitative benchmark for real-time system operation monitoring, prediction and early warning, and risk decision-making.

[0025] S4: Hazard awareness and discovery; S401: This implementation method relies on real-time sensors to continuously monitor key parameters of the unmanned autonomous system and obtain real-time status data of these key parameters. (14); In formula (14), Indicates the first These parameters are acquired in real time. The actual value; S402: To quantitatively assess the distance between the real-time status of key parameters and the performance limit threshold determined in step three, the following risk level index is defined: (15); In formula (15),, For the first Key parameters at real-time monitoring time The level of danger indicators; For the first Key parameters at real-time monitoring time The actual value; The first one calculated from step 5 Key parameters at real-time monitoring time The performance limit threshold; For the first Key parameters at real-time monitoring time The normal reference value is usually taken as the typical value of the parameter under normal operating conditions. When this occurs, it indicates that the corresponding key parameters are approaching or have exceeded the performance limits, and the unmanned autonomous system is at risk of operational danger. When the corresponding key parameters are within a safe range, it indicates that the unmanned autonomous system is operating normally. S403: To uniformly assess the overall operational hazard status of the system, the comprehensive hazard perception index of the system is defined as follows: (16); In formula (16), It is an unmanned autonomous system in real time The comprehensive risk awareness index; It is the first The weighting coefficients of each key parameter need to satisfy... conditions, Obtained through the Analytic Hierarchy Process (AHP).

[0026] S5: Build a real-time prediction architecture; S501: At the current moment Using the parametric coupling model in S2 Predicting from the next moment By a certain prediction window in the future The values ​​of the internal key parameter states, where For predicting time series length: (17); S502: When S4 hazard perception index If the threshold is exceeded, the predictive conservatism of the prediction model is increased to increase the uncertainty boundary of the prediction, and the implementation of the prediction model is revised: (18); In formula (18), Let be a constant, representing the th One parameter predicts the conservative adjustment coefficient. For the first The historical prediction error standard deviation of each parameter.

[0027] S6: Construct a risk assessment model; S601: At each prediction time For each key parameter Predicted value Expected performance limits After processing, the relative risk intensity index is obtained: (19); In formula (19), This indicates a safety situation; a higher value indicates a higher risk. This indicates that the parameter exceeds the safe range, and the unmanned autonomous system is in danger. S602: Similarly, construct a comprehensive system risk index based on the risk intensity of each key parameter: (20); In formula (20), This represents the total number of key parameters. It is the first The weighting coefficients of each key parameter.

[0028] To address the shortcomings and deficiencies of existing unmanned system safety monitoring methods, such as reliance on single parameter thresholds, delayed response, and inability to fully reflect the system state evolution process, this paper establishes a more forward-looking and intelligent operational risk monitoring and control framework based on complex system modeling theory, dynamic Bayesian network technology, and research results on multi-parameter fusion prediction and risk assessment.

[0029] S7: Establish the optimal safe handover mechanism; Set system risk tolerance value ,when Upon establishment, a switching decision process is triggered. If the switching trigger condition is met, the alternative mode set is selected. Selecting the optimal control mode Its goal is to achieve a balance between optimizing the overall operation of the unmanned system and minimizing its risks. (twenty one); In formula (21), For control mode The unit operating energy consumption characterization parameter; For control mode Parameters characterizing adaptability and security assurance capabilities to current risk scenarios; This is used to balance the relationship between security priorities and resource costs. If the system has higher security requirements, it should be set to... If resources or costs are limited, the requirement should be increased. Weights.

[0030] If the optimal control mode Once confirmed, a switchover command is sent to the system execution layer to monitor changes in actual parameters and confirm whether the switchover was successfully completed. If a switchover failure is detected, the backoff mode or alarm system is automatically triggered.

[0031] Optimal switching mechanism such as Figure 2As shown, the optimal switching module includes sub-modules such as threat frequency calculation, vulnerability analysis, real-time situational decision coupling, threat intensity evaluation, threat ranking, risk value calculation, security decision label generation, and fusion analysis and fusion decision output. Specifically, the threat frequency module is used to statistically assess the probability of threat occurrence; the vulnerability module analyzes the system's sensitivity to specific threats; the real-time situational decision coupling module generates decision inputs based on real-time information; the threat intensity module quantitatively evaluates different threats; the threat ranking module integrates threat frequency and intensity for priority ranking; the risk value module comprehensively calculates the overall risk currently faced by the system; the security decision label generation module analyzes the input state parameters and generates security decision labels; the fusion analysis module deeply integrates the machine learning system, the baseline security system, and the decision label information; and the fusion decision output module ultimately outputs the optimal control strategy, enabling the unmanned autonomous system to make safe, real-time, and efficient operational decisions in hazardous scenarios.

[0032] In summary, this invention constructs a unified framework for monitoring the safety status of unmanned systems. By integrating key parameter selection, coupling relationship modeling, performance limit testing, and hazard recognition mechanisms into a unified framework, dynamic Bayesian networks are introduced into the multi-parameter modeling task of unmanned autonomous systems. A coupled model of key operating parameters for dynamic environments is constructed, enabling the characterization of complex interactions between parameters within the system. Furthermore, adaptive learning of the model structure and parameters is achieved through maximum likelihood estimation, significantly improving the system's predictive ability and recognition accuracy for potential risk states. Specific Implementation Method Two: To verify the technical effect of the present invention, this implementation method uses real-time operating data of the UAV flight control system as a specific example, collecting a data sequence of 5000 time steps in real time, including roll rate. Pitch rate yaw rate Triaxial acceleration data Etc., data such as Figure 3 As shown.

[0033] Mutual information analysis was performed on the above real-time data to calculate the correlation coefficient between each parameter and the system security status. This clarified the importance of each parameter, identified them as key parameters, and used them for the next step of modeling and analysis. The actual calculation results are shown in Table 1. Table 1

[0034] A dynamic Bayesian network (DBN) model is trained using historical normal data to determine the network structure and parameters. In practice, the maximum likelihood estimation (MLE) method is used for parameter training to obtain a parameter-coupled model. .

[0035] By fitting historical data using the generalized extreme value distribution (GEV), the performance limit thresholds of key parameters were obtained. The specific calculation results are shown in Table 2. Table 2

[0036] During real-time operation, hazard perception and discovery methods are used to calculate the hazard level of each parameter in real time, thereby obtaining the overall system hazard index: (twenty two); In formula (22), the actual weights of each parameter are... Based on the normalized mutual information values ​​obtained, the actual calculation results are shown in Table 3: Table 3

[0037] When the real-time calculated risk index exceeds the preset threshold of 0.85, the system predicts the status of key parameters for the next 20 steps in a rolling manner based on the real-time prediction mechanism, and performs real-time risk assessment model calculations to determine the specific time step when the risk index exceeds the threshold.

[0038] When the real-time risk index exceeds the threshold, the optimal switching method is triggered, which inputs the real-time predicted parameter state and risk index into the pre-trained reinforcement learning Actor-Critic model; the Critic model calculates the payoff function for each control mode (such as backup flight control mode, backup power mode, and stability augmentation control mode) in real time; the Actor model outputs the best control mode decision in real time based on the payoff function of the Critic model.

[0039] During the real-time operation of 5000 time steps, the optimal control mode switching was triggered 156 times, and 155 switching operations were successfully executed. The actual decision-making performance is shown in Table 4. Table 4

[0040] Dangerous decisions occur in simulations such as Figure 4 As shown, real-time control of risk decisions is as follows: Figure 5 As shown, combined with Figure 4 and Figure 5 Comparison and simulation verification with actual data show that the unmanned system safety monitoring method proposed in this invention performs well in terms of risk identification accuracy, decision switching success rate and real-time response capability, and has potential for engineering applications.

[0041] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A safety monitoring method for unmanned systems based on multimodal perception and autonomous decision-making, characterized in that, include: Step 1: Obtain the raw parameter set during the operation of the unmanned autonomous system. Correlation analysis was used to analyze the original parameter set. The key parameter set was obtained through filtering. ; Step 2: Based on the aforementioned key parameter data matrix set Obtain the historical observation data matrix of key parameters Using dynamic Bayesian networks to analyze historical observation data matrices of key parameters The parameter coupling relationship is modeled to obtain the parameter coupling relationship model. ; Step 3: Analyze the key parameter set based on historical observation data. Key parameters in Preliminary limit value statistics were conducted to construct a historical limit data sequence. Combined with parameter coupling relationship model The set of extreme states is expanded to generate a new set of extreme scenario data. ; Step 4: Fit the new extreme scenario data and historical limit data using the generalized extreme value distribution method to obtain the parameters of the generalized extreme value distribution model. Combine the limit probability threshold to determine the limit performance threshold of each key parameter and obtain the set of limit performance thresholds for key parameters. Step 5: Conduct real-time continuous monitoring of key parameters of the unmanned autonomous system, obtain real-time status data of key parameters, and obtain the comprehensive hazard perception index of the unmanned autonomous system based on the real-time status data of key parameters and the performance limit threshold of the corresponding time data. Step 6: Based on the parameter coupling relationship model Real-time predicted values ​​of key parameters are obtained from the comprehensive risk perception index; Step 7: Construct a relative risk intensity index based on the predicted values ​​of key parameters and the corresponding performance limits at each time point, and construct a comprehensive system risk index based on the relative risk intensity index; Step 8: Set the risk tolerance value for the unmanned autonomous system If the risk prediction value The switching decision process is triggered. If the switching trigger condition is met, the alternative mode set is selected. Selecting the optimal control mode It then sends a switching command to the system execution layer, monitors the actual parameter changes, and confirms whether the switching is successfully completed. If a switching failure is detected, it automatically triggers the backoff mode or alarm system.

2. The unmanned system safety monitoring method based on multimodal perception and autonomous decision-making as described in claim 1, characterized in that, Step 1 specifically includes: For the original parameter set For each parameter in the equation, calculate its Pearson correlation coefficient with the safety status. According to the Pearson correlation coefficients of all parameters with the safety status Sort the parameters in descending order of their size to obtain the parameter set. Set threshold For the sorted parameter set Truncation yields the set of key parameters. ; Original parameter set The expression is: （1）； In formula (1), This represents the total number of data sampling points. The number of parameter types detected; Pearson correlation coefficient The calculation formula is: （2）； In formula (2), For the first The sampling time of the first sampling moment Observed values ​​of each parameter, For the first The average of the observed values ​​of each parameter over all time periods. For the first The security label at any given moment is 0 or 1. The mean of the system security status labels, and the Pearson correlation coefficient. The larger the value, the closer it is to the safety of unmanned autonomous systems; Parameter set The expression is: （3）； In formula (3), This is represented by the parameter that is most relevant to the safety status. It is the parameter with the second highest correlation to safety status. The parameter with the lowest correlation to safety status; Key parameter set The expression is: （4）； In formula (4), These are the key parameters after filtering the parameters in formula (3) based on the threshold.

3. The unmanned system safety monitoring method based on multimodal perception and autonomous decision-making as described in claim 1, characterized in that, Step 2: Historical observation data matrix of key parameters The expression is: （5）； In formula (5), For the first The parameter in the first... The value at time, M The number of key parameters. T The observation time; A dynamic Bayesian network is used to characterize the conditional probabilities and dynamic causal relationships among the parameters in the historical observation data matrix of key parameters, thus obtaining a parameter coupling relationship model. ; Parameter coupling relationship model The calculation formula is: （6）； （7）； In formulas (6) and (7), For the first Key parameters at future time state, For at any time For the The change in the state of a key parameter directly affects the set of other parameters, namely the parent node. To determine the state value of a certain parameter, For a specific combination of values ​​of the parent node set, Conditions appearing in historical data Number of times, For historical data to appear simultaneously and The number of times.

4. The unmanned system safety monitoring method based on multimodal perception and autonomous decision-making as described in claim 1, characterized in that, Step 3 specifically includes: Based on historical observation data Preliminary limit value statistics are performed on each key parameter to construct its historical limit data sequence. Among them, historical extreme data sequences Defined as the set of maximum and minimum values ​​obtained within a certain time period in historical data, using a parameter coupling relationship model. The set of extreme states is expanded to generate a new set of extreme scenario data. ; Historical Limit Data Series The expression is: （8）； In formula (8), For the first A time period interval, The total number of time periods into which historical data is divided; New extreme scenario data set The expression is: （9）； In formula (9), This refers to a set of limit states that are actively defined through expert experience. For a predefined limit state combination space, This is a new set of extreme scenario data generated by the model.

5. The unmanned system safety monitoring method based on multimodal perception and autonomous decision-making as described in claim 1, characterized in that, Step 4 specifically includes: The generalized extreme value distribution method is used to fit new extreme scenario data and historical limit data, and performance limit analysis is performed to obtain the fitting parameters. For the fitted parameters Differentiation yields the parameters of the generalized extreme value distribution model. Set the limit probability threshold Determine the limit performance threshold for each key parameter and output the set of limit performance thresholds for key parameters; Fitting parameters The expression is: （10）； In formula (10), Indicates key parameters The fitting parameters are determined using the maximum likelihood estimation method, and their expression is: （11）； In formula (11), Key parameters in merged data The One limit value, For parameters The total number of limit value data; The formula for calculating the ultimate performance threshold of each key parameter is as follows: （12）； In formula (12), For the first Performance limit thresholds for key parameters; The expression for the set of key parameter performance limit thresholds is: （13）。 6. The unmanned system safety monitoring method based on multimodal perception and autonomous decision-making as described in claim 1, characterized in that, Step 5 specifically includes: Step 5.1: Using real-time sensors, continuously monitor the key parameters of the unmanned autonomous system in real time and obtain a real-time status dataset of the key parameters. ; Step 5.2: Calculate the hazard level index based on the real-time status data of key parameters and the performance limit threshold of the corresponding time point data. , among which, when When this occurs, it indicates that the corresponding key parameters are approaching or have exceeded the performance limits, and the unmanned autonomous system is at risk of operational danger. When the corresponding key parameters are within a safe range, it indicates that the unmanned autonomous system is operating normally. Step 5.3: Obtain the weight coefficients of the corresponding key parameters based on the analytic hierarchy process (AHP). Weight coefficients based on corresponding key parameters and risk level indicators Calculate the comprehensive risk perception index of unmanned autonomous systems ; Real-time status dataset of key parameters The expression is: （14）； In formula (14), Indicates the first These parameters are acquired in real time. The actual value; Risk level indicators The calculation formula is: （15）； In formula (15),, For the first Key parameters at real-time monitoring time The level of danger indicators; For the first Key parameters at real-time monitoring time The actual value; The first one calculated from step 5 Key parameters at real-time monitoring time The performance limit threshold; For the first Key parameters at real-time monitoring time The normal reference value is usually taken as the typical value under normal operating conditions. Comprehensive Risk Perception Index The calculation formula is: （16）； In formula (16), It is an unmanned autonomous system in real time The comprehensive risk awareness index; It is the first The weighting coefficients of each key parameter need to satisfy... conditions, Obtained through the Analytic Hierarchy Process (AHP).

7. The unmanned system safety monitoring method based on multimodal perception and autonomous decision-making as described in claim 1, characterized in that, Step 6 specifically includes: At the current time t, the parameter coupling relationship model is used. Predicting from the next moment By a certain prediction window in the future Values ​​of internal key parameter states When the risk level index When the threshold is exceeded, a predictive conservative adjustment coefficient corresponding to the parameter is introduced. Logarithmic Make corrections and obtain real-time predicted values ​​for the corresponding key parameters; Values ​​of key parameter states The calculation formula is: （17）； The formula for calculating the real-time predicted values ​​of the corresponding key parameters is as follows: （18）； In formula (18), Let be a constant, representing the th One parameter predicts the conservative adjustment coefficient. For the first The historical prediction error standard deviation of each parameter.

8. The unmanned system safety monitoring method based on multimodal perception and autonomous decision-making as described in claim 1, characterized in that, Step 7 specifically includes: At each prediction time For each key parameter Predicted value Expected performance limits The process is performed to obtain the relative risk intensity index. , Based on the relative risk intensity index of each key parameter Construct a comprehensive system risk index; Relative risk intensity index The calculation formula is: （19）； In formula (19), This indicates a safety situation; a higher value indicates a higher risk. This indicates that the parameter exceeds the safe range, and the unmanned autonomous system is in danger. The formula for calculating the system's comprehensive risk index is as follows: （20）； In formula (20), This represents the total number of key parameters. It is the first The weighting coefficients of each key parameter.

9. The method for safety monitoring of unmanned systems based on multimodal perception and autonomous decision-making as described in claim 1, characterized in that, Optimal control mode in step 8 The expression is: （21）； In formula (21), For control mode The unit operating energy consumption characterization parameter; For control mode Parameters characterizing adaptability and security assurance capabilities to current risk scenarios; This is used to balance the relationship between security priorities and resource costs. If the system has higher security requirements, it should be set to... If resources or costs are limited, the requirement should be increased. Weights.

Images