Risk calculation method and system of improved MOE

Abstract

The invention discloses an improved MOE risk calculation method and system. The method comprises the following steps: acquiring related data of a to-be-monitored scene; performing feature extraction and coding on the related data to obtain a three-dimensional attention matrix; processing the three-dimensional attention matrix by adopting an improved MOE module to obtain a multi-modal fusion result; performing comprehensive analysis on each risk factor according to a multi-modal fusion result to obtain a risk type, a risk size and a risk trend so as to obtain multi-dimensional information; and outputting the multi-dimensional information. By implementing the method provided by the invention, complex spatio-temporal data can be better processed, so that the risk prediction and response capabilities in the public security field are improved, and the problems of difficulty in modeling a spatio-temporal coupling relationship, defects of long-term dependence on modeling and stiffness of expert decision in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to a public safety monitoring method, more particularly to an improved risk calculation method and system for MOE. BACKGROUND

[0002] In the field of public safety, especially in scenarios such as crowd management at large events and natural disaster warning, dynamic risk calculation often requires real-time integration of data from multiple different sources and types. These data may include video surveillance images, sensor signals, and various environmental parameters, etc. These information has a high degree of coupling in space and time, and changes over time. In order to effectively identify potential risks and make timely decisions, efficient models are needed to calculate and analyze these data.

[0003] However, traditional convolutional neural networks and recurrent neural networks have significant limitations in modeling spatio-temporal coupling. In particular, when modeling the correlation between sudden changes in crowd density and structural vibration data, these traditional networks struggle to capture these complex dynamics. For example, when crowd density changes rapidly over a short period of time, it can trigger vibrations in buildings or structures, and these changes need to be captured and analyzed from multiple sensor data and video monitoring information in a timely manner to respond. However, CNN and RNN have obvious shortcomings in modeling multi-dimensional and multi-temporal relationships. Although the current standard Transformer model has achieved some success in natural language processing and time series analysis, it often loses some key time series features due to the dilution effect of the attention mechanism when analyzing continuous time series, such as temperature gradient trends. Transformer captures information through global attention mechanisms, but it may ignore important data points in early stages when modeling long-term dependencies, which can affect the accuracy of the prediction results. For example, in disaster warning, if a historical data point is crucial for future risk assessment, but due to the length of the sequence, the model may not be able to maintain effective attention on that data point. Many MOE (Mixture of Experts) models currently use a fixed threshold routing strategy in the decision-making process, i.e., a set of pre-set mechanisms are used to determine which expert models will participate in decision-making at the current time. However, this static routing strategy cannot adapt to the changing needs of real-time scenarios. In practical applications of risk assessment, situations can change rapidly, such as sudden changes in crowd flow or sudden environmental fluctuations, which require some expert models to be given higher weights, while others should reduce their participation. The fixed routing mechanism cannot dynamically adjust the weights of the experts, resulting in insufficient decision-making ability of the expert models at some moments, affecting the accuracy and timeliness of the overall decision-making. Some existing improvements, such as the combination of long short-term memory networks and MOE models, try to solve the long-term dependency problem by introducing long short-term memory capabilities and improving decision-making accuracy through multiple expert models. However, existing methods do not completely solve the problem of multi-expert collaborative decision-making. In these models, while each expert model can play a role in different sub-tasks, collaboration between experts still faces significant obstacles in practical applications. While Transformer-MOE can combine the powerful expression capabilities of Transformer with the multi-expert decision-making of MOE, it still relies on a static routing mechanism and cannot flexibly adjust the weights of each expert according to actual conditions. Therefore, although these improvements have improved the accuracy of risk prediction to some extent, there is still a problem of lag, which cannot achieve real-time and dynamic risk prediction.

[0004] Therefore, it is necessary to design a new method to better process complex spatio-temporal data to improve the risk prediction and response capability in the field of public safety, so as to solve the difficulties of modeling spatio-temporal coupling relationship, the defects of long-term dependence on modeling and the rigidity of expert decision-making in the prior art. SUMMARY

[0005] The purpose of the present application is to overcome the defects of the prior art and provide an improved MOE risk calculation method and system.

[0006] To achieve the above purpose, the present application adopts the following technical solutions: an improved MOE risk calculation method, comprising: acquiring relevant data of a scene to be monitored; performing feature extraction and encoding on the relevant data to obtain a three-dimensional attention matrix; processing the three-dimensional attention matrix using an improved MOE module to obtain a multi-modal fusion result; comprehensively analyzing each risk factor according to the multi-modal fusion result to obtain a risk type, a risk size and a risk trend to obtain multi-dimensional information; outputting the multi-dimensional information.

[0007] Further technical solutions are as follows: the feature extraction and encoding on the relevant data to obtain a three-dimensional attention matrix, comprising: using an improved spatio-temporal coupling attention module to perform multi-scale feature extraction on the relevant data, and constructing a three-dimensional attention matrix by combining spatial position weighting and time decay function.

[0008] Further technical solutions are as follows: the multi-scale feature extraction on the relevant data using an improved spatio-temporal coupling attention module, and constructing a three-dimensional attention matrix by combining spatial position weighting and time decay function, comprising: forming a query by splicing task embedding and environment embedding, and generating keys and values by projecting the relevant data, and constructing a three-dimensional attention matrix by combining spatial position weighting and time decay function.

[0009] Further technical solutions are as follows: the processing of the three-dimensional attention matrix using an improved MOE module to obtain a multi-modal fusion result, comprising: using a multi-modal gating function to amplify abnormal features of the three-dimensional attention matrix, and selecting an expert network by combining a dynamic weight adjustment mechanism; performing multi-modal feature fusion using the expert network to obtain a multi-modal fusion result.

[0010] Further technical solutions of the application are as follows: the multi-modal gating function is used to process the three-dimensional attention matrix using a multi-head attention mechanism.

[0011] Further technical solutions of the application are as follows: the dynamic weight adjustment mechanism includes using KL divergence to measure the difference between each expert network and the environment distribution to adjust the weight of the expert network.

[0012] Further technical solutions of the application are as follows: the multi-modal feature fusion using the expert network includes: The weight of each expert network is calculated by the improved MOE module, and the outputs of multiple expert networks are weighted, and the three-dimensional attention matrix is fused by a Sigmoid function and a risk sensitivity coefficient to obtain a multi-modal fusion result.

[0013] Further technical solutions of the application are as follows: the parameters of each expert network are adjusted by a learning rate and a gradient of a task loss function.

[0014] Further technical solutions of the application are as follows: the task embedding includes a potential risk type, and the environment embedding includes static risk factors and dynamic risk factors.

[0015] The application also provides an improved MOE risk calculation system, which includes: A data acquisition unit is configured to acquire relevant data of a scene to be monitored. A feature encoding unit is configured to perform feature extraction and encoding on the relevant data to obtain a three-dimensional attention matrix. A processing unit is configured to process the three-dimensional attention matrix using an improved MOE module to obtain a multi-modal fusion result. A prediction unit is configured to comprehensively analyze each risk factor based on the multi-modal fusion result to obtain a risk type, a risk size, and a risk trend, and to obtain multi-dimensional information. An output unit is configured to output the multi-dimensional information.

[0016] Compared with the prior art, the application has the following beneficial effects: the application acquires relevant data of a scene to be monitored and performs feature extraction and encoding to generate a three-dimensional attention matrix. Then, the improved MOE module is used to process the matrix to realize multi-modal data fusion, and each risk factor is comprehensively analyzed to obtain multi-dimensional information such as a risk type, a size, and a trend. This method can effectively capture complex spatio-temporal coupling relationships, overcome the limitations of long-term dependence on modeling and flexibility of expert decision-making in the prior art, improve the prediction and response capability of risks in the field of public safety, and enhance the dynamic adaptability and intelligent decision support of the system.

[0017] The application will be described in further detail below with reference to the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0019] Figure 1 A flowchart of the improved MOE risk calculation method provided by the embodiment of the present application is shown in the figure. Figure 2 A sub-flowchart of the improved MOE risk calculation method provided by the embodiment of the present application is shown in the figure. Figure 3 A flowchart of the improved MOE provided by the embodiment of the present application is shown in the figure. Figure 4 A schematic block diagram of the improved MOE risk calculation system provided by the embodiment of the present application is shown in the figure. Figure 5 A schematic block diagram of the processing unit of the improved MOE risk calculation system provided by the embodiment of the present application is shown in the figure. Figure 6 A schematic block diagram of the computer device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort belong to the scope of protection of the present application.

[0021] It should be understood that, when used in the specification and the appended claims, the terms “comprise” and “include” indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0022] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms “a”, “an” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should be further understood that the term "and / or" used in the description and the appended claims of the present application refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0024] Reference is made to Figure 1 , Figure 1 The improved MOE risk calculation method provided by the embodiments of the present application is shown in the schematic flowchart. The improved MOE risk calculation method is applied in a server. The server interacts with sensors to improve the risk prediction and response capability in the field of public security by dynamically processing complex spatio-temporal data. The method first extracts multi-scale features from relevant data through an improved spatio-temporal coupling attention module, constructs a three-dimensional attention matrix, and combines spatial position weighting and time decay function to solve the difficulties of traditional technologies in modeling spatio-temporal coupling relationship and long-term dependence modeling. Subsequently, through the improved MOE module and the multi-modal fusion method, the multi-head attention mechanism and dynamic weight adjustment are used to flexibly select and adjust the expert network, so as to effectively cope with the diversity and complexity of risk factors. The method can overcome the rigid problem of traditional expert decision-making, realize more accurate and efficient risk analysis, and provide more intelligent prediction and response scheme for public security.

[0025] Figure 1 The flowchart of the improved MOE risk calculation method provided by the embodiments of the present application is shown in the schematic flowchart. As shown in Figure 1 , the method includes the following steps S110 to S150.

[0026] S110, acquiring relevant data of a scene to be monitored.

[0027] In the present embodiment, the relevant data includes IoT sensor data, video data, operator data, government data, traffic data, environmental data, and classified data, etc.

[0028] Specifically, acquiring relevant data of a scene to be monitored is the first step in the risk calculation method, and the purpose is to provide basic data support for subsequent analysis and processing. The types of relevant data are diverse, covering multiple fields, including the following categories: IoT sensor data: Internet of Things (IoT) sensors are sensors deployed in various devices to collect and transmit real-time data. These sensors can monitor environmental, temperature, humidity, pressure, gas concentration, etc. In the field of public security, IoT sensors are commonly used in intelligent monitoring, environmental monitoring, etc. For example, sensors installed in intelligent transportation systems can obtain real-time data such as road traffic flow, vehicle speed, etc., and environmental monitoring can obtain air quality data, etc.

[0029] Video Data: Video surveillance systems collect real-time video data through cameras, monitoring cities, traffic, public places, etc. Video data can provide a visual observation of the scene, helping to analyze traffic conditions, event development, public safety incidents, etc. Combined with computer vision technology, specific actions, behaviors, etc. Information can be extracted from the video to assist in risk prediction and early warning.

[0030] Operator Data: Operator data usually refers to network usage data, communication records, traffic data, etc. collected from telecommunications operators. These data can reflect regional communication activities, traffic peaks, group behaviors, etc. In public safety, operator data can help analyze communication congestion, group gatherings, etc. Especially in the event of large-scale events, it can assist in monitoring crowd flow and behavior.

[0031] Government Data: Government data usually refers to data provided by relevant departments, including public policies, laws and regulations, social security data, public event reports, etc. This type of data can help decision-makers understand social conditions, the background of public events, historical risks, etc. to provide important information support for risk prediction. For example, disaster warning, sudden public health events, etc. Government data can directly affect the conclusions of risk analysis.

[0032] Traffic Data: Traffic data refers to road traffic flow, speed, traffic accident records, traffic control, etc. It can reflect the real-time traffic situation and is an indispensable data type in the field of public safety. For example, through real-time traffic flow monitoring, traffic bottlenecks, traffic accident risks, and even accident response scheduling can be predicted.

[0033] Environmental Data: Environmental data refers to information about environmental conditions, such as meteorological data (such as temperature, humidity, wind speed, etc.), air quality data, pollutant concentration, natural disaster monitoring data, etc. This type of data is crucial for assessing the potential risks of sudden natural disasters and climate change on public safety. For example, extreme weather (such as storms, high temperatures, etc.) may cause casualties, traffic accidents, etc. Real-time monitoring of environmental data can help with early warning and response.

[0034] Sensitive Data: Sensitive data refers to data in sensitive fields such as security and economic security. These data usually require special protection and management, but in certain situations, the use of sensitive data is crucial for accurate risk assessment. Although the use of sensitive data is strictly limited, the information it provides is often critical.

[0035] By acquiring and integrating these different types of relevant data, the system can build a comprehensive risk assessment foundation, providing reliable information support for subsequent multi-modal data fusion, risk prediction, and emergency response. These data are collected, stored, and analyzed through various means to ensure accurate information in complex real-time scenarios, providing stronger decision support for public safety.

[0036] S120, feature extraction and encoding are performed on the relevant data to obtain a three-dimensional attention matrix.

[0037] In this embodiment, the three-dimensional attention matrix is obtained by using an improved spatio-temporal coupled attention module for feature extraction and encoding of input data. The three-dimensional attention matrix not only considers the spatial information of the data, but also combines the decay effect in the time dimension, so it can accurately capture the risk features under different time and space conditions.

[0038] Specifically, the relevant data is subjected to multi-scale feature extraction using an improved spatio-temporal coupled attention module, and a three-dimensional attention matrix is constructed by combining spatial position weighting and a time decay function.

[0039] The query is formed by splicing the task embedding and the environment embedding, and the relevant data is projected to generate keys and values, and a three-dimensional attention matrix is constructed by combining spatial position weighting and a time decay function. The task embedding includes potential risk types, and the environment embedding includes static risk factors and dynamic risk factors.

[0040] The role of the spatio-temporal coupled attention module is to perform multi-scale feature extraction by combining spatial and temporal characteristics. This module not only considers each input feature itself, but also performs more detailed feature modeling through the coupling of spatial and temporal information, generating a feature representation with both spatial and temporal sensitivity.

[0041] Task embedding (E task ): represents potential risk types and task objectives. The task embedding can be regarded as a representation of a specific risk task, such as a security check task, a traffic flow prediction task, etc. This task embedding carries context information related to the task.

[0042] Environment embedding (E env ): represents static and dynamic risk factors in the environment. Dynamic risk factors refer to environmental conditions that change over time (such as weather, time period, etc.), while static risk factors may be related to geographic location or specific facilities (such as densely populated areas). The environment embedding combines these spatial and temporal factors.

[0043] By concatenating the task embedding with the environment embedding, we form the query (Q): Q = Etask? Eenv. This concatenation ensures that both the task and environment information are considered simultaneously and play a role in the subsequent attention mechanism.

[0044] The relevant data refers to the input data collected from multiple data sources, such as sensor data, video data, environmental data, etc. To effectively process in the model, these data are transformed into key (K) and value (V) forms through projection operations.

[0045] The concatenated input data (e.g., sensor data, video data, etc.) are linearly transformed to generate new keys (K) and values (V), representing different feature information: Here, and are the sensor and video data, respectively. The concatenated data will jointly generate key-value pairs. The projected data represents the features in different modalities (e.g., video, sensor, etc.).

[0046] When generating the attention matrix, the spatial and temporal factors have a significant impact on risk prediction. Therefore, the model introduces spatial position weighting and temporal decay functions: Spatial weighting (Φ spatial ): This part assigns different weights based on the position characteristics in space. For example, in a security checkpoint scenario, certain areas (such as entrances or densely populated areas) may have higher risks, so these positions will be assigned greater weights when calculating attention weights.

[0047] Temporal decay function (Ψ temporal ): The temporal decay function is used to decay historical data, ensuring that the model is more sensitive to recent data when calculating risks. For example, recent abnormal vibration data may better reflect the current risk, while the influence of historical data is relatively small.

[0048] These two weighting factors jointly affect the generated three-dimensional attention matrix, making it not only capture important positions in space but also identify the importance of time.

[0049] The final three-dimensional attention matrix is calculated by the following formula: where, represents the spatial attention weight, such as the high-risk weight of a crowded area; represents the temporal decay function, such as the higher weight of recent vibration data than historical data; is a scaling factor to prevent the inner product from being too large. is the inner product between the query and the key, measuring their similarity.

[0050] By coupling spatial and temporal attention mechanisms, a comprehensive feature matrix containing spatial, temporal, and task environment information can be obtained. This matrix provides multi-dimensional context information to the model, helping to make more accurate risk predictions. Spatial weighting and temporal decay mechanisms ensure that the model can flexibly adjust the attention level of various features according to the current risk scenario.

[0051] The steps of S120 successfully construct a three-dimensional attention matrix that reflects spatial and temporal factors by combining task embeddings, environment embeddings, and spatio-temporal attention mechanisms. This process effectively improves the accuracy and generalization ability of risk calculation, providing strong support for subsequent expert decision-making and risk prediction.

[0052] S130, the three-dimensional attention matrix is processed using an improved MOE module to obtain a multi-modal fusion result.

[0053] In this embodiment, the multi-modal fusion result refers to the comprehensive output obtained by combining information from different data sources (modalities) after a series of processing. Specifically, these modalities can be different types of sensor data, video data, environmental data, etc. By fusing the information of these modalities, the model can deeply understand the data in multiple dimensions and generate a more comprehensive output result.

[0054] In this process, the purpose of multi-modal fusion is: Integrate heterogeneous information: Each modality provides different types of information, such as sensor data may provide spatial information, while video data may provide dynamic behavior information. By fusing these modalities, a more comprehensive perspective can be obtained to help the model understand more complex situations.

[0055] Enhance accuracy and generalization ability: By utilizing data from multiple modalities, the model can reduce the bias or noise brought by a single modality, improving the accuracy and stability of the final result. For example, a certain modality may not be able to provide sufficient information in some cases, and other modalities can make up for this deficiency, thereby improving overall performance.

[0056] Risk prediction: In this embodiment, the specific application of the multi-modal fusion result may be risk prediction in complex environments. By fusing information from different modalities, the model can better identify abnormal patterns (such as traffic accidents, crowd gathering, etc.) and give more accurate risk assessment.

[0057] In short, the multi-modal fusion result is the comprehensive output obtained by fusing data from different modalities (such as sensors, videos, etc.) after processing, aiming to provide a more accurate, comprehensive, and multi-dimensional prediction result or analysis result.

[0058] In an embodiment, please refer toFigure 2 The step S130 can include steps S131-S132.

[0059] S131, amplify the abnormal features using a multi-modal gating function on the three-dimensional attention matrix, and select the expert network combined with a dynamic weight adjustment mechanism.

[0060] In this embodiment, the multi-modal gating function is used to process the three-dimensional attention matrix using a multi-head attention mechanism.

[0061] In this embodiment, the dynamic weight adjustment mechanism includes using KL divergence to measure the difference between each expert network and the environment distribution to adjust the weight of the expert network.

[0062] Specifically, in this step, we process the aforementioned three-dimensional attention matrix, focusing on amplifying the abnormal features. The three-dimensional attention matrix A(h,i,j)A(h,i,j) includes information of spatial and temporal dimensions, which can effectively capture the heterogeneity features in complex environments (such as crowd gathering areas or abnormal traffic patterns).

[0063] The identification and amplification of abnormal features are carried out through a multi-modal gating function. The form of this function is: ; where Q, K, V represent the query, key and value vectors, combining sensor data, video data, environmental data and other information. And is a gating function based on input data, which generates weights through Softmax operation to amplify or reduce the corresponding features.

[0064] When amplifying abnormal features, special attention should be paid to the role of the risk sensitivity coefficient (β), which adjusts the importance of abnormal features, so that these features are strengthened in risk calculation. For example, when a sudden event (such as an emergency traffic accident or abnormal crowd gathering) occurs in a certain period of time, these features will be given higher weights, thereby affecting the subsequent risk prediction.

[0065] With the changes of data and environment, the selection and allocation of expert networks should also be dynamic. This means that at each input data change, the dynamic weight of the expert network needs to be adjusted according to the changes of the environment and the task .

[0066] In this process, the difference between the input of each expert network and the environment distribution is measured by KL divergence, so as to dynamically adjust the expert selection.

[0067] ; where represents the distribution of the kth expert network, represents the environment distribution, is the adjustment factor. This dynamic mechanism helps to select the most suitable expert network for the current task while avoiding unnecessary computational overhead.

[0068] S132, using the expert network for multi-modal feature fusion to obtain a multi-modal fusion result.

[0069] In this embodiment, the weight of each expert network is calculated by the improved MOE module, and the outputs of multiple expert networks are weighted, and the three-dimensional attention matrix is fused by the Sigmoid function and the risk sensitivity coefficient to obtain a multi-modal fusion result.

[0070] In step S132, the expert network is weighted by its respective weight The data from multiple modalities are fused. Different expert networks focus on different types of features, and these expert networks are fused by weighting to ultimately make a comprehensive evaluation of the input data (such as sensor data, video data, environmental data, etc.).

[0071] Since each expert network has different processing capabilities for specific modal features, using these networks can achieve more accurate multi-modal feature fusion and handle more complex tasks.

[0072] In the process of multi-modal feature fusion, the following formula is used for fusion: ; is the processing result of the kth expert network on the input data X; is the risk sensitivity coefficient, which is used to amplify or reduce specific risk features; is the dynamic weight of the expert, which is adjusted based on the calculation of KL divergence.

[0073] In this step, through the nonlinear activation of the Sigmoid function, the model's ability to recognize abnormal patterns is enhanced, and the balance of the processing results of different expert networks is ensured. The final fusion result can effectively improve the generalization ability and accuracy of the model.

[0074] After multi-modal feature fusion, the result is a comprehensive risk assessment output that can fully reflect the multi-modal information of the input data and provide different dimensional risk prediction results according to the task requirements. This result not only considers spatial and temporal factors, but also combines different types of data sources for deep reasoning, thereby providing support for decision-making in complex environments.

[0075] In step S130, the three-dimensional attention matrix is processed using the improved MOE module combined with a multi-modal gating function and a dynamic weight adjustment mechanism. First, the gating function amplifies abnormal features to select the most suitable expert network for decision-making. Subsequently, the expert network fuses multi-modal features, and the final multi-modal fusion result can more accurately predict risks, improving the calculation accuracy and generalization ability in complex scenarios.

[0076] The parameters of each of the above-described expert networks are adjusted by the learning rate and the gradient of the task loss function.

[0077] ; wherein, represents the parameters of the i-th expert network at the t-th iteration; is the parameter of the i-th expert network after the (t+1)-th iteration; α is the learning rate, controlling the magnitude of each parameter update. is a gating function that determines the activation level of the expert network given input x. is the gradient of the cross-entropy loss function of the risk prediction task with respect to the expert network parameters θ.

[0078] In this way, the parameters of the expert network are adjusted according to the gradient of the task loss at each iteration, thereby continuously optimizing to improve the performance of the risk prediction task.

[0079] S140, based on the multi-modal fusion result, comprehensive analysis is performed on each risk factor to obtain the risk type, risk size and risk trend to obtain multi-dimensional information.

[0080] In this embodiment, this step mainly performs comprehensive analysis based on the fused multi-modal feature data, involving the following key operations: After the aforementioned multi-modal data fusion (such as IoT sensor data, video data, traffic data, etc.), through the improved MOE architecture, the system has dynamically fused information from different data sources. In this step, all fused features will be further analyzed according to their weights (calculated by the dynamic adjustment mechanism) to determine the contribution of different risk factors to the overall risk.

[0081] This analysis process involves multi-dimensional feature processing, such as the spatio-temporal coupling attention module that combines spatial and temporal attention weights to help the model understand the influence of each feature in different time and spatial dimensions.

[0082] Using the dynamic differentiable routing mechanism in the expert network, each input risk data is classified by combining task embedding (latent risk type) and environment embedding (static and dynamic risk factors). Each input modality (e.g., video data, traffic data, etc.) is routed to the corresponding expert network for processing.

[0083] During the multi-modal fusion process, based on the output of each expert network, the model will determine the type of risk according to the predetermined risk classification criteria (such as safety risk, traffic risk, environmental risk, etc.).

[0084] Through the fused multi-modal data, a risk calculation formula (such as R_total) is used to integrate the risk values output by each expert network. Specifically, the fused features will be amplified by a risk-sensitive coefficient (β), focusing on those features with larger abnormal values, which usually represent higher risks.

[0085] The fusion of multi-modal features will perform weighted calculation on risks according to different data sources (such as video monitoring data, sensor data, etc.), and the weights will be dynamically adjusted according to the reliability and timeliness of the data sources.

[0086] With the help of the spatio-temporal coupling attention module, the system can perform weight decay on historical data, especially giving higher weights to recent data (such as real-time monitoring data). This temporal decay function (Ψ_temporal) can ensure that the model pays more attention to the risk factors in the current environment rather than outdated data.

[0087] The generation of risk trends is based on time series data and the changes in historical risk calculations, combined with current environmental and task embedding data, to predict future risk trends.

[0088] In the final output of this step, the system will generate a comprehensive risk assessment report based on the above analysis, taking into account the contribution of various risk factors. The report content includes: Risk type: specific categories such as safety risk, environmental risk, traffic risk, etc.

[0089] Risk size: the severity or scale of impact of each risk.

[0090] Risk trend: the trend of risk changes over time (up or down).

[0091] S150, output the multi-dimensional information.

[0092] In this embodiment, based on S140, the step of S150 aims to output and visualize the aforementioned analysis results, so that relevant personnel can make decisions based on the information provided by the system.

[0093] The output of risk types can be displayed through intuitive classification results, such as marking different risk types as "high risk", "medium risk" and "low risk", or using more refined classification standards to display different labels according to risk types (such as safety risk, traffic congestion risk, etc.).

[0094] The system will provide specific numerical values or risk levels for each risk according to the calculated risk values, which can be a score (for example, a value between 0 and 1) or some quantitative risk assessment result to reflect the urgency and possible impact range of the risk in the current environment.

[0095] This output can be presented in the form of visual charts, such as bar charts, radar charts, etc., to help relevant personnel quickly identify the size of the risk.

[0096] The output of risk trends will show the possible development direction of each risk in the future period of time, which can be visualized through time series charts, curves, etc., so that decision makers can predict the dynamic changes of risks.

[0097] For example, it can show the rising or falling trend of risk index in the future period of time, or present the periodic change trend, so as to help the emergency response team to formulate corresponding prevention or mitigation measures.

[0098] All risk analysis results will be integrated into a complete multi-dimensional risk assessment report and provided to users through appropriate interfaces or interfaces. The report can be real-time or generated periodically, and different information granularity can be provided for different user roles.

[0099] The ultimate goal of this report is to help decision makers understand the current risk status and provide data support for subsequent emergency response, resource allocation and strategy adjustment.

[0100] In summary, the two steps of S140 and S150 are the core links of the system from multi-modal data fusion to the final risk assessment output, ensuring that various risk factors are accurately and dynamically analyzed and provided to decision makers in multi-dimensional form.

[0101] In the method of the embodiment, as shown in Figure 3 The Feed-Forward Network (FFN) module in the traditional Transformer architecture is replaced by an improved MOE (Mixture of Experts) architecture. The improved MOE architecture has the following characteristics: In the MOE architecture, a dynamically differentiable multi-modal gating function is introduced, which can dynamically adjust the weights of each expert network based on the input multi-modal data (e.g. visual, sensor data, etc.). This gating function provides flexible control between different tasks and data sources.

[0102] In the multi-expert computing layer, a spatio-temporal coupled attention module is used, which combines spatial attention functions and temporal decay coefficients. The spatial attention function helps the model capture the importance of different spatial positions, while the temporal decay coefficient ensures that the model places more emphasis on recent data when performing risk assessment and reduces the impact of outdated data.

[0103] In the multi-modal fusion layer, a risk sensitivity coefficient is introduced, which amplifies abnormal risk factors to ensure that the system can respond sensitively to key and sudden high-risk events during risk assessment. These abnormal data will be given higher weights to ensure that the system can quickly identify and respond to potential serious risks.

[0104] Finally, through this improved MOE architecture, different modal data can be effectively fused to provide more accurate risk calculation output for decision-makers to better assess and respond to risks.

[0105] The improved MOE risk calculation method described above obtains relevant data of the scene to be monitored and performs feature extraction and encoding to generate a three-dimensional attention matrix. Then, the improved MOE module is used to process the matrix, realize the fusion of multi-modal data, and then analyze each risk factor to obtain multi-dimensional information such as risk type, size and trend. This method can effectively capture complex spatio-temporal coupling relationships, overcome the limitations of existing technologies in long-term dependence modeling and flexibility of expert decision-making, improve the prediction and response capabilities of risks in the field of public safety, and enhance the dynamic adaptability and intelligent decision support of the system.

[0106] Figure 4 is a schematic block diagram of an improved MOE risk calculation system 300 provided by an embodiment of the present application. As Figure 4 shown, corresponding to the improved MOE risk calculation method described above, the present application also provides an improved MOE risk calculation system 300. The improved MOE risk calculation system 300 includes units for executing the improved MOE risk calculation method described above, and the system can be configured in a server. Specifically, please refer to Figure 4 , the improved MOE risk calculation system 300 includes a data acquisition unit 301, a feature encoding unit 302, a processing unit 303, a prediction unit 304 and an output unit 305.

[0107] The data acquisition unit 301 is configured to acquire related data of a scene to be monitored; the feature encoding unit 302 is configured to perform feature extraction and encoding on the related data to obtain a three-dimensional attention matrix; the processing unit 303 is configured to perform processing on the three-dimensional attention matrix by using an improved MOE module to obtain a multi-modal fusion result; the prediction unit 304 is configured to comprehensively analyze each risk factor according to the multi-modal fusion result to obtain a risk type, a risk size and a risk trend, and to obtain multi-dimensional information; and the output unit 305 is configured to output the multi-dimensional information.

[0108] In an embodiment, the feature encoding unit 302 is configured to perform multi-scale feature extraction on the related data by using an improved spatio-temporal coupling attention module, and construct a three-dimensional attention matrix by combining a spatial position weighting and a time decay function.

[0109] In an embodiment, the feature encoding unit 302 is configured to form a query by splicing a task embedding and an environment embedding, generate keys and values by performing a projection operation on the related data, and construct a three-dimensional attention matrix by combining a spatial position weighting and a time decay function.

[0110] In an embodiment, as shown in Figure 5 The processing unit 303 includes an amplification sub-unit 3031 and a fusion sub-unit 3032.

[0111] The amplification sub-unit 3031 is configured to amplify and process abnormal features of the three-dimensional attention matrix by using a multi-modal gating function, and select an expert network by combining a dynamic weight adjustment mechanism; and the fusion sub-unit 3032 is configured to perform multi-modal feature fusion by using the expert network to obtain a multi-modal fusion result.

[0112] In an embodiment, the fusion sub-unit 3032 is configured to calculate the weight of each expert network by using an improved MOE module, weight the outputs of a plurality of the expert networks, and fuse the three-dimensional attention matrix by using a Sigmoid function and a risk sensitivity coefficient to obtain a multi-modal fusion result.

[0113] It should be noted that the specific implementation process of the improved MOE risk calculation system 300 and each unit can be clearly understood by those skilled in the art, and can be referred to the corresponding description in the foregoing method embodiments. For the convenience and brevity of description, it will not be described here.

[0114] The improved MOE risk calculation system 300 can be implemented in the form of a computer program, which can run on a computer device as shown in Figure 6 .

[0115] Please refer to Figure 6 ,Figure 6 is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0116] Referring to Figure 6 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, wherein the memory can include a non-volatile storage medium 503 and an internal memory 504.

[0117] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform an improved MOE risk calculation method.

[0118] The processor 502 is configured to provide computing and control capabilities to support the operation of the entire computer device 500.

[0119] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503, which, when executed by the processor 502, causes the processor 502 to perform an improved MOE risk calculation method.

[0120] The network interface 505 is configured to perform network communication with other devices. Those skilled in the art can understand that Figure 6 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device 500 to which the scheme of the present application is applied. The specific computer device 500 can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] The processor 502 is configured to run the computer program 5032 stored in the memory to implement the following steps: Obtain relevant data of a scene to be monitored; perform feature extraction and encoding on the relevant data to obtain a three-dimensional attention matrix; process the three-dimensional attention matrix using an improved MOE module to obtain a multi-modal fusion result; comprehensively analyze each risk factor according to the multi-modal fusion result to obtain a risk type, a risk size, and a risk trend to obtain multi-dimensional information; and output the multi-dimensional information.

[0122] In an embodiment, when implementing the step of performing feature extraction and encoding on the relevant data to obtain a three-dimensional attention matrix, the processor 502 specifically implements the following steps: The improved spatio-temporal coupling attention module is used for multi-scale feature extraction on the related data, and a three-dimensional attention matrix is constructed by combining spatial position weighting and a time decay function.

[0123] In an embodiment, when implementing the step of using the improved spatio-temporal coupling attention module for multi-scale feature extraction on the related data, and constructing a three-dimensional attention matrix by combining spatial position weighting and a time decay function, the processor 502 specifically implements the following steps: The query is formed by splicing the task embedding and the environment embedding, and the related data is projected to generate keys and values, and a three-dimensional attention matrix is constructed by combining spatial position weighting and a time decay function.

[0124] The task embedding includes a potential risk type, and the environment embedding includes static risk factors and dynamic risk factors.

[0125] In an embodiment, when implementing the step of using the improved MOE module to process the three-dimensional attention matrix to obtain a multi-modal fusion result, the processor 502 specifically implements the following steps: The multi-modal gating function is used to amplify the processing of abnormal features on the three-dimensional attention matrix, and a dynamic weight adjustment mechanism is used to select an expert network; the multi-modal feature fusion is performed by using the expert network to obtain a multi-modal fusion result.

[0126] The multi-modal gating function is used to process the three-dimensional attention matrix using a multi-head attention mechanism.

[0127] The dynamic weight adjustment mechanism includes using KL divergence to measure the difference between each expert network and the environment distribution to adjust the weight of the expert network.

[0128] The parameters of each expert network are adjusted by the learning rate and the gradient of the task loss function.

[0129] In an embodiment, when implementing the step of using the expert network to perform multi-modal feature fusion to obtain a multi-modal fusion result, the processor 502 specifically implements the following steps: The weight of each expert network is calculated by the improved MOE module, the outputs of multiple expert networks are weighted, and the three-dimensional attention matrix is fused by a Sigmoid function and a risk sensitivity coefficient to obtain a multi-modal fusion result.

[0130] It should be understood that in the embodiments of the present application, the processor 502 can be a central processing unit 303 (CPU), and the processor 502 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0131] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by instructing related hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in the computer system to realize the process steps of the above-mentioned embodiment of the method.

[0132] Therefore, the present application also provides a storage medium. The storage medium can be a computer readable storage medium. The storage medium stores a computer program, wherein the computer program is executed by a processor to make the processor execute the following steps: Obtaining related data of a scene to be monitored; performing feature extraction and encoding on the related data to obtain a three-dimensional attention matrix; processing the three-dimensional attention matrix using an improved MOE module to obtain a multi-modal fusion result; comprehensively analyzing each risk factor according to the multi-modal fusion result to obtain a risk type, a risk size and a risk trend to obtain multi-dimensional information; and outputting the multi-dimensional information.

[0133] In an embodiment, when the processor executes the computer program to realize the step of performing feature extraction and encoding on the related data to obtain a three-dimensional attention matrix, the processor specifically realizes the following steps: Using an improved space-time coupled attention module to perform multi-scale feature extraction on the related data, and constructing a three-dimensional attention matrix by combining spatial position weighting and time decay function.

[0134] In an embodiment, when the processor executes the computer program to realize the step of using an improved space-time coupled attention module to perform multi-scale feature extraction on the related data, and constructing a three-dimensional attention matrix by combining spatial position weighting and time decay function, the processor specifically realizes the following steps: The query is formed by splicing the task embedding and the environment embedding, and the related data is projected to generate keys and values, and a three-dimensional attention matrix is constructed by combining a spatial position weighting and a time decay function.

[0135] The task embedding includes a potential risk type, and the environment embedding includes static risk factors and dynamic risk factors.

[0136] In an embodiment, when the processor implements the step of processing the three-dimensional attention matrix by using the improved MOE module to obtain a multi-modal fusion result by executing the computer program, the following steps are implemented: An abnormal feature is amplified by using a multi-modal gating function on the three-dimensional attention matrix, and a dynamic weight adjustment mechanism is used to select an expert network; and multi-modal feature fusion is performed by using the expert network to obtain a multi-modal fusion result.

[0137] The multi-modal gating function is used to process the three-dimensional attention matrix by using a multi-head attention mechanism.

[0138] The dynamic weight adjustment mechanism includes using a KL divergence to measure the difference between each expert network and an environment distribution to adjust the weight of the expert network.

[0139] The parameters of each expert network are adjusted by a learning rate and a gradient of a task loss function.

[0140] In an embodiment, when the processor implements the step of performing multi-modal feature fusion by using the expert network to obtain a multi-modal fusion result by executing the computer program, the following steps are implemented: The weight of each expert network is calculated by using an improved MOE module, the outputs of multiple expert networks are weighted, and the three-dimensional attention matrix is fused by using a Sigmoid function and a risk sensitivity coefficient to obtain a multi-modal fusion result.

[0141] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, and various computer readable storage media that can store program codes.

[0142] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0143] In several embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are only illustrative. For example, the division of each unit is only a logical functional division, and actual implementation can have another division. For example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not executed.

[0144] The steps in the method embodiments of the present application can be adjusted, combined and deleted in sequence according to actual needs. The units in the system embodiments of the present application can be combined, divided and deleted according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in a processing unit 303, or each unit can exist physically, or two or more units can be integrated in one unit.

[0145] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a terminal or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0146] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An improved method for calculating the risk of MOE, characterized in that, include: Obtain relevant data for the scene to be monitored; Feature extraction and encoding are performed on the relevant data to obtain a three-dimensional attention matrix; The three-dimensional attention matrix is ​​processed using an improved MOE module to obtain multimodal fusion results; Based on the multimodal fusion results, a comprehensive analysis of each risk factor is performed to obtain the risk type, risk magnitude, and risk trend, thereby obtaining multidimensional information. Output the multi-dimensional information.

2. The improved MOE risk calculation method according to claim 1, characterized in that, The step of extracting and encoding features from the relevant data to obtain a three-dimensional attention matrix includes: The relevant data are subjected to multi-scale feature extraction using an improved spatiotemporal coupled attention module, and a three-dimensional attention matrix is ​​constructed by combining spatial location weighting and a time decay function.

3. The improved MOE risk calculation method according to claim 2, characterized in that, The process involves using an improved spatiotemporal coupled attention module to extract multi-scale features from the relevant data, and constructing a three-dimensional attention matrix by combining spatial location weighting and a time decay function, including: The query is formed by concatenating task embedding and environment embedding, and the relevant data is projected to generate keys and values. A three-dimensional attention matrix is ​​constructed by combining spatial location weighting and time decay functions.

4. The improved MOE risk calculation method according to claim 1, characterized in that, The process of processing the three-dimensional attention matrix using an improved MOE module to obtain multimodal fusion results includes: The three-dimensional attention matrix is ​​amplified using a multimodal gating function, and an expert network is selected by combining a dynamic weight adjustment mechanism. The expert network is used to perform multimodal feature fusion to obtain the multimodal fusion result.

5. The improved MOE risk calculation method according to claim 4, characterized in that, The multimodal gating function is used to process the three-dimensional attention matrix using a multi-head attention mechanism.

6. The improved MOE risk calculation method according to claim 4, characterized in that, The dynamic weight adjustment mechanism includes using KL divergence to measure the difference between each expert network and the environment distribution in order to adjust the weights of the expert networks.

7. The improved MOE risk calculation method according to claim 4, characterized in that, The process of using the expert network to perform multimodal feature fusion to obtain a multimodal fusion result includes: The weights of each expert network are calculated using an improved MOE module, and the outputs of multiple expert networks are weighted. The three-dimensional attention matrix is ​​then fused using a sigmoid function and a risk sensitivity coefficient to obtain a multimodal fusion result.

8. The improved MOE risk calculation method according to claim 4, characterized in that, The parameters of each expert network are adjusted by the learning rate and the gradient of the task loss function.

9. The improved MOE risk calculation method according to claim 3, characterized in that, The task embedding includes potential risk types, and the environment embedding includes static risk factors and dynamic risk factors.

10. An improved MOE risk calculation system, characterized in that, include: The data acquisition unit is used to acquire relevant data about the scene to be monitored. The feature encoding unit is used to extract and encode features from the relevant data to obtain a three-dimensional attention matrix; The processing unit is used to process the three-dimensional attention matrix using an improved MOE module to obtain a multimodal fusion result; The prediction unit is used to perform a comprehensive analysis of various risk factors based on the multimodal fusion results to obtain risk type, risk magnitude and risk trend, so as to obtain multi-dimensional information; The output unit is used to output the multi-dimensional information.

Images