Natural disaster risk identification method, device and equipment based on federated learning and storage medium

Abstract

The invention discloses a natural disaster risk identification method, device and equipment based on federated learning, and a storage medium, and relates to the technical field of risk prevention, and the method comprises the steps: collecting and desensitizing natural disaster data, and obtaining initial multi-modal data; based on semantic features obtained by extracting the initial multi-modal data, aligning the initial multi-modal data, and processing a missing value of the initial multi-modal data to obtain target multi-modal data; fusing the spatial-temporal features based on the target multi-modal data to obtain a fusion result, defining nodes and edges based on modal spatial-temporal features in the fusion result, connecting the nodes and the edges, and aggregating neighborhood information of a connection result to determine a to-be-processed graph based on the neighborhood information and the connection result; a federated learning framework is utilized and a model is trained based on a to-be-processed graph, model parameters are issued to an edge server to aggregate the model parameters, aggregation results are issued to a central server to generate a target risk identification model based on the aggregation results, and the efficiency of identifying natural disaster risks is improved.

Description

Technical Field

[0001] This invention relates to the field of hazard prevention technology, and in particular to a method, apparatus, equipment and storage medium for natural disaster hazard identification based on federated learning. Background Technology

[0002] Currently, with the continuous development of technology, the prevention and response to various hazards are receiving increasing attention. Traditional hazard prevention systems often have many limitations. For example, single-dimensional monitoring methods, relying solely on a single type of sensor or monitoring method, cannot comprehensively capture risk information in complex scenarios. Taking industrial production scenarios as an example, if only temperature sensors are used to monitor equipment operation, abnormal vibrations caused by mechanical failures may not be detected in time, leading to potential hazards going undetected.

[0003] Meanwhile, traditional centralized processing architectures suffer from significant latency in their data processing flow, from data acquisition and transmission to analysis at the centralized processing center and finally to early warning decisions. In high-risk scenarios where response speed is extremely critical, such as chemical leaks in chemical production or overheating and fires in power systems, even a delay of just a few seconds can lead to a rapid escalation of danger and irreparable losses.

[0004] Furthermore, in today's context of increasingly close cross-institutional and cross-sectoral collaboration, data sharing and cooperation among different organizations are crucial for more comprehensive and efficient risk prevention. However, data privacy protection has become a major obstacle to data sharing. Data from various organizations often contains a large amount of sensitive information; for example, factory layout diagrams involve production processes and trade secrets, and employee location data involves personal privacy. If this sensitive information is not effectively protected during data sharing, its leakage will have serious negative impacts on organizations and individuals.

[0005] As can be seen from the above, how to improve the efficiency of natural disaster hazard identification in the process of federated learning-based natural disaster hazard identification is an urgent problem to be solved. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for natural disaster hazard identification based on federated learning, which can improve the efficiency of natural disaster hazard identification in the process of natural disaster hazard identification based on federated learning. The specific solution is as follows:

[0007] Firstly, this application provides a method for identifying natural disaster risks based on federated learning, including:

[0008] Collect natural disaster data to be identified, including text data, image data, and sensor data, and desensitize the text data, image data, and sensor data respectively to obtain initial multimodal data;

[0009] Textual semantic features are extracted from the initial multimodal data to obtain semantic features. Based on each semantic feature, the initial multimodal data are aligned to a uniform temporal resolution. Then, the missing values ​​of the initial multimodal data are processed to obtain the target multimodal data.

[0010] The spatiotemporal features of the target multimodal data are determined using a preset spatiotemporal attention mechanism. Then, the spatiotemporal features are fused using a preset attention weight fusion mechanism to obtain a fusion result. Nodes and edges are defined based on the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result. Cross-modal node connections are made between the nodes and edges using a preset semantic matching mechanism. Then, neighborhood information of the connection results is aggregated using a preset graph neural network to determine the graph to be processed based on the neighborhood information and the connection results.

[0011] An initial hazard identification model is trained using a pre-defined federated learning framework based on the graph to be processed to obtain a hazard identification model to be processed. The model parameters corresponding to the hazard identification model to be processed are then sent to a pre-defined edge server, which aggregates the model parameters and sends the aggregation results to a pre-defined central server. The central server then generates a target hazard identification model based on the aggregation results and uses the target hazard identification model to identify natural disaster hazards.

[0012] Optionally, the data to be identified, including text data, image data, and sensor data, comprises:

[0013] The system synchronizes each acquisition sensor in time using a preset time synchronization protocol, and uses a preset log collection tool to capture log files corresponding to the natural disaster videos to be identified in real time. Then, it obtains a structured report corresponding to the natural disaster videos to be identified through the application programming interface.

[0014] The video of the natural disaster to be identified is retrieved through a streaming media protocol and based on the structured report. Keyframes are extracted from the video to obtain image data, and text data is determined based on the image data. The image data includes surveillance video and thermal imaging images, and the text data includes equipment logs and maintenance reports.

[0015] The sampling frequency corresponding to the sensor data is adjusted based on the video frame rate corresponding to the natural disaster to be identified, so as to obtain the target sampling frequency, and the sensor data corresponding to the natural disaster to be identified is collected using the sensor and based on the target sampling frequency; the sensor data includes temperature, pressure and vibration measurement data.

[0016] Optionally, the step of desensitizing the text data, the image data, and the sensor data to obtain initial multimodal data includes:

[0017] The sensitive fields in the text data are replaced with fixed values ​​to obtain the replacement result. The replacement result is then salted using a preset secure hash algorithm to obtain the first desensitized data.

[0018] The image data is subjected to resolution reduction processing to obtain a resolution processing result. Then, a preset detection model is used to identify sensitive regions in the image data, and a preset computer vision library is used to perform Gaussian blur processing on the sensitive regions to obtain the second desensitized data.

[0019] Random noise is added to the sensor data within a preset error range to obtain a noise addition result. The precise timestamp in the noise addition result is replaced with a time period to obtain a result to be processed. Then, the geographic coordinates corresponding to the result to be processed are blurred into a geographic region to obtain the third desensitized data. Initial multimodal data is constructed based on the first desensitized data, the second desensitized data, and the third desensitized data.

[0020] Optionally, the step of determining the spatiotemporal features of the target multimodal data using a preset spatiotemporal attention mechanism, and then fusing the spatiotemporal features based on a preset attention weight fusion mechanism to obtain a fusion result, includes:

[0021] The initial semantic feature extraction model is trained to obtain the target semantic feature extraction model, and the target semantic feature extraction model is used to extract semantic features from the target multimodal data to obtain the initial text semantic features.

[0022] The initial text semantic features are timestamped to obtain timestamping results. Time resolution alignment is performed on each timestamping result to obtain alignment results. Then, missing values ​​in the alignment results are interpolated or filled to obtain missing value processing results.

[0023] The time dependence and spatial correlation of the missing value processing results are captured by a preset spatiotemporal attention mechanism to obtain spatiotemporal features, and the attention weights corresponding to each spatiotemporal feature are determined. The spatiotemporal features are then fused using a preset attention weight fusion mechanism based on the corresponding attention weights to obtain a fusion result.

[0024] Optionally, the step of defining nodes and edges based on the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result, using a preset semantic matching mechanism to connect each node and edge across modal nodes, and then using a preset graph neural network to aggregate the neighborhood information of the connection results, to determine the graph to be processed based on the neighborhood information and the connection results, includes:

[0025] The modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result are determined. Nodes are defined based on the modal characteristics, and edges are constructed based on the intramodal relationships. A semantic matching mechanism is used to connect each node and the edge across modal nodes to obtain the connection result.

[0026] The nodes of different modalities in the connection result are processed by neighborhood aggregation using a preset graph neural network and a preset graph sampling aggregation network to obtain neighborhood information. Then, the neighborhood information is spliced ​​together to obtain the spliced ​​result.

[0027] The modal weights corresponding to each of the splicing results are determined, and the corresponding splicing results are fused using a preset transformer structure and the modal weights to obtain an initial image. Then, the initial image is globally pooled to obtain the image to be processed.

[0028] Optionally, the step of using a preset federated learning framework and training an initial hazard identification model based on the graph to be processed to obtain a hazard identification model to be processed, and then sending the model parameters corresponding to the hazard identification model to a preset edge server, so that the preset edge server aggregates the model parameters and sends the aggregation results to a preset central server, so that the preset central server generates a target hazard identification model based on the aggregation results, and uses the target hazard identification model to identify natural disaster hazards, includes:

[0029] The initial hazard identification model is trained using a pre-defined federated learning framework based on the graph to be processed, resulting in a hazard identification model to be processed. A hierarchical architecture is constructed, comprising clients, pre-defined edge servers, and pre-defined central servers, to divide each client into several training groups according to geographical location and network topology; wherein each pre-defined edge server corresponds one-to-one with each training group.

[0030] Determine the global model parameters corresponding to the hazard identification model to be processed, call the preset central server to divide the global model parameters into several local model parameters, and send each local model parameter to each preset edge server so that each preset edge server can distribute the corresponding local model parameters to the corresponding training group, and define aggregation rules and privacy protection mechanisms;

[0031] The client in each training group is invoked and gradient descent and local periodic training are performed on the local model parameters based on local private data to obtain updated local model parameters. The updated local model parameters are then encrypted using the privacy protection mechanism to obtain encrypted local model parameters. Finally, the encrypted local model parameters are sent to the corresponding preset edge server.

[0032] The preset edge server is invoked to aggregate the encrypted local model parameters according to the aggregation rules to obtain target local model parameters, and the target local model parameters are sent to the preset central server so that the preset central server can determine the model file based on each of the target local model parameters, and generate a target hazard identification model based on the model file, so as to use the target hazard identification model to identify natural disaster hazards.

[0033] Optionally, after using the target hazard identification model to identify natural disaster hazards, the method further includes:

[0034] The edge nodes in the preset edge server continuously receive feedback parameters from clients in each training group, and asynchronously upload the feedback parameters to the preset server. The preset server is then invoked to perform environmental and data distribution adjustments on the model parameters corresponding to the target hazard identification model based on the feedback parameters and local real-time data.

[0035] When an abnormality is detected in the model parameters corresponding to the target hazard identification model, a preset warning mechanism is triggered and the generated warning information is notified to relevant personnel through a preset alarm method. The warning information is recorded and analyzed, and the model parameters corresponding to the target hazard identification model are adjusted based on the analysis results.

[0036] Secondly, this application provides a natural disaster hazard identification device based on federated learning, comprising:

[0037] The data desensitization module is used to collect natural disaster data to be identified, including text data, image data, and sensor data, and to desensitize the text data, image data, and sensor data respectively to obtain initial multimodal data.

[0038] The feature extraction module is used to extract textual semantic features from the initial multimodal data to obtain semantic features, and to align the initial multimodal data to a uniform temporal resolution based on the semantic features. Then, the missing values ​​of the initial multimodal data are processed to obtain the target multimodal data.

[0039] The spatiotemporal feature fusion module is used to determine the spatiotemporal features of the target multimodal data using a preset spatiotemporal attention mechanism, then fuse the spatiotemporal features based on a preset attention weight fusion mechanism to obtain a fusion result, and define nodes and edges based on the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result, so as to connect the nodes and edges across modal nodes using a preset semantic matching mechanism, and then use a preset graph neural network to aggregate the neighborhood information of the connection results, so as to determine the graph to be processed based on the neighborhood information and the connection results;

[0040] The model determination module is used to obtain a hazard identification model to be processed by training an initial hazard identification model based on the graph to be processed using a preset federated learning framework, and to send the model parameters corresponding to the hazard identification model to a preset edge server so that the preset edge server can aggregate the model parameters and send the aggregation results to a preset central server so that the preset central server can generate a target hazard identification model based on the aggregation results and use the target hazard identification model to identify natural disaster hazards.

[0041] Thirdly, this application provides an electronic device, comprising:

[0042] Memory, used to store computer programs;

[0043] A processor is used to execute the computer program to implement the aforementioned federated learning-based natural disaster hazard identification method.

[0044] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned federated learning-based natural disaster hazard identification method.

[0045] As can be seen from the above, before conducting natural disaster hazard identification based on federated learning, this application needs to collect natural disaster data to be identified, including text data, image data, and sensor data. The text data, image data, and sensor data are then de-identified to obtain initial multimodal data. Semantic features are extracted from the initial multimodal data to obtain semantic features. Based on these semantic features, the initial multimodal data are aligned to a unified temporal resolution, and missing values ​​in the initial multimodal data are processed to obtain target multimodal data. A preset spatiotemporal attention mechanism is used to determine the spatiotemporal features of the target multimodal data. Then, based on a preset attention weight fusion mechanism, the spatiotemporal features are fused to obtain the fusion result. Finally, based on the fusion result... Modal spatiotemporal features are defined by modal characteristics and intramodal relationships to define nodes and edges. A pre-defined semantic matching mechanism is used to connect nodes and edges across modalities. Then, a pre-defined graph neural network is used to aggregate neighborhood information of the connection results to determine the graph to be processed based on neighborhood information and connection results. A pre-defined federated learning framework is used to train an initial hazard identification model based on the graph to be processed to obtain a hazard identification model to be processed. The model parameters corresponding to the hazard identification model to be processed are sent to a pre-defined edge server so that the pre-defined edge server can aggregate the model parameters and send the aggregation results to a pre-defined central server so that the pre-defined central server can generate a target hazard identification model based on the aggregation results and use the target hazard identification model to identify natural disaster hazards.

[0046] Therefore, this application first needs to collect natural disaster data to be identified, including text data, image data, and sensor data, and then desensitize the text data, image data, and sensor data respectively to obtain initial multimodal data. Second, semantic features are extracted from the initial multimodal data to obtain semantic features, and the initial multimodal data are aligned to a unified temporal resolution based on each semantic feature. Missing values ​​in the initial multimodal data are then processed to obtain target multimodal data. Next, a preset spatiotemporal attention mechanism is used to determine the spatiotemporal features of the target multimodal data, and then the spatiotemporal features are fused based on a preset attention weight fusion mechanism to obtain a fusion result. Finally, the application uses the corresponding modal spatiotemporal features in the fusion result to determine the target multimodal data. Modal characteristics and intramodal relationships define nodes and edges. A pre-defined semantic matching mechanism is used to connect nodes and edges across modalities. Then, a pre-defined graph neural network aggregates the neighborhood information of the connection results to determine the graph to be processed. Next, a pre-defined federated learning framework is used to train an initial hazard identification model based on the graph to be processed, resulting in a hazard identification model to be processed. The model parameters of the hazard identification model to be processed are then sent to a pre-defined edge server for aggregation. Finally, the aggregation results are sent to a pre-defined central server, which generates a target hazard identification model based on the aggregation results and uses this target hazard identification model to identify natural disaster hazards. This improves the efficiency of natural disaster hazard identification in federated learning-based natural disaster hazard identification, thereby enhancing the safety of the production process. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0048] Figure 1 This is a flowchart of a natural disaster hazard identification method based on federated learning disclosed in this application;

[0049] Figure 2 This is a schematic diagram illustrating a specific process for synchronously acquiring multimodal data as disclosed in this application;

[0050] Figure 3 This is a schematic diagram illustrating a specific process for encrypting and de-identifying data disclosed in this application;

[0051] Figure 4 This is a schematic diagram of a specific process for multimodal feature fusion disclosed in this application;

[0052] Figure 5 This application discloses a flowchart for generating early warning information;

[0053] Figure 6 This is a schematic diagram of a natural disaster hazard identification device based on federated learning disclosed in this application;

[0054] Figure 7 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Currently, with the continuous development of technology, the prevention and response to various hazards are receiving increasing attention. Traditional hazard prevention systems often have many limitations. Therefore, this application provides a federated learning-based method for natural disaster hazard identification, which can improve the efficiency of natural disaster hazard identification in the federated learning-based natural disaster hazard identification process.

[0057] See Figure 1 As shown, this embodiment of the invention discloses a method for identifying natural disaster hazards based on federated learning, including:

[0058] Step S11: Collect natural disaster data to be identified, including text data, image data, and sensor data, and desensitize the text data, image data, and sensor data respectively to obtain initial multimodal data.

[0059] In this embodiment, during the process of natural disaster hazard identification based on federated learning, this application embodiment needs to utilize a multimodal data acquisition module to simultaneously collect heterogeneous data from multiple sources, including text (such as equipment logs, maintenance reports, etc.), images (such as surveillance videos, thermal imaging images, etc.), and sensor data (such as data from IoT devices such as temperature, pressure, and vibration sensors). A flowchart illustrating the process of simultaneously acquiring multimodal data is shown below. Figure 2As shown, this embodiment of the application deploys an NTP (Network Time Protocol) server to ensure time synchronization of all acquisition devices. For devices that cannot connect to the network, PTP (Precision Time Protocol) or hardware clock calibration is used to achieve millisecond-level synchronization accuracy. It is worth noting that data acquisition methods are diverse. This embodiment of the application can utilize log collection tools (such as Fluentd and Logstash) to capture log files in real time, or obtain structured reports through API (Application Programming Interface) interfaces or database queries. It can also use RTSP (Real-Time Streaming Protocol), RTMP (Real-Time Messaging Protocol), or SDKs (Software Development Kits), such as FFmpeg (Fast Forward Moving Picture Experts Group) or OpenCV (Open Source Computer Vision Library), to pull video streams and save keyframes as needed. Sensor data can be pushed to the edge gateway via MQTT (Message Queuing Telemetry Transport), HTTP (Hypertext Transfer Protocol), or OPC UA (OpenPlatform Communications Unified Architecture). At the same time, the sensor sampling frequency is set to match or be an integer multiple of the video frame rate to ensure the consistency of the data in the time dimension.

[0060] Specifically, the acquisition of natural disaster data to be identified, including text data, image data, and sensor data, may include: synchronizing each acquisition sensor in time using a preset time synchronization protocol, and using a preset log collection tool to capture log files corresponding to the natural disaster video to be identified in real time; then obtaining a structured report corresponding to the natural disaster video to be identified through an application programming interface; pulling the natural disaster video to be identified through a streaming media protocol and based on the structured report, extracting keyframes from the natural disaster video to obtain image data, and determining text data based on the image data; wherein, the image data includes monitoring video and thermal imaging images, and the text data includes equipment logs and maintenance reports; adjusting the sampling frequency corresponding to the acquisition sensor data based on the video frame rate corresponding to the natural disaster video to obtain a target sampling frequency, so as to acquire sensor data corresponding to the natural disaster video to be identified using the acquisition sensors and based on the target sampling frequency; the sensor data includes temperature, pressure, and vibration measurement data.

[0061] In this embodiment, the collected data needs to be encrypted and de-identified, and the flowchart for data encryption and de-identification is shown below. Figure 3 As shown. When desensitizing text data, this embodiment replaces sensitive fields with fixed values ​​(e.g., the middle four digits of a phone number become ****), performs irreversible hashing on ID-type fields, such as SHA (Secure Hash Algorithm), and adds salt. When desensitizing image and video data, this embodiment chooses to reduce the image resolution or frame rate to make details unrecognizable. Then, a deep learning model, such as YOLO (a real-time object detection algorithm), is used to identify and replace sensitive objects, or OpenCV (Open Source Computer Vision Library) is used to detect and Gaussian blur sensitive areas. In addition, when desensitizing sensor data, it is achieved by adding random noise to the values ​​(controlled within the error range), replacing precise timestamps with time periods (e.g., 12:00:00 → 12:00 - 12:15), and blurring GPS (Global Positioning System) coordinates to regions (e.g., city level).

[0062] Specifically, the process of desensitizing the text data, image data, and sensor data to obtain initial multimodal data may include: replacing sensitive fields in the text data with fixed values ​​to obtain a replacement result, and salting the replacement result using a preset secure hash algorithm to obtain first desensitized data; reducing the resolution of the image data to obtain a resolution processing result, then using a preset detection model to identify sensitive regions in the image data, and using a preset computer vision library to perform Gaussian blurring on the sensitive regions to obtain second desensitized data; adding random noise to the sensor data within a preset error range to obtain a noise addition result, replacing the precise timestamps in the noise addition result with time periods to obtain a result to be processed, then blurring the geographic coordinates corresponding to the result to be processed into a geographic region to obtain third desensitized data, and constructing initial multimodal data based on the first desensitized data, the second desensitized data, and the third desensitized data.

[0063] Step S12: Extract text semantic features from the initial multimodal data to obtain semantic features, align the initial multimodal data to a uniform temporal resolution based on the semantic features, and then process the missing values ​​of the initial multimodal data to obtain the target multimodal data.

[0064] In this embodiment, the application requires the use of an attention mechanism to align the spatiotemporal dimensions and to model the initial multimodal data using a graph neural network to obtain the correlation between modalities. That is, the application first performs preprocessing and spatiotemporal feature extraction on the data, and then extracts text semantic features and adds timestamps using BERT (Bidirectional Encoder Representations from Transformers) or Word2Vec (Word to Vector). All modal data are aligned to a uniform temporal resolution (e.g., second level), and missing values ​​are interpolated or filled to obtain the target multimodal data.

[0065] Step S13: Determine the spatiotemporal features of the target multimodal data using a preset spatiotemporal attention mechanism, then fuse each spatiotemporal feature based on a preset attention weight fusion mechanism to obtain a fusion result, and define nodes and edges based on the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result, so as to connect each node and edge across modal nodes using a preset semantic matching mechanism, and then use a preset graph neural network to aggregate the neighborhood information of the connection result, so as to determine the graph to be processed based on the neighborhood information and the connection result.

[0066] In this embodiment, a spatiotemporal attention module needs to be constructed to build an attention mechanism capable of simultaneously capturing temporal dependence and spatial correlation. Then, the attention mechanism is used to fuse multimodal spatiotemporal features based on attention weights. Specifically, in the process of processing multimodal data and constructing the graph, this embodiment needs to define nodes according to modal characteristics, construct edges based on intramodal relationships, and then establish cross-modal node connections through semantic matching or external knowledge. A schematic diagram of the multimodal feature fusion process is shown below. Figure 4 As shown.

[0067] Specifically, the step of determining the spatiotemporal features of the target multimodal data using a preset spatiotemporal attention mechanism, and then fusing the spatiotemporal features based on a preset attention weight fusion mechanism to obtain a fusion result, may include: training an initial semantic feature extraction model to obtain a target semantic feature extraction model, and using the target semantic feature extraction model to extract semantic features from the target multimodal data to obtain initial text semantic features; adding timestamps to the initial text semantic features to obtain timestamp addition results, and performing time resolution alignment operations on each timestamp addition result to obtain alignment results; then interpolating or filling missing values ​​in the alignment results to obtain missing value processing results; using a preset spatiotemporal attention mechanism to capture the temporal dependence and spatial correlation of the missing value processing results to obtain spatiotemporal features, and determining the attention weights corresponding to each spatiotemporal feature, so as to use the preset attention weight fusion mechanism and based on the corresponding attention weights to perform fusion processing on each spatiotemporal feature to obtain a fusion result.

[0068] Furthermore, after obtaining the fusion result, this embodiment of the application utilizes a GNN applied to each modality, such as GCN (Graph Convolutional Network), GAT (Graph Attention Network), or GraphSAGE (Graph Sampling and Aggregation), to perform neighborhood information aggregation. Then, inter-modal attention weights are designed to dynamically evaluate the contribution of other modalities to the current modality, and multimodal nodes are merged into the same graph. Notably, this embodiment uses heterogeneous GNNs (such as RGCN) for modeling and allows different modal nodes to exchange information during the GNN aggregation stage. Finally, the node representations of each modality are concatenated, then fused using a fully connected layer or Transformer, and modal weights are learned to dynamically balance the contributions of different modalities. Finally, global pooling is performed on the graph to generate a multimodal joint representation.

[0069] Specifically, the step of defining nodes and edges based on the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result, using a preset semantic matching mechanism to perform cross-modal node connections between each node and each edge, and then using a preset graph neural network to aggregate the neighborhood information of the connection results, and determining the graph to be processed based on the neighborhood information and the connection results, may include: determining the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result, defining nodes based on the modal characteristics, constructing edges based on the intramodal relationships, and using a semantic matching mechanism to perform cross-modal node connections between each node and each edge to obtain connection results; using a preset graph neural network and a preset graph sampling aggregation network to perform neighborhood aggregation processing on nodes of different modalities in the connection results to obtain neighborhood information, and then concatenating each neighborhood information to obtain a concatenated result; determining the modal weights corresponding to each concatenated result, and using a preset transformer structure and the modal weights to fuse the corresponding concatenated results to obtain an initial graph, and then performing global pooling on the initial graph to obtain the graph to be processed.

[0070] Step S14: Using a preset federated learning framework and training an initial hazard identification model based on the graph to be processed, a hazard identification model to be processed is obtained. The model parameters corresponding to the hazard identification model to be processed are sent to a preset edge server so that the preset edge server can aggregate the model parameters and send the aggregation results to a preset central server so that the preset central server can generate a target hazard identification model based on the aggregation results and use the target hazard identification model to identify natural disaster hazards.

[0071] In this embodiment, the federated learning framework is used to aggregate model parameters. Specifically, a hierarchical architecture is constructed, dividing clients (such as mobile devices) into multiple groups based on geographical location, network topology, or function, with each group corresponding to an edge server. The central server distributes global model parameters to all edge servers, which then redistribute these parameters to clients within their groups. Aggregation rules (such as weighting methods and communication frequencies) and privacy protection mechanisms (such as differential privacy and encrypted transmission) are defined for each layer. Subsequently, each client can train using local private data and model parameters, performing gradient descent or local epoch training based on the received model parameters. After updating its local model, the client encrypts and sends the updated model parameters to the corresponding edge server. The edge server receives model parameters from all clients within its group, aggregates them according to preset rules (such as weighted average) to generate local model parameters, and sends these parameters to the central server. After receiving the local model parameters from each edge server, the central server generates new global model parameters and distributes them to the clients, allowing them to replace their local models with the new parameters and enter the next training cycle.

[0072] Specifically, the process of using a preset federated learning framework to train an initial hazard identification model based on the graph to be processed to obtain a hazard identification model to be processed, and then sending the model parameters corresponding to the hazard identification model to a preset edge server so that the preset edge server can aggregate the model parameters and send the aggregation results to a preset central server so that the preset central server can generate a target hazard identification model based on the aggregation results, and use the target hazard identification model to identify natural disaster hazards, may include: using a preset federated learning framework to train the initial hazard identification model based on the graph to be processed to obtain a hazard identification model to be processed, and constructing a hierarchical architecture including clients, preset edge servers, and a preset central server, so as to divide each client into several training groups according to geographical location and network topology; wherein each preset edge server corresponds one-to-one with each training group; determining the global model parameters corresponding to the hazard identification model to be processed, and calling the preset central server to divide the global model parameters into several local parameters. The system takes local model parameters and distributes each of these local model parameters to a preset edge server. Each preset edge server then distributes the corresponding local model parameters to its respective training group, defining aggregation rules and a privacy protection mechanism. The system calls clients in each training group and performs gradient descent and local periodic training on each of the local model parameters based on local private data to obtain updated local model parameters. These updated local model parameters are then encrypted using the privacy protection mechanism to obtain encrypted local model parameters, which are then distributed to the corresponding preset edge server. The preset edge server then aggregates the encrypted local model parameters according to the aggregation rules to obtain target local model parameters, which are sent to the preset central server. The preset central server determines a model file based on these target local model parameters and generates a target hazard identification model based on the model file, which is then used for natural disaster hazard identification.

[0073] In this embodiment, the flowchart for generating early warning information is as follows: Figure 5 As shown: First, edge nodes continuously receive parameter updates from clients within the group and support asynchronous uploads. Then, they fine-tune the model in real time based on local real-time data to adapt to constantly changing environments and data distributions. When an anomaly is detected, an early warning mechanism is quickly triggered, promptly notifying relevant personnel through various means (such as SMS, pop-ups, and audible and visual alarms). Simultaneously, the early warning information is recorded and analyzed in detail for subsequent system optimization and improvement.

[0074] Specifically, after using the target hazard identification model to identify natural disaster hazards, the process may further include: continuously receiving feedback parameters from clients in each training group via edge nodes in the preset edge server, and asynchronously uploading the feedback parameters to the preset server; invoking the preset server and adjusting the model parameters corresponding to the target hazard identification model based on the feedback parameters and local real-time data to optimize the environment and data distribution; when an abnormality is detected in the model parameters corresponding to the target hazard identification model, triggering a preset warning mechanism and notifying relevant personnel of the generated warning information through a preset alarm method, recording and analyzing the warning information, and adjusting the model parameters corresponding to the target hazard identification model based on the analysis results.

[0075] As can be seen from the above, the embodiments of this application first need to collect natural disaster data to be identified, including text data, image data, and sensor data, and then perform desensitization processing on the text data, image data, and sensor data respectively to obtain initial multimodal data; secondly, text semantic features are extracted from the initial multimodal data to obtain semantic features, and the initial multimodal data are aligned to a unified temporal resolution based on each semantic feature, and the missing values ​​of the initial multimodal data are processed to obtain target multimodal data; then, the spatiotemporal features of the target multimodal data are determined using a preset spatiotemporal attention mechanism, and then the spatiotemporal features are fused based on a preset attention weight fusion mechanism to obtain the fusion result, and the modal spatiotemporal features in the fusion result are corresponding to... The modal characteristics and intramodal relationships define nodes and edges. A pre-defined semantic matching mechanism is used to connect nodes and edges across modalities. Then, a pre-defined graph neural network aggregates the neighborhood information of the connection results to determine the graph to be processed. Next, a pre-defined federated learning framework is used to train an initial hazard identification model based on the graph to be processed, resulting in a hazard identification model to be processed. The model parameters of the hazard identification model to be processed are then sent to a pre-defined edge server for aggregation. Finally, the aggregation results are sent to a pre-defined central server, which generates a target hazard identification model based on the aggregation results and uses this target hazard identification model to identify natural disaster hazards. This improves the efficiency of natural disaster hazard identification in federated learning-based natural disaster hazard identification, thereby enhancing the safety of the production process.

[0076] Accordingly, see Figure 6 As shown, this application also provides a natural disaster hazard identification device based on federated learning, comprising:

[0077] The data desensitization module 11 is used to collect natural disaster data to be identified, including text data, image data, and sensor data, and to desensitize the text data, image data, and sensor data respectively to obtain initial multimodal data;

[0078] The feature extraction module 12 is used to extract text semantic features from the initial multimodal data to obtain semantic features, and to align the initial multimodal data to a uniform temporal resolution based on the semantic features. Then, the missing values ​​of the initial multimodal data are processed to obtain the target multimodal data.

[0079] The spatiotemporal feature fusion module 13 is used to determine the spatiotemporal features of the target multimodal data using a preset spatiotemporal attention mechanism, then fuse each spatiotemporal feature based on a preset attention weight fusion mechanism to obtain a fusion result, and define nodes and edges based on the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result, so as to connect each node and edge across modal nodes using a preset semantic matching mechanism, and then use a preset graph neural network to aggregate the neighborhood information of the connection result, so as to determine the graph to be processed based on the neighborhood information and the connection result;

[0080] The model determination module 14 is used to obtain the hazard identification model to be processed by training an initial hazard identification model based on the graph to be processed using a preset federated learning framework, and to send the model parameters corresponding to the hazard identification model to a preset edge server so that the preset edge server can aggregate the model parameters and send the aggregation results to a preset central server so that the preset central server can generate a target hazard identification model based on the aggregation results and use the target hazard identification model to identify natural disaster hazards.

[0081] In some specific embodiments, the data desensitization module 11 may specifically include:

[0082] The structured report acquisition unit is used to synchronize each acquisition sensor in time using a preset time synchronization protocol, and to capture the log file corresponding to the natural disaster video to be identified in real time using a preset log collection tool. Then, it obtains the structured report corresponding to the natural disaster video to be identified through the application programming interface.

[0083] The image data determination unit is used to retrieve the video of the natural disaster to be identified through a streaming media protocol and based on the structured report, extract keyframes from the video of the natural disaster to be identified to obtain image data, and determine text data based on the image data; wherein, the image data includes monitoring video and thermal imaging images, and the text data includes equipment logs and maintenance reports;

[0084] The sampling frequency determination unit is used to adjust the sampling frequency corresponding to the acquired sensor data based on the video frame rate corresponding to the natural disaster video to be identified, so as to obtain a target sampling frequency, so as to acquire sensor data corresponding to the natural disaster video to be identified using the acquired sensor and based on the target sampling frequency; the sensor data includes temperature, pressure and vibration measurement data.

[0085] In some specific embodiments, the data desensitization module 11 may specifically include:

[0086] The sensitive field replacement unit is used to replace sensitive fields in the text data with fixed values ​​to obtain replacement results, and to salt the replacement results using a preset secure hash algorithm to obtain the first desensitized data;

[0087] The sensitive region determination unit is used to perform resolution reduction processing on the image data to obtain a resolution processing result, and then use a preset detection model to identify the sensitive regions in the image data, and use a preset computer vision library to perform Gaussian blur processing on the sensitive regions to obtain the second desensitized data.

[0088] The initial multimodal data determination unit is used to perform random noise addition processing on the sensor data within a preset error range to obtain a noise addition result, and replace the precise timestamp in the noise addition result with a time period to obtain a result to be processed. Then, the geographic coordinates corresponding to the result to be processed are blurred into a geographic region to obtain the third desensitized data, and the initial multimodal data is constructed based on the first desensitized data, the second desensitized data and the third desensitized data.

[0089] In some specific embodiments, the spatiotemporal feature fusion module 13 may specifically include:

[0090] The semantic feature extraction unit is used to train the initial semantic feature extraction model to obtain the target semantic feature extraction model, and to use the target semantic feature extraction model to extract semantic features from the target multimodal data to obtain the initial text semantic features.

[0091] The missing value processing unit is used to add timestamps to the initial text semantic features to obtain timestamp addition results, perform time resolution alignment operation on each timestamp addition result to obtain alignment results, and then perform interpolation or filling processing on the missing values ​​in the alignment results to obtain missing value processing results.

[0092] The spatiotemporal feature determination unit is used to capture the temporal dependence and spatial correlation of the missing value processing result using a preset spatiotemporal attention mechanism to obtain spatiotemporal features, and to determine the attention weight corresponding to each spatiotemporal feature, so as to perform fusion processing on each spatiotemporal feature using a preset attention weight fusion mechanism and based on the corresponding attention weight to obtain a fusion result.

[0093] In some specific embodiments, the spatiotemporal feature fusion module 13 may specifically include:

[0094] The connection result generation unit is used to determine the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result, define nodes based on the modal characteristics, construct edges based on the intramodal relationships, and use a semantic matching mechanism to connect each node and the edge across modal nodes to obtain the connection result;

[0095] The splicing result determination unit is used to perform neighborhood aggregation processing on nodes of different modalities in the connection result using a preset graph neural network and a preset graph sampling aggregation network to obtain neighborhood information, and then splice the neighborhood information to obtain the splicing result;

[0096] The splicing result fusion unit is used to determine the modal weights corresponding to each splicing result, and to fuse the corresponding splicing results using a preset transformer structure and the modal weights to obtain an initial image. Then, the initial image is globally pooled to obtain the image to be processed.

[0097] In some specific embodiments, the model determination module 14 may specifically include:

[0098] A hierarchical architecture building unit is used to train an initial hazard identification model based on the graph to be processed using a preset federated learning framework to obtain a hazard identification model to be processed, and to build a hierarchical architecture including clients, preset edge servers and preset central servers, so as to divide each client into several training groups according to geographical location and network topology; wherein each preset edge server corresponds one-to-one with each training group.

[0099] The model parameter partitioning unit is used to determine the global model parameters corresponding to the hazard identification model to be processed, so as to call the preset central server to partition the global model parameters into several local model parameters, and send each local model parameter to each preset edge server, so that each preset edge server can distribute the corresponding local model parameters to the corresponding training group, and define aggregation rules and privacy protection mechanisms.

[0100] The model parameter division encryption unit is used to call the client in each training group and perform gradient descent and local periodic training on each local model parameter based on local private data to obtain updated local model parameters. The updated local model parameters are then encrypted using the privacy protection mechanism to obtain encrypted local model parameters. Finally, the encrypted local model parameters are sent to the corresponding preset edge server.

[0101] The model file determination unit is used to call the preset edge server to aggregate the encrypted local model parameters according to the aggregation rules to obtain target local model parameters, and send the target local model parameters to the preset central server so that the preset central server determines the model file based on each target local model parameter, and generates a target hazard identification model based on the model file, so as to use the target hazard identification model to identify natural disaster hazards.

[0102] In some specific embodiments, the federated learning-based natural disaster hazard identification device may further include:

[0103] The model parameter adjustment unit is used to call the edge nodes in the preset edge server to continuously receive feedback parameters from the clients in each training group, and asynchronously upload the feedback parameters to the preset server, so as to call the preset server and perform environmental and data distribution adjustment operations on the model parameters corresponding to the target hazard identification model based on the feedback parameters and local real-time data;

[0104] The analysis result generation unit is used to trigger a preset warning mechanism and notify relevant personnel of the generated warning information through a preset alarm method when an abnormality is detected in the model parameters corresponding to the target hazard identification model, and to record and analyze the warning information, and adjust the model parameters corresponding to the target hazard identification model based on the analysis results.

[0105] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the federated learning-based natural disaster hazard identification method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0106] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0107] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0108] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the federated learning-based natural disaster hazard identification method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0109] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed method for identifying natural disaster risks based on federated learning. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0110] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0111] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0112] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0113] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0114] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for identifying natural disaster hazards based on federated learning, characterized in that, include: Collect natural disaster data to be identified, including text data, image data, and sensor data, and desensitize the text data, image data, and sensor data respectively to obtain initial multimodal data; Textual semantic features are extracted from the initial multimodal data to obtain semantic features. Based on each semantic feature, the initial multimodal data are aligned to a uniform temporal resolution. Then, the missing values ​​of the initial multimodal data are processed to obtain the target multimodal data. The spatiotemporal features of the target multimodal data are determined using a preset spatiotemporal attention mechanism. Then, the spatiotemporal features are fused using a preset attention weight fusion mechanism to obtain a fusion result. Nodes and edges are defined based on the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result. Cross-modal node connections are made between the nodes and edges using a preset semantic matching mechanism. Then, neighborhood information of the connection results is aggregated using a preset graph neural network to determine the graph to be processed based on the neighborhood information and the connection results. An initial hazard identification model is trained using a pre-defined federated learning framework based on the graph to be processed to obtain a hazard identification model to be processed. The model parameters corresponding to the hazard identification model to be processed are then sent to a pre-defined edge server, which aggregates the model parameters and sends the aggregation results to a pre-defined central server. The central server then generates a target hazard identification model based on the aggregation results and uses the target hazard identification model to identify natural disaster hazards.

2. The natural disaster hazard identification method based on federated learning according to claim 1, characterized in that, The data collected includes text data, image data, and sensor data related to the natural disaster to be identified, including: The system synchronizes each acquisition sensor in time using a preset time synchronization protocol, and uses a preset log collection tool to capture log files corresponding to the natural disaster videos to be identified in real time. Then, it obtains a structured report corresponding to the natural disaster videos to be identified through the application programming interface. The video of the natural disaster to be identified is retrieved through a streaming media protocol and based on the structured report. Keyframes are extracted from the video to obtain image data, and text data is determined based on the image data. The image data includes surveillance video and thermal imaging images, and the text data includes equipment logs and maintenance reports. The sampling frequency corresponding to the sensor data is adjusted based on the video frame rate corresponding to the natural disaster to be identified, so as to obtain the target sampling frequency, and the sensor data corresponding to the natural disaster to be identified is collected using the sensor and based on the target sampling frequency; the sensor data includes temperature, pressure and vibration measurement data.

3. The natural disaster hazard identification method based on federated learning according to claim 1, characterized in that, The process of desensitizing the text data, image data, and sensor data to obtain initial multimodal data includes: The sensitive fields in the text data are replaced with fixed values ​​to obtain the replacement result. The replacement result is then salted using a preset secure hash algorithm to obtain the first desensitized data. The image data is subjected to resolution reduction processing to obtain a resolution processing result. Then, a preset detection model is used to identify sensitive regions in the image data, and a preset computer vision library is used to perform Gaussian blur processing on the sensitive regions to obtain the second desensitized data. Random noise is added to the sensor data within a preset error range to obtain a noise addition result. The precise timestamp in the noise addition result is replaced with a time period to obtain a result to be processed. Then, the geographic coordinates corresponding to the result to be processed are blurred into a geographic region to obtain the third desensitized data. Initial multimodal data is constructed based on the first desensitized data, the second desensitized data, and the third desensitized data.

4. The natural disaster hazard identification method based on federated learning according to claim 1, characterized in that, The process involves determining the spatiotemporal features of the target multimodal data using a preset spatiotemporal attention mechanism, and then fusing the spatiotemporal features based on a preset attention weight fusion mechanism to obtain a fusion result, including: The initial semantic feature extraction model is trained to obtain the target semantic feature extraction model, and the target semantic feature extraction model is used to extract semantic features from the target multimodal data to obtain the initial text semantic features. The initial text semantic features are timestamped to obtain timestamping results. Time resolution alignment is performed on each timestamping result to obtain alignment results. Then, missing values ​​in the alignment results are interpolated or filled to obtain missing value processing results. The time dependence and spatial correlation of the missing value processing results are captured by a preset spatiotemporal attention mechanism to obtain spatiotemporal features, and the attention weights corresponding to each spatiotemporal feature are determined. The spatiotemporal features are then fused using a preset attention weight fusion mechanism based on the corresponding attention weights to obtain a fusion result.

5. The natural disaster hazard identification method based on federated learning according to claim 1, characterized in that, The process involves defining nodes and edges based on the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result, using a preset semantic matching mechanism to connect each node and edge across modal nodes, and then using a preset graph neural network to aggregate the neighborhood information of the connection results to determine the graph to be processed based on the neighborhood information and the connection results. This includes: The modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result are determined. Nodes are defined based on the modal characteristics, and edges are constructed based on the intramodal relationships. A semantic matching mechanism is used to connect each node and the edge across modal nodes to obtain the connection result. The nodes of different modalities in the connection result are processed by neighborhood aggregation using a preset graph neural network and a preset graph sampling aggregation network to obtain neighborhood information. Then, the neighborhood information is spliced ​​together to obtain the spliced ​​result. The modal weights corresponding to each of the splicing results are determined, and the corresponding splicing results are fused using a preset transformer structure and the modal weights to obtain an initial image. Then, the initial image is globally pooled to obtain the image to be processed.

6. The natural disaster hazard identification method based on federated learning according to claim 1, characterized in that, The process involves using a pre-defined federated learning framework to train an initial hazard identification model based on the graph to be processed, obtaining a hazard identification model to be processed, and then sending the model parameters corresponding to the hazard identification model to a pre-defined edge server. The pre-defined edge server then aggregates the model parameters and sends the aggregation results to a pre-defined central server. The pre-defined central server then generates a target hazard identification model based on the aggregation results and uses the target hazard identification model to identify natural disaster hazards. The initial hazard identification model is trained using a pre-defined federated learning framework based on the graph to be processed, resulting in a hazard identification model to be processed. A hierarchical architecture is constructed, comprising clients, pre-defined edge servers, and pre-defined central servers, to divide each client into several training groups according to geographical location and network topology; wherein each pre-defined edge server corresponds one-to-one with each training group. Determine the global model parameters corresponding to the hazard identification model to be processed, call the preset central server to divide the global model parameters into several local model parameters, and send each local model parameter to each preset edge server so that each preset edge server can distribute the corresponding local model parameters to the corresponding training group, and define aggregation rules and privacy protection mechanisms; The client in each training group is invoked and gradient descent and local periodic training are performed on the local model parameters based on local private data to obtain updated local model parameters. The updated local model parameters are then encrypted using the privacy protection mechanism to obtain encrypted local model parameters. Finally, the encrypted local model parameters are sent to the corresponding preset edge server. The preset edge server is invoked to aggregate the encrypted local model parameters according to the aggregation rules to obtain target local model parameters, and the target local model parameters are sent to the preset central server so that the preset central server can determine the model file based on each of the target local model parameters, and generate a target hazard identification model based on the model file, so as to use the target hazard identification model to identify natural disaster hazards.

7. The natural disaster hazard identification method based on federated learning according to claim 6, characterized in that, After using the target hazard identification model to identify natural disaster hazards, the process further includes: The edge nodes in the preset edge server continuously receive feedback parameters from clients in each training group, and asynchronously upload the feedback parameters to the preset server. The preset server is then invoked to perform environmental and data distribution adjustments on the model parameters corresponding to the target hazard identification model based on the feedback parameters and local real-time data. When an abnormality is detected in the model parameters corresponding to the target hazard identification model, a preset warning mechanism is triggered and the generated warning information is notified to relevant personnel through a preset alarm method. The warning information is recorded and analyzed, and the model parameters corresponding to the target hazard identification model are adjusted based on the analysis results.

8. A natural disaster hazard identification device based on federated learning, characterized in that, include: The data desensitization module is used to collect natural disaster data to be identified, including text data, image data, and sensor data, and to desensitize the text data, image data, and sensor data respectively to obtain initial multimodal data. The feature extraction module is used to extract textual semantic features from the initial multimodal data to obtain semantic features, and to align the initial multimodal data to a uniform temporal resolution based on the semantic features. Then, the missing values ​​of the initial multimodal data are processed to obtain the target multimodal data. The spatiotemporal feature fusion module is used to determine the spatiotemporal features of the target multimodal data using a preset spatiotemporal attention mechanism, then fuse the spatiotemporal features based on a preset attention weight fusion mechanism to obtain a fusion result, and define nodes and edges based on the modal characteristics and intramodal relationships corresponding to the modal spatiotemporal features in the fusion result, so as to connect the nodes and edges across modal nodes using a preset semantic matching mechanism, and then use a preset graph neural network to aggregate the neighborhood information of the connection results, so as to determine the graph to be processed based on the neighborhood information and the connection results; The model determination module is used to obtain a hazard identification model to be processed by training an initial hazard identification model based on the graph to be processed using a preset federated learning framework, and to send the model parameters corresponding to the hazard identification model to a preset edge server so that the preset edge server can aggregate the model parameters and send the aggregation results to a preset central server so that the preset central server can generate a target hazard identification model based on the aggregation results and use the target hazard identification model to identify natural disaster hazards.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the natural disaster hazard identification method based on federated learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the natural disaster hazard identification method based on federated learning as described in any one of claims 1 to 7.

Images