Emergency event processing method and device, equipment, storage medium and program product
By acquiring multimodal data and using deep learning models to process it, the problem of insufficient accuracy and efficiency of emergency event processing models in the prior art is solved, and efficient analysis and processing of complex events is achieved.
Patent Information
- Application Number
- CN202411179693.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-20
AI Technical Summary
In the existing emergency incident handling methods, due to the singularization of input data types, the accuracy and analysis efficiency of the model are insufficient in the face of complex and changeable emergency incidents.
By obtaining multimodal data (including text data, audio data, image data and video data), input it into the emergency event processing model, using the deep learning model and the multi-head self-attention mechanism, we generate emergency event parameters, and determine event types and hazards degree and key data characteristics, and then event processing and scheme analysis are carried out.
It improves model accuracy and analysis efficiency in complex and variable emergency events, can more accurately reflect the event situation and generate effective processing solutions.
Smart Images
Figure CN119167291A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to an emergency event processing method, device, equipment, storage medium and program product. Background Art
[0002] An emergency event refers to an event that occurs suddenly and causes or may cause serious casualties, property losses, ecological environment damage, etc. It covers many aspects such as natural disasters, accidents and disasters, and public health events.
[0003] When dealing with emergency events, the existing technology mainly involves information officers reporting the events and filling in the elements for emergencies that need to be handled urgently.
[0004] However, in the existing emergency event processing methods, due to the single input data type, the model's accuracy and analysis efficiency are insufficient when facing complex and changeable emergency events. Summary of the invention
[0005] The present application provides an emergency event processing method, device, equipment, storage medium and program product to solve the problem of insufficient model accuracy and analysis efficiency due to the single input data type when facing complex and changeable emergency events.
[0006] In a first aspect, the present application provides a method for handling an emergency event, the method comprising:
[0007] Acquiring multimodal data, where the multimodal data is used to indicate multiple types of data corresponding to the emergency event;
[0008] Inputting the multiple types of data into an emergency event processing model to obtain a processing result output by the emergency event processing model, wherein the emergency event processing model is trained based on historical multimodal data and historical processing results corresponding to historical emergency events, and the historical multimodal data is used to indicate historical data of multiple types of historical emergency events;
[0009] The emergency event is processed according to the processing result.
[0010] In a possible implementation, the multimodal data includes: text data, audio data, image data and video data, and the inputting of the multiple types of data into the emergency event processing model to obtain the processing result output by the emergency event processing model includes:
[0011] Determine corresponding word vector features according to the text data;
[0012] Inputting the image data, video data and audio data into a multi-layer perception module to obtain corresponding feature data;
[0013] Using a multi-head self-attention mechanism, data processing is performed on the word vector features and multiple feature data to obtain emergency event parameters corresponding to the emergency event;
[0014] According to the emergency event parameters, a processing result corresponding to the emergency event is determined.
[0015] In a possible implementation, the emergency event parameters include: event type, hazard level, and key data features corresponding to the emergency event, and determining the processing result corresponding to the emergency event according to the emergency event parameters includes:
[0016] Determining first matching weights between a plurality of expert networks and the emergency event according to the event type, the degree of harm, and key data characteristics;
[0017] The first matching weights of multiple expert networks are sorted in descending order, and the first k expert networks are used as target expert networks;
[0018] The emergency event processing solutions sent by the first k expert networks are obtained, and the k emergency event processing solutions are analyzed and integrated to obtain the processing result.
[0019] In a possible implementation, the multimodal data includes: an event type corresponding to the emergency event, the emergency event parameters include: a degree of harm corresponding to the emergency event and key data features, and determining a processing result corresponding to the emergency event according to the emergency event parameters includes:
[0020] Determine a plurality of candidate expert networks corresponding to the event type;
[0021] Determining second matching weights between a plurality of candidate expert networks and the emergency event according to the hazard degree and key data features;
[0022] Determine a target expert network from the plurality of candidate expert networks according to a plurality of second matching weights, wherein the second matching weight of the target expert network is greater than the second matching weights of other expert networks;
[0023] The emergency event processing plan sent by the target expert network is used as the processing result.
[0024] In a possible implementation, before acquiring the multimodal data, the method further includes:
[0025] Acquire event information of multiple historical emergency events, the event information including: multimodal historical data and historical processing results;
[0026] Determine expert networks corresponding to a plurality of event types, wherein each event type corresponds to at least one expert network;
[0027] Based on the multiple expert networks, construct a deep learning model;
[0028] The deep learning model is trained according to the multimodal historical data and the historical processing results to obtain the emergency event processing model.
[0029] In a possible implementation, the training process of the deep learning model according to the multimodal historical data and the historical processing results to obtain the emergency event processing model includes:
[0030] Inputting the multimodal historical data into the deep learning model to obtain a training result output by the deep learning model, wherein the training result is determined based on the multiple expert networks;
[0031] Determining a loss function of the deep learning model according to the training results and the historical processing results;
[0032] When the loss function satisfies a preset condition, it is determined that the deep learning model training is completed and the emergency event handling model is obtained.
[0033] In a possible implementation, performing event processing on the emergency event according to the processing result includes:
[0034] Generate an emergency event reporting message according to the processing result, the degree of harm and the key data characteristics;
[0035] The emergency event reporting message is reported to the emergency event handling server.
[0036] In a second aspect, the present application provides an emergency event handling device, the device comprising:
[0037] An acquisition module, used to acquire multimodal data, where the multimodal data is used to indicate multiple types of data corresponding to an emergency event;
[0038] A processing module, used for inputting the multiple types of data into an emergency event processing model to obtain a processing result output by the emergency event processing model, wherein the emergency event processing model is trained based on historical multimodal data and historical processing results corresponding to historical emergency events, and the historical multimodal data is used to indicate multiple types of historical data of historical emergency events;
[0039] The processing module is further used to perform event processing on the emergency event according to the processing result.
[0040] In one possible implementation, the multimodal data includes: text data, audio data, image data and video data, and the processing module is used to determine the corresponding word vector features based on the text data; input the image data, video data and audio data into a multi-layer perception module to obtain corresponding feature data; adopt a multi-head self-attention mechanism to process the word vector features and multiple feature data to obtain emergency event parameters corresponding to the emergency event; and determine the processing result corresponding to the emergency event based on the emergency event parameters.
[0041] In a possible implementation, the emergency event parameters include: the event type, the degree of harm, and the key data characteristics corresponding to the emergency event; the processing module is used to determine the first matching weights of multiple expert networks and the emergency event according to the event type, the degree of harm, and the key data characteristics; sort the first matching weights of the multiple expert networks in descending order, and use the first k expert networks as target expert networks;
[0042] The acquisition module is also used to acquire the emergency event handling solutions sent by the first k expert networks;
[0043] The processing module is used to analyze and integrate k emergency event processing solutions to obtain the processing result.
[0044] In a possible implementation, the multimodal data includes: the event type corresponding to the emergency event, the emergency event parameters include: the degree of hazard and key data characteristics corresponding to the emergency event, and the processing module is used to determine multiple candidate expert networks corresponding to the event type; determine the second matching weights of the multiple candidate expert networks and the emergency event based on the degree of hazard and the key data characteristics; determine the target expert network from the multiple candidate expert networks based on the multiple second matching weights, the second matching weight of the target expert network is greater than the second matching weights of other expert networks; and use the emergency event handling plan sent by the target expert network as the processing result.
[0045] In a possible implementation, the device further includes: a construction module;
[0046] The acquisition module is further used to acquire event information of multiple historical emergency events, wherein the event information includes: multi-modal historical data and historical processing results;
[0047] The processing module is further used to determine expert networks corresponding to a plurality of event types, wherein each event type corresponds to at least one expert network;
[0048] The building module is used to build a deep learning model based on the multiple expert networks;
[0049] The processing module is also used to train the deep learning model according to the multimodal historical data and the historical processing results to obtain the emergency event processing model.
[0050] In one possible implementation, the processing module is used to input the multimodal historical data into the deep learning model to obtain a training result output by the deep learning model, wherein the training result is determined based on the multiple expert networks; determine the loss function of the deep learning model based on the training result and the historical processing result; and determine that the training of the deep learning model is completed when the loss function meets a preset condition, thereby obtaining the emergency event handling model.
[0051] In a possible implementation, the device further includes: a sending module;
[0052] The processing module is used to generate an emergency event reporting message according to the processing result, the degree of harm and the key data characteristics;
[0053] The sending module is used to report the emergency event reporting message to the emergency event processing server.
[0054] In a third aspect, the present application provides an emergency event handling device, the device comprising:
[0055] Memory;
[0056] processor;
[0057] Wherein, the memory stores computer-executable instructions;
[0058] The processor executes the computer-executable instructions stored in the memory to implement the emergency event handling method as described in the first aspect and various possible implementation methods of the first aspect.
[0059] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the emergency event handling method as described in the first aspect and various possible implementations of the first aspect.
[0060] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the emergency event handling method as described in the first aspect and various possible implementation methods of the first aspect.
[0061] The present application provides an emergency event processing method, device, equipment, storage medium and program product, the method acquires multimodal data, the multimodal data is used to indicate multiple types of data corresponding to the emergency event; the multiple types of data are input into the emergency event processing model to obtain the processing results output by the emergency event processing model, the emergency event processing model is trained based on historical multimodal data and historical processing results corresponding to historical emergency events, the historical multimodal data is used to indicate multiple types of historical data of historical emergency events; and then the emergency event is processed according to the processing results. Since the acquired data is multimodal data, it can improve the accuracy and analysis efficiency of the model when dealing with complex and changeable emergencies. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0063] Figure 1 A schematic diagram of an emergency event handling method provided for this application Figure 1 ;
[0064] Figure 2 A schematic diagram of an emergency event handling method provided for this application Figure 2 ;
[0065] Figure 3 A schematic diagram of the structure of an emergency event handling device provided in this application Figure 3 ;
[0066] Figure 4 A schematic diagram of the structure of an emergency event handling device provided in this application Figure 4 .
[0067] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0068] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims, rather than all embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0069] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, products or devices.
[0070] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0071] An emergency event refers to an event that occurs suddenly and causes or may cause serious casualties, property losses, ecological environment damage, etc. It covers many aspects such as natural disasters, accidents and disasters, and public health events.
[0072] The corresponding emergency event handling is mainly to reduce the impact of disasters, promote post-disaster recovery, and improve response capabilities. It can conduct disaster analysis on the emergency events that have occurred in a relatively short period of time and provide subsequent emergency plans.
[0073] In the prior art, when dealing with emergencies, for emergencies that need to be dealt with urgently, the information officer mainly reports the event and fills in the elements. During the information officer's reporting process, the input data is text data, and only text language descriptions of the emergency events can be made.
[0074] However, in the existing emergency event processing methods, due to the single input data type, the model's accuracy and analysis efficiency are insufficient when facing complex and changeable emergency events.
[0075] In view of the above problems, the present application provides an emergency event processing method, which obtains corresponding processing results by inputting multimodal data corresponding to the emergency event into the emergency event processing model. Then, the event processing is performed based on the processing results. Since the input data is multimodal, it can better reflect the situation of the emergency event, thereby making the processing result more accurate and efficient.
[0076] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in a certain embodiment. The embodiments of the present application are described below in conjunction with the accompanying drawings.
[0077] Figure 1 A process for handling an emergency event provided in the embodiment of the present application Figure 1 .like Figure 1 As shown, the emergency event handling method provided in this embodiment includes:
[0078] S101: Acquire multimodal data, where the multimodal data is used to indicate multiple types of data corresponding to an emergency event.
[0079] Among them, multimodal data refers to data composed of two or more modalities, which can be, for example, different data forms or information sources. Multiple types of data can include, for example, text type data, audio type data, image type data, and video type data.
[0080] In this step, the multimodal data may be obtained by integrating data actively uploaded by personnel at the scene of the emergency, for example, personnel A takes photos of the emergency and uploads them, and personnel B writes a text description of the emergency. In this case, the multimodal data may refer to the sum of the data uploaded by personnel A and personnel B.
[0081] Multimodal data can also be captured by emergency event processing equipment from the Internet.
[0082] It is understandable that the multiple types of data may be, for example, two types of data, or three types of data or other types of data, and the present application does not limit this. The more modes the multimodal data contains, the clearer the emergency event it can reflect.
[0083] S102: Input the multiple types of data into an emergency event processing model to obtain a processing result output by the emergency event processing model.
[0084] Among them, the emergency event processing model is trained based on historical multimodal data and historical processing results corresponding to historical emergency events, and the historical multimodal data is used to indicate multiple types of historical data of historical emergency events.
[0085] The processing result can, for example, represent the processing solution corresponding to the emergency event. There is an association between the processing result and the emergency event, and the processing result is usually the optimal solution to the emergency event. For example, if the event type of the emergency event is a fire, the processing result can represent the optimal solution to the fire. The processing results corresponding to different types of emergency events may be the same or different.
[0086] It can be understood that multiple types of data can be input into the emergency event handling model in batches, for example. After the emergency event handling model receives the input data in the same batch, it will analyze and process the data in the batch to obtain the optimal solution corresponding to the emergency event.
[0087] Multiple types of data can also be divided into multiple batches and input into the emergency event handling model separately, wherein each batch of data contains an event identifier indicating the emergency event, so that the emergency event handling model can obtain the optimal solution to the emergency event corresponding to the event identifier based on the analysis and processing of data with the same event identifier.
[0088] In a possible implementation, the multiple types of data may include, for example, text data, image data, video data, and audio data. After the multiple types of data are input into the emergency event processing model, the model will perform data analysis and processing using corresponding processing methods for different types of data, and perform data processing based on the analysis results corresponding to the multiple types to obtain corresponding processing results.
[0089] The following is an example of the process of analyzing and processing the emergency event processing model based on multimodal data:
[0090] For text data in multimodal data, the emergency event processing model determines the corresponding word vector features based on the text data.
[0091] It can be understood that word vector features are a method of converting words in natural language into numerical vectors, which are used to represent various types of data; the specific method of determining word vector features can be, for example, determined by a semantic analysis algorithm or based on a word segmentation algorithm. For example, the text data can be first segmented to obtain a full word vector, and then the full word vector can be semantically analyzed to extract word vector features that can represent the emergency event.
[0092] For the image data, video data and audio data in the multimodal data, the emergency event processing model can first pre-process the above data, and then input the pre-processed image data, video data and audio data into the multi-layer perception module to obtain the corresponding feature data;
[0093] Among them, the multi-layer perception module is composed of image embedding of image data, video embedding of video data, audio embedding of audio data and multi-layer perceptron; image embedding is a method of converting image data into continuous, low-dimensional vector representation, video embedding is a method of converting video data into high-dimensional vector representation, audio embedding is a method of converting audio data into vector representation, and multi-layer perceptron is a neural network model composed of multiple neuron layers, including an input layer, one or more hidden layers and an output layer. The multi-layer perceptron is used to perform image embedding of image data, video embedding of video data, and audio embedding of audio data for feature extraction.
[0094] After obtaining the word vector features and feature data, a multi-head self-attention mechanism can be used to process the word vector features and multiple feature data to obtain emergency event parameters corresponding to the emergency event; and based on the emergency event parameters, determine the processing results corresponding to the emergency event.
[0095] Among them, the multi-head self-attention mechanism converts each input element into query, key and value vectors, and performs self-attention calculation on these vectors through the processing unit corresponding to the vector converted by each element. The self-attention calculation is to convert each input element into three vectors of query, key and value through different linear transformations.
[0096] Data processing can be, for example, normalization processing, which is a data preprocessing method that limits the data to be processed to a certain range after processing, and the range is usually [0, 1] or [-1, 1]. It is mainly adjusted according to the actual situation, usually to simplify subsequent data processing and analysis.
[0097] Emergency event parameters are key factors in assessing and managing emergency events, covering all aspects of the event and facilitating rapid response and accurate decision making.
[0098] The emergency event parameter may indicate, for example, the severity of the emergency event, or the event type and key data characteristics of the emergency event. Different emergency event parameters may correspond to different processing results.
[0099] In a possible implementation, in the process of determining the processing result corresponding to the emergency event according to the emergency event parameters, it is necessary to first determine whether the multimodal data contains the event type. The specific implementation method of determining the processing result corresponding to the emergency event according to the emergency event parameters is explained below:
[0100] Scenario 1: The multimodal data does not contain the event type corresponding to the emergency event. In this scenario, the emergency event parameters include: the event type corresponding to the emergency event, the degree of harm, and key data characteristics.
[0101] In a possible implementation, the following steps can be used to implement emergency event processing based on multimodal data:
[0102] Determining first matching weights between a plurality of expert networks and the emergency event according to the event type, the degree of harm, and key data characteristics;
[0103] Among them, expert network can broadly refer to any network that can provide professional knowledge, advice or services, including traditional expert consulting systems, decision support systems based on artificial intelligence, and broader social networks and professional communities; the first matching weight is that the gating network dynamically assigns weights to each expert according to the type of event, degree of harm and key data characteristics, reflecting the degree of influence of each expert network on the output result under the input data; the gating network is a method of controlling a neural network by controlling the connection between input and output. The principle is to control the input, output and the transmission process of the network through a group of gating units, so as to achieve specific functional realization.
[0104] After determining the first matching weight of each expert network, the first matching weights of multiple expert networks may be sorted in descending order, and the first k expert networks are used as target expert networks;
[0105] Among them, selecting k expert networks as the target expert networks according to the first matching weight from large to small is to improve the accuracy of the analysis results and prevent the problem that some expert networks cannot be selected due to equipment instability; when the current k experts are activated to handle emergency events, the remaining experts are not activated.
[0106] After determining the first k target expert networks, the emergency event processing solutions sent by the first k expert networks are obtained, and the k emergency event processing solutions are analyzed and integrated to obtain the processing result.
[0107] Among them, in the analysis and fusion processing of the k emergency events, fusion is performed through weighted summation based on gated routing weights; weighted summation based on gated routing weights is achieved by dynamically assigning weights to each expert model and applying these weights to the output of the expert model to achieve the final weighted summation; under the hybrid expert system, the expert model is a specific implementation of the expert network.
[0108] Scenario 2: The multimodal data contains the event type corresponding to the emergency event. In this scenario, the emergency event parameters include: the corresponding degree of harm and key data characteristics; that is, since the multimodal data already contains the event type corresponding to the emergency event, the determined emergency event parameters may not include the event type of the emergency event.
[0109] It is understandable that the event type may be provided by the user when uploading multimodal data, or may be determined based on the multimodal data, and this application does not impose any limitation on this.
[0110] In a possible implementation, the following steps can be used to implement emergency event processing based on multimodal data:
[0111] Determine a plurality of expert candidate networks corresponding to the event type;
[0112] The method for determining the multiple expert candidate networks corresponding to the event type is to classify the expert networks according to the emergency event parameters. When the emergency event type is confirmed, the multiple expert candidate networks are determined according to the emergency event parameters.
[0113] Determining second matching weights between a plurality of candidate expert networks and the emergency event according to the hazard degree and key data features;
[0114] The second matching weight is a weight dynamically assigned by the gating network to each candidate expert according to the degree of harm and key data features.
[0115] Determine a target expert network from the plurality of candidate expert networks according to a plurality of second matching weights, wherein the second matching weight of the target expert network is greater than the second matching weights of other expert networks;
[0116] The emergency event processing plan sent by the target expert network is used as the processing result.
[0117] S103: Perform event processing on the emergency event according to the processing result.
[0118] In a possible implementation, the above process may be implemented by, for example, the following steps:
[0119] According to the determined processing results, the degree of harm corresponding to the emergency event and key data characteristics, an emergency event reporting message is generated and reported to the emergency event processing server.
[0120] The reported information is used to indicate the event situation of the emergency event and the corresponding optimal solution. The key data features can be, for example, a combination of two or more of time feature data, space feature data, event feature data, and environmental feature data, which are not limited here.
[0121] The emergency event processing server is a device or system with the ability to receive, process and store data. It can process emergency events based on emergency event reporting messages.
[0122] In the above steps, in the process of reporting the emergency event reporting message to the emergency event processing server, for example, it can be uploaded to the emergency event processing server through the selected reporting path, the process is monitored to ensure that the information is successfully sent, and the failed message is retried according to the preset retry strategy, and the message is received and confirmed; the emergency event processing server may include a physical server, a virtualized server, a cloud server, and an edge server, which is not limited here.
[0123] The emergency event processing method provided in this embodiment obtains multimodal data, and the multimodal data is used to indicate multiple types of data corresponding to the emergency event; the multiple types of data are input into the emergency event processing model to obtain the processing results output by the emergency event processing model; and the emergency event is processed according to the processing results. Since the acquired data is multimodal data, it can improve the accuracy and analysis efficiency of the model when dealing with complex and changeable emergencies.
[0124] Figure 2 A process for handling an emergency event provided in the embodiment of the present application Figure 2 In this embodiment Figure 1 Based on this, a possible implementation method of the construction and training process of the emergency event handling model is supplemented. Figure 2 As shown, the method includes:
[0125] S201: Acquire a plurality of historical emergency event information, wherein the event information includes: multi-modal historical data and historical processing results.
[0126] Among them, multimodal historical data refers to a collection of data that includes multiple different types, different sources, and accumulated over time. Multimodal historical data can be obtained through online channels.
[0127] The purpose of obtaining multiple historical emergency event information in this step is to generate a corresponding model training set for subsequent model training.
[0128] S202: Determine expert networks corresponding to a plurality of event types, wherein each event type corresponds to at least one expert network.
[0129] Among them, the expert networks corresponding to multiple event types are selected through the gating network. Each event type corresponds to at least one expert network mainly to ensure the professionalism, targeting, resource optimization, efficiency improvement, flexibility and scalability of the processing process, as well as the synergy and complementary relationship between different expert networks.
[0130] S203: Constructing a deep learning model based on the multiple expert networks.
[0131] Among them, deep learning models are usually pre-trained using massive data, and then further adapted to the task through techniques such as fine-tuning.
[0132] In this step, the deep learning model can be constructed using, for example, a backbone network, a gating network, and multiple expert networks.
[0133] S204: Training the deep learning model according to the multimodal historical data and the historical processing results to obtain the emergency event processing model.
[0134] The multimodal historical data and the historical processing results are used to train the deep learning model to obtain the emergency event processing model, including:
[0135] The multimodal historical data is input into the deep learning model to obtain a training result output by the deep learning model, wherein the training result is determined based on the multiple expert networks; the loss function of the deep learning model is determined according to the training result and the historical processing result; when the loss function satisfies a preset condition, it is determined that the training of the deep learning model is completed, and the emergency event handling model is obtained.
[0136] In a possible implementation, the loss function may be determined using the following formula, for example:
[0137] L total =L regressive +α×L aux
[0138] Among them, L toatl is the loss function, L regressive is the free regression loss function, L aux is the auxiliary loss function and α is the scaling factor.
[0139] The free regression loss function usually refers to the loss function used in the regression task, which aims to minimize the error between the predicted value and the actual value; the auxiliary loss function is an additional loss function introduced outside the main task to improve model performance or meet specific needs.
[0140] In the above model training, the free regression loss function can be determined by the following formula, for example:
[0141]
[0142] Among them, ω is the expert network weight, n is the number of samples, y i is the true value of the i-th sample, is the predicted value of the ith sample.
[0143] Since there are multiple types of expert networks, when facing a mixed expert network layer, a load balancing constraint can be used to process the input data to improve the processing efficiency of the expert network. The auxiliary loss function can be determined, for example, by the following formula:
[0144]
[0145] Among them, F represents the proportion of inputs processed by the expert network, G represents the average routing probability of the expert network, and E usually represents the total number of categories or the number of categories in a specific subset; P(x) is a probability distribution or prediction output about the input x, usually a vector, which contains the model's predicted probability that the input x belongs to each category; argmaxP(x) is the index of the maximum value in P(x), that is, the category to which the model believes x is most likely to belong; 1{argmaxP(x)=i} is a function that returns 1 or 0 depending on whether argmaxP(x) is equal to i. If the category predicted by the model (that is, the result of argmaxP(x)) is the same as i, then 1 is returned; otherwise, 0 is returned.
[0146] In this step, the deep learning model is trained based on the multimodal historical data and historical processing results. The specific process of obtaining the emergency event processing model may include, for example:
[0147] According to the above deep learning model constructed by the backbone network, the gating network and multiple expert networks, the data is processed by the backbone network, mainly embedding the data, inputting the multi-layer perceptron for feature alignment, using the multi-modal feature dynamic fusion layer for feature fusion, inputting the multi-head self-attention layer, and performing residual connection and layer normalization to obtain the processing result;
[0148] The obtained processing results are passed through the gating network for expert network selection;
[0149] Then, the selected expert network processes the processing results, fuses the results, performs residual connection and layer normalization, obtains the final output result, completes the model training, and obtains the emergency event processing model.
[0150] Among them, the residual connection uses a direct connection channel to directly connect the corresponding sub-layer input to the output, avoiding the potential gradient vanishing problem caused by the network being too deep during the optimization process, and further improving the stability of training. The corresponding sub-layer is a layer where the connection channel is located in the model training;
[0151] The multimodal feature dynamic fusion layer dynamically adjusts the fusion weights of different modal features according to data quality. It consists of a data quality assessment module, a dynamic weight adjustment module, and a multimodal fusion module.
[0152] The data quality assessment module scores the data quality from multiple dimensions, including clarity, relevance, completeness, consistency, and reliability, which are not restricted here; the dynamic weight adjustment module adjusts the weights through the attention mechanism and data quality scoring, assigning more weight to higher quality data; the multimodal fusion module performs multimodal feature fusion based on the dynamically adjusted weights, and in emergency scenarios, the dynamic fusion of multimodal features can give priority to key information.
[0153] The emergency event processing method provided in this embodiment obtains multiple historical emergency event information, wherein the event information includes: multimodal historical data and historical processing results; determines expert networks corresponding to multiple event types, wherein each event type corresponds to at least one expert network; constructs a deep learning model based on the multiple expert networks; and trains the deep learning model according to the multimodal historical data and the historical processing results to obtain the emergency event processing model. The obtained emergency event processing model can dynamically select an expert network, and determine the processing model through a preset loss function, thereby improving the performance and adaptability of the model in complex environments.
[0154] Figure 3 An emergency event handling device is provided in an embodiment of the present application. Figure 3 As shown, the emergency event handling device 300 provided in the embodiment of the present application includes:
[0155] An acquisition module 301 is used to acquire multimodal data, where the multimodal data is used to indicate multiple types of data corresponding to an emergency event;
[0156] A processing module 302 is used to input the multiple types of data into an emergency event processing model to obtain a processing result output by the emergency event processing model, wherein the emergency event processing model is trained based on historical multimodal data and historical processing results corresponding to historical emergency events, and the historical multimodal data is used to indicate multiple types of historical data of historical emergency events;
[0157] The processing module 302 is further configured to perform event processing on the emergency event according to the processing result.
[0158] In one possible implementation, the multimodal data includes: text data, audio data, image data and video data, and the processing module 302 is used to determine the corresponding word vector features based on the text data; input the image data, video data and audio data into a multi-layer perception module to obtain corresponding feature data; use a multi-head self-attention mechanism to perform data processing on the word vector features and multiple feature data to obtain emergency event parameters corresponding to the emergency event; and determine the processing result corresponding to the emergency event based on the emergency event parameters.
[0159] In a possible implementation, the emergency event parameters include: the event type, the degree of harm, and the key data characteristics corresponding to the emergency event, and the processing module 302 is used to determine the first matching weights of multiple expert networks and the emergency event according to the event type, the degree of harm, and the key data characteristics; sort the first matching weights of the multiple expert networks in descending order, and use the first k expert networks as target expert networks;
[0160] The acquisition module 301 is also used to acquire the emergency event handling solutions sent by the first k expert networks;
[0161] The processing module 302 is used to analyze and integrate k emergency event processing solutions to obtain the processing result.
[0162] In a possible implementation, the multimodal data includes: the event type corresponding to the emergency event, the emergency event parameters include: the degree of hazard and key data characteristics corresponding to the emergency event, the processing module 302 is used to determine multiple candidate expert networks corresponding to the event type; determine the second matching weights of the multiple candidate expert networks and the emergency event based on the degree of hazard and the key data characteristics; determine the target expert network from the multiple candidate expert networks based on the multiple second matching weights, the second matching weight of the target expert network is greater than the second matching weights of other expert networks; and use the emergency event handling plan sent by the target expert network as the processing result.
[0163] In a possible implementation, the device further includes: a construction module 303;
[0164] The acquisition module 301 is also used to acquire event information of multiple historical emergency events, wherein the event information includes: multi-modal historical data and historical processing results;
[0165] The processing module 302 is further used to determine expert networks corresponding to multiple event types, wherein each event type corresponds to at least one expert network;
[0166] The construction module 303 is used to construct a deep learning model based on the multiple expert networks;
[0167] The processing module 302 is also used to train the deep learning model according to the multimodal historical data and the historical processing results to obtain the emergency event processing model.
[0168] In one possible implementation, the processing module 302 is used to input the multimodal historical data into the deep learning model to obtain a training result output by the deep learning model, wherein the training result is determined based on the multiple expert networks; determine the loss function of the deep learning model based on the training result and the historical processing result; and determine that the training of the deep learning model is completed when the loss function meets a preset condition, thereby obtaining the emergency event handling model.
[0169] In a possible implementation, the apparatus further includes: a sending module 304;
[0170] The processing module 302 is used to generate an emergency event reporting message according to the processing result, the degree of harm and the key data characteristics;
[0171] The sending module 304 is used to report the emergency event reporting message to the emergency event handling server.
[0172] Figure 4 This is a schematic diagram of the structure of the emergency event handling equipment provided in this application. Figure 4 As shown, the emergency event handling device 400 provided in the present application includes: a receiver 401, a transmitter 402, a processor 403 and a memory 404.
[0173] Receiver 401, for receiving instructions and data;
[0174] A transmitter 402, used for sending instructions and data;
[0175] Memory 404, for storing computer-executable instructions;
[0176] The processor 403 is used to execute the computer-executable instructions stored in the memory 404 to implement the various steps executed by the emergency event handling method in the above embodiment. For details, please refer to the relevant description in the above emergency event handling method embodiment.
[0177] Optionally, the memory 404 may be independent or integrated with the processor 403 .
[0178] When the memory 404 is independently provided, the electronic device further includes a bus for connecting the memory 404 and the processor 403 .
[0179] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, an emergency event handling method as performed by the above-mentioned emergency event handling device is implemented.
[0180] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the various steps performed by the above-mentioned emergency event handling method. For details, please refer to the relevant description in the above-mentioned emergency event handling method embodiment.
[0181] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0182] So far, the technical solution of the present application has been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments, and the above embodiments are only used to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for handling emergency incidents, characterized in that: The method comprises: Acquiring multimodal data, where the multimodal data is used to indicate multiple types of data corresponding to the emergency event; Inputting the multiple types of data into an emergency event processing model to obtain a processing result output by the emergency event processing model, wherein the emergency event processing model is trained based on historical multimodal data and historical processing results corresponding to historical emergency events, and the historical multimodal data is used to indicate historical data of multiple types of historical emergency events; The emergency event is processed according to the processing result.
2. The method according to claim 1, characterized in that The multimodal data includes: text data, audio data, image data and video data. The inputting of the multiple types of data into the emergency event processing model to obtain the processing result output by the emergency event processing model includes: Determine corresponding word vector features according to the text data; Inputting the image data, video data and audio data into a multi-layer perception module to obtain corresponding feature data; Using a multi-head self-attention mechanism, data processing is performed on the word vector features and multiple feature data to obtain emergency event parameters corresponding to the emergency event; According to the emergency event parameters, a processing result corresponding to the emergency event is determined.
3. The method according to claim 2, characterized in that The emergency event parameters include: the event type, the degree of harm and key data characteristics corresponding to the emergency event. The determining of the processing result corresponding to the emergency event according to the emergency event parameters includes: Determining first matching weights between a plurality of expert networks and the emergency event according to the event type, the degree of harm, and key data characteristics; The first matching weights of multiple expert networks are sorted in descending order, and the first k expert networks are used as target expert networks; The emergency event processing solutions sent by the first k expert networks are obtained, and the k emergency event processing solutions are analyzed and integrated to obtain the processing result.
4. The method according to claim 2, characterized in that: The multimodal data includes: the event type corresponding to the emergency event; the emergency event parameters include: the degree of harm and key data characteristics corresponding to the emergency event; and determining the processing result corresponding to the emergency event according to the emergency event parameters includes: Determine a plurality of candidate expert networks corresponding to the event type; Determining second matching weights between a plurality of candidate expert networks and the emergency event according to the hazard degree and key data features; Determine a target expert network from the plurality of candidate expert networks according to a plurality of second matching weights, wherein the second matching weight of the target expert network is greater than the second matching weights of other expert networks; The emergency event processing plan sent by the target expert network is used as the processing result.
5. The method according to claim 1, characterized in that Before acquiring the multimodal data, the method further includes: Acquire event information of multiple historical emergency events, the event information including: multimodal historical data and historical processing results; Determine expert networks corresponding to a plurality of event types, wherein each event type corresponds to at least one expert network; Based on the multiple expert networks, construct a deep learning model; The deep learning model is trained according to the multimodal historical data and the historical processing results to obtain the emergency event processing model.
6. The method according to claim 5, characterized in that The training process of the deep learning model according to the multimodal historical data and the historical processing results to obtain the emergency event processing model includes: Inputting the multimodal historical data into the deep learning model to obtain a training result output by the deep learning model, wherein the training result is determined based on the multiple expert networks; Determining a loss function of the deep learning model according to the training results and the historical processing results; When the loss function satisfies a preset condition, it is determined that the deep learning model training is completed and the emergency event handling model is obtained.
7. The method according to claim 3 or 4, characterized in that: The performing event processing on the emergency event according to the processing result includes: Generate an emergency event reporting message according to the processing result, the degree of harm and the key data characteristics; The emergency event reporting message is reported to the emergency event handling server.
8. An emergency event handling device, characterized in that: The device comprises: An acquisition module, used to acquire multimodal data, where the multimodal data is used to indicate multiple types of data corresponding to an emergency event; A processing module, used for inputting the multiple types of data into an emergency event processing model to obtain a processing result output by the emergency event processing model, wherein the emergency event processing model is trained based on historical multimodal data and historical processing results corresponding to historical emergency events, and the historical multimodal data is used to indicate multiple types of historical data of historical emergency events; The processing module is further used to perform event processing on the emergency event according to the processing result.
9. An emergency event handling device, characterized in that: include: Memory; processor; Wherein, the memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the emergency event handling method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the emergency event handling method according to any one of claims 1 to 7.
11. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the emergency event handling method as described in any one of claims 1 to 7.