Multimodal expert collaborative fire risk monitoring method for large-scale forest resources

Abstract

The present invention discloses a multimodal expert collaborative fire risk monitoring method for large-scale forestry resources, including constructing a multimodal fire risk monitoring data set containing multimodal samples, wherein each multimodal sample contains meteorological time series data, vegetation images, human activity map data, terrain images and target labels representing fire risk categories; constructing four expert models, including a meteorological expert model, a vegetation expert model, a human activity expert model and a terrain expert model; constructing a multimodal monitoring network to train and obtain a multimodal monitoring model for predicting the fire risk category of the area to be identified. The present invention can extract more information from multimodal data and improve the ability to predict fire risks in complex environments; it can dynamically adjust the weights of each expert model according to the characteristics of the input data to achieve adaptive optimization of the model prediction effect. The present invention is suitable for real-time monitoring and prediction of fire risks, and can improve the accuracy and timeliness of fire warnings.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a multi-modal expert collaborative fire risk monitoring method for large-scale forestry resources. Background Art

[0002] In order to effectively protect and monitor forest resources, accurate fire risk assessment and early warning have become key means. However, traditional monitoring methods often rely on a single data source and are difficult to capture the dynamic changes of forest ecology in a comprehensive and real-time manner. In particular, the effectiveness of traditional methods is often limited under complex terrain or extreme climatic conditions. This is because: (1) The image quality of satellite remote sensing is easily affected by cloudy weather or complex terrain, and it is difficult to monitor sudden situations such as fires in a timely manner, especially in the early stages of a fire. (2) Ground monitoring and manual patrols are limited by small coverage and many blind spots, especially in complex terrain (such as mountains and densely forested areas), making it difficult to conduct comprehensive monitoring. (3) The data points of meteorological stations are sparsely distributed and less representative, and cannot reflect the dynamic changes of the entire forest area. In particular, in complex terrain such as high mountains or hills, the locality of the data makes fire risk assessment inaccurate. (4) Traditional statistical models that rely on historical data respond slowly to climate change and ecological changes, lack flexibility in current emergencies, and cannot effectively respond to rapidly changing fire risks.

[0003] LSTM (Long Short-Term Memory) is a type of RNN (Recurrent Neural Network). Due to its design characteristics, LSTM is very suitable for modeling time series data, such as text data.

[0004] BiLSTM (directional Long Short-Term Memory) is a variant of recurrent neural network, which is composed of forward LSTM and backward LSTM. Both are often used to model contextual information in natural language processing tasks.

[0005] The ViT model (Vision Transformer) is a visual task model based on the Transformer architecture. It mainly consists of three parts: image feature embedding module, Transformer encoder module and MLP classification module (Multilayer Perceptron). The image feature embedding module divides the image into fixed-size image blocks, and converts these image blocks into vectors and sends them to the Transformer encoder. The Transformer encoder is used to capture the relationship between different image blocks based on the attention mechanism, and then the MLP classification module performs classification. Assuming that there are N image blocks, the vectors input to the Transformer encoder are N+1. This is because the ViT algorithm proposes a learnable embedding vector Class Token, which is at the forefront of the vectors corresponding to the N image blocks and is sent to the Transformer encoder together with the N vectors for encoding, outputting N+1 encoded vectors, the first of which is the Class Token, which is used for subsequent classification prediction.

[0006] Since the computational complexity of ViT's self-attention is proportional to the square of the input image size, the computational cost is very high when processing high-resolution images. Therefore, a new network architecture Swin Transformer (Shifted Window Transformer) is proposed, which introduces hierarchical feature mapping and window attention transformation to solve the above problem. Summary of the invention

[0007] The purpose of the present invention is to provide a multimodal expert collaborative fire risk monitoring method for large-scale forestry resources that can solve the above-mentioned problems and accurately monitor forest fire risks under complex terrain or extreme climatic conditions.

[0008] In order to achieve the above-mentioned object, the technical solution adopted by the present invention is as follows: a multi-modal expert collaborative fire risk monitoring method for large-scale forestry resources, comprising the following steps;

[0009] S1, constructing a multimodal fire risk monitoring data set containing multimodal samples, wherein a construction method of a multimodal sample X includes steps S11 to S13;

[0010] S11, divide the fire risk into C categories, each category corresponds to a target label y, ;

[0011] S12, determine a region and time period, and obtain the meteorological time series data for the region and time period , vegetation image , Human Activity Graph Data , Terrain Image , confirm the target label y of fire risk;

[0012] , T is the total number of time steps included in the time period, is the total number of meteorological characteristic dimensions, including temperature, humidity and wind speed;

[0013] Including node feature matrix and the adjacency matrix , , , is the total number of nodes, is the total number of node feature dimensions, the node is a preset human activity concentration point, and the node feature dimensions include pedestrian flow characteristics, traffic flow characteristics, node location temperature, node location humidity, node location precipitation and sunshine intensity;

[0014] S13, Generate multimodal samples ;

[0015] S2, construct four expert models, including meteorological expert model, vegetation expert model, human activity expert model and terrain expert model;

[0016] The meteorological expert model includes a linear transformation layer, a BiLSTM network, a Transformer layer and a global pooling layer, which are arranged in sequence and used to input meteorological time series data. , and get the output of the meteorological expert model ;

[0017] The vegetation expert model includes an image segmentation layer, a linear transformation layer, and a Transformer layer, which are arranged in sequence, and is used to input a vegetation image. , after being divided into blocks by the image block layer, mapped to high dimensions by the linear transformation layer, and encoded by the Transformer layer, its embedding vector is obtained as the output of the vegetation expert model ;

[0018] The human activity expert model includes a first node information propagation layer, a second node information propagation layer, and a global pooling layer; the first node information propagation layer and the second node information propagation layer are used to obtain the first propagation information according to the following formula , Secondary dissemination of information , the global pooling layer is used to Perform global pooling to obtain the output of the human activity expert model ;

[0019] ,

[0020] ,

[0021] In the formula, , are the weight matrices of the first node information propagation layer and the second node information propagation layer, respectively. is the activation function;

[0022] The terrain expert model includes a sliding window layer, a SwinTransformer network and a global pooling layer; used to input a terrain image , get the output of the terrain expert model ;

[0023] S3, construct a multimodal monitoring network, including four expert models, Concat layer, first linear layer, normalization layer, weighted layer, and second linear layer;

[0024] The Concat layer is used according to the formula Get splicing features , For splicing operation;

[0025] The first linear layer is used according to the formula Generate weight Z, where , are the weights and biases of the first linear layer respectively;

[0026] The normalization layer is used to normalize the weight Z to obtain the normalized weight , , where α w , α v , α h , α t H w , H v , H h , H t The normalized weight of

[0027] The weighted layer is used according to the formula Generating weighted features ;

[0028] The second linear layer is used to Classification, output predicted target label ;

[0029] S4, using the multimodal fire risk monitoring data set to train the multimodal monitoring network until convergence, and obtain the multimodal monitoring model;

[0030] S5, model predictions;

[0031] For the identified area, obtain meteorological time series data , vegetation image , Human Activity Graph Data , Terrain Image , forming a multimodal sample to be monitored , sent to the multimodal monitoring model to obtain the predicted target label.

[0032] As a preference: the vegetation image is an RGB image containing vegetation in the area and time period corresponding to the multimodal sample X; the terrain image It is a DEM image in the corresponding area and time period of the multimodal sample X.

[0033] As a preference: construct human activity graph data The methods include Sa1~Sa2;

[0034] Sa1, preset nodes and node dimensions, the nodes are human activity concentration points, including entrances and exits, campsites, viewing platforms, road intersections, and preset distance thresholds;

[0035] Sa2, obtain the distribution map of human activity concentration points in the corresponding area and time period of the multimodal sample X, and construct the node feature matrix based on the nodes and node dimensions and the adjacency matrix , nodes total indivual;

[0036] Construct node feature matrix When the element in row i and column j is Represents the eigenvalue of the jth dimension of the i-th node, 1≤i≤ , 1≤j≤ ;

[0037] Constructing the adjacency matrix When the element in row i and column k is Represents the connection relationship between the i-th node and the k-th node. If the distance between the two nodes is less than the distance threshold, then ,otherwise , 1≤i≤ , 1≤k≤ .

[0038] As a preferred embodiment: in the meteorological expert model;

[0039] The linear transformation layer is used to calculate the ,Will Convert to high-dimensional data , where , are the weights and biases of the linear transformation layer respectively; , , , is the output dimension of the linear transformation layer;

[0040] The BiLSTM network is used as input , capturing the dependencies between time steps and outputting the data , , is the output dimension of the BiLSTM network;

[0041] The Transformer layer is used to Perform feature extraction and output data , , is the output dimension of the Transformer layer;

[0042] The global pooling layer is used to Perform global pooling and output , , The default fixed dimensions.

[0043] As a preference: in the terrain expert model;

[0044] The sliding window layer is used to input the terrain image Perform sliding serial port sampling, and obtain an image block each time sampling;

[0045] The SwinTransformer network is used to encode each image block and then splice them together to obtain data , ,in, is the preset fixed dimension, H DEM , W DEM Terrain images The height and width of

[0046] The global pooling layer is used to Perform global pooling and output , .

[0047] As a preferred embodiment: when the multimodal monitoring network is trained in step S4, one training is specifically as follows:

[0048] S41, preset the batch size B, select B multimodal samples and send them into the multimodal monitoring network, and calculate the loss function L according to the following formula;

[0049] ,

[0050] In the formula, b is the bth multimodal sample in a batch of multimodal samples, c is the cth category, , are the target label and predicted target label of the b-th multimodal sample respectively;

[0051] S42, update the parameters of the multimodal monitoring network using a back propagation algorithm.

[0052] Compared with the prior art, the advantages of the present invention are:

[0053] (1) The present invention introduces a new multimodal data fusion method based on an expert hybrid network structure for fire risk prediction. The present invention integrates four information sources, namely meteorological data, vegetation images, terrain features and human activities, and processes different modal information sources respectively through four independent and targeted expert models. Each expert model is optimized for different data types to ensure that the most effective feature extraction can be obtained for data in different fields. This enables the multimodal monitoring model to extract more information from multimodal data and improve the ability to predict fire risks in complex environments. Compared with relying solely on satellite images, the present invention can not only predict potential fire risks earlier through meteorological and human activity data, but also reduce the impact of weather interference on satellite images. Compared with single meteorological station monitoring, the present invention can provide a more global risk prediction by combining meteorological data from more locations with terrain changes.

[0054] (2) A gating network was designed, which consists of the first linear layer and the normalization layer in the multimodal monitoring network. Different regions or scenarios may have different influencing factors on fire risk. For example, meteorological factors may be more important in some areas, while vegetation or terrain factors may dominate in other areas. The gating network dynamically adjusts the weights through the Softmax mechanism, so that the model can adapt to the data characteristics in different scenarios more quickly and flexibly, thereby improving the accuracy of prediction. This innovative method not only improves the comprehensive processing capabilities of multi-source data, but also enhances the robustness of the model in complex environments.

[0055] (3) The model adopts an end-to-end training method, and each expert model, gating network and final classification layer are optimized together under the same framework. All parameters of the model can be optimized through back propagation, ensuring the coordination and performance of the overall model. Unlike models trained in stages, this method is more efficient and can maximize the use of training data.

[0056] (4) The application of this method in fire risk prediction significantly improves the accuracy and real-time performance of the prediction, providing reliable decision-making support for forestry resource managers. By comprehensively analyzing multimodal data, the expert hybrid network structure can more comprehensively assess fire risks, timely warn of potential threats, and help formulate more effective protective measures. This not only helps to improve the efficiency of forest ecological protection, but also provides a strong technical guarantee for disaster management, and promotes the development of modern forestry management towards a more intelligent and refined direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a schematic diagram of the meteorological expert model structure;

[0058] Figure 2 This is a schematic diagram of the vegetation expert model structure;

[0059] Figure 3 This is a schematic diagram of the structure of the human activity expert model;

[0060] Figure 4 This is a schematic diagram of the structure of the terrain expert model;

[0061] Figure 5 Schematic diagram of the multimodal monitoring network structure. DETAILED DESCRIPTION

[0062] The present invention will be further described below in conjunction with the accompanying drawings.

[0063] Example 1: See Figures 1 to 5 ,A multimodal expert collaborative fire risk monitoring method for large-scale forestry resources includes the following steps;

[0064] S1, constructing a multimodal fire risk monitoring data set containing multimodal samples, wherein a construction method of a multimodal sample X includes steps S11 to S13;

[0065] S11, divide the fire risk into C categories, each category corresponds to a target label y, ;

[0066] S12, determine a region and time period, and obtain the meteorological time series data for the region and time period , vegetation image , Human Activity Graph Data , Terrain Image , confirm the target label y of fire risk;

[0067] , T is the total number of time steps included in the time period, is the total number of meteorological characteristic dimensions, including temperature, humidity and wind speed;

[0068] Including node feature matrix and the adjacency matrix , , , is the total number of nodes, is the total number of node feature dimensions, the node is a preset human activity concentration point, and the node feature dimensions include pedestrian flow characteristics, traffic flow characteristics, node location temperature, node location humidity, node location precipitation and sunshine intensity;

[0069] S13, Generate multimodal samples ;

[0070] S2, construct four expert models, including meteorological expert model, vegetation expert model, human activity expert model and terrain expert model;

[0071] The meteorological expert model includes a linear transformation layer, a BiLSTM network, a Transformer layer and a global pooling layer, which are arranged in sequence and used to input meteorological time series data. , and get the output of the meteorological expert model ;

[0072] The vegetation expert model includes an image segmentation layer, a linear transformation layer, and a Transformer layer, which are arranged in sequence, and is used to input a vegetation image. , after being divided into blocks by the image block layer, mapped to high dimensions by the linear transformation layer, and encoded by the Transformer layer, its embedding vector is obtained as the output of the vegetation expert model ;

[0073] The human activity expert model includes a first node information propagation layer, a second node information propagation layer, and a global pooling layer; the first node information propagation layer and the second node information propagation layer are used to obtain the first propagation information according to the following formula , Secondary dissemination of information , the global pooling layer is used to Perform global pooling to obtain the output of the human activity expert model ;

[0074] ,

[0075] ,

[0076] In the formula, , are the weight matrices of the first node information propagation layer and the second node information propagation layer, respectively. is the activation function;

[0077] The terrain expert model includes a sliding window layer, a SwinTransformer network and a global pooling layer; used to input a terrain image , get the output of the terrain expert model ;

[0078] S3, construct a multimodal monitoring network, including four expert models, Concat layer, first linear layer, normalization layer, weighted layer, and second linear layer;

[0079] The Concat layer is used according to the formula Get splicing features , For splicing operation;

[0080] The first linear layer is used according to the formula Generate weight Z, where , are the weights and biases of the first linear layer respectively;

[0081] The normalization layer is used to normalize the weight Z to obtain the normalized weight , , where α w , α v , α h , α t H w , H v , H h , H t The normalized weight of

[0082] The weighted layer is used according to the formula Generating weighted features ;

[0083] The second linear layer is used to Classification, output predicted target label ;

[0084] S4, using the multimodal fire risk monitoring data set to train the multimodal monitoring network until convergence, and obtain the multimodal monitoring model;

[0085] S5, model predictions;

[0086] For the identified area, obtain meteorological time series data , vegetation image , Human Activity Graph Data , Terrain Image , forming a multimodal sample to be monitored , sent to the multimodal monitoring model to obtain the predicted target label.

[0087] Example 2: See Figures 1 to 5 In this embodiment, the vegetation image is an RGB image containing vegetation in the area and time period corresponding to the multimodal sample X; the terrain image is a DEM image in the area and time period corresponding to the multimodal sample X. The rest is the same as in Example 1.

[0088] Example 3: See Figures 1 to 5 On the basis of Example 1, the human activity graph is based on a specific geographical area, such as a forest, a nature reserve, or a forestry management area, and the time of constructing the human activity graph should be the same as the time and area corresponding to the corresponding multimodal sample X. The construction of the human activity graph is based on important locations or nodes related to human activities in the area, and uses information such as geographic adjacency and activity intensity to determine the connection relationship between nodes.

[0089] For example, a human activity map can be a map of a certain area, where nodes are marked according to the definition, and then the relevant data of the node feature dimension is obtained. The distance threshold is also set manually, assuming it can be 1 km, 2 km, etc. Construct a node feature matrix based on nodes and node feature dimensions , construct the adjacency matrix based on the distance between nodes , construct the adjacency matrix , since each element is a connection between two nodes, it is constructed for This embodiment provides a method for constructing human activity graph data. Specific methods include Sa1~Sa2;

[0090] Sa1, preset nodes and node dimensions, the nodes are human activity concentration points, including entrances and exits, campsites, viewing platforms, road intersections, and preset distance thresholds;

[0091] Sa2, obtain the distribution map of human activity concentration points in the corresponding area and time period of the multimodal sample X, and construct the node feature matrix based on the nodes and node dimensions and the adjacency matrix , nodes total indivual;

[0092] Construct node feature matrix When the element in row i and column j is Represents the eigenvalue of the jth dimension of the i-th node, 1≤i≤ , 1≤j≤ ;

[0093] Constructing the adjacency matrix When the element in row i and column k is Represents the connection relationship between the i-th node and the k-th node. If the distance between the two nodes is less than the distance threshold, then ,otherwise , 1≤i≤ , 1≤k≤ .

[0094] Example 4: See Figures 1 to 5 , based on Example 1, in the meteorological expert model;

[0095] The linear transformation layer is used to calculate the ,Will Convert to high-dimensional data , where , are the weights and biases of the linear transformation layer respectively; , , , is the output dimension of the linear transformation layer;

[0096] The BiLSTM network is used as input , capturing the dependencies between time steps and outputting the data , , is the output dimension of the BiLSTM network;

[0097] The Transformer layer is used to Perform feature extraction and output data , , is the output dimension of the Transformer layer;

[0098] The global pooling layer is used to Perform global pooling and output , , The default fixed dimensions.

[0099] Example 5: See Figures 1 to 5 , based on Example 1, in the terrain expert model;

[0100] The sliding window layer is used to input the terrain image Perform sliding serial port sampling, and obtain an image block each time sampling;

[0101] The SwinTransformer network is used to encode each image block and then splice them together to obtain data , ,in, is the preset fixed dimension, H DEM, W DEM Terrain images The height and width of

[0102] The global pooling layer is used to Perform global pooling and output , .

[0103] Example 6: See Figures 1 to 5 , based on Example 1, when step S4 trains the multimodal monitoring network, one training is specifically as follows;

[0104] S41, preset the batch size B, select B multimodal samples and send them into the multimodal monitoring network, and calculate the loss function L according to the following formula;

[0105] ,

[0106] In the formula, b is the bth multimodal sample in a batch of multimodal samples, c is the cth category, , are the target label and predicted target label of the b-th multimodal sample respectively;

[0107] S42, update the parameters of the multimodal monitoring network using a back propagation algorithm.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-modal expert collaborative fire risk monitoring method for large-scale forestry resources, characterized by: The steps include: S1, constructing a multimodal fire risk monitoring data set containing multimodal samples, wherein a construction method of a multimodal sample X includes steps S11 to S13; S11, divide the fire risk into C categories, each category corresponds to a target label y, ; S12, determine a region and time period, and obtain the meteorological time series data for the region and time period , vegetation image , Human Activity Graph Data , Terrain Image , confirm the target label y of fire risk; , T is the total number of time steps included in the time period, is the total number of meteorological characteristic dimensions, including temperature, humidity and wind speed; Including node feature matrix and the adjacency matrix , , , is the total number of nodes, is the total number of node feature dimensions, the node is a preset human activity concentration point, and the node feature dimensions include pedestrian flow characteristics, traffic flow characteristics, node location temperature, node location humidity, node location precipitation and sunshine intensity; S13, Generate multimodal samples ; S2, construct four expert models, including meteorological expert model, vegetation expert model, human activity expert model and terrain expert model; The meteorological expert model includes a linear transformation layer, a BiLSTM network, a Transformer layer and a global pooling layer, which are arranged in sequence and used to input meteorological time series data. , and get the output of the meteorological expert model ; The vegetation expert model includes an image segmentation layer, a linear transformation layer, and a Transformer layer, which are arranged in sequence, and is used to input a vegetation image. , after being divided into blocks by the image block layer, mapped to high dimensions by the linear transformation layer, and encoded by the Transformer layer, its embedding vector is obtained as the output of the vegetation expert model ; The human activity expert model includes a first node information propagation layer, a second node information propagation layer, and a global pooling layer; The first node information propagation layer and the second node information propagation layer are used to obtain the first propagation information according to the following formula , Secondary dissemination of information , the global pooling layer is used to Perform global pooling to obtain the output of the human activity expert model ; ， ， In the formula, , are the weight matrices of the first node information propagation layer and the second node information propagation layer, respectively. is the activation function; The terrain expert model includes a sliding window layer, a SwinTransformer network and a global pooling layer; For input terrain image , get the output of the terrain expert model ; S3, constructing a multimodal monitoring network, including a Concat layer, a first linear layer, a normalization layer, a weighted layer, and a second linear layer; The Concat layer is used according to the formula Get splicing features , For splicing operation; The first linear layer is used according to the formula Generate weight Z, where , are the weights and biases of the first linear layer respectively; The normalization layer is used to normalize the weight Z to obtain the normalized weight , , where α w , α v , α h , α t H w , H v , H h , H t The normalized weight of The weighted layer is used according to the formula Generating weighted features ; The second linear layer is used to Classification, output predicted target label ; S4, using the multimodal fire risk monitoring data set to train the multimodal monitoring network until convergence, and obtain the multimodal monitoring model; S5, model predictions; For the identified area, obtain meteorological time series data , vegetation image , Human Activity Graph Data , Terrain Image , forming a multimodal sample to be monitored , sent to the multimodal monitoring model to obtain the predicted target label.

2. The multi-modal expert collaborative fire risk monitoring method for large-scale forestry resources according to claim 1 is characterized by: The vegetation image is an RGB image containing vegetation in the area and time period corresponding to the multimodal sample X; the terrain image It is a DEM image in the corresponding area and time period of the multimodal sample X.

3. The multi-modal expert collaborative fire risk monitoring method for large-scale forestry resources according to claim 1 is characterized in that: Constructing human activity graph data The methods include Sa1~Sa2; Sa1, preset nodes and node dimensions, the nodes are human activity concentration points, including entrances and exits, campsites, viewing platforms, road intersections, and preset distance thresholds; Sa2, obtain the distribution map of human activity concentration points in the corresponding area and time period of the multimodal sample X, and construct the node feature matrix based on the nodes and node dimensions and the adjacency matrix , nodes total indivual; Construct node feature matrix When the element in row i and column j is Represents the eigenvalue of the jth dimension of the i-th node, 1≤i≤ , 1≤j≤ ; Constructing the adjacency matrix When the element in row i and column k is Represents the connection relationship between the i-th node and the k-th node. If the distance between the two nodes is less than the distance threshold, then ,otherwise , 1≤i≤ , 1≤k≤ .

4. The multi-modal expert collaborative fire risk monitoring method for large-scale forestry resources according to claim 1 is characterized by: In the meteorological expert model; The linear transformation layer is used to calculate the ,Will Convert to high-dimensional data , where , are the weights and biases of the linear transformation layer respectively; , , , is the output dimension of the linear transformation layer; The BiLSTM network is used as input , capturing the dependencies between time steps and outputting the data , , is the output dimension of the BiLSTM network; The Transformer layer is used to Perform feature extraction and output data , , is the output dimension of the Transformer layer; The global pooling layer is used to Perform global pooling and output , , The default fixed dimensions.

5. The multi-modal expert collaborative fire risk monitoring method for large-scale forestry resources according to claim 1 is characterized by: In the terrain expert model; The sliding window layer is used to input the terrain image Perform sliding serial port sampling, and obtain an image block each time sampling; The SwinTransformer network is used to encode each image block and then splice them together to obtain data , ,in, is the preset fixed dimension, H DEM , W DEM Terrain images The height and width of The global pooling layer is used to Perform global pooling and output , .

6. The multi-modal expert collaborative fire risk monitoring method for large-scale forestry resources according to claim 1 is characterized by: When step S4 trains the multimodal monitoring network, one training is specifically as follows: S41, preset the batch size B, select B multimodal samples and send them into the multimodal monitoring network, and calculate the loss function L according to the following formula; ， In the formula, b is the bth multimodal sample in a batch of multimodal samples, c is the cth category, , are the target label and predicted target label of the b-th multimodal sample respectively; S42, update the parameters of the multimodal monitoring network using a back propagation algorithm.

Images