Space-ground cooperative mountain torrent disaster forecasting method

Abstract

The invention relates to a sky-ground collaborative mountain torrent disaster forecasting method, which comprises the following steps of: comprehensively processing multi-source data, and obtaining mountain torrent situation data in a monitoring area based on three-dimensional space interpolation, second-order grid refinement and strong variation space sensing technologies; fusing the mountain torrent situation data and the unmanned aerial vehicle remote sensing image data, comprehensively analyzing water level changes and topographic changes, and building a multi-mode mountain torrent early warning large model; a technology of combining P-Tuning V2 and low-rank adaptive LoRA is adopted to carry out fine tuning on the multi-mode mountain torrent early warning large model; and non-autoregression decoding is adopted to accelerate the reasoning speed of the large model. The mountain torrent early warning large model improves early warning accuracy and timeliness, optimizes emergency response, and provides technical support for flood control and disaster relief. And the fine-tuned mountain torrent early warning large model can predict the mountain torrent risk more accurately and efficiently in real time, process multi-modal data, and provide powerful technical support for disaster prevention and reduction.

Description

Technical Field

[0001] The present invention relates to the technical field of mountain torrent disaster forecasting, and in particular to a method for forecasting mountain torrent disasters in coordination between heaven and earth. Background Art

[0002] The outbreak of flash floods is affected by complex external environments such as meteorology, soil moisture, and river water levels. The dynamic changes of these factors make the forecast of flash floods particularly complex. Currently commonly used technical means include inversion prediction based on hydrological models, physical models, and hydrodynamic models, but these methods are insufficient in the ability to accurately warn in time and place, and are usually limited to a single data source. At the same time, traditional models often have low recognition accuracy when processing unseen data, and the complexity and diversity of flash floods make it difficult for traditional methods to effectively deal with them. In contrast, large models usually contain hundreds of millions to tens of billions or even more parameters, have stronger generalization capabilities, and can learn and store a large number of complex features and information. Large models can process and fuse multimodal information from different data sources to provide more comprehensive analysis and prediction. Therefore, large models can provide more accurate predictions when facing different environments and new data, thereby more effectively carrying out flash flood warnings. Therefore, the present invention constructs a large model for flash flood warning based on multimodal fusion, and the core content includes cleaning, characterization and aggregation technology of multimodal data, as well as pre-training and fine-tuning technology of large models. At the same time, we conduct in-depth research on inference acceleration technology for large models to reduce the computational cost and memory usage of neural network models, minimize precision loss, and improve the inference efficiency of large models in resource-constrained environments. Summary of the invention

[0003] The purpose of the present invention is to provide a method for forecasting mountain torrent disasters in coordination with the heaven and earth, so as to solve the technical problems mentioned in the background technology.

[0004] The technical solution adopted by the present invention to solve its technical problem is:

[0005] A method for predicting mountain torrent disasters in coordination with heaven and earth, comprising the following steps;

[0006] S1. Flash flood situation assessment: Comprehensively process multi-source data, which comes from rainfall forecast, ground sensors, drones, etc. The multi-source data specifically includes rainfall forecast data, cloud volume collected by drones, soil settlement, rock fragmentation, soil moisture, river water level and flow data collected by ground sensors, and obtain flash flood situation data in the monitoring area based on three-dimensional spatial interpolation, second-order grid refinement and strong variation spatial perception technology;

[0007] S2. Construct a large model for flash flood warning: integrate flash flood situation data and drone remote sensing image data, comprehensively analyze water level changes, terrain changes and other information, and build a multi-modal flash flood warning model;

[0008] S3. Fine-tuning of large models: The multi-modal flash flood warning large model is fine-tuned using a combination of P-TuningV2 and low-rank adaptive LoRA technology;

[0009] S4. Non-autoregressive decoding inference acceleration: Non-autoregressive decoding is used to accelerate the inference speed of large models.

[0010] As a further technical solution of the present invention, step S1 includes the following steps:

[0011] The two-dimensional plane representation of the deployed sensors in the flash flood monitoring area is obtained, and the three-dimensional distribution of the sensors in the three-dimensional space is obtained based on the adaptive ellipse space interpolation method. In the three-dimensional distribution, each visualized line segment affects the surrounding area, which is called the affected area. The shape of the affected area is an ellipse, and its curvature is adaptively matched with the length of the line segment. The values ​​of all grid cells in the affected area are equal to the attribute values ​​of the corresponding line segments. If a grid cell P is located in multiple affected areas at the same time, the value of the grid cell is jointly determined by the values ​​of these affected areas based on the ToP-k method. First, the values ​​v of all affected areas related to the grid cell P are calculated. i Sort and get a sorted value sequence v1,v2,...,v n , where v1≥v2≥···≥v n , select the first k largest values ​​from the sorted sequence, that is, v1,v2,...,v k , the final value V of the grid cell P P By taking a time-decayed weighted average of these k values, the final value V P The calculation process is shown in Formula 1:

[0012]

[0013] where w j is the corresponding v j The weight of j is the corresponding v j The timestamp of , λ is the time decay rate constant;

[0014] By repeating the above process, the distribution of all sensor information in the three-dimensional space of the monitoring area can be obtained. This project intends to spatially fuse the multi-dimensional features to generate a three-dimensional grid distribution for flash flood situation assessment, and use the second-order refinement method to refine the monitoring area space to achieve accurate assessment of flash flood situation;

[0015] In the first stage of grid refinement, a sensor signal acquisition optimization strategy is proposed to use three-dimensional segmentation technology to preliminarily determine the flash flood area. First, the final environmental situation influence coefficient E is obtained through environmental situation information. env(x,y,z), the environmental situation information is a multidimensional vector Z(x,y,z)=[e1(x,y,z),e2(x,y,z),...,e m (x,y,z)], where e j (x, y, z) represents the value of the jth environmental factor at point (x, y, z), m is the number of environmental factors, and the final environmental situation impact coefficient calculation process is shown in Formula 2:

[0016]

[0017] where β j is the weight associated with the jth environmental factor, reflecting its impact on flash flood disaster assessment. env The spatial distribution of the flash flood area of ​​(x,y,z) is V'(x,y,z)=V(x,y,z)·E env (x, y, z), where V(x, y, z) is the original flash flood area distribution obtained by three-dimensional segmentation technology;

[0018] Define indicators to establish the connection between the flash flood area distribution pattern and the sensor spatial layout. The indicators include trajectory crossing ratio and trajectory-flash flood area center distance, and construct reasonable optimization conditions and fitness functions. The fitness function is constructed as shown in Formula 3:

[0019]

[0020] in is the average distance between the trajectory and the center of the flash flood area, is the average value of the environmental situation influence coefficients of all points on the trajectory, and α1, α2, α3 and α4 are the corresponding weight coefficients;

[0021] The global optimization algorithm is applied to search for the optimal solution of the sensor spatial distribution in the flash flood monitoring area. The algorithm obtains the optimal spatial layout of the sensor by inputting the initial position of the sensor, the objective function and the constraint conditions, and then performs secondary signal acquisition to generate a more refined three-dimensional grid distribution. In the second stage of grid refinement, a filter based on the GaP-filled method is proposed to resample the grid group to further generate a high-resolution three-dimensional grid distribution for more accurate characterization of the flash flood situation in the monitoring area. Finally, a multi-scale Gaussian difference model is proposed to enhance the attributes of the entire tree space based on the visual preference mechanism. The principle is shown in Formula 4:

[0022]

[0023] where σ i represents the standard deviation of the i-th scale, and G'(x,y,z) is a three-dimensional grid.

[0024] At the same time, a strong variation spatial perception function is designed. The spatial variation degree of each point in the monitoring area is obtained by combining the spatial data of the monitoring area with the environmental situation information. Directional filtering is used to obtain significant edge information. The boundaries of the flash flood area are screened in combination with the inherent variation. The flash flood situation data in the monitoring area is obtained in combination with the multi-dimensional environmental situation information.

[0025] As a further technical solution of the present invention, step S2 includes the following steps:

[0026] First, the UAV remote sensing image data (such as terrain changes, vegetation coverage, etc.) and flash flood situation data (such as water level, rainfall, etc.) are input into the linear mapping layer and embedded into a unified feature space. Then, the features are deepened layer by layer through multiple Transformer encoder layers. Each encoder layer includes position encoding, multi-head self-attention mechanism and feedforward neural network. The position encoding module helps the model understand the temporal characteristics of sequence data, such as the impact of rainfall at different time points on flash floods. The multi-head self-attention mechanism can capture the complex relationship between different modal data, such as the association between geomorphic features in remote sensing images and flash flood situation data.

[0027] For a given input sequence X = [x1, x2, ..., x n ], where each x i Is a vector of dimension d, the self-attention mechanism converts it into query Q, key K and value V, vector Q = XW Q , K = XW K , V = XW V , where W Q , W K , W V is a learnable weight matrix;

[0028] The output of self-attention is obtained by calculating the dot product between the query and the key to obtain the attention weight, and then the value is weighted averaged using the attention weight. The specific process is shown in Formula 5:

[0029]

[0030] in is a scaling factor, d k is the dimension of the key vector;

[0031] The multi-head self-attention mechanism uses multiple different linear transformation matrices to perform multiple different self-attention calculations on the input sequence, and then concatenates and linearly transforms the results of these different attention heads to obtain the final output. The multi-head self-attention mechanism captures the features in the input data from different angles by concatenating the outputs of h given attention heads. The process is shown in Formula 6:

[0032] Formula 6: MultiHead(Q,K,V)=Concat(head1,head n ,...,head h )W o ;

[0033] Where W O is the final linear transformation matrix, which is used to map the concatenated result back to the original dimension;

[0034] Finally, the model maps the extracted high-level features to the output space of flash flood warning through the fully connected layer and the linear layer, thereby realizing the probability prediction of flash floods.

[0035] For the proposed multimodal flash flood warning model, a multi-task joint loss function is proposed to simultaneously optimize the characteristics of flash flood situation data and UAV remote sensing image data and the flash flood forecast accuracy. The specific expression of the loss function is shown in Formula 7:

[0036] Formula 7: L = αL t +βL i +γL f +δL r ;

[0037] Where L t Used to indicate the loss of flash flood situation data, L i Used to represent the loss of remote sensing image data, L f is the multimodal consistency loss, which aims to minimize the difference between feature representations of different modalities. r is a regularization term used to prevent the model from overfitting, and α, β, γ, and δ are weight hyperparameters that determine the contribution of each loss term to the total loss. This design of multi-task joint loss function can effectively optimize the dual-channel large model and make it perform well in multimodal flash flood monitoring tasks.

[0038] As a further technical solution of the present invention, step S3 includes the following steps:

[0039] Compared with F-Tuning, P-Tuning only optimizes a small segment of learnable continuous task-specific vectors (Prefix) during the model adjustment process, rather than the parameters of the entire model. This method solves the problem that the effect of traditional prompt learning methods is highly dependent on the quality of manually designed templates. P-TuningV2 adds continuous prompts to the front end of the sequence and introduces trainable prompts at each layer. In addition, P-TuningV2 removes reparameterization, abandons the vocabulary mapping vocabulary, reuses CLS and character labels, enhances versatility, and can better adapt to sequence labeling tasks. P-TuningV2 no longer uses reparameterization for representation, and no longer replaces pre-trained word embeddings, but directly fine-tunes the parameters of the deep model corresponding to the prompt markers.

[0040] Different from the prompt-based P-Tuning algorithm, the low-rank adaptive LoRA freezes the weights of the pre-trained model, injects the trainable low-rank decomposition matrix into the Transformer layer and operates in parallel with it, thereby significantly reducing the number of trainable parameters of downstream tasks. During the training process, the parameter W of the pre-trained model remains unchanged and does not receive gradient updates. The parameter matrices A and B introduced by LoRA contain trainable parameters and are calculated in parallel with the parameter W. W and ΔW = BA both process the same input, and finally sum their outputs, where B∈R d×r , A∈R r×d , the parameter matrix ΔW satisfies r<<min(d,k);

[0041] P-TuningV2 improves the model's adaptability to tasks by optimizing task-specific prompts, while LoRA improves training efficiency by reducing the number of trainable parameters. By combining P-TuningV2 and Lora to fine-tune the model, it can not only process multimodal data in flash flood warnings more accurately, but also achieve efficient optimization under limited resource conditions.

[0042] As a further technical solution of the present invention, step S4 includes the following steps:

[0043] Non-autoregressive decoding is used to accelerate the inference speed of large models and achieve rapid response to sudden flash flood disasters. Non-autoregressive decoding generates all elements in the sequence in parallel. After the model inputs the information at the initial moment t0, it directly generates warning information at subsequent moments t1, t2, t3, and t4 without waiting for the generation result of the previous moment. Assuming that the input information is X, the target output sequence is Y = (y1, y2, ..., y T ), the goal of non-autoregressive decoding is to directly generate the probability distribution of the entire sequence, as shown in Formula 8:

[0044]

[0045] Among them, each y t It depends only on the input information X and not on other elements in the sequence, so all y can be generated in parallel t .

[0046] The beneficial effects of the present invention are:

[0047] The flash flood warning large model combines flash flood situation information and drone remote sensing image data, and can comprehensively analyze multimodal data such as water level changes, rainfall and terrain, greatly improving the accuracy of warnings and reducing false alarms and missed reports. The model can process data in real time and quickly generate warning information, providing a critical time window for emergency rescue. It is also good at processing complex, multi-source data, extracting effective features, and comprehensively assessing risk areas to help decision makers deploy resources in advance and reduce losses. The large model has adaptive capabilities, can continue to learn, and maintain efficient warnings. Overall, the model improves the accuracy and timeliness of warnings, optimizes emergency responses, and provides technical support for flood control and disaster relief.

[0048] In the present invention, fine-tuning of the large model can make the pre-trained model better adapt to the flash flood warning task, optimize the flash flood situation information and drone remote sensing data, and improve the warning accuracy and risk identification ability. At the same time, fine-tuning retains the generalization ability of the model, enabling it to effectively handle flood characteristics and complex scenes in different regions and enhance its adaptability to tasks. Compared with training from scratch, fine-tuning only needs to update some parameters, reducing computing resources and time consumption, and is particularly suitable for the application of large-scale data and complex models. In addition, through fine-tuning, the model can better integrate multimodal data, improve the perception and prediction capabilities of complex environments, thereby enhancing real-time warning performance, quickly responding to data changes, and improving the speed and accuracy of warnings. Fine-tuning can also be customized and optimized according to different regions, climate conditions or disaster types to enhance the flexibility and adaptability of the model. In general, the fine-tuned flash flood warning large model can predict flash flood risks more accurately, in real time and efficiently, process multimodal data, and provide strong technical support for disaster prevention and mitigation.

[0049] In the large model of flash flood warning, non-autoregressive decoding inference acceleration has significant advantages. First, it can break the limitation of autoregressive step-by-step generation by processing multiple time steps in parallel, significantly improve the inference efficiency, and ensure that the system can quickly generate warning results, which is crucial for real-time monitoring of flash flood situations. Secondly, non-autoregressive decoding is particularly suitable for processing long sequence data, such as water level changes and rainfall trends, which greatly reduces latency and ensures that the model is still efficient when facing complex data. In addition, inference acceleration can also help the system make emergency decisions in a shorter time, take preventive measures in advance, and reduce disaster losses. At the same time, this acceleration method optimizes resource utilization efficiency. Even in scenarios with limited computing resources, the model can run efficiently, which is particularly suitable for flash flood warning systems that process massive multimodal data in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flow chart of a method for forecasting flash flood disasters in coordination between heaven and earth provided in an embodiment of the present invention.

[0051] Figure 2 A flow chart of flash flood situation assessment in a ground-sky coordinated flash flood disaster forecasting method provided in an embodiment of the present invention.

[0052] Figure 3 A flow chart of a large model for flash flood warning in a method for forecasting flash flood disasters in coordination between heaven and earth provided in an embodiment of the present invention.

[0053] Figure 4 A flowchart of fine-tuning a large model in a method for forecasting flash flood disasters in coordination with heaven and earth provided in an embodiment of the present invention.

[0054] Figure 5 A flowchart of non-autoregressive decoding inference acceleration in a method for forecasting flash flood disasters in coordination with heaven and earth provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] See also Figure 1 , the embodiment of the present invention provides a method for forecasting mountain torrent disasters in coordination between heaven and earth, comprising the following steps;

[0057] S1. Flash flood situation assessment: Comprehensively process multi-source data, which comes from rainfall forecast, ground sensors, drones, etc. The multi-source data specifically includes rainfall forecast data, cloud volume collected by drones, soil settlement, rock fragmentation, soil moisture, river water level and flow data collected by ground sensors, and obtain flash flood situation data in the monitoring area based on three-dimensional spatial interpolation, second-order grid refinement and strong variation spatial perception technology;

[0058] See also Figure 2 In this embodiment, in order to accurately assess the flash flood situation, the present invention conducts a study on the comprehensive assessment of the chain disaster of rainstorm-flash floods in coordination with the sky and the earth based on the rainfall forecast data, the amount of fog collected by the drone, and the soil settlement, rock crushing degree, soil moisture, river water level and flow collected by the ground sensors; each sensor and drone can provide the real-time environmental status of the surrounding area, and this information will be continuously updated over time to form environmental situation information. Various types of sensors are deployed in the flash flood monitoring area, and a multi-dimensional ground environmental situation distribution is formed through the same sensors at different locations. Combined with the rainfall situation and the atmospheric fog situation, different environmental situations are integrated and refined, and finally an accurate flash flood situation assessment result is obtained. The present invention constructs a flash flood situation assessment model that combines three-dimensional spatial interpolation, second-order grid refinement and strong variation spatial perception to achieve an accurate assessment of the flash flood situation in the monitoring area.

[0059] The two-dimensional plane representation of the deployed sensors in the flash flood monitoring area is obtained, and the three-dimensional distribution of the sensors in the three-dimensional space is obtained based on the adaptive ellipse space interpolation method. In the three-dimensional distribution, each visualized line segment affects the surrounding area, which is called the affected area. The shape of the affected area is an ellipse, and its curvature is adaptively matched with the length of the line segment. The values ​​of all grid cells in the affected area are equal to the attribute values ​​of the corresponding line segments. If a grid cell P is located in multiple affected areas at the same time, the value of the grid cell is jointly determined by the values ​​of these affected areas based on the ToP-k method. First, the values ​​v of all affected areas related to the grid cell P are calculated. i Sort and get a sorted value sequence v1,v2,...,v n , where v1≥v2≥···≥v n , select the first k largest values ​​from the sorted sequence, that is, v1,v2,...,v k , the final value V of the grid cell P P By taking a time-decayed weighted average of these k values, the final value V P The calculation process is shown in Formula 1:

[0060]

[0061] where w jis the corresponding v j The weight of j is the corresponding v j The timestamp of , λ is the time decay rate constant;

[0062] By repeating the above process, the distribution of all sensor information in the three-dimensional space of the monitoring area can be obtained. This project intends to spatially fuse the multi-dimensional features to generate a three-dimensional grid distribution for flash flood situation assessment, and use the second-order refinement method to refine the monitoring area space to achieve accurate assessment of flash flood situation;

[0063] In the first stage of grid refinement, a sensor signal acquisition optimization strategy is proposed to use three-dimensional segmentation technology to preliminarily determine the flash flood area. First, the final environmental situation influence coefficient E is obtained through environmental situation information. env (x,y,z), the environmental situation information is a multidimensional vector Z(x,y,z)=[e1(x,y,z),e2(x,y,z),...,e m (x,y,z)], where e j (x, y, z) represents the value of the jth environmental factor at point (x, y, z), m is the number of environmental factors, and the final environmental situation impact coefficient calculation process is shown in Formula 2:

[0064]

[0065] where β j is the weight associated with the jth environmental factor, reflecting its impact on flash flood disaster assessment. env The spatial distribution of the flash flood area of ​​(x,y,z) is V'(x,y,z)=V(x,y,z)·E env (x, y, z), where V(x, y, z) is the original flash flood area distribution obtained by three-dimensional segmentation technology;

[0066] Define indicators to establish the connection between the flash flood area distribution pattern and the sensor spatial layout. The indicators include trajectory crossing ratio and trajectory-flash flood area center distance, and construct reasonable optimization conditions and fitness functions. The fitness function is constructed as shown in Formula 3:

[0067]

[0068] in is the average distance between the trajectory and the center of the flash flood area, is the average value of the environmental situation influence coefficient of all points on the trajectory, α1, α2, α 3 and α4 are the corresponding weight coefficients;

[0069] The global optimization algorithm is applied to search for the optimal solution of the sensor spatial distribution in the flash flood monitoring area. The algorithm obtains the optimal spatial layout of the sensor by inputting the initial position of the sensor, the objective function and the constraint conditions, and then performs secondary signal acquisition to generate a more refined three-dimensional grid distribution. In the second stage of grid refinement, a filter based on the GaP-filled method is proposed to resample the grid group to further generate a high-resolution three-dimensional grid distribution for more accurate characterization of the flash flood situation in the monitoring area. Finally, a multi-scale Gaussian difference model is proposed to enhance the attributes of the entire tree space based on the visual preference mechanism. The principle is shown in Formula 4:

[0070]

[0071] where σ i represents the standard deviation of the i-th scale, and G'(x,y,z) is a three-dimensional grid.

[0072] At the same time, a strong variation spatial perception function is designed. The spatial variation degree of each point in the monitoring area is obtained by combining the spatial data of the monitoring area with the environmental situation information. Directional filtering is used to obtain significant edge information. The boundaries of the flash flood area are screened in combination with the inherent variation. The flash flood situation data in the monitoring area is obtained in combination with the multi-dimensional environmental situation information.

[0073] S2. Construct a large model for flash flood warning: integrate flash flood situation data and drone remote sensing image data, comprehensively analyze water level changes, terrain changes and other information, and build a multi-modal flash flood warning model;

[0074] See also Figure 3 In step S2, first, the UAV remote sensing image data (such as terrain changes, vegetation coverage, etc.) and flash flood situation data (such as water level, rainfall, etc.) are input into the linear mapping layer and embedded into a unified feature space. Then, the features are deepened layer by layer through multiple Transformer encoder layers. Each encoder layer includes position encoding, multi-head self-attention mechanism and feedforward neural network. The position encoding module helps the model understand the temporal characteristics of sequence data, such as the impact of rainfall at different time points on flash floods. The multi-head self-attention mechanism can capture the complex relationship between different modal data, such as the association between geomorphic features in remote sensing images and flash flood situation data.

[0075] For a given input sequence X = [x1, x2, ..., x n ], where each x i Is a vector of dimension d, the self-attention mechanism converts it into query Q, key K and value V, vector Q = XW Q , K = XW K , V = XW V , where WQ , W K , W V is a learnable weight matrix;

[0076] The output of self-attention is obtained by calculating the dot product between the query and the key to obtain the attention weight, and then the value is weighted averaged using the attention weight. The specific process is shown in Formula 5:

[0077]

[0078] in is a scaling factor, d k is the dimension of the key vector;

[0079] The multi-head self-attention mechanism uses multiple different linear transformation matrices to perform multiple different self-attention calculations on the input sequence, and then concatenates and linearly transforms the results of these different attention heads to obtain the final output. The multi-head self-attention mechanism captures the features in the input data from different angles by concatenating the outputs of h given attention heads. The process is shown in Formula 6:

[0080] Formula 6: MultiHead(Q,K,V)=Concat(head1,head n ,...,head h )W o ;

[0081] Where W O is the final linear transformation matrix, which is used to map the concatenated result back to the original dimension;

[0082] Finally, the model maps the extracted high-level features to the output space of flash flood warning through the fully connected layer and the linear layer, thereby realizing the probability prediction of flash floods.

[0083] For the proposed multimodal flash flood warning model, a multi-task joint loss function is proposed to simultaneously optimize the characteristics of flash flood situation data and UAV remote sensing image data and the flash flood forecast accuracy. The specific expression of the loss function is shown in Formula 7:

[0084] Formula 7: L = αL t +βL i +γL f +δL r ;

[0085] Where L t Used to indicate the loss of flash flood situation data, L i Used to represent the loss of remote sensing image data, L f is the multimodal consistency loss, which aims to minimize the difference between feature representations of different modalities. ris a regularization term used to prevent model overfitting, and α, β, γ, and δ are weight hyperparameters that determine the contribution of each loss term to the total loss. This design of multi-task joint loss function can effectively optimize the dual-channel large model and make it perform well in multimodal flash flood monitoring tasks.

[0086] S3, large model fine-tuning: P-TuningV2 and low-rank adaptation (LoRA) technology are used to fine-tune the multimodal flash flood warning large model;

[0087] See also Figure 4 , the pre-trained large models are usually trained on a wide range of general data sets, while the actual flash flood warning data may have unique characteristics. Fine-tuning can make the model better adapt to these specific data characteristics and improve the accuracy and applicability of the prediction. At the same time, fine-tuning can achieve the optimization of the model in practical applications without consuming a lot of computing resources. The present invention intends to fine-tune the multimodal flash flood warning large model by combining P-TuningV2 and low-rank adaptation (LoRA).

[0088] In step S3, compared with F-Tuning, P-Tuning only optimizes a small segment of learnable continuous task-specific vectors (Prefix) in the process of adjusting the model, rather than the parameters of the entire model; this method solves the problem that the effect of traditional prompt learning methods is highly dependent on the quality of manually designed templates. P-TuningV2 adds continuous prompts to the front end of the sequence and introduces trainable prompts at each layer. In addition, P-TuningV2 removes reparameterization, abandons the vocabulary mapping vocabulary, reuses CLS and character labels, enhances versatility, and can better adapt to sequence labeling tasks. P-TuningV2 no longer uses reparameterization for representation, and no longer replaces pre-trained word embeddings, but directly fine-tunes the parameters of the deep model corresponding to the prompt markers;

[0089] Different from the prompt-based P-Tuning algorithm, the low-rank adaptive LoRA freezes the weights of the pre-trained model, injects the trainable low-rank decomposition matrix into the Transformer layer and operates in parallel with it, thereby significantly reducing the number of trainable parameters of downstream tasks. During the training process, the parameter W of the pre-trained model remains unchanged and does not receive gradient updates. The parameter matrices A and B introduced by LoRA contain trainable parameters and are calculated in parallel with the parameter W. W and ΔW = BA both process the same input, and finally sum their outputs, where B∈R d×r , A∈R r×d , the parameter matrix ΔW satisfies r<<min(d,k).

[0090] P-TuningV2 improves the model's adaptability to tasks by optimizing task-specific prompts, while LoRA improves training efficiency by reducing the number of trainable parameters. By combining P-TuningV2 and Lora to fine-tune the model, it can not only process multimodal data in flash flood warnings more accurately, but also achieve efficient optimization under limited resource conditions.

[0091] S4. Non-autoregressive decoding inference acceleration: Non-autoregressive decoding is used to accelerate the inference speed of large models.

[0092] See also Figure 5 Flash flood disasters are characterized by strong suddenness and rapid development. Traditional early warning methods often find it difficult to provide accurate and timely early warning information in a short period of time. This leads to a slow response to flash flood disasters and poor early warning effects, thereby expanding the scope of disaster impact. When processing sequence data, the traditional autoregressive decoding method has a slow generation speed due to the serialization characteristics of its generation process, that is, the generation of each element depends on the previous element. This sequence generation method is difficult to meet the high real-time requirements of flash flood warnings and cannot provide timely and effective decision-making support for disaster prevention and mitigation. To this end, the present invention intends to use non-autoregressive decoding to accelerate the inference speed of large models and achieve rapid response to sudden flash flood disasters. Non-autoregressive decoding generates all elements in the sequence in parallel. After the model inputs the information at the initial time t0, it directly generates warning information for subsequent multiple time points t1, t2, t3, and t4 without waiting for the generation result of the previous time point. Assuming that the input information is X, the target output sequence is Y=(y1, y2, ..., y T ), the goal of non-autoregressive decoding is to directly generate the probability distribution of the entire sequence, as shown in Formula 8:

[0093]

[0094] Among them, each y t It depends only on the input information X and not on other elements in the sequence, so all y can be generated in parallel t . This parallel generation method significantly improves the model's reasoning speed, allowing the model to integrate meteorological and remote sensing data more quickly and generate real-time and accurate flash flood warning information. Specifically, non-autoregressive decoding avoids the time delay caused by step-by-step generation by processing the generation process at all moments in parallel, thereby significantly shortening the reasoning time. For a sequence of length T, the traditional autoregressive method requires O(T) iterations, while the non-autoregressive method can complete the generation of the entire sequence in one parallel operation, theoretically reducing the reasoning time to O(1). This significant time reduction improves the real-time performance and efficiency of the flash flood warning model, making the reasoning process of the flash flood warning model faster and more accurate.

[0095] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0096] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for predicting mountain torrent disasters in coordination with heaven and earth, characterized in that: The steps include: S1. Flash flood situation assessment: Comprehensively process multi-source data, which comes from rainfall forecast, sensors, drones, etc. The multi-source data specifically includes rainfall forecast data, cloud volume collected by drones, soil settlement, rock fragmentation, soil moisture, and river water level and flow collected by sensors, and obtain flash flood situation data in the monitoring area based on three-dimensional spatial interpolation, second-order grid refinement and strong variation spatial perception technology; S2. Construct a large model for flash flood warning: integrate flash flood situation data and drone remote sensing image data, comprehensively analyze water level changes and terrain changes, and build a multi-modal flash flood warning large model; S3. Fine-tuning of large models: The multi-modal flash flood warning large model is fine-tuned using a combination of P-TuningV2 and low-rank adaptive LoRA technology; S4. Non-autoregressive decoding inference acceleration: Non-autoregressive decoding is used to accelerate the inference speed of large models.

2. The method for predicting mountain torrent disasters in coordination with heaven and earth according to claim 1, characterized in that: Step S1 includes the following steps: The two-dimensional plane representation of the deployed sensors in the flash flood monitoring area is obtained, and the three-dimensional distribution of the sensors in the three-dimensional space is obtained based on the adaptive ellipse space interpolation method. In the three-dimensional distribution, each visualized line segment affects the surrounding area, which is called the affected area. The shape of the affected area is an ellipse, and its curvature is adaptively matched with the length of the line segment. The values ​​of all grid cells in the affected area are equal to the attribute values ​​of the corresponding line segments. If a grid cell P is located in multiple affected areas at the same time, the value of the grid cell is jointly determined by the values ​​of these affected areas based on the ToP-k method. First, the values ​​v of all affected areas related to the grid cell P are calculated. i Sort and get a sorted value sequence v1,v2,...,v n , where v1≥v2≥···≥v n , select the first k largest values ​​from the sorted sequence, that is, v1,v2,...,v k , the final value V of the grid cell P P By taking a time-decayed weighted average of these k values, the final value V P The calculation process is shown in Formula 1: Formula 1: where w j is the corresponding v j The weight of j is the corresponding v j The timestamp of , λ is the time decay rate constant; In the first stage of grid refinement, a sensor signal acquisition optimization strategy is proposed to use three-dimensional segmentation technology to preliminarily determine the flash flood area. First, the final environmental situation influence coefficient E is obtained through environmental situation information. env (x,y,z), the environmental situation information is a multidimensional vector Z(x,y,z)=[e1(x,y,z),e2(x,y,z),...,e m (x,y,z)], where e j (x, y, z) represents the value of the jth environmental factor at point (x, y, z), m is the number of environmental factors, and the final environmental situation impact coefficient calculation process is shown in Formula 2: Formula 2: where β j is the weight associated with the jth environmental factor, reflecting its impact on flash flood disaster assessment. env The spatial distribution of the flash flood area of ​​(x,y,z) is V'(x,y,z)=V(x,y,z)·E env (x, y, z), where V(x, y, z) is the original flash flood area distribution obtained by three-dimensional segmentation technology; Define indicators to establish the connection between the flash flood area distribution pattern and the sensor spatial layout. The indicators include trajectory crossing ratio and trajectory-flash flood area center distance, and construct reasonable optimization conditions and fitness functions. The fitness function is constructed as shown in Formula 3: Formula 3: in is the average distance between the trajectory and the center of the flash flood area, is the average value of the environmental situation influence coefficients of all points on the trajectory, and α1, α2, α3 and α4 are the corresponding weight coefficients; The global optimization algorithm is applied to search for the optimal solution of the sensor spatial distribution in the flash flood monitoring area. The algorithm obtains the optimal spatial layout of the sensor by inputting the initial position of the sensor, the objective function and the constraint conditions, and then performs secondary signal acquisition to generate a more refined three-dimensional grid distribution. In the second stage of grid refinement, a filter based on the GaP-filled method is proposed to resample the grid group to further generate a high-resolution three-dimensional grid distribution for more accurate characterization of the flash flood situation in the monitoring area. Finally, a multi-scale Gaussian difference model is proposed to enhance the attributes of the entire tree space based on the visual preference mechanism. The principle is shown in Formula 4: where σ i represents the standard deviation of the i-th scale, G'(x,y,z) is a three-dimensional grid; At the same time, a strong variation spatial perception function is designed. The spatial variation degree of each point in the monitoring area is obtained by combining the spatial data of the monitoring area with the environmental situation information. Directional filtering is used to obtain significant edge information. The boundaries of the flash flood area are screened in combination with the inherent variation. The flash flood situation data in the monitoring area is obtained in combination with the multi-dimensional environmental situation information.

3. The method for predicting mountain torrent disasters in coordination with heaven and earth according to claim 1, characterized in that: Step S2 includes the following steps: First, the UAV remote sensing image data and flash flood situation data are input into the linear mapping layer respectively and embedded into a unified feature space. Then, the features are deepened layer by layer through multiple Transformer encoder layers. Each encoder layer includes position encoding, multi-head self-attention mechanism and feedforward neural network. The position encoding module helps the model understand the temporal characteristics of sequence data, and the multi-head self-attention mechanism captures the complex relationship between different modal data. For a given input sequence X = [x1, x2, ..., x n ], where each x i Is a vector of dimension d, the self-attention mechanism converts it into query Q, key K and value V, vector Q = XW Q , K = XW K , V = XW V , where W Q , W K , W V is a learnable weight matrix; The output of self-attention is obtained by calculating the dot product between the query and the key to obtain the attention weight, and then the value is weighted averaged using the attention weight. The specific process is shown in Formula 5: Formula 5: in is a scaling factor, d k is the dimension of the key vector; The multi-head self-attention mechanism uses multiple different linear transformation matrices to perform multiple different self-attention calculations on the input sequence, and then concatenates and linearly transforms the results of these different attention heads to obtain the final output. The multi-head self-attention mechanism captures the features in the input data from different angles by concatenating the outputs of h given attention heads. The process is shown in Formula 6: Equation 6: MultiHead(Q, K, V) = Concat(head1, head n ,..., head h )W o ; Where W O is the final linear transformation matrix, which is used to map the concatenated result back to the original dimension; Finally, the model maps the extracted high-level features to the output space of flash flood warning through the fully connected layer and the linear layer, thereby realizing the probability prediction of flash floods. For the proposed multimodal flash flood warning model, a multi-task joint loss function is proposed to simultaneously optimize the characteristics of flash flood situation data and UAV remote sensing image data and the flash flood forecast accuracy. The specific expression of the loss function is shown in Formula 7: Equation 7: L = αL t + βL i + γL f + δL r ; Where L t Used to indicate the loss of flash flood situation data, L i Used to represent the loss of remote sensing image data, L f is the multimodal consistency loss, which aims to minimize the difference between feature representations of different modalities. r is a regularization term used to prevent the model from overfitting, and α, β, γ, and δ are weight hyperparameters that determine the contribution of each loss term to the total loss.

4. The method for predicting mountain torrent disasters in coordination with heaven and earth according to claim 3, characterized in that: Step S3 includes the following steps: P-Tuning only optimizes a small segment of learnable continuous task-specific vectors rather than the parameters of the entire model during the model adjustment process. P-TuningV2 adds continuous prompts to the front end of the sequence and introduces trainable prompts at each layer. In addition, P-TuningV2 removes reparameterization, abandons the vocabulary of vocabulary mapping, reuses CLS and character labels, enhances versatility, and can better adapt to sequence labeling tasks. P-TuningV2 no longer uses reparameterization for representation and no longer replaces pre-trained word embeddings, but directly fine-tunes the parameters of the deep model corresponding to the prompt markers; Different from the prompt-based P-Tuning algorithm, the low-rank adaptive LoRA freezes the weights of the pre-trained model, injects the trainable low-rank decomposition matrix into the Transformer layer and operates in parallel with it, thereby significantly reducing the number of trainable parameters of downstream tasks. During the training process, the parameter W of the pre-trained model remains unchanged and does not receive gradient updates. The parameter matrices A and B introduced by LoRA contain trainable parameters and are calculated in parallel with the parameter W. W and ΔW = BA both process the same input, and finally sum their outputs, where B∈R d×r , A∈R r×d , the parameter matrix ΔW satisfies r<<min(d,k).

5. The method for predicting mountain torrent disasters in coordination with heaven and earth according to claim 1, characterized in that: Step S4 includes the following steps: Non-autoregressive decoding is used to accelerate the inference speed of large models and achieve rapid response to sudden flash flood disasters. Non-autoregressive decoding generates all elements in the sequence in parallel. After the model inputs the information at the initial moment t0, it directly generates warning information at subsequent moments t1, t2, t3, and t4 without waiting for the generation result of the previous moment. Assuming that the input information is X, the target output sequence is Y = (y1, y2, ..., y T ), the goal of non-autoregressive decoding is to directly generate the probability distribution of the entire sequence, as shown in Formula 8: Formula 8: Among them, each y t It depends only on the input information X and not on other elements in the sequence, so all y can be generated in parallel t .

Images