Single-mode landslide identification prediction model and landslide identification prediction base model training method, landslide identification prediction model localization method and landslide identification prediction method, landslide identification prediction system

Abstract

The present disclosure provides a training method for a single-mode landslide identification prediction model and a landslide identification prediction base model, a landslide identification prediction model localization method, a landslide identification prediction method, and a landslide identification prediction system, and relates to computer systems based on specific models, disaster prediction, deep learning, model training, model training and fine-tuning, and other fields. Each first device end uses the collected single-mode landslide data to train a single-mode landslide identification prediction model, and the second device end fuses the model parameters of multiple single-mode landslide identification prediction models to obtain a landslide identification prediction base model. The third device end sends the target mode to the second device end, and the model that is cut or lightweighted according to the target mode is fine-tuned using the localized landslide training sample to obtain the localized landslide identification model of each third device end, thereby improving the accuracy of landslide identification prediction and obtaining disaster prediction results in the absence of original data.

Description

Technical Field

[0001] The present disclosure belongs to the field of computer systems and data processing based on specific models, and specifically relates to technical fields such as disaster prediction, deep learning algorithms, machine learning, model training, model training and fine-tuning, and model localization, and in particular to a training method for a single-modal landslide identification and prediction model, a training method for a landslide identification and prediction base model, a localization method for a landslide identification and prediction model, a landslide identification and prediction method, a landslide identification and prediction system, and corresponding systems, electronic devices, computer-readable storage media, and computer program products. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] Existing artificial intelligence models mainly adopt a centralized architecture, with training data located on the server side and model training also performed on the server side. This leads to problems of data privacy leakage and server performance bottlenecks, and cannot meet the needs of landslide monitoring data security and continuous collection of landslide monitoring data.

[0004] For address disaster identification, since it may involve the use of high-precision maps and address data, the collected raw data involves national security, and different departments are reluctant to share their own data. In addition, due to the improvement of legal compliance, the free flow of raw data has become increasingly difficult. How to ensure data security and obtain disaster prediction results while missing raw data has become particularly urgent.

[0005] The existing landslide model training must require the training data to be carried out on the server side, which requires the transmission of a large amount of raw data. There are problems of data leakage and server performance bottlenecks, which cannot meet the needs of landslide monitoring data security and continuous collection of landslide monitoring data.

[0006] When a server with a centralized architecture needs to interact with multiple interactive objects, it needs to wait for data from all interactive objects. Different interactive objects take different amounts of time to process data, making it difficult to implement asynchronous processing, resulting in low efficiency of the server with a centralized architecture.

[0007] The prior art predicts the probability of landslides in a region, such as predicting certain regions or grids and obtaining the probability of landslides in the regions or grids. However, the location coordinates (longitude and latitude) of the landslide cannot be accurately predicted, which increases the scope of detection and search by address detection professionals.

[0008] In order to obtain more features of the same object in the same place, it is often necessary to preprocess multi-source heterogeneous data of the same object to obtain aligned data or consistent data. The preprocessing process requires writing specific algorithms to process the data, which consumes computing resources and loses accuracy. Some people cannot process heterogeneous data from different objects and sources, making disaster prediction impossible or inaccurate.

[0009] Therefore, accurate identification of landslide hazards is an urgent problem to be solved. Summary of the invention

[0010] In view of the problems existing in the above-mentioned prior art, a training method for a single-mode landslide identification and prediction model, a training method for a landslide identification and prediction base model, a localization method for a landslide identification and prediction model, a landslide identification and prediction method, and a matching landslide identification and prediction system, an electronic device, a computer-readable storage medium and a computer program product are proposed. By using this method, electronic device and computer-readable storage medium, all or part of the above-mentioned problems can be solved.

[0011] In a first aspect, an embodiment of the present disclosure proposes a training method for a unimodal landslide identification and prediction model, which is applied to a first device end, and includes: obtaining unimodal landslide data collected by a preset unimodal landslide data collection device, and the unimodal landslide data includes any of the following: satellite optical image data, airborne optical image data, satellite synthetic aperture radar image data, ground-based synthetic aperture radar data, airborne synthetic aperture radar data, airborne lidar data, on-site survey photos, monitoring data collected by surface sensors of the landslide body, and monitoring data collected by sensors inside the slope body; using the unimodal landslide data as sample input and the landslide result data corresponding to the unimodal landslide data as expected sample output, to construct a training sample including the sample input and the expected sample output; wherein the landslide result data includes: whether the landslide occurs and the type of landslide. The training samples are used to train the unimodal landslide recognition and prediction model to be trained to obtain a target unimodal landslide recognition and prediction model; wherein the unimodal landslide recognition and prediction model is at least used to characterize the corresponding relationship between the unimodal landslide data and the occurrence probability of different types of landslides.

[0012] In the second aspect, the embodiment of the present disclosure proposes a training method for a landslide identification and prediction base model, which is applied to a second device end, including: obtaining model parameters of multiple single-modal landslide identification and prediction models respectively sent from multiple first device ends; wherein the single-modal landslide identification and prediction model is trained based on the training method of the single-modal landslide identification and prediction model described in the first aspect above; based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, multimodal data fusion is performed on multiple single-modal landslide identification and prediction models corresponding to multiple model parameters to obtain a target landslide identification and prediction base model.

[0013] In a third aspect, an embodiment of the present disclosure proposes a localization method for a landslide identification and prediction model, which is applied to a third device end, and includes: initiating a model localization request for a target modality containing locally used landslide data to a second device end; obtaining a trimmed model or a lightweight model based on the target landslide identification and prediction base model of the second device end; wherein the trimmed model is a model obtained by trimming model parameters and model structures in the target landslide identification and prediction base model whose correlation with the target modality is lower than a preset degree, the target landslide identification and prediction base model is obtained based on the training method of the landslide identification and prediction base model described in the second aspect above, and the lightweight model is a model obtained by performing a preset lightweight processing on the target landslide identification and prediction base model when the trimming of model parameters and model structures in the target landslide identification and prediction base model cannot be completed based on the target modality; the trimmed model or the lightweight model is fine-tuned and trained using localized landslide training samples with the target modality to obtain a localized landslide identification and prediction model.

[0014] In a fourth aspect, an embodiment of the present disclosure proposes a landslide identification and prediction method, which is applied to a third device end, and includes: receiving actual landslide data of a target mode collected from a target geographic location; calling a preset localized landslide identification and prediction model to process the actual landslide data to obtain an output landslide identification result; wherein the localized landslide identification and prediction model is obtained based on a localization method of the landslide identification and prediction model described in the third aspect; and determining the occurrence probability of different types of landslides according to the landslide identification results.

[0015] In a fifth aspect, an embodiment of the present disclosure provides a first device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, the training method of the single-modal landslide identification and prediction model described in the first aspect can be implemented.

[0016] In a sixth aspect, an embodiment of the present disclosure provides a second device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, the training method of the landslide identification and prediction base model as described in the second aspect can be implemented.

[0017] In the seventh aspect, an embodiment of the present disclosure provides a third device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, it can implement the localization method of the landslide identification and prediction model or the landslide identification and prediction method as described in the third or fourth aspect.

[0018] In an eighth aspect, an embodiment of the present disclosure proposes a landslide identification and prediction system, comprising: a plurality of first device terminals, for acquiring single-modal landslide data collected by a preset single-modal landslide data collection device, wherein the single-modal landslide data comprises any of the following: satellite optical image data, airborne optical image data, satellite synthetic aperture radar image data, ground-based synthetic aperture radar data, airborne synthetic aperture radar data, airborne lidar data, on-site survey photos, monitoring data collected by surface sensors of the landslide body, and monitoring data collected by sensors inside the slope body; taking the single-modal landslide data as sample input, and taking the landslide result data corresponding to the single-modal landslide data as expected sample output, to construct a training sample including the sample input and the expected sample output, the landslide result data comprising: whether the landslide occurs and the type of landslide; using the training samples to train a single-modal landslide identification and prediction model to be trained, to obtain a target single-modal landslide identification and prediction model, the single-modal landslide identification and prediction model being at least used to characterize the single-modal The invention relates to a method for detecting a landslide in a target area and a prediction model of a landslide. The method comprises the following steps: determining a corresponding relationship between landslide data and the occurrence probability of different types of landslides; a second device end is used to receive model parameters of single-mode landslide identification and prediction models respectively sent by each first device end; based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, multimodal data fusion is performed on multiple single-mode landslide identification and prediction models corresponding to multiple model parameters to obtain a target landslide identification and prediction base model; a model localization request of a target mode containing landslide data used locally is received from a third device end; a third device end is used to receive a cropped model or a lightweight model returned by the second device end; the cropped model or the lightweight model is fine-tuned using a localized landslide training sample of the target mode to obtain a localized landslide identification and prediction model; actual landslide data of the target mode collected from a target geographical location is received; a preset localized landslide identification and prediction model is called to process the actual landslide data to obtain an output landslide identification result; and the occurrence probability of different types of landslides is determined according to the landslide identification result.

[0019] In a ninth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to implement the method described in the first aspect and / or the second aspect and / or the third aspect and / or the fourth aspect when executed. In a tenth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, wherein the computer program implements the steps of the method described in the first aspect and / or the second aspect and / or the third aspect and / or the fourth aspect when executed by a processor.

[0020] In the eleventh aspect, an embodiment of the present disclosure provides a computer program that enables a computer or processor to implement the steps of the method described in the first aspect and / or the second aspect and / or the third aspect and / or the fourth aspect when executed.

[0021] The present disclosure provides a landslide identification prediction model training and localization method, a landslide identification prediction method, and a landslide identification prediction system, and relates to computer systems based on specific models, disaster prediction, deep learning, machine learning, model training and fine-tuning, model localization, and other technical fields. The first device end uses the single-modal landslide data collected by each device end to train and obtain a single-modal landslide identification prediction model, and the second device end receives the model parameters of the single-modal landslide identification prediction models transmitted by multiple first device ends, and combines the polynomial neural network of the channel attention mechanism and the spatial attention mechanism to fuse the model parameters of multiple single-modal landslide identification prediction models to obtain a target landslide identification prediction base model, and the third device end sends the target mode of the landslide data used locally to the second device end, so that the second device end performs trimming or lightweight processing on the target landslide identification prediction base model according to the target mode, so that the third device end uses the localized landslide training sample to perform fine-tuning training on the received trimmed model or lightweight model, thereby finally obtaining the localized landslide identification model required to be used by each third device end. That is, by fusing multiple single-modal landslide identification and prediction models into a landslide identification and prediction base model, it is possible to summarize the characteristics and knowledge of landslide address disasters in various dimensions, so that the landslide identification and prediction base model has a more accurate landslide disaster identification capability. Even if it is adjusted in combination with the localization needs of different third-party devices, the model itself already has a more powerful landslide disaster identification capability before trimming or lightweighting. Therefore, the landslide identification and prediction model after localization processing has better recognition and prediction accuracy than the single-modal landslide identification and prediction model, and can ensure data security and obtain disaster prediction results when the original data is missing.

[0022] Other advantages of certain embodiments of the present disclosure will be explained in more detail with reference to the following description and accompanying drawings.

[0023] It should be understood that the above description is only an overview of the technical solutions of certain embodiments of the present disclosure, so that the technical means of certain embodiments of the present disclosure can be more clearly understood, and can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of certain embodiments of the present disclosure more obvious and easy to understand, the specific implementation methods of certain embodiments of the present disclosure are specifically described below by way of example. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The advantages and benefits described herein and other advantages and benefits will be apparent to those of ordinary skill in the art by reading the detailed description of the exemplary embodiments below. The accompanying drawings are only for the purpose of illustrating exemplary embodiments and are not to be considered as limiting the present disclosure. Also, the same reference numerals are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0025] Figure 1 A flowchart of a training method for a single-mode landslide identification and prediction model provided in an embodiment of the present disclosure;

[0026] Figure 2 A flowchart of a method for training a landslide identification and prediction base model provided in an embodiment of the present disclosure;

[0027] Figure 3 A schematic diagram of a process for obtaining a landslide identification and prediction base model based on multiple single-mode landslide identification and prediction models provided in an embodiment of the present disclosure;

[0028] Figure 4 A flowchart of a localization method of a landslide identification prediction model provided by an embodiment of the present disclosure;

[0029] Figure 5 A structural flow chart of a landslide identification system provided by an embodiment of the present disclosure;

[0030] Figure 6 A flow chart of a landslide identification and prediction method provided by an embodiment of the present disclosure;

[0031] Figure 7 A timing diagram of a landslide identification system provided by an embodiment of the present disclosure;

[0032] Figure 8 A structural diagram of a training device for a single-mode landslide identification and prediction model provided in an embodiment of the present disclosure;

[0033] Fig. 9 A structural diagram of a training device for a landslide identification and prediction base model provided in an embodiment of the present disclosure;

[0034] Fig.10 A structural diagram of a localization device for a landslide identification and prediction model provided in an embodiment of the present disclosure;

[0035] Fig.11 A structural diagram of a landslide identification and prediction device provided in an embodiment of the present disclosure;

[0036] Fig.12 A schematic diagram of the structure of an electronic device suitable for executing a training method for a single-modal landslide identification and prediction model and / or a training method for a landslide identification and prediction base model and / or a localization method for a landslide identification and prediction model and / or a landslide identification and prediction method provided in an embodiment of the present disclosure.

[0037] Description of Reference Numerals

[0038] 11, 12, 13 are first devices; 20 is a second device; 31, 32 are third devices; 100 is a flowchart of a training method for a single-mode landslide identification prediction model in an implementation column; 101, 102, 103 are training steps of the training method for a single-mode landslide identification prediction model; 200 is a flowchart of a training method for a landslide identification prediction base model in an implementation column; 201, 202 are training steps of a training method for a landslide identification prediction base model; 400 is a flowchart of a localization method for a landslide identification prediction model in an implementation column; 401, 402, 403 are training steps of a localized landslide identification prediction model; 600 is a flowchart of a landslide identification prediction method in an implementation column; 601, 602, 603 are prediction steps of a landslide identification prediction method; 800 is a training device for a single-mode landslide identification prediction model in an implementation column; 801 is a single-mode data acquisition unit; 802 is a training sample Construction unit; 803 is a single-mode recognition prediction model training unit; 900 is a training device for a multi-modal landslide recognition prediction base model in an implementation column; 901 is a model parameter acquisition unit; 902 is a multi-modal data fusion unit; 1000 is a localization device for a landslide recognition prediction model in an implementation column; 1001 is a localization request initiating unit; 1002 is a receiving unit; 1003 is a fine-tuning training unit; 1100 is a landslide recognition prediction device in an implementation column; 1101 is an actual landslide data receiving unit; 1102 is a model calling and processing unit; 1103 is an occurrence probability determination unit; 1200 is a structure of an electronic device in an implementation column; 1201 is a computing unit; 1202 is a ROM; 1203 is a RAM; 1204 is a bus; 1205 is an I / O interface; 1206 is an input unit; 1207 is an output unit; 1208 is a storage unit; 1209 is a communication unit.

[0039] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0040] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0041] In the description of the embodiments of the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present disclosure, and are not intended to exclude the possibility of the presence of one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof.

[0042] Unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. The “and / or” in this article is merely a way to describe the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0043] The terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

[0044] As used herein, a "model" is capable of processing inputs and providing corresponding outputs, and typically includes an input layer and an output layer and one or more hidden layers between the input layer and the output layer. Models used in deep learning applications typically include many hidden layers, thereby extending the depth of the network. The various layers of the neural network are connected in sequence, so that the output of the previous layer is provided as the input of the next layer, wherein the input layer receives the input of the neural network, and the output of the output layer serves as the final output of the neural network. Each layer includes one or more nodes (also referred to as processing nodes or neurons), each of which processes the input from the previous layer. In this article, the terms "neural network", "network", "neural network model" and "model" are used interchangeably.

[0045] In order to clearly explain the embodiments of the present disclosure, some technical problems discovered by the inventors during the scientific research process will be introduced first. The description here is not admitted to be prior art because it is included in this section. This section is intended to provide a background or context for the embodiments of the present disclosure stated in the claims.

[0046] For address disaster identification, since it may involve the use of high-precision maps and address data, the collected raw data involves national security, and different departments are reluctant to share their own data. In addition, due to the improvement of legal compliance, the free flow of raw data has become increasingly difficult. How to ensure data security and obtain disaster prediction results while missing raw data has become particularly urgent.

[0047] In the centralized artificial intelligence model, the training data is located on the server side, and the model training is also performed on the server side. There are problems of data privacy leakage and server performance bottlenecks, and it cannot meet the needs of landslide monitoring data security and continuous collection of landslide monitoring data.

[0048] The prior art predicts the probability of landslides in a region, such as predicting certain regions or grids and obtaining the probability of landslides in the regions or grids. However, the location coordinates (longitude and latitude) of the landslide cannot be accurately predicted, which increases the scope of detection by address detection professionals.

[0049] In order to obtain more features of the same object in the same place, it is often necessary to preprocess multi-source heterogeneous data of the same object to obtain aligned data or consistent data. The preprocessing process requires writing specific algorithms to process the data, which consumes computing resources and loses accuracy. Some people cannot process heterogeneous data from different objects and sources, making disaster prediction impossible or inaccurate.

[0050] In order to at least partially solve one or more of the above-mentioned problems and other potential problems, the example embodiments of the present disclosure propose a training method for a landslide identification and prediction base model based on continuous learning, a localization method and device for a landslide identification and prediction model, which are applied to a landslide identification and prediction system. The system adopts a distributed C / S architecture of a production-service-consumption model, including a first device end, a second device end, a third device end and / or a fourth device end, wherein the first device end and the second device end are communicatively connected, the second device end and the third device end are communicatively connected, and / or the third device end and the fourth device end are communicatively connected.

[0051] In this way, each first device does not share the original data, thus protecting the security of the original data. At the same time, by preprocessing the feature graph or compressing similar feature graphs (including compression and deduplication), the scale of the landslide identification prediction base model and the amount of input data are effectively reduced, the amount of calculation is reduced, and the calculation efficiency is improved. By fusing the feature graphs of multiple first devices through a data fusion module, various landslide laws can be comprehensively integrated to improve the accuracy of the landslide identification prediction base model in identifying landslide disasters.

[0052] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0053] Please refer to Figure 1 , Figure 1 A flowchart of a method for training a single-mode landslide identification prediction model provided by an embodiment of the present disclosure, wherein process 100 includes the following steps:

[0054] Step 101: acquiring single-mode landslide data collected by a preset single-mode landslide data collection device;

[0055] This step is intended to enable the executor of the training method of the unimodal landslide identification and prediction model (such as geological monitoring stations, monitoring points, various remote sensing aircraft, various weather balloons or various satellites scattered in different locations) to obtain the unimodal landslide data collected by a preset unimodal landslide data acquisition device, wherein the unimodal landslide data is the original data.

[0056] Specifically, the single-mode landslide data may include any of the following: satellite optical image data, airborne optical image data, satellite synthetic aperture radar image data, ground-based synthetic aperture radar data, airborne synthetic aperture radar data, airborne lidar data, on-site survey photos, monitoring data collected by sensors on the surface of the landslide body, and monitoring data collected by sensors inside the slope body. Specifically, the monitoring data may include data in various forms, such as vibration data, voice data, environmental sound data, etc. The corresponding collection devices may be: cameras, radars, laser scanners, vibration sensors, remote sensing data collectors, microphones, etc.

[0057] At present, landslide disasters still need to rely on public reports or rescue workers to search for disaster sites in a large area to accurately determine the location of the landslide disaster, and it is difficult to accurately locate the location coordinates of the landslide disaster. In order to accurately predict the location coordinates of the landslide on the basis of being able to identify whether a landslide disaster has occurred, what type of landslide, and predict the probability of occurrence after that, the single-mode landslide data can also include: geographic location information with a lower than preset accuracy, and the geographic location information includes: longitude information, latitude information and elevation information. Similarly, in order to clarify whether the occurrence of landslide disasters is related to and affected by environmental and weather information, environmental weather information, load-bearing information of the landslide body, vibration information, etc. at the same time can also be used as part of the landslide data. Among them, the environmental weather information includes but is not limited to the environmental temperature, light rain, moderate rain, heavy rain, wind force levels from 0 to 12 (windless-hurricane), 13 to 17, etc.; the environmental temperature includes the average temperature of the air, the temperature generated by sunlight heating, the temperature generated by geothermal heat, the temperature of the ground surface heated by wildfires, etc.; the load-bearing information of the landslide body represents the external pressure applied to the surface or the inside of the surface or the side of the surface at the possible landslide location, including but not limited to the pressure of river water on the side of the dam, the pressure of water on the side of the road, the pressure of the mountain on the side of the road, the pressure of vehicles and other transportation tools on the road surface, the pressure of the gravity of the snow body on the ground at the snow mountain, etc. The vibration information includes the vibration generated by the roller, the vibration of the earthquake, the disturbance generated by the airflow, the disturbance generated by the environmental sound, etc. Preferably, the landslide data includes any data related to the landslide and can be used for landslide prediction.

[0058] Step 102: using the unimodal landslide data as sample input and the landslide result data corresponding to the unimodal landslide data as expected sample output, to construct a training sample including the sample input and the expected sample output;

[0059] On the basis of step 101, this step aims to use the unimodal landslide data as sample input and the landslide result data corresponding to the unimodal landslide data as expected sample output, to construct a training sample including the sample input and the expected sample output, that is, the training sample includes multiple sample pairs, each sample pair consists of a sample input and a corresponding expected sample output.

[0060] The landslide result data includes: whether the landslide occurs (which can be simply expressed as a binary classification of 100% and 0%, or as a quantitative probability value between 0% and 100%), and the type of landslide. In one embodiment, whether the landslide occurs can also be expressed as the probability of the landslide. The single-mode landslide data also includes geographic location information with a precision not less than a preset precision, in the hope that the subsequent model can also learn the ability to predict high-precision geographic location information through low-precision geographic location information during the learning process.

[0061] An initial unimodal landslide identification prediction model is constructed. The specific implementation method can be that the first device end or the second device end uses transformer, support vector machine, etc. to automatically construct it. A location information embedding network is established in the initial unimodal landslide identification prediction model, and geographical location information such as longitude, latitude, and elevation are integrated into the above-mentioned initial unimodal landslide identification prediction model. The method is to add geographical location information as three separate channels (longitude, latitude, and elevation) to the input data, and input it through the location information embedding network. Optionally, the location information embedding network is connected to the multimodal feature fusion module of the unimodal landslide identification prediction model. The location information embedding network and the initial unimodal landslide identification prediction model together constitute a unimodal landslide identification prediction model to be trained. A neural network branch for predicting longitude, latitude, and elevation is established in the output module of the unimodal landslide identification prediction model to be trained, and the prediction error of the geographical location is included in the loss function of the model.

[0062] Step 103: using the training samples to train the single-mode landslide recognition and prediction model to be trained, to obtain a target single-mode landslide recognition and prediction model.

[0063] On the basis of step 102, this step aims to train the unimodal landslide recognition prediction model to be trained by the above-mentioned execution subject using the training samples to obtain the target unimodal landslide recognition prediction model. The unimodal landslide recognition prediction model is at least used to characterize the corresponding relationship between the unimodal landslide data and the occurrence probability of different types of landslides.

[0064] Specifically, the training process often involves determining a suitable loss function and selecting a suitable optimization algorithm under back propagation, such as a gradient descent optimization algorithm, to finally complete the training through multiple rounds of iterations. The training cutoff conditions may also include multiple conditions, such as the loss function is reduced to a preset value, the training difference is less than a preset difference, the number of training rounds exceeds a preset number of rounds, etc. The appropriate training cutoff conditions can be flexibly selected according to the actual situation.

[0065] An implementation method including but not limited to may be:

[0066] The training samples containing geographic location information are used to train the unimodal landslide identification prediction model to be trained based on the target loss function containing landslide identification prediction loss and geographic location prediction loss, and the target unimodal landslide identification prediction model is obtained; wherein the target unimodal landslide identification prediction model is used to characterize the correspondence between unimodal landslide data, the occurrence probability of different types of landslides and geographic location information. That is, the target loss function can be split into a landslide identification prediction sub-loss function and a geographic location prediction sub-loss function, and the contribution of each of the two sub-loss functions to the target loss function is determined respectively, so as to determine the corresponding weighted weights.

[0067] The training method of the unimodal landslide recognition prediction model using the first device end provided in the embodiment of the present disclosure first collects unimodal landslide data respectively, then constructs training samples based on the unimodal landslide data, and finally trains the target unimodal landslide recognition prediction model using the training samples. After the unimodal landslide recognition prediction model is trained, the first device end can send the respective target unimodal landslide recognition prediction model or the model parameters of the target unimodal landslide recognition prediction model to the second device end. Preferably, the model parameters of the target unimodal landslide recognition prediction model include but are not limited to a variety of model parameters such as unimodal landslide recognition prediction feature maps, which can transmit as little data as possible and save time while ensuring accuracy.

[0068] On the basis of the above embodiment, considering the difficulty of obtaining the expected sample output labeled with landslide result data on the one hand, and on the other hand, in order to improve the accuracy of the trained single-mode landslide recognition prediction model as much as possible, this embodiment also provides a solution combining supervised training and self-supervised training in the process of constructing training samples, including the following steps:

[0069] Firstly, the unimodal landslide data is used as sample input, the landslide result data corresponding to the unimodal landslide data is used as the expected sample output, and a basic sample pair consisting of the sample input and the expected sample output is constructed; that is, the basic sample pair corresponds to supervised training;

[0070] Then, a supervision signal is constructed by self-supervised learning for the single-mode landslide data as sample input, and the expected supervision output corresponding to the supervision signal is determined, and a self-supervised incremental sample pair constructed by the sample input and the expected supervision output is constructed;

[0071] Among them, the self-supervised learning method includes: randomly blocking a part of the area in the unimodal landslide data, and using the unblocked area to predict the blocked area, or using the unimodal landslide data at the previous moment to predict the unimodal landslide data at the current moment, or other similar methods. That is, the incremental sample pair corresponds to self-supervised training.

[0072] Finally, training samples are constructed based on basic sample pairs and self-supervised incremental sample pairs. That is, when the training samples are subsequently used to train the single-mode landslide recognition and prediction model, they contain both supervised samples and self-supervised samples, that is, the sample size of the training samples is expanded through self-supervision, thereby improving the recognition and prediction accuracy of the single-mode landslide recognition and prediction model obtained by the final training.

[0073] Please refer to Figure 2 , Figure 2A flowchart of a method for training a landslide identification prediction base model provided by an embodiment of the present disclosure, wherein process 200 includes the following steps:

[0074] Step 201: obtaining model parameters of single-mode landslide identification and prediction models or target single-mode landslide identification and prediction models respectively sent from multiple first device ends;

[0075] This step is intended to enable the executor of the training method of the landslide identification and prediction base model (e.g., a remote service end providing multimodal data fusion services) to obtain the model parameters of the single-modal landslide identification and prediction models or the target single-modal landslide identification and prediction models respectively sent from multiple first device ends. The model parameters may specifically exist in the form of feature maps. In another embodiment, multiple first device ends may also send their respective target single-modal landslide identification and prediction models to the second device end, and the second device end may obtain the feature map according to the target single-modal landslide identification and prediction model. The second device end may subsequently process the model parameters through a multimodal data alignment module, and the multimodal data alignment may align the differences between different single-modal data to improve the accuracy of the prediction.

[0076] Furthermore, if the received model parameters are in a compressed and / or encrypted state, the above-mentioned execution subject also needs to process them in a pre-agreed decompression and / or decryption manner to obtain model parameters that can be subsequently processed. The compression process is to reduce the amount of data in the transmission process as much as possible, and the encryption process is to prevent unknown or unauthorized third parties from obtaining the model parameters in plain text and to ensure that only known or authorized third parties obtain the model parameters in plain text.

[0077] Step 202: Based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, multimodal data fusion is performed on multiple single-modal landslide recognition and prediction models corresponding to multiple model parameters to obtain a target landslide recognition and prediction base model.

[0078] On the basis of step 201, this step is intended to be performed by the above-mentioned execution subject. Based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, a plurality of single-mode landslide identification prediction models corresponding to a plurality of model parameters are subjected to multimodal data fusion, and after iterative training, a target landslide identification prediction base model is finally obtained. In another embodiment, the second device end can be based on the fusion of the single-mode landslide identification prediction models respectively sent from the plurality of first device ends to obtain a target landslide identification prediction base model.

[0079] Furthermore, considering that the amount of unimodal landslide data collected by different first device ends is huge and sparse, and all follow the same mechanical law, mainly with different boundary conditions, it is entirely possible that different first device ends collect the same or similar modal landslide data. Therefore, in order to reduce the amount of computational data as much as possible and avoid duplication, before multimodal data fusion is performed on multiple unimodal landslide recognition and prediction models corresponding to multiple model parameters based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, the following preprocessing is also performed using a unimodal landslide recognition and prediction feature map preprocessing module:

[0080] The above-mentioned execution entity aligns the model parameters of each single-mode landslide identification and prediction model, and then merges or removes duplicates; wherein, merging or de-duplication refers to merging or removing the model parameters of any two single-mode landslide identification and prediction models that have a similarity exceeding a preset degree so that only one is retained in the end.

[0081] The model parameters obtained after deduplication or merging can be finally used for multi-modal feature fusion according to the following steps:

[0082] Firstly, the feature maps of each single-modal landslide recognition and prediction model after deduplication are spliced ​​to obtain the spliced ​​feature map; then, the channel attention weights of each part of the feature map in the spliced ​​feature map are determined based on the channel attention mechanism; then, the spatial attention weights of each part of the feature map in the spliced ​​feature map are determined based on the spatial attention mechanism; finally, the feature maps of each part of the spliced ​​feature map are combined with the corresponding channel attention weights and spatial attention weights for feature fusion.

[0083] It should be noted that the spatial self-attention mechanism and the channel self-attention mechanism are two variants of the self-attention mechanism in different application scenarios. Among them, the spatial self-attention mechanism is used to process data with spatial structure, such as images or sequence data with spatial arrangement (such as sentences in natural language). It focuses on the relationship between different positions (or spaces) in the data. For each position or pixel, the spatial self-attention mechanism calculates its correlation with all other positions in order to understand the global context information. This correlation calculation can help the model capture the dependencies and importance between different positions in the data.

[0084] The channel self-attention mechanism is used to process the channel dimension of data. In image data, the channel dimension usually represents different features or feature maps. In text data, the channel dimension may represent different semantic features or word embeddings. The channel self-attention mechanism focuses on the relationship and importance between different channels. It can help the model effectively integrate and utilize information in different channels when processing data to improve the richness of data representation and model performance.

[0085] In one embodiment of the present disclosure, these two self-attention mechanisms can be used alone or in combination. For example, when processing two-dimensional image data, the spatial self-attention mechanism can be applied simultaneously to consider the spatial relationship between pixels, and the channel self-attention mechanism can be applied to process the relationship between different feature channels. This combination can usually better capture the structural information and semantic information in the data, thereby improving the performance of the model.

[0086] Among them, the Polynomial Neural Network (PNN) is a special type of neural network, whose main feature is the ability to perform polynomial mapping and nonlinear transformation on input data. It is usually used to handle data modeling tasks with complex nonlinear relationships. The core of PNN is the polynomial activation function. PNN uses the polynomial activation function to perform nonlinear transformation on input features, so as to capture high-order feature interactions in the data. During the network training process, the parameters (weights and biases) in the polynomial activation function are optimized through the back-propagation algorithm, so that the network can adapt to the given data pattern and task.

[0087] The present disclosure uses a polynomial neural network to complete the fusion of multimodal features. The reason why a polynomial neural network is used is that the characteristics of its nonlinear activation function can better handle the complex nonlinear relationships and interaction patterns in multimodal data. This ability enables it to excel in capturing the complex interactions between different perceptual modalities; and it can model the interaction of high-order features of data, which is particularly important for fusing features from different modalities. For example, some unimodal landslide data have a large and sparse amount of data, while others have a dense amount of data. Different unimodal landslide data capture information on different aspects of landslide disasters and present a nonlinear relationship. When there is a complex nonlinear relationship between image features and text features, PNN can more accurately capture the high-order interactions between them, thereby improving the performance of the model; and the flexibility of PNN enables it to adaptively adjust the order and parameters of the polynomial to maximize the reflection of the characteristics and relationships of different modal data. This expressive ability enables the model to more comprehensively utilize multimodal information and improve the overall performance of the task.

[0088] In summary, polynomial neural networks, due to the characteristics of their nonlinear polynomial activation functions, are suitable for multimodal feature fusion tasks and can effectively process and utilize the complex relationships between different perceptual modalities, thereby improving the effect and performance of the model in multimodal data processing and applications.

[0089] Specifically, in one embodiment of the present disclosure, the polynomial neural network can be obtained in the following manner:

[0090] On the base network served by the residual network, the addition operation originally used by each residual module on the cross-layer connection is converted into a product operation, and the original activation function is removed.

[0091] The training method of the landslide recognition prediction base model applied to the second device end provided in this embodiment first receives the model parameters of the single-mode landslide recognition prediction models transmitted from the first device end respectively, and then fuses the model parameters of the multiple single-mode landslide recognition prediction models by combining the polynomial neural network of the channel attention mechanism and the spatial attention mechanism to obtain the target landslide recognition prediction base model. The landslide features and knowledge under different modes are fused by multimodal fusion to improve the recognition and prediction accuracy of the model.

[0092] On the basis of the above-mentioned embodiments, the second device end only receives the model parameters of the new unimodal landslide identification prediction model sent by each first device end respectively within the preset time length. Considering that each first device end is likely to update the previously trained unimodal landslide identification prediction model based on the newly collected unimodal landslide data in the subsequent time to include more effective information, it is necessary to update the landslide identification prediction base model of the second device end accordingly to realize the continuous learning of the landslide identification prediction base model, wherein the subsequent time refers to the time after the preset time length. In one embodiment, the subsequent time refers to the preset cycle time, the preset time period, the time when each first device end updates the unimodal landslide identification prediction model after the preset time length, the time when the first device end needs to send the model or model parameters at any time, and the time when the second device end actively retrieves the model or model parameters from the first device end. In one embodiment, during the continuous learning process, since the collected unimodal landslide data are generated or collected at different periods and the computing power of each first device end is different, the speeds at which each first device end feeds back the unimodal landslide identification prediction model are different. In order to avoid the second device end waiting for the feedback of each first device end for a long time and reduce the training speed of the model, it can be considered that the second device end only receives the updated model parameters of each unimodal landslide identification prediction model sent within a preset time period. For example, the second device end can start a timer after issuing a model update instruction to each first device segment, so that after the preset time indicated by the timer arrives, the receiving channel is closed and the subsequent updated model parameters are refused to be received, or the updated model parameters received after the preset time arrives can be directly discarded, thereby realizing asynchronous processing of data and ensuring the high efficiency of continuous learning and timeliness of updating of the landslide identification prediction base model.

[0093] For example, assuming that there are three updated model parameters A, B and C, where A and B arrive within the preset time period, and C arrives after a timeout, the second device will only consider A and B when updating the model. The preset time period in this embodiment can be determined comprehensively based on factors such as the complexity of the model to be trained, the average computing power of each working node, and the communication rate between nodes.

[0094] A specific implementation method may be:

[0095] Firstly, the execution subject receives the model parameters of the new unimodal landslide identification and prediction model sent by each first device end respectively only within the preset time; wherein, the receiving start time is the training completion time of the target landslide identification and prediction base model, the preset time is determined based on the device performance of each first device end and the data transmission performance between each first device end and the second device end, the model parameters of the new unimodal landslide identification and prediction model are the updated model parameters described in the above content, that is, the new unimodal landslide identification and prediction model is the updated unimodal landslide identification and prediction model, thereby realizing asynchronous processing of data and ensuring the high efficiency of continuous learning and timeliness of updating of the landslide identification and prediction base model.

[0096] Then, based on the model parameters of each new single-mode landslide identification and prediction model, the target landslide identification and prediction base model is updated.

[0097] Based on the above embodiment, the above execution subject can also be

[0098] no longer receiving the model parameters of the new single-mode landslide identification and prediction model sent by each first device end respectively beyond the preset time period; or

[0099] discarding the model parameters of the new single-mode landslide identification and prediction model sent by each first device end respectively beyond the preset time period; or

[0100] The model parameters of the new unimodal landslide identification and prediction model sent from the first device beyond the preset time length are temporarily stored, and the model parameters of the temporarily stored new unimodal landslide identification and prediction model are used together with the model parameters of other new unimodal landslide identification and prediction models received within the next preset time length to participate in the subsequent update of the landslide identification and prediction base model.

[0101] The preset duration mentioned in the above embodiment can be determined in the following way:

[0102] The actual time taken for each first device end to send the generated new single-mode landslide identification prediction model or the model parameters of the new single-mode landslide identification prediction model to the second device end is counted respectively under the device performance and data transmission performance of each first device end;

[0103] In each actual time consumption, a first target duration corresponding to a preset number of model parameters of new single-modal landslide identification prediction models can be received, and a second target duration corresponding to a preset proportion of the total number of single-modal landslide identification prediction models can be received;

[0104] A preset duration is determined based on the first target duration or the second target duration.

[0105] An implementation method including but not limited to may be: performing redundancy correction on the first target duration or the second target duration in combination with a preset incremental correction value, and determining the duration obtained after the redundant correction as the preset duration; wherein the duration corresponding to the incremental correction value is determined based on the excess duration of the cutoff moment when the moment when at least one first device end sends model parameters of a new single-mode landslide identification model exceeds the preset duration.

[0106] On the basis of the above embodiment, when the second device uses the updated model parameters to update the landslide identification prediction base model, it may also be necessary to determine whether the updated model parameters have improved accuracy compared to the original model parameters. One verification method may be:

[0107] Firstly, for each model parameter of the new single-mode landslide identification and prediction model, the accuracy difference between the prediction results output by the new single-mode landslide identification and prediction model and the corresponding single-mode landslide identification and prediction model on the preset verification samples is verified;

[0108] Then, when the accuracy difference is manifested as a higher prediction accuracy of the new single-mode landslide identification prediction model, the corresponding new single-mode landslide identification prediction model is determined as a valid update model; conversely, when the accuracy difference is manifested as a lower prediction accuracy of the new single-mode landslide identification prediction model, the new single-mode landslide identification prediction model is determined as an invalid update model;

[0109] Finally, the above-mentioned execution entity only uses each valid update model to update the corresponding single-mode landslide identification and prediction model part in the target landslide identification and prediction base model to ensure that the updated landslide identification and prediction base model has higher identification and prediction accuracy.

[0110] In order to deepen the understanding of the interaction process between the first device end and the second device end, the present disclosure also Figure 3 A flow chart showing the interactive process is shown, which is easier to understand and more intuitive:

[0111] The n first device ends respectively send the feature graphs of the trained unimodal landslide models to the unimodal landslide identification prediction feature graph preprocessing module of the second device end. After preprocessing, the unimodal landslide identification prediction feature graph is sent to the multimodal data fusion module. In the multimodal data fusion module, the feature graphs are first spliced, and then processed by polynomial neural network, channel attention and spatial attention mechanism to obtain the processing results. The processing results are then sent to the convolution layer for feature extraction, and then the landslide labels and predicted landslide coordinates are predicted, combined with the corresponding loss function to guide the training under the landslide real labels and landslide real coordinates, and finally the training of the landslide identification prediction base model with the fusion of multimodal features is completed. It should be noted that Figure 3The darker black dashed arrows represent forward propagation, and the lighter black dashed arrows represent reverse propagation. The landslide identification and prediction base model includes a single-mode landslide identification and prediction feature map preprocessing module, a multi-mode feature fusion module, and a prediction accuracy evaluation module.

[0112] At present, there is no disaster prediction solution that can re-output the prediction results in real time according to the user-defined environmental conditions, and it is difficult to take the user's judgment into account in the prediction model. For the trained landslide identification prediction base model, the actual landslide data is input into the landslide identification prediction base model, and the landslide identification prediction base model can output the predicted landslide result data corresponding to the actual landslide data.

[0113] In one embodiment of the present disclosure, a landslide prediction method based on a landslide identification prediction base model is provided, which is applied to a second device end to obtain model parameters of a plurality of target single-modal landslide identification prediction models respectively sent from a plurality of first device ends; based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, a plurality of target single-modal landslide identification prediction models corresponding to the plurality of model parameters are subjected to multimodal data fusion to obtain a target landslide identification prediction base model. The landslide identification prediction base model outputs landslide result data according to the actual landslide data in response to a user's custom operation, and generates correction data based on the actual landslide data; the landslide identification prediction base model outputs corrected landslide result data according to the corrected data; wherein the corrected landslide result data includes: whether a landslide occurs and the type of landslide. Preferably, the landslide result data also includes: geographic location information not less than the preset accuracy, and the geographic location information includes: longitude information, latitude information and elevation information. The second device end is provided with a database or shares a database with the third device end, and the database contains standard environmental condition data information. The standard environmental condition data information can be entered in advance by various professionals and presented to the user in the form of menu options, etc., for the user to customize the operation. The customized operation includes selecting the data information identified by the specified menu option as the customized environmental condition data information, and selecting the "plus" or "minus" instruction button. In response to the user's customized operation, the correction data is generated based on the actual landslide data, including: calling the standard environmental condition data information and the customized environmental condition data information in the database of the second device end; adding the standard environmental condition data information or the customized environmental condition data information to the actual landslide data, or subtracting it from the actual landslide data to obtain the correction data; wherein the standard environmental condition data information is identified by the menu option. Further, based on the trained landslide identification and prediction base model, the user can customize the operation based on the actual landslide data, and the training device of the landslide identification and prediction base model or the second device end will automatically correct the actual landslide data according to the customized environmental conditions to obtain the correction data; the landslide identification and prediction base model can re-output the predicted landslide result according to the correction data. The custom operation includes adding standard environmental condition data, subtracting standard environmental condition data or changing actual landslide data based on actual landslide data. The training samples of the landslide identification prediction base model obtained by training also include the environment, weather information, vibration information, pressure information, etc. at the corresponding time. When the landslide identification prediction base model is called in this embodiment, when the actual landslide data is input, the user can also customize the increase, decrease or change of environmental conditions through the second device end. For example, the landslide result data has been predicted. On this basis, the user can add new environmental conditions based on the actual landslide data, and the landslide identification prediction base model can re-output the predicted landslide result data.Users can remove certain environmental conditions that exist in actual landslide data, and the landslide identification and prediction base model can re-output predicted landslide result data.

[0114] In one embodiment, the acquisition of model parameters of target unimodal landslide identification prediction models respectively sent by multiple first device ends includes: acquiring characteristic graphs of target unimodal landslide identification prediction models respectively sent by multiple first device ends. The model parameters of new target unimodal landslide identification prediction models respectively sent by each first device end are received only within a preset time period; wherein the receiving start time is the training completion time of the target landslide identification prediction base model, and the preset time period is determined based on the device performance of each first device end and the data transmission performance between each first device end and the second device end; based on the model parameters of each new target unimodal landslide identification prediction model, the target landslide identification prediction base model is updated. The model parameters of new target unimodal landslide identification prediction models respectively sent by each first device end beyond the preset time period are no longer received; or the model parameters of new target unimodal landslide identification prediction models respectively sent by each first device end beyond the preset time period are discarded.

[0115] In one embodiment, there is no rainfall information in the actual landslide data, and the user has called the landslide identification prediction base model to process the actual landslide data and obtain the output landslide identification result; after obtaining the output landslide identification result, for example, there is a high slope at (m1, n1, h1), and the high slope has a 30% probability of a landslide disaster. The user customizes the addition of rainfall information based on the actual landslide data, but the user wants to see how the predicted landslide result will change if it rains heavily. For example, the user customizes the operation of adding heavy rain, and the training device or the second device end of the landslide identification prediction base model automatically adds the data information of heavy rain to the actual landslide data to obtain corrected data; the landslide identification prediction base model outputs the corresponding landslide identification result according to the corrected data. In another embodiment, the actual landslide data includes water pressure data information on the dam. The user has called the landslide identification and prediction base model to process the actual landslide data to obtain the landslide identification result of the dam corresponding to the actual landslide data. For example, in the dam at (m2, n2, h2), there is a 20% probability that a landslide disaster will occur in the dam; however, after obtaining the output landslide identification result, the user wants to see how large the water pressure on the dam becomes when the probability of a landslide at the dam exceeds 60%. The user customizes the data information of gradually increasing the water pressure on the dam based on the actual landslide data. The training device or the second device end of the landslide identification and prediction base model automatically adds the customized data information of gradually increasing the water pressure on the dam to the actual landslide data to obtain multiple corrected data; the landslide identification and prediction base model outputs the corresponding landslide identification result based on the multiple corrected data. In another embodiment, the user has called the landslide identification and prediction base model to process the actual landslide data, and obtained the landslide identification results of different positions of the highway corresponding to the actual landslide data. For example, at (m3, n3, h3), there is a highway with a slope on one side, and there is a 0% probability of a highway landslide. However, after obtaining the output landslide identification result, the user wants to see whether a landslide will occur if a 10-ton truck is added to the road surface of the highway. The user customizes the data information of adding 10 tons of load-bearing capacity based on the actual landslide data, such as simulating a 10-ton truck passing through the road surface of the highway. The training device or the second device end of the landslide identification and prediction base model automatically adds the data information of 10 tons of load-bearing capacity to the actual landslide data to obtain the corrected data. The landslide identification and prediction base model outputs the corresponding landslide identification result according to the corrected data.

[0116] In one embodiment, the second device end is provided with a database, which contains standard environmental condition data information, which can be entered in advance by various professionals and presented to the user in the form of menu options, etc., for the user to customize the operation, which includes adding the standard environmental condition or customized environmental condition data information to the actual landslide data, or subtracting it from the existing actual landslide data to obtain the correction data. Optionally, the database also contains customized environmental condition data information, which also includes the user customizing the addition, reduction or change of environmental condition data information at the second device end, and can add the customized environmental condition data information to the actual landslide data, or subtract it from the existing actual landslide data to obtain the correction data. The landslide identification prediction base model re-predicts the occurrence probability and position coordinates of different types of landslides under the increased external conditions according to the correction data. In order to make full use of the knowledge of the association between the environment, weather information, load-bearing and other data information of all single-mode landslide identification prediction models and the landslide results.

[0117] Based on the above embodiments that have described the training schemes of the single-mode landslide identification prediction model and the landslide identification prediction base model from the first device end and the second device end, this embodiment also describes the process of how to request the localized landslide identification prediction model that meets the localized use requirements from the second device end from the perspective of a third device end that is different from the first device end and the second device end. The third device end that is different from the first device end and the second device end ensures data security and avoids model unavailability due to device damage.

[0118] Please refer to Figure 4 , Figure 4 A flowchart of a localization method of a landslide identification prediction model provided by an embodiment of the present disclosure, wherein process 400 includes the following steps:

[0119] Step 401: initiating a model localization request of a target mode including landslide data used locally to a second device end;

[0120] This step is intended to initiate a model localization request of the target modality containing the landslide data used locally to the second device end by the execution subject of the localization method of the landslide identification prediction model (e.g., the service end that needs to perform landslide identification prediction or the individual user end that needs to use the landslide identification prediction service, i.e., the third device end). The target modality may be a field survey photo. Optionally, the target modality may be satellite optical image data, etc.

[0121] Step 402: receiving the cropped model or lightweight model returned by the second device;

[0122] Based on step 401, this step is intended to cause the above-mentioned execution subject to receive the cropped model or lightweight model returned by the second device end.

[0123] Among them, the trimmed model is the model obtained after the second device end trims the model parameters and model structures in the target landslide identification prediction base model whose correlation with the target mode is lower than the preset degree, and the lightweight model is the model obtained after the second device end performs a preset lightweight processing on the target landslide identification prediction base model when the model parameters and model structures in the target landslide identification prediction base model cannot be trimmed based on the target mode. In one embodiment, the second device end obtains the trimmed model or lightweight model according to the actual situation or demand of each third device end, and sends the trimmed model or lightweight model to each third device end respectively. In another embodiment, if the computing power of the third device end meets the conditional requirements, for example, the second device end determines that the computing power of the third device end is greater than or equal to the preset threshold, the second device end can send the target landslide identification prediction base model to the third device end, and the third device end directly uses the target landslide identification prediction base model or obtains the trimmed model or lightweight model based on the above-mentioned model trimming or lightweighting process, which will not be repeated here.

[0124] A specific cutting method can be:

[0125] Firstly, in the target landslide identification and prediction base model, the target model parameters and target model structure whose correlation with the target mode is lower than the preset degree are determined; then, the target model parameters and target model structure in the target landslide identification and prediction base model are trimmed to obtain the trimmed model.

[0126] The lightweight processing method can also be used to lightweight the target landslide identification and prediction base model when the target model parameters and target model structure whose correlation with the target mode is lower than the preset degree cannot be determined. Specifically, the lightweight processing includes model distillation processing, or other similar lightweight processing methods.

[0127] Step 403: fine-tune the pruned model or lightweight model using localized landslide training samples with the target mode to obtain a localized landslide recognition and prediction model.

[0128] On the basis of step 402, this step aims to have the above-mentioned execution subject fine-tune the trimmed model or the lightweight model using the localized landslide training samples with the target mode to obtain a localized landslide identification and prediction model.

[0129] The localization method of the landslide identification and prediction model applied to the third device end provided in this embodiment sends the target mode of the landslide data used locally to the second device end, so that the second device end crops or lightweights the target landslide identification and prediction base model according to the target mode, so that the third device end uses the localized landslide training samples to fine-tune the received cropped model or lightweight model, thereby finally obtaining the localized landslide identification model required to be used by each third device end. In another embodiment, a localization method for a landslide identification prediction model applied to a third device end is provided, by sending a model localization request of a target mode of the landslide data used locally to a second device end, the model localization request includes the computing power size and / or verification information of the third device end, the second device end determines that the computing power and / or verification information of the third device end meet the requirements, the second device end can send the target landslide identification prediction base model to the third device end, the third device end directly uses the target landslide identification prediction base model or obtains a trimmed model or a lightweight model based on the above-mentioned model trimming or lightweighting process, the third device end uses the localized landslide training sample to fine-tune the trimmed model or the lightweight model, and finally obtains the localized landslide identification model required for use by each third device end. Optionally, in order to make the prediction result more accurate, in the case where the second device end sends the target landslide identification prediction base model to the third device end for direct use, the third device end uses the localized landslide training sample to fine-tune the target landslide identification prediction base model, and finally obtains the localized landslide identification model required for use by each third device end. In response to a user's customized operation, correction data is generated based on actual landslide data; the localized landslide identification prediction model outputs corrected landslide result data according to the correction data; wherein the corrected landslide result data includes: whether a landslide occurs and the type of landslide. Actual landslide data of a target mode acquired from a target geographic location is received; a preset localized landslide identification prediction model is called to process the actual landslide data to obtain an output landslide identification result; wherein the localized landslide identification prediction model is obtained by a localized method of a landslide identification prediction model according to an embodiment of the present disclosure; the probability of occurrence of different types of landslides is determined according to the landslide identification result.

[0130] In one embodiment, the present disclosure provides a landslide identification and prediction system, which includes: a plurality of first device terminals, which are used to obtain unimodal landslide data collected by a preset unimodal landslide data collection device; using the unimodal landslide data as sample input, using landslide result data corresponding to the unimodal landslide data as expected sample output, and constructing a training sample including the sample input and the expected sample output, wherein the landslide result data includes: whether a landslide occurs and the type of landslide; using the training samples to train a unimodal landslide identification and prediction model to be trained, and obtaining a target unimodal landslide identification and prediction model, wherein the unimodal landslide identification and prediction model is at least used to characterize the corresponding relationship between the unimodal landslide data and the occurrence probability of different types of landslides;

[0131] The second device end is used to receive the model parameters of the single-modal landslide recognition and prediction models respectively sent by each of the first device ends; based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, multimodal data fusion is performed on multiple single-modal landslide recognition and prediction models corresponding to multiple model parameters to obtain a target multimodal landslide recognition and prediction model; a model localization request of a target mode containing the landslide data used locally is received from a third device end; in response to a user's custom operation, correction data is generated based on the actual landslide data; the landslide recognition and prediction base model outputs corrected landslide result data according to the correction data; wherein the landslide recognition and prediction base model can at least be used to predict the probability of occurrence of a landslide, and the corrected landslide result data includes: whether a landslide occurs, the type of landslide

[0132] The third device end is used to receive the cropped model or the lightweight model returned by the second device end; fine-tune the cropped model or the lightweight model using the localized landslide training sample of the target mode to obtain a localized landslide identification and prediction model; receive actual landslide data of the target mode collected from the target geographical location; call the preset localized landslide identification and prediction model to process the actual landslide data to obtain an output landslide identification result; determine the occurrence probability of different types of landslides according to the landslide identification result; generate correction data based on the actual landslide data in response to a user's custom operation; the localized landslide identification and prediction model outputs corrected landslide result data according to the corrected data; wherein the localized landslide identification and prediction model can at least be used to predict the occurrence probability of a landslide, and the corrected landslide result data includes: whether a landslide occurs and the type of landslide.

[0133] In one embodiment, the third device end may be a service end that needs to perform landslide identification and prediction or a personal user end that needs to use landslide identification and prediction services. Preferably, the third device end may be a service end that needs to perform landslide identification and prediction, such as a small server, a laptop computer, a tablet, a mobile phone, a smart TV and other electronic devices. The third device end generates different warning information according to the probability of occurrence of different types of landslides, and sends the warning information to at least one fourth device end. The fourth device end may be a personal user end, such as a TV, a mobile phone, a tablet, a personal computer, a small server and other electronic devices. The fourth device end displays the warning information to the user. Preferably, the third device end may be a personal user end that needs to use landslide identification and prediction services, such as a small server, a laptop computer, a tablet, a mobile phone, a smart TV and other electronic devices. The third device end displays the warning information to the user.

[0134] In order to enhance the understanding of the interaction process between the first device end, the second device end, and the third device end, this embodiment also uses Figure 5 A more intuitive multi-terminal interaction diagram is shown:

[0135] The data end corresponding to the first device end is trained based on different single-modal landslide data to obtain a single-modal landslide characterization model, and then the model parameters are sent to the server corresponding to the second device end. The server can obtain a landslide identification prediction base model by continuous learning through the multi-modal data alignment module, the single-modal landslide identification prediction feature map preprocessing module, the multi-modal feature fusion module and the prediction accuracy evaluation module. Among them, the multi-modal data alignment is to align the differences between different modal data. The fusion process of the multi-modal feature fusion module can be seen in Figure 3 ,The prediction accuracy evaluation module verifies the accuracy difference between the new and old model parameters by using the benchmark data provided by the benchmark dataset to confirm whether the new model parameters correspond to a more accurate landslide identification and prediction model.

[0136] The user end corresponding to the third device end generates a variety of localized landslide identification and prediction models based on its own needs through transfer learning, with the help of the landslide identification and prediction base model and local landslide detection data, so as to transfer the landslide knowledge in the landslide identification and prediction base model to the local area, so as to provide the user end with more accurate landslide identification and prediction.

[0137] The above embodiments illustrate how to train a landslide recognition prediction model from various aspects. In order to highlight the effect of the trained localized landslide recognition prediction model from the actual usage scenario as much as possible, the present disclosure also specifically provides a solution for using the trained localized landslide recognition prediction model to solve practical problems. Please refer to Figure 6 , Figure 6A flow chart of a landslide identification and prediction method provided by an embodiment of the present disclosure, wherein process 600 includes the following steps:

[0138] Step 601: receiving actual landslide data of a target mode acquired from a target geographical location;

[0139] This step is intended to receive actual landslide data of the target modality collected from the target geographic location by the executor of the landslide identification and prediction method (e.g., an individual user terminal that needs to use the landslide identification and prediction service). The target modality may be any one of on-site survey photos, monitoring data collected by sensors inside the slope, satellite optical image data, or airborne optical image data.

[0140] Step 602: calling a preset localized landslide identification prediction model to process actual landslide data to obtain an output landslide identification result;

[0141] Based on step 601, this step aims to call a preset localized landslide identification prediction model to process actual landslide data and obtain an output landslide identification result.

[0142] Step 603: Determine the occurrence probability and / or location coordinates of different types of landslides according to the landslide identification result.

[0143] On the basis of step 602, this step aims at determining the occurrence probability of different types of landslides according to the landslide identification result by the above-mentioned execution subject.

[0144] At present, there is no disaster prediction scheme that can re-output the prediction results according to the user-defined environmental conditions, and it is difficult to add the user's judgment to the prediction. Further, step 604: on the basis of step 603, the user can customize the environmental conditions based on the actual landslide data, and the localization device or the third device end will automatically correct the actual landslide data according to the customized environmental conditions to obtain the corrected data; the localized landslide identification prediction model re-predicts the landslide result according to the corrected data. The customized environmental conditions include adding new environmental condition data, reducing environmental condition data or changing the actual landslide data on the basis of the actual landslide data. When the training samples of the single-mode landslide identification prediction model are trained and also contain the environment and weather information at the corresponding time, when the localized landslide identification prediction model is called in this embodiment, in addition to inputting the actual landslide data of the target mode, the user end can also customize to increase, reduce or change the environmental conditions. For example, a landslide result has been predicted. On this basis, the user can add the changed environmental conditions, and the localized landslide identification prediction model can re-output the prediction result.

[0145] In one embodiment, there is no rainfall information in the actual landslide data, and the user has called the preset localized landslide identification prediction model to process the actual landslide data to obtain the output landslide identification result; after obtaining the output landslide identification result, for example, there is a mountain at (x1, y1, z1), and there is a 40% probability of a landslide disaster in the mountain. The user customizes the rainfall information based on the actual landslide data, but the user wants to see how the landslide identification result will change if it rains lightly, moderately, or heavily. For example, the user customizes the setting of adding light rain, moderately, or heavily, and the localization device or the third device automatically adds the data information of light rain, moderately, or heavily to the actual landslide data in sequence to obtain different correction data; the localized landslide identification prediction model outputs the corresponding landslide identification results in sequence according to the different correction data. In another embodiment, the actual landslide data includes data information of a level 10 gale. The user has called a preset localized landslide identification and prediction model to process the actual landslide data to obtain a landslide identification result corresponding to the actual landslide data. For example, there is a snow slope at (x2, y2, z2), and there is a 90% probability of an avalanche landslide on the snow slope. However, after obtaining the output landslide identification result, the user wants to see whether a landslide will occur on the snow slope if there is a level 2 light wind. The user customizes the data information of reducing the wind volume based on the actual landslide data. For example, the user sets the wind level to a level 2 light wind. The localization device or the third device automatically corrects the data information of the level 10 gale in the actual landslide data to the data information of the level 2 light wind, and obtains the corrected data. The localized landslide identification and prediction model outputs the corresponding landslide identification result according to the corrected data. In another embodiment, the user has called a preset localized landslide identification and prediction model to process actual landslide data, and obtained landslide identification results at different positions on the highway corresponding to the actual landslide data. For example, at (x3, y3, z3), there is a highway with a slope on one side, and there is a 5% probability of a highway landslide. However, after obtaining the output landslide identification result, the user wants to see whether a landslide will occur if a 1-ton truck is added to the road surface of the highway. The user customizes the data information of adding 1 ton of load-bearing capacity based on the actual landslide data, such as simulating a 1-ton truck passing through the road surface of the highway. The localization device or the third equipment end automatically adds 1 ton of load-bearing capacity data information to the actual landslide data to obtain the corrected data. The localized landslide identification and prediction model outputs the corresponding landslide identification result according to the corrected data.Among them, the third device end is provided with a database or shares a database with the second device end, and the database contains standard environmental condition data information, and the standard environmental condition data information can be entered in advance by various professionals, and presented to the user in the form of menu options, etc., for the user to customize the operation, and the custom operation includes selecting the data information identified by the specified menu option as the custom environmental condition data information, and selecting the plus or minus instruction button, and the second device end adds the custom environmental condition data information to the actual landslide data, or subtracts it from the existing actual landslide data to obtain the correction data. In one embodiment, the standard environmental condition data information is a certain type of single-mode landslide data. Optionally, the database also contains customized environmental condition data information, and the custom operation includes the user customizing the increase, reduction or change of environmental condition data information on the third device end, and can add the customized environmental condition data information to the actual landslide data, or subtract it from the existing actual landslide data to obtain the correction data. In one embodiment, the third device end generates different early warning information according to the probability of occurrence and / or position coordinates of different types of landslides, and sends the early warning information to at least one fourth device end.

[0146] In one embodiment, the second device end and the third device end have their own databases or share the same database, and the database contains standard environmental condition data information, which can be entered in advance by various professionals and presented to the user in the form of menu options, etc., for the user to customize the operation, and the customized operation includes adding the customized environmental condition data information to the actual landslide data, or subtracting it from the existing actual landslide data to obtain the correction data. Optionally, the database also contains customized environmental condition data information, and the customized operation also includes the user end (i.e., the third device end) customizing the increase, reduction or change of environmental condition data information, and can add the customized environmental condition data information to the actual landslide data, or subtract it from the existing actual landslide data to obtain the correction data. The localized landslide identification prediction model re-predicts the occurrence probability and location coordinates of different types of landslides under the increased external conditions according to the correction data. In order to make full use of the knowledge of the association between the environment, weather information, load-bearing and other data information of the single-mode landslide identification prediction model and the landslide identification prediction base model and the landslide identification prediction base model on the landslide results.

[0147] The landslide identification and prediction method applied to the third device end provided in this embodiment integrates multiple single-modal landslide identification and prediction models into a landslide identification and prediction base model, thereby summarizing the characteristics and knowledge of landslide address disasters in various dimensions, so that the landslide identification and prediction base model has a more accurate landslide disaster identification capability. Even if it is adjusted in combination with the localization needs of different third device ends, the model itself has a more powerful landslide disaster identification capability before trimming or lightweighting, and can adapt to the computing power requirements of different third device ends, so that landslide disaster identification can run smoothly on multiple devices. Therefore, the landslide identification and prediction model after localization processing also has better recognition and prediction accuracy than the single-modal landslide identification and prediction model.

[0148] In order to more fully understand the complete solution, this embodiment also passes Figure 7 A landslide identification and prediction system consisting of a plurality of first device ends, a second device end and a plurality of third device ends is shown:

[0149] First devices 11, 12, 13 (taking the first device 12 as an example):

[0150] S701: Acquire single-mode landslide data collected by a preset single-mode landslide data collection device;

[0151] S702: taking the unimodal landslide data as sample input and the landslide result data corresponding to the unimodal landslide data as expected sample output, and constructing a training sample including the sample input and the expected sample output, wherein the landslide result data includes: whether the landslide occurs and the landslide type;

[0152] S703: using the training samples to train the unimodal landslide recognition prediction model to be trained, to obtain a target unimodal landslide recognition prediction model, where the unimodal landslide recognition prediction model is at least used to characterize the corresponding relationship between the unimodal landslide data and the occurrence probability of different types of landslides;

[0153] Second device 20:

[0154] S704: receiving model parameters of the single-mode landslide identification and prediction model respectively sent by each first device end;

[0155] S705: Based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, multimodal data fusion is performed on multiple single-modal landslide recognition and prediction models corresponding to multiple model parameters to obtain a target landslide recognition and prediction base model;

[0156] S706: receiving a model localization request of a target mode including landslide data used locally initiated by a third device;

[0157] The third devices 31 and 32 (taking the third device 32 as an example):

[0158] S707: Receive the cropped model or lightweight model returned by the second device end;

[0159] S708: fine-tuning the trimmed model or the lightweight model using a localized landslide training sample with the target mode to obtain a localized landslide recognition and prediction model;

[0160] S709: receiving actual landslide data of a target mode acquired from a target geographic location; calling a preset localized landslide identification prediction model to process the actual landslide data to obtain an output landslide identification result; and determining the occurrence probability of different types of landslides according to the landslide identification result.

[0161] S710: receiving actual landslide data of a target mode acquired from a target geographic location; calling a preset localized landslide identification prediction model to process the actual landslide data to obtain an output landslide identification result; and determining the occurrence probability of different types of landslides according to the landslide identification result.

[0162] Further references Figure 8-Figure 11 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a training device for a single-mode landslide identification and prediction model, an embodiment of a training device for a landslide identification and prediction base model, an embodiment of a localization device for a landslide identification and prediction model, and an embodiment of a landslide identification and prediction device. Figure 1 Corresponding to the embodiment of the single-mode landslide identification and prediction method shown in FIG, the embodiment of the training device for the landslide identification and prediction base model is as follows Figure 2 Corresponding to the training method embodiment of the landslide identification prediction base model, the localization device embodiment of the landslide identification prediction model is Figure 4 Corresponding to the embodiment of the localization method of the landslide identification prediction model shown in FIG. Figure 6 The above device can be applied to various electronic devices.

[0163] like Figure 8As shown, the training device 800 of the unimodal landslide identification prediction model of this embodiment may include: a unimodal data acquisition unit 801, a training sample construction unit 802, and a unimodal identification prediction model training unit 803. The unimodal data acquisition unit 801 is configured to acquire unimodal landslide data acquired by a preset unimodal landslide data acquisition device, and the training sample construction unit 802 is configured to use the unimodal landslide data as sample input and the landslide result data corresponding to the unimodal landslide data as expected sample output, and construct a training sample including the sample input and the expected sample output; wherein the landslide result data includes: whether the landslide occurs and the type of landslide. The unimodal identification prediction model training unit 803 is configured to train the unimodal landslide identification prediction model to be trained using the training samples to obtain a target unimodal landslide identification prediction model; wherein the unimodal landslide identification prediction model is at least used to characterize the corresponding relationship between the unimodal landslide data and the occurrence probability of different types of landslides.

[0164] In this embodiment, in the training device 800 for the single-mode landslide identification prediction model, the specific processing of the single-mode data acquisition unit 801, the training sample construction unit 802, and the single-mode identification prediction model training unit 803 and the technical effects thereof can be referred to respectively. Figure 1 The relevant descriptions of steps 101 - 103 in the corresponding embodiment are not repeated here.

[0165] In some optional implementations of this embodiment, the single-mode landslide data includes any of the following:

[0166] Satellite optical image data, airborne optical image data, satellite synthetic aperture radar image data, ground-based synthetic aperture radar data, airborne synthetic aperture radar data, airborne lidar data, on-site survey photos, monitoring data collected by sensors on the surface of the landslide, monitoring data collected by sensors inside the slope, voice data, and environmental sound data.

[0167] In some optional implementations of this embodiment, the single-modal landslide data also includes: geographic location information with a precision lower than a preset value, and correspondingly, the landslide result data also includes: geographic location information with a precision not lower than a preset value, and the geographic location information includes: longitude information, latitude information and elevation information.

[0168] In some optional implementations of this embodiment, the single-modality recognition prediction model training unit 803 may be further configured as follows:

[0169] The training samples containing geographic location information are used to train the unimodal landslide identification and prediction model to be trained based on the target loss function containing landslide identification prediction loss and geographic location prediction loss, so as to obtain the target unimodal landslide identification and prediction model; wherein, the target unimodal landslide identification and prediction model is used to characterize the correspondence between unimodal landslide data, the occurrence probability of different types of landslides and geographic location information.

[0170] In some optional implementations of this embodiment, the training sample construction unit 802 may be further configured to:

[0171] Taking the unimodal landslide data as sample input, taking the landslide result data corresponding to the unimodal landslide data as expected sample output, and constructing a basic sample pair consisting of the sample input and the expected sample output;

[0172] A supervision signal is constructed by self-supervised learning for the single-mode landslide data as sample input, and an expected supervision output corresponding to the supervision signal is determined, and a self-supervised incremental sample pair constructed by the sample input and the expected supervision output is constructed;

[0173] Training samples are constructed based on basic sample pairs and self-supervised incremental sample pairs.

[0174] In some optional implementations of this embodiment, the self-supervised learning method includes: randomly blocking a portion of the area in the unimodal landslide data, and using the unblocked area to predict the blocked area, or using the unimodal landslide data at the previous moment to predict the unimodal landslide data at the current moment.

[0175] like Fig. 9 As shown, the training device 900 of the landslide identification prediction base model of this embodiment may include: a model parameter acquisition unit 901 and a multimodal data fusion unit 902. The model parameter acquisition unit 901 is configured to acquire model parameters of single-modal landslide identification prediction models respectively sent from multiple first device ends; wherein the single-modal landslide identification prediction model is obtained by the single-modal landslide identification prediction device 800; the multimodal data fusion unit 902 is configured to perform multimodal data fusion on multiple single-modal landslide identification prediction models corresponding to multiple model parameters based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, so as to obtain a target landslide identification prediction base model.

[0176] In this embodiment, in the training device 900 for the landslide identification prediction base model, the specific processing of the model parameter acquisition unit 901 and the multimodal data fusion unit 902 and the technical effects thereof can be respectively referred to in Figure 2 The relevant descriptions in the method embodiment shown are not repeated here.

[0177] In some optional implementations of this embodiment, the model parameter acquisition unit 901 is further configured to:

[0178] The characteristic graphs of the single-mode landslide identification and prediction models respectively sent from the plurality of first device ends are obtained.

[0179] In some optional implementations of this embodiment, the training device 900 for the landslide identification and prediction base model may further include:

[0180] The new model parameter receiving unit is configured to receive the model parameters of the new single-mode landslide identification and prediction model sent by each first device end only within a preset time period; wherein the receiving start time is the training completion time of the target landslide identification and prediction base model, and the preset time period is determined based on the device performance of each first device end and the data transmission performance between each first device end and the second device end;

[0181] The model parameter updating unit is configured to update the target landslide identification and prediction base model based on the model parameters of each new single-mode landslide identification and prediction model.

[0182] In some optional implementations of this embodiment, the model parameter acquisition unit 901 is further configured to:

[0183] The non-receiving processing unit is configured to no longer receive the model parameters of the new single-mode landslide identification and prediction model sent by each first device end respectively beyond the preset time period; or

[0184] The discard processing unit is configured to discard the model parameters of the new single-mode landslide identification and prediction model sent by each first device end respectively beyond a preset time period.

[0185] In some optional implementations of this embodiment, the model parameter acquisition unit 901 is further configured to:

[0186] The temporary storage processing unit is configured to temporarily store the model parameters of the new single-mode landslide identification and prediction model sent from the first device end beyond the preset time length, and use the temporarily stored model parameters of the new single-mode landslide identification and prediction model together with the model parameters of other new single-mode landslide identification and prediction models received within the next preset time length to participate in the subsequent update of the landslide identification and prediction base model.

[0187] In some optional implementations of this embodiment, the preset duration may be determined in the following manner:

[0188] The actual time taken for each first device end to send the generated new single-mode landslide identification prediction model or the model parameters of the new single-mode landslide identification prediction model to the second device end is counted respectively under the device performance and data transmission performance of each first device end;

[0189] In each actual time consumption, a first target duration corresponding to a preset number of model parameters of new single-modal landslide identification prediction models can be received, and a second target duration corresponding to a preset proportion of the total number of single-modal landslide identification prediction models can be received;

[0190] A preset duration is determined based on the first target duration or the second target duration.

[0191] In some optional implementations of this embodiment, determining the preset duration based on the first target duration or the second target duration includes:

[0192] The first target duration or the second target duration is subjected to redundancy correction in combination with a preset incremental correction value, and the duration obtained after the redundant correction is determined as the preset duration; wherein the duration corresponding to the incremental correction value is determined based on the excess duration of the cutoff moment when the moment when the model parameters of a new single-mode landslide identification model are sent from at least one first device end exceeds the preset duration.

[0193] In some optional implementations of this embodiment, the multimodal data fusion unit 902 may be further configured to:

[0194] For each model parameter of the new single-mode landslide identification and prediction model, the accuracy difference between the prediction results output by the new single-mode landslide identification and prediction model and the corresponding single-mode landslide identification and prediction model for the preset verification samples is verified;

[0195] In response to the accuracy difference being manifested as the new single-mode landslide identification and prediction model having a higher prediction accuracy, the corresponding new single-mode landslide identification and prediction model is determined as a valid updated model;

[0196] Only the valid updated models are used to update the corresponding single-mode landslide identification and prediction model part in the target landslide identification and prediction base model.

[0197] In some optional implementations of this embodiment, the training device 900 for the landslide identification and prediction base model may further include:

[0198] In response to the received model parameters being in a compressed and / or encrypted state, the model parameters are processed in a pre-agreed decompression and / or decryption manner.

[0199] In some optional implementations of this embodiment, before performing multimodal data fusion on multiple single-modal landslide identification and prediction models corresponding to multiple model parameters based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, the method further includes:

[0200] The merging and deduplication processing unit is configured to merge or deduplication the model parameters of each single-mode landslide identification and prediction model; wherein, the merging or deduplication processing refers to merging or removing the model parameters of any two single-mode landslide identification and prediction models that have a similarity exceeding a preset degree so that only one is retained in the end.

[0201] In some optional implementations of this embodiment, the multimodal data fusion unit 902 may be further configured to:

[0202] Splicing the deduplicated feature maps of each single-mode landslide identification and prediction model to obtain a spliced ​​feature map;

[0203] Determine the channel attention weights of each part of the feature map in the concatenated feature map based on the channel attention mechanism;

[0204] Determine the spatial attention weights of each part of the feature map after concatenation based on the spatial attention mechanism;

[0205] The feature maps of each part of the spliced ​​feature map are combined with the corresponding channel attention weights and spatial attention weights for feature fusion.

[0206] In some optional implementations of this embodiment, the polynomial neural network is obtained in the following manner:

[0207] On the base network served by the residual network, the addition operation originally used by each residual module on the cross-layer connection is converted into a product operation, and the original activation function is removed.

[0208] like Fig.10 As shown, the localization device 1000 of the landslide identification and prediction model of this embodiment may include: a localization request initiating unit 1001, a receiving unit 1002, and a fine-tuning training unit 1003. Among them, the localization request initiating unit 1001 is configured to initiate a model localization request of the target mode containing the landslide data used locally to the second device end; the receiving unit 1002 is configured to receive the trimmed model or lightweight model returned by the second device end; wherein the trimmed model is a model obtained by the second device end after trimming the model parameters and model structures in the target landslide identification and prediction base model whose correlation with the target mode is lower than a preset degree, the target landslide identification and prediction base model is obtained by the training device 900 of the landslide identification and prediction base model, and the lightweight model is a model obtained by the second device end after the target landslide identification and prediction base model is subjected to a preset lightweight processing when the trimming of the model parameters and model structures in the target landslide identification and prediction base model cannot be completed based on the target mode; the fine-tuning training unit 1003 is configured to fine-tune the trimmed model or lightweight model using the localized landslide training samples with the target mode to obtain a localized landslide identification and prediction model.

[0209] In this embodiment, in the localization device 1000 of the landslide identification prediction model, the specific processing of the localization request initiating unit 1001, the receiving unit 1002, and the fine-tuning training unit 1003 and the technical effects thereof can be respectively referred to in Figure 4 The relevant descriptions in the method embodiment shown are not repeated here.

[0210] like Fig.11 As shown, the landslide identification and prediction device 1100 of this embodiment may include: an actual landslide data receiving unit 1101, a model calling and processing unit 1102, and an occurrence probability determination unit 1103. The actual landslide data receiving unit 1101 is configured to receive actual landslide data of a target mode acquired from a target geographical location; the model calling and processing unit 1102 is configured to call a preset localized landslide identification and prediction model to process the actual landslide data to obtain an output landslide identification result; the localized landslide identification and prediction model is obtained by the localization device 1000 of the landslide identification and prediction model; the occurrence probability determination unit 1103 is configured to determine the occurrence probability of different types of landslides according to the landslide identification result.

[0211] In this embodiment, the landslide identification and prediction device 1100 includes: an actual landslide data receiving unit 1101, a model calling and processing unit 1102, and an occurrence probability determining unit 1103. The specific processing and the technical effects thereof can be respectively referred to in Figure 6 The relevant descriptions in the method embodiment shown are not repeated here.

[0212] The above embodiments exist as apparatus embodiments corresponding to the above method embodiments respectively. The first device end uses the single-modal landslide data collected by each of them to train a single-modal landslide recognition prediction model. The second device end receives the model parameters of the single-modal landslide recognition prediction models transmitted by multiple first device ends respectively, and fuses the model parameters of multiple single-modal landslide recognition prediction models with a polynomial neural network combining a channel attention mechanism and a spatial attention mechanism to obtain a target landslide recognition prediction base model. The third device end sends the target mode of the landslide data used locally to the second device end, so that the second device end performs cropping or lightweight processing on the target landslide recognition prediction base model according to the target mode, so that the third device end uses the localized landslide training samples to fine-tune the received cropped model or lightweight model, thereby finally obtaining the localized landslide recognition model required to be used by each third device end. That is, by fusing multiple single-modal landslide identification and prediction models into a landslide identification and prediction base model, the characteristics and knowledge of landslide address disasters in various dimensions can be summarized, so that the landslide identification and prediction base model has a more accurate landslide hazard identification capability. Even if it is adjusted in combination with the localization needs of different third-party devices, the model itself already has a more powerful landslide hazard identification capability before trimming or lightweighting. Therefore, the landslide identification and prediction model after localization processing also has better recognition and prediction accuracy than the single-modal landslide identification and prediction model.

[0213] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, it can implement the training method of the single-modal landslide identification and prediction model (for example, corresponding to the first device) and / or the training method of the landslide identification and prediction base model (for example, corresponding to the second device) and / or the localization method of the landslide identification and prediction model (for example, corresponding to the third device) and / or the modal landslide identification and prediction method (for example, corresponding to the third device) described in any of the above embodiments.

[0214] According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the training method of a single-modal landslide identification and prediction model and / or the training method of a landslide identification and prediction base model and / or the localization method of a landslide identification and prediction model and / or the modal landslide identification and prediction method described in any of the above embodiments when executed.

[0215] The embodiments of the present disclosure provide a computer program product, which, when executed by a processor, can implement the steps of the single-modal landslide identification and prediction model training method and / or the landslide identification and prediction base model training method and / or the landslide identification and prediction model localization method and / or the modal landslide identification and prediction method described in any of the above embodiments.

[0216] Fig.12 A schematic block diagram of an example electronic device 1200 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices, or the electronic device can be a PC, a portable computer, a mobile electronic device, a centralized server, a distributed server, an image acquisition terminal, a sensor terminal, a satellite, a mobile phone, a vehicle-mounted terminal, a large screen, and other terminal devices. The first device, the second device, the third device, and / or the fourth device are preferably any one or more combinations of the electronic device 1200. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0217] like Fig.12 As shown, the device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the device 1200 can also be stored. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0218] A number of components in the device 1200 are connected to the I / O interface 1205, including: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0219] The computing unit 1201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1201 performs the various methods and processes described above, such as the training method of a single-mode landslide identification prediction model and / or the training method of a landslide identification prediction base model and / or the localization method of a landslide identification prediction model and / or the modal landslide identification prediction method. For example, in some embodiments, the training method of a single-mode landslide identification prediction model and / or the training method of a landslide identification prediction base model and / or the localization method of a landslide identification prediction model and / or the modal landslide identification prediction method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the training method of the single-modal landslide identification prediction model and / or the training method of the landslide identification prediction base model and / or the localization method of the landslide identification prediction model and / or the modal landslide identification prediction method described above may be executed. Alternatively, in other embodiments, the computing unit 1201 may be configured to execute the training method of the single-modal landslide identification prediction model and / or the training method of the landslide identification prediction base model and / or the localization method of the landslide identification prediction model and / or the modal landslide identification prediction method in any other appropriate manner (e.g., by means of firmware).

[0220] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0221] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0222] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0223] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0224] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0225] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private server (VPS) services.

[0226] According to the technical solution of the embodiment of the present disclosure, the first device end uses the single-modal landslide data collected by each of them to train a single-modal landslide recognition prediction model, the second device end receives the model parameters of the single-modal landslide recognition prediction models transmitted by multiple first device ends, and fuses the model parameters of multiple single-modal landslide recognition prediction models with a polynomial neural network combining a channel attention mechanism and a spatial attention mechanism to obtain a target landslide recognition prediction base model, and the third device end sends the target mode of the landslide data used locally to the second device end, so that the second device end performs cropping or lightweight processing on the target landslide recognition prediction base model according to the target mode, so that the third device end uses the localized landslide training samples to perform fine-tuning training on the received cropped model or lightweight model, thereby finally obtaining the localized landslide recognition model required to be used by each third device end. That is, by fusing multiple single-modal landslide identification and prediction models into a landslide identification and prediction base model, the characteristics and knowledge of landslide address disasters in various dimensions can be summarized, so that the landslide identification and prediction base model has a more accurate landslide hazard identification capability. Even if it is adjusted in combination with the localization needs of different third-party devices, the model itself already has a more powerful landslide hazard identification capability before trimming or lightweighting. Therefore, the landslide identification and prediction model after localization processing also has better recognition and prediction accuracy than the single-modal landslide identification and prediction model.

[0227] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0228] It should be noted that the device in the embodiment of the present disclosure can implement each process of the embodiment of the aforementioned method and achieve the same effects and functions, which will not be repeated here.

[0229] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, apparatuses (devices or systems), and computer-readable storage media according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device for specifying functions in a box or multiple boxes. The various embodiments can be combined with each other without causing contradictions. The various implementation methods in the present disclosure are described in a progressive manner, and the same and similar parts between the various implementation methods can be referred to each other, and each implementation method focuses on the differences from other implementation methods. In particular, for the device, equipment and computer-readable storage medium implementation methods, since they are basically similar to the method implementation methods, their descriptions are simplified, and the relevant parts can be referred to the partial description of the method implementation methods.

[0230] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, models of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. In addition, although the operations of the disclosed method are described in a particular order in the accompanying drawings, this does not require or imply that these operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0231] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A training method for a single-mode landslide identification and prediction model, characterized in that: Applied to the first device, including: Obtaining unimodal landslide data collected by a preset unimodal landslide data collection device; wherein the unimodal landslide data includes any one of the following: satellite optical image data of the location where the landslide occurs, airborne optical image data, satellite synthetic aperture radar image data, ground-based synthetic aperture radar data, airborne synthetic aperture radar data, airborne lidar data, on-site survey photos, monitoring data collected by sensors on the surface of the landslide body, and monitoring data collected by sensors inside the slope body; The unimodal landslide data is used as sample input, and the landslide result data corresponding to the unimodal landslide data is used as expected sample output, to construct a training sample including the sample input and the expected sample output; wherein the landslide result data includes: whether the landslide occurs and the type of the landslide; The training samples are used to train the unimodal landslide identification and prediction model to be trained, so as to obtain a target unimodal landslide identification and prediction model; wherein the unimodal landslide identification and prediction model is at least used to characterize the corresponding relationship between the unimodal landslide data and the occurrence probability of different types of landslides; There are multiple first device ends, and the first device ends send model parameters of their respective target single-mode landslide identification and prediction models to the second device ends.

2. The method according to claim 1, characterized in that The single-mode landslide data also includes: geographic location information with a precision lower than a preset value. Correspondingly, the landslide result data also includes: geographic location information with a precision not lower than the preset value. The geographic location information includes: longitude information, latitude information and elevation information.

3. The method according to claim 2, characterized in that The method of using the training samples to train the single-mode landslide recognition prediction model to be trained to obtain a target single-mode landslide recognition prediction model includes: The training samples containing geographic location information are used to train the unimodal landslide identification and prediction model to be trained based on the target loss function containing the landslide identification prediction loss and the geographic location prediction loss to obtain the target unimodal landslide identification and prediction model; wherein the target unimodal landslide identification and prediction model is used to characterize the correspondence between unimodal landslide data, the probability of occurrence of different types of landslides and geographic location information.

4. The method according to any one of claims 1 to 3, characterized in that: The method of using the unimodal landslide data as sample input and the landslide result data corresponding to the unimodal landslide data as expected sample output to construct a training sample including the sample input and the expected sample output includes: Taking the unimodal landslide data as sample input, taking the landslide result data corresponding to the unimodal landslide data as expected sample output, and constructing a basic sample pair consisting of the sample input and the expected sample output; A supervision signal is constructed by self-supervised learning for the single-mode landslide data as the sample input, and an expected supervision output corresponding to the supervision signal is determined, and a self-supervised incremental sample pair constructed by the sample input and the expected supervision output is constructed; The training samples are constructed based on the basic sample pairs and the self-supervised incremental sample pairs.

5. The method according to claim 4, characterized in that The self-supervised learning method includes: randomly blocking a part of the area in the unimodal landslide data, and using the unblocked area to predict the blocked area, or using the unimodal landslide data at the previous moment to predict the unimodal landslide data at the current moment.

6. A method for training a landslide identification and prediction base model, characterized in that: Applied to the second device, including: Obtaining model parameters of the single-mode landslide identification and prediction model respectively sent from multiple first device ends; wherein the single-mode landslide identification and prediction model is trained based on the training method of the single-mode landslide identification and prediction model according to any one of claims 1 to 5; Based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, multimodal data fusion is performed on multiple single-modal landslide identification and prediction models corresponding to multiple model parameters to obtain a target landslide identification and prediction base model.

7. The method according to claim 6, characterized in that The obtaining of model parameters of the single-mode landslide identification and prediction models respectively sent by the plurality of first device ends includes: Acquire characteristic graphs of the single-mode landslide identification and prediction models respectively sent by the plurality of first device ends.

8. The method according to claim 6, characterized in that Also includes: The model parameters of the new single-mode landslide identification and prediction model respectively sent by each of the first device ends are received only within a preset time period; wherein the receiving start time is the training completion time of the target landslide identification and prediction base model, and the preset time period is determined based on the device performance of each of the first device ends and the data transmission performance between each of the first device ends and the second device end; Based on the model parameters of each of the new single-mode landslide identification and prediction models, the target landslide identification and prediction base model is updated.

9. The method according to claim 8, characterized in that Also includes: no longer receiving the model parameters of the new single-mode landslide identification and prediction model sent by each of the first device ends respectively beyond the preset time length; or The model parameters of the new single-mode landslide identification and prediction model sent by each of the first device ends respectively beyond the preset time length are discarded.

10. The method according to claim 8, characterized in that Also includes: The model parameters of the new unimodal landslide identification and prediction model sent by the first device end beyond the preset time length are temporarily stored, and the model parameters of the temporarily stored new unimodal landslide identification and prediction model are used together with the model parameters of other new unimodal landslide identification and prediction models received within the next preset time length to participate in the subsequent update of the landslide identification and prediction base model.

11. The method according to claim 8, characterized in that The preset duration can be determined by the following method: respectively counting the actual time taken by each first device end to send the generated new single-mode landslide identification prediction model or the model parameters of the new single-mode landslide identification prediction model to the second device end under the device performance and the data transmission performance; In each of the actual time consumptions, a first target duration corresponding to a preset number of model parameters of new unimodal landslide identification and prediction models can be received, and a second target duration corresponding to a preset proportion of the total number of unimodal landslide identification and prediction models can be received; The preset duration is determined based on the first target duration or the second target duration.

12. The method according to claim 11, characterized in that The determining the preset duration based on the first target duration or the second target duration includes: The first target duration or the second target duration is subjected to redundancy correction in combination with a preset incremental correction value, and the duration obtained after the redundant correction is determined as the preset duration; wherein the duration corresponding to the incremental correction value is determined based on the excess duration of the moment when at least one of the first device ends sends model parameters of a new single-mode landslide identification model that exceeds the cutoff moment of the preset duration.

13. The method according to claim 8, characterized in that The updating of the target landslide identification and prediction base model based on the model parameters of each of the new single-mode landslide identification and prediction models comprises: For each model parameter of the new single-mode landslide identification and prediction model, the accuracy difference between the prediction results output by the new single-mode landslide identification and prediction model and the corresponding single-mode landslide identification and prediction model for the preset verification samples is verified; In response to the accuracy difference being manifested as the new single-mode landslide identification and prediction model having a higher prediction accuracy, determining the corresponding new single-mode landslide identification and prediction model as a valid update model; Only the valid update models are used to update the corresponding single-mode landslide identification and prediction model part in the target landslide identification and prediction base model.

14. The method according to claim 6, characterized in that Also includes: In response to the received model parameters being in a compressed and / or encrypted state, the model parameters are processed in a pre-agreed decompression and / or decryption manner.

15. The method according to claim 6, characterized in that Before the multimodal data fusion of the multiple single-modal landslide identification and prediction models corresponding to the multiple model parameters is performed based on the polynomial neural network with the channel attention mechanism and the spatial attention mechanism, the method further includes: The model parameters of each of the single-mode landslide identification and prediction models are merged or deduplicated; wherein the merging or deduplication refers to merging or removing the model parameters of any two single-mode landslide identification and prediction models that have a similarity greater than a preset degree so that only one is retained in the end.

16. The method according to claim 15, characterized in that The method of performing multimodal data fusion on a plurality of single-modal landslide identification and prediction models corresponding to a plurality of model parameters based on a polynomial neural network having a channel attention mechanism and a spatial attention mechanism comprises: Splicing the deduplicated feature maps of each single-mode landslide identification and prediction model to obtain a spliced ​​feature map; Determine the channel attention weights of each part of the feature map in the spliced ​​feature map based on the channel attention mechanism; Determine the spatial attention weights of each part of the feature map in the spliced ​​feature map based on the spatial attention mechanism; The partial feature maps of the spliced ​​feature map are combined with the corresponding channel attention weights and spatial attention weights to perform feature fusion.

17. The method according to any one of claims 6 to 16, characterized in that: The polynomial neural network is obtained in the following way: On the base network served by the residual network, the addition operation originally used by each residual module on the cross-layer connection is converted into a product operation, and the original activation function is removed.

18. A localization method for a landslide identification prediction model, characterized in that: Applied to the third-party device, including: Initiating a model localization request of a target mode including the landslide data used locally to the second device end; Obtaining a trimmed model or a lightweight model based on the target landslide identification and prediction base model at the second device end; The trimmed model is a model obtained by trimming the model parameters and model structures in the target landslide identification and prediction base model whose correlation with the target modality is lower than a preset degree, the target landslide identification and prediction base model is obtained based on the training method of the landslide identification and prediction base model according to any one of claims 6 to 16, and the lightweight model is a model obtained by performing a preset lightweight processing on the target landslide identification and prediction base model when the trimming of the model parameters and model structures in the target landslide identification and prediction base model cannot be completed based on the target modality; The trimmed model or the lightweight model is fine-tuned using localized landslide training samples with the target mode to obtain a localized landslide identification and prediction model.

19. A landslide identification and prediction method, characterized in that: Applied to the third-party device, including: receiving actual landslide data of a target mode acquired at a target geographic location; Calling a preset localized landslide identification prediction model to process the actual landslide data to obtain an output landslide identification result; wherein the localized landslide identification prediction model is obtained based on the localization method of the landslide identification prediction model of claim 18; The occurrence probabilities of different types of landslides are determined according to the landslide identification results.

20. The method according to claim 19, characterized in that The third device terminal generates different warning information according to the occurrence probability of different types of landslides, and sends the warning information to at least one fourth device terminal.

21. A first device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method for the single-modal landslide identification and prediction model according to any one of claims 1-5.

22. A second device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for training a landslide identification and prediction base model according to any one of claims 6-17.

23. A third device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the localization method of the landslide identification and prediction model described in claim 18 and / or the landslide identification and prediction method described in any one of claims 19-20.

24. A landslide identification and prediction system, characterized in that: include: A plurality of first device terminals are used to obtain unimodal landslide data collected by a preset unimodal landslide data collection device, wherein the unimodal landslide data includes any one of the following: satellite optical image data, airborne optical image data, satellite synthetic aperture radar image data, ground-based synthetic aperture radar data, airborne synthetic aperture radar data, airborne lidar data, on-site survey photos, monitoring data collected by sensors on the surface of the landslide body, and monitoring data collected by sensors inside the slope body; the unimodal landslide data is used as sample input, and the landslide result data corresponding to the unimodal landslide data is used as expected sample output to construct a training sample including the sample input and the expected sample output, wherein the landslide result data includes: whether the landslide occurs and the type of the landslide; the unimodal landslide recognition and prediction model to be trained is trained using the training samples to obtain a target unimodal landslide recognition and prediction model, wherein the unimodal landslide recognition and prediction model is at least used to characterize the corresponding relationship between the unimodal landslide data and the occurrence probability of different types of landslides; The second device end is used to receive the model parameters of the single-modal landslide recognition prediction model sent by each of the first device ends respectively; based on a polynomial neural network with a channel attention mechanism and a spatial attention mechanism, perform multimodal data fusion on multiple single-modal landslide recognition prediction models corresponding to multiple model parameters to obtain a target landslide recognition prediction base model; receive a model localization request of the target mode containing the landslide data used locally initiated by the third device end; The third device end is used to receive the cropped model or the lightweight model returned by the second device end; fine-tune the cropped model or the lightweight model using the localized landslide training sample of the target mode to obtain a localized landslide identification and prediction model; receive actual landslide data of the target mode collected from the target geographical location; call the preset localized landslide identification and prediction model to process the actual landslide data to obtain an output landslide identification result; determine the occurrence probability of different types of landslides according to the landslide identification result.

25. The system according to claim 24, characterized in that The second device is also used for: In the target landslide identification and prediction base model, determining target model parameters and target model structures having a correlation degree with the target mode lower than a preset degree; The target model parameters and the target model structure in the target landslide identification and prediction base model are trimmed to obtain the trimmed model.

26. The system according to claim 25, characterized in that The second device is also used for: In response to failure to determine the target model parameters and the target model structure having a correlation degree with the target modality lower than the preset degree, lightweight processing is performed on the target landslide identification and prediction base model.

27. The system according to claim 26, characterized in that The lightweighting process includes a model distillation process.

28. The system according to claim 24, characterized in that The system further comprises: A plurality of fourth device ends, wherein the fourth device ends are used to receive and present different warning information generated by the third device end according to the occurrence probability of different types of landslides.

29. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the training method of the single-modal landslide identification and prediction model described in any one of claims 1-5 and / or the training method of the landslide identification and prediction base model described in any one of claims 6-17 and / or the localization method of the landslide identification and prediction model described in claim 18 and / or the landslide identification and prediction method described in any one of claims 19-20.

30. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the steps of the training method of the single-modal landslide identification and prediction model according to any one of claims 1-5 and / or the steps of the training method of the landslide identification and prediction base model according to any one of claims 6-17 and / or the steps of the localization method of the landslide identification and prediction model according to claim 18 and / or the steps of the landslide identification and prediction method according to any one of claims 19-20.

Images