Landslide hazard correlation element identification method and device based on multi-modal fusion and pruning

Abstract

The application discloses a landslide hidden danger related element identification method and device based on multi-modal fusion and pruning, and relates to the technical field of geological disaster identification.The landslide hidden danger related element identification method based on multi-modal fusion and pruning comprises the following steps: acquiring initial data, wherein the initial data comprises aerial optical spectrum data, InSAR deformation data and terrain data; performing wave band level superposition on the initial data to generate a multi-channel fusion data matrix; performing channel pruning, network reconstruction and fine-tuning learning processing on a first lightweight model network based on the multi-channel fusion data matrix to obtain a target lightweight model network; acquiring to-be-identified data and inputting the to-be-identified data into the target lightweight model network to output landslide hidden danger related element data.The landslide hidden danger related element identification method based on multi-modal fusion and pruning can combine multi-modal data fusion and lightweight pruning technology, and improve the identification precision and real-time processing capability of landslide hidden danger related elements.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of geological disaster identification, and more specifically to a landslide hidden danger associated element identification method and device based on multi-modal fusion and pruning. BACKGROUND

[0002] The formation process of geological disasters is influenced by disaster environment, inducing factors, disaster bearing law, etc., and the apparent characteristics on remote sensing images are complex, making intelligent interpretation difficult. Existing landslide hidden danger associated element intelligent identification is mainly based on InSAR data, optical image data, terrain data and other multi-source data, and optimizes the classical deep learning network model framework to realize automatic identification of associated elements.

[0003] With the development of earth observation technology, a large amount of remote sensing images and data have been generated, which puts higher requirements on the rapid extraction of regional or large-scale geological disasters. At present, with the increase of multi-modal input data, the model complexity is significantly improved, and the network parameter quantity is inflated, making the training process more dependent on computing resources and consuming more hardware resources, which further causes the current situation of low processing efficiency of landslide hidden danger associated element intelligent identification. Specifically, the current geological disaster hidden danger intelligent identification technology mainly faces the following bottlenecks: (1) directly using traditional natural language general model to process multi-modal data, the data collaborative gain information extraction and utilization is insufficient; (2) in the large-scale geological disaster monitoring scene, there are problems such as model parameter redundancy, imbalance between model reasoning effect and speed, etc. SUMMARY

[0004] The present application provides a landslide hidden danger associated element identification method and device based on multi-modal fusion and pruning, which combines multi-modal data fusion and lightweight pruning technology to improve the identification accuracy and real-time processing capability of landslide hidden danger associated elements.

[0005] According to a first aspect of the present application, the present application provides a landslide hidden danger associated element identification method based on multi-modal fusion and pruning, which comprises: acquiring initial data, the initial data comprising: aerial optical spectrum data, InSAR deformation data, and terrain data; performing band level superposition on the initial data to generate a multi-channel fusion data matrix; performing channel pruning, network reconstruction, and fine-tuning learning processing on a first lightweight model network based on the multi-channel fusion data matrix to obtain a target lightweight model network; acquiring to-be-identified data and inputting the to-be-identified data into the target lightweight model network to output landslide hidden danger associated element data.

[0006] The above-mentioned embodiment of the present application can comprehensively apply multi-source remote sensing data, and prune redundant feature layers in the model based on the multi-modal data fusion network model, significantly reduce the model parameter quantity, and improve the inference efficiency of the model business application.

[0007] In some embodiments of the present application, the multi-modal fusion and pruning landslide hazard correlation element identification method further comprises: registering the initial data of different modalities, unifying the spatial resolution and pixel size, and then performing band-level superposition.

[0008] In some embodiments of the present application, the first lightweight model network is a model network based on a U-shaped structure, wherein the encoder of the U-shaped structure adopts part of the network structure of MobileNetV2, and the decoding part of the U-shaped structure adopts inverse convolution and reverse residual structure.

[0009] In some embodiments of the present application, the target lightweight model network obtained by performing channel pruning, network reconstruction, and fine-tuning learning processing on the first lightweight model network based on the multi-channel fusion data matrix comprises: extracting feature maps from the multi-channel fusion data matrix through the first lightweight model network; performing normalization preprocessing on the input feature maps through the batch normalization layer in the first lightweight model network, performing linear transformation on the normalized values through the first adjustment factor and the second adjustment factor, and obtaining the second lightweight model network in which the first adjustment factor is sparse; obtaining a preset pruning ratio and the absolute value distribution of the first adjustment factor of each channel in the batch normalization layer, calculating the quantile threshold of the absolute value distribution according to the preset pruning ratio, traversing the channels of each convolution layer in the second lightweight model network, removing the channels and their corresponding input and output connections and weights in which the absolute value of the first adjustment factor is less than the quantile threshold, and obtaining the third lightweight model network after pruning; and fine-tuning the third lightweight model network to obtain the target lightweight model network.

[0010] In some embodiments of the present application, the input feature map is normalized by the batch normalization layer in the first lightweight model network, and the linear transformation of the normalized value by the first and second learnable adjustment factors comprises: mapping the parameters of the input feature map to a normal distribution range with a mean of 0 and a variance of 1 by the following formula, and linearly transforming the normalized value by the first and second learnable adjustment factors:

[0011]

[0012] a o = γ·τ i + β

[0013] wherein a i is the input feature map, μ i and σ i are the mean and standard deviation obtained according to the statistics of the current batch data i, τ i is the normalized activation value, ∈ is an extremely small constant, γ and β are the first and second adjustment factors learned independently for each channel feature map, and a o is the output feature map.

[0014] In some embodiments of the present application, the first adjustment factor of the batch normalization layer is subjected to L1 regularization constraint by the following formula:

[0015]

[0016] wherein (x, y) is the training input and target, W is the training weight, ∑ (x,y) l(f((x, W)), y) is the normal training loss function of the convolutional neural network, λ∑ γ∈Γ g(γ) is the L1 regularization of γ, wherein γ is the scaling factor, λ is the penalty coefficient, and L is the total loss function of the entire training process.

[0017] According to the second aspect of the present application, the embodiments of the present application provide a landslide hazard correlation element identification device for multi-modal fusion and pruning, which comprises: a fusion data generation module, configured to obtain initial data and perform waveband-level superposition on the initial data to generate a multi-channel fusion data matrix, wherein the initial data comprises: aerial optical spectrum data, InSAR deformation data, and terrain data; a lightweight model network generation module, configured to perform channel pruning, network reconstruction, and fine-tuning learning processing on a first lightweight model network based on the multi-channel fusion data matrix to obtain a target lightweight model network; and a landslide hazard correlation element identification module, configured to obtain to-be-identified data and input the to-be-identified data into the target lightweight model network to output landslide hazard correlation element data.

[0018] The above-mentioned embodiments of the present application can comprehensively apply multi-source remote sensing data, and prune redundant feature layers in the model based on the multi-modal data fusion network model, significantly reduce the model parameter quantity, and improve the inference efficiency of the model business application.

[0019] In some embodiments of the present application, the fusion data generation module is further configured to register the initial data of different modalities, unify the spatial resolution and pixel size, and then perform band-level superposition.

[0020] In some embodiments of the present application, the first lightweight model network is a model network based on a U-shaped structure, wherein the encoder of the U-shaped structure adopts part of the network structure of MobileNetV2, and the decoding part of the U-shaped structure adopts inverse convolution and reverse residual structure.

[0021] In some embodiments of the present application, the target lightweight model network obtained by performing channel pruning, network reconstruction, and fine-tuning learning processing on the first lightweight model network based on the multi-channel fusion data matrix comprises: extracting feature maps from the multi-channel fusion data matrix through the first lightweight model network; performing normalization preprocessing on the input feature maps through the batch normalization layer in the first lightweight model network, performing linear transformation on the normalized values through the first adjustment factor and the second adjustment factor, and obtaining the second lightweight model network in which the first adjustment factor is sparse; obtaining a preset pruning ratio and the absolute value distribution of the first adjustment factor of each channel in the batch normalization layer, calculating the quantile threshold of the absolute value distribution according to the preset pruning ratio, traversing the channels of each convolution layer in the second lightweight model network, removing the channels and their corresponding input and output connections and weights in which the absolute value of the first adjustment factor is less than the quantile threshold, and obtaining the third lightweight model network after pruning; and fine-tuning the third lightweight model network to obtain the target lightweight model network.

[0022] In summary, the landslide hazard associated element identification method and device provided by the embodiment of the application are characterized in that: multi-source remote sensing data are comprehensively applied, and the redundant feature layers in the model are pruned on the basis of the multi-modal data fusion network model, so that the model parameter quantity is significantly reduced, and the inference efficiency of the model in business application is improved. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a flowchart of a landslide hazard associated element identification method according to an embodiment of the application;

[0024] Figure 2 is a schematic diagram of a landslide hazard associated element identification network structure according to an embodiment of the application;

[0025] Figure 3 is a schematic diagram of a landslide hazard associated element identification device according to an embodiment of the application. DETAILED DESCRIPTION

[0026] The various aspects of the application will be described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that the various embodiments described below are only used for illustration, and are not used to limit the protection scope of the application, and various different combinations of the various embodiments of the application can be made without departing from the essence of the application.

[0027] The terms used below are briefly described as follows.

[0028] InSAR: Interferometric Synthetic Aperture Radar, interferometric synthetic aperture radar, refers to the interferometric synthetic aperture radar measurement technology.

[0029] MobileNetV2: a lightweight convolutional neural network architecture.

[0030] Figure 1 is a flowchart of a landslide hazard associated element identification method according to an embodiment of the application.

[0031] As Figure 1 shown, in an embodiment of the application, the landslide hazard associated element identification method can at least include the following steps S11, S12, S13 and S14, which will be described in detail below.

[0032] In step S11, initial data is obtained, which includes aerial optical spectrum data, InSAR deformation data and terrain data.

[0033] In step S12, the initial data is subjected to band-level superposition to generate a multi-channel fusion data matrix. In this embodiment, the initial data of different modalities is registered, unified in spatial resolution and pixel size, and then subjected to band-level superposition.

[0034] In step S13, a target lightweight model network is obtained by subjecting the first lightweight model network to channel pruning, network reconstruction and fine-tuning learning based on the multi-channel fusion data matrix. In this embodiment, the first lightweight model network is a model network based on a U-shaped structure, wherein the encoder of the U-shaped structure adopts part of the network structure of MobileNetV2, and the decoding part of the U-shaped structure adopts inverse convolution and reverse residual structure.

[0035] In some embodiments, obtaining the target lightweight model network by subjecting the first lightweight model network to channel pruning, network reconstruction and fine-tuning learning based on the multi-channel fusion data matrix comprises: extracting feature maps from the multi-channel fusion data matrix by the first lightweight model network; performing normalization preprocessing on the input feature maps by the batch normalization layer in the first lightweight model network, performing linear transformation on the normalized values by the first and second learnable adjustment factors; applying L1 regularization constraint to the first adjustment factor of the batch normalization layer, and synchronously optimizing the network weight and the first adjustment factor to obtain a second lightweight model network in which the first adjustment factor is sparsely distributed; obtaining a preset pruning ratio and an absolute value distribution of the first adjustment factor of each channel in the batch normalization layer, calculating a quantile threshold value of the absolute value distribution according to the preset pruning ratio; traversing the channels of each convolution layer in the second lightweight model network, and removing the channels whose absolute values of the first adjustment factor are less than the quantile threshold value, as well as the corresponding input and output connections and weights, to obtain a third lightweight model network after pruning; fine-tuning the third lightweight model network to obtain a target lightweight model network.

[0036] In further embodiments, performing normalization preprocessing on the input feature maps by the batch normalization layer in the first lightweight model network, and performing linear transformation on the normalized values by the first and second learnable adjustment factors comprises:

[0037] The parameters of the input feature maps are mapped to a normal distribution range with a mean of 0 and a variance of 1 by the following formulas (1) and (2), and the normalized values are linearly transformed by the first and second learnable adjustment factors:

[0038]

[0039] a o = γ · τ i + β (2)

[0040] wherein, a i is an input feature map, μ i and σ i are respectively a mean value and a standard deviation obtained according to a statistic of a current batch data i, τ i is a normalized activation value, ∈ is a very small constant, γ and β are respectively a first adjustment factor and a second adjustment factor corresponding to each channel feature map independently learned, a o is an output feature map. The first adjustment factor is a scaling factor, and the second adjustment factor is an offset factor.

[0041] The first adjustment factor of the batch normalization layer is subjected to L1 regularization constraint through the following formula (3):

[0042] L = ∑ (x,y) l(f((x,W)),y) + λ∑ γ∈Γ g(γ) (3)

[0043] wherein, (x,y) is a training input and target, W is a training weight, ∑ (x,y) l(f((x,W)),y) is a normal training loss function of a convolutional neural network, λ∑ γ∈Γ g(γ) is L1 regularization of γ, wherein γ is a scaling factor (i.e. the first adjustment factor), λ is a penalty coefficient, and L represents a total loss function of the entire training process.

[0044] In step S14, the to-be-identified data is obtained and input into the target lightweight model network, and landslide hazard correlation element data is output. In the embodiment, the to-be-identified data is 7 channel data of test data, and the output hazard correlation element data includes but is not limited to deformation characteristics, atlas characteristics or disaster-bearing body elements of landslide hazards.

[0045] By using the above method of the embodiment of the application, the multi-source remote sensing initial data is fused, the first lightweight model network is processed by channel pruning and the like based on the fused data to generate a target lightweight model network, and then the to-be-identified data is input into the target lightweight model network to obtain landslide hazard correlation element data, which can comprehensively apply multi-source remote sensing data, prune redundant feature layers in the model based on the multi-modal data fusion network model, significantly reduce the model parameter quantity, and improve the inference efficiency of the model business application.

[0046] The following gives a kind of using Figure 1The multi-modal fusion and pruning landslide hazard correlation element identification method shown in the specific example of landslide hazard correlation element identification, the following will be combined with Figure 2 Specifically described:

[0047] In this example, the multi-modal fusion and pruning landslide hazard correlation element identification method can at least include the following steps:

[0048] Step one: comprehensively apply the aerial optical spectrum data, InSAR deformation data, and terrain data of landslide hazard correlation elements, adopt a parallel fusion strategy, perform band-level superposition, and generate a multi-channel fusion data matrix. Among them, the aerial optical spectrum data is 3 bands of visible light red green blue data, the InSAR deformation data is 3 channels of deformation phase data, and the terrain data is 1 channel of mountain shadow data.

[0049] In this embodiment, the initial data of different modalities (such as aerial optical spectrum data, InSAR deformation data, terrain data, etc.) are registered, unified spatial resolution and pixel size, and then band-level superposition is performed. When the resolution of the initial data is inconsistent, resampling operation is performed to ensure that the data is unified to the same resolution. In this way, the data of different modalities can be aligned in geographical space and have the same spatial resolution and pixel size, which facilitates subsequent band-level superposition of the initial data and generation of a multi-channel fusion data matrix.

[0050] Step two: construct a landslide hazard correlation element identification network structure as shown in Figure 2 The network structure is based on a U-shaped structure, and the encoding part of the U-shaped structure is composed of the main network mobilenetv2 part structure, and the decoding part of the U-shaped structure introduces inverse convolution and residual structure. The encoding part of the U-shaped structure extracts features of different levels from the fusion data (multi-channel fusion data matrix) generated in step one to obtain feature maps of different levels; the decoding part of the U-shaped structure serially fuses the inverse convolution and residual structure with the feature maps of different levels obtained by the encoding part to obtain high-level abstract semantic features.

[0051] In the landslide hazard correlation element identification network structure of this embodiment, the decoding part of the U-shaped structure uses inverse convolution and residual structure instead of the original up-sampling convolution layer, feature splicing layer and two 3x3 ordinary convolution layers. By introducing inverse convolution and residual structure, the correlation element feature expression and the model calculation efficiency are enhanced.

[0052] Among them, the layer structure of the aforementioned landslide hazard correlation element identification network structure is shown in Table 1:

[0053] Table 1

[0054]

[0055]

[0056] In Table 1, Conv2d 3x3 is a two-dimensional convolution layer with a convolution kernel size of 3x3, Bneckx1 is a single bottleneck module; Bneckx2 is the use of 2 bottleneck modules in succession; Bneckx3 is the use of 3 bottleneck modules in succession; Bneckx7 is the use of 7 bottleneck modules in succession; Bneckx4 is the use of 4 bottleneck modules in succession; Conv2d 1x1 is a two-dimensional convolution layer with a convolution kernel size of 1x1; TransConv 4x4 is a transpose convolution with a convolution kernel size of 4x4; InvertedRes is an inverted residual module.

[0057] Step three: On the basis of the lightweight model network constructed in step two, a channel-level pruning method is used to prune unimportant feature layers. Specifically, a Batch Normalization layer (BN layer) is set for normalization preprocessing, and all parameters of the input feature map are mapped to a normal distribution range with a mean of 0 and a variance of 1 through the foregoing formulas (1) and (2), so that each mapped parameter learns the corresponding first adjustment factor γ and second adjustment factor β in the training process. Among them, γ can be regarded as the weight of each channel of the BN layer feature map. When r i tends to 0, the output channel corresponding to the feature map is all constant, that is, the channel has no contribution to the subsequent operation.

[0058] Step four: sparse constraint training. Specifically, the first adjustment factor (i.e. scaling factor) γ of the Batch Normalization layer is subjected to L1 regularization constraint through the foregoing formula (3), and the network weight and scaling factor are simultaneously optimized in the training process, so as to induce the scaling factor to present sparse distribution.

[0059] Step five: channel pruning. Specifically, after the channel sparse regularization training in step four, a network model is obtained in which many scaling factors γ tend to 0. The absolute value distribution of all BN layer scaling factors γ ∈ Γ in the network model is counted, and the quantile threshold value of the absolute value distribution is calculated according to the set pruning percentage.

[0060] Step six: network reconstruction. Specifically, the channels of each convolution layer in the network obtained in step five are traversed, the channels that do not meet the foregoing quantile threshold value are removed, and the corresponding input and output connections and the corresponding weights of the removed channels are also deleted synchronously, to obtain a compact pruned network. For example, the channel number comparison results of part of the convolution layers before and after pruning by 50% are shown in Table 2 as follows:

[0061] Table 2

[0062]

[0063]

[0064]

[0065] wherein, layer index is the number identifying each layer in the neural network, total channel number is the original channel number of a certain layer before pruning, and remaining channel number is the channel number reserved after pruning. For example, the last row in Table 1 indicates that the original channel number of the 230th layer in the neural network before pruning is 192, and the remaining channel number is 98.

[0066] Step seven: fine-tuning the pruned network through the fine-tuning technology to obtain a more compact target model. For example, the volume, parameter quantity (Parameters) and calculation quantity (FLOPs) of the original model and the target model pruned by 50% are shown in Table 3 as follows:

[0067] Table 3

[0068]

[0069] wherein, Baselines is the baseline model, i.e. the original model (the initial version without pruning or optimization) serving as a comparison benchmark; and γ-Prune 50% refers to the target model pruned by 50% based on the γ parameter. As shown in Table 3, the volume, parameter quantity and calculation quantity of the target model are significantly reduced, and the precision and accuracy of the target model are improved. It should be understood that the value range of the pruning ratio used for channel pruning in step five is between 0.01 and 0.99, and the selection of the pruning ratio needs to seek a balance point between model volume, calculation efficiency and calculation performance. Generally, the final pruning ratio needs to be determined through experimental verification, for example, a gradual adjustment strategy can be adopted to gradually increase the pruning ratio and simultaneously evaluate the model precision loss, and finally the best pruning ratio considering light weight and performance is determined based on experimental data.

[0070] Step eight: obtaining the to-be-identified data and inputting the to-be-identified data into the target model network to output landslide hazard associated element data.

[0071] The landslide hazard associated element identification method of the above multi-modal fusion and pruning according to the embodiments of the present application comprehensively applies multi-source remote sensing data, adopts a serial and parallel interactive fusion strategy, constructs a multi-modal data fusion network model, further prunes the redundant feature layers in the model by using a channel pruning method, which can significantly reduce the model parameter quantity, improve the inference efficiency of the model in business application, and improve the identification accuracy and real-time processing capability of the landslide hazard associated elements.

[0072] Figure 3 FIG. 1 is a schematic diagram of an architecture of a landslide hazard correlation element identification device according to an embodiment of the present application.

[0073] As shown in Figure 3 FIG. 1, the landslide hazard correlation element identification device includes a fusion data generation module 310, a lightweight model network generation module 320, and a landslide hazard correlation element identification module 330.

[0074] The fusion data generation module 310 is configured to obtain initial data and perform waveband-level superposition on the initial data to generate a multi-channel fusion data matrix, wherein the initial data includes aerial optical spectrum data, InSAR deformation data, and terrain data.

[0075] In this embodiment, the fusion data generation module is further configured to perform waveband-level superposition after registering the initial data of different modalities and unifying the spatial resolution and pixel size.

[0076] The lightweight model network generation module 320 is configured to perform channel pruning, network reconstruction, and fine-tuning learning processing on a first lightweight model network based on the multi-channel fusion data matrix to obtain a target lightweight model network.

[0077] In this embodiment, the first lightweight model network is a model network based on a U-shaped structure, wherein the encoder of the U-shaped structure adopts part of the network structure of MobileNetV2, and the decoding part of the U-shaped structure adopts inverse convolution and reverse residual structure.

[0078] In some embodiments, the target light-weight model network obtained by performing channel pruning, network reconstruction, and fine-tuning processing on the first light-weight model network based on the multi-channel fusion data matrix comprises: performing feature map extraction on the multi-channel fusion data matrix by the first light-weight model network; performing normalization preprocessing on the input feature map by a batch normalization layer in the first light-weight model network, performing linear transformation on the normalized value by a first adjustment factor and a second adjustment factor, applying L1 regularization constraint to the first adjustment factor of the batch normalization layer, and synchronously optimizing the network weight and the first adjustment factor to obtain a second light-weight model network in which the first adjustment factor is sparse; obtaining a preset pruning ratio and an absolute value distribution of the first adjustment factor of each channel in the batch normalization layer, calculating a quantile threshold of the absolute value distribution according to the preset pruning ratio, traversing the channels of each convolution layer in the second light-weight model network, and removing the channels and corresponding input and output connections and weights of the first adjustment factor of the channels corresponding to each convolution layer in the second light-weight model network whose absolute value is less than the quantile threshold to obtain a third light-weight model network after pruning; and performing fine-tuning on the third light-weight model network to obtain a target light-weight model network.

[0079] The landslide hazard correlation element identification module 330 is configured to obtain to-be-identified data and input the to-be-identified data into the target light-weight model network to output landslide hazard correlation element data.

[0080] The device described above in the embodiments of the present application can comprehensively apply multi-source remote sensing data, prune redundant feature layers in the model based on the multi-modal data fusion network model, significantly reduce the number of model parameters, and improve the inference efficiency of the model in business application.

[0081] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software combined with a hardware platform. Based on this understanding, all or part of the technical solutions of the present application that contribute to the background art can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in each embodiment or some part of the embodiment of the present application.

[0082] Correspondingly, the embodiment of the present application also provides a computer readable storage medium, which stores computer readable instructions or programs, and the computer readable instructions or programs are executed by a processor to make the processor perform the following operations: the operations include the steps of the method according to any one of the above embodiments, which will not be repeated here. Wherein, the storage medium can include: for example, optical disc, hard disk, floppy disk, flash memory, magnetic tape, etc. Part or all of the above embodiments of the present application can be implemented by a computer device including the above storage medium and the processor, and the processor executes the computer readable instructions in the storage medium to perform the steps, operations, processes recorded in any one of the above embodiments.

[0083] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. Therefore, the protection scope of the present application should be subject to the claims.

Claims

1. A landslide hazard correlation factor identification method based on multi-modal fusion and pruning, the method comprising: obtaining initial data, the initial data comprising: aerial optical spectrum data, InSAR deformation data, and terrain data; performing band-level superposition on the initial data to generate a multi-channel fusion data matrix; performing channel pruning, network reconstruction, and fine-tuning learning on a first lightweight model network based on the multi-channel fusion data matrix to obtain a target lightweight model network, wherein the first lightweight model network is a model network based on a U-shaped structure, wherein the encoder of the U-shaped structure adopts part of the network structure of MobileNetV2, and the decoding part of the U-shaped structure adopts an inverse convolution and an inverted residual structure; obtaining to-be-identified data and inputting the to-be-identified data into the target lightweight model network to output landslide hazard correlation factor data; wherein performing channel pruning, network reconstruction, and fine-tuning learning on the first lightweight model network based on the multi-channel fusion data matrix to obtain the target lightweight model network comprises: extracting feature maps from the multi-channel fusion data matrix through the first lightweight model network; performing normalization preprocessing on the input feature maps through a batch normalization layer in the first lightweight model network, performing linear transformation on the normalized values through a first learnable adjustment factor and a second learnable adjustment factor; applying L1 regularization constraint to the first adjustment factor of the batch normalization layer, and simultaneously optimizing the network weight and the first adjustment factor to obtain a second lightweight model network in which the first adjustment factor is sparsely distributed; obtaining a preset pruning ratio and an absolute value distribution of the first adjustment factor of each channel in the batch normalization layer, calculating a quantile threshold of the absolute value distribution according to the preset pruning ratio; traversing the channels of each convolution layer in the second lightweight model network, and removing the channels and their corresponding input-output connections and weights whose absolute values of the first adjustment factor are less than the quantile threshold, to obtain a third lightweight model network after pruning; performing fine-tuning learning on the third lightweight model network to obtain the target lightweight model network.

2. The method of claim 1, wherein, The method further comprises: registering the initial data of different modalities, unifying the spatial resolution and pixel size, and then performing band-level superposition.

3. The method of claim 1, wherein, performing normalization preprocessing on the input feature maps through a batch normalization layer in the first lightweight model network, performing linear transformation on the normalized values through a first learnable adjustment factor and a second learnable adjustment factor comprises: mapping the parameters of the input feature maps to a normal distribution range with a mean of 0 and a variance of 1 through the following formula, performing linear transformation on the normalized values through a first learnable adjustment factor and a second learnable adjustment factor, and restoring the expression ability of the data: a o = γ · τ i + β wherein a i is the input feature map, μ i and σ i are the mean and standard deviation obtained according to the statistics of the current batch data i, τ i is the normalized activation value, ∈ is a very small constant, γ and β are the first and second adjustment factors corresponding to each channel feature map independently learned, and a o is the output feature map.

4. The method of claim 1, wherein, applying L1 regularization constraint to the first adjustment factor of the batch normalization layer through the following formula: where (x, y) is the training input and target, W is the training weight, ∑ (x,y) l(f((x,W)),y) is the normal training loss function of the convolutional neural network, λ∑ γ∈Γ g(γ) is the L1 regularization of γ, where γ is the first adjustment factor, λ is the penalty coefficient, and L is the total loss function throughout the training process.

5. A landslide hazard correlation factor identification device with multi-modal fusion and pruning, characterized in that, The apparatus comprises: The fusion data generation module is configured to obtain initial data and perform band-level superposition on the initial data to generate a multi-channel fusion data matrix, wherein the initial data includes aerial optical spectrum data, InSAR deformation data and terrain data. The lightweight model network generation module is configured to obtain a target lightweight model network by performing channel pruning, network reconstruction and fine-tuning learning processing on a first lightweight model network based on the multi-channel fusion data matrix, wherein the first lightweight model network is a model network based on a U-shaped structure, the encoder of the U-shaped structure adopts a partial network structure of MobileNetV2, and the decoding part of the U-shaped structure adopts an inverse convolution and reverse residual structure. The landslide hidden danger correlation element identification module is configured to obtain to-be-identified data and input the to-be-identified data into the target lightweight model network to output landslide hidden danger correlation element data. The target lightweight model network is obtained by performing channel pruning, network reconstruction and fine-tuning learning processing on the first lightweight model network based on the multi-channel fusion data matrix, including: extracting feature maps from the multi-channel fusion data matrix through the first lightweight model network; performing normalization preprocessing on the input feature maps through a batch normalization layer in the first lightweight model network, performing linear transformation on the normalized values through a first adjustment factor and a second adjustment factor, and obtaining a second lightweight model network in which the first adjustment factor is sparse; applying L1 regularization constraint to the first adjustment factor of the batch normalization layer, and synchronously optimizing network weights and the first adjustment factor to obtain the second lightweight model network in which the first adjustment factor is sparse; obtaining a preset pruning ratio and an absolute value distribution of the first adjustment factor of each channel in the batch normalization layer, calculating a quantile threshold of the absolute value distribution according to the preset pruning ratio, traversing channels of each convolution layer in the second lightweight model network, and removing channels and corresponding input-output connections and weights of the channels in the second lightweight model network whose absolute values of the first adjustment factors are less than the quantile threshold, to obtain a third lightweight model network after pruning; performing fine-tuning learning on the third lightweight model network to obtain the target lightweight model network. The fusion data generation module is further configured to register initial data of different modalities, unify spatial resolution and pixel size, and then perform band-level superposition.

6. The apparatus of claim 5, wherein, ​

Images