Sand storm multi-task detection method based on multi-source data fusion and long-distance modeling

Abstract

A sand storm multi-task detection method based on multi-source data fusion and long-distance modeling belongs to the field of sand storm detection, and comprises the following steps: S1, inputting meteorological reanalysis data into a meteorological reanalysis data processing network, extracting meteorological reanalysis data features step by step through two-dimensional convolution and pooling operation, and obtaining meteorological reanalysis data features; at least extracting meteorological reanalysis data features of four scales; s2, connecting the initial satellite data with the meteorological reanalysis data to obtain initial multi-source data, inputting the multi-source data into a multi-source data processing network, carrying out long-distance feature extraction to obtain fused data features, then connecting meteorological reanalysis data features of the same scale, continuing to carry out long-distance feature extraction, and obtaining fused data features; obtaining data features capable of representing the sand storm; and S3, performing up-sampling on the obtained data features, expanding the size of the data features to be consistent with the input data, and finally obtaining the occurrence range or the occurrence intensity of the sand storm through a classifier. According to the invention, the performance of sand storm strength detection is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of real-time sandstorm detection, and in particular relates to a sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling. Background Art

[0002] As one of the natural disasters in arid and semi-arid areas, sandstorms mainly occur in northwest China, including the southern Xinjiang Basin and southwestern Qinghai. The occurrence of sandstorms is affected by many factors, including temperature, wind speed, wind direction, etc. Sandstorm detection uses meteorological data including satellite data to detect the occurrence of sandstorms in real time. Since strong sandstorms can cause great damage to people's production and life, it is very necessary to detect the scope and intensity of sandstorms in real time to prevent such weather disasters.

[0003] Generally speaking, traditional methods focus on building physical models, but due to the complexity of weather systems, it is difficult for physical models to take into account all influencing factors. Therefore, with the development of artificial intelligence technology, many studies have proposed various deep learning methods to detect sandstorms. At present, most deep learning-based methods tend to detect the scope of sandstorms and ignore the intensity detection of sandstorms. In addition, from a technical point of view, deep learning-based research is often based on pure convolutional architectures, which makes it difficult to overcome the inherent locality problem of convolution. At the same time, these studies focus on a single data source and lack the exploration of using multi-source data to detect sandstorms. These problems have led to bottlenecks in detection performance. Summary of the invention

[0004] In order to overcome the shortcomings of the existing technology, the present invention provides a sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling to improve the performance of sandstorm intensity detection. Multi-source data fusion and long-distance modeling are used to solve the problem of the lack of simultaneous detection of sandstorm occurrence range and intensity in the existing technology solutions, as well as the lack of multi-source data fusion and the performance bottleneck of pure convolution structure.

[0005] The technical solution adopted by the present invention to solve its technical problem is: A sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling includes the following steps: S1. Input the meteorological reanalysis data into the meteorological reanalysis data processing network, extract the meteorological reanalysis data features step by step through two-dimensional convolution and pooling operations, and extract the meteorological reanalysis data features at least at four scales; S2. Connect the initial satellite data with the meteorological reanalysis data to obtain the initial multi-source data, input the multi-source data into the multi-source data processing network, perform long-distance feature extraction through two-dimensional convolution, pooling operation, and VSS module with attention mechanism to obtain fused data features, and then connect the meteorological reanalysis data features of the same scale to continue long-distance feature extraction. Repeat the feature extraction and connection operations to obtain data features that can characterize sandstorms; S3. Upsample the obtained data features, expand the data feature size to be consistent with the input data, and finally pass it through a classifier to obtain the occurrence range or occurrence intensity of the sandstorm.

[0006] Further, in S1, the meteorological reanalysis data processing network includes four meteorological reanalysis data feature extraction units, each feature extraction unit is composed of two-dimensional convolution and pooling operations; In each feature extraction unit, the input features are sequentially subjected to two-dimensional convolution and pooling operations to obtain output features. After each feature extraction unit, the number of channels of the output features is twice that of the input features, and the length and width of the output features are halved, thereby achieving multi-scale feature extraction.

[0007] Furthermore, in S2, the multi-source data connection performs the following operations: In the multi-source data processing network, the initial satellite data features or the data features extracted from the upper layer are connected with the meteorological reanalysis data features of the same layer in the channel dimension to obtain the fused data features as input; the initial satellite data features or the data features extracted from the upper layer are in front, and the meteorological reanalysis data features of the same layer are in the back.

[0008] In S2, the multi-source data processing network includes five fused data feature extraction units, wherein the first two extraction units include VSS modules with attention mechanisms, two-dimensional convolution and pooling operations; the third to fourth feature extraction units include two-dimensional convolution and pooling operations; and the fifth feature extraction unit includes two-dimensional convolution operations. In the first two fused data feature extraction units, the fused data features are sequentially subjected to VSS modules with attention mechanisms, two-dimensional convolution and pooling operations; through the VSS module, the feature extraction unit can effectively capture long-distance feature relationships; through the attention mechanism, the feature extraction unit can focus on data features closely related to sandstorms; in the third to fourth feature extraction units, the fused data are sequentially subjected to two-dimensional convolution and pooling operations, and finally subjected to two-dimensional convolution of the fifth feature extraction unit for feature extraction to obtain sandstorm data features; for each feature extraction unit with pooling operations, the length and width of the output features are halved.

[0009] Preferably, the two-dimensional convolution performs the following operations: first, the input features are processed by a two-dimensional convolution operation and batch normalization; then, the features are nonlinearly transformed using the ReLU activation function; next, the features after the activation function are again subjected to a two-dimensional convolution operation and batch normalization; finally, the final features are output through the ReLU activation function.

[0010] Furthermore, in S3, the upsampling is performed based on bilinear interpolation: first, the spatial size of the feature is expanded using bilinear interpolation; then, the interpolated features are processed and standardized through two-dimensional convolution, and nonlinear changes are performed using the ReLU activation function; finally, the fused features of the same scale obtained by the multi-source data processing network are connected through jump connections and dense convolution layers; bilinear interpolation, two-dimensional convolution, ReLU activation, jump connections and dense convolution layers are repeated to gradually restore the feature size to the same as the input.

[0011] In S2, the VSS module with attention mechanism consists of a normalization operation, a linear layer, a depth-separable convolution, a SS2D module, a SILU activation function, a channel attention module, and a spatial attention module; The input features pass through two branches. Branch one goes through normalization operation, linear layer, depth-wise separable convolution, SS2D module, normalization operation, and linear layer in sequence to obtain the output; branch two goes through channel attention mechanism and spatial attention mechanism; the outputs of branch one and branch two are bitwise added to obtain the output features.

[0012] Preferably, the SS2D module performs the following operations: Scan expansion: The input is divided into several small blocks, and then expanded into a one-dimensional sequence along four different scan paths; S6 module processing: Each sequence is processed by a different S6 module; Scan merging: Merge the outputs of each S6 module to obtain the final output data features.

[0013] More preferably, the S6 module performs the following operations: linearly transform the input x to generate matrices A, B, and C, obtain matrices A and B by calculation, and use A and B to transform the hidden state h at the previous moment t-1 and the current input x t Weighted, get the current hidden state h t , use C and D to transform the hidden state and generate the output y t ; Concatenate the outputs of each time step into the final output sequence y.

[0014] The attention mechanism is a CBAM attention mechanism, including a channel attention mechanism module and a spatial attention mechanism module. First, the input features pass through the channel attention module. The channel attention module uses adaptive maximum pooling and average pooling to extract global features respectively, processes the pooling results through a shared fully connected layer, and uses the sigmoid activation function to normalize the results to obtain the final channel attention map; then, the features are input to the spatial attention module. The spatial attention module integrates channel information through maximum pooling and average pooling operations, splices the pooling results in the channel dimension, extracts spatial features through convolution operations, and uses the sigmoid activation function to generate an attention map of the spatial dimension; the final output result has channel and spatial attention, which enhances the overall model's attention to key features and important spatial positions.

[0015] In the present invention, the data required for detection include satellite data and meteorological reanalysis data related to sandstorms. Satellite data include satellite band data such as BT10.8 and BT12.0 of Fengyun-4A; meteorological reanalysis data include ERA5 reanalysis data, two-meter air temperature, ten-meter air temperature, etc. in MERRA-2 reanalysis data. The present invention adopts a multi-source data fusion method to overcome the influence of data with different spatiotemporal resolutions, so that data from different sources can have positive help for sandstorm detection. The VSS module with CBAM attention mechanism is used to enhance the global modeling ability of the model, and valuable information is obtained from complex multi-source data. Finally, the occurrence or intensity information of sandstorms is obtained. At the same time, transfer learning strategies, such as joint training strategy and transfer parameter strategy, are adopted to greatly improve the performance of sandstorm intensity detection.

[0016] The beneficial effects of the present invention are mainly manifested in: improving the performance of sandstorm intensity detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Flowchart of the multi-task sandstorm detection method based on multi-source data fusion and long-distance modeling.

[0018] Figure 2 Schematic diagram of the sandstorm detection network model structure.

[0019] Figure 3 Schematic diagram of the VSS module structure with CBAM attention mechanism.

[0020] Figure 4 Different training strategies for the sandstorm intensity detection model, where (a) represents the joint training strategy and (b) represents the migration parameter training strategy. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings.

[0022] Reference Figure 1~Figure 4 ,A multi-task sandstorm detection method based on multi-source data fusion and long-distance modeling, ,is constructed by training a sandstorm detection model, and then the trained ,detection network model is used to process multi-source meteorological data to obtain the ,real-time occurrence and intensity of sandstorms.

[0023] The sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling includes the following steps: Step S1: input the meteorological reanalysis data into the meteorological reanalysis data processing network, extract the meteorological reanalysis data features step by step through two-dimensional convolution and pooling operations, and extract meteorological reanalysis data features at least at four scales.

[0024] Reference Figure 2 The detection network model structure shown in FIG. 1 is as follows: Figure 2 As shown, it includes a feature extraction module, a feature upsampling module and a classifier, wherein the feature extraction module includes two branches, namely a meteorological reanalysis data processing network and a multi-source data processing network. The meteorological reanalysis data processing network is used for extracting features of meteorological reanalysis data, and the multi-source data processing network is used for fusing multi-source data and extracting features of fused data.

[0025] The meteorological reanalysis data processing network includes four meteorological reanalysis data feature extraction units, each of which consists of a two-dimensional convolution and pooling operation. The two-dimensional convolution includes a two-dimensional convolution operation, a batch normalization operation, and a ReLU activation function.

[0026] In this embodiment, in the meteorological reanalysis data processing network, through two-dimensional convolution and pooling operations, it is possible to extract the features of multi-scale meteorological reanalysis data, and finally obtain multi-scale meteorological reanalysis data features. The multi-scale meteorological reanalysis data features will be used for subsequent multi-source data fusion as one of the input data of the multi-source data processing network.

[0027] Specifically, the meteorological reanalysis data is subjected to two-dimensional convolution to obtain meteorological reanalysis data features, and the number of channels is doubled. Then, the features are subjected to the maximum pooling operation to reduce the length and width by half, and the meteorological reanalysis data feature extraction unit outputs a scale of meteorological reanalysis features, which are input to the next meteorological reanalysis data feature extraction unit to continue to extract another scale of meteorological reanalysis data features.

[0028] In this embodiment, the first meteorological reanalysis data feature extraction unit can obtain the first scale meteorological reanalysis data feature. When it is necessary to obtain meteorological reanalysis data features at multiple scales, multiple meteorological reanalysis data feature extraction units are required. Figure 2In order to obtain meteorological reanalysis data features of four scales, the first-scale meteorological reanalysis data features obtained by the first meteorological reanalysis data feature extraction unit are input into the second meteorological reanalysis data feature extraction unit again to obtain the second-scale meteorological reanalysis data features. Similarly, the second-scale meteorological reanalysis data features are input into the third meteorological reanalysis data feature extraction unit to obtain the third-scale meteorological reanalysis data features. The third-scale meteorological reanalysis data features are input into the fourth meteorological reanalysis data feature extraction unit to obtain the fourth-scale meteorological reanalysis data features. For situations where features of more scales are required, this can be achieved by adding meteorological reanalysis data feature extraction units, which will not be repeated here.

[0029] In a specific embodiment, the number of intermediate layer channels, the number of output channels, the convolution kernel size, the convolution step size and the padding width of the convolution layer of the first two-dimensional convolution module are 32, 32, 3, 1, and the convolution kernel size, the convolution step size and the padding width of the maximum pooling layer are 2, 2, 0; the number of intermediate layer channels, the number of output channels, the convolution kernel size, the convolution step size and the padding width of the convolution layer of the second two-dimensional convolution module are 64, 64, 3, 1, and the convolution kernel size, the convolution step size and the padding width of the maximum pooling layer are 2, 2, 0; the number of intermediate layer channels, number of output channels, convolution kernel size, convolution step and padding width of the convolution layer of the third two-dimensional convolution module are 128, 128, 3, 1, and the convolution kernel size, convolution step and padding width of the maximum pooling layer are 2, 2, 0; the number of intermediate layer channels, number of output channels, convolution kernel size, convolution step and padding width of the convolution layer of the fourth two-dimensional convolution module are 256, 256, 3, 1, and the convolution kernel size, convolution step and padding width of the maximum pooling layer are 2, 2, 0. From the input and output perspective, the original input data size is 448×896. After the first convolution layer and pooling operation, the scale of the obtained meteorological reanalysis data features is 224×448. After the second convolution layer and pooling operation, the scale of the obtained meteorological reanalysis data features is 112×224. After the third convolution layer and pooling operation, the scale of the obtained meteorological reanalysis data features is 56×112. After the fourth convolution layer and pooling operation, the scale of the obtained meteorological reanalysis data features is 28×56.

[0030] This embodiment extracts meteorological reanalysis data features step by step through two-dimensional convolution and pooling operations. Multi-scale meteorological reanalysis data features will be used for subsequent multi-source data fusion, laying a good foundation for subsequent feature fusion and processing.

[0031] It should be noted that, for simplicity, Figure 2 The four complete feature extraction units of meteorological reanalysis data are not listed, and “×2” is used to represent two feature extraction units.

[0032] Step S2: Connect the initial satellite data with the meteorological reanalysis data to obtain the initial multi-source data. Input the multi-source data into the multi-source data processing network, perform long-distance feature extraction through two-dimensional convolution, pooling operation, and VSS module with attention mechanism to obtain fused data features, and then connect the meteorological reanalysis data features of the same scale to continue long-distance feature extraction. Repeat the feature extraction and connection operations to obtain data features that can characterize sandstorms.

[0033] Reference Figure 2 As shown, the features of this embodiment are obtained by connecting the initial satellite data and the meteorological reanalysis data to obtain multi-source data input, and long-distance feature extraction and fusion are performed through a multi-source data processing network, and finally data features that can characterize sandstorms can be obtained.

[0034] Specifically, the initial meteorological reanalysis data features plus the meteorological analysis data features of four different scales that have been obtained have a total of five meteorological reanalysis data features. They are respectively input into the multi-source data processing network at the same layer, and connected to the initial satellite data features or the fusion features extracted from the upper layer of the multi-source data processing network. The connection method is the "cat" connection operation, which is connected in the channel dimension. The initial satellite data features or the fusion data features extracted from the upper layer are in front, and the meteorological reanalysis data features are in the back.

[0035] The multi-source data processing network includes five fused data feature extraction units. Among them, in the first two extraction units, the multi-source data features first pass through the VSS module with the CBAM attention mechanism, and then undergo two-dimensional convolution and pooling operations to obtain the fused features of the intermediate steps. In the third and fourth feature extraction units, the multi-source data features are further subjected to two-dimensional convolution and pooling operations to obtain fused data features. In the last feature extraction unit, data features that can characterize sandstorms are obtained through two-dimensional convolution as input to the subsequent upsampling module. At the same time, the features extracted by the first four feature extraction units through two-dimensional convolution will be input to the upsampling module through jump connections and dense convolution layers for subsequent upsampling operations.

[0036] Reference Figure 3In branch 1 of the VSS module with CBAM attention mechanism, the multi-source data feature input is enhanced by the channel attention module and the spatial attention module, respectively, to obtain the output of branch 1. In branch 2, the input feature is first processed by layer normalization linear layer, then depthwise separable convolution, SILU activation function activation, and input to the SS2D module. The output of the SS2D module is layer normalized and bitwise multiplied with the other branch of the depthwise separable convolution, and then passed through the linear layer to obtain the output of branch 2. The outputs of branch 1 and branch 2 are bitwise added to obtain the final fused feature output, at which time the feature contains attention information and long-distance modeling information.

[0037] The SS2D module includes the following steps: Scan expansion: Divide the input into several small blocks, and then expand them into a one-dimensional sequence along four different scanning paths. Specifically, the four scanning paths include horizontal scanning from the upper left corner to the lower right corner, vertical scanning from the upper left corner to the lower right corner, horizontal scanning from the lower right corner to the upper left corner, and vertical scanning from the lower right corner to the upper left corner. S6 module processing: Each sequence is processed by a different S6 module. For the S6 module, the input x is linearly transformed, the hidden state at the current moment is calculated, and the output y is calculated through the hidden state at the current moment. Scan merging: The outputs of each S6 module are merged to obtain the final output data features. At this time, each small block contains the information of other blocks, thereby realizing long-distance modeling.

[0038] In a specific embodiment, the number of characteristic channels of the initial satellite data is 6, and the number of characteristic channels of the meteorological reanalysis data is 12, so the number of characteristic channels of the fused data is 18. The number of characteristic channels of the processed data is 64, and the number of channels of the meteorological reanalysis data to be fused is 32, so the number of characteristic channels of the second fused data is 96. The number of characteristic channels of the processed data is 128, and the number of channels of the meteorological reanalysis data to be fused is 64, so the number of characteristic channels of the third fused data is 192. The number of characteristic channels of the processed data is 256, and the number of channels of the meteorological reanalysis data to be fused is 128, so the number of characteristic channels of the fourth fused data is 384. The number of characteristic channels of the processed data is 512, and the number of channels of the meteorological reanalysis data to be fused is 256, so the number of characteristic channels of the fifth fused data is 768.

[0039] The number of intermediate layer channels, the number of output channels, the convolution kernel size, the convolution step size, and the padding width of the convolution layer of the first two-dimensional convolution module are 64, 64, 3, and 1, and the convolution kernel size, the convolution step size, and the padding width of the maximum pooling layer are 2, 2, and 0; the number of intermediate layer channels, the number of output channels, the convolution kernel size, the convolution step size, and the padding width of the convolution layer of the second two-dimensional convolution module are 128, 128, 3, and 1, and the convolution kernel size, the convolution step size, and the padding width of the maximum pooling layer are 2, 2, and 0; The number of intermediate layer channels, number of output channels, convolution kernel size, convolution step and padding width of the convolution layers of the three 2D convolution modules are 256, 256, 3, 1, and the convolution kernel size, convolution step and padding width of the maximum pooling layer are 2, 2, 0; the number of intermediate layer channels, number of output channels, convolution kernel size, convolution step and padding width of the convolution layer of the fourth 2D convolution module are 512, 512, 3, 1, and the convolution kernel size, convolution step and padding width of the maximum pooling layer are 2, 2, 0. The number of intermediate layer channels, number of output channels, convolution kernel size, convolution step and padding width of the convolution layer of the fifth 2D convolution module are 1024, 1024, 3, 1. The hidden state of the second VSS module is 18, and the drop rate is 0.1; the hidden state of the first VSS module is 96, and the drop rate is 0.1.

[0040] By connecting the initial satellite data features or the fused data features extracted from the upper layer with the meteorological reanalysis data features at various scales, the advantages of multi-source data can be fully utilized, so that the model can extract features from different perspectives and then detect sandstorms. Through the attention mechanism, the sandstorm detection model can focus on the meteorological features closely related to sandstorms and possible occurrence areas; through the VSS module, the sandstorm detection model can perform long-distance modeling and take into account meteorological conditions at a distance. Combining the attention mechanism and long-distance modeling can effectively improve the performance of sandstorm detection.

[0041] It should be noted that Figure 2 For the sake of simplicity, the complete five fusion data feature extraction units are not listed, and “×2” is used to represent two feature extraction units.

[0042] Step S3: upsample the obtained fused data features, and expand the data feature size to be consistent with the input data. Finally, the occurrence range or occurrence intensity of the sandstorm is obtained through a classifier.

[0043] In the upsampling stage, this embodiment gradually expands the size of the fused features obtained by the multi-source data processing network through bilinear interpolation and two-dimensional convolution, and obtains the occurrence range or intensity of the sandstorm through the final classifier.

[0044] Specifically, refer to Figure 2, four upsamplings are required in the upsampling stage. For each upsampling layer, bilinear interpolation is first used to expand the spatial size of the feature. Then, after two-dimensional convolution, the interpolated features are processed and standardized, and nonlinear mapping is performed using the ReLU activation function. Finally, through jump connections and dense convolution layers, the fusion features of the same scale obtained by the multi-source data processing network are connected. The upsampling stage outputs feature data of the same size as the input feature, which is classified by the classifier and outputs the range or intensity of the sandstorm.

[0045] In a specific embodiment, the data feature dimension output by the multi-source data fusion network is 28×56. After the first upsampling, the data dimension is expanded to 56×1112; after the second upsampling, the data dimension is expanded to 112×224; after the third upsampling, the data dimension is expanded to 224×448; after the fourth upsampling, the data dimension is expanded to 448×896, which is consistent with the input data.

[0046] For the sandstorm occurrence detection task, the categories are "no sandstorm occurrence" and "sandstorm occurrence", so in the matrix output by the classifier, 0 represents no sandstorm occurrence and 1 represents sandstorm occurrence; for the sandstorm intensity detection task, the categories are "intensity 0 (no sandstorm occurrence)", "intensity 1", "intensity 2", "intensity 3", "intensity 4", "intensity 5", and "intensity 6", representing different levels of sandstorm intensity. Therefore, in the matrix output by the classifier, 0 represents no sandstorm occurrence, and 1-6 represent different levels of sandstorm intensity.

[0047] It should be noted that Figure 2 For simplicity, “×4” is used to represent 4 times upsampling.

[0048] The training process of the sandstorm detection network model of the present invention is as follows: First, the dataset is preprocessed. The dataset is divided into three parts: training set, validation set, and test set. The training set contains satellite data, meteorological reanalysis data, and sandstorm label data with a longitude and latitude range of 20.0N-58.2N and 38.9E-153.8E from March to May 2020-2021. The validation set and test set contain satellite data, meteorological reanalysis data, and sandstorm label data from March 2022 and April-May 2022, respectively. The dataset division ratio is approximately 7:1:2. The satellite data and meteorological reanalysis data are normalized to make the data more conducive to model training. In order to adapt to the model, all data sizes are 448×896.

[0049] While training: The convolutional layers and other parameters in the detection network model were initialized with random weights, and the model was optimized by the model optimizer AdamW with a dynamic decay learning rate of 0.0005, and trained for 200 cycles. The meteorological reanalysis data were input into the meteorological reanalysis data processing network, and the satellite data were input into the multi-source data processing network.

[0050] For sandstorm occurrence detection, the occurrence results of the model detection are compared with the actual situation to obtain the loss L1. The parameters of the detection network model are adjusted by minimizing the loss function through AdamW to optimize the performance of the occurrence detection model. For sandstorm intensity detection, the occurrence results of the model detection are compared with the actual situation to obtain the loss L2. The parameters of the detection network model are adjusted by minimizing the loss function through AdamW to optimize the performance of the intensity detection model.

[0051] For the strength detection model, refer to Figure 4 , there are two other training methods: Figure 4 (a) in the figure is a joint training strategy, that is, the model uses two outputs, the model randomly generates initialization parameters, and trains the occurrence and strength models at the same time. Figure 4 (b) in the figure is the migration parameter training strategy, which uses the backbone network parameters of the trained occurrence detection model as the initial backbone network parameters, randomly generates the parameters of the output layer, and trains the intensity detection model.

[0052] The detection network model is trained until convergence, and the best performance detection network model is obtained to detect the occurrence or intensity of sandstorms.

[0053] This application also provides experimental data under the data set for training the detection network model, and compares the technical solution of this application with traditional deep learning classification methods (U-net, U-net++, Attention U-net, Transunet, Swinunet and Vm-unet), and compares the performance indicators using Dice coefficient, intersection-over-union ratio Iou, accuracy Precision, recall rate Recall, and Kappa coefficient.

[0054] The comparison results are shown in Table 1 and Table 2: Table 1 Comparison results of sandstorm occurrence detection

[0055] Table 2 Comparison results of sandstorm intensity detection

[0056] Table 1 is the comparison result of the sandstorm occurrence detection task, and Table 2 is the comparison result of the sandstorm intensity detection task. From the comparison results of Table 1 and Table 2, it can be seen that the method of the present application is superior to all previous methods in all indicators. It should be noted that the joint training method has a significant effect on intensity detection, but not on occurrence detection, so it is not shown in Table 1.

[0057] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A multi-task sandstorm detection method based on multi-source data fusion and long-distance modeling, characterized in that: The method comprises the following steps: S1. Input the meteorological reanalysis data into the meteorological reanalysis data processing network, extract the meteorological reanalysis data features step by step through two-dimensional convolution and pooling operations, and extract the meteorological reanalysis data features at least at four scales; S2. Connect the initial satellite data with the meteorological reanalysis data to obtain the initial multi-source data, input the multi-source data into the multi-source data processing network, perform long-distance feature extraction through two-dimensional convolution, pooling operation, and VSS module with attention mechanism to obtain fused data features, then connect the meteorological reanalysis data features of the same scale, continue to perform long-distance feature extraction, repeat feature extraction and connection operations to obtain data features that can characterize sandstorms; S3. Upsample the obtained data features, expand the data feature size to be consistent with the input data, and finally pass it through a classifier to obtain the occurrence range or occurrence intensity of the sandstorm.

2. The sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling as claimed in claim 1 is characterized in that: In S1, the meteorological reanalysis data processing network includes four meteorological reanalysis data feature extraction units, each feature extraction unit is composed of two-dimensional convolution and pooling operations; In each feature extraction unit, the input features are sequentially subjected to two-dimensional convolution and pooling operations to obtain output features. After each feature extraction unit, the number of channels of the output features is twice that of the input features, and the length and width of the output features are halved, thereby achieving multi-scale feature extraction.

3. The sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling according to claim 1 or 2, characterized in that: In S2, the multi-source data connection performs the following operations: In the multi-source data processing network, the initial satellite data features or the data features extracted from the upper layer are connected with the meteorological reanalysis data features of the same layer in the channel dimension to obtain the fused data features as input; The initial satellite data features or the data features extracted from the upper layer are in the front, and the meteorological reanalysis data features of the same layer are in the back.

4. The sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling as claimed in claim 3 is characterized in that: In S2, the multi-source data processing network includes five fused data feature extraction units, wherein the first two extraction units include a VSS module with an attention mechanism, a two-dimensional convolution, and a pooling operation; the third to fourth feature extraction units include a two-dimensional convolution and a pooling operation; the fifth feature extraction unit includes a two-dimensional convolution operation; in the first two fused data feature extraction units, the fused data features are sequentially subjected to a VSS module with an attention mechanism, a two-dimensional convolution, and a pooling operation; through the VSS module, the feature extraction unit can effectively capture long-distance feature relationships; Through the attention mechanism, the feature extraction unit can focus on data features that are closely related to sandstorms; in the third to fourth feature extraction units, the fused data undergoes two-dimensional convolution and pooling operations in turn, and finally undergoes feature extraction through two-dimensional convolution of the fifth feature extraction unit to obtain sandstorm data features; for each feature extraction unit with pooling operation, the length and width of the output feature are halved.

5. The sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling as claimed in claim 4 is characterized in that: The two-dimensional convolution performs the following operations: first, the input features are processed by a two-dimensional convolution operation and batch normalization; then, the ReLU activation function is used to perform a nonlinear transformation on the features; next, the features after the activation function are again subjected to a two-dimensional convolution operation and batch normalization; finally, the final features are output through the ReLU activation function.

6. The sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling according to claim 1 or 2, characterized in that: In S3, the upsampling is performed based on bilinear interpolation: first, the spatial size of the feature is expanded using bilinear interpolation; then, the interpolated features are processed and standardized through two-dimensional convolution, and nonlinear changes are performed using the ReLU activation function; finally, the fused features of the same scale obtained by the multi-source data processing network are connected through jump connections and dense convolution layers; bilinear interpolation, two-dimensional convolution, ReLU activation, jump connections and dense convolution layers are repeated to gradually restore the feature size to the same as the input.

7. The sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling as claimed in claim 4 is characterized in that: In S2, the VSS module with attention mechanism consists of a normalization operation, a linear layer, a depth-separable convolution, a SS2D module, a SILU activation function, a channel attention module, and a spatial attention module; The input features pass through two branches. Branch one goes through normalization operation, linear layer, depth-wise separable convolution, SS2D module, normalization operation, and linear layer in sequence to obtain the output; branch two goes through channel attention mechanism and spatial attention mechanism; the outputs of branch one and branch two are bitwise added to obtain the output features.

8. The sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling as claimed in claim 7, characterized in that: The SS2D module performs the following operations: Scan expansion: The input is divided into several small blocks, and then expanded into a one-dimensional sequence along four different scan paths; S6 module processing: Each sequence is processed by a different S6 module; Scan merging: Merge the outputs of each S6 module to obtain the final output data features.

9. The sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling as claimed in claim 8, characterized in that: The S6 module performs the following operations: linearly transform the input x to generate matrices A, B, and C, obtain matrices A and B through calculation, and use A and B to transform the hidden state h at the previous moment t-1 and the current input x t Weighted, get the current hidden state h t , use C and D to transform the hidden state and generate the output y t ; Concatenate the outputs of each time step into the final output sequence y.

10. The sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling according to claim 7, characterized in that: The attention mechanism is a CBAM attention mechanism, including a channel attention mechanism module and a spatial attention mechanism module. First, the input features pass through the channel attention module, which uses adaptive maximum pooling and average pooling to extract global features respectively, processes the pooling results through a shared fully connected layer, and uses a sigmoid activation function to normalize the results to obtain the final channel attention map; Then, the features are input into the spatial attention module, which integrates channel information through maximum pooling and average pooling operations, concatenates the pooling results in the channel dimension, extracts spatial features through convolution operations, and uses the sigmoid activation function to generate an attention map in the spatial dimension; the final output result has channel and spatial attention, which enhances the overall model's attention to key features and important spatial positions.

Images