Multi-task Detection Method for Sandstorms Based on Multi-source Data Fusion and Long-distance Modeling

Abstract

A multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling, belonging to the field of sandstorm detection, includes the following steps: S1. Input meteorological reanalysis data into a meteorological reanalysis data processing network, and gradually extract meteorological reanalysis data features through two-dimensional convolution and pooling operations, extracting at least four scales of meteorological reanalysis data features; S2. Connect the initial satellite data with the meteorological reanalysis data to obtain initial multi-source data, input the multi-source data into a multi-source data processing network, perform long-distance feature extraction to obtain fused data features, then connect the meteorological reanalysis data features of the same scale, and continue to perform long-distance feature extraction to obtain data features that can characterize sandstorms; S3. Upsample the obtained data features, expand the data feature size to be consistent with the input data, and finally obtain the occurrence range or occurrence intensity of sandstorms through a classifier. The present invention improves the performance of sandstorm intensity detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of real-time detection of sandstorms, and particularly relates to a multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling. Background Art

[0002] As one of the natural disasters in arid and semi-arid regions, sandstorms mainly occur in the northwestern regions of China, including the southern Xinjiang Basin, southwestern Qinghai, etc. The occurrence of sandstorms is affected by various factors, including temperature, wind speed, wind direction, etc. Sandstorm detection is to use meteorological data including satellite data to detect the occurrence of sandstorms in real time. Since strong sandstorms will cause huge damage to people's production and life, in order to prevent such weather disasters, it is very necessary to detect the occurrence range and intensity of sandstorms in real time.

[0003] Generally speaking, traditional methods focus on establishing physical models. However, due to the complexity of weather systems, it is difficult for physical models to consider all influencing factors. Therefore, with the development of artificial intelligence technology, many studies have proposed various deep learning methods to achieve sandstorm detection. At present, most deep learning-based methods tend to detect the occurrence range of sandstorms, ignoring the intensity detection of sandstorms. In addition, from a technical perspective, deep learning-based research often relies on a pure convolutional architecture, making it difficult to overcome the inherent locality problem of convolution. At the same time, these studies focus on a single data source and lack exploration of using multi-source data to detect sandstorms. These problems all lead to bottlenecks in detection performance. Summary of the Invention

[0004] In order to overcome the deficiencies of the prior art, the present invention provides a multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling, which can improve the performance of sandstorm intensity detection. By multi-source data fusion and long-distance modeling, it solves the problems existing in the prior art solutions, such as the lack of simultaneous detection of the occurrence range and intensity of sandstorms, as well as the lack of multi-source data fusion and the performance bottleneck of the pure convolutional structure.

[0005] The technical solution adopted by the present invention to solve its technical problems is:

[0006] A multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling, comprising the following steps:

[0007] S1. Input meteorological reanalysis data into a meteorological reanalysis data processing network, and gradually extract meteorological reanalysis data features through two-dimensional convolution and pooling operations, extracting at least four scales of meteorological reanalysis data features;

[0008] S2. Connect the initial satellite data with the meteorological reanalysis data to obtain the initial multi-source data. Input the multi-source data into the multi-source data processing network, and perform long-distance feature extraction through two-dimensional convolution, pooling operations, and the VSS module with an attention mechanism to obtain the fused data features. Then connect the meteorological reanalysis data features at the same scale and continue with long-distance feature extraction. Repeat the feature extraction and connection operations to obtain the data features that can represent sandstorms.

[0009] S3. Upsample the obtained data features so that the data feature size is expanded to be the same as the input data. Finally, pass through the classifier to obtain the occurrence range or occurrence intensity of sandstorms.

[0010] Furthermore, in S1, the meteorological reanalysis data processing network includes four meteorological reanalysis data feature extraction units, and each feature extraction unit consists of two-dimensional convolution and pooling operations.

[0011] In each feature extraction unit, the input features are sequentially passed through two-dimensional convolution and pooling operations to obtain the output features; for each feature extraction unit passed through, the number of channels of the output features is twice that of the input features, and the length and width dimensions of the output features are reduced by half, thereby realizing multi-scale feature extraction.

[0012] Still further, in S2, the multi-source data connection performs the following operations:

[0013] In the multi-source data processing network, connect the initial satellite data features or the data features extracted from the upper layer with the meteorological reanalysis data features at the same layer in the channel dimension to obtain the fused data features as the input; where the initial satellite data features or the data features extracted from the upper layer are in the front and the meteorological reanalysis data features at the same layer are in the back.

[0014] In S2, the multi-source data processing network includes five fused data feature extraction units. The first two extraction units include the VSS module with an attention mechanism, two-dimensional convolution, and pooling operations; the third to fourth feature extraction units include two-dimensional convolution and pooling operations; the fifth feature extraction unit includes two-dimensional convolution operations. In the first two fused data feature extraction units, the fused data features are sequentially passed through the VSS module with an attention mechanism, two-dimensional convolution, and pooling operations; through the VSS module, the feature extraction unit can effectively capture long-distance feature relationships; through the attention mechanism, the feature extraction unit can focus on the data features closely related to sandstorms; in the third to fourth feature extraction units, the fused data is sequentially passed through two-dimensional convolution and pooling operations, and finally through the two-dimensional convolution of the fifth feature extraction unit for feature extraction to obtain the sandstorm data features; for each feature extraction unit with a pooling operation, the length and width dimensions of the output features are reduced by half.

[0015] Preferably, the two-dimensional convolution performs the following operations: First, the input features are processed through a two-dimensional convolution operation and batch normalization; then, the ReLU activation function is used to perform a non-linear transformation on the features; immediately afterwards, the features after passing through the activation function are again processed through a two-dimensional convolution operation and batch normalization; finally, the final features are output through the ReLU activation function.

[0016] Furthermore, in S3, the upsampling is based on bilinear interpolation: First, bilinear interpolation is used to expand the spatial dimensions of the features; then, the interpolated features are processed and normalized through a two-dimensional convolution, and the ReLU activation function is used for non-linear transformation; finally, through skip connections and a dense convolutional layer, the fused features of the same scale obtained by the multi-source data processing network are connected; bilinear interpolation, two-dimensional convolution, ReLU activation, skip connections, and dense convolutional layers are repeated to gradually restore the feature size to the same as the input.

[0017] In S2, the VSS module with an attention mechanism consists of a normalization operation, a linear layer, a depthwise separable convolution, an SS2D module, a SiLU activation function, a channel attention module, and a spatial attention module;

[0018] The input features pass through two branches. Branch one sequentially passes through a normalization operation, a linear layer, a depthwise separable convolution, an SS2D module, a normalization operation, and a linear layer to obtain an output; branch two passes through a channel attention mechanism and a spatial attention mechanism; the outputs of branch one and branch two are added bitwise to obtain the output features.

[0019] Preferably, the SS2D module performs the following operations:

[0020] Scan expansion: The input is divided into several small blocks, and then along four different scan paths, it is expanded into a one-dimensional sequence;

[0021] S6 module processing: Each sequence is respectively processed through different S6 modules;

[0022] Scan merging: The outputs of each S6 module are merged to obtain the final output data features.

[0023] More preferably, the S6 module performs the following operations: perform a linear transformation on the input x to generate matrices B and C, calculate matrices A and B, and use A and B to weight the hidden state h at the previous time step t-1 and the current input x t to obtain the current hidden state h t , use C and D to transform the hidden state to generate the output y t ; the outputs of each time step are concatenated into the final output sequence y.

[0024] The attention mechanism is the CBAM attention mechanism, which includes a channel attention mechanism module and a spatial attention mechanism module. First, the input features pass through the channel attention module. The channel attention module uses adaptive max-pooling and average pooling to extract global features respectively, processes the pooling results through a shared fully-connected layer, and normalizes the results using the sigmoid activation function to obtain the final channel attention map. Then, the features are input into the spatial attention module. The spatial attention module integrates channel information through max-pooling and average pooling operations, concatenates the pooling results in the channel dimension, and extracts spatial features through convolutional operations, and generates an attention map in the spatial dimension using the sigmoid activation function. The final output result has channel and spatial attention, enhancing the overall model's attention to key features and important spatial positions.

[0025] In the present invention, the data required for detection includes satellite data related to sandstorms and meteorological reanalysis data. The satellite data includes satellite band data such as BT10.8 and BT12.0 of Fengyun 4A; the meteorological reanalysis data includes ERA5 reanalysis data, two-meter air temperature, ten-meter air temperature, etc. in MERRA-2 reanalysis data. The present invention adopts a multi-source data fusion method to overcome the influence of data with different spatio-temporal resolutions, enabling data from different sources to have a positive effect on sandstorm detection. It uses the VSS module with the CBAM attention mechanism to enhance the model's global modeling ability, obtains valuable information from complex multi-source data, and finally obtains the occurrence or intensity information of sandstorms. At the same time, it adopts transfer learning strategies, such as joint training strategies and transfer parameter strategies, which greatly improve the performance of sandstorm intensity detection.

[0026] The beneficial effects of the present invention are mainly manifested in: improving the performance of sandstorm intensity detection. Brief Description of the Drawings

[0027] Figure 1 It is a flowchart of a sandstorm multi-task detection method based on multi-source data fusion and long-distance modeling.

[0028] Figure 2 It is a schematic diagram of the structure of a sandstorm detection network model.

[0029] Figure 3 It is a schematic diagram of the structure of the VSS module with the CBAM attention mechanism.

[0030] Figure 4 It is different training strategies of the sandstorm intensity detection model, where (a) represents the joint training strategy and (b) represents the transfer parameter training strategy. Detailed Embodiments

[0031] The present invention will be further described below with reference to the accompanying drawings.

[0032] Reference Figures 1 to 4 , a multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling. By training the constructed sandstorm detection model, and then using the trained detection network model to process multi-source meteorological data to obtain the occurrence and intensity of real-time sandstorms.

[0033] The multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling includes the following steps:

[0034] Step S1: Input meteorological reanalysis data into the meteorological reanalysis data processing network, and extract meteorological reanalysis data features step by step through two-dimensional convolution and pooling operations, extracting at least four scales of meteorological reanalysis data features.

[0035] Reference Figure 2 The detection network model structure shown. The detection network model constructed in this embodiment is as Figure 2 shown, including a feature extraction module, a feature upsampling module and a classifier. The feature extraction module includes two branches, namely the meteorological reanalysis data processing network and the multi-source data processing network. The meteorological reanalysis data processing network is used for extracting meteorological reanalysis data features, and the multi-source data processing network is used for fusing multi-source data and extracting fused data features.

[0036] The meteorological reanalysis data processing network includes four meteorological reanalysis data feature extraction units, and each feature extraction unit consists of two-dimensional convolution and pooling operations. The two-dimensional convolution includes two-dimensional convolution operation, batch normalization operation, and ReLU activation function.

[0037] In this embodiment, in the meteorological reanalysis data processing network, through two-dimensional convolution and pooling operations, the feature extraction of multi-scale meteorological reanalysis data can be realized, and finally multi-scale meteorological reanalysis data features are obtained. The multi-scale meteorological reanalysis data features will be used for subsequent multi-source data fusion as one of the input data of the multi-source data processing network.

[0038] Specifically, the meteorological reanalysis data passes through two-dimensional convolution to obtain meteorological reanalysis data features, and at this time the number of channels doubles. Then, the features pass through the max-pooling operation to reduce the length and width dimensions by half, obtaining one scale of meteorological reanalysis features output by the meteorological reanalysis data feature extraction unit, and inputting it into the next meteorological reanalysis data feature extraction unit to continue extracting another scale of meteorological reanalysis data features.

[0039] In this embodiment, through the first meteorological reanalysis data feature extraction unit, the first scale of meteorological reanalysis data features can be obtained. When multiple scales of meteorological reanalysis data features are required, multiple meteorological reanalysis data feature extraction units are needed. For example, in Figure 2In [the above], in order to obtain the meteorological reanalysis data features of four scales, the first-scale meteorological reanalysis data features obtained by the first meteorological reanalysis data feature extraction unit are input into the second meteorological reanalysis data feature extraction unit again to obtain the second-scale meteorological reanalysis data features. Similarly, the second-scale meteorological reanalysis data features are input into the third meteorological reanalysis data feature extraction unit to obtain the third-scale meteorological reanalysis data features. The third-scale meteorological reanalysis data features are input into the fourth meteorological reanalysis data feature extraction unit to obtain the fourth-scale meteorological reanalysis data features. For the case where more scale features are required, it can be achieved by adding meteorological reanalysis data feature extraction units, which will not be elaborated here.

[0040] In a specific embodiment, the number of intermediate layer channels, output channels, convolution kernel size, convolution stride, and padding width of the convolution layer of the first two-dimensional convolution module are 32, 32, 3, 1, respectively, and the convolution kernel size, convolution stride, and padding width of the max pooling layer are 2, 2, 0; the number of intermediate layer channels, output channels, convolution kernel size, convolution stride, and padding width of the convolution layer of the second two-dimensional convolution module are 64, 64, 3, 1, respectively, and the convolution kernel size, convolution stride, and padding width of the max pooling layer are 2, 2, 0; the number of intermediate layer channels, output channels, convolution kernel size, convolution stride, and padding width of the convolution layer of the third two-dimensional convolution module are 128, 128, 3, 1, respectively, and the convolution kernel size, convolution stride, and padding width of the max pooling layer are 2, 2, 0; the number of intermediate layer channels, output channels, convolution kernel size, convolution stride, and padding width of the convolution layer of the fourth two-dimensional convolution module are 256, 256, 3, 1, respectively, and the convolution kernel size, convolution stride, and padding width of the max pooling layer are 2, 2, 0. From the perspective of input and output, the size of the original input data is 448×896. After the first convolution layer and pooling operation, the scale of the meteorological reanalysis data features obtained is 224×448. After the second convolution layer and pooling operation, the scale of the meteorological reanalysis data features obtained is 112×224. After the third convolution layer and pooling operation, the scale of the meteorological reanalysis data features obtained is 56×112. After the fourth convolution layer and pooling operation, the scale of the meteorological reanalysis data features obtained is 28×56.

[0041] In this embodiment, the meteorological reanalysis data is used to extract the meteorological reanalysis data features step by step through two-dimensional convolution and pooling operations. The multi-scale meteorological reanalysis data features will be used for subsequent multi-source data fusion, laying a good foundation for subsequent feature fusion and processing.

[0042] It should be noted that, for simplicity, Figure 2 the complete four meteorological reanalysis data feature extraction units are not listed, and "×2" is used to represent two feature extraction units.

[0043] Step S2: Connect the initial satellite data with the meteorological reanalysis data to obtain the initial multi-source data. Input the multi-source data into the multi-source data processing network, and perform long-distance feature extraction through two-dimensional convolution, pooling operations, and the VSS module with an attention mechanism to obtain fused data features. Then connect the meteorological reanalysis data features at the same scale and continue long-distance feature extraction. Repeat the feature extraction and connection operations to obtain data features that can represent sandstorms.

[0044] Referring to Figure 2 As shown, the features of this embodiment connect the initial satellite data and the meteorological reanalysis data to obtain multi-source data input, and perform long-distance feature extraction and fusion through the multi-source data processing network. Finally, data features that can represent sandstorms can be obtained.

[0045] Specifically, the initial meteorological reanalysis data features plus the four different-scale meteorological analysis data features that have been obtained, there are a total of five meteorological reanalysis data features. They are respectively input into the multi-source data processing network at the same layer, and the initial satellite data features or the fused features extracted from the previous layer of the multi-source data processing network are connected. The connection method is the "cat" connection operation, which is connected in the channel dimension. The initial satellite data features or the fused data features extracted from the upper layer are in front, and the meteorological reanalysis data features are behind.

[0046] The multi-source data processing network includes five fused data feature extraction units. Among them, in the first two extraction units, the multi-source data features first pass through the VSS module with the CBAM attention mechanism, and then through two-dimensional convolution and pooling operations to obtain the fused features in the intermediate steps. In the third and fourth feature extraction units, the multi-source data features are further processed through two-dimensional convolution and pooling operations to obtain the fused data features. In the last feature extraction unit, the data features that can represent sandstorms are obtained through two-dimensional convolution and used as the input of the subsequent upsampling module. At the same time, the features extracted by two-dimensional convolution in the first four feature extraction units will be input into the upsampling module through skip connections and dense convolutional layers for subsequent upsampling operations.

[0047] Referring to Figure 3, in Branch 1 of the VSS module with the CBAM attention mechanism, the multi-source data features are input through the channel attention module and the spatial attention module to enhance the attention of the data in the channel dimension and the spatial dimension respectively, obtaining the output of Branch 1. In Branch 2, the input features are first processed by the layer normalization linear layer, then subjected to depthwise separable convolution and activated by the SILU activation function, and input into the SS2D module. The output of the SS2D module is normalized by the layer and multiplied bitwise with another branch of the depthwise separable convolution, and the output of Branch 2 is obtained through the linear layer. The outputs of Branch 1 and Branch 2 are added bitwise to obtain the final fused feature output. At this time, the features contain attention information and long-range modeling information.

[0048] The SS2D module includes the following steps: Scan expansion: The input is divided into several small blocks and then expanded into a one-dimensional sequence along four different scan paths. Specifically, the four scan paths include horizontal scanning from the upper left corner to the lower right corner, vertical scanning from the upper left corner to the lower right corner, horizontal scanning from the lower right corner to the upper left corner, and vertical scanning from the lower right corner to the upper left corner. S6 module processing: Each sequence is processed by a different S6 module. For the S6 module, a linear transformation is performed on the input x to calculate the hidden state at the current moment, and the output y is calculated through the hidden state at the current moment. Scan merging: The outputs of each S6 module are merged to obtain the final output data features. At this time, each small block contains information of other blocks, thus realizing long-range modeling.

[0049] In a specific embodiment, the number of channels of the initial satellite data features is 6, and the number of channels of the meteorological reanalysis data features is 12. Therefore, the number of channels of the fused data features is 18. The number of channels of the processed data features is 64, and the number of channels of the meteorological reanalysis data to be fused is 32. Therefore, the number of channels of the second fused data features is 96. The number of channels of the processed data features is 128, and the number of channels of the meteorological reanalysis data to be fused is 64. Therefore, the number of channels of the third fused data features is 192. The number of channels of the processed data features is 256, and the number of channels of the meteorological reanalysis data to be fused is 128. Therefore, the number of channels of the fourth fused data features is 384. The number of channels of the processed data features is 512, and the number of channels of the meteorological reanalysis data to be fused is 256. Therefore, the number of channels of the fifth fused data features is 768.

[0050] The number of intermediate layer channels, output channels, convolution kernel size, convolution stride, and padding width of the convolution layer of the first two-dimensional convolution module are 64, 64, 3, 1, respectively, and the convolution kernel size, convolution stride, and padding width of the max pooling layer are 2, 2, 0; the number of intermediate layer channels, output channels, convolution kernel size, convolution stride, and padding width of the convolution layer of the second two-dimensional convolution module are 128, 128, 3, 1, respectively, and the convolution kernel size, convolution stride, and padding width of the max pooling layer are 2, 2, 0; the number of intermediate layer channels, output channels, convolution kernel size, convolution stride, and padding width of the convolution layer of the third two-dimensional convolution module are 256, 256, 3, 1, respectively, and the convolution kernel size, convolution stride, and padding width of the max pooling layer are 2, 2, 0; the number of intermediate layer channels, output channels, convolution kernel size, convolution stride, and padding width of the convolution layer of the fourth two-dimensional convolution module are 512, 512, 3, 1, respectively, and the convolution kernel size, convolution stride, and padding width of the max pooling layer are 2, 2, 0. The number of intermediate layer channels, output channels, convolution kernel size, convolution stride, and padding width of the convolution layer of the fifth two-dimensional convolution module are 1024, 1024, 3, 1. The hidden state of the second VSS module is 18, and the dropout rate is 0.1; the hidden state of the first VSS module is 96, and the dropout rate is 0.1.

[0051] By connecting the initial satellite data features or the fused data features extracted from the upper layer with the meteorological reanalysis data features at various scales, the advantages of multi-source data can be fully utilized, enabling the model to extract features from different perspectives and then detect sandstorms. Through the attention mechanism, the sandstorm detection model can focus on the meteorological features closely related to sandstorms and the possible occurrence areas; through the VSS module, the sandstorm detection model can perform long-distance modeling and take into account the meteorological conditions in more distant areas. Combining the attention mechanism and long-distance modeling can effectively improve the performance of sand and dust detection.

[0052] It should be noted that Figure 2 For simplicity, the complete five fused data feature extraction units are not listed, and "×2" is used to represent two feature extraction units.

[0053] Step S3: Upsample the obtained fused data features to expand the data feature size to be the same as the input data. Finally, through the classifier, the occurrence range or occurrence intensity of sandstorms is obtained.

[0054] In this embodiment, in the upsampling stage, the fused feature size obtained by the multi-source data processing network is gradually enlarged through bilinear interpolation and two-dimensional convolution, and the occurrence range or intensity of sandstorms is obtained through the final classifier.

[0055] Specifically, refer to Figure 2, four upsamplings are required in the upsampling stage. For each upsampling layer, first, bilinear interpolation is used to expand the spatial dimensions of the features. Then, through two-dimensional convolution, the interpolated features are processed and normalized, and the ReLU activation function is used for non-linear mapping. Finally, through skip connections and dense convolutional layers, the fused features of the same scale obtained by the multi-source data processing network are connected. The upsampling stage outputs feature data with the same size as the input features, which are classified by a classifier to output the occurrence range or intensity of sandstorms.

[0056] In a specific embodiment, the data feature dimension output by the multi-source data fusion network is 28×56. After the first upsampling, the data dimension is expanded to 56×1112; after the second upsampling, the data dimension is expanded to 112×224; after the third upsampling, the data dimension is expanded to 224×448; after the fourth upsampling, the data dimension is expanded to 448×896, which is the same as the input data.

[0057] For the sandstorm occurrence detection task, the categories are "no sandstorm occurrence" and "sandstorm occurrence". Therefore, in the matrix output by the classifier, 0 represents no sandstorm occurrence, and 1 represents sandstorm occurrence; for the sandstorm intensity detection task, the categories are "intensity 0 (no sandstorm occurrence)", "intensity 1", "intensity 2", "intensity 3", "intensity 4", "intensity 5", "intensity 6", representing different levels of sandstorm intensity respectively. Therefore, in the matrix output by the classifier, 0 represents no sandstorm occurrence, and 1-6 represent different levels of sandstorm intensity respectively.

[0058] It should be noted that Figure 2 for simplicity, "×4" is used to represent 4 upsamplings.

[0059] For the sandstorm detection network model of the present invention, its training process is as follows:

[0060] First, perform dataset preprocessing. The dataset is divided into three parts: training set, validation set, and test set. The training set contains satellite data, meteorological reanalysis data, and sandstorm label data with longitude and latitude ranges of 20.0N-58.2N, 38.9E-153.8E from March to May in 2020-2021. The validation set and test set respectively contain satellite data, meteorological reanalysis data, and sandstorm label data in March 2022 and from April to May 2022. The dataset division ratio is approximately 7:1:2. The satellite data and meteorological reanalysis data are normalized to make the data more conducive to model training. To adapt to the model, all data sizes are 448×896.

[0061] During training:

[0062] Initialize the parameters of the convolutional layer in the detection network model with random weights, and optimize the model using the AdamW optimizer with a dynamic decay learning rate of 0.0005 for 200 epochs. Input the meteorological reanalysis data into the meteorological reanalysis data processing network and input the satellite data into the multi-source data processing network.

[0063] For dust storm occurrence detection, the occurrence results detected by the model are compared with the actual situation to obtain the loss L1, and the parameters of the detection network model are adjusted by minimizing the loss function with AdamW to optimize the performance of the occurrence detection model; for dust storm intensity detection, the occurrence results detected by the model are compared with the actual situation to obtain the loss L2, and the parameters of the detection network model are adjusted by minimizing the loss function with AdamW to optimize the performance of the intensity detection model.

[0064] For the intensity detection model, referring to Figure 4 , two other training methods can also be adopted: Figure 4 In (a) of Figure 4 is the joint training strategy, that is, the model uses two outputs, randomly generates initialization parameters, and trains the occurrence and intensity models simultaneously.

[0065] Train the detection network model until convergence, and obtain the best performance detection network model for dust storm occurrence or intensity detection.

[0066] This application also provides experimental data under the dataset for training the detection network model. Compare the technical solution of this application with traditional deep learning classification methods (U-net, U-net++, Attention U-net, Transunet, Swinunet, and Vm-unet), and the performance metrics used are the Dice coefficient, intersection over union Iou, precision, recall, and Kappa coefficient.

[0067] The comparison results are shown in Tables 1 and 2:

[0068] Table 1 Comparison results of dust storm occurrence detection

[0069]

[0070] Table 2 Comparison results of dust storm intensity detection

[0071]

[0072] Table 1 above shows the comparison results of the sandstorm occurrence detection task, and Table 2 shows the comparison results of the sandstorm intensity detection task. From the comparison results in Table 1 and Table 2, it can be seen that the method of this application is superior to all previous methods in all indicators. It should be noted that the joint training method has obvious effects on intensity detection, but not obvious effects on occurrence detection, so it is not shown in Table 1.

[0073] The content described in the embodiments of this specification is only an enumeration of the implementation forms of the inventive concept and is only for illustrative purposes. The protection scope of the present invention should not be regarded as limited to the specific forms stated in this embodiment. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those of ordinary skill in the art based on the inventive concept.

Claims

1. A multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling, characterized in that, The method includes the following steps: S1. Input the meteorological reanalysis data into the meteorological reanalysis data processing network, and extract the meteorological reanalysis data features step by step through two-dimensional convolution and pooling operations, extracting at least four scales of meteorological reanalysis data features; S2. Connect the initial satellite data with the meteorological reanalysis data to obtain the initial multi-source data, input the multi-source data into the multi-source data processing network, perform long-distance feature extraction through two-dimensional convolution, pooling operations, and the VSS module with an attention mechanism to obtain the fused data features, then connect the meteorological reanalysis data features of the same scale, continue long-distance feature extraction, and repeat the feature extraction and connection operations to obtain the data features that can characterize sandstorms; S3. Upsample the obtained data features, expand the data feature size to be consistent with the input data, and finally pass through a classifier to obtain the occurrence range or occurrence intensity of sandstorms; In the above S2, the multi-source data processing network includes five fused data feature extraction units. The first two extraction units include the VSS module with an attention mechanism, two-dimensional convolution, and pooling operations; the third to fourth feature extraction units include two-dimensional convolution and pooling operations; the fifth feature extraction unit includes two-dimensional convolution operations. In the first two fused data feature extraction units, the fused data features sequentially pass through the VSS module with an attention mechanism, two-dimensional convolution, and pooling operations; through the VSS module, the feature extraction unit can effectively capture long-distance feature relationships; through the attention mechanism, the feature extraction unit can focus on the data features closely related to sandstorms; in the third to fourth feature extraction units, the fused data sequentially passes through two-dimensional convolution and pooling operations, and finally passes through the two-dimensional convolution of the fifth feature extraction unit for feature extraction to obtain the sandstorm data features; for each feature extraction unit with a pooling operation, the length and width dimensions of the output feature are reduced by half; In the above S2, the VSS module with an attention mechanism consists of a normalization operation, a linear layer, a depthwise separable convolution, an SS2D module, a SILU activation function, a channel attention module, and a spatial attention module; The input feature passes through two branches. Branch one sequentially passes through a normalization operation, a linear layer, a depthwise separable convolution, an SS2D module, a normalization operation, and a linear layer to obtain an output; branch two passes through a channel attention mechanism and a spatial attention mechanism; the outputs of branch one and branch two are added bitwise to obtain the output feature.

2. The multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling according to claim 1, characterized in that In the above S1, the meteorological reanalysis data processing network includes four meteorological reanalysis data feature extraction units, and each feature extraction unit consists of two-dimensional convolution and pooling operations; In each feature extraction unit, the input feature sequentially passes through two-dimensional convolution and pooling operations to obtain the output feature; for each feature extraction unit passed through, the number of channels of the output feature is twice that of the input feature, and the length and width dimensions of the output feature are reduced by half, thereby realizing multi-scale feature extraction.

3. The multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling according to claim 1 or 2, characterized in that In the above S2, the multi-source data connection performs the following operations: In a multi-source data processing network, the initial satellite data features or the data features extracted from the upper layer are concatenated with the meteorological reanalysis data features of the same layer in the channel dimension to obtain the fused data features as the input; wherein the initial satellite data features or the data features extracted from the upper layer are in the front, and the meteorological reanalysis data features of the same layer are in the back.

4. The multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling according to claim 1, wherein, The two-dimensional convolution performs the following operations: First, the input features are processed through a two-dimensional convolution operation and batch normalization; then, the ReLU activation function is used to perform a non-linear transformation on the features; immediately afterwards, the features after the activation function are again processed through a two-dimensional convolution operation and batch normalization; finally, the final features are output through the ReLU activation function.

5. The multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling according to claim 1 or 2, characterized in that, In S3, the upsampling is based on bilinear interpolation: First, the features are enlarged in spatial size using bilinear interpolation; then, the interpolated features are processed and normalized through two-dimensional convolution, and the ReLU activation function is used for non-linear transformation; finally, through skip connections and dense convolutional layers, the fused features of the same scale obtained by the multi-source data processing network are connected; the bilinear interpolation, two-dimensional convolution, ReLU activation, skip connections and dense convolutional layers are repeated to gradually restore the feature size to the same as the input.

6. The multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling according to claim 1, characterized in that The SS2D module performs the following operations: Scan expansion: The input is divided into several small blocks, and then expanded into a one-dimensional sequence along four different scan paths; S6 module processing: Each sequence is processed through a different S6 module respectively; Scan merging: The outputs of each S6 module are merged to obtain the final output data features.

7. The multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling according to claim 6, characterized in that The S6 module performs the following operations: linearly transform the input x to generate matrices B and C, calculate matrices A and B, weight the previous hidden state h t-1 and the current input x t to obtain the current hidden state h t , transform the hidden state using C and D to generate the output y t ; concatenate the outputs of each time step into the final output sequence y.

8. The multi-task detection method for sandstorms based on multi-source data fusion and long-distance modeling according to claim 1, characterized in that The attention mechanism is the CBAM attention mechanism, including a channel attention mechanism module and a spatial attention mechanism module. First, the input features pass through the channel attention module. The channel attention module uses adaptive max pooling and average pooling to extract global features respectively, processes the pooling results through a shared fully connected layer, and uses the sigmoid activation function to normalize the results to obtain the final channel attention map; Then, the features are input into the spatial attention module. The spatial attention module integrates the channel information through max pooling and average pooling operations, concatenates the pooling results in the channel dimension, and extracts spatial features through convolution operations, and uses the sigmoid activation function to generate the attention map in the spatial dimension; the final output result has channel and spatial attention, enhancing the overall model's attention to key features and important spatial positions.

Images