Satellite remote sensing data processing system and method thereof
By using convolutional neural networks to perform feature extraction and correlation analysis on cloud and water vapor images in satellite remote sensing data processing system, the limitations of traditional rainfall prediction methods are solved, real-time rainfall monitoring and accurate prediction of the monitoring area are achieved.
Patent Information
- Application Number
- CN202311203468.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-18
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional rainfall prediction methods are based on discrete observation data points, with low temporal and spatial resolution, resulting in limitations in the prediction results, unable to fully cover the prediction area, and lack real-time monitoring and accurate prediction capabilities.
The meteorological sensor collects cloud images and water vapor images at different times in the area to be monitored, and uses a convolutional neural network to extract high-dimensional implicit feature distribution information from both local and global directions, and conducts joint analysis and correlation of cloud and water vapor features to generate classified features The chart is used to predict rainfall.
Real-time rainfall monitoring and more accurate and efficient rainfall prediction in the monitoring area are achieved, improving the accuracy and coverage of weather forecasts.
Smart Images

Figure CN119091309A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of satellite remote sensing, and more specifically, to a satellite remote sensing data processing system and method thereof. Background Art
[0002] With the rapid development of science and technology and the continuous pursuit of weather forecast accuracy by humans, satellite data plays an increasingly important role in the field of weather forecasting. At present, satellite weather forecasts have become one of the most important means of obtaining global weather information. With the continuous updating and advancement of satellite launch technology and satellite remote sensing technology, the quality and accuracy of satellite data are also getting higher and higher, enabling us to make more accurate weather forecasts. For example, we can use satellite remote sensing data to predict rainfall. Traditional rainfall prediction methods are usually based on discrete observation data points and have low temporal and spatial resolutions. The spatial distribution of these data is limited and may not fully cover the forecast area, resulting in limitations in the forecast results.
[0003] Therefore, a satellite remote sensing data processing system and method thereof are desired to monitor rainfall conditions in real time through satellite remote sensing data so as to make more accurate and efficient rainfall prediction. Summary of the invention
[0004] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a satellite remote sensing data processing system and method thereof, which collects cloud images and water vapor images at different times in the monitored area through meteorological sensors, further extracts high-dimensional implicit feature distribution information of the cloud images and the water vapor images from local and global directions through convolutional neural networks, then respectively combines the obtained cloud local feature map and water vapor local feature map, and combines the obtained cloud global feature map and water vapor global feature map, and then fuses and analyzes the obtained cloud-water vapor local correlation feature map and cloud-water vapor global correlation feature map, so as to analyze and predict whether it will rain in the monitored area.
[0005] According to a first aspect of the present application, there is provided a satellite remote sensing data processing system, comprising:
[0006] The monitored image acquisition unit is used to collect cloud images and water vapor images at different times in the monitored area through meteorological sensors;
[0007] a first convolutional encoding unit, configured to pass the cloud image and the water vapor image through a first convolutional neural network as a local feature extractor to obtain a cloud local feature map and a water vapor local feature map;
[0008] a second convolutional encoding unit, configured to pass the cloud image and the water vapor image through a second convolutional neural network as a non-local feature extractor to obtain a cloud global feature map and a water vapor global feature map;
[0009] A local feature association unit, used for associating the cloud local feature map with the water vapor local feature map to obtain a cloud-water vapor local association feature map;
[0010] A global feature association unit, used for associating the cloud global feature map with the water vapor global feature map to obtain a cloud-water vapor global association feature map;
[0011] A feature fusion unit, used for associating the cloud-water vapor local correlation feature map with the cloud-water vapor global correlation feature map to obtain a classification feature map;
[0012] A feature optimization unit, used for performing homography density domain correlation analysis in feature space on the classification feature map to obtain an optimized classification feature map;
[0013] A result generating unit is used for passing the optimized classification feature map through a classifier to obtain a classification result, wherein the classification result is used to indicate whether it will rain in the monitored area.
[0014] In combination with the first aspect of the present application, in a satellite remote sensing data processing system of the first aspect of the present application, the first convolutional coding unit is used to: use each layer of the first convolutional neural network serving as a local feature extractor to perform convolution processing, mean pooling processing based on the local feature matrix, and nonlinear activation processing on the input data in the forward pass of the layer so that the last layer of the first convolutional neural network serving as a local feature extractor outputs the cloud local feature map and the water vapor local feature map, wherein the input of the first convolutional neural network serving as a local feature extractor is the cloud image and the water vapor image.
[0015] In combination with the first aspect of the present application, in a satellite remote sensing data processing system of the first aspect of the present application, the second convolutional encoding unit is used to: use each layer of the second convolutional neural network as a non-local feature extractor to perform the following on the input data in the forward transfer of the layer: based on the convolution kernel, perform convolution processing on the input data to generate a cloud convolution feature map and a water vapor convolution feature map; perform mean pooling processing based on the local feature matrix on the cloud convolution feature map and the water vapor convolution feature map to generate a cloud pooling feature map and a water vapor pooling feature map; and perform nonlinear activation on the eigenvalues of each position in the cloud pooling feature map and the water vapor pooling feature map to generate a cloud activation feature map and a water vapor activation feature map; wherein the output of the last layer of the second convolutional neural network as a non-local feature extractor is the cloud global feature map and the water vapor global feature map, respectively, the input from the second layer to the last layer of the second convolutional neural network as a non-local feature extractor is the output of the previous layer, and the input of the second convolutional neural network as a non-local feature extractor is the cloud image and the water vapor image, respectively.
[0016] According to a second aspect of the present application, a satellite remote sensing data processing method is provided, which includes:
[0017] Collect cloud images and water vapor images at different times in the monitored area through meteorological sensors;
[0018] Passing the cloud image and the water vapor image through a first convolutional neural network as a local feature extractor to obtain a cloud local feature map and a water vapor local feature map respectively;
[0019] Passing the cloud image and the water vapor image through a second convolutional neural network as a non-local feature extractor to obtain a cloud global feature map and a water vapor global feature map respectively;
[0020] Associating the cloud local feature map with the water vapor local feature map to obtain a cloud-water vapor local correlation feature map;
[0021] Associating the cloud global feature map with the water vapor global feature map to obtain a cloud-water vapor global correlation feature map;
[0022] Associating the cloud-water vapor local correlation feature map with the cloud-water vapor global correlation feature map to obtain a classification feature map;
[0023] Performing homography density domain correlation analysis in feature space on the classification feature map to obtain an optimized classification feature map;
[0024] The optimized classification feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether it will rain in the area to be monitored.
[0025] According to another aspect of the present application, a computer-readable medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the satellite remote sensing data processing method as described above.
[0026] Compared with the prior art, the present application provides a satellite remote sensing data processing system and method thereof, which collects cloud images and water vapor images at different times in the monitored area through meteorological sensors, further extracts high-dimensional implicit feature distribution information of the cloud images and the water vapor images from local and global directions through a convolutional neural network, then respectively combines the obtained cloud local feature map and water vapor local feature map, and combines the obtained cloud global feature map and water vapor global feature map, and then fuses and analyzes the obtained cloud-water vapor local correlation feature map and cloud-water vapor global correlation feature map, so as to analyze and predict whether it will rain in the monitored area. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0028] Figure 1 The figure illustrates a schematic block diagram of a satellite remote sensing data processing system according to an embodiment of the present application.
[0029] Figure 2 The figure illustrates a flow chart of a satellite remote sensing data processing method according to an embodiment of the present application.
[0030] Figure 3 A schematic diagram of the system architecture of a satellite remote sensing data processing method according to an embodiment of the present application is illustrated.
[0031] Figure 4 A block diagram of an electronic device according to an embodiment of the present application is illustrated. DETAILED DESCRIPTION
[0032] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.
[0033] Scenario Overview
[0034] With the rapid development of science and technology and the continuous pursuit of weather forecast accuracy by humans, satellite data plays an increasingly important role in the field of weather forecasting. At present, satellite weather forecasts have become one of the most important means of obtaining global weather information. With the continuous updating and advancement of satellite launch technology and satellite remote sensing technology, the quality and accuracy of satellite data are also getting higher and higher, enabling us to make more accurate weather forecasts. For example, we can use satellite remote sensing data to predict rainfall. Traditional rainfall prediction methods are usually based on discrete observation data points and have low temporal and spatial resolutions. The spatial distribution of these data is limited and may not fully cover the forecast area, resulting in limitations in the forecast results.
[0035] Therefore, a satellite remote sensing data processing system and method thereof are desired to monitor rainfall conditions in real time through satellite remote sensing data so as to make more accurate and efficient rainfall prediction.
[0036] Accordingly, in the technical solution of the present application, the technical principle of the satellite remote sensing data processing system is explained by taking the real-time distribution of clouds and water vapor in the monitored area as an example. Specifically, considering that rainfall is condensed by water droplets or ice crystals in the clouds, based on this, the rainfall in the monitored area can be observed and predicted in real time by monitoring the shape, size, density and spatial distribution characteristics of the clouds. Specifically, cloud images at different times in the monitored area are collected by meteorological sensors, wherein the cloud images at different times in the monitored area show information such as the position, shape, type and other information of clouds in the atmosphere, that is, by obtaining cloud images at different times in the monitored area, the accumulation and growth of clouds can be observed, and then it can be inferred whether rainfall occurs. In addition, considering that water vapor forms water droplets or ice crystals when it sublimates or condenses in cold, thereby causing rainfall, therefore, it is also possible to determine whether rainfall will occur in the monitored area by obtaining the distribution characteristics of water vapor in the monitored area. Specifically, water vapor images at different times in the monitored area are collected by meteorological sensors, wherein the water vapor images at different times in the monitored area show the distribution of water vapor content in the atmosphere. That is, by obtaining water vapor images at different times in the monitored area, the distribution and changes of water vapor in the atmosphere can be understood, and then whether there is a possibility of rainfall.
[0037] After collecting the above data, how to analyze it to obtain the classification result is the key. It should be understood that the clouds and water vapor in the monitored area not only have local features such as color, shape, texture, etc., but also global features such as cloud coverage, average cloud height, overall water vapor content, atmospheric humidity, geography and topography in the monitored area. Therefore, the cloud image and the water vapor image are feature extracted and analyzed from both local and global perspectives.
[0038] Specifically, considering that the cloud image in the area to be monitored and the water vapor image in the area to be monitored are both image data, therefore, in the technical solution of the present application, a convolutional neural network model with excellent performance in the field of image processing is used to encode the cloud image in the area to be monitored and the water vapor image in the area to be monitored respectively. Further, considering that the distribution of clouds and water vapor usually has complex textures, shapes and structures, by extracting local features, the detailed information of clouds and water vapor, such as edges, textures, shapes, etc., can be captured. These detailed information can distinguish different types of clouds, detect cloud boundaries, and segment cloud and water vapor and other feature information. Therefore, in order to capture the local high-dimensional implicit change characteristics of clouds and water vapor in the area to be monitored in the cloud image and the water vapor image in the spatial dimension, a first convolutional neural network as a local feature extractor is used to encode the cloud image and the water vapor image to obtain the cloud local feature map and the water vapor local feature map.
[0039] In addition, it should be understood that global features such as cloud coverage, average cloud height, overall water vapor content, atmospheric humidity, geography and topography in the monitored area are also key factors affecting rainfall. By extracting the global features of clouds and water vapor in the monitored area, the overall distribution information of clouds and water vapor in the monitored area can be obtained, including their overall coverage, density distribution, concentrated areas, etc. By analyzing the overall distribution information, the relationship between clouds and water vapor and factors such as geography and climate can be understood. Therefore, a second convolutional neural network as a non-local feature extractor is used to encode the cloud image and the water vapor image to capture the global high-dimensional implicit change features of clouds and water vapor in the cloud image and the water vapor image in the spatial dimension, thereby obtaining the cloud global feature map and the water vapor global feature map.
[0040] Next, considering that there is a certain correlation between the distribution of clouds and the distribution of water vapor in the monitored area, for example, when the water vapor in the air reaches the saturation point and there are suitable condensation nuclei, the water vapor will condense into cloud droplets or ice crystals to form clouds, that is, the increase in water vapor content will increase the potential for cloud formation. Therefore, the correlation technology in machine learning can be used to extract and analyze the distribution of clouds and water vapor in the monitored area. Specifically, the cloud local feature map and the water vapor local feature map are correlated to obtain a cloud-water vapor local correlation feature map, and the cloud global feature map and the water vapor global feature map are correlated to obtain a cloud-water vapor global correlation feature map.
[0041] Then, the rainfall in the monitoring area can be monitored by fusing the local cloud-water vapor correlation feature map and the global cloud-water vapor correlation feature map. Specifically, the local cloud-water vapor correlation feature map and the global cloud-water vapor correlation feature map are correlated to obtain a classification feature map.
[0042] In particular, in the technical solution of the present application, it is taken into account that the classification feature map is essentially a collection of local features represented by its channel dimension, that is, on the one hand, the classification feature map participates in the subsequent classification judgment as a whole feature collection, and on the other hand, if the feature distribution association between the local features in the classification feature map can be utilized, the feature distribution information implied in the classification feature map can be further utilized to optimize the feature expression of the classification feature map.
[0043] Specifically, the classification feature map is subjected to homography density domain correlation analysis in feature space to obtain an optimized classification feature map, including: a position mean calculation subunit, which calculates the position mean of each feature matrix along the channel dimension of the classification feature map to obtain a feature self-distribution cluster center matrix; a cosine similarity calculation subunit, which calculates the cosine similarity between each feature matrix along the channel dimension of the classification feature map and the feature self-distribution cluster center matrix to obtain a plurality of feature density domain eigenvalues; a feature activation subunit, which arranges the plurality of feature density domain eigenvalues as an input vector and then passes the Softmax activation function to obtain a homography density domain weighted feature vector; a feature weighting subunit, which uses the eigenvalue of each position in the homography density domain weighted feature vector as a weight to weight each feature matrix along the channel dimension of the classification feature map to obtain the optimized classification feature map.
[0044] Here, in the technical solution of the present application, the positional means of each feature matrix along the channel dimension of the classification feature map are calculated to obtain a feature self-distribution cluster center matrix, that is, along the channel dimension of the classification feature map, the classification feature map is subjected to self-clustering analysis to obtain a feature self-distribution cluster center matrix, which can be regarded as the self-clustering center of the classification feature map from the perspective of the feature manifold. Then, the contribution of each local feature of the classification feature map to the final classification judgment is re-weighed by taking the feature self-distribution cluster center matrix as the pivot and measuring the intra-class consistency between each feature subset of the classification feature map and the pivot. In this way, the intrinsic structure and relationship in the data can be found without reducing the data dimension, that is, the original information and features of the data are retained to avoid information loss and distortion caused by dimensionality reduction.
[0045] Finally, the optimized classification feature map is classified by a classifier to obtain a classification result indicating whether it will rain in the monitored area. That is, in the technical solution of the present application, the classification labels of the classifier include whether it will rain in the monitored area and whether it will not rain in the monitored area. Based on this, the rainfall in the monitored area can be monitored in real time.
[0046] Exemplary Systems
[0047] Figure 1 FIG. 1 is a schematic block diagram of a satellite remote sensing data processing system according to an embodiment of the present application. Figure 1 As shown, according to the satellite remote sensing data processing system 100 of the embodiment of the present application, it includes: a monitored image acquisition unit 110, which is used to collect cloud images and water vapor images at different times in the monitored area through a meteorological sensor; a first convolutional coding unit 120, which is used to pass the cloud image and the water vapor image through a first convolutional neural network as a local feature extractor to obtain a cloud local feature map and a water vapor local feature map; a second convolutional coding unit 130, which is used to pass the cloud image and the water vapor image through a second convolutional neural network as a non-local feature extractor to obtain a cloud global feature map and a water vapor global feature map; a local feature association unit 140, which is used to associate the cloud local feature map with the water vapor local feature map. The water vapor local feature map is associated with the water vapor local correlation feature map to obtain a cloud-water vapor local correlation feature map; a global feature association unit 150 is used to associate the cloud global feature map with the water vapor global feature map to obtain a cloud-water vapor global correlation feature map; a feature fusion unit 160 is used to associate the cloud-water vapor local correlation feature map with the cloud-water vapor global correlation feature map to obtain a classification feature map; a feature optimization unit 170 is used to perform homography density domain association analysis in feature space on the classification feature map to obtain an optimized classification feature map; a result generation unit 180 is used to pass the optimized classification feature map through a classifier to obtain a classification result, and the classification result is used to indicate whether there will be rainfall in the monitored area.
[0048] In an embodiment of the present application, the image acquisition unit 110 to be monitored is used to collect cloud images and water vapor images at different times in the area to be monitored through a meteorological sensor. Considering that rainfall is condensed by water droplets or ice crystals in the cloud, based on this, the rainfall situation in the area to be monitored can be observed and predicted in real time by monitoring the shape, size, density and spatial distribution characteristics of the cloud. Specifically, cloud images at different times in the area to be monitored are collected by meteorological sensors, wherein the cloud images at different times in the area to be monitored show information such as the position, shape, type and other information of the clouds in the atmosphere, that is, by acquiring cloud images at different times in the area to be monitored, the accumulation and growth of clouds can be observed, and then it can be inferred whether rainfall occurs. In addition, considering that water vapor forms water droplets or ice crystals when it sublimates or condenses in cold, thereby causing rainfall, therefore, it is also possible to determine whether rainfall will occur in the area to be monitored by acquiring the distribution characteristics of water vapor in the area to be monitored. Specifically, water vapor images at different times in the monitored area are collected by meteorological sensors, wherein the water vapor images at different times in the monitored area show the distribution of water vapor content in the atmosphere. That is, by obtaining water vapor images at different times in the monitored area, the distribution and changes of water vapor in the atmosphere can be understood, and then whether there is a possibility of rainfall.
[0049] In the embodiment of the present application, the first convolutional coding unit 120 is used to pass the cloud image and the water vapor image through the first convolutional neural network as a local feature extractor to obtain a cloud local feature map and a water vapor local feature map. Considering that the cloud image in the area to be monitored and the water vapor image in the area to be monitored are both image data, therefore, in the technical solution of the present application, a convolutional neural network model with excellent performance in the field of image processing is used to encode the cloud image in the area to be monitored and the water vapor image in the area to be monitored. Further, considering that the distribution of clouds and water vapor usually has complex textures, shapes and structures, by extracting local features, the detailed information of clouds and water vapor, such as edges, textures, shapes, etc., can be captured. These detailed information can distinguish different types of clouds, detect cloud boundaries, and segment cloud and water vapor and other feature information. Therefore, in order to capture the local high-dimensional implicit change characteristics of clouds and water vapor in the area to be monitored in the cloud image and the water vapor image in the spatial dimension, the first convolutional neural network as a local feature extractor is used to encode the cloud image and the water vapor image to obtain the cloud local feature map and the water vapor local feature map.
[0050] In the embodiment of the present application, the second convolutional coding unit 130 is used to pass the cloud image and the water vapor image through the second convolutional neural network as a non-local feature extractor to obtain a cloud global feature map and a water vapor global feature map. It should be understood that global features such as cloud coverage, average cloud height, overall water vapor content, atmospheric humidity, geography and topography in the monitored area are also key to affecting rainfall. By extracting the global features of clouds and water vapor in the monitored area, the overall distribution information of clouds and water vapor in the monitored area can be obtained, which includes their overall coverage, density distribution, concentrated area, etc. By analyzing the overall distribution information, the relationship between clouds and water vapor and factors such as geography and climate can be understood. Therefore, the second convolutional neural network as a non-local feature extractor is used to encode the cloud image and the water vapor image to capture the global high-dimensional implicit change characteristics of clouds and water vapor in the cloud image and the water vapor image in the spatial dimension, thereby obtaining the cloud global feature map and the water vapor global feature map.
[0051] In a specific embodiment of the present application, the second convolutional encoding unit 130 is used to: use each layer of the second convolutional neural network as a non-local feature extractor to perform on the input data in the forward pass of the layer: based on the convolution kernel, convolution processing is performed on the input data to generate a cloud convolution feature map and a water vapor convolution feature map; the cloud convolution feature map and the water vapor convolution feature map are respectively subjected to mean pooling processing based on the local feature matrix to generate a cloud pooling feature map and a water vapor pooling feature map; and the feature values of each position in the cloud pooling feature map and the water vapor pooling feature map are respectively nonlinearly activated to generate a cloud activation feature map and a water vapor activation feature map;
[0052] Among them, the output of the last layer of the second convolutional neural network serving as a non-local feature extractor is the cloud global feature map and the water vapor global feature map, respectively, the input from the second layer to the last layer of the second convolutional neural network serving as a non-local feature extractor is the output of the previous layer, and the input of the second convolutional neural network serving as a non-local feature extractor is the cloud image and the water vapor image, respectively.
[0053] In an embodiment of the present application, the local feature association unit 140 is used to associate the cloud local feature map with the water vapor local feature map to obtain a cloud-water vapor local correlation feature map. Considering that there is a certain correlation between the distribution of clouds and the distribution of water vapor in the monitored area, for example, when the water vapor in the air reaches the saturation point and there are suitable condensation nuclei, the water vapor will condense into cloud droplets or ice crystals to form clouds, that is, the increase in water vapor content will increase the potential for cloud formation. Therefore, the association technology in machine learning can be used to extract and analyze the distribution of clouds and water vapor in the monitored area. Specifically, the cloud local feature map and the water vapor local feature map are associated to obtain a cloud-water vapor local correlation feature map, and the cloud global feature map and the water vapor global feature map are associated to obtain a cloud-water vapor global correlation feature map.
[0054] In the embodiment of the present application, the global feature association unit 150 is used to associate the cloud global feature map with the water vapor global feature map to obtain a cloud-water vapor global association feature map.
[0055] In a specific embodiment of the present application, the global feature association unit 150 is used to: fuse the cloud global feature map and the water vapor global feature map using the following cascade formula to obtain the cloud-water vapor global association feature map;
[0056] Wherein, the cascade formula is:
[0057] F=Concat[F1,F2]
[0058] Among them, F1 represents the cloud global feature map, F2 represents the water vapor global feature map, F represents the cloud-water vapor global correlation feature map, and Concat[·,·] represents the cascade function.
[0059] In an embodiment of the present application, the feature fusion unit 160 is used to associate the cloud-water vapor local correlation feature map with the cloud-water vapor global correlation feature map to obtain a classification feature map. It should be understood that the clouds and water vapor in the monitored area not only have local features such as color, shape, texture, etc., but also global features such as cloud coverage, average cloud height, overall water vapor content, atmospheric humidity, geography and topography in the monitored area. Therefore, the cloud image and the water vapor image are more comprehensively extracted and analyzed from both local and global perspectives. Specifically, the cloud-water vapor local correlation feature map and the cloud-water vapor global correlation feature map are fused to obtain a classification feature map.
[0060] In an embodiment of the present application, the feature optimization unit 170 is used to perform homography density domain correlation analysis of the classification feature map in feature space to obtain an optimized classification feature map. In particular, in the technical solution of the present application, it is considered that the classification feature map is essentially a collection of local features represented by its channel dimension, that is, on the one hand, the classification feature map participates in subsequent classification judgment as a whole feature set, and on the other hand, if the feature distribution association between the local features in the classification feature map can be utilized, the feature distribution information implied in the classification feature map can be further utilized to optimize the feature expression of the classification feature map.
[0061] Specifically, the classification feature map is subjected to homography density domain correlation analysis in feature space to obtain an optimized classification feature map, including: a position mean calculation subunit, which calculates the position mean of each feature matrix along the channel dimension of the classification feature map to obtain a feature self-distribution cluster center matrix; a cosine similarity calculation subunit, which calculates the cosine similarity between each feature matrix along the channel dimension of the classification feature map and the feature self-distribution cluster center matrix to obtain a plurality of feature density domain eigenvalues; a feature activation subunit, which arranges the plurality of feature density domain eigenvalues as an input vector and then passes the Softmax activation function to obtain a homography density domain weighted feature vector; a feature weighting subunit, which uses the eigenvalue of each position in the homography density domain weighted feature vector as a weight to weight each feature matrix along the channel dimension of the classification feature map to obtain the optimized classification feature map.
[0062] Here, in the technical solution of the present application, the positional means of each feature matrix along the channel dimension of the classification feature map are calculated to obtain a feature self-distribution cluster center matrix, that is, along the channel dimension of the classification feature map, the classification feature map is subjected to self-clustering analysis to obtain a feature self-distribution cluster center matrix, which can be regarded as the self-clustering center of the classification feature map from the perspective of the feature manifold. Then, the contribution of each local feature of the classification feature map to the final classification judgment is re-weighed by taking the feature self-distribution cluster center matrix as the pivot and measuring the intra-class consistency between each feature subset of the classification feature map and the pivot. In this way, the intrinsic structure and relationship in the data can be found without reducing the data dimension, that is, the original information and features of the data are retained to avoid information loss and distortion caused by dimensionality reduction.
[0063] In the embodiment of the present application, the result generating unit 180 is used to pass the optimized classification feature map through a classifier to obtain a classification result, and the classification result is used to indicate whether it will rain in the area to be monitored. It should be understood that the classifier can classify and predict the data, so the classification result of the optimized classification feature map can be obtained by using the classifier, and the classification result is used to indicate whether it will rain in the area to be detected.
[0064] In a specific embodiment of the present application, the result generating unit 180 is used to: use the classifier to process the optimized classification feature map using the following classification formula to obtain the classification result;
[0065] Wherein, the classification formula is: O = softmax{(W c ,B c )|Project(F)}, where Project(F) represents projecting the optimized classification feature map into a vector, W c is the weight matrix, B c represents a bias vector, softmax represents a normalized exponential function, and O represents the classification result.
[0066] In summary, the satellite remote sensing data processing system based on the embodiment of the present application collects cloud images and water vapor images at different times in the monitored area through meteorological sensors, and further extracts high-dimensional implicit feature distribution information of the cloud images and the water vapor images from both local and global directions through a convolutional neural network, and then respectively combines the obtained cloud local feature map and water vapor local feature map, and combines the obtained cloud global feature map and water vapor global feature map, and then fuses and analyzes the obtained cloud-water vapor local correlation feature map and cloud-water vapor global correlation feature map, so as to analyze and predict whether it will rain in the monitored area.
[0067] As described above, the satellite remote sensing data processing system 100 according to the embodiment of the present application can be implemented in various terminal devices, such as a server deployed with a satellite remote sensing data processing algorithm. In one example, the satellite remote sensing data processing system 100 can be integrated into the terminal device as a software module and / or a hardware module. For example, the satellite remote sensing data processing system 100 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the satellite remote sensing data processing system 100 can also be one of the many hardware modules of the terminal device.
[0068] Alternatively, in another example, the satellite remote sensing data processing system 100 and the terminal device may also be separate devices, and the satellite remote sensing data processing system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.
[0069] Exemplary Methods
[0070] Figure 2 FIG. 1 is a flowchart of a satellite remote sensing data processing method according to an embodiment of the present application. Figure 2 As shown, according to the satellite remote sensing data processing method of the embodiment of the present application, the method includes: S110, collecting cloud images and water vapor images at different times in the monitored area through a meteorological sensor; S120, respectively passing the cloud image and the water vapor image through a first convolutional neural network as a local feature extractor to obtain a cloud local feature map and a water vapor local feature map; S130, respectively passing the cloud image and the water vapor image through a second convolutional neural network as a non-local feature extractor to obtain a cloud global feature map and a water vapor global feature map; S140, the cloud local feature map and the water vapor local feature map are respectively obtained. S150, associating the cloud global feature map with the water vapor global feature map to obtain a cloud-water vapor global correlation feature map; S160, associating the cloud-water vapor local correlation feature map with the cloud-water vapor global correlation feature map to obtain a classification feature map; S170, performing homography density domain association analysis in feature space on the classification feature map to obtain an optimized classification feature map; S180, passing the optimized classification feature map through a classifier to obtain a classification result, which is used to indicate whether it will rain in the area to be monitored.
[0071] Figure 3 FIG. 1 is a schematic diagram of a system architecture of a satellite remote sensing data processing method according to an embodiment of the present application. Figure 3As shown, first, cloud images and water vapor images at different times in the monitored area are collected by meteorological sensors respectively. Considering that the clouds and water vapor in the monitored area not only have local features such as color, shape, texture, etc., but also have global features such as cloud coverage, average cloud height, overall water vapor content, atmospheric humidity, geography and topography in the monitored area, a more comprehensive feature extraction and analysis of the cloud image and the water vapor image is performed from both local and global perspectives. Specifically, the cloud image and the water vapor image are passed through a first convolutional neural network as a local feature extractor to obtain the cloud local feature map and the water vapor local feature map. At the same time, the cloud image and the water vapor image are passed through a second convolutional neural network as a non-local feature extractor to obtain the cloud global feature map and the water vapor global feature map. Then, the cloud local feature map and the water vapor local feature map are feature-correlated to obtain a cloud-water vapor local correlation feature map, and the cloud global feature map and the water vapor global feature map are feature-correlated to obtain a cloud-water vapor global correlation feature map. Then, the cloud-water vapor local correlation feature map and the cloud-water vapor global correlation feature map are fused to obtain a classification feature map, and the classification feature map is subjected to homography density domain correlation analysis in feature space to obtain an optimized classification feature map. Finally, the optimized classification feature map is passed through a classifier to obtain a classification result indicating whether there is rainfall in the monitored area.
[0072] In a specific embodiment of the present application, the cloud image and the water vapor image are respectively passed through a first convolutional neural network as a local feature extractor to obtain a cloud local feature map and a water vapor local feature map, which are used to: use each layer of the first convolutional neural network as a local feature extractor to perform convolution processing, mean pooling processing based on the local feature matrix, and nonlinear activation processing on the input data in the forward pass of the layer to output the cloud local feature map and the water vapor local feature map from the last layer of the first convolutional neural network as a local feature extractor, wherein the input of the first convolutional neural network as a local feature extractor is the cloud image and the water vapor image.
[0073] In a specific embodiment of the present application, the cloud image and the water vapor image are respectively passed through a second convolutional neural network as a non-local feature extractor to obtain a cloud global feature map and a water vapor global feature map, for: using each layer of the second convolutional neural network as a non-local feature extractor to perform the following on the input data in the forward pass of the layer:
[0074] Based on the convolution kernel, the input data is respectively convolved to generate a cloud convolution feature map and a water vapor convolution feature map;
[0075] Performing mean pooling processing based on a local feature matrix on the cloud convolution feature map and the water vapor convolution feature map to generate a cloud pooling feature map and a water vapor pooling feature map respectively; and
[0076] Nonlinearly activate the feature values of each position in the cloud pooling feature map and the water vapor pooling feature map to generate a cloud activation feature map and a water vapor activation feature map;
[0077] Among them, the output of the last layer of the second convolutional neural network serving as a non-local feature extractor is the cloud global feature map and the water vapor global feature map, respectively, the input from the second layer to the last layer of the second convolutional neural network serving as a non-local feature extractor is the output of the previous layer, and the input of the second convolutional neural network serving as a non-local feature extractor is the cloud image and the water vapor image, respectively.
[0078] In a specific embodiment of the present application, the cloud local feature map and the water vapor local feature map are associated to obtain a cloud-water vapor local correlation feature map, which is used to: fuse the cloud local feature map and the water vapor local feature map using the following fusion formula to obtain the cloud-water vapor local correlation feature map, wherein the fusion formula is:
[0079] F1=αF s +βF d
[0080] Among them, F1 is the cloud-water vapor local correlation feature map, F s is the local feature map of the cloud, F d is the water vapor local feature map, “+” represents the addition of the elements at corresponding positions of the cloud local feature map and the water vapor local feature map, and α and β are weighting parameters for controlling the balance between the cloud local feature map and the water vapor local feature map in the cloud-water vapor local correlation feature map.
[0081] Here, those skilled in the art will appreciate that the specific functions and operations of each step in the above satellite remote sensing data processing method have been described in detail above. Figure 1 The invention has been described in detail in the description of the satellite remote sensing data processing system, and therefore, its repeated description will be omitted.
[0082] Exemplary Electronic Devices
[0083] Below, reference Figure 4 To describe an electronic device according to an embodiment of the present application.
[0084] Figure 4 A block diagram of an electronic device according to an embodiment of the present application is illustrated.
[0085] like Figure 4As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .
[0086] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
[0087] The memory 12 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the satellite remote sensing data processing system of each embodiment of the present application described above and / or other desired functions.
[0088] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0089] The input device 13 may include, for example, a keyboard, a mouse, etc.
[0090] The output device 14 can output various information to the outside, including decoded values, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.
[0091] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.
[0092] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.
[0093] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.
[0094] It should be understood that the specific examples herein are only intended to help those skilled in the art better understand the embodiments of the present application, and do not limit the scope of the embodiments of the present application. In addition, in various embodiments of the present application, the size of the sequence number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0095] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A satellite remote sensing data processing system, characterized in that: include: The monitored image acquisition unit is used to collect cloud images and water vapor images at different times in the monitored area through meteorological sensors; a first convolutional encoding unit, configured to pass the cloud image and the water vapor image through a first convolutional neural network as a local feature extractor to obtain a cloud local feature map and a water vapor local feature map; a second convolutional encoding unit, configured to pass the cloud image and the water vapor image through a second convolutional neural network as a non-local feature extractor to obtain a cloud global feature map and a water vapor global feature map; A local feature association unit, used for associating the cloud local feature map with the water vapor local feature map to obtain a cloud-water vapor local association feature map; A global feature association unit, used for associating the cloud global feature map with the water vapor global feature map to obtain a cloud-water vapor global association feature map; A feature fusion unit, used for associating the cloud-water vapor local correlation feature map with the cloud-water vapor global correlation feature map to obtain a classification feature map; A feature optimization unit, used for performing homography density domain correlation analysis in feature space on the classification feature map to obtain an optimized classification feature map; A result generating unit is used for passing the optimized classification feature map through a classifier to obtain a classification result, wherein the classification result is used to indicate whether it will rain in the monitored area.
2. The satellite remote sensing data processing system according to claim 1, characterized in that: The first convolutional encoding unit is used to: use each layer of the first convolutional neural network as a local feature extractor to perform convolution processing, mean pooling processing based on the local feature matrix and nonlinear activation processing on the input data in the forward pass of the layer so that the last layer of the first convolutional neural network as a local feature extractor outputs the cloud local feature map and the water vapor local feature map, wherein the input of the first convolutional neural network as a local feature extractor is the cloud image and the water vapor image.
3. The satellite remote sensing data processing system according to claim 2, characterized in that: The second convolutional encoding unit is used to: use each layer of the second convolutional neural network as a non-local feature extractor to respectively perform the following on the input data in the forward pass of the layer: Based on the convolution kernel, the input data is respectively convolved to generate a cloud convolution feature map and a water vapor convolution feature map; Performing mean pooling processing based on a local feature matrix on the cloud convolution feature map and the water vapor convolution feature map to generate a cloud pooling feature map and a water vapor pooling feature map respectively; as well as Nonlinearly activate the feature values of each position in the cloud pooling feature map and the water vapor pooling feature map to generate a cloud activation feature map and a water vapor activation feature map; Among them, the output of the last layer of the second convolutional neural network serving as a non-local feature extractor is the cloud global feature map and the water vapor global feature map, respectively, the input from the second layer to the last layer of the second convolutional neural network serving as a non-local feature extractor is the output of the previous layer, and the input of the second convolutional neural network serving as a non-local feature extractor is the cloud image and the water vapor image, respectively.
4. The satellite remote sensing data processing system according to claim 3, characterized in that: The local feature association unit is used to fuse the cloud local feature map and the water vapor local feature map using the following fusion formula to obtain the cloud-water vapor local correlation feature map, wherein the fusion formula is: F1=αF s +βF d Among them, F1 is the cloud-water vapor local correlation feature map, F s is the local feature map of the cloud, F d is the water vapor local feature map, "+" represents the addition of the elements at corresponding positions of the cloud local feature map and the water vapor local feature map, and α and β are weighting parameters for controlling the balance between the cloud local feature map and the water vapor local feature map in the cloud-water vapor local correlation feature map.
5. The satellite remote sensing data processing system according to claim 4, characterized in that: The global feature association unit is used to: fuse the cloud global feature map and the water vapor global feature map using the following cascade formula to obtain the cloud-water vapor global association feature map; Wherein, the cascade formula is: F=Concat[F1,F2] Among them, F1 represents the cloud global feature map, F2 represents the water vapor global feature map, F represents the cloud-water vapor global correlation feature map, and Concat[·,·] represents the cascade function.
6. The satellite remote sensing data processing system according to claim 5, characterized in that: The feature optimization unit comprises: A position mean calculation subunit is used to calculate the position mean of each feature matrix along the channel dimension of the classification feature map to obtain a feature self-distribution cluster center matrix; A cosine similarity calculation subunit calculates the cosine similarity between each feature matrix along the channel dimension of the classification feature map and the feature self-distribution cluster center matrix to obtain a plurality of feature density domain feature values; A feature activation subunit, which arranges the plurality of feature density domain feature values into an input vector and then passes the Softmax activation function to obtain a homography density domain weight feature vector; The feature weighting subunit uses the feature values of each position in the homography density domain weight feature vector as weights to weight each feature matrix along the channel dimension of the classification feature map to obtain the optimized classification feature map.
7. The satellite remote sensing data processing system according to claim 6, characterized in that: The result generating unit is used to: use the classifier to process the optimized classification feature map according to the following classification formula to obtain the classification result; Wherein, the classification formula is: O = softmax{(W c ,B c )|Project(F)}, where Project(F) represents projecting the optimized classification feature map into a vector, W c is the weight matrix, B c represents a bias vector, softmax represents a normalized exponential function, and O represents the classification result.
8. A satellite remote sensing data processing method, characterized in that: include: Collect cloud images and water vapor images at different times in the monitored area through meteorological sensors; Passing the cloud image and the water vapor image through a first convolutional neural network as a local feature extractor to obtain a cloud local feature map and a water vapor local feature map respectively; Passing the cloud image and the water vapor image through a second convolutional neural network as a non-local feature extractor to obtain a cloud global feature map and a water vapor global feature map respectively; Associating the cloud local feature map with the water vapor local feature map to obtain a cloud-water vapor local correlation feature map; Associating the cloud global feature map with the water vapor global feature map to obtain a cloud-water vapor global correlation feature map; Associating the cloud-water vapor local correlation feature map with the cloud-water vapor global correlation feature map to obtain a classification feature map; Performing homography density domain correlation analysis in feature space on the classification feature map to obtain an optimized classification feature map; The optimized classification feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether it will rain in the area to be monitored.
9. The satellite remote sensing data processing method according to claim 8, characterized in that: The cloud image and the water vapor image are respectively passed through the first convolutional neural network as a local feature extractor to obtain a cloud local feature map and a water vapor local feature map, and are used to: use each layer of the first convolutional neural network as a local feature extractor to perform convolution processing, mean pooling processing based on the local feature matrix and nonlinear activation processing on the input data in the forward transfer of the layer so that the last layer of the first convolutional neural network as a local feature extractor outputs the cloud local feature map and the water vapor local feature map, wherein the input of the first convolutional neural network as a local feature extractor is the cloud image and the water vapor image.
10. The satellite remote sensing data processing method according to claim 9, characterized in that: The cloud image and the water vapor image are respectively passed through a second convolutional neural network as a non-local feature extractor to obtain a cloud global feature map and a water vapor global feature map, and are used to: use each layer of the second convolutional neural network as a non-local feature extractor to respectively perform the following on the input data in the forward pass of the layer: Based on the convolution kernel, the input data is respectively convolved to generate a cloud convolution feature map and a water vapor convolution feature map; Performing mean pooling processing based on a local feature matrix on the cloud convolution feature map and the water vapor convolution feature map to generate a cloud pooling feature map and a water vapor pooling feature map respectively; as well as Nonlinearly activate the feature values of each position in the cloud pooling feature map and the water vapor pooling feature map to generate a cloud activation feature map and a water vapor activation feature map; Among them, the output of the last layer of the second convolutional neural network serving as a non-local feature extractor is the cloud global feature map and the water vapor global feature map, respectively, the input from the second layer to the last layer of the second convolutional neural network serving as a non-local feature extractor is the output of the previous layer, and the input of the second convolutional neural network serving as a non-local feature extractor is the cloud image and the water vapor image, respectively.
