Sea ice range multi-modal fusion method, equipment, medium and product
By preprocessing and grouping the sea ice data set, a multimodal data fusion model is constructed, which solves the problem that different expressions and spatial resolution differences in the existing technology are not able to be integrated, and the efficiency, accuracy and efficiency of sea ice monitoring is achieved. Reliable multimodal fusion.
Patent Information
- Application Number
- CN202411165304.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-12
AI Technical Summary
The prior art cannot effectively fuse sea ice parameters of different manifestations, and cannot take into account spatial resolution differences in the fusion process of multi-source sea ice products, resulting in inaccurate and unreliable sea ice monitoring.
Using the multimodal fusion method of sea ice range, the multimodal data fusion model is constructed by pre-processing and grouping the sea ice data set, and using dynamic grouping convolution and adaptive pooling technology to transfer data of different spatial resolutions. Perform matching and fusion to generate multimodal fusion results.
The effective fusion of sea ice parameters in different manifestations is achieved, which improves the accuracy and reliability of sea ice monitoring, can process data with different spatial resolutions, improves the fusion accuracy and reduces calculation costs.
Smart Images

Figure CN118940218A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of microwave remote sensing technology, and in particular to a method, device, medium and product for multi-modal fusion of sea ice range. Background Art
[0002] Arctic and Antarctic sea ice is one of the core links of the global climate system, and its changes are one of the most concerned climate change phenomena. With the observed dramatic changes in polar sea ice and the interest in regional resource development, human activities have increased, and concerns about polar sea ice coverage have also increased. Polar sea ice observations are needed for many purposes, from sea ice forecasting, maritime traffic routes and search and rescue, to climate change analysis, long-term modeling, prediction and future planning.
[0003] Using data fusion and data assimilation technology to integrate observational data from multiple sources and model simulation data is an effective means to obtain high-precision, high-quality, and spatiotemporally continuous multi-source fusion sea ice distribution products. In response to scientific research and business needs, a variety of high-quality sea ice concentration fusion products have been developed internationally. For example, the National Oceanic and Atmospheric Administration (NOAA) of the United States used the median filtering method to develop and generate the global 0.25° daily sea ice concentration product OISST (Optimum Interpolation Sea Surface Temperature) (Reynolds et al., 2007; Banzon et al., 2016). This product has the characteristics of long time series and high timeliness and is widely used. The National Meteorological Office (Met Office) of the United Kingdom used bilinear interpolation and other methods to generate the global 0.05° daily sea ice concentration product OSTIA (Operational Sea Surface Temperature and Sea Ice Analysis) (Donlon et al., 2012). The UK Met Office Hadley Centre used digital charts of the same period to correct microwave remote sensing data for climate change research, and used climatological sea ice data to interpolate on grids without observation data to generate global 1° daily sea ice concentration HadISST data (Titchner and Rayner, 2014). Zhang Lei et al. (2021) used a variety of sea ice concentration products to generate a super observation value, and then used the Space-Time Multi-Scale Analysis System (STMAS) method to merge the super observation value with ERA-Interim (European Meteorological Center data, ECMWF Re-Analysis-Interim) sea ice concentration data (background field) to generate a 1-year merged global daily sea ice concentration data with a resolution of 0.25°.Regionally, the National Ice Center (NIC) of the United States has developed the Interactive Multisensor Snow and Ice Mapping System (IMS) (Helfrich et al., 2007) and the Multisensor Analyzed Sea Ice Extent project for the Northern Hemisphere (MASIE) for sea ice coverage data for the Northern Hemisphere, which mainly integrates multiple satellite data (such as visible light, passive microwave, active microwave, etc.) and ice and snow digital charts and other data sources through human-computer interaction. Based on the above discussion, the sea ice density products generated by passive microwave satellite remote sensing constitute the main source of data fusion, and only IMS and MASIE for specific regions consider additional sea ice parameters during fusion. In terms of data processing methods, manual interaction, filtering technology, interpolation methods and pattern analysis are widely used in the data fusion process, but the spatial resolution processing of single data source data is relatively simple.
[0004] Thanks to the advancement of remote sensing technology, there are now a variety of sea ice parameter products available for use by the scientific community. Given the diversity of currently available sea ice parameter products and the wide range of observational objectives, from advancing long-term climate predictions and future planning to supporting navigation, commerce, disaster monitoring, search and rescue, and disaster response, generating a more comprehensive and accurate sea ice distribution depends on a variety of factors, including the area of interest to the user, parameter product characteristics, including temporal and spatial coverage, temporal and spatial resolution, accuracy, and data availability and latency.
[0005] Based on the above description, the prior art has the following disadvantages:
[0006] 1. Because sea ice density, sea ice type, and sea ice range can all be used to indicate sea ice distribution, but these data have different forms of expression and storage methods, existing technologies cannot effectively integrate sea ice parameters in different forms.
[0007] 2. In order to ensure the accuracy of fusion, a coarser resolution is generally used as a reference during the fusion process, and the method adopted is mainly interpolation. The processing is relatively simple and cannot take into account the spatial resolution differences in the fusion process of multi-source sea ice products, which leads to the inability to accurately and reliably monitor sea ice. Summary of the invention
[0008] The purpose of this application is to provide a sea ice range multimodal fusion method, equipment, medium and product, which can effectively fuse sea ice parameters in different forms, while taking into account the spatial resolution differences in the fusion process of multi-source sea ice products, and effectively improving the accuracy and reliability of sea ice monitoring.
[0009] To achieve the above objectives, this application provides the following solutions:
[0010] In a first aspect, the present application provides a sea ice extent multi-modal fusion method, comprising:
[0011] Preprocessing the sea ice data set; the sea ice data set includes: NSIDC-SSMIS (National Snow and Ice Data Center / Special Sensor Microwave-Imager / Sounder) sea ice density, OSISAF-SSMIS (Ocean and Sea Ice Satellite Application Facility / Special Sensor Microwave-Imager / Sounder) sea ice density, OSISAF-AMSR (Ocean and Sea Ice Satellite Application Facility / Advanced Microwave Scanning Radiometer-2 / Eos) sea ice density, Bremen-AMSR (University of Bremen / Advanced Microwave Scanning Radiometer-2 / Eos) sea ice density, IMS sea surface type, OISAF (Ocean and Sea Ice Satellite Application Facility) sea ice range, OSISAF (Ocean and Sea Ice Satellite Application Facility) sea ice type, OISST (Optimum Interpolation Sea Surface Temperature) sea ice density;
[0012] The preprocessed sea ice data set is grouped to obtain a grouping result; the grouping result includes: a parameter data set, a first spatial resolution data set, a second spatial resolution data set, and a third spatial resolution data set;
[0013] Construct a multimodal data fusion model;
[0014] The grouping result is input into the multimodal data fusion model to obtain a multimodal fusion result.
[0015] Optionally, inputting the grouping result into the multimodal data fusion model to obtain a multimodal fusion result specifically includes:
[0016] Connecting the data in the first spatial resolution data set to obtain first connected data, and performing dynamic group convolution on the first connected data to obtain a first feature data set;
[0017] Connecting the data in the second spatial resolution data set to obtain second connected data, and performing dynamic group convolution on the second connected data to obtain an initial feature set;
[0018] Matching the initial feature set to the first spatial resolution data set using adaptive pooling to generate a second feature data set;
[0019] Convolving the data in the third spatial resolution data set to obtain convolution features, and matching the convolution features to the first spatial resolution data set using adaptive pooling to generate a third feature data set;
[0020] Connecting the first feature data set and the second feature data set, and performing dynamic group convolution to generate a fourth feature data set;
[0021] Connect the third feature data set and the fourth feature data set, and perform dynamic group convolution to obtain a fifth feature data set;
[0022] The multimodal fusion result is obtained by performing convolution transposition and back propagation on the fifth feature data set.
[0023] Optionally, preprocess the sea ice dataset, including:
[0024] The data in the sea ice dataset are projected onto the polar grid and the spatial range is unified to obtain a preprocessed sea ice dataset.
[0025] Optionally, the data in the sea ice dataset is projected onto a polar grid using the rasterio library of Python, and the spatial range is unified to obtain a preprocessed sea ice dataset.
[0026] Optionally, the polar grid for the South Pole is EPSG3412; the spatial extent of the South Pole is [West: -3950000m, East: 3950000m, North: -3950000m, South: 4350000m]; the polar grid for the North Pole is EPSG3411; the spatial extent of the North Pole is [West: -3850000m, East: 3750000m, North: -5350000m, South: 5850000m].
[0027] Optionally, the parameter data set includes: the parameter data set includes: pre-processed Bremen-AMSR sea ice density and first grid spacings of the Arctic and Antarctic;
[0028] The first spatial resolution dataset includes the preprocessed NSIDC-SSMIS sea ice density, the preprocessed OSISAF-SSMIS sea ice density, the preprocessed OSISAF-AMSR sea ice density and the preprocessed OISST sea ice density and the second grid spacing of the Arctic and Antarctic;
[0029] The second spatial resolution dataset includes the pre-processed OISAF sea ice extent and the pre-processed OISAF sea ice type and the third grid spacing for the Arctic and Antarctic;
[0030] The third spatial resolution dataset includes the pre-processed IMS sea surface types and a fourth grid spacing for the Arctic and Antarctic.
[0031] Optionally, the spatial resolution of the first spatial resolution dataset is greater than the spatial resolution of the second spatial resolution dataset; the spatial resolution of the second spatial resolution dataset is greater than the spatial resolution of the third spatial resolution dataset; and the spatial resolution of the first spatial resolution dataset is 25km; the spatial resolution of the second spatial resolution dataset is 10km; and the spatial resolution of the third spatial resolution dataset is 4km.
[0032] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multimodal fusion method for sea ice extent provided above.
[0033] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned sea ice extent multimodal fusion method.
[0034] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned sea ice extent multimodal fusion method.
[0035] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0036] The present application provides a sea ice range multimodal fusion method, equipment, medium and product. By grouping the preprocessed sea ice data set, the spatial resolution difference problem in the fusion process of multi-source sea ice products is taken into account. The grouping results are input into the multimodal data fusion model to obtain the multimodal fusion results. The sea ice parameters in different forms can be effectively fused, and at the same time, the accuracy and reliability of sea ice monitoring can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0038] Figure 1 This is an application environment diagram of a sea ice range multi-modal fusion method in one embodiment of the present application;
[0039] Figure 2 A schematic diagram of a flow chart of a multi-modal fusion method for sea ice extent provided in one embodiment of the present application;
[0040] Figure 3 A flowchart of an implementation of a sea ice range multi-modal fusion method provided in an embodiment of the present application;
[0041] Figure 4 A schematic diagram of a dynamic group convolution process provided by another embodiment of the present application;
[0042] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0044] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0045] Multimodal fusion is a technology that integrates and analyzes data from different modalities or multiple sources to improve overall information understanding and decision-making efficiency. Multimodal fusion technology is widely used in many scientific and engineering fields (such as remote sensing, machine learning, computer vision, medical imaging, and human-computer interaction).
[0046] Separable dynamic convolutional neural network (SepDGConv) is an efficient and scalable multi-source remote sensing data fusion framework based on deep convolutional neural network. Its characteristics are: it can contain multi-stream branches with differences between input data groups (Yang et al., 2022). SepDGConv first groups the input data by type (such as sensor type, data type, spatial resolution, and temporal resolution), performs convolution within each group, and uses the relationship matrix to retain the characteristics between groups, which is a separable convolution. Among them, the relationship matrix can be used to model the input-output channel mapping by introducing a binary matrix. Unlike previous dense convolutions, SepDGConv can fuse the characteristics of different source data at an early stage and improve the generalization ability of the model. In this context, the use of SepDGConv to integrate various sea ice parameters can comprehensively utilize the advantages of different parameters and data sources, make up for the shortcomings of a single data source, and improve the accuracy and reliability of sea ice monitoring.
[0047] Based on the above description, the present application embodiment provides a sea ice range multi-modal fusion method, which can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the sea ice data set to the server 104, and after the server 104 receives the sea ice data set, it pre-processes the sea ice data set. The pre-processed sea ice data set is grouped to obtain a grouping result. The grouping result is input into the constructed multimodal data fusion model to obtain a multimodal fusion result. The server 104 can feed back the obtained multimodal fusion result to the terminal 102. In addition, in some embodiments, the sea ice range multimodal fusion method can also be implemented separately by the server 104 or the terminal 102, such as the terminal 102 can directly perform multimodal fusion on the sea ice data set, or the server 104 can obtain the sea ice data set from the data storage system and perform multimodal fusion on the sea ice data set.
[0048] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.
[0049] In an exemplary embodiment, Figure 2 As shown, a method for multimodal fusion of sea ice extent is provided. The method is executed by a computer device, and specifically can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following steps 201 to 203. Among them:
[0050] Step 200: Preprocess the sea ice dataset. The sea ice dataset can be downloaded from relevant websites, as shown in Table 1, which may include: NSIDC-SSMIS sea ice density, OSISAF-SSMIS sea ice density, OSISAF-AMSR sea ice density, Bremen-AMSR sea ice density, IMS sea surface type, OISAF sea ice extent, OSISAF sea ice type, OISST sea ice density.
[0051] Table 1 List of gridded satellite sea ice datasets with integrated sea ice density
[0052]
[0053]
[0054] Furthermore, in actual application, the rasterio library of Python can be used to project the data in the sea ice dataset to the polar grid and unify the spatial range to obtain the preprocessed sea ice dataset. For example, the polar grid of the Antarctic is EPSG3412. The spatial range of the Antarctic is [West: -3950000m, East: 3950000m, North: -3950000m, South: 4350000m]; the polar grid of the Arctic is EPSG3411; the spatial range of the Arctic is [West: -3850000m, East: 3750000m, North: -5350000m, South: 5850000m].
[0055] Step 201: grouping the preprocessed sea ice dataset to obtain a grouping result, which includes a parameter dataset, a first spatial resolution dataset, a second spatial resolution dataset, and a third spatial resolution dataset.
[0056] For example, the preprocessed Bremen-AMSR sea ice density can be divided into a parameter data set. In this data set, the grid spacing of the Arctic and Antarctic is the first grid spacing, for example, 3.125 kilometers.
[0057] The preprocessed NSIDC-SSMIS sea ice density, the preprocessed OSISAF-SSMIS sea ice density, the preprocessed OSISAF-AMSR sea ice density, and the preprocessed OISST sea ice density are divided into the first spatial resolution dataset. The first spatial resolution dataset is recorded as Group 1, and its spatial resolution is uniformly 25 km. The grid spacing of the Arctic and Antarctic in this dataset is the second grid spacing, for example, 25 km.
[0058] The preprocessed OISAF sea ice extent and preprocessed OISAF sea ice type are divided into the second spatial resolution dataset. The second spatial resolution dataset is recorded as Group2, and its spatial resolution is uniformly 10km. The grid spacing of the Arctic and Antarctic in this dataset is the third grid spacing, for example, 10km.
[0059] The preprocessed IMS sea surface types are divided into the third spatial resolution dataset. The third spatial resolution dataset is recorded as Group 3, and its spatial resolution is uniformly 4 km. The grid spacing of the North Pole and the South Pole in this dataset is the fourth grid spacing, for example, 4 km.
[0060] In actual application, the spatial resolution of the first spatial resolution data set is greater than the spatial resolution of the second spatial resolution data set. The spatial resolution of the second spatial resolution data set is greater than the spatial resolution of the third spatial resolution data set.
[0061] Step 202: Construct a multimodal data fusion model.
[0062] Step 203: Input the grouping result into the multimodal data fusion model to obtain a multimodal fusion result.
[0063] By implementing the above-mentioned steps 201 to 203, the present application can effectively fuse sea ice parameters in different forms, and at the same time, take into account the difference in spatial resolution in the fusion process of multi-source sea ice products, thereby effectively improving the accuracy and reliability of sea ice monitoring.
[0064] In another exemplary embodiment of the present application, Figure 3 As shown, in step 203, the process of performing dynamic convolution according to the grouping situation includes:
[0065] Step 1: Connect the data in the first spatial resolution data set to obtain first connected data, and perform dynamic group convolution on the first connected data to obtain a first feature data set, wherein the first feature data set is recorded as Feature1.
[0066] For example, the four data sets of Group1 (preprocessed NSIDC-SSMIS sea ice density, preprocessed OSISAF-SSMIS sea ice density, preprocessed OSISAF-AMSR sea ice density, and preprocessed OISST sea ice density) are connected and dynamic group convolution is performed to generate the first feature data set Feature1.
[0067] Step 2: Connect the data in the second spatial resolution dataset to obtain the second connected data, and perform dynamic group convolution on the second connected data to obtain the initial feature set. Use adaptive pooling to match the initial feature set to the first spatial resolution dataset to generate a second feature dataset. The second feature dataset is recorded as Feature2.
[0068] For example, the two datasets of Group2 (preprocessed OISAF sea ice extent and preprocessed OISAF sea ice type) are connected, dynamic group convolution is performed, and adaptive pooling is used to match their size to Group1 to generate the second feature dataset Feature2.
[0069] Step 3: Convolve the data in the third spatial resolution data set to obtain convolution features, and use adaptive pooling to match the convolution features to the first spatial resolution data set to generate a third feature data set, where the third feature data set is recorded as Feature3.
[0070] For example, convolution is performed on one dataset of Group3 (preprocessed IMS sea surface type), and its size is matched to Group1 using adaptive pooling to generate the third feature dataset Feature3.
[0071] Step 4: Connect the first feature data set Feature1 and the second feature data set Feature2, and perform dynamic group convolution to generate a fourth feature data set, wherein the fourth feature data set is recorded as Feature4.
[0072] Step 5: Connect the third feature data set Feature3 and the fourth feature data set Feature4, and perform dynamic group convolution to obtain a fifth feature data set, where the fifth feature data set is recorded as Feature.
[0073] Step 6: Perform convolution transposition and back propagation on the fifth feature data set Feature to obtain the multimodal fusion result.
[0074] In the implementation process, the purpose of convolution transposition of the fifth feature dataset Feature is to generate target data so as to use transposed convolution to adjust the size to make it the same as the target data. Finally, the multimodal fusion result obtained by backpropagation is the sea ice density, which can be expressed as a decimal between 0 and 1. For example, 0-0.4 represents sea water, 0.4-0.7 represents fast drift ice, and 0.7-1 represents dense drift ice.
[0075] Furthermore, in this embodiment, the purpose of the connection is to stitch images of the same size from different data sources on channels to form a multi-channel image.
[0076] Furthermore, in actual application, Figure 3 As shown in FIG. 1 , the preprocessed sea ice dataset can also be grouped and processed according to spatial resolutions of 25 km, 3.125 km, and 1 km, and is not limited to the grouping method provided in this application. Users can make adaptive adjustments according to actual needs. The processing process of dynamic group convolution is shown in FIG. Figure 4 shown. Figure 4 The first, second and third groups correspond to Figure 3 Shown are the first group (a first spatial resolution data set with a spatial resolution of 25 km), the second group (a second spatial resolution data set with a spatial resolution of 3.125 km), and the third group (a third spatial resolution data set with a spatial resolution of 1 km).
[0077] In summary, compared with the prior art, the sea ice range multimodal fusion method provided by this application has the following advantages:
[0078] 1. Ability to process the multimodal characteristics of different data sources, provide more accurate and comprehensive sea ice distribution information, and achieve efficient fusion of multimodal data. This application can efficiently integrate various forms of sea ice parameters, including sea ice density, sea ice type, and sea ice extent.
[0079] 2. Improve fusion accuracy. This application makes full use of the advantages of deep learning and improves the accuracy of fusion results by learning complex nonlinear relationships.
[0080] 3. Adaptive spatial resolution processing. This application can process sea ice data of different spatial resolutions. Through the multi-level feature extraction and fusion mechanism in the network, it can realize adaptive processing of data of different resolutions, avoiding the limitation of using coarse resolution as reference in traditional methods.
[0081] 4. Reduce computational costs. This application automatically learns and simplifies traditional dense convolution, which can significantly reduce computational costs and memory usage, and maintain high computational efficiency when processing high-resolution data.
[0082] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store sea ice range multimodal fusion processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a sea ice range multimodal fusion method is implemented.
[0083] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0084] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0085] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0087] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0088] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.
[0089] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0090] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A multi-modal fusion method for sea ice extent, characterized in that: The sea ice extent multimodal fusion method comprises: Preprocessing the sea ice data set; the sea ice data set includes: NSIDC-SSMIS sea ice density, OSISAF-SSMIS sea ice density, OSISAF-AMSR sea ice density, Bremen-AMSR sea ice density, IMS sea surface type, OISAF sea ice extent, OSISAF sea ice type, OISST sea ice density; The preprocessed sea ice data set is grouped to obtain a grouping result; the grouping result includes: a parameter data set, a first spatial resolution data set, a second spatial resolution data set, and a third spatial resolution data set; Construct a multimodal data fusion model; The grouping result is input into the multimodal data fusion model to obtain a multimodal fusion result.
2. The sea ice extent multimodal fusion method according to claim 1, characterized in that: Inputting the grouping result into the multimodal data fusion model to obtain a multimodal fusion result specifically includes: Connecting the data in the first spatial resolution data set to obtain first connected data, and performing dynamic group convolution on the first connected data to obtain a first feature data set; Connecting the data in the second spatial resolution data set to obtain second connected data, and performing dynamic group convolution on the second connected data to obtain an initial feature set; Matching the initial feature set to the first spatial resolution data set using adaptive pooling to generate a second feature data set; Convolving the data in the third spatial resolution data set to obtain convolution features, and matching the convolution features to the first spatial resolution data set using adaptive pooling to generate a third feature data set; Connecting the first feature data set and the second feature data set, and performing dynamic group convolution to generate a fourth feature data set; Connect the third feature data set and the fourth feature data set, and perform dynamic group convolution to obtain a fifth feature data set; The multimodal fusion result is obtained by performing convolution transposition and back propagation on the fifth feature data set.
3. The sea ice range multimodal fusion method according to claim 1, characterized in that: Preprocess the sea ice dataset, including: The data in the sea ice dataset are projected onto the polar grid and the spatial range is unified to obtain a preprocessed sea ice dataset.
4. The sea ice range multimodal fusion method according to claim 3, characterized in that: The data in the sea ice dataset are projected onto the polar grid using the python rasterio library, and the spatial range is unified to obtain the preprocessed sea ice dataset.
5. The sea ice range multimodal fusion method according to claim 3, characterized in that: The polar grid of the South Pole is EPSG3412; the spatial extent of the South Pole is [West: -3950000m, East: 3950000m, North: -3950000m, South: 4350000m]; the polar grid of the North Pole is EPSG3411; the spatial extent of the North Pole is [West: -3850000m, East: 3750000m, North: -5350000m, South: 5850000m].
6. The sea ice extent multimodal fusion method according to claim 1, characterized in that: The parameter data set includes: pre-processed Bremen-AMSR sea ice density and the first grid spacing of the Arctic and Antarctic; The first spatial resolution dataset includes the preprocessed NSIDC-SSMIS sea ice density, the preprocessed OSISAF-SSMIS sea ice density, the preprocessed OSISAF-AMSR sea ice density and the preprocessed OISST sea ice density and the second grid spacing of the Arctic and Antarctic; The second spatial resolution dataset includes the pre-processed OISAF sea ice extent and the pre-processed OISAF sea ice type and the third grid spacing for the Arctic and Antarctic; The third spatial resolution dataset includes the pre-processed IMS sea surface types and a fourth grid spacing for the Arctic and Antarctic.
7. The sea ice extent multimodal fusion method according to claim 1, characterized in that: The spatial resolution of the first spatial resolution data set is greater than the spatial resolution of the second spatial resolution data set; The spatial resolution of the second spatial resolution data set is greater than the spatial resolution of the third spatial resolution data set; Furthermore, the spatial resolution of the first spatial resolution data set is 25 km; The spatial resolution of the second spatial resolution data set is 10 km; The spatial resolution of the third spatial resolution dataset is 4 km.
8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multimodal fusion method for sea ice extent described in any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the sea ice extent multimodal fusion method described in any one of claims 1 to 7 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the sea ice extent multimodal fusion method described in any one of claims 1 to 7 is implemented.
Citation Information
Cited By
Method for simulating and generating sea ice area on sea surface
CN119648892A
Multi-source fusion sea ice concentration acquisition method, device, equipment and medium
CN120451952A
Arctic atmosphere coupling forecasting method capable of automatically fusing sea ice concentration and thickness
CN120579146A
