Unmanned aerial vehicle optical and thermal infrared data sea ice classification and density inversion method

Abstract

The invention relates to an unmanned aerial vehicle optical and thermal infrared data sea ice classification and intensity inversion method, and the method comprises the steps: firstly obtaining registered unmanned aerial vehicle optical and thermal infrared images through the preprocessing technologies of orbit correction, radiometric calibration, geometric correction and the like, and providing high-quality data for the subsequent sea ice information extraction; on the basis, an Ice-Unet deep learning model is constructed, and a high-precision sea ice classification result is obtained; and then selecting surface temperature values corresponding to an open water area and fixed ice in the sea ice classification to construct a system point value of the thermal infrared data, and constructing a linear inversion model based on a radiation transmission model so as to obtain a sea ice concentration inversion result based on the thermal infrared data. Therefore, the problems of difficult realization of rapid reporting of sea ice information in the north pole field seawater and field guarantee in the prior art are solved.

Description

Technical Field

[0001] This application relates to the technical field of remote sensing image processing, and particularly relates to a method for classifying sea ice and retrieving its concentration from unmanned aerial vehicle optical and thermal infrared data. Background Art

[0002] As an important part of the marine environment, Arctic sea ice accounts for about 2% of the global ocean area. On the one hand, the rapid change of polar sea ice affects the heat and water balance of the ocean and is an important monitoring factor for global climate change; on the other hand, the spatio-temporal distribution of polar sea ice also affects the route planning of ship navigation and the development of polar resources, which is a current research hotspot. By monitoring and analyzing information such as the distribution, concentration, and thickness of sea ice in real time, the ice conditions in the waterway can be better understood and predicted, and corresponding navigation strategies can be adopted, which can not only ensure navigation safety, but also the obtained sea ice observation data can better understand the ecological and environmental changes in the Arctic region, and then predict the impact of these changes on the global climate and ecosystem. Therefore, Arctic sea ice monitoring has important scientific and economic value.

[0003] In terms of sea ice classification, hyperspectral data can be used to construct a complete sea ice feature extraction and analysis framework by means of spectral-spatial-joint features to obtain high-precision sea ice category information. High-resolution SAR (Synthetic Aperture Radar) data can be used to obtain fine sea ice distribution information. At the same time, combined with traditional sea ice concentration products, a richer multi-scale ice condition map can be constructed to support polar sea ice monitoring and route optimization. In terms of the selection of classification models, they can be mainly divided into support vector machines, Bayesian algorithms, neural networks, principal component analysis, etc. Among them, the support vector machine combines texture feature analysis, uses the gray-level co-occurrence matrix to extract eigenvalue, improving the accuracy and reliability of classification. However, the training time increases significantly with the increase of data volume. The selection of kernel function and regularization parameters has a great impact on the results. The tuning process is complex, and the efficiency is low when dealing with multi-classification problems. The Bayesian algorithm calculates the posterior probability of each category by using Bayes' formula through statistically calculating the prior probability and conditional probability of different categories, and selects the category with the maximum posterior probability as the classification result. Naive Bayes assumes that feature conditions are independent, while the spectral, texture and spatial features of sea ice are often highly correlated, and this algorithm is sensitive to the prior distribution and cannot handle complex distributions. The neural network is a classifier based on neuron simulation, suitable for dealing with complex non-linear problems. However, it is prone to overfitting when the training samples are insufficient or the data noise is large, and the selection of hyperparameters has a great impact on the results. The tuning process is complex and time-consuming. Principal component analysis compresses high-dimensional spectral or texture data into a low-dimensional space to extract main features. However, data dimensionality reduction may lead to the loss of some useful information, affecting the classification effect and the interpretability of the model. And principal component analysis assumes that the data is linearly structured and is difficult to handle complex non-linear relationships.

[0004] In terms of sea ice concentration inversion, current research mainly includes methods based on passive microwave concentration inversion algorithms, optical data inversion algorithms, and multi-source data fusion, etc., which are specifically described as follows:

[0005] 1) Single-frequency inversion method based on polarization ratio. For example, the Bootstrap method uses the polarization ratio information of the brightness temperature in the vertical and horizontal directions at 19 GHz or 37 GHz for concentration inversion. However, the polarization ratio has a weak response to snow cover and melting ice, and large errors are likely to occur during the summer melting period. Secondly, the resolution of passive microwave brightness temperature is usually on the order of dozens of kilometers, making it difficult to reflect the small-scale sea ice distribution.

[0006] 2) Single-polarization method based on frequency gradient. By describing the distribution of sea ice and seawater within the resolution unit through the electromagnetic scattering differences of microwaves at different frequencies for concentration inversion, such as the CalVal and NORSEX algorithms. These algorithms rely on empirical models, and the thresholds of frequency gradient and model parameters need to be adjusted according to regions and seasons, with limited generality.

[0007] 3) Intensity inversion algorithms that comprehensively consider polarization ratio and frequency gradient, such as the ASI (ARTIST Sea Ice, Arctic Radiation and Turbulent Exchange Study on Sea Ice) algorithm and the NASATeam algorithm. These methods have a high algorithm complexity, require simultaneous processing of data from multiple channels, and thus have a high computational cost.

[0008] 4) Intensity inversion algorithms based on multi-channel data;

[0009] 5) Sea ice concentration inversion algorithms based on multi-source data fusion. For example, the sea ice concentration inversion method based on a neural network model uses features such as passive microwave polarization ratio and SAR backscattering to establish a neural network model, constructs hidden layer units of the network based on feature fusion, performs iterative updates through error backpropagation, and uses the maximum likelihood constraint criterion to obtain the intensity inversion results of the time series.

[0010] In summary, the existing technologies still have problems such as the trade-off between resolution and coverage, poor adaptability to complex environments, and insufficient model generality.

[0011] With the continuous progress and update of detection means, unmanned aerial vehicles (UAVs) have gradually been applied to the research of sea ice at home and abroad. Compared with other traditional satellite remote sensing, aerial remote sensing, ship-based and helicopter observations, etc., satellite remote sensing can cover a wider area, but its data acquisition usually requires waiting for the satellite to pass by, and effective data may not be obtained under adverse weather conditions. In contrast, UAVs can provide real-time data, reduce the dependence on satellite and aerial remote sensing data, thereby reducing costs and improving the timeliness of data acquisition. In terms of spatial resolution and data accuracy, UAVs can provide a spatial resolution of up to sub-meter level, which enables them to identify and measure the physical properties of sea ice in detail, such as ice surface roughness and melt pond information. In addition, UAVs can be quickly deployed, are flexible in operation, can enter inaccessible areas, and provide first-hand data. Although the operation of UAVs is greatly restricted by weather conditions, such as strong winds and adverse weather may limit their flight ability, UAVs can provide more detailed local data and can ensure its continuity through more frequent flight missions.

[0012] However, the existing technologies are difficult to achieve rapid reporting and on-site guarantee of sea ice information in the Arctic seawater, which urgently needs to be solved. Summary of the Invention

[0013] The present application provides a method for classifying and inverting the concentration of sea ice from UAV optical and thermal infrared data to solve the problems that the existing technologies are difficult to achieve rapid reporting and on-site guarantee of sea ice information in the Arctic seawater, etc.

[0014] An embodiment of the first aspect of the present application provides a method for classifying sea ice and inverting ice concentration from optical and thermal infrared data of an unmanned aerial vehicle, including the following steps: obtaining the optical and thermal infrared data of a target unmanned aerial vehicle, and performing preprocessing and orthorectification stitching operations on the optical and thermal infrared data to obtain an orthorectified image corresponding to the optical and thermal infrared data; constructing a sea ice classification training dataset based on the orthorectified image and a preset SAM model, training a pre-constructed Ice-Unet model through the sea ice classification training dataset, and inputting the orthorectified image into the trained Ice-Unet model to output sea ice category information corresponding to the orthorectified image; determining target reference temperatures for target type waters and target type sea ice according to the sea ice category information, and constructing a sea ice concentration inversion model based on the target reference temperatures to quantitatively evaluate the inversion accuracy of the sea ice concentration using the sea ice concentration inversion model.

[0015] Optionally, in an embodiment of the present application, the obtaining the optical and thermal infrared data of a target unmanned aerial vehicle, and performing preprocessing and orthorectification stitching operations on the optical and thermal infrared data to obtain an orthorectified image corresponding to the optical and thermal infrared data includes: performing geometric correction and radiometric calibration operations on the optical and thermal infrared data to obtain corresponding calibration results; constructing a corresponding digital elevation model based on the calibration results, and performing orthorectification operations on the optical and thermal infrared data according to the digital elevation model to obtain corresponding orthorectified data; performing orthorectification stitching processing on the orthorectified data to obtain the orthorectified image.

[0016] Optionally, in an embodiment of the present application, the training the pre-constructed Ice-Unet model through the sea ice classification training dataset includes: constructing the Ice-Unet model based on a preset backbone feature extraction network, enhanced feature extraction network, and prediction network; inputting the training data in the sea ice classification training dataset into the backbone feature extraction network in the Ice-Unet model to perform multiple convolution and max pooling operations on the training data to generate multiple preliminary effective feature layers; inputting the multiple preliminary effective feature layers into the enhanced feature extraction network to perform upsampling and feature fusion processing on the multiple preliminary effective feature layers to obtain a target fusion effective feature layer; using a target convolutional layer to perform channel adjustment operations on the target fusion effective feature layer to obtain the trained Ice-Unet model.

[0017] Optionally, in an embodiment of the present application, determining a target reference temperature for a target type of water area and a target type of sea ice according to the sea ice category information, and constructing an sea ice concentration inversion model based on the target reference temperature to quantitatively evaluate the sea ice concentration inversion accuracy by using the sea ice concentration inversion model, including: determining a target pixel grid corresponding to each pixel point in the orthorectified image, and dividing the target pixel grid into a plurality of sub-grids; in each of the plurality of sub-grids, selecting the sea ice surface temperature at a target percentile, and determining a preliminary sea ice reference temperature for the target type of sea ice according to the sea ice surface temperature at the target percentile; determining a final reference temperature for the target type of sea ice based on the preliminary sea ice reference temperature and a preset linear regression strategy; performing multiple coverage operations on each pixel point based on a preset per-pixel sliding window strategy to obtain the final reference temperature of the target type of sea ice corresponding to each coverage operation, and calculating the target reference temperature of the target type of sea ice corresponding to each pixel point according to the final reference temperature of the target type of sea ice corresponding to each coverage operation; calculating the sea ice concentration corresponding to each pixel according to the target reference temperature of the target type of water area, the target reference temperature of the target type of sea ice, and the sea ice concentration inversion model, and projecting the sea ice concentration to obtain a corresponding projection result, and quantitatively evaluating the sea ice concentration inversion accuracy through the projection result.

[0018] Optionally, in an embodiment of the present application, the mathematical expression of the sea ice concentration inversion model is:

[0019]

[0020] where SIC represents the sea ice concentration; t pwate represents the target reference temperature of the target type of water area; t pice represents the target reference temperature of the target type of sea ice; IST represents the sea ice surface temperature.

[0021] The second aspect of the present application provides an apparatus for classifying sea ice and inverting ice concentration from optical and thermal infrared data of an unmanned aerial vehicle, including: a preprocessing module, configured to obtain optical and thermal infrared data of a target unmanned aerial vehicle, and perform preprocessing and orthorectification stitching operations on the optical and thermal infrared data to obtain an orthorectified image corresponding to the optical and thermal infrared data; a classification module, configured to construct a sea ice classification training dataset based on the orthorectified image and a preset SAM model, train a pre-constructed Ice-Unet model through the sea ice classification training dataset, and input the orthorectified image into the trained Ice-Unet model to output sea ice category information corresponding to the orthorectified image; an inversion module, configured to determine target reference temperatures of a target type of water area and a target type of sea ice according to the sea ice category information, and construct a sea ice concentration inversion model based on the target reference temperatures to quantitatively evaluate the inversion accuracy of the sea ice concentration by using the sea ice concentration inversion model.

[0022] Optionally, in an embodiment of the present application, the preprocessing module includes: a radiometric calibration unit, configured to perform geometric calibration and radiometric calibration operations on the optical and thermal infrared data to obtain corresponding calibration results; an orthorectification unit, configured to construct a corresponding digital elevation model based on the calibration results, and perform orthorectification operations on the optical and thermal infrared data according to the digital elevation model to obtain corresponding orthorectified data; an orthorectification stitching unit, configured to perform orthorectification stitching processing on the orthorectified data to obtain the orthorectified image.

[0023] Optionally, in an embodiment of the present application, the classification module includes: a modeling unit, configured to construct the Ice-Unet model based on a preset backbone feature extraction network, an enhanced feature extraction network, and a prediction network; a backbone feature extraction unit, configured to input training data in the sea ice classification training dataset into the backbone feature extraction network in the Ice-Unet model to perform multiple convolution and max-pooling operations on the training data to generate multiple preliminary effective feature layers; an enhanced feature extraction unit, configured to input the multiple preliminary effective feature layers into the enhanced feature extraction network to perform upsampling and feature fusion processing on the multiple preliminary effective feature layers to obtain a target fusion effective feature layer; a prediction unit, configured to perform channel adjustment operations on the target fusion effective feature layer by using a target convolutional layer to obtain the trained Ice-Unet model.

[0024] Optionally, in an embodiment of the present application, the inversion module includes: a division unit configured to determine a target pixel grid corresponding to each pixel point in the orthorectified image and divide the target pixel grid into a plurality of sub-grids; a selection unit configured to select a target percentile of the sea ice surface temperature in each of the plurality of sub-grids and determine a preliminary sea ice reference temperature of the target type of sea ice according to the target percentile of the sea ice surface temperature; a determination unit configured to determine a final reference temperature of the target type of sea ice based on the preliminary sea ice reference temperature and a preset linear regression strategy; a coverage unit configured to perform multiple coverage operations on each pixel point based on a preset per-pixel sliding window strategy to obtain a final reference temperature of the target type of sea ice corresponding to each coverage operation, and calculate a target reference temperature of the target type of sea ice corresponding to each pixel point according to the final reference temperature of the target type of sea ice corresponding to each coverage operation; a quantitative evaluation unit configured to calculate the sea ice concentration corresponding to each pixel according to the target reference temperature of the target type of water area and the target type of sea ice and the sea ice concentration inversion model, project the sea ice concentration, obtain a corresponding projection result, and quantitatively evaluate the inversion accuracy of the sea ice concentration through the projection result.

[0025] Optionally, in an embodiment of the present application, the mathematical expression of the sea ice concentration inversion model is:

[0026]

[0027] where SIC represents the sea ice concentration; t pwate represents the target reference temperature of the target type of water area; t pice represents the target reference temperature of the target type of sea ice; IST represents the sea ice surface temperature.

[0028] An embodiment of the third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the method for classifying and inverting the sea ice concentration from the optical and thermal infrared data of the drone as described in the above embodiment.

[0029] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the program is executed by a processor, it implements the method for classifying and inverting the sea ice concentration from the optical and thermal infrared data of the drone as described above.

[0030] An embodiment of the fifth aspect of the present application provides a computer program product, including a computer program, where the computer program is executed to implement the method for classifying and inverting the sea ice concentration from the optical and thermal infrared data of the drone as described above.

[0031] Accordingly, the embodiments of the present application have the following beneficial effects:

[0032] The embodiments of the present application can obtain the optical and thermal infrared data of the target unmanned aerial vehicle (UAV), and perform preprocessing and orthorectification stitching operations on the optical and thermal infrared data to obtain the orthorectified images corresponding to the optical and thermal infrared data; based on the orthorectified images and a preset SAM model, construct a sea ice classification training dataset, and train a pre-constructed Ice-Unet model through the sea ice classification training dataset, and input the orthorectified images into the trained Ice-Unet model to output the sea ice category information corresponding to the orthorectified images; determine the target reference temperatures of the target type water area and the target type sea ice according to the sea ice category information, and based on the target reference temperatures, construct a sea ice concentration inversion model to quantitatively evaluate the sea ice concentration inversion accuracy by using the sea ice concentration inversion model, so as to obtain high-precision sea ice category and concentration data, and be able to achieve simple route planning, providing reliable reference data for regional Arctic sea ice changes. Accordingly, the problems in the prior art that it is difficult to achieve rapid reporting of sea ice information in Arctic in-situ seawater and on-site guarantee are solved.

[0033] Some of the additional aspects and advantages of the present application will be given in the following description, some will become obvious from the following description, or be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and / or additional aspects and advantages of the present application will become obvious and easy to understand from the following description of the embodiments in conjunction with the drawings, wherein:

[0035] Figure 1 FIG. is a flowchart of a method for classifying sea ice and inverting sea ice concentration using optical and thermal infrared data of an unmanned aerial vehicle according to an embodiment of the present application;

[0036] Figure 2 FIG. is an execution logic diagram of a method for classifying sea ice and inverting sea ice concentration using optical and thermal infrared data of an unmanned aerial vehicle provided by an embodiment of the present application;

[0037] Figure 3 FIG. is a schematic diagram of an ISAT-SAM dataset annotation software and annotation results provided by an embodiment of the present application;

[0038] Figure 4 FIG. is a schematic diagram of an Ice-Unet sea ice classification model provided by an embodiment of the present application;

[0039] Figure 5 FIG. is a schematic diagram of sea ice classification results of different methods provided by an embodiment of the present application;

[0040] Figure 6 Schematic diagram for verifying sea ice results of different methods provided by an embodiment of the present application;

[0041] Figure 7 Schematic diagram of sea ice concentration inversion results provided by an embodiment of the present application;

[0042] Figure 7 Among them, (a) is a schematic diagram of optical data provided by an embodiment of the present application;

[0043] Figure 7 Among them, (b) is a schematic diagram of thermal infrared data provided by an embodiment of the present application;

[0044] Figure 7 Among them, (c) is a schematic diagram of sea ice concentration inversion results of optical data provided by an embodiment of the present application;

[0045] Figure 7 Among them, (d) is a schematic diagram of sea ice concentration inversion results of thermal infrared data provided by an embodiment of the present application;

[0046] Figure 8 Schematic diagram for verifying sea ice concentration results provided by an embodiment of the present application;

[0047] Figure 8 Among them, (a) is a schematic diagram of comparison between optical and ASI concentrations provided by an embodiment of the present application;

[0048] Figure 8 Among them, (b) is a schematic diagram of comparison between thermal infrared and ASI concentrations provided by an embodiment of the present application;

[0049] Figure 8 Among them, (c) is a schematic diagram of comparison between optical and thermal infrared concentrations provided by an embodiment of the present application;

[0050] Figure 9 Schematic diagram of an unmanned aerial vehicle optical and thermal infrared data sea ice classification and concentration inversion device according to an embodiment of the present application;

[0051] Figure 10 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application.

[0052] Among them, 10 - unmanned aerial vehicle optical and thermal infrared data sea ice classification and concentration inversion device; 100 - preprocessing module, 200 - classification module, 300 - inversion module; 1001 - memory, 1002 - processor, 1003 - communication interface. Detailed implementation manners

[0053] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application and should not be construed as limiting the present application.

[0054] The following describes a method for classifying sea ice and retrieving ice concentration from unmanned aerial vehicle (UAV) optical and thermal infrared data according to an embodiment of the present application. In view of the problems mentioned in the above background art, the present application provides a method for classifying sea ice and retrieving ice concentration from UAV optical and thermal infrared data. In this method, by acquiring the optical and thermal infrared data of the target UAV and performing preprocessing and orthorectification stitching operations on the optical and thermal infrared data, an orthorectified image corresponding to the optical and thermal infrared data is obtained; based on the orthorectified image and a preset SAM model, a sea ice classification training dataset is constructed, and a pre-constructed Ice-Unet model is trained through the sea ice classification training dataset, and the orthorectified image is input into the trained Ice-Unet model to output the sea ice category information corresponding to the orthorectified image; according to the sea ice category information, the target reference temperatures of the target type water area and the target type sea ice are determined, and based on the target reference temperatures, an ice concentration inversion model is constructed to quantitatively evaluate the ice concentration inversion accuracy by using the ice concentration inversion model, so that high-precision sea ice category and concentration data can be obtained, simple route planning can be realized, and reliable reference data for regional Arctic sea ice changes is provided. Thus, the problems in the prior art that it is difficult to realize rapid reporting of sea ice information in Arctic field sea water and on-site guarantee are solved.

[0055] Specifically, Figure 1 FIG. is a flowchart of a method for classifying sea ice and retrieving ice concentration from UAV optical and thermal infrared data according to an embodiment of the present application.

[0056] As Figure 1 shown, the method for classifying sea ice and retrieving ice concentration from UAV optical and thermal infrared data includes the following steps:

[0057] In step S101, the optical and thermal infrared data of the target UAV are acquired, and preprocessing and orthorectification stitching operations are performed on the optical and thermal infrared data to obtain an orthorectified image corresponding to the optical and thermal infrared data.

[0058] Embodiments of the present application first need to acquire remote sensing data (i.e., optical and thermal infrared data) of UAVs and satellites, and improve the reliability of the data through orthorectification, geometric correction, and radiometric calibration. In view of the influence of geometric factors such as flight trajectories and terrain undulations on UAV and remote sensing satellite images, terrain correction is performed on UAV images based on ground control points and vehicle poses to obtain orthoimages.

[0059] In the actual implementation process, the embodiments of the present application can perform radiometric calibration on the infrared images of the UAV, and use the radiative transfer equation to eliminate the influence of the absorption and scattering of the radiative signal between the ground object and the sensor, calculate the true surface temperature, and at the same time correct the measurement error caused by the change of the environmental temperature and humidity of the sensor by combining the placed standard radiation source.

[0060] For the problem of spatial misalignment of multi-source data, the embodiments of the present application can perform rough spatial alignment based on the high-precision pose data of the satellite and the UAV combined with the preprocessed orthoimage; then, based on the computer vision strategy, extract the homologous points on the multi-modal data to construct feature matching to find the geometric transformation model between the multi-source images, and map the multi-source images to the same spatial coordinate system based on the transformation model; in addition, the embodiments of the present application also need to perform local registration fine-tuning based on the mutual information registration principle to obtain the registered UAV optical and thermal infrared images, so as to provide high-quality data for the subsequent extraction of sea ice information.

[0061] Optionally, in an embodiment of the present application, obtaining the optical and thermal infrared data of the target UAV, and performing preprocessing and orthoimage stitching operations on the optical and thermal infrared data to obtain the orthorectified images corresponding to the optical and thermal infrared data, including: performing geometric correction and radiometric calibration operations on the optical and thermal infrared data to obtain the corresponding calibration results; based on the calibration results, constructing the corresponding digital elevation model, and performing orthorectification operations on the optical and thermal infrared data according to the digital elevation model to obtain the corresponding orthorectified data; performing orthoimage stitching processing on the orthorectified data to obtain the orthorectified image.

[0062] It should be noted that the specific steps of the embodiments of the present application for preprocessing and orthoimage stitching operations on the optical and thermal infrared data are described as follows:

[0063] Step 1, Geometric correction

[0064] First, the embodiments of the present application can establish a parameter model of the ground control points and the image sensor based on the ground control points arranged for on-site observation of sea ice, and determine the accurate position of the image pixels in the geographical space; secondly, the embodiments of the present application can obtain the exterior orientation parameters of the camera, such as position and attitude angles, through the GPS and IMU data of the UAV, so as to establish an accurate relationship between the image and the ground coordinates; finally, the embodiments of the present application can eliminate the offset error caused by the terrain undulation through terrain correction;

[0065] In the specific implementation process, the embodiments of the present application can select a second-order polynomial to implement the coordinate transformation between the original image and the corrected image; after the coordinate transformation, the position of the pixel center usually changes. Therefore, the embodiments of the present application need to resample the original image according to certain rules based on the position of each pixel in the output image in the original image, and establish a new raster matrix by recalculating the raster values; the embodiments of the present application can select the bilinear interpolation method for resampling to assign gray values to the pixels of the output image of the distorted image to achieve the geometric correction of the UAV image data and be used for subsequent sea ice feature extraction and parameter inversion;

[0066] Step 2: Radiometric calibration

[0067] The radiometric calibration of the UAV image is to convert the original digital quantization value (DN value) into radiance, which is described as follows:

[0068] (1) The thermal infrared sensor carried by the UAV senses the infrared photons radiated by the ground object, and based on the calibration coefficients of the sensor, including the radiation gain and offset, converts them into electrical signals and quantifies them into DN values;

[0069] (2) Using the radiative transfer equation, eliminate the influence of the absorption and scattering of the radiation signal between the ground object and the sensor to obtain the radiance of the sea ice surface;

[0070] (3) Based on the radiative transfer model, convert the radiance into brightness temperature, calculate the true surface temperature, and combine the standard radiation source placed during the on-site observation to correct the measurement error caused by the changes in the environmental temperature and humidity of the sensor, and output the final calibration result;

[0071] Step 3: Construct a DEM (Digital Elevation Model)

[0072] After obtaining the UAV data and performing Steps 1 and 2, the embodiments of the present application can select image pairs with sufficient overlapping areas, extract the feature points in the images using the SIFT algorithm in the feature extraction algorithm, find the homologous feature points in the adjacent images through the matching algorithm, and calculate the three-dimensional spatial coordinates of these feature points according to the principle of photogrammetry, so as to obtain the point cloud data and construct a point cloud data with sufficient density and accuracy to reflect the undulation changes of the terrain;

[0073] Step 4: Orthorectification

[0074] Orthorectification is based on the photogrammetry principle of central projection. By eliminating the geometric deformation of the image, the image of central projection is converted into orthographic projection, which is described as follows:

[0075] (1) The principle of step 1 geometric correction can be adopted. According to the UAV images, a collinearity equation is established. The sensor parameters calculated in step 1 geometric correction are substituted into the equation. At the same time, the DEM and plane coordinates obtained in step 3 are combined to relate the image point coordinates to the actual geographical coordinates of the ground object points;

[0076] (2) Image point coordinate reprojection and gray resampling are performed. For each image point in the image, according to the established geometric correction model and DEM data, the bilinear interpolation method is used to obtain a relatively smooth image, and finally its new coordinates under the orthographic projection are calculated;

[0077] Step 5, Orthographic stitching

[0078] According to the above data preprocessing steps, the embodiments of the present application can import the processed UAV data into the corresponding software. For example, after performing the above corrections on the data obtained by the Pegasus V500, import it into the UAV Butler software, add the image data and the corresponding POS data, and then the orthorectified image can be generated according to the feature extraction results and the generated point cloud.

[0079] In step S102, based on the orthorectified image and a preset SAM model, a sea ice classification training dataset is constructed, and the pre-constructed Ice-Unet model is trained through the sea ice classification training dataset. And the orthorectified image is input into the trained Ice-Unet model to output the sea ice category information corresponding to the orthorectified image.

[0080] Furthermore, the embodiments of the present application can utilize the orthophoto image data obtained above, and construct a training dataset by using SAM (Segment Anything Model), and on this basis, construct an Ice-Unet network, as Figure 2 shown, to achieve high-precision sea ice classification of optical images.

[0081] Optionally, in an embodiment of the present application, training the pre-constructed Ice-Unet model through the sea ice classification training dataset includes: constructing an Ice-Unet model based on a preset backbone feature extraction network, enhanced feature extraction network, and prediction network; inputting the training data in the sea ice classification training dataset into the backbone feature extraction network in the Ice-Unet model to perform multiple convolution and max-pooling operations on the training data to generate multiple preliminary effective feature layers; inputting the multiple preliminary effective feature layers into the enhanced feature extraction network to perform upsampling and feature fusion processing on the multiple preliminary effective feature layers to obtain a target fusion effective feature layer; using a target convolutional layer to perform channel adjustment operations on the target fusion effective feature layer to obtain the trained Ice-Unet model.

[0082] As a feasible implementation, the embodiments of the present application construct a sea ice classification training dataset based on orthorectified images and a preset SAM model, and train a pre-constructed Ice-Unet model through the sea ice classification training dataset. The specific steps are as follows:

[0083] Step 1: Sample selection based on the SAM model

[0084] The embodiments of the present application can adopt an interactive sea ice annotation method based on the SAM segmentation model to construct a sea ice classification dataset. Those skilled in the art should understand that ISAT-SAM is an interactive semi-automatic image segmentation annotation tool based on SAM. This tool supports two annotation modes: manual polygon drawing and semi-automatic annotation. In the semi-automatic annotation mode, you can first left-click on the area to be annotated. As a prompt for ISAT-SAM, the tool automatically recognizes the same type of area, and the recognition effect is generally good. If there is a misrecognition, you can cancel the point annotation by clicking the right mouse button. Finally, the entire image can be annotated by clicking multiple times, as Figure 3 shown; the tool defaults to generating a json label file, and then uses the format conversion function of the tool to convert the json format to the png format to obtain the required label file;

[0085] Step 2: Construction of the Ice-Unet model

[0086] The Ice-Unet model of the embodiments of the present application is as Figure 4 shown. This model can be divided into three parts: a backbone feature extraction network, an enhanced feature extraction network, and a prediction network, which are specifically described as follows:

[0087] (1) The first part is the backbone feature extraction network. This part is similar to the VGG16 network structure and obtains five preliminary effective feature layers through multiple convolutions and max-poolings as the input of the enhanced feature extraction network. The specific process is as follows:

[0088] 1) The number of channels of the input sea ice image is 3. First, two 3×3 convolutions with an output channel number of 64 are performed to obtain the first preliminary effective feature layer;

[0089] 2) Max-pooling and two 3×3 convolutions with an output channel number of 128 are performed in sequence to obtain the second preliminary effective feature layer;

[0090] 3) Max-pooling and three 3×3 convolutions with an output channel number of 256 are performed to obtain the third preliminary effective feature layer;

[0091] 4) Max-pooling and three 3×3 convolutions with an output channel number of 512 are performed to obtain the fourth preliminary effective feature layer;

[0092] 5) Perform max pooling and a 3×3 convolution with 512 output channels for three times to obtain the fifth preliminary effective feature layer;

[0093] In the above process, the convolution kernel and stride of max pooling are both set to 2, realizing that the width and height of the picture are halved while the number of channels remains unchanged.

[0094] (2) The second part is the enhanced feature extraction network. This part performs upsampling and feature fusion on the five preliminary effective feature layers, and finally obtains an effective feature layer that integrates all features (i.e., the target fusion effective feature layer); the fifth preliminary effective feature layer is upsampled by 2 times and then concatenated with the fourth preliminary effective feature layer, and then passes through two convolutions to obtain the first enhanced feature layer; the first enhanced feature layer is upsampled by 2 times and then concatenated with the third preliminary effective feature layer, and then passes through two convolutions to obtain the second enhanced feature layer; the second enhanced feature layer is upsampled by 2 times and then concatenated with the second preliminary effective feature layer, and then passes through two convolutions to obtain the third enhanced feature layer; the third enhanced feature layer is upsampled by 2 times and then concatenated with the first preliminary effective feature layer, and then passes through two convolutions to obtain the final effective feature layer, whose size is the same as the original input picture and the number of channels is 64.

[0095] (3) The third part is the prediction network. This part uses a 1×1 convolution to adjust the channels of the final effective feature layer, so that the number of channels is equal to the number of target categories to be classified (in the embodiment of the present application, the number of target categories to be classified is 4, namely ice, melt pond, water, and ship), which is equivalent to classifying each feature point of the feature layer. As Figure 5 shown, thus realizing the semantic segmentation of each pixel of the sea ice image.

[0096] Compared with other classification algorithms, as Figure 6 shown, the Ice-Unet model of the embodiment of the present application has better sea ice classification accuracy.

[0097] In step S103, determine the target reference temperatures of the target type water area and the target type sea ice according to the sea ice category information, and based on the target reference temperatures, construct an inversion model of sea ice concentration to quantitatively evaluate the inversion accuracy of sea ice concentration by using the inversion model of sea ice concentration.

[0098] Furthermore, the embodiment of the present application can use the obtained sea ice category information as auxiliary information for reference temperature selection, obtain the reference temperatures of open water and sea ice, and construct an inversion model of sea ice concentration based on thermal infrared data, so as to quantitatively evaluate the inversion accuracy of sea ice concentration.

[0099] Accordingly, the embodiments of the present application achieve high-precision sea ice classification by introducing the Ice-Unet deep learning model, providing auxiliary information for the selection of thermal infrared sea ice system points; on this basis, the embodiments of the present application establish an inversion model for sea ice concentration based on thermal infrared data to achieve high-precision sea ice concentration inversion. Therefore, the research on sea ice information extraction based on unmanned observation proposed in the embodiments of the present application can obtain high-precision sea ice category and concentration data, thereby realizing simple route planning and providing reference data for regional Arctic sea ice changes.

[0100] Optionally, in an embodiment of the present application, the target reference temperatures of the target type water area and the target type sea ice are determined according to the sea ice category information, and based on the target reference temperatures, an inversion model for sea ice concentration is constructed to quantitatively evaluate the inversion accuracy of the sea ice concentration using the inversion model for sea ice concentration, including: determining the target pixel grid corresponding to each pixel point in the orthorectified image and dividing the target pixel grid into multiple sub-grids; in each of the multiple sub-grids, selecting the sea ice surface temperature at the target percentile and determining the preliminary sea ice reference temperature of the target type sea ice according to the sea ice surface temperature at the target percentile; determining the final reference temperature of the target type sea ice based on the preliminary sea ice reference temperature and a preset linear regression strategy; based on a preset per-pixel sliding window strategy, performing multiple coverage operations on each pixel point to obtain the final reference temperature of the target type sea ice corresponding to each coverage operation, and calculating the target reference temperature of the target type sea ice corresponding to each pixel point according to the final reference temperature of the target type sea ice corresponding to each coverage operation; calculating the sea ice concentration corresponding to each pixel according to the target reference temperatures of the target type water area and the target type sea ice and the inversion model for sea ice concentration, and projecting the sea ice concentration to obtain the corresponding projection result, and quantitatively evaluating the inversion accuracy of the sea ice concentration through the projection result.

[0101] In the embodiments of the present application, the calculation of sea ice concentration mainly uses a linear equation to solve the pixel points, and the more important parameter is the surface temperature, which is divided into the reference temperature of the open water sea water (i.e., the target reference temperature of the target type water area) t pwater and the reference temperature of the sea ice (i.e., the target reference temperature of the target type sea ice) t pice , that is, the temperature when the current pixel is completely covered by water or sea ice. Among them, the positions of sea water and sea ice can be obtained by using the sea ice classification method based on the Ice-Unet model.

[0102] In the actual implementation process, the embodiments of the present application can select the temperature corresponding to the seawater category in the sea ice category information on the thermal infrared data, and select the minimum value of the mean square error of the temperature as the reference temperature of the seawater category; while the ice surface temperature is affected by the air temperature and varies greatly, so it is impossible to select a fixed Arctic temperature value as the reference. To cope with local variations, each pixel is assigned an independent t pice .

[0103] In the specific processing process, in order to obtain the density inversion result of the target resolution from the high-resolution original UAV data, the embodiments of the present application can divide the original image into grids of 20×20 pixels, and each grid represents a pixel point of the density inversion result of the target resolution. To determine the sea ice reference temperature of each pixel point, each grid can be further divided into small blocks of 5×5 (i.e., multiple sub-grids). In each small block, the 25th percentile value of the sea ice surface temperature (i.e., the sea ice surface temperature of the target percentile) is selected as the preliminary sea ice reference temperature, and then the final reference temperature is determined through the linear regression formula as shown in the following formula:

[0104] t pice (x,y) = a·x + b·y + c

[0105] where x and y are the coordinates of each pixel, and a, b, and c are regression coefficients.

[0106] Optionally, in an embodiment of the present application, the mathematical expression of the sea ice density inversion model is:

[0107]

[0108] where SIC represents the sea ice density; t pwate represents the target reference temperature of the target type water area; t pice represents the target reference temperature of the target type sea ice; IST represents the sea ice surface temperature.

[0109] After that, the embodiments of the present application can adopt the per-pixel sliding window method to cover each pixel 20 times; then select the average value of the 20 iterations as t pice , and estimate SIC (Sea Ice Concentration) through a linear model. The mathematical expression of this sea ice density inversion model is as follows:

[0110]

[0111] where SIC represents the sea ice density; t pwate represents the target reference temperature of the target type water area; t pice represents the target reference temperature of the target type sea ice; IST represents the sea ice surface temperature.

[0112] Finally, the embodiments of the present application can project the SIC results based on a 1m grid and generate the final sea ice density inversion product by combining data of the same orbit.

[0113] It should be noted that Figure 7 and Figure 8 are respectively the schematic diagram of the sea ice concentration inversion result and the schematic diagram of the verification of the sea ice concentration result. Among them, Figure 7 the (a) in Figure 7 is the schematic diagram of optical data, Figure 7 the (b) in Figure 7 is the schematic diagram of thermal infrared data, Figure 8 the (c) in Figure 8 is the schematic diagram of the sea ice concentration inversion result of optical data, Figure 8 the (d) in

[0114] such as Figure 7 the (a)-(d) in Figure 8 and the (a)-(c) in

[0115] show, the embodiments of the present application can perform reliable sea ice concentration inversion and realize the quantitative evaluation of the inversion accuracy of sea ice concentration.

[0116] In summary, the embodiments of the present application first obtain registered UAV optical and thermal infrared images through preprocessing techniques such as orbit correction, radiometric calibration, and geometric correction, providing high-quality data for subsequent sea ice information extraction to improve the reliability of multi-modal data; at the same time, based on the sea ice feature learning method of the deep learning model, the Ice-Unet model is introduced to achieve high-precision sea ice classification results, providing auxiliary information for the selection of thermal infrared sea ice system points, so as to select the surface temperature values corresponding to open water and fixed ice in sea ice classification to construct the system point values of thermal infrared data; on this basis, a sea ice concentration inversion model based on thermal infrared data is established to obtain the sea ice concentration inversion result based on thermal infrared data, so as to achieve high-precision sea ice concentration inversion and quantitatively evaluate the uncertainty of sea ice concentration.

[0117] The method for classifying sea ice and inverting its concentration from the optical and thermal infrared data of an unmanned aerial vehicle (UAV) according to the embodiments of the present application includes obtaining the optical and thermal infrared data of the target UAV, performing preprocessing and orthorectification stitching operations on the optical and thermal infrared data to obtain the orthorectified images corresponding to the optical and thermal infrared data; constructing a sea ice classification training dataset based on the orthorectified images and a preset SAM model, training a pre-constructed Ice-Unet model through the sea ice classification training dataset, and inputting the orthorectified images into the trained Ice-Unet model to output the sea ice category information corresponding to the orthorectified images; determining the target reference temperatures of the target type of water area and the target type of sea ice according to the sea ice category information, and constructing a sea ice concentration inversion model based on the target reference temperatures to quantitatively evaluate the inversion accuracy of the sea ice concentration by using the sea ice concentration inversion model. Thus, high-precision sea ice category and concentration data can be obtained, simple route planning can be realized, and reliable reference data for regional Arctic sea ice changes can be provided.

[0118] Secondly, a device for classifying sea ice and inverting its concentration from the optical and thermal infrared data of an unmanned aerial vehicle according to the embodiments of the present application will be described with reference to the accompanying drawings.

[0119] Figure 9 It is a block diagram of the device for classifying sea ice and inverting its concentration from the optical and thermal infrared data of an unmanned aerial vehicle according to the embodiments of the present application.

[0120] As Figure 9 shown, the device 10 for classifying sea ice and inverting its concentration from the optical and thermal infrared data of an unmanned aerial vehicle includes: a preprocessing module 100, a classification module 200, and an inversion module 300.

[0121] Among them, the preprocessing module 100 is used to obtain the optical and thermal infrared data of the target UAV, and perform preprocessing and orthorectification stitching operations on the optical and thermal infrared data to obtain the orthorectified images corresponding to the optical and thermal infrared data.

[0122] The classification module 200 is used to construct a sea ice classification training dataset based on the orthorectified images and a preset SAM model, train a pre-constructed Ice-Unet model through the sea ice classification training dataset, and input the orthorectified images into the trained Ice-Unet model to output the sea ice category information corresponding to the orthorectified images.

[0123] The inversion module 300 is used to determine the target reference temperatures of the target type of water area and the target type of sea ice according to the sea ice category information, and construct a sea ice concentration inversion model based on the target reference temperatures to quantitatively evaluate the inversion accuracy of the sea ice concentration by using the sea ice concentration inversion model.

[0124] Optionally, in an embodiment of the present application, the preprocessing module 100 includes: a radiometric calibration unit, an orthorectification unit, and an orthomosaic unit.

[0125] Among them, the radiometric calibration unit is used to perform geometric correction and radiometric calibration operations on optical and thermal infrared data to obtain corresponding calibration results.

[0126] The orthorectification unit is used to construct a corresponding digital elevation model based on the calibration results, and perform orthorectification operations on optical and thermal infrared data according to the digital elevation model to obtain corresponding orthorectified data.

[0127] The orthomosaic unit is used to perform orthomosaic processing on the orthorectified data to obtain an orthorectified image.

[0128] Optionally, in an embodiment of the present application, the classification module 200 includes: a modeling unit, a backbone feature extraction unit, an enhanced feature extraction unit, and a prediction unit.

[0129] Among them, the modeling unit is used to construct an Ice-Unet model based on a preset backbone feature extraction network, an enhanced feature extraction network, and a prediction network.

[0130] The backbone feature extraction unit is used to input the training data in the sea ice classification training dataset into the backbone feature extraction network in the Ice-Unet model to perform multiple convolution and max-pooling operations on the training data to generate multiple preliminary effective feature layers.

[0131] The enhanced feature extraction unit is used to input multiple preliminary effective feature layers into the enhanced feature extraction network to perform upsampling and feature fusion processing on the multiple preliminary effective feature layers to obtain a target fusion effective feature layer.

[0132] The prediction unit is used to perform channel adjustment operations on the target fusion effective feature layer using a target convolutional layer to obtain a trained Ice-Unet model.

[0133] Optionally, in an embodiment of the present application, the inversion module 300 includes: a division unit, a selection unit, a determination unit, a coverage unit, and a quantitative evaluation unit.

[0134] Among them, the division unit is used to determine the target pixel grid corresponding to each pixel point in the orthorectified image, and divide the target pixel grid into multiple sub-grids.

[0135] The selection unit is used to select the target percentile of the sea ice surface temperature in each of the multiple sub-grids, and determine the preliminary sea ice reference temperature of the target type of sea ice according to the target percentile of the sea ice surface temperature.

[0136] A determination unit, configured to determine a final reference temperature of sea ice of a target type based on a preliminary sea ice reference temperature and a preset linear regression strategy.

[0137] An overlay unit, configured to perform multiple overlay operations on each pixel point based on a preset pixel-by-pixel sliding window strategy, so as to obtain a final reference temperature of sea ice of the target type corresponding to each overlay operation, and calculate a target reference temperature of sea ice of the target type corresponding to each pixel point according to the final reference temperature of sea ice of the target type corresponding to each overlay operation.

[0138] A quantitative evaluation unit, configured to calculate the sea ice concentration corresponding to each pixel according to the target reference temperature of the target type water area and the target type sea ice and a sea ice concentration inversion model, project the sea ice concentration, so as to obtain a corresponding projection result, and quantitatively evaluate the sea ice concentration inversion accuracy through the projection result.

[0139] Optionally, in an embodiment of the present application, the mathematical expression of the sea ice concentration inversion model is:

[0140]

[0141] Wherein, SIC represents the sea ice concentration; t pwate represents the target reference temperature of the target type water area; t pice represents the target reference temperature of the target type sea ice; IST represents the sea ice surface temperature

[0142] It should be noted that the foregoing explanations of the embodiments of the method for classifying and inverting the sea ice concentration of the optical and thermal infrared data of the unmanned aerial vehicle are also applicable to the device for classifying and inverting the sea ice concentration of the optical and thermal infrared data of the unmanned aerial vehicle in this embodiment, and will not be elaborated here.

[0143] The device for classifying sea ice and inverting the concentration of optical and thermal infrared data of an unmanned aerial vehicle according to an embodiment of the present application includes a preprocessing module, which is used to obtain the optical and thermal infrared data of a target unmanned aerial vehicle, and perform preprocessing and orthorectification stitching operations on the optical and thermal infrared data to obtain orthorectified images corresponding to the optical and thermal infrared data; a classification module, which is used to construct a sea ice classification training dataset based on the orthorectified images and a preset SAM model, train a pre-constructed Ice-Unet model through the sea ice classification training dataset, and input the orthorectified images into the trained Ice-Unet model to output sea ice category information corresponding to the orthorectified images; an inversion module, which is used to determine the target reference temperatures of target type waters and target type sea ice according to the sea ice category information, and construct a sea ice concentration inversion model based on the target reference temperatures to quantitatively evaluate the inversion accuracy of the sea ice concentration by using the sea ice concentration inversion model, so as to obtain high-precision sea ice category and concentration data, and can realize simple route planning, providing reliable reference data for regional Arctic sea ice changes.

[0144] Figure 10 The structural schematic diagram of the electronic device provided by an embodiment of the present application. The electronic device may include:

[0145] A memory 1001, a processor 1002, and a computer program stored on the memory 1001 and executable on the processor 1002.

[0146] When the processor 1002 executes the program, it implements the method for classifying sea ice and inverting the concentration of optical and thermal infrared data of an unmanned aerial vehicle provided in the above embodiment.

[0147] Furthermore, the electronic device further includes:

[0148] A communication interface 1003, which is used for communication between the memory 1001 and the processor 1002.

[0149] The memory 1001 is used to store a computer program executable on the processor 1002.

[0150] The memory 1001 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0151] If the memory 1001, the processor 1002, and the communication interface 1003 are implemented independently, the communication interface 1003, the memory 1001, and the processor 1002 can be interconnected via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 only a thick line is used to represent it in Figure 10 , but it does not mean that there is only one bus or one type of bus.

[0152] Optionally, in a specific implementation, if the memory 1001, the processor 1002, and the communication interface 1003 are integrated on a single chip, the memory 1001, the processor 1002, and the communication interface 1003 can communicate with each other through an internal interface.

[0153] The processor 1002 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0154] The embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned method for classifying sea ice and inverting the concentration of drone optical and thermal infrared data is implemented.

[0155] The embodiments of the present application further provide a computer program product, including a computer program, and when the computer program is executed, it is used to implement the above-mentioned method for classifying sea ice and inverting the concentration of drone optical and thermal infrared data.

[0156] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or N embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0157] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of this application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0158] Any process or method description shown in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or N executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of this application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in the reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of this application belong.

[0159] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definable sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or used in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection part (electronic device) having one or N wirings, a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then stored in a computer memory.

[0160] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0161] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of implementing the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0162] In addition, each functional unit in various embodiments of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0163] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for sea ice classification and density inversion using unmanned aerial vehicle optical and thermal infrared data, characterized in that: The following steps are involved: Acquire optical and thermal infrared data of the target UAV, and perform preprocessing and ortho-stitching operations on the optical and thermal infrared data to obtain ortho-rectified images corresponding to the optical and thermal infrared data; Based on the orthorectified image and the preset SAM model, a sea ice classification training data set is constructed, and a pre-constructed Ice-Unet model is trained by the sea ice classification training data set, and the orthorectified image is input into the trained Ice-Unet model to output sea ice category information corresponding to the orthorectified image; The target reference temperature of the target type of waters and the target type of sea ice is determined according to the sea ice category information, and a sea ice density inversion model is constructed based on the target reference temperature, so as to quantitatively evaluate the sea ice density inversion accuracy using the sea ice density inversion model.

2. The method according to claim 1, characterized in that The step of acquiring optical and thermal infrared data of the target UAV, and performing preprocessing and ortho-stitching operations on the optical and thermal infrared data to obtain ortho-rectified images corresponding to the optical and thermal infrared data includes: Performing geometric correction and radiation calibration operations on the optical and thermal infrared data to obtain corresponding calibration results; Based on the calibration results, a corresponding digital elevation model is constructed, and an orthorectification operation is performed on the optical and thermal infrared data according to the digital elevation model to obtain corresponding orthorectified data; The orthorectified data are ortho-stitched to obtain the orthorectified image.

3. The method according to claim 2, characterized in that The method of training the pre-built Ice-Unet model using the sea ice classification training data set includes: Based on the preset backbone feature extraction network, enhanced feature extraction network and prediction network, the Ice-Unet model is constructed; Inputting the training data in the sea ice classification training data set into the backbone feature extraction network in the Ice-Unet model to perform multiple convolution and maximum pooling operations on the training data to generate multiple preliminary effective feature layers; Inputting the multiple preliminary effective feature layers into an enhanced feature extraction network to perform upsampling and feature fusion processing on the multiple preliminary effective feature layers to obtain a target fused effective feature layer; The target convolution layer is used to perform a channel adjustment operation on the target fusion effective feature layer to obtain the trained Ice-Unet model.

4. The method according to claim 3, characterized in that The step of determining a target reference temperature for a target type of water area and a target type of sea ice according to the sea ice category information, and constructing a sea ice density inversion model based on the target reference temperature, so as to quantitatively evaluate the sea ice density inversion accuracy using the sea ice density inversion model, includes: Determine a target pixel grid corresponding to each pixel point in the orthorectified image, and divide the target pixel grid into a plurality of sub-grids; In each of the plurality of subgrids, selecting a sea ice surface temperature of a target percentile, and determining a preliminary sea ice reference temperature of the target type of sea ice according to the sea ice surface temperature of the target percentile; Determining a final reference temperature of the target type of sea ice based on the preliminary sea ice reference temperature and a preset linear regression strategy; Based on a preset pixel-by-pixel sliding window strategy, multiple covering operations are performed on each pixel point to obtain a final reference temperature of the target type of sea ice corresponding to each covering operation, and a target reference temperature of the target type of sea ice corresponding to each pixel point is calculated according to the final reference temperature of the target type of sea ice corresponding to each covering operation; The sea ice density corresponding to each pixel is calculated according to the target reference temperature of the target type of waters and the target type of sea ice and the sea ice density inversion model, and the sea ice density is projected to obtain a corresponding projection result, and the sea ice density inversion accuracy is quantitatively evaluated through the projection result.

5. The method according to claim 4, characterized in that The mathematical expression of the sea ice density inversion model is: Wherein, SIC represents the sea ice concentration; t pwate Indicates the target reference temperature of the target type of water area; t pice represents the target reference temperature of the target type of sea ice; IST represents the sea ice surface temperature.

6. A device for sea ice classification and density inversion using optical and thermal infrared data from unmanned aerial vehicles, characterized in that: include: A preprocessing module, used for acquiring optical and thermal infrared data of the target UAV, and performing preprocessing and ortho-stitching operations on the optical and thermal infrared data to obtain ortho-rectified images corresponding to the optical and thermal infrared data; A classification module, for constructing a sea ice classification training data set based on the orthorectified image and a preset SAM model, and training a pre-constructed Ice-Unet model through the sea ice classification training data set, and inputting the orthorectified image into the trained Ice-Unet model to output sea ice category information corresponding to the orthorectified image; An inversion module is used to determine the target reference temperature of the target type of waters and the target type of sea ice according to the sea ice category information, and to construct a sea ice density inversion model based on the target reference temperature, so as to quantitatively evaluate the sea ice density inversion accuracy using the sea ice density inversion model.

7. The device according to claim 6, characterized in that The preprocessing module comprises: A radiation calibration unit, used for performing geometric correction and radiation calibration operations on the optical and thermal infrared data to obtain corresponding calibration results; An orthorectification unit, configured to construct a corresponding digital elevation model based on the calibration result, and to perform an orthorectification operation on the optical and thermal infrared data according to the digital elevation model to obtain corresponding orthorectified data; The ortho-stitching unit is used to perform ortho-stitching processing on the ortho-corrected data to obtain the ortho-corrected image.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for sea ice classification and density inversion using optical and thermal infrared data from unmanned aerial vehicle as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the sea ice classification and density inversion method using unmanned aerial vehicle optical and thermal infrared data as described in any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that The computer program is executed to implement the method for sea ice classification and density inversion using unmanned aerial vehicle optical and thermal infrared data as described in any one of claims 1 to 5.

Images