Image-based deep-sea polymetallic nodule identification and reserve evaluation method and system

Abstract

The application belongs to the technical field of deep-sea polymetallic nodules, and particularly relates to a deep-sea polymetallic nodule recognition and reserve evaluation method and system based on images. In view of the problems of brightness imbalance and variable nodule shape and size in deep-sea nodule pictures, the application improves the image degradation problem through illumination distribution optimization preprocessing, combines a Transformer and a convolutional neural network, takes into account global background consistency and local fine-grained reservation, and realizes the cross-layer flow of multi-scale nodule information through the long and short distance combined skip connection, so as to improve the nodule segmentation performance in a complex deep-sea scene. Based on the segmentation result, the reserve evaluation is further realized in combination with actual sampling data, and reliable technical support is provided for deep-sea polymetallic nodule resource evaluation.

Description

Technical Field

[0001] This application belongs to the field of deep-sea polymetallic nodule technology, specifically involving an image-based method and system for deep-sea polymetallic nodule identification and reserve assessment. Background Technology

[0002] Deep-sea polymetallic nodules are important strategic metal resources. Accurate segmentation of nodule regions based on seabed optical images is crucial for automated nodule identification and reserve assessment. However, the severe imbalance of lighting and the varied textures of the seabed in the natural deep-sea environment limit image quality and render manual processing methods unreliable, placing higher demands on automated and refined nodule segmentation methods. Existing traditional image processing and machine learning methods are constrained by the instability of underwater imaging and struggle to meet the segmentation accuracy requirements of complex seabed scenes. Convolutional neural networks (CNNs) possess excellent local structural information extraction capabilities, but due to their reliance on local receptive space, they struggle to fully establish overall structural relationships when dealing with nodules of diverse shapes and sizes. In recent years, Transformers have demonstrated strong global spatial correlation capabilities in image analysis, but their extraction of local structural information remains relatively insufficient. Summary of the Invention

[0003] To address the aforementioned problems, this application proposes an image-based method and system for deep-sea polymetallic nodule identification and reserve assessment. This method improves image degradation by optimizing illumination distribution and, by combining Transformer and convolutional neural networks, fully utilizes local nodule morphology information and overall seabed distribution information, balancing global background consistency with local fine-grained preservation. This enhances segmentation accuracy in complex deep-sea scenarios and provides a reliable basis for polymetallic nodule reserve assessment. The technical solution is as follows:

[0004] A method for image-based identification and reserve assessment of deep-sea polymetallic nodules, comprising the following steps:

[0005] S1. Acquire raw images of deep-sea nodules using deep-sea optical imaging equipment;

[0006] S2. Perform illumination distribution optimization preprocessing on the input image;

[0007] S3. Segment nodule images based on a multi-scale deep-sea nodule image segmentation network to obtain pixel-level nodule prediction maps;

[0008] S4. Count the number of pixels identified as nodules based on the pixel-level nodule prediction map, and calculate the nodule coverage rate by combining it with the total number of pixels in the image; establish a conversion model between the two-dimensional occupied area and mass of nodules based on the measured sampling data, thereby realizing the conversion from pixel-level segmentation results to resource reserve quantification.

[0009] Preferably, step S2 performs illumination distribution optimization preprocessing on the input image, and the specific steps are as follows:

[0010] S11. Convert the input RGB image to the HSV color space;

[0011] S12. Smooth the luminance V channel in the HSV color space using a large-scale Gaussian function to extract the global illumination representation of the image. ;

[0012] S13. Based on global illumination characterization Constructing an adaptive deep-sea illumination distribution optimization factor The calculation formula is as follows:

[0013] ;

[0014] in, This represents the average value of the image's brightness channel.

[0015] S14. Smooth the luminance V channel of the HSV color space using a small-scale Gaussian function to extract local illumination characteristics. ;

[0016] S15. Characterization of local illumination Standard deviation normalization is performed to obtain the local illumination imbalance estimation factor. s；

[0017] S16. Estimation factor based on local illumination imbalance s Dynamically adjust the main parameter values ​​of CLAHE.

[0018] Preferred, based on adaptive deep-sea illumination distribution optimization factor An adaptive deep-sea illumination distribution optimization function is constructed to adjust the brightness V channel of the input image:

[0019] ;

[0020] in, and The brightness V channel intensity values ​​of the input image and the corrected image are respectively. Maximum value of the brightness V channel.

[0021] Preferably, the main parameter values ​​of CLAHE in step S15 include the contrast constraint factor. and local regional division scale :

[0022] .

[0023] Preferably, the multi-scale deep-sea nodule image segmentation network in step S3 includes CSWin Transformer encoding and CNN decoding: the encoding steps are as follows:

[0024] S21. Convolutional Label Embedding: First, the image after illumination distribution optimization... pass The convolution with a stride of 4 is used to divide the data into blocks, and then the spatial features of the convolutional feature maps are rearranged into token features suitable for the Transformer model. ;

[0025] ;

[0026] in Indicates the convolution operation; This represents the feature map spatial feature rearrangement operation; h, w, and c represent the image height, width, and channels, respectively.

[0027] S22. Token characteristics T Input three layers of coded blocks and obtain the output features respectively. The first layer of the coding block includes two CSWin Transformer modules and a downsampling module; the second layer includes four CSWin Transformer modules and a downsampling module; the third layer includes 32 CSWin Transformer modules and a downsampling module; the third layer outputs features. The output features are obtained from two CSWin Transformer modules. ;

[0028] S23. Will To achieve effective integration, the specific steps are as follows:

[0029] ;

[0030] in, , To adjust the parameters, Indicates splicing and fusion. , , These represent the skip connection outputs of the first, second, and third layers, respectively.

[0031] Preferably, the final output of the CSWin Transformer encoder As input to the CNN decoding, the CNN decoding consists of three decoding layers. Each decoding layer includes an upsampling module and a convolutional module. The outputs of the three upsampling modules are as follows: , and The outputs of the three decoding blocks are respectively , and ;

[0032] The upsampling module reconstructs the spatial scale of the feature map through transposed convolution: the input of the first upsampling module is the final output of the CSWin Transformer encoder. The input to the second upsampling module is the output of the first layer decoding block. The input to the third upsampling module is the output of the second-layer decoding block. The details are as follows:

[0033] ;

[0034] in, This indicates the transpose convolution operation; express COMING Nonlinear activation function;

[0035] The convolutional module first concatenates and fuses the skip connections and the corresponding upsampled feature maps, and then effectively extracts the nodule edge texture details through two consecutive convolution operations; the specific steps are as follows:

[0036] ;

[0037] ;

[0038] ;

[0039] in, , , These are the outputs of the first, second, and third convolutional modules, respectively, spliced ​​and fused together. , , These are the outputs of the first convolution operation in the first, second, and third convolutional modules, respectively. , , These are the outputs of the second convolution operation in the first, second, and third convolutional modules, respectively.

[0040] The output of the last layer of decoding block The nodule is input to the segmentation head and undergoes a convolution operation to obtain the final nodule segmentation result. The specific steps are as follows:

[0041] .

[0042] Preferably, calculate tuberculosis coverage rate. C :

[0043] ;

[0044] Number of kernel pixels Total number of pixels in the image ;

[0045] A conversion model between the two-dimensional occupied area and mass of nodules was established based on measured sampling data. This enables the transformation from pixel-level segmentation results to resource reserve quantification.

[0046] An image-based system for identifying and assessing the reserves of deep-sea polymetallic nodules includes an image acquisition module, an image processing module, a calculation module, and an output module.

[0047] Image acquisition module: used to acquire images of polymetallic nodules on the seabed in the target area;

[0048] Image processing module: Adjusts the brightness and contrast of the input image through adaptive deep-sea illumination distribution optimization and dynamic enhancement of details based on illumination perception;

[0049] Calculation module: Utilizing Transformer and convolutional neural network, it makes full use of the local morphological information of nodules and the overall seabed distribution information, taking into account both global background consistency and local fine-grained preservation, and further calculates the nodule coverage based on the segmentation results;

[0050] Output module: Visualizes the results.

[0051] Preferably, the computation module includes encoding and decoding. In the encoding stage, encoding blocks are used to extract the overall semantic representation of nodules and seabed background at different scales, thereby modeling the spatial relationship between nodule regions. In the decoding stage, convolution operations are used to recover nodule edges and texture details, and a skip connection structure combining short and long distances is used to effectively transfer the global spatial relationship extracted by the encoder at multiple levels to the decoder, thereby realizing the spatial scale reconstruction of feature maps step by step. Finally, the decoder outputs pixel-level nodule prediction maps through the segmentation head.

[0052] Compared with the prior art, the beneficial effects of this application are as follows:

[0053] 1. Illumination Distribution Optimization Preprocessing. This application effectively improves the brightness distribution and contrast of deep-sea nodule images by adaptive deep-sea illumination distribution optimization and illumination-aware dynamic detail enhancement, mitigating the impact of factors such as illumination imbalance on image quality.

[0054] 2. Multi-scale Deep-sea Nodule Image Segmentation Network. This application proposes a multi-scale deep-sea nodule image segmentation network that combines the advantages of CSWin Transformer and CNN. Through skip connections combining short-range and long-range connections, it can effectively capture both local fine-grained features and global relationships in images. Compared to single-scale feature extraction methods, this approach has higher accuracy in capturing features at different scales in deep-sea nodule images, enabling more precise identification of nodule targets of different shapes and sizes. Attached Figure Description

[0055] Figure 1 For flowcharts;

[0056] Figure 2 This is a network diagram for segmenting images of deep-sea nodules at multiple scales. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] A method for image-based identification and reserve assessment of deep-sea polymetallic nodules includes a nodule image acquisition stage, an illumination distribution optimization preprocessing stage, a CSWinTransformer encoding stage, a CNN decoding stage, and a reserve assessment stage. In the nodule image acquisition stage, raw images of deep-sea nodules are acquired using deep-sea optical imaging equipment. In the illumination distribution optimization preprocessing stage, the brightness and contrast of the input image are adjusted through adaptive deep-sea illumination distribution optimization and illumination-aware detail dynamic enhancement to mitigate the impact of image degradation caused by factors such as illumination imbalance in the deep-sea environment on subsequent segmentation tasks. In the CSWinTransformer encoding stage, a three-layer CSWinTransformer encoding block is used to extract the overall semantic representation of the nodules and the seabed background, realizing the modeling of the spatial relationship between nodule regions. In the CNN decoding stage, convolutional operations are used to recover the nodule edges and texture details, and a skip connection structure combining short-range and long-range connections is used to effectively transfer the overall semantic representation extracted by the encoder at multiple levels to the decoder, thereby progressively reconstructing the spatial scale of the feature map. Finally, the decoder outputs pixel-level nodule prediction maps through the segmentation head. During the reserve assessment phase, the nodule coverage rate is further calculated based on the segmentation results, providing key parameter support for subsequent reserve estimation of nodules.

[0059] Step 1: Acquire raw images of deep-sea nodules using deep-sea optical imaging equipment, and perform basic storage management on the nodule images to provide input data for subsequent illumination distribution optimization and segmentation processing.

[0060] Step 2: Perform illumination distribution optimization preprocessing on the input image, including adaptive deep-sea illumination distribution optimization and illumination-aware dynamic detail enhancement. The specific steps are as follows:

[0061] ① Convert the input RGB image to the HSV color space.

[0062] ② A large-scale Gaussian function is used to smooth the luminance V channel in the HSV color space to extract the global illumination representation of the image. .

[0063] ③Based on global illumination characterization Constructing an adaptive deep-sea illumination distribution optimization factor The calculation formula is as follows:

[0064] ;

[0065] in, This represents the average value of the image's brightness channel.

[0066] ④ Optimization factor based on adaptive deep-sea illumination distribution An adaptive deep-sea illumination distribution optimization function is constructed to adjust the brightness V channel of the input image:

[0067] ;

[0068] in, and The brightness V channel intensity values ​​of the input image and the corrected image are respectively. Maximum value of the brightness V channel.

[0069] ⑤ Use a small-scale Gaussian function to smooth the luminance V channel of the HSV color space to extract local illumination characteristics. .

[0070] ⑥ Characterization of local illumination Standard deviation normalization is performed to obtain the local illumination imbalance estimation factor. s .

[0071] ⑦ Limit Contrast Adaptive Histogram Equalization (CLAHE) enhances contrast by equalizing the local brightness distribution and limits the enhancement intensity to prevent over-enhancement. Light-perceived detail dynamic enhancement is further based on factors estimated from the local illumination imbalance. s Dynamically adjust the main parameter values ​​of CLAHE, including the contrast constraint factor. and local regional division scale This was used to perform CLAHE contrast enhancement on the V channel of the image after adaptive deep-sea illumination distribution optimization.

[0072] ;

[0073] ⑧ Convert the enhanced HSV image back to an RGB image.

[0074] Step 3: Construct a multi-scale deep-sea nodule image segmentation network to segment nodule images, including CSWinTransformer encoding and CNN decoding. The specific steps are as follows:

[0075] ① Convolutional Label Embedding: First, the image after illumination distribution optimization... pass The convolution with a stride of 4 is used to divide the data into blocks, and then the spatial features of the convolutional feature maps are rearranged into token features suitable for the Transformer model. T :

[0076] ;

[0077] in Indicates the convolution operation; This represents the feature map spatial feature rearrangement operation. h、w、c These represent the image height, width, and channels, respectively.

[0078] ② Token characteristics T Input three layers of coded blocks and obtain the output features respectively. The first layer of encoding includes two CSWin Transformer modules and a downsampling module; the second layer includes four CSWin Transformer modules and a downsampling module; the third layer includes 32 CSWin Transformer modules and a downsampling module; the output of the third layer encoding passes through two CSWin Transformer modules sequentially and is then output. .

[0079] ③The CSWin Transformer module is normalized by layers ( LN ), cross-shaped window self-attention mechanism ( CSWin- Attention ) and multilayer perceptron ( MLP )composition.

[0080] ④ The downsampling module reduces the spatial size of the feature map through convolution operations, as follows:

[0081] ;

[0082] in express ReLU Nonlinear activation function; Presentation layer normalization operation; , These represent the outputs of the previous CSWin Transformer module and the downsampling module, respectively.

[0083] ⑤ To better facilitate the cross-layer flow of multi-scale tuberculosis information, a skip connection structure combining short-distance and long-distance connections is adopted. To achieve effective integration, the specific steps are as follows:

[0084] ;

[0085] in, , To adjust the parameters; Indicates splicing and merging; , , These represent the skip connection outputs of the first, second, and third layers, respectively.

[0086] ⑥ The final output of the CSWin Transformer encoder This serves as the input for CNN decoding. CNN decoding consists of three decoding layers, each including an upsampling module and a convolutional module; the outputs of the three upsampling modules are... , and The outputs of the three decoding blocks are respectively , and .

[0087] ⑦ The upsampling module reconstructs the spatial scale of the feature map through transposed convolution. The input to the first upsampling module is the final output of the CSWin Transformer encoder. The input to the second upsampling module is the output of the first-layer CNN decoding block. The input to the third upsampling module is the output of the second-layer CNN decoding block. The details are as follows:

[0088] ;

[0089] in, This indicates the transpose convolution operation; express COMING Nonlinear activation function.

[0090] ⑧ The convolution module first concatenates and fuses the skip connections and the corresponding upsampled feature maps, and then effectively extracts the edges and texture details of the nodules through two consecutive convolution operations. The specific steps are as follows:

[0091] ;

[0092] ;

[0093] ;

[0094] in, , , These are the outputs of the first, second, and third convolutional modules, respectively, spliced ​​and fused together. , , These are the outputs of the first convolution operation in the first, second, and third convolutional modules, respectively. , , These are the outputs of the second convolution operation in the first, second, and third convolutional modules, respectively.

[0095] ⑨ Output the last layer of the decoding block The nodule is input to the segmentation head and undergoes a convolution operation to obtain the final nodule segmentation result. The specific steps are as follows:

[0096] .

[0097] Step 4: Obtaining the segmentation results of deep-sea polymetallic nodules Then, the number of pixels identified as nodules is first counted based on the pixel-level nodule prediction map. And combined with the total number of pixels in the image Calculate tuberculosis coverage Subsequently, a conversion model between the two-dimensional occupied area and mass of nodules was established based on measured sampling data. This enables the transformation from pixel-level segmentation results to resource reserve quantification.

[0098] An image-based system for identifying and assessing the reserves of deep-sea polymetallic nodules includes an image acquisition module, an image processing module, a calculation module, and an output module.

[0099] Image acquisition module: used to acquire images of polymetallic nodules on the seabed in the target area;

[0100] Image processing module: Adjusts the brightness and contrast of the input image through adaptive deep-sea illumination distribution optimization and dynamic enhancement of details based on illumination perception;

[0101] Calculation module: Utilizing Transformer and convolutional neural network, it makes full use of the local morphological information of nodules and the overall seabed distribution information, taking into account both global background consistency and local fine-grained preservation, and further calculates the nodule coverage based on the segmentation results;

[0102] Output module: Visualizes the results.

[0103] The computation module includes encoding and decoding. In the encoding stage, encoding blocks are used to extract the overall semantic representation of nodules and seabed background at different scales, thereby modeling the spatial relationship between nodule regions. In the decoding stage, convolution operations are used to recover the nodule edges and texture details, and a skip connection structure combining short and long distances is used to effectively pass the overall semantic representation extracted by the encoder at multiple levels to the decoder, thereby realizing the spatial scale reconstruction of the feature map step by step. Finally, the decoder outputs pixel-level nodule prediction maps through the segmentation head.

[0104] The foregoing has shown and described the basic principles, main features, and advantages of this application. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the application. Various changes and modifications can be made without departing from the spirit and scope of this application, and all such changes and modifications fall within the scope of the claims. The scope of protection of this application is defined by the appended claims and their equivalents.

Claims

1. An image-based method for identifying and assessing the reserves of deep-sea polymetallic nodules, characterized in that, Includes the following steps: S1. Acquire raw images of deep-sea nodules using deep-sea optical imaging equipment; S2. Perform illumination distribution optimization preprocessing on the input image; S3. Segment nodule images based on a multi-scale deep-sea nodule image segmentation network to obtain pixel-level nodule prediction maps; The multi-scale deep-sea nodule image segmentation network includes CSWin Transformer encoding and CNN decoding: the encoding steps are as follows: S21. Convolutional Label Embedding: First, the image after illumination distribution optimization... The feature maps are divided into blocks using convolution, and then the spatial features of the convolutional feature maps are rearranged into token features suitable for the Transformer model. ; ； in Indicates the convolution operation; This represents the feature rearrangement operation in the feature map space; h, w, and c represent the image height, width, and channels, respectively. S22. Input the token feature T into the three-layer coding block to obtain the output features respectively. The first layer of the coding block includes two CSWin Transformer modules and a downsampling module; the second layer includes four CSWin Transformer modules and a downsampling module; the third layer includes 32 CSWin Transformer modules and a downsampling module; the third layer outputs features. The output features are obtained from two CSWin Transformer modules. ; S23. Will To achieve effective integration, the specific steps are as follows: ； in, , To adjust the parameters, Indicates splicing and fusion. , , These represent the skip connection outputs of the first, second, and third layers, respectively; Final output of the CSWin Transformer encoder As input to the CNN decoding, the CNN decoding consists of three decoding layers. Each decoding layer includes an upsampling module and a convolutional module. The outputs of the three upsampling modules are as follows: , and The outputs of the three decoding blocks are respectively , and ; The upsampling module reconstructs the spatial scale of the feature map through transposed convolution: the input of the first upsampling module is the final output of the CSWin Transformer encoder. The input to the second upsampling module is the output of the first layer decoding block. The input to the third upsampling module is the output of the second-layer decoding block. The details are as follows: ； in, This indicates the transpose convolution operation; This represents the GELU nonlinear activation function; The convolutional module first concatenates and fuses the skip connections and the corresponding upsampled feature maps, and then effectively extracts the edges and texture details of the nodules through two consecutive convolution operations; the specific steps are as follows: ； ； ； in, , , These are the outputs of the first, second, and third convolutional modules, respectively, spliced ​​and fused together. , , These are the outputs of the first convolution operation in the first, second, and third convolutional modules, respectively. , , These are the outputs of the second convolution operation in the first, second, and third convolutional modules, respectively. The output of the last layer of decoding block As input to the segmentation head, the kernels undergo convolution operations to obtain the final pixel-level segmentation results. The specific steps are as follows: ； S4. Count the number of pixels identified as nodules based on the pixel-level nodule prediction map, and calculate the nodule coverage rate by combining it with the total number of pixels in the image; establish a conversion model between the two-dimensional occupied area and mass of nodules based on the measured sampling data, thereby realizing the conversion from pixel-level segmentation results to resource reserve quantification.

2. The image-based method for identifying and assessing the reserves of deep-sea polymetallic nodules according to claim 1, characterized in that, Step S2 involves performing illumination distribution optimization preprocessing on the input image. The specific steps are as follows: S11. Convert the input RGB image to the HSV color space; S12. Smooth the luminance V channel in the HSV color space using a large-scale Gaussian function to extract the global illumination representation of the image. ; S13. Based on global illumination characterization Constructing an adaptive deep-sea illumination distribution optimization factor The calculation formula is as follows: ； in, This represents the average value of the image's brightness channel. S14. Smooth the luminance V channel of the HSV color space using a small-scale Gaussian function to extract local illumination characteristics. ; S15. Characterization of local illumination Standard deviation normalization is performed to obtain the local illumination imbalance estimation factor s; S16. Dynamically adjust the main parameter values ​​of CLAHE based on the local illumination imbalance estimation factor s.

3. The image-based method for identifying and assessing the reserves of deep-sea polymetallic nodules according to claim 2, characterized in that, Based on adaptive deep-sea illumination distribution optimization factor An adaptive deep-sea illumination distribution optimization function is constructed to adjust the brightness V channel of the input image: ； in, and The brightness V channel intensity values ​​of the input image and the corrected image are respectively. Maximum value of the brightness V channel.

4. The image-based method for identifying and assessing the reserves of deep-sea polymetallic nodules according to claim 2, characterized in that, The main parameter values ​​of CLAHE in step S16 include the contrast constraint factor. and local regional division scale : 。 5. The image-based method for identifying and assessing the reserves of deep-sea polymetallic nodules according to claim 1, characterized in that, Calculate the tuberculosis coverage rate C: ； Number of kernel pixels Total number of pixels in the image ; A conversion model between the two-dimensional occupied area and mass of nodules was established based on measured sampling data. This enables the transformation from pixel-level segmentation results to resource reserve quantification.

6. An image-based deep-sea polymetallic nodule identification and reserve assessment system, employing the image-based deep-sea polymetallic nodule identification and reserve assessment method as described in any one of claims 1-5, characterized in that, It includes an image acquisition module, an image processing module, a calculation module, and an output module; Image acquisition module: used to acquire images of polymetallic nodules on the seabed in the target area; Image processing module: Adjusts the brightness and contrast of the input image through adaptive deep-sea illumination distribution optimization and dynamic enhancement of details based on illumination perception; Calculation module: Utilizing Transformer and convolutional neural network, it makes full use of the local morphological information of nodules and the overall seabed distribution information, taking into account both global background consistency and local fine-grained preservation, and further calculates the nodule coverage based on the segmentation results; Output module: Visualizes the results.

7. The image-based deep-sea polymetallic nodule identification and reserve assessment system according to claim 6, characterized in that, The computation module includes encoding and decoding. In the encoding stage, encoding blocks are used to extract the overall semantic representation of nodules and seabed background at different scales, so as to model the spatial relationship between regions. During the decoding stage, convolutional operations are used to restore the nodule edges and texture details. A skip connection structure combining short and long distances is used to effectively pass the overall semantic representation extracted by the encoder at multiple levels to the decoder, thereby realizing the spatial scale reconstruction of the feature map step by step. Finally, the decoder outputs a pixel-level nodule prediction map through the segmentation head.

Images