Deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation

Abstract

The present application relates to the technical field of image data processing, and more particularly to a deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation, comprising: a deep-sea image enhancement module for illuminance compensation and noise suppression of deep-sea multispectral images, comprising: a multi-band illuminance compensation unit; an adaptive noise suppression unit; a multispectral semantic segmentation module comprising: a multiscale feature fusion network; a hydrothermal vent suppression unit; a lightweight deployment module connected to the output end of the multispectral semantic segmentation module for deploying the model to deep-sea hardware, comprising: a dynamic fixed-point quantization unit; a parallel pipeline acceleration unit based on FPGA to split the image enhancement and segmentation network into independent computing units, and to realize parallel execution of data reading and writing and calculation through a double buffering mechanism. The system can not only ensure accurate identification of sulfide deposits in deep-sea environments, but also efficiently run on low-power hardware, improving the automation level and work efficiency of deep-sea deposit exploration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and particularly relates to a deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation. BACKGROUND

[0002] As an important seabed mineral resource, the efficient exploration of deep-sea polymetallic sulfide deposits depends on the accurate identification of sulfide structures in seabed images. However, due to the extreme environment of the deep sea, existing technologies face the following bottlenecks in practical application:

[0003] In the deep-sea environment, the penetration difference between visible light and near-infrared light spectrum is significant. The absorption and scattering of light by water result in low overall contrast of the image, and the gray difference between the sulfide area and the background sediment is weak. Existing image enhancement algorithms (such as histogram equalization and Retinex algorithm) are mostly designed for land or shallow sea environments, with the core assumption of uniform light distribution or single noise type, which cannot effectively distinguish the deep-sea multi-band attenuation characteristics (for example, the near-infrared frequency band attenuates much faster than visible light). In addition, the mixed superposition of Gaussian noise introduced by the long exposure requirement of deep-sea cameras and impulse noise caused by water disturbance, traditional filtering methods (such as mean filtering and bilateral filtering) will destroy the weak details of sulfide edges while smoothing noise, further reducing the image usability.

[0004] Traditional semantic segmentation models (such as FCN and DeepLab series) mainly rely on spatial texture features in the visible light band, while the identification of deep-sea sulfides needs to comprehensively utilize the reflection difference between visible light and near-infrared spectrum. Due to the similar high reflectivity of hydrothermal vent area in the near-infrared band as sulfide, existing models are prone to misjudgment, resulting in blurred segmentation boundaries and category confusion. Although some studies attempt to introduce multi-spectral data, the feature fusion method is mostly simple channel splicing, which fails to model the coupling relationship between sulfide multi-scale texture (such as blocky and chimney-like structure) and spectral reflectance, making it difficult to meet the actual exploration requirements in terms of segmentation accuracy.

[0005] Existing deep-sea image processing systems usually rely on manually labeled data to train high-parameter models (such as ResNet and VGG), and are not optimized for embedded hardware, resulting in low inference speed (usually less than 10 FPS). Real-time navigation and sampling operations of deep-sea vehicles require online access to deposit distribution information, while existing systems cannot achieve end-to-end deployment on low-power FPGA / ASIC hardware due to high computational resource occupation, which severely restricts the exploration efficiency.

[0006] Therefore, there is an urgent need for a technical solution that optimizes the whole chain from image enhancement, semantic segmentation to hardware deployment, to break through the bottleneck of deposit identification in the extreme environment of the deep sea. SUMMARY

[0007] Based on the above purpose, the present application provides a deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation, comprising:

[0008] A deep-sea image enhancement module is used for illumination compensation and noise suppression of a deep-sea multi-spectral image, comprising:

[0009] A multi-band illumination compensation unit decomposes an original image into a visible light band and a near-infrared band, generates a compensation curve according to the attenuation characteristics of each band in a dark area, and superimposes a compensation value pixel by pixel to improve the contrast of a sulfide area;

[0010] An adaptive noise suppression unit divides a filtering area based on local texture complexity of an image, uses non-local mean filtering to retain edge details for a low-texture area, and uses adaptive median filtering to suppress impulse noise for a high-texture area;

[0011] A multi-spectral semantic segmentation module is connected to an output end of the deep-sea image enhancement module and is used for generating a pixel-level segmentation result of a sulfide deposit, comprising:

[0012] A multi-scale feature fusion network inputs a visible light and near-infrared dual-channel enhanced image, extracts multi-scale sulfide texture features through parallel hollow convolution, and dynamically fuses the features by using a channel attention mechanism;

[0013] A hydrothermal vent suppression unit screens a high-reflection area according to a near-infrared channel reflectivity difference, and removes a misjudgment candidate area overlapping with a hydrothermal vent through morphological operation;

[0014] A lightweight deployment module is connected to an output end of the multi-spectral semantic segmentation module and is used for deploying a model to a deep-sea hardware, comprising:

[0015] A dynamic fixed-point quantization unit adaptively allocates quantization bit width according to network layer weight distribution characteristics, retains high precision for a high dynamic range layer, and compresses to a low bit width for a low dynamic range layer;

[0016] A parallel pipeline acceleration unit splits the image enhancement and segmentation network into independent computing units based on FPGA, and realizes parallel execution of data reading and writing and calculation through a double-buffering mechanism.

[0017] Preferably, in the multi-band illumination compensation unit, the generation process of the compensation curve comprises:

[0018] After the original image is decomposed into a visible light band and a near-infrared band, the gray distribution histograms of each band in a dark area of the image are respectively counted;

[0019] The attenuation ratios of the visible light band and the near-infrared band are calculated according to the proportion of low-frequency components in the dark area gray distribution, and the attenuation ratio of the near-infrared band is dynamically corrected through a water penetration depth model.

[0020] The compensation curve is superimposed on the original image and output to the noise suppression unit.

[0021] Preferably, in the adaptive noise suppression unit, the method for dividing the local texture complexity comprises:

[0022] The enhanced image is processed in blocks, and the gradient amplitude variance of each sub-block is calculated, the gradient amplitude being extracted by a Sobel operator;

[0023] The division threshold of the low-texture area and the high-texture area is dynamically set according to the gradient variance: if the gradient variance of the sub-block is lower than a first threshold, the sub-block is determined as a low-texture area, and if the gradient variance of the sub-block is higher than a second threshold, the sub-block is determined as a high-texture area, the first threshold and the second threshold being calculated based on the noise level of the whole image, the noise level being obtained by the gray level fluctuation statistics of the flat area of the image;

[0024] The non-local mean filter is used for the low-texture area, and the filter window size is adaptively adjusted according to the size of the sub-block; the adaptive median filter is used for the high-texture area, and the maximum window size of the filter is dynamically set according to the density of the impulse noise.

[0025] Preferably, the structure of the multi-scale feature fusion network comprises:

[0026] The encoder part comprises three groups of parallel hole convolution layers, respectively using convolution kernels with expansion rates of 1, 3 and 5, for extracting sulfide edge features and texture features under different receptive fields;

[0027] The decoder part fuses the feature maps and the up-sampling results of each stage of the encoder through a jump connection, and introduces a channel attention module before the fusion, the channel attention module generating a channel description vector through global average pooling, learning the correlation between channels through a fully connected layer to generate attention weights, and weighting and enhancing important feature channels;

[0028] The output layer inputs the preliminary segmentation result to a hydrothermal vent suppression unit, the hydrothermal vent suppression unit screening a high-reflectivity area according to a reflectivity threshold of the near-infrared channel, the reflectivity threshold being obtained by the statistical difference of the spectral reflectivity of sulfides and hydrothermal vents in the training data set.

[0029] Preferably, the execution process of the morphological operation in the hydrothermal vent suppression unit comprises:

[0030] The near-infrared channel image is segmented by the reflectivity threshold to obtain a binary mask of the high-reflectivity area;

[0031] The binary mask is closed with a circular structural element, which merges adjacent high reflectivity regions, and the radius of the structural element is set according to the typical size range of the hydrothermal vent;

[0032] The closed mask is logically XORed with the preliminary result of the segmentation network to remove the misjudged sulfide candidate regions in the overlapping area, and the final segmentation mask is generated.

[0033] Preferably, the quantization bit width allocation method in the dynamic fixed-point quantization unit comprises:

[0034] The distribution range of the absolute values of the weights of the fully connected layer is counted, and if the ratio of the maximum weight value to the minimum weight value exceeds a preset threshold, a high bit width is allocated; otherwise, a low bit width is allocated;

[0035] The kurtosis of the weights of the convolutional layer is analyzed, and if the kurtosis value is lower than a preset threshold, it indicates that the weight distribution is flat, and a low bit width is allocated; if the kurtosis value is higher than the threshold, a high bit width is retained;

[0036] A symmetric quantization strategy is used to map the weights to a quantization interval centered at zero, and the truncated error of the quantized weights is reduced by an error compensation algorithm.

[0037] Preferably, the double buffering mechanism of the parallel pipeline acceleration unit comprises:

[0038] Two independent buffers are set, the first buffer is used to store the current frame data output by the image enhancement module, and the second buffer is used to store the previous frame data in the processing of the segmentation module;

[0039] When the image enhancement module completes a frame of processing, the buffer address is switched to make the segmentation module immediately read the latest data, and at the same time the image enhancement module writes the next frame of data to the idle buffer;

[0040] The buffer capacity is dynamically configured according to the on-chip storage resources of the FPGA and the input image resolution, to ensure that the data throughput rates of the two modules match.

[0041] Preferably, it further comprises a multispectral image acquisition unit, which specifically comprises:

[0042] A visible light-near infrared dual-band camera carried by the deep-sea vehicle, which synchronously records water depth data when shooting;

[0043] The exposure time of the camera is dynamically adjusted according to the water depth data: short exposure time is used in shallow water area to suppress overexposure, and long exposure time is used in deep water area to improve the signal-to-noise ratio, and the adjustment step of the exposure time is adaptively set according to the water depth gradient;

[0044] The acquired multispectral image is input to the image enhancement module and bound with the water depth data for dynamic correction of the subsequent band compensation parameters.

[0045] Preferably, the weight generation process of the channel attention module comprises:

[0046] Global average pooling is performed on the multi-scale feature map to generate a global feature description vector for each channel.

[0047] The description vector is input into a subnetwork comprising two fully connected layers, the first fully connected layer compresses the dimension to 1 / 4 of the original channel number, and the second layer restores to the original channel number.

[0048] The channel attention weight between 0 and 1 is generated by the Sigmoid function, and the weight is multiplied with the original feature map channel by channel to strengthen the response of the sulfide-related feature channel.

[0049] Preferably, the real-time control method of the system comprises:

[0050] Step 1: Map the segmentation mask to the navigation coordinate system of the deep-sea vehicle, and generate a three-dimensional distribution map of the sulfide deposit combined with the vehicle pose data.

[0051] Step 2: Plan the sampling path according to the density distribution of the deposit in the three-dimensional distribution map, and avoid the high reflection area of the hydrothermal vent.

[0052] Step 3: Trigger the mechanical arm control signal through the FPGA hardware, and perform fixed-point sampling on the target sulfide area. The delay time of the sampling instruction is synchronized and calibrated according to the calculation period of the pipeline acceleration unit.

[0053] The beneficial effects of the present application are:

[0054] 1. The present application combines the multi-band attenuation characteristics of deep sea, designs a multi-band light compensation unit and an adaptive noise suppression unit according to the attenuation characteristics of different wave bands (such as visible light and near infrared), which can more effectively enhance the sulfide structure in deep sea image, reduce the influence of noise on image details, and effectively maintain the weak details of sulfide edge. Therefore, the present application can significantly improve the image quality and enhance the contrast in the process of deep sea image enhancement, and overcome the shortcomings of traditional methods in deep sea environment.

[0055] 2. The present application introduces the joint segmentation technology of visible light and near infrared wave band in deep sea image, combines the multi-scale texture features (such as blocky and chimney structure) of sulfide, effectively solves the problems of fuzzy segmentation boundary and category confusion. The method can make the system accurately identify sulfide in deep sea environment through more accurate feature modeling of multispectral data by deep learning model, and avoid the risk of misjudging hydrothermal vent, thereby improving the segmentation accuracy.

[0056] 3、The application optimizes the system architecture, combines image enhancement and semantic segmentation, and is specially optimized for low-power hardware (such as FPGA), adopts hardware acceleration and pipeline calculation strategy, and effectively improves the system inference speed. The system can process deep sea images in real time under low power consumption conditions, accurately identify sulfide ore deposits and plan the sampling path, and meet the real-time navigation and sampling needs of deep sea vehicles. The application not only improves the real-time performance, but also reduces the power consumption of the system, greatly improves the exploration efficiency, and breaks through the bottleneck of the traditional method.

[0057] 4、The application provides a whole-chain optimization scheme integrating image enhancement, semantic segmentation and hardware deployment, and each link in the deep sea ore deposit exploration is finely designed to form an efficient and accurate deep sea ore deposit identification system. The system can not only provide accurate ore deposit distribution information, but also plan the optimal sampling path according to the density distribution, and control the mechanical arm for point sampling through FPGA, so that the whole process is seamlessly connected, and the efficiency and accuracy of deep sea ore deposit identification are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only a part of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0059] Fig. 1 It is a structural block diagram of the system of the application;

[0060] Fig. 2 It is a step flow chart of the double buffering mechanism of the parallel pipeline acceleration unit of the system of the application;

[0061] Fig. 3 It is a step flow chart of the real-time control method of the system of the application. DETAILED DESCRIPTION

[0062] The application will be described in detail below in combination with the drawings and specific embodiments. It should be noted here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and other alternative ways can also be used by those skilled in the art to implement them; and the drawings are only used to more specifically describe the embodiments, and are not intended to specifically limit the application.

[0063] Please refer to Figs. 1-3 The embodiment of the application provides a deep sea polymetallic sulfide ore deposit identification system based on image enhancement and semantic segmentation, wherein the task of the deep sea image enhancement module is to perform illumination compensation and noise suppression on the deep sea multispectral image.

[0064] The purpose of the multi-band illumination compensation unit is to decompose the original image into visible light bands and near-infrared bands. According to the gray characteristics of the dark areas in the image, a compensation curve is generated respectively. By superimposing the compensation value on each pixel pixel by pixel, the contrast of the ore deposit area, especially the sulfide area, is improved. Specifically, the generation of the compensation curve takes into account the attenuation characteristics of different bands and the illumination attenuation model of the water body, ensuring that the illumination deficiency can be effectively compensated in water areas at different depths, thereby improving the visibility of the target area.

[0065] The adaptive noise suppression unit dynamically divides the filtering area according to the local texture complexity of the image. For low-texture areas, a non-local mean filtering method is used to preserve the edge details of the image and avoid blurring of the details; for high-texture areas, an adaptive median filter is used to effectively suppress impulse noise. This method can dynamically adjust the noise processing strategy, thereby achieving optimal noise suppression effect in different areas.

[0066] The multi-spectral semantic segmentation module performs multi-scale feature extraction and semantic segmentation on the enhanced image.

[0067] Specifically, the multi-scale feature fusion network inputs the visible light and near-infrared enhanced image, uses parallel hole convolution to extract multi-scale sulfide texture features, further enhancing the local and global feature representation of the image. In addition, the network also focuses on the feature channels that have a key impact on sulfide ore deposits through channel attention mechanism, thereby improving the recognition accuracy.

[0068] The hydrothermal vent suppression unit filters out high-reflectance areas in the image based on the difference in near-infrared reflectance, and removes false positives in the hydrothermal vent area through morphological operations. Morphological closing operation can merge adjacent high-reflectance areas and accurately remove false positive ore deposit areas overlapping with hydrothermal vents, ensuring the accuracy of the segmentation result.

[0069] Further, the lightweight deployment module deploys the trained model to deep-sea hardware, ensuring efficient operation of the system in resource-constrained environments.

[0070] Specifically, the dynamic fixed-point quantization unit dynamically adjusts the quantization bit width according to the weight distribution characteristics of the network layer. For layers with high dynamic range, higher quantization precision is preserved, while for layers with low dynamic range, bit width is compressed to reduce computational burden. This dynamic quantization method effectively balances the calculation accuracy and the use of hardware resources, reducing energy consumption and computational delay.

[0071] The parallel pipeline acceleration unit utilizes FPGA technology to split the image enhancement and semantic segmentation tasks into independent computing units, and realizes the parallel execution of data reading and calculation through a double buffering mechanism. This parallel design significantly improves the real-time processing capability of the system, and is particularly suitable for rapid data processing requirements in deep sea environments.

[0072] In one possible implementation, first, the original deep sea image is decomposed into two different frequency bands, the visible light band and the near-infrared band. The visible light band and the near-infrared band have different illumination attenuation characteristics in the image processing process, so they need to be compensated independently. After decomposing the image, the gray level distribution histogram of the dark area in each frequency band is counted, that is, the pixel distribution of the low gray value area (i.e. dark illumination) in the image is counted. This step provides basic data for subsequent calculation of the attenuation ratio, helping to judge the illumination attenuation degree of each frequency band.

[0073] After obtaining the gray level distribution histogram of the dark area in each frequency band, the next task is to calculate the attenuation ratio according to the low frequency component (i.e. the dark part) in the histogram. Specifically, the attenuation ratio is analyzed by the proportion of the low frequency component to measure the attenuation degree of the image in different frequency bands. For the visible light band, the attenuation ratio mainly reflects the absorbance and depth of the water body; for the near-infrared band, the attenuation ratio needs to be further considered according to the penetration depth of the water body, which is usually dynamically corrected by a water body penetration depth model. This model adjusts the attenuation ratio according to different properties, transparency and wavelength of the water body, so as to more accurately calculate the attenuation effect of the near-infrared band.

[0074] Based on the attenuation ratio calculated above, the system generates an exponential compensation curve to correct the insufficient illumination part of the image. The generation of the compensation curve takes into account the characteristics of illumination attenuation, so that the brightness of the dark area of the image can be effectively compensated. For the near-infrared band, the compensation strength will be dynamically adjusted according to the output value of the penetration depth model, to ensure that the compensation value can adapt to different water depths and illumination conditions.

[0075] After the compensation curve is generated, it will be superimposed on the original image pixel by pixel to perform illumination enhancement processing on the image. This enhancement process not only improves the brightness of the dark area, but also effectively enhances the contrast of the target area such as sulfide ore deposit, providing a clearer image basis for subsequent noise suppression and semantic segmentation. The superimposed image will be input to the noise suppression unit for further noise filtering and processing.

[0076] In one possible implementation, the generation formula of the compensation curve is represented by the following formula:

[0077] ;

[0078] wherein, is a spectral band identifier;

[0079] is a pixel coordinate;

[0080] is a compensation value;

[0081] is a basic compensation coefficient, calibrated by experiment, taking 1.0 in shallow water and 1.2 in deep water in the embodiment;

[0082] is an attenuation proportion coefficient, and the determination formula is , is a water body turbidity coefficient, is a frequency band attenuation rate, in the embodiment, taking 0.03 for visible light and 0.15 for near-infrared light;

[0083] is a water depth, is a dark area pixel value, is a frequency band maximum gray value;

[0084] In a possible implementation, first, the enhanced image after the illumination compensation processing is subjected to block processing. The image is divided into a plurality of sub-blocks, and each sub-block is a relatively independent region in the image. For each sub-block, the gradient amplitude variance thereof is calculated. The gradient amplitude can be extracted through a Sobel operator, which obtains the gradient amplitude of each pixel point in the image by calculating the horizontal and vertical gradients of the pixel point. The greater the gradient amplitude, the more obvious the edge or texture information of the region, and vice versa. By calculating the gradient amplitude variance of each sub-block, a quantitative index of the texture complexity of the region can be obtained.

[0085] According to the gradient amplitude variance of each sub-block, the division threshold of the low-texture region and the high-texture region in the image is dynamically set. If the gradient variance of the sub-block is lower than a predetermined first threshold, the sub-block is determined as a low-texture region; if the gradient variance of the sub-block is higher than a predetermined second threshold, the sub-block is determined as a high-texture region. The setting of the two thresholds is dynamically calculated based on the noise level of the whole image. The calculation of the noise level depends on the gray level fluctuation statistics of the flat region in the image, that is, the noise intensity is estimated by analyzing the region with small gray value change in the image. The threshold is adjusted according to the noise level, so that the division of the low-texture region and the high-texture region can adapt to different noise backgrounds.

[0086] Low-texture region noise suppression: For low-texture regions (i.e., regions with less texture variation), non-local mean filtering is employed for noise suppression. Non-local mean filtering smooths noise by utilizing similar pixel blocks in the image, effectively preserving the details of flat regions. In this method, the size of the filtering window is adaptively adjusted according to the size of the sub-block, ensuring that low-texture regions can be properly smoothed while avoiding excessive blurring.

[0087] For high-texture regions (i.e., regions with more intense texture variation), adaptive median filtering is employed. Adaptive median filtering dynamically adjusts the maximum window size of the filter based on the density of impulse noise in the signal. For regions with higher noise density, the filter window is larger to better eliminate noise; while for regions with lower noise density, the filter window is relatively smaller, preserving the details and texture structure of the image.

[0088] Through the above process, different noise suppression techniques are used for low-texture regions and high-texture regions, ensuring that the noise in each region of the image is effectively reduced while important image details are preserved. The noise-suppressed image will be passed as input to the subsequent semantic segmentation module for further identification of target regions of deep-sea polymetallic sulfide deposits.

[0089] In one possible implementation, the division threshold of low-texture regions and high-texture regions in the image is expressed by the following formula:

[0090] Low-texture region threshold: ;

[0091] High-texture region threshold: ;

[0092] ;

[0093] wherein, is the noise variance of the flat region of the image, is the flat region (low-texture region) in the image;

[0094] is the total number of pixels in the region, is the gray value of the pixel , is the average gray value of the region.

[0095] In one possible implementation, the encoder part contains three sets of parallel dilated convolution layers, respectively using convolution kernels with dilation rates of 1, 3, and 5. Dilated convolution is a technique to expand the receptive field of convolution, by using convolution kernels with different dilation rates, multi-scale features in the image can be extracted under different receptive fields. Specifically: ​

[0096] Dilation rate 1 convolution kernel: This kernel is similar to regular convolution and can extract local area detail features, suitable for capturing texture detail information.

[0097] Dilation rate 3 convolution kernel: By increasing the size of the receptive field, it can capture medium-scale features such as edges and some texture changes.

[0098] Dilation rate 5 convolution kernel: Further expand the receptive field, suitable for capturing large-scale texture features, which can help identify the boundaries of larger areas of sulfide deposits and other key areas in the image.

[0099] These dilated convolution layers work in parallel, allowing the extraction of different features of the image at multiple scales, thus helping to better understand the complex structures in the image, especially the sulfide edge features and texture features.

[0100] The decoder part fuses the feature maps extracted by each stage of the encoder with the up-sampling results through a skip connection. The skip connection helps to preserve the details of the encoding stage and combine them with the coarse features of the decoding stage, ensuring that details are not lost during image reconstruction. To further enhance the feature fusion effect, a channel attention module is introduced before fusion.

[0101] The role of the channel attention module is to enhance the model's attention to important feature channels. The specific operation steps are as follows:

[0102] First, the feature map of each channel is reduced dimensionally through global average pooling to obtain a global feature representation of each channel. The channel description vector after global average pooling is fed into a fully connected layer to learn the correlation between channels. After the fully connected layer, the attention weight of each channel is generated. Through this weight, each channel in the feature map is weighted to enhance those more important feature channels for sulfide deposit recognition.

[0103] In the output layer, the feature map after decoder processing is input to the hydrothermal vent suppression unit. The purpose of the hydrothermal vent suppression unit is to filter out areas with high reflectivity based on the near-infrared channel reflectivity information in the image. These areas usually correspond to hydrothermal vents (i.e. high-temperature liquid areas generated by deep-sea hydrothermal vents), which are usually represented as bright areas with high reflectivity in the image. The specific implementation steps are as follows:

[0104] The reflectivity threshold is obtained by statistical analysis of the spectral reflectivity difference between sulfide and hydrothermal vent in the training data set. This threshold can effectively distinguish between sulfide deposits and hydrothermal vent areas. When the reflectivity of a certain area in the image exceeds the threshold, the area will be marked as a hydrothermal vent and removed from the final segmentation result to avoid misidentifying hydrothermal vents as deposits.

[0105] In one possible implementation, first, the near-infrared channel image is extracted for reflectance threshold segmentation. The change of near-infrared channel reflectance can effectively distinguish different objects and areas, because the reflection characteristics of hydrothermal vents and sulfide deposits are significantly different in the near-infrared band.

[0106] A reasonable reflectance threshold is obtained by analyzing the training data. This threshold is used to segment the near-infrared channel image to generate a binary mask. In the binary mask, the area with a value of 1 represents a high reflectance area (such as a hydrothermal vent), and the area with a value of 0 represents other areas (such as a sulfide deposit or background).

[0107] In one possible implementation, the reflectance threshold can be represented by the following formula:

[0108] Sulfide area reflectance threshold: ;

[0109] Hydrothermal vent area reflectance threshold: ;

[0110] wherein is the mean reflectance of the sulfide area, which is obtained by statistical analysis of the training set;

[0111] is the standard deviation of the reflectance of the sulfide area, which is obtained by statistical analysis of the training set;

[0112] is the mean reflectance of the hydrothermal vent area, which is obtained by statistical analysis of the training set;

[0113] is the standard deviation of the reflectance of the hydrothermal vent area, which is obtained by statistical analysis of the training set;

[0114] To further optimize the reflectance segmentation result, a circular structure element is used to perform a closing operation on the binary mask after reflectance threshold segmentation. The closing operation is a morphological operation that can effectively remove small holes and connect adjacent high reflectance areas by first performing an expansion operation and then performing an erosion operation.

[0115] The radius of the circular structure element is set according to the typical size range of the hydrothermal vent. The hydrothermal vent usually has a certain size, and reasonable setting of the radius of the structure element can ensure that the merged area conforms to the typical morphology of the hydrothermal vent and avoid mismerging small areas. The closing operation can effectively connect adjacent high reflectance areas, reduce errors, and improve the integrity of the hydrothermal vent area, providing a more accurate mask for subsequent screening.

[0116] The closed mask and the preliminary segmentation result of the segmentation network (i.e., the segmentation result of the ore deposit) are subjected to logical exclusive or operation. The role of the logical exclusive or operation is to eliminate the misjudged sulfide candidate regions in the overlapping area.

[0117] The logical exclusive or operation compares two binary images, and only the area where the two masks are inconsistent is retained in the result. Therefore, the overlapping part, i.e., the area marked as both a hydrothermal vent and a sulfide deposit, will be removed. This step helps to remove misidentified sulfide regions, ensuring the accuracy of the final segmentation result. This step can eliminate the misidentified deposit regions due to the high reflectivity of the hydrothermal vent area, ensuring that the final segmentation mask only contains the true sulfide deposit area and avoiding misjudgment of the hydrothermal vent and the deposit.

[0118] After the above steps, the final segmentation mask is generated. The mask is the result of reflectivity threshold segmentation, closed operation optimization, and logical exclusive or operation to remove misjudgments. This final mask accurately marks the sulfide deposit area in the deep-sea image.

[0119] In one possible implementation, in the quantization process, first, the weights of the full connection layer are counted to calculate the distribution range of their absolute values. Specifically, first, the maximum and minimum values of each weight in the full connection layer are obtained. Then, the ratio of the two is calculated:

[0120] If the ratio of the maximum weight value to the minimum weight value is greater than a preset threshold, it indicates that the distribution of the weight values is significantly different. In order to maintain high precision and detail, the system will allocate a higher quantization bit width to ensure that the large weight range can be accurately represented.

[0121] It can be understood that the value of the full connection layer weight range threshold is selected by analyzing the weight distribution histogram of a typical neural network full connection layer, and selecting a critical ratio that can distinguish between "wide dynamic range" and "narrow dynamic range" as the threshold;

[0122] When the distribution range of the weights is large, high-bit-width quantization can more finely represent these weight values, avoiding information loss.

[0123] Next, the kurtosis of the convolution layer weights is analyzed. Kurtosis is an indicator of the shape of a probability distribution, reflecting the sharpness of the weight distribution:

[0124] If the kurtosis value of the weights is low, it means that the weight distribution is relatively flat and the information concentration is low. At this time, the system considers that it does not need too high quantization accuracy, so it will allocate a lower bit width. If the weight distribution presents a relatively sharp peak (i.e., the kurtosis value is high), it means that the weight information is concentrated, and higher quantization accuracy is needed to preserve these important weight information. Therefore, the system will allocate a higher bit width.

[0125] It can be understood that the kurtosis threshold of the convolution layer is based on the kurtosis statistical histogram of the convolution layer weights, and the standard normal distribution kurtosis value is taken as the kurtosis threshold for distinguishing between "flat distribution" and "spike distribution".

[0126] To ensure that the quantized weights are uniformly distributed and not biased to either side, the system adopts a symmetric quantization strategy. Under this strategy, the quantization interval of the weights is centered on zero, i.e., the positive and negative weights are distributed in the same quantization range. This means that the weight values will be mapped to a symmetric quantization interval to reduce bias and maintain information balance during the quantization process. This symmetric quantization helps to improve the accuracy of the quantized weights, especially in cases where the weight distribution is not symmetric, effectively avoiding truncation error and information loss.

[0127] During the quantization process, weights are quantized to integers or limited precision representations, which can result in truncation error. To reduce this error, the system uses an error compensation algorithm:

[0128] The error compensation algorithm corrects the quantized weights to reduce the error generated during the quantization process, ensuring that the performance of the network does not decrease significantly due to the reduction in quantization precision.

[0129] Specifically, the kurtosis The calculation formula is as follows:

[0130] ;

[0131] wherein, is the weight value of the convolution layer, is the mean of the weights, is the number of weights.

[0132] Assuming the quantization interval is , each weight can be quantized and mapped by the following formula:

[0133] ;

[0134] wherein, is the quantization step (the step size of the quantization interval), and round represents the rounding operation. The quantized weight will be mapped to the quantization interval, preserving as much information as possible from the original weight.

[0135] Assuming is the quantized weight, is the original floating-point weight, and the error compensation can be represented as:

[0136] ; ​

[0137] wherein, is the error calculated by the compensation algorithm, which is dynamically adjusted by an optimization algorithm (such as back propagation).

[0138] In one possible implementation, the core of the double buffering mechanism is to set up two independent buffers:

[0139] First buffer: used to store the current frame data output by the image enhancement module.

[0140] Second buffer: used to store the previous frame data in the segmentation module processing.

[0141] This buffer design allows the two modules (image enhancement and semantic segmentation modules) to work in parallel, avoiding resource conflicts between them and improving overall processing efficiency.

[0142] Whenever the image enhancement module completes a frame of data processing, the system switches the buffer address, i.e.:

[0143] The image enhancement module writes the next frame of data into the free buffer (i.e. the buffer that is not currently in use).

[0144] The segmentation module will immediately read data from the first buffer (the buffer storing the current frame data) and begin processing the next frame of segmentation.

[0145] This switching mechanism allows the two modules to seamlessly connect, ensuring data flow and parallelism, thus avoiding mutual waiting between the image enhancement module and the semantic segmentation module, allowing the system to work continuously and stably.

[0146] The capacity of the buffer is dynamically configured according to the on-chip memory resources of the system and the input image resolution. Specifically:

[0147] According to the memory size of the FPGA chip, the system dynamically adjusts the size of the buffer to ensure that the memory resources are used reasonably. As the input image resolution changes, the capacity of the buffer needs to be adjusted accordingly. Higher resolution images require more storage space, while lower resolution images can use smaller buffers.

[0148] Through this dynamic configuration, the buffer ensures that it meets the data throughput requirements while not exceeding the system's hardware resource limitations.

[0149] In one possible implementation, the deep-sea vehicle carries a visible light-near infrared dual-band camera that integrates the imaging capabilities of visible light and near infrared bands, capable of collecting multispectral images in deep-sea environments. Through multi-band imaging, the spectral characteristics of different substances can be better identified, especially suitable for the identification and analysis of deep-sea mineral deposits.

[0150] The camera records water depth data through the sensor while capturing images, which is crucial for subsequent exposure time adjustment and image enhancement processing.

[0151] In areas with shallow water depth, the light is strong, which can easily cause image overexposure. Therefore, the system uses short exposure time to avoid loss of image details and reduce overexposure, ensuring image clarity.

[0152] In areas with deep water depth, the light is weak. To improve the signal-to-noise ratio of the image, the system uses long exposure time. Long exposure time can increase the light sensitivity of the camera, thus improving image quality, reducing noise, and obtaining higher quality images.

[0153] The adjustment of exposure time is based on the adaptive setting of water depth gradient, that is, automatically determining the adjustment step and exposure time according to the actual water depth change. This dynamic adjustment mechanism enables the system to automatically optimize image acquisition at different depths, avoiding the problem of insufficient or excessive manual setting.

[0154] The collected multispectral images are transmitted to the image enhancement module for further processing. In this process:

[0155] The image is combined with the synchronously recorded water depth data, and the two data are used together. Water depth information and image data work together to help dynamic correction of subsequent band compensation parameters, further improving image quality, especially in deep sea complex environment, band compensation plays an important role in enhancing image details and improving resolution.

[0156] Dynamic adjustment of exposure time can automatically optimize the exposure level of the image according to the change of water depth, ensuring that the best image quality can be obtained in shallow or deep water environment, thus improving the accuracy of subsequent image analysis and processing.

[0157] Through the adjustment of long exposure time, the signal-to-noise ratio of deep water area is effectively improved, reducing the noise in the image and ensuring the clarity of the image, providing more accurate data support for the identification of deep sea mineral deposits.

[0158] The system can adaptively adjust the exposure time according to different water depth gradients, avoiding the problem of image overexposure or insufficient exposure caused by traditional fixed exposure time, and improving the adaptability of the system in various water depth environments.

[0159] The binding of water depth data and image data enables more accurate dynamic correction of band compensation, thus optimizing the image enhancement effect and improving the accuracy of deep sea mineral deposit identification.

[0160] In one possible implementation, the multi-scale feature maps are deep feature maps at different scales obtained after the pre-image processing and feature extraction module. These feature maps generally contain image information at different levels and different spatial positions.

[0161] Global average pooling is performed on these feature maps, which averages all pixel values in each channel to generate a global feature description vector for each channel. The purpose of global average pooling is to capture the global features of the channel by aggregating the overall information in the channel and retaining important information for each channel.

[0162] The fully connected layer subnetwork includes two fully connected layers. The first layer compresses the dimension of the global feature description vector to 1 / 4 of the original channel number, with the purpose of dimension reduction processing to reduce the amount of calculation and improve the efficiency of the model.

[0163] The second fully connected layer restores the feature dimension to the original channel number, which ensures that the final output attention weight has the same number of channels as the input feature map. The core purpose of this process is to abstract the importance of each channel through dimension reduction and restoration.

[0164] After processing by the two fully connected layers, a channel attention vector is obtained. The values of this vector are activated by the Sigmoid function, limiting the range to between 0 and 1. The Sigmoid function normalizes the weight of each channel to a value between 0 and 1, indicating the activation degree of each channel.

[0165] The final channel attention weight represents the importance of each channel in the feature map. The closer the weight is to 1, the more important the features of that channel are, and the closer the weight is to 0, the less important the features of that channel are.

[0166] The generated channel attention weight is multiplied with the original feature map channel by channel to obtain a weighted feature map. In this way, the network will strengthen the response to sulfide-related feature channels and suppress the response to irrelevant channels. This weighting process enables the network to pay more attention to channel features that are critical to deep-sea mineral deposit images in subsequent feature learning, improving the system's ability to recognize deep-sea polymetallic sulfide deposits.

[0167] In one possible implementation, the segmentation mask is pixel-level segmentation information of the mineral deposit area obtained by a deep learning network in the pre-image enhancement and semantic segmentation processing process. It represents the spatial distribution of sulfide deposits in deep-sea images.

[0168] The segmentation mask is mapped to the navigation coordinate system of the deep-sea vehicle. This step is achieved by coordinate transformation technology, which connects the pixel position in the image with the real-world coordinates of the vehicle. Typically, the vehicle's pose data (such as position, attitude, heading, etc.) is obtained through sensors, and combined with the position data of each point in the segmentation mask, the coordinates in three-dimensional space are calculated.

[0169] The generated three-dimensional distribution map contains spatial distribution information of the ore deposit area, which can provide reference for subsequent ore deposit sampling. This three-dimensional map not only considers the location of the ore deposit, but also reflects the shape and distribution characteristics of the ore deposit, helping the navigation system accurately understand the spatial layout of the ore deposit.

[0170] Based on the three-dimensional distribution map, the sampling path is planned through density distribution analysis of the ore deposit. The density distribution information reflects the distribution density of the ore deposit area, so as to determine which areas are more suitable for sampling.

[0171] When planning the sampling path, the high reflection area of the hydrothermal vent must be avoided. The hydrothermal vent area usually has strong reflection signals. These areas may contain unfavorable interference information, and even may mislead the ore deposit recognition system, so these high reflection areas must be avoided when planning the path, so as to ensure the accuracy of sampling.

[0172] FPGA hardware triggering refers to the efficient and low-delay control signal output through field programmable gate array (FPGA). As a hardware acceleration platform, FPGA can provide faster and more stable signal response in scenarios with high real-time requirements.

[0173] The system triggers the control signal of the mechanical arm through FPGA to accurately sample the target sulfide area. This process requires the mechanical arm to accurately reach the target position according to the sampling path planning and perform sampling operations.

[0174] Delay time synchronization calibration of sampling instructions refers to the adjustment of sampling instruction time according to the calculation period of pipeline acceleration units (including image processing, data analysis, etc.) in the system, so that the sampling operation is consistent with the output of image analysis and path planning, thereby avoiding errors caused by instruction delay.

[0175] Through accurate three-dimensional distribution map generation, path planning, mechanical arm fixed-point sampling, and low-delay control triggering mechanism, the efficiency and accuracy of deep-sea polymetallic sulfide ore deposit sampling can be effectively improved. This technical method has significant application value, not only optimizing the sampling process, but also enhancing the working ability of the system in complex seabed environment.

[0176] The present application encompasses any alternatives, modifications, equivalent methods and solutions made to the essence and scope of the present application. In order to make the public have a thorough understanding of the present application, specific details are described in the following preferred embodiments of the present application, and the present application can also be fully understood without the description of these details to those skilled in the art. In addition, in order to avoid unnecessary confusion to the essence of the present application, well-known methods, processes, procedures, elements and circuits, etc. are not described in detail.

[0177] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can also be made, which should be considered as the protection scope of the present application.

Claims

1. A deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation, characterized in that, The deep-sea image enhancement module is used for illumination compensation and noise suppression of a deep-sea multispectral image, and comprises: A multi-band illumination compensation unit decomposes an original image into a visible light band and a near-infrared band, generates a compensation curve according to the attenuation characteristics of each band in a dark area, and superimposes a compensation value pixel by pixel to improve the contrast of a sulfide area; An adaptive noise suppression unit divides a filtering area based on local texture complexity of an image, adopts non-local mean filtering to retain edge details for a low-texture area, and adopts adaptive median filtering to suppress impulse noise for a high-texture area; The adaptive noise suppression unit, wherein the division method of the local texture complexity comprises: Block processing is performed on the enhanced image, and the gradient amplitude variance of each sub-block is calculated, wherein the gradient amplitude is extracted through a Sobel operator; The division threshold of the low-texture area and the high-texture area is dynamically set according to the gradient variance: if the gradient variance of the sub-block is lower than a first threshold, the sub-block is determined as the low-texture area, and if the gradient variance of the sub-block is higher than a second threshold, the sub-block is determined as the high-texture area, wherein the first threshold and the second threshold are calculated based on the noise level of the entire image, and the noise level is obtained by statistical analysis of the gray level fluctuation of the flat area of the image; Non-local mean filtering is adopted for the low-texture area, and the size of the filtering window is adaptively adjusted according to the size of the sub-block; adaptive median filtering is adopted for the high-texture area, and the maximum window size of the filter is dynamically set according to the density of impulse noise; The multispectral semantic segmentation module is connected to the output end of the deep-sea image enhancement module, and is used for generating a pixel-level segmentation result of a sulfide deposit, and comprises: A multi-scale feature fusion network inputs the visible light and near-infrared dual-channel enhanced image, extracts multi-scale sulfide texture features through parallel hole convolution, and dynamically fuses the features by using a channel attention mechanism; A hydrothermal vent suppression unit screens a high-reflection area according to the reflectivity difference of the near-infrared channel, and removes the misjudgment candidate area overlapping with the hydrothermal vent through morphological operation; The lightweight deployment module is connected to the output end of the multispectral semantic segmentation module, and is used for deploying the model to a deep-sea hardware, and comprises: A dynamic fixed-point quantization unit adaptively allocates quantization bit width according to the weight distribution characteristics of the network layer, retains high precision for high dynamic range layers, and compresses low dynamic range layers to low bit width; A parallel pipeline acceleration unit splits the image enhancement and segmentation network into independent calculation units based on FPGA, and realizes parallel execution of data reading and writing and calculation through a double-buffering mechanism; The double-buffering mechanism of the parallel pipeline acceleration unit comprises: Two independent buffers are set, a first buffer is used to store the current frame data output by the image enhancement module, and a second buffer is used to store the previous frame data in the processing of the segmentation module; When the image enhancement module completes one frame of processing, the buffer addresses are switched, so that the segmentation module immediately reads the latest data, and at the same time, the image enhancement module writes the next frame of data to the idle buffer; The buffer capacity is dynamically configured according to the on-chip storage resources of FPGA and the input image resolution, so as to ensure that the data throughput rates of the two modules match. In the multi-band illumination compensation unit, the generation process of the compensation curve comprises:

2. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1, characterized in that, ​ After decomposing the original image into visible band and near-infrared band, the gray level distribution histogram of each band in the dark area of the image is counted respectively; According to the proportion of low-frequency components in the dark area gray level distribution, the attenuation ratio of the visible band and the near-infrared band is calculated, and the attenuation ratio of the near-infrared band is dynamically corrected through the water body penetration depth model; Based on the attenuation ratio, an exponential compensation curve is generated, and the compensation intensity of the near-infrared band is dynamically adjusted according to the output value of the penetration depth model. The compensation curve is superimposed on the original image and output to the noise suppression unit.

3. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1, characterized in that, The structure of the multi-scale feature fusion network includes: The encoder part includes three groups of parallel hole convolution layers, respectively using convolution kernels with expansion rates of 1, 3 and 5, for extracting sulfide edge features and texture features under different receptive fields; The decoder part fuses the feature maps of each stage of the encoder with the up-sampling results through a jump connection, and introduces a channel attention module before fusion. The channel attention module generates a channel description vector through global average pooling, and generates an attention weight after learning the correlation between channels through a fully connected layer, which weights and enhances important feature channels; The output layer inputs the preliminary segmentation result to the hydrothermal vent suppression unit, which filters high-reflectivity areas according to the reflectivity threshold of the near-infrared channel. The reflectivity threshold is obtained by statistical analysis of the spectral reflectivity difference between sulfides and hydrothermal vents in the training data set.

4. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation of claim 1, characterized in that, The execution process of the morphological operation in the hydrothermal vent suppression unit includes: Perform reflectivity threshold segmentation on the near-infrared channel image to obtain a binary mask of high reflectivity areas; Perform a closing operation on the binary mask using a circular structure element to merge adjacent high reflectivity areas. The radius of the structure element is set according to the typical size range of the hydrothermal vent; Perform an exclusive-OR operation on the mask after the closing operation and the preliminary result of the segmentation network to remove false positive sulfide candidate regions in the overlapping area and generate a final segmentation mask.

5. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1, characterized in that, The quantization bit width allocation method in the dynamic fixed-point quantization unit includes: Statistical analysis of the distribution range of the absolute values of the weights of the fully connected layer. If the ratio of the maximum weight value to the minimum weight value exceeds a preset threshold, allocate a high bit width; otherwise, allocate a low bit width; Perform kurtosis analysis on the weights of the convolution layer. If the kurtosis value is below a preset threshold, it indicates that the weight distribution is flat, and a low bit width is allocated; if the kurtosis value is above the threshold, the high bit width is retained; Use a symmetric quantization strategy to map the weights to a zero-centered quantization interval. The quantized weights reduce the truncation error through an error compensation algorithm.

6. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation of claim 1, characterized in that, It also includes a multispectral image acquisition unit, which specifically includes: A visible-near-infrared dual-band camera mounted on a deep-sea vehicle, which records water depth data synchronously during shooting; Adjust the exposure time of the camera according to the water depth data: use short exposure time in shallow water to suppress overexposure, and use long exposure time in deep water to improve the signal-to-noise ratio. The adjustment step of the exposure time is adaptively set according to the water depth gradient; Input the collected multispectral images into the image enhancement module and bind them with the water depth data for dynamic correction of subsequent band compensation parameters.

7. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation of claim 3, characterized in that, The weight generation process of the channel attention module includes: Global average pooling is performed on the multi-scale feature maps to generate global feature description vectors for each channel; The description vectors are input into a subnetwork containing two fully connected layers, the first layer compresses the dimension to 1 / 4 of the original channel number, and the second layer restores to the original channel number; The channel attention weight between 0 and 1 is generated by the Sigmoid function, and the weight is multiplied with the original feature map channel by channel to strengthen the response of the sulfide related feature channel.

8. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation of claim 1, characterized in that, The real-time control method of the system comprises: Step 1: Map the segmentation mask to the navigation coordinate system of the deep-sea vehicle, and generate a three-dimensional distribution map of the sulfide deposit combined with the vehicle pose data; Step 2: Plan the sampling path according to the density distribution of the deposit in the three-dimensional distribution map, and avoid the high reflection area of the hydrothermal vent; Step 3: Trigger the mechanical arm control signal through FPGA hardware to perform fixed-point sampling on the target sulfide area, and the delay time of the sampling instruction is synchronized and calibrated according to the calculation period of the pipeline acceleration unit.

Images