Deep sea polymetallic sulfide deposit recognition system based on image enhancement and semantic segmentation

Abstract

The invention relates to the technical field of image data processing, in particular to a deep-sea polymetallic sulfide deposit recognition system based on image enhancement and semantic segmentation, and the system comprises a deep-sea image enhancement module which is used for carrying out illumination compensation and noise suppression on a deep-sea multispectral image and comprises a multiband illumination compensation unit; an adaptive noise suppression unit; the multispectral semantic segmentation module comprises a multi-scale feature fusion network; a hot liquid nozzle suppression unit; the lightweight deployment module is connected with the output end of the multispectral semantic segmentation module, is used for deploying the model to deep sea hardware, and comprises a dynamic fixed-point quantization unit; and the parallel pipeline acceleration unit is used for splitting the image enhancement and segmentation network into independent calculation units based on the FPGA and realizing parallel execution of data read-write and calculation through a double-buffer mechanism. The system not only can ensure the precise identification of the sulfide ore deposit in the deep sea environment, but also can efficiently operate on low-power hardware, and improves the automation level and the working efficiency of deep sea ore deposit exploration.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation. Background Art

[0002] Deep-sea polymetallic sulfide deposits are important seabed mineral resources, and their efficient exploration relies on the accurate identification of sulfide structures in seabed images. However, due to the extreme deep-sea environment, existing technologies face the following bottlenecks in practical application: In deep-sea environments, the penetration of visible light and near-infrared spectra differ significantly. The absorption and scattering of light by the water results in low overall image contrast, with subtle grayscale differences between sulfide areas and background sediments. Existing image enhancement algorithms (such as histogram equalization and Retinex) are mostly designed for terrestrial or shallow-sea environments. Their core assumptions are uniform illumination distribution or a single type of noise, making them unable to effectively distinguish the multi-band attenuation characteristics of the deep sea (for example, the near-infrared band attenuates much faster than visible light). Furthermore, the Gaussian noise introduced by deep-sea cameras due to their long exposure requirements is mixed and superimposed with the impulse noise caused by water disturbances. Traditional filtering methods (such as mean filtering and bilateral filtering) smooth out the noise while destroying subtle details at the edges of the sulfides, further reducing image usability.

[0003] Traditional semantic segmentation models (such as FCN and the DeepLab series) primarily rely on spatial texture features in the visible light band. However, the identification of deep-sea sulfides requires a comprehensive utilization of reflectance differences between the visible and near-infrared spectra. Because hydrothermal vent regions exhibit high reflectance in the near-infrared band, similar to sulfides, existing models tend to misclassify them as sulfides, leading to blurred segmentation boundaries and category confusion. Although some studies have attempted to incorporate multispectral data, these feature fusion methods often rely on simple channel splicing, failing to model the coupled relationship between the multi-scale texture of sulfides (such as blocky and chimney-like structures) and spectral reflectance. Consequently, the segmentation accuracy is insufficient to meet actual exploration needs.

[0004] Existing deep-sea image processing systems typically rely on manually annotated data to train high-parameter models (such as ResNet and VGG) and are not optimized for embedded hardware, resulting in slow inference speeds (typically below 10 FPS). Real-time navigation and sampling operations for deep-sea vehicles require online access to mineral deposit distribution information. However, existing systems, due to their high computing resource requirements, cannot be deployed end-to-end on low-power FPGA / ASIC hardware, severely limiting exploration efficiency.

[0005] Therefore, there is an urgent need for a technical solution that optimizes the entire chain from image enhancement, semantic segmentation to hardware deployment to break through the bottleneck of mineral deposit identification in extreme deep-sea environments. Summary of the Invention

[0006] Based on the above objectives, the present invention provides a deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation, comprising: The deep-sea image enhancement module is used to perform illumination compensation and noise suppression on deep-sea multispectral images, including: The multi-band illumination compensation unit decomposes the original image into visible light and near-infrared bands, generates compensation curves based on the attenuation characteristics of each band in dark areas, and superimposes the compensation values ​​pixel by pixel to enhance the contrast of the sulfide area. Adaptive noise suppression unit, which divides the filtering area based on the local texture complexity of the image, uses non-local mean filtering to preserve edge details in low-texture areas, and uses adaptive median filtering to suppress impulse noise in high-texture areas; The multispectral semantic segmentation module is connected to the output of the deep-sea image enhancement module and is used to generate pixel-level segmentation results of sulfide deposits, including: The multi-scale feature fusion network takes the visible light and near-infrared dual-channel enhanced images as input, extracts multi-scale sulfide texture features through parallel dilated convolution, and dynamically fuses the features using the channel attention mechanism; The hydrothermal vent suppression unit screens high-reflection areas based on the reflectivity differences in the near-infrared channel and removes false positive candidate areas that overlap with hydrothermal vents through morphological operations. A lightweight deployment module, connected to the output of the multispectral semantic segmentation module, is used to deploy the model to deep-sea hardware. It includes: Dynamic fixed-point quantization unit, which adaptively allocates quantization bit width according to the weight distribution characteristics of the network layer, retains high precision for high dynamic range layers, and compresses low dynamic range layers to low bit width; The parallel pipeline acceleration unit splits the image enhancement and segmentation networks into independent computing units based on FPGA, and realizes parallel execution of data reading and writing and calculation through a double buffering mechanism.

[0007] Preferably, in the multi-band illumination compensation unit, the process of generating the compensation curve includes: After decomposing the original image into visible light band and near infrared band, the grayscale distribution histogram of each band in the dark area of ​​the image is counted respectively; The attenuation ratio of the visible light band and the near-infrared band is calculated based on the proportion of low-frequency components in the grayscale distribution of the dark area. The attenuation ratio of the near-infrared band is dynamically corrected by the water penetration depth model. An exponential compensation curve is generated based on the attenuation ratio, and the compensation intensity of the near-infrared band is dynamically adjusted according to the output value of the penetration depth model. The compensation curve is superimposed on the original image and then output to the noise suppression unit.

[0008] Preferably, in the adaptive noise suppression unit, the method for dividing the local texture complexity includes: The enhanced image is divided into blocks and the gradient amplitude variance of each sub-block is calculated, and the gradient amplitude is extracted by the Sobel operator; Dynamically set the threshold for dividing low-texture and high-texture areas based on gradient variance: if the sub-block gradient variance is lower than the first threshold, it is determined to be a low-texture area; if it is higher than the second threshold, it is determined to be a high-texture area. The first and second thresholds are calculated based on the noise level of the entire image, which is obtained by statistically analyzing the grayscale fluctuations in the flat areas of the image. Non-local mean filtering is used for low-texture areas, and the filter window size is adaptively adjusted according to the sub-block size; adaptive median filtering is used for high-texture areas, and the maximum window size of the filter is dynamically set according to the impulse noise density.

[0009] Preferably, the structure of the multi-scale feature fusion network includes: The encoder consists of three sets of parallel dilated convolutional layers, using convolution kernels with dilation rates of 1, 3, and 5, respectively, to extract sulfide edge features and texture features under different receptive fields. The decoder uses skip connections to fuse the feature maps of each encoder stage with the upsampling results. Before fusion, a channel attention module is introduced. The channel attention module generates a channel description vector through global average pooling. After learning the correlation between channels through the fully connected layer, it generates attention weights and performs weighted enhancement on important feature channels. The output layer inputs the preliminary segmentation results to the hydrothermal vent suppression unit, which filters the high-reflection areas according to the reflectivity threshold of the near-infrared channel. The reflectivity threshold is obtained by statistically analyzing the difference in spectral reflectivity between sulfides and hydrothermal vents in the training dataset.

[0010] Preferably, the execution process of the morphological operation in the hydrothermal vent suppression unit includes: Perform reflectance threshold segmentation on the near-infrared channel image to obtain a binary mask of the high reflectance area; A circular structuring element is used to close the binary mask to merge adjacent high reflectivity areas, where the radius of the structuring element is set according to the typical size range of hydrothermal vents; The mask after the closing operation is logically XORed with the preliminary result of the segmentation network to eliminate the misjudged sulfide candidate areas in the overlapping area and generate the final segmentation mask.

[0011] Preferably, the quantization bit width allocation method in the dynamic fixed-point quantization unit includes: Count the distribution range of the absolute value of the weight of the fully connected layer. If the ratio of the maximum weight value to the minimum weight value exceeds the preset threshold, the high bit width is allocated; otherwise, the low bit width is allocated. Perform kurtosis analysis on the convolutional layer weights. If the kurtosis value is lower than the preset threshold, it indicates that the weight distribution is flat and a low bit width is allocated. If the kurtosis value is higher than the threshold, a high bit width is retained. A symmetric quantization strategy is adopted to map the weights to a quantization interval centered on zero, and the quantized weights are used to reduce the truncation error through an error compensation algorithm.

[0012] Preferably, the double buffering mechanism of the parallel pipeline acceleration unit includes: Two independent buffers are set up, the first buffer is used to store the current frame data output by the image enhancement module, and the second buffer is used to store the previous frame data being processed by the segmentation module; When the image enhancement module completes processing of a frame, it switches the buffer address so that the segmentation module immediately reads the latest data, while the image enhancement module writes the next frame of data to the free buffer; The buffer capacity is dynamically configured based on the FPGA on-chip storage resources and the input image resolution to ensure that the data throughput rates of the two modules match.

[0013] Preferably, a multispectral image acquisition unit is also included, specifically including: A visible light and near-infrared dual-band camera carried by the deep-sea vehicle, which simultaneously records water depth data while shooting; Dynamically adjust the camera exposure time based on water depth data: short exposure time is used to suppress overexposure in shallow water areas, and long exposure time is used to improve the signal-to-noise ratio in deep water areas. The exposure time adjustment step is adaptively set according to the water depth gradient; The collected multispectral images are input into the image enhancement module and bound with the water depth data for the subsequent dynamic correction of the frequency band compensation parameters.

[0014] Preferably, the weight generation process of the channel attention module includes: Perform global average pooling on the multi-scale feature map to generate the global feature description vector of each channel; The description vector is input into a sub-network consisting of two fully connected layers. The first fully connected layer compresses the dimension to 1 / 4 of the original number of channels, and the second layer restores it to the original number of channels. The channel attention weight between 0 and 1 is generated by the Sigmoid function, and the weight is multiplied with the original feature map channel by channel to strengthen the response of the sulfide-related feature channel.

[0015] Preferably, the real-time control method of the system includes: Step 1: Map the segmentation mask to the navigation coordinate system of the deep-sea vehicle and generate a 3D distribution map of the sulfide deposits based on the vehicle’s pose data. Step 2: Plan the sampling path based on the density distribution of the mineral deposit in the 3D distribution map, avoiding the highly reflective areas of the hydrothermal vents; Step 3: The FPGA hardware triggers the robotic arm control signal to perform fixed-point sampling of the target sulfide area. The delay time of the sampling instruction is synchronously calibrated according to the calculation cycle of the pipeline acceleration unit.

[0016] Beneficial effects of the present invention: 1. By incorporating the multi-band attenuation characteristics of the deep sea, this invention designs a multi-band illumination compensation unit and an adaptive noise suppression unit tailored to the attenuation characteristics of different bands (such as visible light and near-infrared). This effectively enhances sulfide structures in deep-sea images, reduces the impact of noise on image detail, and effectively preserves subtle details at the edges of sulfides. Consequently, this invention significantly improves image quality and contrast during deep-sea image enhancement, overcoming the shortcomings of traditional methods in deep-sea environments.

[0017] 2. This invention effectively addresses the issues of blurred segmentation boundaries and category confusion by introducing a joint segmentation technique using visible and near-infrared bands in deep-sea imagery, combined with the multi-scale texture features of sulfides (such as blocky and chimney-like structures). This method utilizes a deep learning model to more accurately model the features of multispectral data, enabling the system to accurately identify sulfides in deep-sea environments, while avoiding the risk of misidentifying hydrothermal vents and improving segmentation accuracy.

[0018] 3. This invention optimizes the system architecture, combines image enhancement with semantic segmentation, and specifically optimizes for low-power hardware (such as FPGAs). It employs hardware acceleration and pipeline computing strategies to effectively improve system inference speed. The system can process deep-sea imagery in real time, accurately identify sulfide deposits, and plan sampling paths while maintaining low power consumption, meeting the requirements of real-time navigation and sampling for deep-sea vehicles. This invention not only improves real-time performance but also reduces system power consumption, significantly enhancing exploration efficiency and breaking through the bottlenecks of traditional methods.

[0019] 4. This invention integrates a full-chain optimization solution that integrates image enhancement, semantic segmentation, and hardware deployment, meticulously designing every step in deep-sea mineral exploration to form an efficient and accurate deep-sea mineral deposit identification system. This system not only provides accurate mineral deposit distribution information but also plans the optimal sampling path based on density distribution. It also uses an FPGA-controlled robotic arm for fixed-point sampling. The entire process is seamless, significantly improving the efficiency and accuracy of deep-sea mineral deposit identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 It is a structural block diagram of the system of the present invention; Figure 2 A flowchart of the steps of the double buffering mechanism of the parallel pipeline acceleration unit of the system of the present invention; Figure 3 The figure is a flow chart of the steps of the real-time control method of the system of the present invention. DETAILED DESCRIPTION

[0022] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. It is also noted that, to provide a more detailed description, the following embodiments are best and preferred embodiments, and those skilled in the art may employ alternative methods for implementing certain known technologies. Furthermore, the accompanying drawings are intended only to provide a more detailed description of the embodiments and are not intended to limit the present invention.

[0023] See Figure 1-Figure 3 An embodiment of the present invention provides a deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation, wherein the task of the deep-sea image enhancement module is to perform illumination compensation and noise suppression on deep-sea multispectral images.

[0024] The multi-band illumination compensation unit decomposes the original image into visible and near-infrared bands. Based on the grayscale characteristics of dark areas within the image, compensation curves are generated. By overlaying compensation values ​​pixel by pixel, the contrast of mineral deposits, particularly sulfide areas, is enhanced. Specifically, the compensation curves are generated by taking into account the attenuation characteristics of different frequency bands and the light attenuation model of the water body, ensuring effective compensation for light loss at various depths, thereby improving visibility of the target area.

[0025] The adaptive noise suppression unit dynamically divides the filtering area based on the local texture complexity of the image. In low-texture areas, a non-local means filter is used to preserve edge details and avoid blurring. In high-texture areas, an adaptive median filter is used to effectively suppress impulse noise. This method dynamically adjusts the noise processing strategy to achieve optimal noise suppression in different areas.

[0026] The multispectral semantic segmentation module performs multi-scale feature extraction and semantic segmentation on the enhanced image.

[0027] Specifically, the multi-scale feature fusion network takes visible and near-infrared enhanced images as input and uses parallel dilated convolutions to extract multi-scale sulfide texture features, further enhancing both local and global feature representations. Furthermore, the network uses a channel-attention mechanism to automatically focus on feature channels that are critical to sulfide deposits, thereby improving recognition accuracy.

[0028] The hydrothermal vent suppression unit identifies highly reflective areas within the image based on differences in near-infrared reflectivity and removes misidentified hydrothermal vent areas through morphological operations. Morphological closing operations merge adjacent highly reflective areas, accurately removing misidentified mineral deposits that overlap with hydrothermal vents and ensuring accurate segmentation results.

[0029] Furthermore, the lightweight deployment module deploys the trained model to deep-sea hardware, ensuring that the system runs efficiently in a resource-constrained environment.

[0030] Specifically, the dynamic fixed-point quantization unit dynamically adjusts the quantization bit width based on the weight distribution characteristics of the network layer. For layers with a high dynamic range, high quantization accuracy is retained, while for layers with a low dynamic range, the bit width is compressed to reduce the computational burden. This dynamic quantization method effectively balances computational accuracy and hardware resource utilization, reducing energy consumption and computational latency.

[0031] The parallel pipeline acceleration unit utilizes FPGA technology to split image enhancement and semantic segmentation tasks into independent computational units, enabling parallel execution of data reading and computation through a double-buffering mechanism. This parallel design significantly enhances the system's real-time processing capabilities, making it particularly well-suited for the rapid data processing demands of deep-sea environments.

[0032] In one possible implementation, the original deep-sea image is first decomposed into two distinct frequency bands: the visible light band and the near-infrared band. The visible light band and the near-infrared band each have different light attenuation characteristics during image processing, requiring independent compensation. After image decomposition, the grayscale distribution histogram of the dark regions within each frequency band is calculated. Specifically, the distribution of pixels in regions of low grayscale values ​​(i.e., darker illumination) within the image is calculated for each frequency band. This step provides basic data for the subsequent calculation of attenuation ratios, helping to determine the degree of light attenuation in each frequency band.

[0033] After obtaining the grayscale distribution histogram for the dark regions of each frequency band, the next task is to calculate the attenuation ratio based on the low-frequency components (i.e., the darker grayscale portions) in the histogram. Specifically, the attenuation ratio measures the degree of image attenuation in different frequency bands by analyzing the proportion of low-frequency components. For the visible light band, the attenuation ratio primarily reflects the absorbance of water and the influence of depth. For the near-infrared band, the attenuation ratio also needs to consider the water penetration depth, typically dynamically corrected using a water penetration depth model. This model adjusts the attenuation ratio based on parameters such as the different properties of the water, transparency, and wavelength, thereby more accurately calculating the attenuation effect in the near-infrared band.

[0034] Based on the calculated attenuation ratio, the system generates an exponential compensation curve to correct for insufficiently illuminated portions of the image. This compensation curve takes into account the characteristics of light attenuation, effectively compensating for brightness in dark areas. For the near-infrared band, the compensation intensity is dynamically adjusted based on the output of the penetration depth model, ensuring that the compensation value is adaptable to varying water depths and lighting conditions.

[0035] Once the compensation curve is generated, it is superimposed pixel by pixel onto the original image to perform illumination enhancement. This enhancement process not only increases the brightness of dark areas but also effectively enhances the contrast of target areas such as sulfide deposits, providing a clearer image foundation for subsequent noise suppression and semantic segmentation. The superimposed image is then passed as input to the noise suppression unit for further noise filtering and processing.

[0036] In a possible implementation, the compensation curve generation formula is expressed by the following formula: ; in, is the spectrum band identifier; is the pixel coordinate; is the compensation value; is the basic compensation coefficient, which is calibrated through experiments. In this embodiment, it is set to 1.0 for shallow water and 1.2 for deep water; is the attenuation proportional coefficient, and the formula is: , is the water turbidity coefficient, is the frequency band attenuation rate, In this embodiment, the value for visible light is 0.03 and the value for near infrared is 0.15; For water depth, is the pixel value of the dark area, is the maximum grayscale value of the band;

[0037] In one possible implementation, the enhanced image after illumination compensation is first subjected to block processing. The image is divided into several sub-blocks, each of which is a relatively independent region in the image. For each sub-block, the gradient amplitude variance is calculated. The gradient amplitude can be extracted using the Sobel operator, which calculates the horizontal and vertical gradients of each pixel in the image to obtain the gradient amplitude of that point. A larger gradient amplitude indicates more pronounced edge or texture information in the region, while a smaller gradient amplitude indicates a relatively flat region. By calculating the gradient amplitude variance of each sub-block, a quantitative indicator of regional texture complexity can be obtained.

[0038] The thresholds for demarcating low- and high-texture regions in an image are dynamically set based on the gradient amplitude variance of each sub-block. If the sub-block's gradient variance is below a predetermined first threshold, the sub-block is classified as a low-texture region; if the sub-block's gradient variance is above a predetermined second threshold, the sub-block is classified as a high-texture region. These two thresholds are dynamically calculated based on the overall noise level of the image. The calculation of the noise level relies on the grayscale fluctuation statistics of flat areas in the image, estimating the noise intensity by analyzing areas of the image with minimal grayscale value variation. Adjusting the threshold based on the noise level allows the demarcation of low- and high-texture regions to adapt to varying noise backgrounds.

[0039] Low-Texture Noise Suppression: Non-local mean filtering is used to suppress noise in low-texture areas (i.e., areas with minimal texture variation). Non-local mean filtering smoothes noise by utilizing similar pixel blocks within the image, effectively preserving details in flat areas. In this method, the filter window size is adaptively adjusted based on the sub-block size, ensuring that low-texture areas are properly smoothed while avoiding excessive blurring.

[0040] For highly textured areas (i.e., those with drastic texture changes), an adaptive median filter is used. When processing highly textured areas, the adaptive median filter dynamically adjusts the maximum filter window size based on the density of impulse noise in the signal. In areas with higher noise density, the filter window is larger to better eliminate noise; in areas with lower noise density, the filter window is smaller to preserve image detail and texture structure.

[0041] Through this process, different noise suppression techniques are applied to low-texture and high-texture areas, ensuring that noise is effectively reduced in all regions of the image while preserving important image details. The noise-suppressed image is then passed as input to a subsequent semantic segmentation module, which is used to further identify target areas for deep-sea polymetallic sulfide deposits.

[0042] In a possible implementation, the threshold for dividing the low-texture area and the high-texture area in the image is expressed by the following formula: Low texture area threshold: ; High texture area threshold: ; ;

[0043] in, is the noise variance in the flat area of ​​the image, is the flat area (low texture area) in the image; for The total number of pixels in the region, Pixels The gray value of for The average gray value of the area.

[0044] In one possible implementation, the encoder consists of three parallel dilated convolution layers, each using convolution kernels with dilation rates of 1, 3, and 5. Dilated convolution is a technique for expanding the receptive field of convolution. By using convolution kernels with different dilation rates, multi-scale features in images can be extracted under different receptive fields. Specifically: Convolution kernel with expansion rate 1: This convolution kernel is similar to conventional convolution and can extract detailed features of local areas. It is suitable for capturing detailed texture information.

[0045] Convolution kernel with dilation rate 3: By increasing the size of the receptive field, it can capture medium-scale features such as edges and some texture changes.

[0046] Convolution kernel with a dilation rate of 5: further expands the receptive field and is suitable for capturing a wide range of texture features. It can help identify the boundaries of sulfide deposits and other key areas in larger areas of the image.

[0047] These atrous convolutional layers work in parallel to simultaneously extract different features of the image at multiple scales, thereby helping to more comprehensively understand the complex structures in the image, especially the edge features and texture features of sulfides.

[0048] The decoder uses skip connections to fuse the feature maps extracted from each encoder stage with the upsampling results. Skip connections help preserve detailed information from the encoding stage and combine it with the coarse features from the decoding stage to ensure that details are not lost during image reconstruction. To further improve feature fusion, a channel attention module is introduced before fusion.

[0049] The channel attention module is used to enhance the model's attention to important feature channels. The specific steps are as follows: First, global average pooling is used to reduce the dimensionality of each channel's feature map, yielding a global feature representation for each channel. This global average pooled channel description vector is fed into a fully connected layer to learn inter-channel correlations. After passing through the fully connected layer, an attention weight is generated for each channel. This weight is used to weight each channel in the feature map, enhancing those channels that are more important for sulfide deposit identification.

[0050] At the output layer, the feature map processed by the decoder is input to the hydrothermal vent suppression unit. The purpose of the hydrothermal vent suppression unit is to filter out areas of high reflectivity based on the near-infrared channel reflectivity information in the image. These areas typically correspond to hydrothermal vents (i.e., areas of high-temperature liquid produced by deep-sea hydrothermal vents). These areas usually appear as bright areas with high reflectivity in the image. The specific implementation steps are as follows: The reflectance threshold is determined by statistically analyzing the difference in spectral reflectance between sulfides and hydrothermal vents in the training dataset. This threshold effectively distinguishes sulfide deposits from hydrothermal vent areas. When the reflectance of an area in the image exceeds the threshold, it is marked as a hydrothermal vent and removed from the final segmentation results, preventing misidentification of hydrothermal vents as deposits.

[0051] In one possible implementation, the near-infrared channel image is first extracted for reflectance threshold segmentation. Variations in near-infrared channel reflectance can effectively distinguish different objects and regions, as hydrothermal vents and sulfide deposits have significantly different reflectance characteristics in the near-infrared band.

[0052] By analyzing the training data, a reasonable reflectance threshold is determined. This threshold is used to segment the near-infrared channel image, generating a binary mask. In this binary mask, areas with a value of 1 represent areas with high reflectance (such as hydrothermal vents), while areas with a value of 0 represent other areas (such as sulfide deposits or background).

[0053] In a possible implementation, the reflectivity threshold can be expressed by the following formula: Sulfide region reflectivity threshold: ; Hydrothermal vent region reflectivity threshold: ; in, is the mean reflectivity of the sulfide area, obtained through statistics of the training set; is the standard deviation of reflectivity in the sulfide area, obtained through statistics of the training set; is the mean reflectivity of the hydrothermal vent area, obtained through statistics of the training set; is the standard deviation of reflectivity in the hydrothermal vent area, obtained through statistics of the training set;

[0054] To further optimize the reflectivity segmentation results, a circular structuring element is used to close the binary mask after reflectivity threshold segmentation. Closing is a morphological operation that effectively removes small holes and connects adjacent high-reflectivity areas by first performing a dilation operation and then an erosion operation.

[0055] The radius of the circular structuring element is set based on the typical size range of hydrothermal vents. Hydrothermal vents typically have a certain size. Choosing the appropriate structuring element radius ensures that the merged region conforms to the typical morphology of hydrothermal vents and avoids mistakenly merging too small regions. The closing operation effectively connects adjacent high-reflectivity regions, reducing errors and improving the integrity of the hydrothermal vent region, providing a more accurate mask for subsequent screening.

[0056] The mask after the closing operation is logically XORed with the initial segmentation result of the segmentation network (i.e., the segmentation result of the mineral deposit). The purpose of the logical XOR operation is to eliminate the misjudged sulfide candidate areas in the overlapping areas of the two.

[0057] A logical XOR operation compares the two binary images, retaining only areas where the two masks disagree. Consequently, overlapping regions, labeled as both hydrothermal vents and sulfide deposits, are removed. This step helps eliminate misidentified sulfide regions and ensures the accuracy of the final segmentation. This step eliminates areas of mineral deposits that were misidentified due to the higher reflectivity of hydrothermal vents, ensuring that the final segmentation mask contains only true sulfide deposits, avoiding the misidentification of hydrothermal vents and mineral deposits.

[0058] After completing these steps, a final segmentation mask is generated. This mask is the result of reflectance threshold segmentation, closed optimization, and logical XOR operations to remove false positives. This final mask accurately marks the sulfide deposit areas in the deep-sea image.

[0059] In one possible implementation, during the quantization process, the weights of the fully connected layer are first counted and the distribution range of their absolute values ​​is calculated. Specifically, the maximum and minimum values ​​of each weight in the fully connected layer are first obtained. Then the ratio of the two is calculated: If the ratio of the maximum weight value to the minimum weight value is greater than a preset threshold, this indicates that the distribution of weight values ​​is quite different. To maintain high accuracy and detail, the system will allocate a higher quantization bit width to ensure that a larger weight range can be accurately represented.

[0060] It can be understood that the value of the fully connected layer weight range threshold is determined by analyzing the weight distribution histogram of the fully connected layer of a typical neural network, and selecting the critical ratio that can distinguish between "wide dynamic range" and "narrow dynamic range" as the threshold;

[0061] When the distribution range of weights is large, high-bitwidth quantization can be used to represent these weight values ​​more finely and avoid information loss.

[0062] Next, we analyze the kurtosis of the convolutional layer weights. Kurtosis is an indicator of the shape of the probability distribution, reflecting the degree of sharpness of the weight distribution: If the weight kurtosis value is low, it indicates that the weight distribution is relatively flat and the information concentration is low. In this case, the system believes that too high quantization precision is not required and will allocate a lower bit width. If the weight distribution shows a sharp peak (i.e., a high kurtosis value), it means that the weight information is relatively concentrated and higher quantization precision is required to retain this important weight information. Therefore, the system will allocate a higher bit width.

[0063] It can be understood that the convolution layer kurtosis threshold is based on the kurtosis statistical histogram of the convolution layer weights, and the standard normal distribution kurtosis value is used as the kurtosis threshold to distinguish between "flat distribution" and "peaked distribution".

[0064] To ensure that the quantized weights are evenly distributed and not biased towards either side, the system adopts a symmetric quantization strategy. Under this strategy, the quantization interval of the weights is centered on zero, that is, the positive and negative weights are distributed within the same quantization range. This means that the weight values ​​are mapped to a symmetrical quantization interval, reducing bias and maintaining information balance during the quantization process. This symmetric quantization helps improve the accuracy of the quantized weights, especially when dealing with asymmetric weight distribution, and can effectively avoid truncation errors and information loss.

[0065] During the quantization process, weights are quantized into integers or finite precision representations, which may lead to truncation errors. In order to reduce this error, the system uses an error compensation algorithm: The error compensation algorithm corrects the quantized weights to reduce the errors generated during the quantization process, ensuring that the performance of the network will not be significantly reduced due to the reduction in quantization accuracy.

[0066] Specifically, kurtosis The calculation formula is as follows: ; in, It is the convolutional layer weight values, is the mean of the weights, is the number of weights.

[0067] Assume that the quantization interval is , then each weight The quantitative mapping can be performed using the following formula: ; in, is the quantization step size (the step size of the quantization interval), and round indicates the rounding operation. It will be mapped to the quantization interval, preserving the original weight information as much as possible.

[0068] Assumptions is the quantized weight, is the original floating point weight, and the error compensation can be expressed as: ; in, To compensate for the errors calculated by the algorithm, dynamic adjustments are usually made through optimization algorithms (such as backpropagation).

[0069] In one possible implementation, the core of the double buffering mechanism is to set up two independent buffers: The first buffer is used to store the current frame data output by the image enhancement module.

[0070] Second buffer: used to store the previous frame data being processed by the segmentation module.

[0071] This buffer design allows the two modules (image enhancement and semantic segmentation modules) to work in parallel, avoiding resource conflicts between the two and improving overall processing efficiency.

[0072] Whenever the image enhancement module completes processing of a frame of data, the system switches the buffer address, namely: The image enhancement module writes the next frame data into an idle buffer (i.e., a buffer that is not currently in use).

[0073] The segmentation module will immediately read data from the first buffer (the buffer storing the current frame data) and start the segmentation process for the next frame.

[0074] This switching mechanism enables the two modules to connect seamlessly, ensuring data fluidity and parallelism, thereby avoiding the image enhancement module and the semantic segmentation module waiting for each other, allowing the system to work continuously and stably.

[0075] The buffer capacity is dynamically configured based on the system's FPGA on-chip storage resources and the input image resolution. Specifically: The system dynamically adjusts the buffer size based on the FPGA chip's memory capacity to ensure optimal use of memory resources. As the input image resolution changes, the buffer capacity needs to be adjusted accordingly. Higher-resolution images require more storage space, while lower-resolution images can use a smaller buffer.

[0076] This dynamic configuration ensures that the buffer zone meets data throughput requirements while not exceeding the system's hardware resource limitations.

[0077] In one possible implementation, a dual-band visible-near-infrared camera onboard a deep-sea vehicle integrates visible and near-infrared imaging capabilities, enabling the acquisition of multispectral images of deep-sea environments. Multispectral imaging can better identify the spectral characteristics of different substances, making it particularly suitable for identifying and analyzing deep-sea mineral deposits.

[0078] While capturing images, the camera also records water depth data through sensors. This data is crucial for subsequent exposure time adjustment and image enhancement processing.

[0079] In shallow water areas, the strong light can easily lead to overexposure of the image. Therefore, the system will use a short exposure time to avoid loss of image details, reduce overexposure, and ensure image clarity.

[0080] In deep water, where light is weak, the system uses a long exposure time to improve the signal-to-noise ratio (SNR). This increases the camera's light sensitivity, improving image quality and reducing noise, resulting in higher-quality images.

[0081] Exposure time adjustment is an adaptive setting based on the water depth gradient, automatically adjusting the step size and exposure time based on actual water depth. This dynamic adjustment mechanism enables the system to automatically optimize image acquisition at different depths, avoiding the problem of under- or over-setting manually.

[0082] The collected multispectral images are then transferred to the image enhancement module for further processing. During this process: The image is combined with the simultaneously recorded depth data, and the two data are used together. The combined effect of depth information and image data can help to dynamically correct the subsequent band compensation parameters, further improving image quality. Band compensation is particularly important for enhancing image detail and improving resolution in complex deep-sea environments.

[0083] Dynamically adjusting the exposure time can automatically optimize the image exposure according to changes in water depth, ensuring the best image quality regardless of shallow or deep water environments, thereby improving the accuracy of subsequent image analysis and processing.

[0084] By adjusting the long exposure time, the signal-to-noise ratio in deep-water areas is effectively improved, reducing the noise in the image, ensuring image clarity, and providing more accurate data support for the identification of deep-sea mineral deposits.

[0085] The system can adaptively adjust the exposure time according to different water depth gradients, avoiding the image overexposure or underexposure problems caused by traditional fixed exposure time, and improving the system's adaptability in various water depth environments.

[0086] The binding of water depth data and image data enables more accurate dynamic correction of frequency band compensation, thereby optimizing the image enhancement effect and improving the accuracy of deep-sea mineral deposit identification.

[0087] In one possible implementation, multi-scale feature maps are depth feature maps at different scales obtained after preliminary image processing and feature extraction modules. These feature maps typically contain image information at different levels and different spatial locations.

[0088] Global average pooling is performed on these feature maps. This operation averages all pixel values ​​in each channel to generate a global feature description vector for each channel. The purpose of global average pooling is to capture the global characteristics of the channel by aggregating the overall information within the channel and retain the important information of each channel.

[0089] The fully connected layer subnetwork contains two fully connected layers. The first layer compresses the dimension of the global feature description vector to 1 / 4 of the original number of channels. The purpose is to reduce the dimensionality of the features, thereby reducing the amount of calculation and improving the efficiency of the model.

[0090] The second fully connected layer restores the feature dimension to the original number of channels, ensuring that the final output attention weights have the same number of channels as the input feature map. The core purpose of this process is to abstract the importance of each channel through dimensionality reduction and dimensionality recovery.

[0091] After passing through two fully connected layers, a channel attention vector is obtained. The value of this vector is activated by the Sigmoid function, limiting its range to between 0 and 1. The Sigmoid function normalizes the weight of each channel to a value between 0 and 1, which represents the degree of activation of each channel.

[0092] The final channel attention weight represents the importance of each channel in the feature map. The closer the weight is to 1, the more important the feature of the channel is, and the closer the weight is to 0, the less important the feature of the channel is.

[0093] The generated channel attention weights are multiplied channel by channel with the original feature map to produce a weighted feature map. This approach strengthens the network's response to sulfide-related feature channels and suppresses responses to irrelevant channels. This weighting process enables the network to focus more on key channel features in deep-sea mineral deposit images during subsequent feature learning, improving the system's ability to identify deep-sea polymetallic sulfide deposits.

[0094] In one possible implementation, the segmentation mask is pixel-level segmentation information of the mineral deposit area obtained by a deep learning network during the initial image enhancement and semantic segmentation process. It represents the spatial distribution of sulfide deposits in deep-sea images.

[0095] The segmentation mask is mapped to the deep-sea vehicle's navigation coordinate system. This step uses coordinate transformation techniques to align pixel positions in the image with the vehicle's real-world coordinates. Typically, the vehicle's pose data (such as position, attitude, and heading) is acquired through sensors and combined with the position data of each point in the segmentation mask to calculate its coordinates in 3D space.

[0096] The resulting 3D distribution map contains spatial information about the deposit area, providing a reference for subsequent deposit sampling. This 3D map not only considers the location of the deposit but also reflects its morphology and distribution characteristics, helping the navigation system accurately understand the spatial layout of the deposit.

[0097] Based on the 3D distribution map, the sampling path is planned by analyzing the density distribution of the ore deposit. The density distribution information reflects the distribution density of the ore deposit area, thus determining which areas are more suitable for sampling.

[0098] When planning sampling routes, it is necessary to avoid the highly reflective areas of hydrothermal vents. Hydrothermal vents typically have strong reflection signals, which can contain unfavorable interference information and even mislead the mineral deposit identification system. Therefore, these highly reflective areas must be avoided during route planning to ensure sampling accuracy.

[0099] FPGA hardware triggering uses a field-programmable gate array (FPGA) to achieve efficient, low-latency control signal output. As a hardware acceleration platform, FPGAs can provide faster and more stable signal responses in scenarios with high real-time requirements.

[0100] The system uses an FPGA to trigger control signals from the robotic arm, allowing it to precisely sample the target sulfide area. This process requires the robotic arm to accurately reach the target location and perform sampling operations based on the planned sampling path.

[0101] The delay time synchronization calibration of the sampling instruction means that during the multi-step processing process, due to the certain calculation cycle delay of the pipeline acceleration unit in the system (involving image processing, data analysis, etc.), the timing of the sampling instruction needs to be adjusted according to these calculation cycles to make the sampling operation consistent with the output of image analysis and path planning, thereby avoiding errors caused by instruction delays.

[0102] Through precise 3D distribution map generation, path planning, robotic arm fixed-point sampling, and a low-latency control trigger mechanism, the sampling efficiency and accuracy of deep-sea polymetallic sulfide deposits can be effectively improved. This technical approach has significant application value, not only optimizing the sampling process but also enhancing the system's ability to operate in complex seabed environments.

[0103] The present invention encompasses any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention. To provide a thorough understanding of the present invention, specific details are described in detail below in connection with the preferred embodiments of the present invention, but those skilled in the art will be able to fully understand the present invention without these detailed descriptions. Furthermore, to avoid unnecessary confusion regarding the essence of the present invention, well-known methods, processes, procedures, components, and circuits have not been described in detail.

[0104] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation, characterized in that: include: The deep-sea image enhancement module is used to perform illumination compensation and noise suppression on deep-sea multispectral images, including: The multi-band illumination compensation unit decomposes the original image into visible light and near-infrared bands, generates compensation curves based on the attenuation characteristics of each band in dark areas, and superimposes the compensation values ​​pixel by pixel to enhance the contrast of the sulfide area. Adaptive noise suppression unit, which divides the filtering area based on the local texture complexity of the image, uses non-local mean filtering to preserve edge details in low-texture areas, and uses adaptive median filtering to suppress impulse noise in high-texture areas; The multispectral semantic segmentation module is connected to the output of the deep-sea image enhancement module and is used to generate pixel-level segmentation results of sulfide deposits, including: The multi-scale feature fusion network takes the visible light and near-infrared dual-channel enhanced images as input, extracts multi-scale sulfide texture features through parallel dilated convolution, and dynamically fuses the features using the channel attention mechanism; The hydrothermal vent suppression unit screens highly reflective areas based on the reflectivity differences in the near-infrared channel and removes falsely identified candidate areas that overlap with hydrothermal vents through morphological operations. A lightweight deployment module, connected to the output of the multispectral semantic segmentation module, is used to deploy the model to deep-sea hardware. It includes: Dynamic fixed-point quantization unit, which adaptively allocates quantization bit width according to the weight distribution characteristics of the network layer, retains high precision for high dynamic range layers, and compresses low dynamic range layers to low bit width; The parallel pipeline acceleration unit splits the image enhancement and segmentation networks into independent computing units based on FPGA, and realizes parallel execution of data reading and writing and calculation through a double buffering mechanism.

2. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1 is characterized in that: In the multi-band illumination compensation unit, the compensation curve generation process includes: After decomposing the original image into visible light band and near infrared band, the grayscale distribution histogram of each band in the dark area of ​​the image is counted respectively; The attenuation ratio of the visible light band and the near-infrared band is calculated based on the proportion of low-frequency components in the grayscale distribution of the dark area. The attenuation ratio of the near-infrared band is dynamically corrected by the water penetration depth model. An exponential compensation curve is generated based on the attenuation ratio, and the compensation intensity of the near-infrared band is dynamically adjusted according to the output value of the penetration depth model. The compensation curve is superimposed on the original image and then output to the noise suppression unit.

3. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1 is characterized in that: In the adaptive noise suppression unit, the method for dividing the local texture complexity includes: The enhanced image is divided into blocks and the gradient amplitude variance of each sub-block is calculated, and the gradient amplitude is extracted by the Sobel operator; Dynamically set the threshold for dividing low-texture and high-texture areas based on gradient variance: if the sub-block gradient variance is lower than the first threshold, it is determined to be a low-texture area; if it is higher than the second threshold, it is determined to be a high-texture area. The first and second thresholds are calculated based on the noise level of the entire image, which is obtained by statistically analyzing the grayscale fluctuations in the flat areas of the image. Non-local mean filtering is used for low-texture areas, and the filter window size is adaptively adjusted according to the sub-block size; adaptive median filtering is used for high-texture areas, and the maximum window size of the filter is dynamically set according to the impulse noise density.

4. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1 is characterized in that: The structure of the multi-scale feature fusion network includes: The encoder consists of three sets of parallel dilated convolutional layers, using convolution kernels with dilation rates of 1, 3, and 5, respectively, to extract sulfide edge features and texture features under different receptive fields. The decoder uses skip connections to fuse the feature maps of each encoder stage with the upsampling results. Before fusion, a channel attention module is introduced. The channel attention module generates a channel description vector through global average pooling. After learning the correlation between channels through the fully connected layer, it generates attention weights and performs weighted enhancement on important feature channels. The output layer inputs the preliminary segmentation results to the hydrothermal vent suppression unit, which filters the high-reflection areas according to the reflectivity threshold of the near-infrared channel. The reflectivity threshold is obtained by statistically analyzing the difference in spectral reflectivity between sulfides and hydrothermal vents in the training dataset.

5. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1 is characterized in that: The execution process of the morphological operation in the hydrothermal vent suppression unit includes: Perform reflectance threshold segmentation on the near-infrared channel image to obtain a binary mask of the high reflectance area; A circular structuring element is used to close the binary mask to merge adjacent high reflectivity areas, where the radius of the structuring element is set according to the typical size range of hydrothermal vents; The mask after the closing operation is logically XORed with the preliminary result of the segmentation network to eliminate the misjudged sulfide candidate areas in the overlapping area and generate the final segmentation mask.

6. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1 is characterized in that: The method for allocating quantization bit width in the dynamic fixed-point quantization unit includes: Count the distribution range of the absolute value of the weight of the fully connected layer. If the ratio of the maximum weight value to the minimum weight value exceeds the preset threshold, the high bit width is allocated; otherwise, the low bit width is allocated. Perform kurtosis analysis on the convolutional layer weights. If the kurtosis value is lower than the preset threshold, it indicates that the weight distribution is flat and a low bit width is allocated. If the kurtosis value is higher than the threshold, a high bit width is retained. A symmetric quantization strategy is adopted to map the weights to a quantization interval centered on zero, and the quantized weights are used to reduce the truncation error through an error compensation algorithm.

7. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1 is characterized in that: The double buffering mechanism of the parallel pipeline acceleration unit includes: Two independent buffers are set up, the first buffer is used to store the current frame data output by the image enhancement module, and the second buffer is used to store the previous frame data being processed by the segmentation module; When the image enhancement module completes processing of a frame, it switches the buffer address so that the segmentation module immediately reads the latest data, while the image enhancement module writes the next frame of data to the free buffer; The buffer capacity is dynamically configured based on the FPGA on-chip storage resources and the input image resolution to ensure that the data throughput rates of the two modules match.

8. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1 is characterized in that: It also includes a multispectral image acquisition unit, specifically including: A visible light and near-infrared dual-band camera carried by the deep-sea vehicle, which simultaneously records water depth data while shooting; Dynamically adjust the camera exposure time based on water depth data: short exposure time is used to suppress overexposure in shallow water areas, and long exposure time is used to improve the signal-to-noise ratio in deep water areas. The exposure time adjustment step is adaptively set according to the water depth gradient; The collected multispectral images are input into the image enhancement module and bound with the water depth data for the subsequent dynamic correction of the frequency band compensation parameters.

9. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 4, characterized in that: The weight generation process of the channel attention module includes: Perform global average pooling on the multi-scale feature map to generate the global feature description vector of each channel; The description vector is input into a sub-network consisting of two fully connected layers. The first fully connected layer compresses the dimension to 1 / 4 of the original number of channels, and the second layer restores it to the original number of channels. The channel attention weight between 0 and 1 is generated by the Sigmoid function, and the weight is multiplied with the original feature map channel by channel to strengthen the response of the sulfide-related feature channel.

10. The deep-sea polymetallic sulfide deposit identification system based on image enhancement and semantic segmentation according to claim 1, characterized in that: The real-time control method of the system includes: Step 1: Map the segmentation mask to the navigation coordinate system of the deep-sea vehicle and generate a 3D distribution map of the sulfide deposits based on the vehicle’s pose data. Step 2: Plan the sampling path based on the density distribution of the mineral deposit in the 3D distribution map, avoiding the highly reflective areas of the hydrothermal vents; Step 3: The FPGA hardware triggers the robotic arm control signal to perform fixed-point sampling of the target sulfide area. The delay time of the sampling instruction is synchronously calibrated according to the calculation cycle of the pipeline acceleration unit.

Images