A method for quantitatively observing submarine rock outcrop area based on cascaded deep learning

Abstract

The present application relates to the field of image data processing, in particular to a method for quantitatively observing the outcrop area of seabed rock based on cascaded deep learning, which specifically comprises the following steps: obtaining the current frame image of the underwater video stream, generating a bottom material confirmation signal after determining that the bottom is seen by the first model; in response to the signal, using the second model to distinguish the bottom material type to distinguish the first type of bottom material that does not need to be segmented and the second type of bottom material that needs to be segmented, and simultaneously using the third model to dynamically calibrate the scale factor; then, according to the classification result, the area calculation is performed: for the first type of bottom material, the physical area is calculated based on the whole frame pixel area and the scale factor; for the second type of bottom material, the fourth model is used to segment the target individual, and the physical area is calculated based on the total pixel area of the target individual and the scale factor. The present application is based on cascaded deep learning, and realizes adaptive, full-process, high-precision real-time quantitative observation of the outcrop area of seabed rock.

Description

Technical Field

[0001] This invention relates to the field of image data processing, specifically to a method for quantitative observation of the exposed area of ​​seabed rocks based on cascaded deep learning. Background Technology

[0002] The exposed area and spatial distribution of seabed rocks (including bedrock, crusts, and gravel) are key parameters for deep-sea mineral resource exploration, seabed geomorphological mapping, marine engineering geological assessment, and benthic ecosystem research. For rocks associated with mineral resources such as cobalt-rich crusts and polymetallic nodules, quantitative information such as their coverage and grain size distribution is crucial for resource estimation and mining feasibility assessment. Accurately and efficiently acquiring these parameters has significant scientific and engineering value for improving the automation level of marine surveys and reducing the cost of manual interpretation.

[0003] However, the attenuation of light and water turbidity in underwater environments pose significant challenges to automated quantitative image-based observations. In deep-sea near-bottom observations, the field of view covered by each image frame is limited (typically on the order of meters), and within this field of view, the seabed sediment usually exhibits a single dominant type: a pure sediment-covered area, a continuous crust-covered area, or a mixed sediment area with both rocks and sediments (collectively referred to as "rubble" in this invention). Existing technologies have the following main shortcomings in such tasks: (1) Lack of a multi-branch processing strategy that adapts to seabed sediment. Existing methods typically use a single segmentation or detection model to process all types of seabed images. This "one-size-fits-all" strategy ignores the fundamental differences in visual features and physical morphology between different seabed sediments. For example, sediment-covered areas are often continuous, homogeneous regions without fixed boundaries, which are suitable for calculating the total area; while gravel areas are composed of a large number of discrete individuals with different shapes and overlapping each other, requiring individual segmentation before calculating the total area, which a single model cannot handle simultaneously.

[0004] (2) The observation chain is incomplete and the level of automation is low. From determining whether the detection equipment is in place (bottom visible), to distinguishing the type of seabed, and then to performing targeted quantitative calculations, existing technologies often separate these steps, requiring a lot of manual intervention or offline post-processing. In particular, in dynamic scale measurements based on laser calibration, there is a lack of an end-to-end automated framework that can be seamlessly integrated with seabed identification and area calculation, resulting in a lag in the output of measurement results, which cannot meet the needs of real-time or near-real-time deep-sea operations.

[0005] (3) The quantitative analysis capability for exposed rock area, especially mixed substrate (gravel) area, is insufficient. For large areas of bedrock or crust, a single model is difficult to handle large-area coverage and local details at the same time. For mixed substrate areas where rocks and sediments coexist, the rocks often have a highly aggregated and mutually adhered distribution. Traditional segmentation methods often lack an effective overlap processing mechanism, which can easily lead to the same pixel being counted multiple times or the inaccurate determination of rock boundaries, directly affecting the calculation accuracy of the core indicator of "exposed area".

[0006] In summary, there is an urgent need for a cascaded automated observation method that can dynamically adapt to different substrate types and integrates bottom detection, substrate classification, adaptive quantitative analysis, and dynamic scale calibration. Summary of the Invention

[0007] To address the problems of poor seabed adaptability, incomplete observation chains, and susceptibility to failure due to dense gravel fragmentation in existing technologies, this invention proposes a quantitative observation method for seabed rock exposure area based on cascaded deep learning, specifically including the following steps: S1: Acquire the current frame image in the underwater video stream, and use the first model to determine the bottom-reaching status of the current frame image; when the bottom-reaching status is determined, generate a bottom confirmation signal and trigger subsequent processing procedures; S2: In response to the substrate confirmation signal, the substrate type of the current frame image is determined using the second model to obtain a substrate classification result; wherein, the substrate classification result includes a first type of substrate and a second type of substrate; the first type of substrate is a substrate type that does not require instance segmentation, and the second type of substrate is a substrate type that requires instance segmentation; simultaneously, the current frame image is dynamically scaled using the third model to obtain a scale factor; S3: Execute the corresponding area calculation branch according to the substrate classification result: when it is determined to be the first type of substrate, directly calculate the physical area based on the pixel area of ​​the whole frame image and the scale factor; when it is determined to be the second type of substrate, use the fourth model to perform instance segmentation on the target individuals in the second type of substrate, and calculate the physical area based on the total pixel area of ​​the segmented target individuals and the scale factor.

[0008] Based on a cascaded deep learning architecture, this invention completes adaptive seabed discrimination, full-process observation, and stable segmentation of dense targets. It can perform automated, high-precision real-time quantitative calculation of the exposed area of ​​seabed rocks, effectively improving the reliability, completeness, and engineering application value of area calculation in complex seabed scenarios.

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows: To address the problem of poor adaptability to sediment in existing technologies, this invention employs a multi-branch quantitative strategy. Sediment is classified immediately after bottom exposure, and its classification is routed to different computational branches based on the different attributes of "seeds," "crusts," and "gravel." For homogeneous sediments and crusts, a simple and efficient full-map area calculation is used; for gravel with complex composition requiring individual-level analysis, a specialized segmentation model is activated. This on-demand customized strategy avoids wasted computational resources in "simple" scenarios while ensuring accurate calculations in "complex" scenarios, achieving an optimal balance between computational efficiency and measurement accuracy.

[0010] To address the incomplete observation chain in existing technologies, this invention uses "bottom detection" as the master switch for automation, organically linking bottom classification, subsurface registration, and laser calibration. The entire process, from the bottom detection frame to the final data output, requires no manual intervention in model selection or offline data registration. Laser point detection and segmentation tasks are executed simultaneously, and scale factors and area calculation results are automatically correlated, achieving end-to-end automation from state judgment and scene understanding to multi-dimensional quantitative parameter output.

[0011] To address the problem of frequent failures in dense gravel segmentation in existing technologies, this invention decomposes the task of calculating the area of ​​the gravel region into the sum of the segmentation results of all individual stones. This effectively solves the area calculation deviation caused by gaps and edges in traditional region segmentation methods. Even when stones are highly adhered and stacked, the instance segmentation model can ensure that the final accumulated area reflects the actual exposed stone area, significantly improving the quantitative accuracy of gravel in this special substrate. Attached Figure Description

[0012] The invention will now be further described with reference to the accompanying drawings.

[0013] Figure 1 This is a flowchart of the steps of the method of the present invention; Figure 2 These are schematic diagrams of the results of sediment classification, laser calibration, and rock segmentation; (a) is a schematic diagram of the detection results for the sediment scene; (b) is a schematic diagram of the detection results for the crust scene; (c) is a schematic diagram of the detection results for the small-diameter gravel scene; and (d) is a schematic diagram of the detection results for the large-diameter gravel scene. Detailed Implementation

[0014] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, such as... Figure 1 As shown, the quantitative observation method for seabed rock exposure area based on cascaded deep learning proposed in this invention specifically includes the following steps: S1: Identify the visual condition of the substrate The system continuously acquires underwater video streams and inputs the acquired video frames into a pre-trained lightweight image classification model (i.e., the first model). The first model performs binary classification on the current frame image to determine whether the current field of view is in a "bottom-view" state. The "bottom-view" state determination criteria are as follows: the first model, based on the overall texture, clarity, and color distribution characteristics of the current frame image, determines whether the current field of view meets the effective feature extraction conditions for seabed texture. When the visual features of the seabed texture in the image are identifiable and not obstructed by suspended matter causing feature extraction failure, it is determined to be a "bottom-view" state. When a "bottom-view" state is determined, a seabed confirmation signal is generated, triggering a cascaded seabed classification, laser point detection, and quantitative analysis process. If a "bottom-not-view" state is determined, the current frame image is discarded, and subsequent video frames are acquired.

[0015] S2: Parallel processing of substrate classification and laser calibration After receiving the substrate confirmation signal, the substrate type determination and laser point dynamic scale calibration operations are performed in parallel on the current frame image, specifically including: S2.1: Substrate type identification The current frame image is input into a pre-trained seabed classification model (i.e., the second model), which is used to identify the seabed seabed type. The output of the second model includes at least three mutually exclusive types: "sediment", "crust" and "gravel".

[0016] Based on the unique classification result output by the second model, perform the corresponding subsequent processing steps: 1. If it is determined to be "sediment", it means that the current field of view is covered by pure sediment and there are no exposed rocks. Jump to step S4 to calculate the overall area. 2. If it is determined to be "crust", it means that the current field of view is completely covered by continuous crust and there is no exposed sediment. Jump to step S4 to calculate the overall area. 3. If it is determined to be "gravel", it represents the mixed substrate state in which stones and sediments coexist in the current field of view, and is not limited by the size, density or coverage ratio of the stones. This triggers the stone segmentation process and proceeds to step S3.

[0017] S2.2: Laser point detection and dynamic scale calibration The current frame image is input into a pre-trained laser spot detection model (i.e., the third model). The third model is used to detect the light spots formed by the projection of parallel laser beams in real time and calculate the pixel distance between the light spots. Based on the known physical spacing of the parallel laser beams and the pixel distance, the pixel-physical size conversion ratio of the current frame image is calculated to obtain the scale factor.

[0018] S3: Stone Segmentation and Area Calculation in the Gravel Zone This step is only performed in step S2 when the sediment type is determined to be "gravel". When the determination result is "sediment" or "crust", since there is no state where stones and sediments coexist in the current field of view (i.e., there are no individual stones in a pure sediment scene, while the stone coverage area can be directly used as the entire map area in a continuous crust scene), there is no need to perform individual-level instance segmentation, so this step is skipped.

[0019] The current frame image is input into a pre-trained stone segmentation model (i.e., the fourth model). The fourth model is used to segment individual stones that are stacked or stuck together in the gravel area, and outputs the category label, instance mask and bounding box of each individual stone. Based on the instance mask of each individual stone output by the fourth model, the sum of the pixel areas of all individual stones is calculated to obtain the total pixel area of ​​the stones in the current gravel region.

[0020] S4: Multi-branch quantitative calculation Obtain the scale factor determined in step S2.2, and perform the corresponding physical area calculation based on the substrate type determination result in step S2.1: 1. When the determination result is "sediment", the pixel area of ​​the entire frame image is multiplied by the square of the scale factor to obtain the physical area covered by the sediment. The calculation formula is as follows: ; in, Indicates the physical coverage area of ​​sediments. and These represent the number of pixels corresponding to the width and height of the current frame image, respectively. The scale factor is expressed in centimeters per pixel.

[0021] 2. When the determination result is "shelling", the pixel area of ​​the entire frame image is multiplied by the square of the scale factor to obtain the physical area of ​​the exposed shell. The calculation formula is as follows: ; in, Indicates the physical coverage area of ​​the crust. , and The definition is the same as above.

[0022] 3. When the determination result is "rubble", the total pixel area of ​​the stones obtained in step S3 is multiplied by the square of the scale factor to obtain the physical area of ​​exposed stones in the rubble region. The calculation formula is as follows: ; in, This represents the total physical exposed area of ​​the target individual within the gravel. This represents the total number of target individuals detected by the fourth model. For the first The instance mask pixel area of ​​each target individual. The scaling factor; In the above formula, the physical coverage area of ​​the first type of substrate or the total physical exposed area of ​​the target individual in the second type of substrate is obtained by multiplying the corresponding pixel area by the square of the scale factor. Finally, at least one of the following is output: sediment coverage area, crust exposure area, and stone exposure area in the gravel region within the current observation area.

[0023] The following detailed description of the technical solution of this invention is based on a quantitative observation scenario of exposed rock area in a deep-sea multi-bed sediment region. It should be understood that this embodiment is only for explaining the invention and does not limit its application scope. This invention is also applicable to other quantitative observation scenarios of seabed rock, such as investigation of bedrock outcrops, mapping of deep-sea gravel beaches, and assessment of rock distribution along subsea pipelines.

[0024] Example 1: Quantitative observation of exposed rock area in deep-sea multi-bed sedimentary regions To fully verify the effectiveness and universality of the method of this invention in actual deep-sea operations, this embodiment selects near-bottom high-definition videos (dive 1 and dive 2) taken during two typical dives in a seamount area for system-level experiments. The videos were taken by a high-definition underwater camera system equipped with parallel lasers spaced 10cm apart, with a resolution of 1920×1080 and a frame rate of 50fps. The two dives were close in time and adjacent in space, and the terrain and seabed conditions of the operating areas differed significantly, which can comprehensively test the adaptability and application performance of this invention under different seabed conditions. Specifically, the operating area of ​​dive 1 was a steep seamount ridge area with a seabed mainly composed of crust and locally accompanied by gravel; the operating area of ​​dive 2 was a gentle seamount valley area with a seabed mainly composed of sediment and gravel. The two dives covered a rich variety of seabed and terrain types. This experiment processed a total of 6053 valid image frames, including 2965 valid image frames of latent level 1 and 3088 valid image frames of latent level 2. The sufficient and differentiated experimental samples can provide strong support for the reliability verification of the method of this invention.

[0025] Acquisition and processing of experimental data This embodiment uses four independent datasets to train the four models in the cascaded framework. All datasets are generated from deep-sea near-bottom optical survey videos after frame extraction and manual annotation. Detailed overviews of each dataset are shown in Table 1.

[0026] Table 1 Overview of training datasets for each model

[0027] The sediment classification dataset contains 5280 images: 880 sediment images, 2300 crust images, and 2300 gravel images. The second model, VGG16, achieved a classification accuracy of 88.91% on the test set with a loss value of 0.021, a computational cost of 15.47 GFLOPs, and 13.84 MB of parameters. The fourth model, YOLOv11s-seg, achieved an mAP50 of 67.4 and an mAP50-90 of 40.6 on the stone instance segmentation task, with a computational cost of 21.5 GFLOPs.

[0028] All four models employed independent training strategies. The training convergence criterion was that the validation set loss did not decrease for 50 consecutive epochs, or reached the preset maximum number of epochs (200 epochs for the first and second models, and 300 epochs for the third and fourth models). Each model achieved optimal performance on its respective task under the given dataset conditions.

[0029] Experimental Design and Results Analysis The method described in this invention is used to automatically process continuously acquired underwater video streams, count the number of rocks in each frame, and map the bottom sediment discrimination results to a geographic coordinate system in combination with positioning and navigation data.

[0030] Experimental results show that the method of this invention can maintain stable operation during long-term continuous observation tasks, effectively distinguish between three types of bottom sediments: sediment, crust, and gravel, and output quantitative area data correlated with geographical location. Furthermore, the cascaded processing framework employed in this invention has good modularity, facilitating the subsequent updates and maintenance of individual models.

[0031] Typical Scene Visualization Analysis To visually demonstrate the processing effect of the method described in this invention, four typical images were selected from 6053 valid images for detailed analysis, corresponding to four scenarios: sediment, crust, small-diameter gravel, and large-diameter nodules. The original images were all acquired by an underwater camera system equipped with dual parallel lasers (10cm apart). The detection and segmentation results are as follows: Figure 2 As shown, the specific processing flow is as follows: S1: Substrate Confirmation Four images are input into the first model (SqueezeNet), and all of them are judged to be in a "bottoming out" state, generating a bottom confirmation signal, which triggers the subsequent cascaded processing flow.

[0032] S2: Parallel processing of substrate classification and laser calibration In response to the sediment confirmation signal, the second model (VGG16) and the third model (YOLOv11s) are initiated in parallel. The sediment classification result output by the second model is as follows: Figure 2In the first two categories, (a) represents "sediment," (b) represents "crust," and (c) and (d) both represent "gravel." Based on this classification, Figure 2 Image data from (a) and (b) directly proceed to step S4; while Figure 2 In steps (c) and (d), branch routing is triggered, activating the fourth model to proceed to step S3. Simultaneously, the third model completes laser spot detection, with the detection results as follows: Figure 2 As shown in the diagram (circles mark the positions of the light spots, and lines mark the spacing between the light spots). Furthermore, in Figure 2 In (c) and (d), the outline of each individual stone is marked with a semi-transparent mask of a different color and labeled with a numerical number. The specific scale calibration results are shown in Table 2.

[0033] Table 2 Results of laser spot detection and scale calibration

[0034] S3: Stone Segmentation and Area Calculation in the Gravel Zone Will Figure 2 Images (c) and (d) are input to the fourth model (YOLOv11s-seg), which performs instance segmentation on the stones in the image. In the segmentation results, each individual stone is identified by a semi-transparent mask and labeled with a unique number (e.g., ...). Figure 2 (As shown in (c) and (d)). According to statistics, Figure 2 A total of 65 individual stones were detected in (c). Figure 2 A total of 22 individual stones were detected in (d). For pixels in overlapping areas, a confidence-based priority allocation strategy was used to assign them to the corresponding stones to avoid duplicate area calculations. The specific area statistics are shown in Table 3.

[0035] Table 3. Statistical Results of Stone Segmentation and Area in the Gravel Zone

[0036] As can be seen from Table 3, Figure 2 (c) and (d) show significant differences in particle size distribution. Figure 2 The stones in (c) are numerous but small in size, mainly small-diameter gravel, with stones smaller than 10 cm² accounting for more than half. Figure 2 The stones in the middle (d) are fewer in number but larger in size, with nearly half of them having an area of ​​more than 30 cm², and the largest single stone having an area of ​​284.96 cm².

[0037] S4: Multi-branch quantitative calculation Based on the scale factor and the branch discrimination results, the physical area conversion was completed, and the specific results are summarized in Table 4.

[0038] Table 4 Summary of Quantitative Observation Results for Cascaded Deep Learning Methods

[0039] As shown in Table 4, the four images cover three typical scenarios of cascaded frames: Figure 2 In (a) and (b), the area can be calculated directly without separating the sediment and crust, which is highly efficient. Figure 2 (c) and (d) refer to the automatic activation segmentation model for gravel, achieving refined surface accumulation. The scale factor dynamically adapts to the shooting distance. Figure 2 The middle (d) image was taken at a closer distance and had a larger scale factor, resulting in larger individual stones being detected.

[0040] This embodiment, based on experimental results from two complete dives totaling 6053 images, fully verifies the effectiveness and universality of the method of the present invention. The two independent dives exhibited consistent seabed distribution patterns in adjacent areas, confirming the operational stability of the method in complex deep-sea environments.

[0041] S5: Experimental Comparison and Verification To verify the superiority of the method of this invention in the quantitative observation of rock outcrop area, a continuous video of about 15 minutes, containing three typical states of "sediment-gravel transition zone-dense gravel", was selected from dive 1 for ablation comparison experiment. Using the actual rock area annotated at the pixel level as a benchmark, the observation errors of the following three schemes were compared: Scheme A is the method of this invention, i.e., the complete cascaded framework; Scheme B is the ablation substrate classification module, which directly performs instance segmentation and accumulates the area for all frames; Scheme C uses the traditional semantic segmentation model U-Net to directly extract the rock cover area.

[0042] Table 5. Comparison of errors of different methods in quantitative observation of rock area

[0043] As shown in Table 5, in pure sediment and crust scenarios, the method of this invention directly routes to the full map area calculation through substrate classification, with zero error, and does not require calling the segmentation model. Schemes B and C, lacking prior guidance from the substrate, are prone to misclassifying background noise as gravel in sediment areas, resulting in errors. In dense gravel areas, due to the high adhesion of rocks, Scheme C struggles to accurately delineate boundaries, leading to overestimation of the area calculated. While Scheme B can perform individual segmentation, the lack of substrate classification guidance often results in missed detections. In crust areas, Scheme B performs unnecessary instance segmentation of continuous surfaces, causing boundary errors, while Scheme C suffers from missegmentation due to the natural texture of the crust surface. In contrast, the method of this invention, through its "determine first, then segment" on-demand scheduling mechanism, maintains the lowest quantitative observation error in all scenarios. Furthermore, thanks to the cascaded framework saving computational overhead of the segmentation model in sediment and crust scenarios, the average processing time of the method of this invention is only 37.5% of that of Scheme C, achieving a balance between high accuracy and high real-time performance.

[0044] In video-level experiments, this invention comprehensively covers the continuous changes in substrate type. The entire cascaded process achieves end-to-end automated processing, from bottom detection and identification, parallel initiation of substrate classification and laser calibration, branch route determination, gravel area segmentation, to quantitative output. Specifically, for sediment and crust scenarios, only two models are needed to complete the calculation; while for gravel scenarios, the segmentation model is automatically activated for refined processing. This mechanism effectively achieves the optimal balance between computational efficiency and measurement accuracy for different substrate types.

[0045] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A method for quantitatively observing the exposed area of ​​seabed rocks based on cascaded deep learning, characterized in that, Includes the following steps: S1: Acquire the current frame image in the underwater video stream, and use the first model to determine the bottom-reaching status of the current frame image; when the bottom-reaching status is determined, generate a bottom confirmation signal and trigger subsequent processing procedures; S2: In response to the substrate confirmation signal, the substrate type of the current frame image is determined using the second model to obtain a substrate classification result; wherein, the substrate classification result includes a first type of substrate and a second type of substrate; the first type of substrate is a substrate type that does not require instance segmentation, and the second type of substrate is a substrate type that requires instance segmentation; simultaneously, the current frame image is dynamically scaled using the third model to obtain a scale factor; S3: Execute the corresponding area calculation branch according to the substrate classification result: when it is determined to be the first type of substrate, directly calculate the physical area based on the pixel area of ​​the whole frame image and the scale factor; when it is determined to be the second type of substrate, use the fourth model to perform instance segmentation on the target individuals in the second type of substrate, and calculate the physical area based on the total pixel area of ​​the segmented target individuals and the scale factor. The calculation steps for the scaling factor in S2 are as follows: The third model is used to detect light spots formed by parallel laser beam projection in real time and to calculate the pixel distance between light spots; The ratio of the preset physical spacing of parallel laser beams to the pixel distance is used as the pixel-physical size conversion ratio of the current frame image to obtain the scale factor. The formula for calculating the physical area based on the pixel area of ​​the entire frame image and the scale factor in S3 is as follows: ； in, This represents the physical coverage area of ​​the first type of substrate. and These represent the number of pixels corresponding to the width and height of the current frame image, respectively. The scale factor is expressed in centimeters per pixel. The formula for calculating the physical area of ​​the target individual based on segmentation and the scale factor is as follows: ； in, This represents the total physical exposed area of ​​the target individual in the second type of substrate. This represents the total number of target individuals detected by the fourth model. For the first The instance mask pixel area of ​​each target individual. The scaling factor is denoted as .

2. The method for quantitative observation of seabed rock outcrop area based on cascaded deep learning according to claim 1, characterized in that, The criteria for determining the bottoming state in S1 are as follows: the first model determines whether the current field of view meets the effective feature extraction conditions for the seabed texture based on the overall texture, clarity and color distribution characteristics of the current frame image. When the visual features of the seabed sediment in the image are identifiable and feature extraction fails due to obstruction by suspended matter in the water, it is determined to be a bottom-revealed state.

3. The method for quantitative observation of seabed rock outcrop area based on cascaded deep learning according to claim 1, characterized in that, The sediment classification results in S2 include sediments, crusts, and gravel; The first type of substrate includes the sediment and the crust; The second type of substrate is the gravel.

4. The method for quantitative observation of seabed rock outcrop area based on cascaded deep learning according to claim 3, characterized in that, The sediments represent a state in which the current field of view is covered by pure sediments with no exposed rocks; The crust characterizes a state in which there is a continuous crust covering the entire field of view and no exposed sediment. The gravel represents a mixed substrate state in which rocks and sediments coexist within the current field of view, and the mixed substrate state is not limited by the rock particle size, distribution density, or coverage ratio.

Images