A submarine cable defect detection method and device based on multi-modal perception spatial correlation

Abstract

The application relates to a submarine cable defect detection method and device based on multi-modal perception space correlation, which comprises the following steps: synchronously acquiring submarine cable observation data by using an underwater detection platform, preprocessing the submarine cable observation data, and correlating the submarine cable observation data through a three-dimensional space transformation matrix; adopting a residual network as a backbone network, performing multi-scale fusion through a feature pyramid network structure, and acquiring fusion features; in the multi-scale fusion, a water quality prior parameter is introduced as a gate factor to adjust feature weights, and the water quality prior parameter comprises a light attenuation coefficient of a current water area and a sonar signal-to-noise ratio; the fusion features are input into a pre-trained target detection head, the submarine cable defect category and a bounding box are acquired, laser ranging information is combined, the actual physical size of the defect is calculated according to a pinhole camera model, and submarine cable defect three-dimensional detection results are output, so that submarine cable defect detection is completed. Compared with the prior art, the application realizes automatic identification of submarine cable defects in a complex underwater environment and is suitable for underwater submarine cable defect detection tasks.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection of underwater robots, and in particular to a method and device for detecting defects in submarine cables based on multimodal perception spatial correlation. Background Technology

[0002] Submarine power cables and communication cables are crucial infrastructure for cross-sea interconnection, power transmission, and information communication. Their operational safety directly impacts the reliability of power and communication systems. Therefore, regular condition monitoring and inspections are necessary in marine engineering and marine resource development. With the large-scale construction of offshore wind farms, large-scale marine engineering projects, and cross-sea power transmission projects, the length and number of submarine cables are rapidly increasing, significantly increasing the pressure on operation and maintenance throughout the cable's life cycle. This places higher demands on the automatic identification and precise location of cable defects.

[0003] In the field of underwater cable inspection and tracking, existing technologies have systematically studied and reviewed detection methods based on vision, sonar, magnetism, and multi-sensor fusion from the perspectives of overall morphology detection and trajectory tracking. These studies point out that different methods have limitations in terms of spatial resolution, detection distance, environmental adaptability, and engineering complexity, and overall, it remains difficult to balance detection accuracy and robustness in complex environments. Vision-based detection methods typically utilize underwater cameras to acquire cable images, and then use image feature extraction and tracking algorithms to achieve cable target identification and path following, assisting underwater work platforms in maintaining cable-line inspections. While the implementation process of these methods is relatively intuitive, they rely on single optical imaging and are sensitive to factors such as insufficient lighting, water turbidity, and scattering. Cable edges in the images are often blurred and have low contrast, leading to unstable detection results. Furthermore, these methods primarily focus on the overall cable trajectory, limiting their ability to automatically identify and assess the size of local defects.

[0004] To improve environmental adaptability, existing technologies have proposed underwater pipeline and cable inspection methods based on multi-sensor data fusion. These methods combine multi-source data such as multibeam echo sounding and side-scan sonar to construct a three-dimensional model in a unified coordinate system, analyzing the three-dimensional geometry, burial depth, and suspension status of pipelines or cables. While these methods excel in overall morphological modeling, they primarily focus on the macroscopic geometry of linear targets, lacking the ability to finely represent local damage features at the centimeter level, making it difficult to meet the needs for quantitative assessment and precise spatial location of specific defects. In low-visibility and highly turbid water environments, side-scan sonar is widely used for pipeline and cable inspection due to its strong imaging capabilities. Related methods automatically extract and track linear targets in sonar images to achieve automatic cable path following. However, limited by sonar resolution and imaging mechanisms, it remains difficult to accurately characterize subtle defects such as sheath damage and localized dents.

[0005] In recent years, deep learning methods have been introduced into the task of detecting pipelines and cables in side-scan sonar images. Existing techniques have automatically identified and detected pipeline targets in side-scan sonar images by constructing deep neural network frameworks, and have studied the impact of different pre-training datasets on detection performance, achieving certain improvements in detection accuracy and robustness. However, most of these works focus on the detection of "the presence of pipelines / cables" and the location of linear targets. The training data is mainly two-dimensional images and their annotations, and has not fully incorporated carrier attitude, multi-sensor ranging, and three-dimensional environmental information, making it difficult to detect specific defects (such as exposed, suspended, or locally damaged objects).

[0006] In summary, existing underwater cable inspection methods have made some progress in overall shape detection, path tracking, and deep learning-based two-dimensional target detection. However, they generally lack the ability to unify observations from multiple sensors into the same coordinate system, and thus cannot fully meet the engineering needs for fine detection and accurate positioning of submarine cable defects in complex underwater environments. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method and device for detecting submarine cable defects based on multimodal sensing spatial correlation.

[0008] The objective of this invention can be achieved through the following technical solutions: A method for detecting defects in submarine cables based on multimodal sensing spatial correlation, the method comprising: The underwater inspection platform was used to acquire submarine cable observation data synchronously, and the data was preprocessed and associated with each submarine cable observation data through a three-dimensional spatial transformation matrix. Based on the preprocessed submarine cable observation data, a residual network is used as the backbone network, and multi-scale fusion is performed through a feature pyramid network structure to obtain fused features. In the multi-scale fusion, water quality prior parameters are also introduced as gate factors to adjust feature weights. The water quality prior parameters include the light attenuation coefficient and sonar signal-to-noise ratio of the current water area. The fused features are input into a pre-trained target detection head to obtain the cable defect category and bounding box. Combined with laser ranging information, the actual physical size of the defect is calculated based on the pinhole camera model. Output three-dimensional detection results of submarine cable defects, including the defect type and actual physical size, to complete the submarine cable defect detection.

[0009] Furthermore, the submarine cable observation data is collected synchronously by an underwater platform equipped with an optical camera, sonar, and laser rangefinder. The optical camera collects high-frequency texture details of the submarine cable, the sonar collects geometric structural features of the submarine cable, and the laser rangefinder collects distance measurement information of the submarine cable. The distance measurement information of the submarine cable is a distance measurement point cloud or scan line data.

[0010] Furthermore, the process of correlating the aforementioned submarine cable observation data includes: Based on the preset external calibration matrix, a three-dimensional spatial transformation matrix is ​​established to spatially correlate the data of each submarine cable, and the submarine cable ranging information collected by the laser rangefinder is mapped to the optical camera coordinate system.

[0011] Furthermore, the process of obtaining the fusion features includes: A residual network is used as the backbone network, and a feature pyramid network structure is superimposed as the basic framework for feature extraction. By fusing semantic and detail features at different levels from the high-frequency texture details and geometric features of the submarine cable through convolution and upsampling operations, optical features reflecting the texture of defects in the submarine cable are obtained. and acoustic characteristics reflecting the geometry of submarine cables ; The ranging point cloud or scan line data collected by the laser rangefinder are characterized to extract three-dimensional geometric features that can reflect local depressions and deformation defects in the submarine cable. ; The gate factor is adjusted according to the current water turbidity, and a dynamic weight is generated based on the gate factor. By introducing a feature fusion formula for the dynamic weight, the optical features, acoustic features and three-dimensional geometric features are fused to obtain the fused features.

[0012] Furthermore, the process of adjusting the gate factor based on the current water turbidity and calculating the dynamic weights includes: Obtaining the light attenuation coefficient: The underwater standard grayscale target image in the current water area is acquired by an optical camera mounted on an underwater platform; grayscale values ​​are extracted from the target image, and the grayscale value attenuation of the same grayscale block at different propagation distances in the image is calculated; exponential fitting is performed on multiple sets of acquired data to solve for the α value in the Beer-Lambert law, which is used as the light attenuation coefficient of the current water area. Obtaining the sonar signal-to-noise ratio: Sound waves are emitted to the seabed and echoes are received using a sonar mounted on an underwater platform to obtain sonar echo data. This sonar echo data includes sonar images, voltage and power data of the original echo signals. Within the sonar echo data, the echo signal area corresponding to the submarine cable location and the pure seabed background area without submarine cables or seabed obstacles are selected. The average echo noise power at the submarine cable location and the average echo noise power in the pure seabed background area are extracted respectively. The sonar signal-to-noise ratio is calculated based on the extracted average echo noise power. Dynamic weights are generated by standardizing the optical attenuation coefficient and sonar signal-to-noise ratio based on a preset standardization formula, and combining optical features, acoustic features and three-dimensional geometric features through an attention mechanism.

[0013] Furthermore, the process of obtaining the optical attenuation coefficient also includes: Using a submarine cable of known size or grayscale as a reference, images of the submarine cable are captured by an optical camera mounted on an underwater platform. Combined with the camera-to-submarine cable distance obtained by a laser rangefinder, the optical attenuation coefficient is calculated in a simplified manner based on the Beer-Lambert law.

[0014] Furthermore, the expression for the feature fusion formula is: in, As a feature of fusion, For activation function, For dynamic weight tensors, This is an element-wise multiplication operation.

[0015] Furthermore, the target detection head is a single-stage detection head or an anchor-frame-free detection head designed for the characteristics of submarine cable defects.

[0016] A submarine cable defect detection device based on multimodal sensing spatial correlation, the device comprising: Data acquisition module: synchronously acquires submarine cable observation data using an underwater detection platform, preprocesses the data, and associates the observation data of each submarine cable through a three-dimensional spatial transformation matrix; Adaptive Feature Fusion Module: Based on the preprocessed submarine cable observation data, a residual network is used as the backbone network, and multi-scale fusion is performed through a feature pyramid network structure to obtain fused features; in the multi-scale fusion, water quality prior parameters are also introduced as gate factors to adjust feature weights, and the water quality prior parameters include the light attenuation coefficient and sonar signal-to-noise ratio of the current water area; Defect quantitative assessment module: Input the fused features into the pre-trained target detection head to obtain the cable defect category and bounding box, and calculate the actual physical size of the defect based on the pinhole camera model by combining laser ranging information; 3D Result Output Module: Outputs 3D detection results of submarine cable defects, including the defect category and actual physical dimensions, to complete the submarine cable defect detection.

[0017] A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the submarine cable defect detection method based on multimodal sensing spatial correlation as described above.

[0018] Compared with the prior art, the beneficial effects of the present invention include: 1. This invention achieves precise full-process detection of submarine cable defects by collecting submarine cable observation data, extracting and fusing features, obtaining the defect category and bounding box of the submarine cable through the fused features and the target detection head, and calculating the actual physical size of the defects. Compared with traditional manual inspection or semi-automatic inspection methods, it greatly improves the efficiency of submarine cable inspection, is suitable for routine inspection of large-scale submarine cable networks, and alleviates the pressure of full life cycle operation and maintenance brought about by the rapid increase in the number of submarine cables.

[0019] 2. This invention introduces prior water quality parameters (light attenuation coefficient, sonar signal-to-noise ratio) as gate factors during feature fusion. It constructs an environment-prior-driven adaptive gate fusion mechanism through an attention mechanism, dynamically adjusting the fusion weights of optical, acoustic, and laser modal features based on the underwater water environment. In scenarios where optical imaging fails, such as turbid water or insufficient lighting, the weight ratio of acoustic and laser features is automatically increased; in scenarios with clear water, optical texture features are prioritized. This effectively overcomes the shortcomings of traditional single or dual-modal detection methods, such as environmental sensitivity and unstable detection results, achieving continuity and reliability in submarine cable detection under complex underwater environments.

[0020] 3. This invention achieves automatic measurement of the physical size of defects through laser modal intervention, providing data support for the later maintenance of submarine cables and solving the problem of the lack of quantitative assessment of submarine cable defects in the prior art.

[0021] 4. This invention uses a residual network as the backbone network and combines it with a feature pyramid network (FPN) for multi-scale feature extraction. It accurately captures the texture details of minute defects on the centimeter scale, such as cracks and local depressions in the submarine cable sheath, through shallow high-resolution features. It uses deep strong semantic features to lock the overall trajectory of the submarine cable. This solves the problem that traditional detection methods cannot simultaneously take into account the global positioning of the submarine cable and the identification of local small defects, which greatly improves the detection accuracy of minute defects in submarine cables and meets the engineering requirements for refined defect detection in submarine cables.

[0022] 5. This invention, after obtaining the pixel-level bounding box of defects by fusion feature input target detection head, combines it with the high-precision physical distance of laser ranging. Based on a pinhole camera model, it accurately converts the defect pixel size to the actual physical size, and can quantitatively output key parameters such as the physical width and height of the defects. This solves the pain point of traditional detection methods, which can only achieve qualitative defect identification but cannot perform quantitative analysis. The quantitative data of defects provides accurate decision-making basis for the subsequent graded maintenance and operation and maintenance plan formulation of submarine cables, improving the scientific nature and pertinence of submarine cable operation and maintenance.

[0023] 6. This invention constructs a three-branch feature extraction structure involving a camera, sonar, and laser. The camera branch depicts the detailed texture of defects, the sonar branch captures the geometric structure and macroscopic spatial location of the submarine cable, and the laser branch provides depth data and 3D geometric features of the defects. An adaptive fusion module achieves deep coupling of these three types of features. This design retains sensitivity to small, localized defects in the submarine cable while also possessing the ability to identify the overall structure of the cable and the macroscopic location of defects. Compared to traditional methods, it more comprehensively mines the feature information of submarine cable defects, improving the accuracy of defect identification. Attached Figure Description

[0024] Figure 1 This is a flowchart of the method of the present invention; Figure 2 A schematic diagram of a submarine cable defect detection device based on multimodal sensing spatial correlation provided in an embodiment of the present invention; Figure 3 This is a hardware structure diagram of any device with data processing capabilities for the submarine cable defect detection method based on multimodal sensing spatial correlation provided in the embodiments of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] Example 1 A method for detecting defects in submarine cables based on multimodal sensing spatial correlation, the method is as follows: Figure 1 As shown, steps S1-S4 are included, and each step is described in detail below: Step S1: Use an underwater detection platform to synchronously acquire submarine cable observation data, preprocess the data, and associate the observation data of each submarine cable with a three-dimensional spatial transformation matrix.

[0027] The underwater inspection platform simultaneously acquires observation data of the submarine cable. The platform is equipped with an optical camera, sonar, and laser rangefinder. The optical camera primarily captures high-frequency texture details of the submarine cable, enabling it to detect minute cracks and color changes on the cable surface. The sonar equipment collects geometric features of the cable, obtaining its outline and deformation information in turbid water. The laser rangefinder collects distance measurement information from the cable, including distance point clouds or scan line data, used to accurately measure the distance from the cable to the inspection platform.

[0028] After acquiring the data, a three-dimensional spatial transformation matrix is ​​established based on a preset external calibration matrix to spatially correlate the data from various submarine cables. Specifically, the submarine cable ranging information collected by the laser rangefinder is mapped to the optical camera coordinate system, and pixel-level correlation of multimodal data in spatial scale is achieved through a rotation matrix R and a translation vector T. This spatial alignment ensures that data acquired by different sensors can be fused and processed in a unified coordinate system.

[0029] Based on the preset external calibration matrix, the laser point cloud is calculated using a formula. Mapped to camera coordinate system : in, For rotation matrix, It is a translation vector that enables pixel-level correlation of data from different modalities at spatial scales.

[0030] Step S2: Based on the preprocessed submarine cable observation data, a Residual Network (ResNet) is used as the backbone network, and multi-scale fusion is performed through a Feature Pyramid Network (FPN) structure to obtain fused features.

[0031] Specifically, the backbone network is a ResNet-101 pre-trained network model on ImageNet.

[0032] Through formula By integrating semantics and details at different levels, the ability to identify centimeter-level submarine cable defects is enhanced. and Its ability to capture.

[0033] In multi-scale fusion, prior water quality parameters are also introduced as gate factors to adjust feature weights. These prior water quality parameters include the light attenuation coefficient and sonar signal-to-noise ratio of the current water area.

[0034] Based on preprocessed submarine cable observation data, a residual network is used as the backbone network, and a feature pyramid network structure is superimposed as the basic framework for feature extraction. Through convolution and upsampling operations, semantic and detail features at different levels from the high-frequency texture details and geometric structural features of the submarine cable are fused to obtain optical features reflecting the texture of cable defects. and acoustic characteristics reflecting the geometry of submarine cables .

[0035] The ranging point cloud or scan line data acquired by the laser rangefinder are characterized to extract three-dimensional geometric features that can reflect local depressions and deformation defects in the submarine cable. This step fully utilizes the high precision of laser ranging data, enabling the detection of minute deformations and structural anomalies.

[0036] In the multi-scale fusion process, prior water quality parameters are introduced as gate factors to adjust feature weights. These prior water quality parameters include the light attenuation coefficient and sonar signal-to-noise ratio of the current water area, which directly affect the reliability of data from each sensor.

[0037] There are two methods to obtain the optical attenuation coefficient: Method 1: Acquire target images of underwater standard grayscale targets in the current water area using an optical camera mounted on an underwater platform; extract grayscale values ​​from the target images and calculate the grayscale attenuation of the same grayscale block at different propagation distances in the image; perform exponential fitting on multiple sets of acquired data to solve for the α value in the Beer-Lambert law, which is used as the light attenuation coefficient of the current water area.

[0038] Method 2: Using a submarine cable of known size or grayscale as a reference, images of the submarine cable are captured by an optical camera mounted on an underwater platform. The distance between the camera and the submarine cable is obtained by a laser rangefinder. The optical attenuation coefficient is then calculated using the Beer-Lambert law.

[0039] The process of obtaining the sonar signal-to-noise ratio is as follows: A sonar mounted on an underwater platform transmits sound waves to the seabed and receives the echoes, acquiring sonar echo data, which includes sonar images, voltage and power data of the original echo signals. Within the sonar echo data, the echo signal area corresponding to the submarine cable location and the pure seabed background area without submarine cables or seabed obstacles are selected. The average echo noise power at the submarine cable location and the average echo noise power in the pure seabed background area are extracted respectively. The sonar signal-to-noise ratio is calculated based on the extracted average echo noise power.

[0040] The gate factor is adjusted based on the current water turbidity. The light attenuation coefficient and sonar signal-to-noise ratio are standardized using a pre-defined formula. Dynamic weights are generated by combining optical, acoustic, and three-dimensional geometric features through an attention mechanism. A feature fusion formula based on these dynamic weights is then introduced to fuse optical, acoustic, and three-dimensional geometric features, resulting in a fused feature.

[0041] The expression for the feature fusion formula is: in, As a feature of fusion, For activation function, For dynamic weight tensors, This is an element-wise multiplication operation.

[0042] This fusion mechanism ensures that the system can adaptively adjust the weight of each sensor's data under different water quality conditions. When the quality of the optical image decreases due to turbidity, the system will automatically increase the weight of the acoustic and laser features to ensure the reliability of the detection results.

[0043] Step S3: Input the fused features into the pre-trained target detection head to obtain the cable defect category and bounding box. Combine the laser ranging information and calculate the actual physical size of the defect based on the pinhole camera model.

[0044] The fused features are input into a pre-trained target detection head to obtain the cable defect category and bounding box. The target detection head is a single-stage detection head or an anchor-frame-free detection head specifically designed for the characteristics of cable defects. These detection heads are specially trained to adapt to various morphological features of cable defects.

[0045] By combining laser ranging information with a pinhole camera model, the actual physical size of the defect is calculated. The specific formula is: the actual physical size of the defect equals the defect pixel width in the image multiplied by the laser ranging distance, then divided by the camera focal length. This calculation method converts the pixel size in the image into the actual physical size, providing a quantitative basis for subsequent defect assessment and repair.

[0046] Specifically, fusion features Input the target detection head to predict the category and 2D bounding box of submarine cable defects. (Refer to...) Figure 1 This solution uses laser ranging information. The detection head is calibrated in real time, and the actual width of the defect is calculated using physical dimension estimation formulas. : in, The width is in pixels. It is the focal length.

[0047] Step S4: Output the three-dimensional detection results of submarine cable defects, including the defect type and actual physical size, to complete the submarine cable defect detection.

[0048] The system outputs 3D inspection results of submarine cable defects, including the defect type and actual physical dimensions, thus completing the defect detection. The results include information such as defect type (e.g., cracks, corrosion, wear), location coordinates, bounding box, and actual physical dimensions, providing accurate data support for submarine cable maintenance and repair.

[0049] This method effectively overcomes the limitations of single sensors in complex underwater environments through multimodal data fusion and environmental adaptation mechanisms. The feature pyramid network structure solves the problem of cross-scale defect identification in submarine cables, demonstrating stronger detection capabilities for localized damage. The environmentally adaptive gate mechanism effectively copes with extreme turbidity conditions, improving the robustness of the detection system. The introduction of a laser ranging module enables precise measurement of the physical dimensions of defects, providing reliable data support for subsequent cable maintenance.

[0050] Example 2 This embodiment, based on Embodiment 1 above, discloses a submarine cable defect detection device based on multimodal sensing spatial correlation. The device is described in [reference needed]. Figure 2 It includes one or more processors for implementing the submarine cable defect detection method based on multimodal sensing spatial correlation in the above embodiments.

[0051] The device specifically includes: Data acquisition module: synchronously acquires submarine cable observation data using an underwater detection platform, preprocesses the data, and associates the observation data of each submarine cable through a three-dimensional spatial transformation matrix; Adaptive Feature Fusion Module: Based on preprocessed submarine cable observation data, a residual network is used as the backbone network, and multi-scale fusion is performed through a feature pyramid network structure to obtain fused features. In the multi-scale fusion, water quality prior parameters are also introduced as gate factors to adjust feature weights. The water quality prior parameters include the light attenuation coefficient and sonar signal-to-noise ratio of the current water area. Defect quantitative assessment module: Input the fused features into the pre-trained target detection head to obtain the defect category and bounding box of the submarine cable, and calculate the actual physical size of the defect based on the pinhole camera model by combining laser ranging information. 3D Result Output Module: Outputs 3D detection results of submarine cable defects, including the defect category and actual physical dimensions, to complete the submarine cable defect detection.

[0052] For details regarding the above modules, please refer to the relevant descriptions and effects in Example 1 for further understanding.

[0053] The embodiments of the multimodal sensing-based submarine cable defect detection device of the present invention can be applied to any device with data processing capabilities, such as a computer. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 3 The diagram shown is a hardware structure diagram of any device with data processing capabilities, including the multimodal sensing-based submarine cable defect detection device of this invention. (Except for...) Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0054] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0055] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0056] Corresponding to the aforementioned embodiments of the submarine cable defect detection method based on multimodal sensing spatial correlation, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the submarine cable defect detection method based on multimodal sensing spatial correlation as described above. Figure 3 The diagram shown is a hardware structure diagram of any device with data processing capabilities for the submarine cable defect detection method based on multimodal sensing spatial correlation provided in this application embodiment, except... Figure 3 In addition to the processor, memory, DMA controller, disk, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0057] Corresponding to the aforementioned embodiments of the submarine cable defect detection method based on multimodal sensing spatial correlation, this embodiment of the invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the submarine cable defect detection method based on multimodal sensing in the above embodiments.

[0058] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0059] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only.

[0060] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0061] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting defects in submarine cables based on multimodal sensing spatial correlation, characterized in that, The method includes: The underwater inspection platform was used to acquire submarine cable observation data synchronously, and the data was preprocessed and associated with each submarine cable observation data through a three-dimensional spatial transformation matrix. Based on the preprocessed submarine cable observation data, a residual network is used as the backbone network, and multi-scale fusion is performed through a feature pyramid network structure to obtain fused features. In the multi-scale fusion, water quality prior parameters are also introduced as gate factors to adjust feature weights. The water quality prior parameters include the light attenuation coefficient and sonar signal-to-noise ratio of the current water area. The fused features are input into a pre-trained target detection head to obtain the cable defect category and bounding box. Combined with laser ranging information, the actual physical size of the defect is calculated based on the pinhole camera model. Output three-dimensional detection results of submarine cable defects, including the defect type and actual physical size, to complete the submarine cable defect detection.

2. The submarine cable defect detection method based on multimodal sensing spatial correlation according to claim 1, characterized in that, The submarine cable observation data is collected synchronously by an underwater platform equipped with an optical camera, sonar, and laser rangefinder. The optical camera collects high-frequency texture details of the submarine cable, the sonar collects geometric structural features of the submarine cable, and the laser rangefinder collects distance measurement information of the submarine cable, which is a distance measurement point cloud or scan line data.

3. The method for detecting submarine cable defects based on multimodal sensing spatial correlation according to claim 2, characterized in that, The process of correlating the submarine cable observation data includes: Based on the preset external calibration matrix, a three-dimensional spatial transformation matrix is ​​established to spatially correlate the data of each submarine cable, and the submarine cable ranging information collected by the laser rangefinder is mapped to the optical camera coordinate system.

4. The submarine cable defect detection method based on multimodal sensing spatial correlation according to claim 2, characterized in that, The process of obtaining the fusion features includes: A residual network is used as the backbone network, and a feature pyramid network structure is superimposed as the basic framework for feature extraction. By fusing semantic and detail features at different levels from the high-frequency texture details and geometric features of the submarine cable through convolution and upsampling operations, optical features reflecting the texture of defects in the submarine cable are obtained. and acoustic characteristics reflecting the geometry of submarine cables ; The ranging point cloud or scan line data collected by the laser rangefinder are characterized to extract three-dimensional geometric features that can reflect local depressions and deformation defects in the submarine cable. ; The gate factor is adjusted according to the current water turbidity, and a dynamic weight is generated based on the gate factor. By introducing a feature fusion formula for the dynamic weight, the optical features, acoustic features and three-dimensional geometric features are fused to obtain the fused features.

5. The submarine cable defect detection method based on multimodal sensing spatial correlation according to claim 4, characterized in that, The process of adjusting the gate factor based on the current water turbidity and calculating the dynamic weights includes: Obtaining the light attenuation coefficient: The underwater standard grayscale target image in the current water area is acquired by an optical camera mounted on an underwater platform; grayscale values ​​are extracted from the target image, and the grayscale value attenuation of the same grayscale block at different propagation distances in the image is calculated; exponential fitting is performed on multiple sets of acquired data to solve for the α value in the Beer-Lambert law, which is used as the light attenuation coefficient of the current water area. Obtaining the sonar signal-to-noise ratio: Sound waves are emitted to the seabed and echoes are received using a sonar mounted on an underwater platform to obtain sonar echo data. This sonar echo data includes sonar images, voltage and power data of the original echo signals. Within the sonar echo data, the echo signal area corresponding to the submarine cable location and the pure seabed background area without submarine cables or seabed obstacles are selected. The average echo noise power at the submarine cable location and the average echo noise power in the pure seabed background area are extracted respectively. The sonar signal-to-noise ratio is calculated based on the extracted average echo noise power. Dynamic weights are generated by standardizing the optical attenuation coefficient and sonar signal-to-noise ratio based on a preset standardization formula, and combining optical features, acoustic features and three-dimensional geometric features through an attention mechanism.

6. The submarine cable defect detection method based on multimodal sensing spatial correlation according to claim 5, characterized in that, The process of obtaining the optical attenuation coefficient also includes: Using a submarine cable of known size or grayscale as a reference, images of the submarine cable are captured by an optical camera mounted on an underwater platform. Combined with the camera-to-submarine cable distance obtained by a laser rangefinder, the optical attenuation coefficient is calculated in a simplified manner based on the Beer-Lambert law.

7. The submarine cable defect detection method based on multimodal sensing spatial correlation according to claim 4, characterized in that, The expression for the feature fusion formula is: in, As a feature of fusion, For activation function, For dynamic weight tensors, This is an element-wise multiplication operation.

8. The submarine cable defect detection method based on multimodal sensing spatial correlation according to claim 1, characterized in that, The target detection head is a single-stage detection head or an anchor-frame-free detection head designed for the characteristics of submarine cable defects.

9. A submarine cable defect detection device based on multimodal sensing spatial correlation, characterized in that, The device includes: Data acquisition module: synchronously acquires submarine cable observation data using an underwater detection platform, preprocesses the data, and associates the observation data of each submarine cable through a three-dimensional spatial transformation matrix; Adaptive Feature Fusion Module: Based on the preprocessed submarine cable observation data, a residual network is used as the backbone network, and multi-scale fusion is performed through a feature pyramid network structure to obtain fused features; in the multi-scale fusion, water quality prior parameters are also introduced as gate factors to adjust feature weights, and the water quality prior parameters include the light attenuation coefficient and sonar signal-to-noise ratio of the current water area; Defect quantitative assessment module: Input the fused features into the pre-trained target detection head to obtain the cable defect category and bounding box, and calculate the actual physical size of the defect based on the pinhole camera model by combining laser ranging information; 3D Result Output Module: Outputs 3D detection results of submarine cable defects, including the defect category and actual physical dimensions, to complete the submarine cable defect detection.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the submarine cable defect detection method based on multimodal sensing spatial correlation as described in any one of claims 1-8.

Images