Seafloor pipeline acoustic-optical combined investigation method, system, device and medium

Abstract

The application relates to the technical field of submarine pipeline investigation, and discloses a submarine pipeline acoustic-optical combined investigation method, system, device and medium. An AUV carrying an acoustic sensor is used to carry out coverage investigation on a task area, acoustic data are collected, and acoustic characteristics of an object to be investigated are extracted, a suspected pipeline candidate area is generated based on echo intensity in the acoustic characteristics and a preset acoustic detection threshold; an AUV carrying an optical imaging device is used to carry out fine imaging on the suspected pipeline candidate area, optical data are collected, and optical characteristics of the object to be investigated are extracted, clustering analysis is carried out based on the optical characteristics to obtain a clustering result containing a suspected target cluster, a noise cluster and an uncertain cluster; and a multi-modal information fusion method is used to jointly judge the object to be investigated according to the acoustic characteristics and the optical characteristics of the suspected target cluster. The wide-area coverage advantage of the acoustic and the high-resolution advantage of the optical are fused, and efficient, accurate and reliable investigation of the submarine pipeline is realized.

Description

Technical Field

[0001] This application relates to the field of submarine pipeline survey technology, specifically to a method, system, equipment, and medium for combined acoustic and optical survey of submarine pipelines. Background Technology

[0002] In current submarine pipeline survey technology, there are two main types of core detection methods: acoustic sensing detection and optical imaging detection.

[0003] Acoustic sensing technologies, such as side-scan sonar and multibeam sonar, have become the mainstream method for large-scale marine target surveys due to their advantages of being unaffected by underwater visibility, having a wide detection range, and high operational efficiency. However, acoustic sensing technologies have significant limitations: First, the resolution of acoustic images is relatively low, making it difficult to capture the fine structure and texture features of targets, resulting in insufficient target identification accuracy and a high risk of misjudgment; second, complex seabed topography (such as trenches and reefs), water noise, and interference from seabed sediments can severely affect the propagation and reflection of acoustic signals, further reducing the reliability of the detection results; third, it is difficult to effectively distinguish between different targets with similar acoustic reflection characteristics, making accurate target confirmation impossible.

[0004] Optical imaging detection technology, including high-definition camera imaging and laser imaging systems, can acquire high-resolution two-dimensional or three-dimensional images of targets, clearly presenting detailed features such as texture, edges, and geometric shapes, with extremely high recognition accuracy. Its core advantage lies in its ability to perform detailed observation of targets, providing direct evidence for target classification and structural integrity assessment. However, optical imaging detection technology also has insurmountable drawbacks: First, underwater optical signals suffer severe attenuation, greatly affected by factors such as water turbidity, water depth, and lighting conditions, resulting in limited effective detection distance and making it difficult to achieve full coverage surveys over large sea areas; second, in complex marine environments such as turbid water and suspended sediments, the clarity of optical images drops sharply, or even becomes completely ineffective; third, the operating efficiency of optical imaging equipment is low. Using optical equipment for large-scale scanning would consume a lot of time and energy, failing to meet the high-efficiency requirements of engineering applications.

[0005] In summary, existing single acoustic detection methods suffer from wide coverage but low resolution and insufficient identification accuracy, while single optical detection methods suffer from high resolution but small effective range and poor adaptability. Neither can simultaneously meet the dual requirements of "wide coverage" and "high-precision confirmation" for submarine pipeline surveys.

[0006] Therefore, developing a joint survey method that can organically integrate the advantages of acoustic detection and optical imaging to achieve "large-scale rapid survey" and "high-precision accurate confirmation" and overcome the limitations of existing technologies has become an urgent technical problem to be solved in the field of marine exploration. Summary of the Invention

[0007] To address the aforementioned issues, this application provides a method, system, equipment, and medium for joint acoustic-optical investigation of submarine pipelines. It aims to resolve the contradiction between the limited resolution of traditional acoustic detection (which offers a large coverage area) and the limited effective range of optical imaging (which offers high resolution). The core idea is to use acoustic detection as a precursor to acquire a large candidate area, followed by fine imaging through secondary optical scanning. Feature extraction and cluster analysis are then introduced to automate and intelligently process the optical images, ultimately fusing them with the acoustic detection results.

[0008] The embodiments of this application adopt the following technical solutions: Firstly, this application provides a method for combined acoustic and optical investigation of submarine pipelines, including: An autonomous underwater vehicle equipped with acoustic sensors is used to conduct a comprehensive survey of the mission area, collect acoustic data and extract acoustic features of the target to be investigated, and generate candidate areas for suspected pipelines based on the echo intensity in the acoustic features and a preset acoustic detection threshold. An autonomous underwater vehicle equipped with an optical imaging device was used to perform detailed imaging of the suspected pipeline candidate area, collect optical data and extract the optical features of the target to be investigated, and perform cluster analysis based on the optical features to obtain cluster results including suspected target clusters, noise clusters and uncertain clusters; Based on acoustic characteristics and optical characteristics of suspected target clusters, a multimodal information fusion method is used to jointly determine the target under investigation.

[0009] Secondly, this application also provides an acoustic-optical joint investigation system for submarine pipelines, comprising: The acoustic data acquisition unit is used to conduct a comprehensive survey of the mission area using an autonomous underwater vehicle equipped with acoustic sensors, collect acoustic data and extract acoustic features of the target to be investigated, and generate suspected pipeline candidate areas based on the echo intensity in the acoustic features and a preset acoustic detection threshold. The optical data acquisition unit is used to perform fine imaging of the suspected pipeline candidate area using an autonomous underwater vehicle equipped with optical imaging equipment, acquire optical data and extract the optical features of the target to be investigated, and perform cluster analysis based on the optical features to obtain cluster results including suspected target clusters, noise clusters and uncertain clusters. The joint judgment unit is used to make a joint judgment on the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method.

[0010] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the aforementioned acoustic-optical joint investigation method for submarine pipelines.

[0011] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aforementioned acoustic-optical joint investigation method for submarine pipelines.

[0012] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: This application combines the wide-area coverage advantage of acoustic detection with the high-resolution advantage of optical imaging. Through a hierarchical survey mode of "acoustic wide-area scanning to generate candidate areas + optical fine imaging to confirm targets", it not only ensures the integrity of coverage over a large area of ​​sea, but also achieves high-precision identification of submarine pipelines, effectively solving the technical contradiction of low resolution of traditional single acoustic methods and small coverage of single optical methods.

[0013] In complex marine environments such as turbid water, sediment cover, and noise interference, this application relies on acoustic detection to provide stable candidate area clues, and then supplements detailed information through optical fine imaging. Even if the quality of optical imaging is reduced due to environmental influences, it can still achieve stable target identification by combining acoustic features, overcoming the limitation of existing optical methods that are prone to failure under low visibility conditions.

[0014] A multimodal information fusion mechanism was introduced, which integrates acoustic and optical recognition results through weighted calculation, significantly reducing false alarms and missed alarms caused by a single sensor. Simultaneously, an uncertainty cluster feedback mechanism and parameter optimization strategy were implemented to conduct supplementary investigations or manual verification of areas with questionable recognition results, further improving the reliability and robustness of target confirmation results.

[0015] It supports a collaborative survey mode for multiple autonomous underwater vehicles (AUVs), achieving a balanced allocation of tasks through regional division. Different AUVs undertake acoustic detection and optical confirmation tasks respectively, and data is shared in real time, significantly improving the survey efficiency of large-scale and highly complex tasks.

[0016] This application is applicable to various submarine pipeline survey tasks, including but not limited to submarine cable inspection, pipeline inspection, shipwreck search, seabed archaeology, and underwater infrastructure inspection. It can meet the safety inspection needs in engineering applications and the detailed survey requirements in scientific research tasks, and has broad application value and good engineering feasibility. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic flowchart of an acoustic-optical combined survey method for submarine pipelines according to an embodiment of this application is shown; Figure 2 A flowchart illustrating a method for combined acoustic and optical investigation of submarine pipelines according to another embodiment of this application is shown; Figure 3 A schematic diagram of the structure of an acoustic-optical joint investigation system for submarine pipelines according to an embodiment of this application is shown; Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Figure 1 A schematic flowchart of an acoustic-optical combined investigation method for subsea pipelines according to an embodiment of this application is shown. (Refer to...) Figure 1 As shown, this embodiment includes steps S110 to S130: Step S110: Use an autonomous underwater vehicle equipped with acoustic sensors to conduct a comprehensive survey of the mission area, collect acoustic data and extract acoustic features of the target to be investigated, and generate a candidate area for suspected pipelines based on the echo intensity in the acoustic features and a preset acoustic detection threshold.

[0020] First, using an autonomous underwater vehicle (AUV) equipped with acoustic sensors, a large-scale coverage scan of the mission area is completed, the acoustic features of the target to be investigated are extracted, and candidate areas of suspected pipelines are generated, providing navigation basis for subsequent fine optical imaging.

[0021] It should be noted that the target to be investigated is a submarine pipeline of a pre-defined type. For example, the target to be investigated may include, but is not limited to: submarine cables, submarine pipelines, shipwrecks, etc.

[0022] Before conducting acoustic detection, environmental information and historical survey data for the mission area should be collected. Environmental information may include, but is not limited to, topography, water depth, seabed sediment types, and water turbidity. Historical survey data may include, but is not limited to, past detection records and target distribution patterns in the mission area.

[0023] A prior model is constructed based on environmental data and historical survey data. The prior model can reflect the environmental characteristics and prior knowledge of the task area, providing a basis for acoustic scanning path planning and acoustic detection threshold setting.

[0024] The optimal acoustic scanning path is determined based on the prior model. Possible acoustic scanning paths include, but are not limited to, lawnmower-style paths, spiral paths, or zigzag paths. The specific path selection is determined using mature methods from existing technologies based on the prior model, and will not be elaborated further here.

[0025] Lawnmower paths are more suitable for large task areas with regular shapes and relatively flat terrain. Lawnmower paths scan back and forth along a baseline, ensuring full coverage of the scanned area.

[0026] The spiral path is more suitable for small areas that require focused investigation. It starts from the center and gradually expands outwards in a spiral pattern.

[0027] A zigzag path is more suitable for areas with complex terrain and many obstacles. It allows for flexible avoidance of obstacles.

[0028] After determining the optimal acoustic scanning path, an autonomous underwater vehicle equipped with acoustic sensors is deployed to conduct a comprehensive survey of the mission area according to the scanning path. Acoustic sensors may include, but are not limited to, side-scan sonar, multibeam sonar, or synthetic aperture sonar.

[0029] Raw acoustic data of the task area is continuously collected using acoustic sensors. This raw acoustic data may include, but is not limited to, reflected sound waves.

[0030] The collected raw acoustic data are processed sequentially by target enhancement, linear feature extraction, probability heatmap construction, information entropy distribution, and attitude estimation to extract the acoustic features of the target under investigation.

[0031] Target enhancement can employ filtering algorithms to remove noise and other interference signals from the original acoustic data, thereby enhancing the signal contrast between the target area under investigation and other background signals.

[0032] Linear feature extraction can extract features with linear distribution characteristics from enhanced raw acoustic data, highlighting the acoustic signals of the target area under investigation.

[0033] Probabilistic heatmap construction can build an acoustic probability heatmap of the task area based on the linear feature extraction results. The value of each pixel in the heatmap can represent the probability that the pixel is suspected to contain the target to be investigated.

[0034] By analyzing the information entropy distribution of the enhanced original acoustic signal, the target area under investigation can be further distinguished from other backgrounds.

[0035] Attitude estimation processing can preliminarily determine the extension direction and location information of the target under investigation.

[0036] It should be understood that the target enhancement, linear feature extraction, probability heatmap construction, information entropy distribution, and pose estimation processing described above can all be achieved using mature processing methods in the existing technology, which will not be elaborated here.

[0037] Therefore, in some optional implementations, step S110, which involves using an autonomous underwater vehicle (AUV) equipped with acoustic sensors to conduct a comprehensive survey of the mission area, collecting acoustic data and extracting acoustic features of the target to be investigated, includes: collecting environmental information and historical survey data of the mission area; constructing a priori model based on the environmental information and historical survey data; wherein the environmental information includes: topography, water depth, seabed sediment type, and water turbidity; determining an acoustic scanning path based on the priori model; wherein the scanning path includes: a lawnmower path, a spiral path, or a zigzag path; scheduling the AUV equipped with acoustic sensors to conduct a comprehensive survey of the mission area using the scanning path; wherein the acoustic sensors include: side-scan sonar, multibeam sonar, or synthetic aperture sonar; collecting raw acoustic data of the mission area through the acoustic sensors; and sequentially performing target enhancement, linear feature extraction, probability heatmap construction, information entropy distribution generation, and attitude estimation processing on the raw acoustic data to extract the acoustic features of the target to be investigated.

[0038] Furthermore, echo intensity is extracted from acoustic features based on the above processing. For any spatial coordinate in the task region... Extract the corresponding echo intensity This reflects the likelihood that the spatial coordinates may contain a target to be investigated.

[0039] Based on the echo intensity and the preset acoustic detection threshold, a candidate region for a suspected pipeline is generated according to the following formula (1): , formula (1); in, This indicates a potential pipeline candidate area. Represents the spatial coordinates within the task area. Echo intensity representing spatial coordinates, This indicates the preset acoustic detection threshold.

[0040] The preset acoustic detection threshold can be adaptively adjusted based on environmental information, historical survey data, and prior models to ensure that the candidate area for suspected pipelines can both include the target to be investigated and effectively filter noise interference.

[0041] Through acoustic detection, suspected pipeline candidate areas are screened out in the mission area, and acoustic characteristics of the target under investigation are provided based on the acoustic detection.

[0042] Therefore, in some optional implementations, step S110, generating a suspected pipeline candidate region based on the echo intensity in the acoustic features and a preset acoustic detection threshold, includes: generating a suspected pipeline candidate region based on the following formula according to the echo intensity in the acoustic data and the preset acoustic detection threshold. ,in, This indicates a potential pipeline candidate area. Represents the spatial coordinates within the task area. Echo intensity representing spatial coordinates, This indicates the preset acoustic detection threshold.

[0043] Step S120: Use an autonomous underwater vehicle equipped with an optical imaging device to perform fine imaging of the suspected pipeline candidate area, collect optical data and extract the optical features of the target to be investigated, and perform cluster analysis based on the optical features to obtain cluster results including suspected target clusters, noise clusters and uncertain clusters.

[0044] Next, an autonomous underwater vehicle equipped with optical imaging equipment was used to perform close-range fine imaging of the candidate areas of suspected pipelines generated by acoustic detection, extract the optical features of the targets to be investigated, and distinguish suspected target clusters, noise clusters and uncertain clusters through cluster analysis, so as to provide refined data support for the final confirmation of the subsea pipelines.

[0045] Based on the coordinates of the suspected pipeline candidate area, an autonomous underwater vehicle equipped with optical imaging equipment was dispatched to navigate to the suspected pipeline candidate area.

[0046] During navigation, an adaptive path planning algorithm can be employed, combined with reinforcement learning strategies to dynamically optimize the navigation path. For example, when approaching a suspected pipeline candidate area, the autonomous underwater vehicle can adjust its speed and path based on real-time environmental data to ensure it reaches the candidate area at the optimal distance and angle for optical imaging. It should be understood that the reinforcement learning-based adaptive path planning algorithm can utilize mature methods from existing technologies, which will not be elaborated upon here.

[0047] Optical imaging equipment may include, but is not limited to, high-definition cameras or laser imaging systems. High-definition cameras are suitable for scenes with good visibility in water bodies and can acquire high-resolution two-dimensional images. Laser imaging systems are suitable for scenes with turbid water and low visibility, and acquire three-dimensional images through laser scanning.

[0048] After the autonomous underwater vehicle reaches the suspected pipeline candidate area, it uses optical imaging equipment to perform comprehensive and multi-angle fine imaging of the suspected pipeline candidate area in order to collect raw optical data.

[0049] The acquired raw optical data is preprocessed, edge detected, and target feature points are extracted sequentially to obtain pixel optical features for each pixel, including grayscale statistical features, texture energy features, and edge direction information.

[0050] Preprocessing can perform denoising, image enhancement, and geometric correction on the raw optical data to eliminate noise interference and distortion in the image.

[0051] Edge detection can use edge detection algorithms to extract target edge information from preprocessed raw optical data to highlight the boundary between the target under investigation and other backgrounds.

[0052] Target feature point extraction can be achieved by using feature point detection algorithms to extract key feature points of the target under investigation, so as to reflect information such as the local structure, shape, and texture of the target.

[0053] It should be understood that the above preprocessing, edge detection and target feature point extraction can all adopt mature processing methods in the existing technology, which will not be elaborated here.

[0054] Through the above processing, the pixel optical features of each pixel are obtained, which are expressed in the form of the following formula (2): , formula (2); in, Indicates pixel index, Indicates the optical characteristics of a pixel. Represents the statistical characteristics of grayscale. Represents texture energy characteristics, This indicates edge direction information.

[0055] Then, the pixel optical features of all pixels are integrated in the form of the following formula (3) to construct an optical feature matrix: , formula (3); in, Represents the optical characteristic matrix. Indicates the number of pixels.

[0056] Therefore, in some optional implementations, step S120, using an autonomous underwater vehicle equipped with an optical imaging device to perform fine imaging of the suspected pipeline candidate area, collect optical data, and extract the optical features of the target to be investigated, includes: according to the coordinate information of the suspected pipeline candidate area, scheduling the autonomous underwater vehicle equipped with the optical imaging device to navigate to the suspected pipeline candidate area according to an adaptive path planning algorithm based on reinforcement learning; wherein, the optical imaging device includes: a high-definition camera or a laser imaging system; collecting the original optical data of the suspected pipeline candidate area through the optical imaging device; wherein, the original optical data includes two-dimensional images or three-dimensional images; performing preprocessing, edge detection, and target feature point extraction on the original optical data in sequence to obtain pixel optical features for each pixel including grayscale statistical features, texture energy features, and edge direction information, expressed based on the following formula; ,in, Indicates pixel index, Indicates the optical characteristics of a pixel. Represents the statistical characteristics of grayscale. Represents texture energy characteristics, Represents edge direction information; constructs an optical feature matrix based on the pixel optical features of each pixel, expressed by the following formula; ,in, Represents the optical characteristic matrix. Indicates the number of pixels.

[0057] Clustering algorithms are used to perform cluster analysis on optical features. Cluster analysis can be performed using, but is not limited to, K-means clustering, density-based clustering (DBSCAN), or hierarchical clustering algorithms.

[0058] For example, when using the K-means clustering algorithm, the objective function is in the form shown in the following formula (4): , formula (4); in, Describe the objective function. Indicates the cluster index, This indicates the total number of predefined clusters. Indicates a cluster, Indicates the cluster center.

[0059] Preset cluster size threshold (i.e., the threshold for the number of pixels within a cluster) and distribution density threshold (i.e., the threshold for the spatial distribution density of pixels within a cluster) are used to classify the clustering results.

[0060] Based on the comparison results of the cluster size and cluster size threshold of each cluster in the cluster analysis results, and the comparison results of the distribution density and distribution density threshold of each cluster, all clusters are divided into suspected target clusters, noise clusters, and uncertain clusters.

[0061] Clusters whose cluster size is greater than the cluster size threshold and whose distribution density is greater than the distribution density threshold are classified as suspected target clusters.

[0062] Clusters whose cluster size is less than the cluster size threshold and whose distribution density is less than the distribution density threshold are classified as noise clusters.

[0063] Other clusters that do not meet the criteria of suspected target clusters and noise clusters are classified as uncertain clusters.

[0064] To improve the reliability of optical detection, a feedback mechanism is introduced for further processing of the uncertain clusters obtained from cluster analysis.

[0065] The proportion of uncertain clusters is calculated based on the following formula (5): , formula (5); in, This indicates the number of pixels contained in an uncertain cluster. This indicates the total number of pixels contained in the optical feature.

[0066] when If this indicates that there are many areas within the suspected pipeline candidate area that cannot be clearly identified, further acoustic and optical data are needed to improve the reliability of the judgment. In this case, a combined acoustic and optical re-investigation of the task area should be conducted, and acoustic and optical data should be collected again after adjusting the investigation parameters. Adjusting the investigation parameters may include, but is not limited to, the detection parameters of the acoustic sensors, the imaging parameters of the optical imaging equipment, and the scanning path. Alternatively, a manual review process can be triggered, where professional technicians make a comprehensive judgment based on the acoustic data and environmental information.

[0067] when If the uncertainty cluster has a relatively small impact, the process can proceed directly to the subsequent multimodal information fusion stage.

[0068] Preset feedback threshold It can be flexibly determined based on the actual situation and historical experience.

[0069] Therefore, in some optional implementations, step S120, performing cluster analysis based on optical features to obtain clustering results including suspected target clusters, noise clusters, and uncertain clusters, includes: performing cluster analysis on optical features using a clustering algorithm; wherein, the clustering algorithm includes: K-means clustering algorithm, density-based clustering algorithm, or hierarchical clustering algorithm; preset cluster size threshold and distribution density threshold, and according to the comparison results of the cluster size of each cluster with the cluster size threshold and the comparison results of the distribution density of each cluster with the distribution density threshold in the cluster analysis results, dividing all clusters into suspected target clusters, noise clusters, and uncertain clusters; calculating the proportion of uncertain clusters based on the following formula; ,in, This indicates the number of pixels contained in an uncertain cluster. Indicates the total number of pixels contained in the optical feature; if In this case, a combined audio-visual re-investigation of the task area will be conducted, or a manual review will be performed; among these, Indicates the preset feedback threshold; if Then, the step of jointly judging the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method is executed.

[0070] Step S130: Based on the acoustic features and the optical features of the suspected target cluster, a joint judgment is made on the target to be investigated using a multimodal information fusion method.

[0071] By using a multimodal information fusion method, the target under investigation is jointly identified based on the acoustic features obtained from acoustic detection and the optical features of suspected target clusters obtained from optical clustering analysis, and then the subsea pipeline is finally confirmed.

[0072] Before performing multimodal information fusion, the acoustic recognition probability and optical recognition probability are calculated, and the comprehensive confidence level is calculated based on the acoustic recognition probability and optical recognition probability to determine the credibility of the joint acoustic-optical survey.

[0073] The acoustic recognition probability is calculated based on the matching degree between the acoustic features of the target to be investigated and the corresponding type of acoustic feature template.

[0074] The acoustic feature template is obtained by statistical analysis and modeling of the known acoustic features of a large number of known types of submarine pipelines (such as cables, pipes, shipwrecks, etc.). Each type corresponds to an acoustic feature template, which contains the typical acoustic feature parameters of that type.

[0075] Since the type of the target to be investigated is already clearly defined, the extracted acoustic features of the target are matched with the corresponding type's acoustic feature templates to calculate the acoustic matching similarity. The calculation method for acoustic matching similarity can employ existing, mature methods (e.g., distance metrics such as Euclidean distance, or correlation analysis such as Pearson correlation coefficient, Spearman correlation coefficient, etc.). A higher acoustic matching similarity indicates a greater likelihood that the target to be investigated belongs to that type.

[0076] Based on the acoustic matching similarity, a probability transformation method is used to convert it into an acoustic recognition probability. This probability transformation method can employ mature existing techniques, such as establishing a mapping relationship between acoustic matching similarity and acoustic recognition probability based on a statistical model, thereby directly outputting the corresponding acoustic recognition probability based on the calculated acoustic matching similarity.

[0077] Similarly, the optical recognition probability is calculated based on the matching degree between the optical features of the target under investigation and the corresponding type of optical feature template.

[0078] The optical feature templates are obtained by statistical analysis and modeling of the optical features of a large number of known types of subsea pipelines. Each type corresponds to an optical feature template, which contains typical optical feature parameters of that type.

[0079] Since the type of the target under investigation is already clearly defined, the extracted optical features of the target are matched with the corresponding type's optical feature template to calculate the optical matching similarity. The calculation method for optical matching similarity can employ existing mature methods (such as using a feature point matching algorithm to match the optical features of each pixel with the optical feature template, and then calculating the overall optical matching similarity). A higher optical matching similarity indicates a greater likelihood that the target under investigation belongs to that type.

[0080] Based on the optical matching similarity, the probability conversion method, which is the same as or similar to the acoustic recognition probability calculation, is used to convert it into optical recognition probability.

[0081] Based on the acoustic recognition probability and the optical recognition probability, the overall confidence level of the target is calculated using the following formula (6): , formula (6); in, Indicates the overall confidence level of the target. and Represents the weight coefficient and , Indicates the probability of acoustic recognition. This indicates the probability of optical recognition.

[0082] Weighting coefficient and The determination of the probability of optical recognition can be based on the reliability and importance of acoustic detection and optical imaging in different detection scenarios. For example, in environments with high water turbidity and poor optical imaging, the reliability of optical recognition probability is low, and the probability can be appropriately increased. The value of decreases The value of is determined by the fact that in environments with complex terrain and high acoustic noise, the reliability of acoustic recognition probability is low, and the value can be appropriately increased. The value of decreases The value of .

[0083] like This indicates that the combined confidence level of the acoustic detection results and optical imaging results has reached the preset requirements, and the target under investigation is likely to be confirmed. Subsequent multimodal information fusion steps can then be performed to determine the specific information of the target under investigation.

[0084] like This indicates insufficient overall confidence in the target, a low probability of confirming the target, or that existing acoustic and optical detection data are insufficient to accurately identify the target. In this case, a joint acoustic and optical re-survey of the mission area is required, with adjustments to the survey parameters followed by re-collection of acoustic and optical data. Adjustments to the survey parameters may include, but are not limited to, the detection parameters of the acoustic sensors, the imaging parameters of the optical imaging equipment, and the scanning path.

[0085] Therefore, in some optional implementations, before step S130, which involves jointly judging the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method, the method further includes: calculating the acoustic recognition probability based on the matching degree between the acoustic features of the target under investigation and the corresponding type of acoustic feature template; calculating the optical recognition probability based on the matching degree between the optical features of the target under investigation and the corresponding type of optical feature template; and calculating the target comprehensive confidence level based on the acoustic recognition probability and the optical recognition probability using the following formula. ,in, Indicates the overall confidence level of the target. and Represents the weight coefficient and , Indicates the probability of acoustic recognition. Indicates the probability of optical recognition; if Then, the step of jointly judging the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method is executed; whereby, This indicates a preset reliability threshold; if If so, a combined audio-visual re-investigation of the mission area will be conducted.

[0086] When the target comprehensive confidence level At the same time, a multimodal information fusion method is used to combine acoustic features and optical features of suspected target clusters for joint judgment to generate detailed information of the target to be investigated.

[0087] Multimodal information fusion includes feature-level fusion and decision-level fusion.

[0088] Feature layer fusion: The acoustic geometric features of the target under investigation are extracted from the acoustic characteristics. These features may include, but are not limited to, the acoustic length, acoustic width, acoustic orientation, and acoustic continuity of the target. The optical geometric features of the target under investigation are extracted from the optical characteristics of the suspected target cluster. These features may include, but are not limited to, parameters such as the optical length, optical width, optical orientation, optical edge roughness, and optical texture distribution of the target.

[0089] Acoustic and optical geometric features are fused to construct a joint feature vector. During the fusion process, the two types of features are normalized to eliminate dimensional differences. Then, methods such as feature concatenation and weighted summation are used to construct the joint feature vector. This joint feature vector integrates geometric information from both acoustic and optical dimensions, providing a more comprehensive and accurate reflection of the actual morphological characteristics of the target object.

[0090] Integration of decision-making levels: Linear structure fitting is performed based on the constructed joint feature vector. Existing mature fitting algorithms can be used to fit the geometric morphological parameters in the joint feature vector to obtain a linear structure model of the target object.

[0091] Based on a linear structure model, detailed information about the target to be investigated is generated. This detailed information may include, but is not limited to, the type, location, orientation, and length of the target, thereby confirming the accuracy of the investigation results.

[0092] Therefore, in some optional implementations, step S130, which involves jointly judging the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method, includes: extracting acoustic geometric features of the target under investigation from acoustic features, extracting optical geometric features of the target under investigation from optical features of suspected target clusters, constructing a joint feature vector based on acoustic and optical geometric features, performing linear structure fitting based on the joint feature vector, and generating the type, location, orientation, and length of the target under investigation based on the linear structure fitting result.

[0093] When the joint acoustic-optical survey mission is large in scale, covers a wide area, or has higher requirements for survey efficiency, multiple autonomous underwater vehicles (AUVs) can be used for collaborative surveys. In this scenario, some AUVs are equipped with acoustic sensors, while others are equipped with optical imaging equipment, and the data from each AUV is shared.

[0094] Based on the Voronoi partitioning principle, the task area can be divided into multiple sub-task areas, and the suspected pipeline candidate area can be divided into multiple sub-suspected pipeline candidate areas. Voronoi partitioning can ensure that the boundaries of each sub-task area or each sub-suspected pipeline candidate area are clear, and that there is no overlap between the sub-task areas or each sub-suspected pipeline candidate area.

[0095] Multiple autonomous underwater vehicles (AUVs) equipped with acoustic sensors are responsible for conducting comprehensive surveys of each sub-task area, collecting acoustic data, and extracting acoustic features. Multiple AUVs equipped with optical imaging devices are responsible for performing detailed imaging of each suspected pipeline candidate area, collecting optical data, and extracting optical features.

[0096] For each main underwater vehicle, reasonable allocation can be made based on the performance and location of each main underwater vehicle, as well as the difficulty and workload of each sub-task area or each sub-suspected pipeline candidate area.

[0097] During the investigation, each autonomous underwater vehicle can transmit the data it collects in real time to the central control system. The central control system then aggregates, stores, and processes the data, and shares the relevant data with other autonomous underwater vehicles that need it.

[0098] For example, for uncertain clusters obtained in cluster analysis, the central control system transmits the data in real time to all relevant autonomous underwater vehicles through a sharing mechanism, and then schedules the relevant autonomous underwater vehicles to readjust the survey parameters for the uncertain cluster area and conduct a joint acoustic and optical survey.

[0099] Figure 2 A schematic flowchart of a combined acoustic and optical survey method for submarine pipelines according to another embodiment of this application is shown. (Refer to...) Figure 2 As shown, this method can be used for submarine cable inspection, specifically including the following steps S201 to S207: Step S201: Prior model construction and acoustic scanning path determination. Collect environmental information and historical survey data of the task area, construct a prior model based on the environmental information and historical survey data, and determine the acoustic scanning path based on the prior model. Proceed to step S202.

[0100] Step S202: Acoustic data acquisition and acoustic feature extraction. An autonomous underwater vehicle equipped with acoustic sensors is deployed to conduct a comprehensive survey of the mission area using a scanning path. Raw acoustic data of the mission area is acquired through the acoustic sensors. This raw acoustic data is then processed sequentially, including target enhancement, linear feature extraction, probability heatmap construction, information entropy distribution generation, and attitude estimation, to extract the acoustic features of the target under investigation. Proceed to step S203.

[0101] Step S203: Generation of suspected pipeline candidate regions. Based on the echo intensity in the acoustic data and a preset acoustic detection threshold, suspected pipeline candidate regions are generated. Proceed to step S204.

[0102] Step S204: Optical data acquisition and optical feature extraction. Based on the coordinate information of the suspected pipeline candidate area, an autonomous underwater vehicle equipped with an optical imaging device is navigating to the suspected pipeline candidate area using a reinforcement learning-based adaptive path planning algorithm. The optical imaging device acquires the raw optical data of the suspected pipeline candidate area. The raw optical data is then preprocessed, edge detected, and target feature points extracted sequentially to obtain pixel optical features for each pixel, including grayscale statistical features, texture energy features, and edge direction information. An optical feature matrix is ​​constructed based on the pixel optical features of each pixel. Proceed to step S205.

[0103] Step S205, Cluster Analysis. A clustering algorithm is used to perform cluster analysis on the optical features. Preset cluster size and distribution density thresholds are used. Based on the comparison results of the cluster size and cluster size threshold, and the comparison results of the distribution density and distribution density threshold, all clusters are divided into suspected target clusters, noise clusters, and uncertain clusters. The proportion of uncertain clusters is calculated. If the proportion of uncertain clusters is greater than the preset feedback threshold, proceed to step S201; if the proportion of uncertain clusters is not greater than the preset feedback threshold, proceed to step S206.

[0104] Step S206: Calculate the overall confidence score of the target. Calculate the acoustic recognition probability based on the matching degree between the acoustic features of the target and the corresponding type of acoustic feature template; calculate the optical recognition probability based on the matching degree between the optical features of the target and the corresponding type of optical feature template; and calculate the overall confidence score of the target based on the acoustic and optical recognition probabilities. If the overall confidence score is less than a preset confidence threshold, proceed to step S201; if the overall confidence score is not less than the preset confidence threshold, proceed to step S207.

[0105] Step S207: Target identification. The acoustic geometric features of the target are extracted from the acoustic features. The optical geometric features of the target are extracted from the optical features of the suspected target cluster. A joint feature vector is constructed based on the acoustic and optical geometric features. Linear structure fitting is performed based on the joint feature vector. The type, location, orientation, and length of the target are generated based on the linear structure fitting results.

[0106] In one implementation case, this method is used for pipeline inspection. Acoustic surveys provide the overall direction of pipeline extension, while optical surveys distinguish normal pipe sections from abnormal ones.

[0107] In one implementation, this method was used for shipwreck detection. Acoustic surveys were used to determine the general outline of the wreckage, while optical surveys clarified the skeletal structure.

[0108] Figure 3 An acoustic-optical combined survey system for subsea pipelines according to one embodiment of this application is shown. Figure 3As shown, the acoustic-optical joint survey system 300 for submarine pipelines includes: The acoustic data acquisition unit 310 is used to conduct a comprehensive survey of the mission area using an autonomous underwater vehicle equipped with acoustic sensors, collect acoustic data and extract acoustic features of the target to be investigated, and generate a candidate area for suspected pipelines based on the echo intensity in the acoustic features and a preset acoustic detection threshold. The optical data acquisition unit 320 is used to perform fine imaging of the suspected pipeline candidate area using an autonomous underwater vehicle equipped with optical imaging equipment, acquire optical data and extract the optical features of the target to be investigated, and perform cluster analysis based on the optical features to obtain cluster results including suspected target clusters, noise clusters and uncertain clusters. The joint judgment unit 330 is used to make a joint judgment on the target under investigation based on acoustic features and optical features of suspected target clusters through a multimodal information fusion method.

[0109] In some optional embodiments, in the above system, the acoustic data acquisition unit 310 is used to: collect environmental information and historical survey data of the task area; construct a prior model based on the environmental information and historical survey data; wherein the environmental information includes: topography, water depth, seabed sediment type, and water turbidity; determine the acoustic scanning path based on the prior model; wherein the scanning path includes: lawnmower path, spiral path, or zigzag path; schedule an autonomous underwater vehicle equipped with acoustic sensors to conduct a comprehensive survey of the task area using the scanning path; wherein the acoustic sensors include: side-scan sonar, multibeam sonar, or synthetic aperture sonar; collect raw acoustic data of the task area through the acoustic sensors; and sequentially perform target enhancement, linear feature extraction, probability heatmap construction, information entropy distribution generation, and attitude estimation processing on the raw acoustic data to extract the acoustic features of the target to be investigated.

[0110] In some alternative implementations, in the above system, the acoustic data acquisition unit 310 is further configured to: generate a suspected pipeline candidate region based on the following formula according to the echo intensity in the acoustic data and a preset acoustic detection threshold; ,in, This indicates a potential pipeline candidate area. Represents the spatial coordinates within the task area. Echo intensity representing spatial coordinates, This indicates the preset acoustic detection threshold.

[0111] In some optional embodiments, in the above system, the optical data acquisition unit 320 is used to: schedule an autonomous underwater vehicle equipped with an optical imaging device to navigate to the suspected pipeline candidate area according to the coordinate information of the suspected pipeline candidate area using an adaptive path planning algorithm based on reinforcement learning; wherein the optical imaging device includes: a high-definition camera or a laser imaging system; acquire raw optical data of the suspected pipeline candidate area through the optical imaging device; wherein the raw optical data includes two-dimensional images or three-dimensional images; perform preprocessing, edge detection and target feature point extraction on the raw optical data in sequence to obtain pixel optical features for each pixel including grayscale statistical features, texture energy features and edge direction information, expressed based on the following formula; ,in, Indicates pixel index, Indicates the optical characteristics of a pixel. Represents the statistical characteristics of grayscale. Represents texture energy characteristics, Represents edge direction information; constructs an optical feature matrix based on the pixel optical features of each pixel, expressed by the following formula; ,in, Represents the optical characteristic matrix. Indicates the number of pixels.

[0112] In some optional embodiments, in the above system, the optical data acquisition unit 320 is further configured to: perform cluster analysis on the optical features using a clustering algorithm; wherein the clustering algorithm includes: K-means clustering algorithm, density-based clustering algorithm, or hierarchical clustering algorithm; preset cluster size threshold and distribution density threshold, and divide all clusters into suspected target clusters, noise clusters, and uncertain clusters based on the comparison results of the cluster size of each cluster with the cluster size threshold and the comparison results of the distribution density of each cluster with the distribution density threshold in the cluster analysis results; and calculate the proportion of uncertain clusters based on the following formula. ,in, This indicates the number of pixels contained in an uncertain cluster. Indicates the total number of pixels contained in the optical feature; if In this case, a combined audio-visual re-investigation of the task area will be conducted, or a manual review will be performed; among these, Indicates the preset feedback threshold; if Then, the step of jointly judging the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method is executed.

[0113] In some optional implementations, in the above system, the joint judgment unit 330 is used to: calculate the acoustic recognition probability based on the matching degree between the acoustic features of the target under investigation and the corresponding type of acoustic feature template; calculate the optical recognition probability based on the matching degree between the optical features of the target under investigation and the corresponding type of optical feature template; and calculate the target comprehensive confidence level based on the acoustic recognition probability and the optical recognition probability using the following formula. ,in, Indicates the overall confidence level of the target. and Represents the weight coefficient and , Indicates the probability of acoustic recognition. Indicates the probability of optical recognition; if Then, the step of jointly judging the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method is executed; whereby, This indicates a preset reliability threshold; if If so, a combined audio-visual re-investigation of the mission area will be conducted.

[0114] In some optional embodiments, in the above system, the joint judgment unit 330 is further configured to: extract the acoustic geometric features of the target under investigation from the acoustic features, extract the optical geometric features of the target under investigation from the optical features of the suspected target cluster, construct a joint feature vector based on the acoustic geometric features and the optical geometric features; perform linear structure fitting based on the joint feature vector, and generate the type, position, orientation and length of the target under investigation based on the linear structure fitting result.

[0115] In some optional implementations, in the above system, the acoustic data acquisition unit 310 uses multiple autonomous underwater vehicles (AUVs) equipped with acoustic sensors to conduct a comprehensive survey of the multiple sub-task areas included in the mission area, and the optical data acquisition unit 320 uses multiple AUVs equipped with optical imaging devices to perform detailed imaging of the multiple sub-suspected pipeline candidate areas included in the suspected pipeline candidate area; wherein, each sub-task area and each sub-suspected pipeline candidate area is divided based on the Voronoi region division principle, and the data of each AUV is shared.

[0116] It should be noted that the aforementioned acoustic-optical joint investigation system 300 for submarine pipelines can implement the aforementioned acoustic-optical joint investigation methods for submarine pipelines, which will not be elaborated further.

[0117] Figure 4 This invention illustrates a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4As shown, the electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external devices via a network connection. When executed by the processor, the computer program implements the functions or steps of the acoustic-optical combined survey method for submarine pipelines.

[0118] In one embodiment, the electronic device provided in this application includes a memory and a processor. The memory stores a database and a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the acoustic-optical joint investigation method for submarine pipelines.

[0119] The above is as stated in this application. Figure 3 The method for the acoustic-optical joint investigation system for submarine pipelines disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0120] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the acoustic-optical joint investigation method for submarine pipelines.

[0121] It should be noted that the functions or steps that the above-mentioned electronic devices or computer-readable storage media can achieve can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0122] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0123] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0124] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for combined acoustic and optical investigation of submarine pipelines, characterized in that, include: An autonomous underwater vehicle equipped with acoustic sensors is used to conduct a comprehensive survey of the mission area, collect acoustic data and extract acoustic features of the target to be investigated, and generate candidate areas for suspected pipelines based on the echo intensity in the acoustic features and a preset acoustic detection threshold. An autonomous underwater vehicle equipped with an optical imaging device was used to perform detailed imaging of the suspected pipeline candidate area, collect optical data and extract the optical features of the target to be investigated, and perform cluster analysis based on the optical features to obtain cluster results including suspected target clusters, noise clusters and uncertain clusters; Based on acoustic characteristics and optical characteristics of suspected target clusters, a multimodal information fusion method is used to jointly determine the target under investigation.

2. The method for combined acoustic and optical investigation of submarine pipelines according to claim 1, characterized in that, The method of using an autonomous underwater vehicle equipped with acoustic sensors to conduct a comprehensive survey of the mission area, collect acoustic data, and extract the acoustic features of the targets under investigation includes: Collect environmental information and historical survey data of the mission area, and construct a priori model based on the environmental information and historical survey data; the environmental information includes: topography, water depth, seabed sediment type and water turbidity; The acoustic scanning path is determined based on the prior model; the scanning path includes: lawnmower path, spiral path or zigzag path; The autonomous underwater vehicle equipped with acoustic sensors is deployed to conduct a comprehensive survey of the mission area by scanning a path; the acoustic sensors include: side-scan sonar, multibeam sonar or synthetic aperture sonar. Raw acoustic data of the task area is collected by acoustic sensors. The raw acoustic data is then processed sequentially to perform target enhancement, linear feature extraction, probability heatmap construction, information entropy distribution generation, and attitude estimation to extract the acoustic features of the target to be investigated.

3. The acoustic-optical combined investigation method for submarine pipelines according to claim 1, characterized in that, The generation of suspected pipeline candidate regions based on echo intensity in acoustic features and a preset acoustic detection threshold includes: Based on the echo intensity in the acoustic data and the preset acoustic detection threshold, a candidate area for suspected pipelines is generated using the following formula; ， in, This indicates a potential pipeline candidate area. Represents the spatial coordinates within the task area. Echo intensity representing spatial coordinates, This indicates the preset acoustic detection threshold.

4. The method for combined acoustic and optical investigation of submarine pipelines according to claim 1, characterized in that, The process of using an autonomous underwater vehicle equipped with optical imaging equipment to perform detailed imaging of suspected pipeline candidate areas, collect optical data, and extract optical features of the target under investigation includes: Based on the coordinates of the suspected pipeline candidate area, an autonomous underwater vehicle equipped with optical imaging equipment is scheduled to navigate to the suspected pipeline candidate area according to an adaptive path planning algorithm based on reinforcement learning; the optical imaging equipment includes: a high-definition camera or a laser imaging system; Raw optical data of suspected pipeline candidate areas are acquired using optical imaging equipment; the raw optical data includes two-dimensional or three-dimensional images. The raw optical data is preprocessed, edge detection is performed, and target feature points are extracted in sequence to obtain the pixel optical features of each pixel, including gray-level statistical features, texture energy features, and edge direction information, which are expressed based on the following formula; ， in, Indicates pixel index, Indicates the optical characteristics of a pixel. Represents the statistical characteristics of grayscale. Represents texture energy characteristics, Indicates edge direction information; An optical feature matrix is ​​constructed based on the pixel optical features of each pixel, expressed by the following formula; ， in, Represents the optical characteristic matrix. Indicates the number of pixels.

5. The method for combined acoustic and optical investigation of submarine pipelines according to claim 1, characterized in that, The clustering analysis based on optical features yields clustering results including suspected target clusters, noise clusters, and uncertain clusters, including: Clustering algorithms are used to perform cluster analysis on optical features; among them, the clustering algorithms include: K-means clustering algorithm, density-based clustering algorithm, or hierarchical clustering algorithm; Preset cluster size threshold and distribution density threshold. Based on the comparison results of the cluster size of each cluster with the cluster size threshold and the comparison results of the distribution density of each cluster with the distribution density threshold in the cluster analysis results, all clusters are divided into suspected target clusters, noise clusters and uncertain clusters. The proportion of uncertain clusters is calculated based on the following formula; ， in, This indicates the number of pixels contained in an uncertain cluster. This indicates the total number of pixels contained in the optical feature; like In this case, a combined audio-visual re-investigation of the task area will be conducted, or a manual review will be performed; among these, Indicates the preset feedback threshold; like Then, the step of jointly judging the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method is executed.

6. The method for combined acoustic and optical investigation of submarine pipelines according to claim 1, characterized in that, Before the step of jointly judging the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method, the method further includes: The acoustic recognition probability is calculated based on the matching degree between the acoustic features of the target under investigation and the corresponding type of acoustic feature template. The optical recognition probability is calculated based on the matching degree between the optical features of the target under investigation and the corresponding type of optical feature template. Based on the acoustic recognition probability and the optical recognition probability, the overall confidence level of the target is calculated using the following formula; ， in, Indicates the overall confidence level. and Represents the weight coefficient and , Indicates the probability of acoustic recognition. Indicates the probability of optical recognition; like Then, the step of jointly judging the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method is executed; among which, This indicates a preset confidence threshold; like If so, a combined audio-visual re-investigation of the mission area will be conducted.

7. The method for combined acoustic and optical investigation of submarine pipelines according to claim 1, characterized in that, The method of jointly judging the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method includes: The acoustic geometric features of the target under investigation are extracted from the acoustic features, and the optical geometric features of the target under investigation are extracted from the optical features of the suspected target cluster. A joint feature vector is constructed based on the acoustic geometric features and the optical geometric features. Linear structure fitting is performed based on the joint feature vectors, and the type, location, orientation, and length of the target under investigation are generated based on the linear structure fitting results.

8. The method for combined acoustic and optical investigation of submarine pipelines according to claim 1, characterized in that, The method further includes: Multiple autonomous underwater vehicles (AUVs) equipped with acoustic sensors were used to conduct a comprehensive survey of the mission area, which included multiple sub-mission areas. Multiple AUVs equipped with optical imaging devices were used to perform detailed imaging of the suspected pipeline candidate area, which included multiple sub-suspected pipeline candidate areas. The sub-task areas and the sub-suspected pipeline candidate areas are divided based on the Voronoi area division principle, and each underwater vehicle shares data.

9. A combined acoustic and optical survey system for submarine pipelines, characterized in that, include: The acoustic data acquisition unit is used to conduct a comprehensive survey of the mission area using an autonomous underwater vehicle equipped with acoustic sensors, collect acoustic data and extract acoustic features of the target to be investigated, and generate suspected pipeline candidate areas based on the echo intensity in the acoustic features and a preset acoustic detection threshold. The optical data acquisition unit is used to perform fine imaging of the suspected pipeline candidate area using an autonomous underwater vehicle equipped with optical imaging equipment, acquire optical data and extract the optical features of the target to be investigated, and perform cluster analysis based on the optical features to obtain cluster results including suspected target clusters, noise clusters and uncertain clusters. The joint judgment unit is used to make a joint judgment on the target under investigation based on acoustic features and optical features of suspected target clusters using a multimodal information fusion method.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the acoustic-optical combined investigation method for submarine pipelines as described in any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the acoustic-optical combined investigation method for submarine pipelines as described in any one of claims 1 to 8.

Images