Intelligent auxiliary decision-making method for underwater operation based on multi-modal data fusion

Abstract

The invention discloses an underwater operation intelligent auxiliary decision-making method based on multi-modal data fusion, and relates to the technical field of underwater operation, and the method comprises the steps: ST1, preloading topological information based on a pipeline CAD model, and dynamically planning a sensor combination of underwater operation equipment and a corresponding execution path; sT2, underwater operation equipment carries sensors, pre-executes monitoring operation according to a planned sensor combination and a corresponding execution path, identifies underwater environment characteristics, adaptively adjusts the acquisition frequency of the sensors according to the underwater environment characteristics, and gathers to obtain detection data of the underwater operation equipment; sT3, generating a pipeline 3D deformation thermodynamic diagram and a corrosion thermodynamic diagram based on underwater operation equipment detection data; the accuracy of the data collected by the underwater operation equipment is ensured, so that the accuracy of subsequent damage determination of the underwater pipeline is ensured; collected data redundancy is avoided, and the value of the collected data is ensured; and the accuracy of pipeline deformation and corrosion analysis is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater operations, and in particular to an intelligent auxiliary decision-making method for underwater operations based on multimodal data fusion. Background Art

[0002] Underwater operations constantly face the dual challenges of both the natural environment and technical requirements. In environments characterized by low visibility, surging water pressure, and limited communications, traditional operations, particularly during underwater pipeline monitoring, face a dual squeeze of efficiency bottlenecks and safety risks. With breakthroughs in artificial intelligence and the Internet of Things (IoT) technologies, the development of intelligent underwater decision-making assistance methods is crucial.

[0003] There are still certain deficiencies in the existing technology for underwater pipeline detection, which are specifically reflected in the following aspects: traditional fixed-mode data collection is difficult to meet the needs of key areas of the pipeline, resulting in inaccurate collected data, which in turn affects the accuracy of subsequent underwater pipeline damage determination. In addition, it is difficult to make adaptive adjustments according to the characteristics of the underwater environment during actual collection. The different underwater environmental characteristics also affect the credibility of the data collected by the sensor. The existing technology lacks this level of adjustment, resulting in low value and redundancy of the collected data. At the same time, traditional analysis of pipeline deformation and corrosion risks mostly adopts a fixed weight method, and pays little attention to setting weights according to the confidence of the collected data, thereby reducing the accuracy of pipeline deformation and corrosion analysis. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent auxiliary decision-making method for underwater operations based on multimodal data fusion, which solves the problems existing in the background technology.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions: The present invention provides an intelligent auxiliary decision-making method for underwater operations based on multimodal data fusion,

[0006] ST1. Based on the pre-loaded topology information of the pipeline CAD model, the sensor combination and corresponding execution path of the underwater operation equipment are dynamically planned.

[0007] ST2: The underwater operation equipment is equipped with sensors and pre-executes monitoring operations according to the planned sensor combination and its corresponding execution path to identify underwater environmental characteristics. Based on the underwater environmental characteristics, the sensor acquisition frequency is adaptively adjusted to obtain the underwater operation equipment detection data.

[0008] ST3. Generate 3D pipeline deformation thermograms and corrosion thermograms based on underwater operation equipment detection data.

[0009] The beneficial effects of the present invention are: (1) the present invention sets sensors and their corresponding parameters according to the key areas of the pipeline to ensure the accuracy of the collected data, thereby ensuring the accuracy of the subsequent determination of damage to the underwater pipeline.

[0010] (2) The present invention makes adaptive adjustments according to the underwater environment characteristics during actual data collection to avoid data redundancy and ensure the value of the collected data.

[0011] (3) When analyzing pipeline deformation and corrosion risks, the present invention sets weights according to the confidence level of the collected data, thereby improving the accuracy of pipeline deformation and corrosion analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0014] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0015] Reference Figure 1 As shown, the present invention provides an intelligent auxiliary decision-making method for underwater operations based on multimodal data fusion, including: ST1, preloading topology information based on the pipeline CAD model, and dynamically planning the sensor combination of underwater operation equipment and its corresponding execution path.

[0016] In a specific embodiment of the present invention, the dynamic planning of the sensor combination of the underwater operation equipment and its corresponding execution path, the specific steps are: ST1001: Based on the preloaded topology information of the pipeline CAD model, the characteristic parameters of several key areas of the pipeline are obtained, the several key areas specifically reflect the differences in the construction process of the pipeline, and the characteristic parameters of the several key areas specifically reflect the different quality of the construction process of the pipeline.

[0017] ST1002: Determine sensors in several key areas based on several key areas of the pipeline, and set sensor parameters according to characteristic parameters of the several key areas.

[0018] ST1003: Generate a sensor combination for underwater operation equipment based on sensors in several key areas of the pipeline, and generate an execution path for the underwater operation equipment based on the locations of several key areas of the pipeline.

[0019] In a specific embodiment of the present invention, the sensor parameters are set according to the characteristic parameters of several key areas, and the specific setting method is: based on the characteristic parameters of several key areas of the pipeline and the parameter set of several sensors preset in the data warehouse under the characteristic parameter combination, the sensor parameters of several key areas of the pipeline are screened.

[0020] The parameter sets of the plurality of sensors under the characteristic parameter combination specifically reflect the appropriate acquisition parameter sets of the sensors under different working environments.

[0021] The characteristic parameters specifically reflect the characteristics of several key areas.

[0022] For example, for the weld area, the corresponding characteristic parameters are weld width and weld depth, and the corresponding sensor is a lidar scanning sensor. The horizontal density of the lidar scanning sensor is negatively correlated with the weld width, and the vertical density of the lidar scanning sensor is positively correlated with the weld depth. For the elbow area, the corresponding characteristic parameter is bending curvature. The corresponding sensors are a sonar sensor and a lidar scanning sensor. The sonar frequency of the sonar sensor is positively correlated with the bending curvature, the sonar scanning angle is positively correlated with the bending curvature, and the horizontal density and vertical density of the lidar scanning sensor are positively correlated with the bending curvature, respectively. Secondly, there are also bolt connection areas, coating areas, etc., which will not be elaborated here.

[0023] It should be noted that the parameter sets of several sensors pre-installed in the data warehouse under the characteristic parameter combination are specifically obtained through training with historical data. By collecting historical samples of weld morphology data (width, depth) and corresponding sensor parameters (horizontal / vertical scanning density) covering various working conditions, a quantitative mapping relationship between parameters can be established in combination with data mining technology. For example, regression analysis or machine learning models are used to capture the association between weld characteristics and optimal scanning density, thereby generating a recommended parameter set that adapts to complex weld changes. This data-driven approach can significantly improve the engineering rationality and environmental adaptability of parameter configuration.

[0024] The present invention sets sensors and their corresponding parameters according to key areas of the pipeline to ensure the accuracy of collected data, thereby ensuring the accuracy of subsequent damage determination of the underwater pipeline.

[0025] ST2: The underwater operation equipment is equipped with sensors and pre-executes monitoring operations according to the planned sensor combination and its corresponding execution path to identify underwater environmental characteristics. Based on the underwater environmental characteristics, the sensor acquisition frequency is adaptively adjusted to obtain the underwater operation equipment detection data.

[0026] In a specific embodiment of the present invention, the specific identification method for identifying underwater environmental characteristics is: based on the sensors carried by the underwater operating equipment, the underwater environmental parameters of the underwater operating equipment are obtained, and the underwater environmental parameters include a number of underwater environmental data.

[0027] Identify underwater environmental features based on the underwater environmental features and underwater environmental data mapping table preset in the data warehouse.

[0028] It should be noted that the underwater environment characteristics include turbidity, strong light, dim light, high salinity, high temperature, etc.

[0029] It should be noted that the underwater environmental characteristics and underwater environmental data mapping table is specifically constructed by defining measurement indicators for each underwater environmental characteristic, such as turbidity corresponding to suspended particulate matter concentration, strong light and dark light corresponding to light intensity, high salt corresponding to salinity, and high heat corresponding to water temperature, and is trained through historical data.

[0030] It is also worth adding that by combining historical observation data collected from multi-source sensors (such as optical, acoustic, and electrical sensors) with machine learning or data-driven modeling methods, it is possible to quantitatively analyze the nonlinear relationships between environmental parameters such as suspended solids concentration, water temperature, and salinity, and underwater environmental characteristics such as turbidity and glare, and to establish interpretable mapping models. This process relies on the statistical regularity of historical data to discover characteristic boundaries under typical environmental patterns (such as the correlation between the optical failure threshold and the critical value of suspended solids concentration). Through cross-validation and physical model constraints, the robustness of the mapping table is enhanced, ultimately achieving accurate identification of environmental states under complex hydrological conditions and data validity assessment, providing a reliable basis for underwater detection.

[0031] In a specific embodiment of the present invention, the sensor acquisition frequency is adaptively adjusted according to the underwater environmental characteristics, and its specific implementation steps are: according to the underwater environmental characteristics, combined with the currently called sensor of the underwater operating equipment and its corresponding parameters, through the underwater environmental characteristics-suppression sensor rule library preset in the data warehouse, it is determined whether the underwater environmental characteristics are inconsistent with the currently called sensor. If inconsistent, the sensor acquisition frequency is adaptively adjusted according to the underwater environmental data corresponding to the underwater environmental characteristics.

[0032] It should be noted that the underwater environment feature-suppression sensor rule base includes a number of underwater environment features corresponding to a number of suppression sensors, which are specifically set by staff based on experience, such as turbidity suppression optical sensors.

[0033] It should also be noted that the specific judgment method for judging whether the underwater environment characteristics are inconsistent with the currently called sensor is: based on the underwater environment characteristics, identify several suppression sensors corresponding to the underwater environment characteristics. If the currently called sensor is the same as a suppression sensor, it is judged that the underwater environment characteristics are inconsistent with the currently called sensor.

[0034] It should be noted again that the underwater environmental characteristics corresponding to the underwater environmental data are specifically determined through a mapping table of underwater environmental characteristics and underwater environmental data.

[0035] In this embodiment, the underwater environmental characteristics corresponding to the associated underwater environmental data are compared with the suitable underwater environmental data, and the excess value P of the underwater environmental characteristics corresponding to the associated underwater environmental data is calculated. Assuming that the interval of the excess value of the underwater environmental characteristics corresponding to the associated underwater environmental data is (P_min, P_max), , where P_min is the minimum value of the underwater environmental feature corresponding to the associated underwater environmental data exceeding the value interval, P_max represents the maximum value of the underwater environmental feature corresponding to the associated underwater environmental data exceeding the value interval, and the corresponding sensor acquisition frequency interval is (f_min, f_max), f_min represents the minimum value of the sensor acquisition frequency interval, and f_max represents the maximum value of the sensor acquisition frequency interval. Assuming that there is a linear relationship between the environmental data and the sensor acquisition frequency, it can be mapped by the following formula: f=f_max-(P-P_min) / P_max-P_min)×(f_max-f_min).

[0036] In this way, each value of the associated underwater environment data corresponds to a sensor acquisition frequency.

[0037] Assuming that there is a nonlinear relationship between underwater environmental data and sensor acquisition frequency, it can be mapped through a polynomial function. The specific formula is: , where the coefficients E and B must satisfy the boundary conditions f=f_min when P=PX_min and f=f_max when P=P_max.

[0038] The present invention performs adaptive adjustment according to underwater environmental characteristics during actual data collection, thereby avoiding redundancy in collected data and ensuring the value of the collected data.

[0039] ST3. Generate 3D pipeline deformation thermograms and corrosion thermograms based on underwater operation equipment detection data.

[0040] In a specific embodiment of the present invention, the pipeline 3D deformation thermogram and corrosion thermogram are generated based on the underwater operation equipment detection data. The specific method is: using the ICP algorithm to align the scanning point cloud in the underwater operation equipment detection data with the pipeline CAD model to obtain the detection pipeline CAD model.

[0041] Pipeline 3D deformation thermal map: The pipeline surface is divided into grids, and the underwater environmental feature set based on underwater equipment detection data is used to determine the average deformation within each grid. , and according to ,in Generate the deformation risk level in each grid for the preset deformation variable interval corresponding to the jth deformation risk level , combined with the color labels corresponding to several deformation risk levels preset in the data warehouse, determine the deformation color label of each grid, and perform real-time rendering according to the deformation color label. i is the number of the grid, , j is the number of deformation risk level, .

[0042] 3D Pipeline Corrosion Thermogram: Based on the optical, electromagnetic, and acoustic data from underwater equipment detection data, combined with the underwater environment feature set from the underwater equipment detection data, the corrosion risk index within each grid is determined. The corrosion risk level within each grid is generated in the same way as the deformation risk level within each grid. Combined with the color labels corresponding to several corrosion risk levels preset in the data warehouse, the corrosion color label for each grid is determined, and real-time rendering is performed based on the corrosion color label.

[0043] In a specific embodiment of the present invention, the method for determining the average deformation amount in each grid is as follows: based on the pipeline CAD model and the detected pipeline CAD model, the Euclidean distance difference, i.e., the deformation amount, is calculated point by point. ,in It represents the Euclidean distance difference between the mth point in the i-th grid in the pipeline CAD model and the mth point in the i-th grid in the detection pipeline CAD model. is the deformation correction parameter corresponding to the environmental characteristics when detecting the i-th grid of the pipeline CAD model, m is the number of the point in the grid, .

[0044] The deformation correction parameter corresponding to the environmental feature specifically reflects the impact of the environmental feature on the confidence of the scanned point cloud data during detection by underwater operating equipment. It can be specifically determined by the staff through experiments. For example, when the environmental feature is strong light, the corresponding deformation correction parameter is 0.5, and when the environmental feature is turbidity, the corresponding deformation correction parameter is 0.4.

[0045] In a specific embodiment of the present invention, the corrosion risk index in each grid is determined by the following method: based on the optical data, electromagnetic data and acoustic data in the detection data of the underwater operation equipment, the optical image, electromagnetic frequency, eddy current response signal amplitude and sonar echo intensity in each grid of the detection pipeline CAD model are respectively obtained.

[0046] The pipe wall thickness in each grid is calculated by combining the electromagnetic frequency and eddy current response signal amplitude in each grid with the metal thickness-signal attenuation relationship formula.

[0047] According to the optical image within each grid, the U-Net model is used to identify the rust area and coating peeling area of ​​the optical image, and the volume of the rust area and coating peeling volume within each grid is obtained.

[0048] The rust area volume, coating peeling volume, pipe wall thickness, and sonar echo intensity in each grid are homogenized and then imported into the corrosion risk index model. Output the corrosion risk index within each grid, where 、 、 、 are respectively represented by the volume of the rust spot area, the volume of the coating peeling, the pipe wall thickness, and the sonar echo intensity in the i-th grid after homogenization. 、 、 are the confidence factors of the optical data, electromagnetic data, and acoustic data in the i-th grid under environmental characteristics. The confidence factors under environmental characteristics specifically reflect the impact of environmental characteristics on the confidence of optical data, electromagnetic data, and acoustic data during detection by underwater operating equipment, and can be specifically determined by staff through experiments.

[0049] It should be noted that when homogenizing the volume of the rust area, the volume of coating peeling, the pipe wall thickness, and the sonar echo intensity within each grid, measures are taken to force the sonar echo intensity to return to zero. For example, if biological attachment is identified through the optical image within the grid, the sonar echo intensity is forced to return to zero.

[0050] In a specific embodiment of the present invention, the metal thickness-signal attenuation formula is specifically established as follows: when the metal thickness When , the eddy current is only distributed on the surface, and the signal attenuation is saturated, where is the penetration depth of eddy current in the conductor, ,in 、 and are angular frequency, material magnetic permeability, and material electrical conductivity, respectively.

[0051] when When the thickness decreases, the signal attenuation is exponential, which can be approximated by the following model: ,in is the detection signal voltage when the metal thickness is D, and V0 is the reference signal voltage when there is no thickness limit (obtained through calibration).

[0052] Establish the metal thickness-signal attenuation relationship formula: Because the underwater environment is more complex, temperature changes affect the conductivity of the material, thereby changing the penetration depth. Therefore, temperature compensation is introduced to correct the conductivity. ,in is the conductivity temperature coefficient, update the pipeline penetration depth The pipe wall thickness is then recalculated. The temperature is specifically collected by a temperature sensor carried by the underwater detection equipment.

[0053] The present invention sets weights according to the confidence of collected data when analyzing pipeline deformation and corrosion risks, thereby improving the accuracy of pipeline deformation and corrosion analysis.

[0054] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they should all fall within the scope of protection of the present invention.

Claims

1. An intelligent auxiliary decision-making method for underwater operations based on multimodal data fusion, characterized in that: include: ST1: Based on the pre-loaded topology information of the pipeline CAD model, dynamically plan the sensor combination and corresponding execution path of the underwater operation equipment; ST2: The underwater equipment is equipped with sensors and pre-executes monitoring operations according to the planned sensor combination and its corresponding execution path to identify underwater environmental characteristics. Based on these characteristics, the sensor acquisition frequency is adaptively adjusted to obtain detection data from the underwater equipment. ST3. Generate 3D pipeline deformation thermograms and corrosion thermograms based on underwater operation equipment detection data.

2. The underwater operation intelligent auxiliary decision-making method based on multimodal data fusion according to claim 1 is characterized in that: The specific steps of dynamically planning the sensor combination of underwater operation equipment and its corresponding execution path are as follows: ST1001: Based on preloaded topology information of the pipeline CAD model, characteristic parameters of several key areas of the pipeline are obtained. The several key areas specifically reflect differences in the construction process of the pipeline, and the characteristic parameters of the several key areas specifically reflect differences in the quality of the construction process of the pipeline; ST1002: Determine sensors in several key areas based on several key areas of the pipeline, and set sensor parameters according to characteristic parameters of the several key areas; ST1003: Generate a sensor combination for underwater operation equipment based on sensors in several key areas of the pipeline, and generate an execution path for the underwater operation equipment based on the locations of several key areas of the pipeline.

3. The underwater operation intelligent auxiliary decision-making method based on multimodal data fusion according to claim 2 is characterized in that: The sensor parameters are set according to the characteristic parameters of several key areas, and the specific setting method is as follows: According to the characteristic parameters of several key areas of the pipeline and the parameter sets of several sensors preset in the data warehouse under the characteristic parameter combination, the sensor parameters of several key areas of the pipeline are screened and obtained; The parameter sets of the plurality of sensors under the characteristic parameter combination specifically reflect the appropriate acquisition parameter sets of the sensors under different working environments; The characteristic parameters specifically reflect the characteristics of several key areas.

4. The underwater operation intelligent auxiliary decision-making method based on multimodal data fusion according to claim 1 is characterized in that: The specific identification method of the underwater environment characteristics is as follows: Acquiring underwater environmental parameters of the underwater operating equipment based on sensors carried by the underwater operating equipment, wherein the underwater environmental parameters include a plurality of underwater environmental data; Identify underwater environmental features based on the underwater environmental features and underwater environmental data mapping table preset in the data warehouse.

5. The underwater operation intelligent auxiliary decision-making method based on multimodal data fusion according to claim 1 is characterized in that: The specific implementation steps of adaptively adjusting the sensor acquisition frequency according to the underwater environment characteristics are as follows: According to the underwater environment characteristics, combined with the currently called sensors of the underwater operating equipment and their corresponding parameters, the underwater environment characteristics-suppression sensor rule library preset in the data warehouse is used to determine whether the underwater environment characteristics are inconsistent with the currently called sensors. If they are inconsistent, the sensor collection frequency is adaptively adjusted according to the corresponding associated underwater environment data based on the underwater environment characteristics.

6. The underwater operation intelligent auxiliary decision-making method based on multimodal data fusion according to claim 1 is characterized in that: The specific method for generating the pipeline 3D deformation thermogram and corrosion thermogram based on the detection data of the underwater operation equipment is as follows: Use the ICP algorithm to align the scan point cloud in the underwater operation equipment detection data with the pipeline CAD model to obtain the detection pipeline CAD model; Pipeline 3D deformation heat map: The pipeline surface is divided into grids, and the underwater environmental feature set of underwater equipment detection data is combined to determine the average deformation within each grid. The deformation risk level within each grid is generated. The deformation color label corresponding to several deformation risk levels preset in the data warehouse is combined to determine the deformation color label of each grid, and real-time rendering is performed based on the deformation color label. Pipeline 3D corrosion thermodynamic map: Based on the optical, electromagnetic, and acoustic data from underwater equipment detection data, combined with the underwater environment feature set from the underwater equipment detection data, the corrosion risk index within each grid is determined, and the corrosion risk level within each grid is generated. Combined with the color labels corresponding to several corrosion risk levels preset in the data warehouse, the corrosion color label for each grid is determined, and real-time rendering is performed based on the corrosion color label.

7. The underwater operation intelligent auxiliary decision-making method based on multimodal data fusion according to claim 6 is characterized in that: The specific method for determining the average deformation amount in each grid is as follows: Based on the pipeline CAD model and the detected pipeline CAD model, the Euclidean distance difference, i.e. the deformation variable, is calculated point by point ,in It represents the Euclidean distance difference between the mth point in the i-th grid in the pipeline CAD model and the mth point in the i-th grid in the detection pipeline CAD model. is the deformation correction parameter corresponding to the environmental characteristics when detecting the i-th grid of the pipeline CAD model, i is the number of the grid, , m is the number of the point in the grid, ; The deformation correction parameter corresponding to the environmental feature specifically reflects the impact of the environmental feature on the confidence of the scanned point cloud data during detection by underwater operating equipment.

8. The underwater operation intelligent auxiliary decision-making method based on multimodal data fusion according to claim 7 is characterized in that: The specific method for determining the corrosion risk index in each grid is as follows: Based on the optical data, electromagnetic data and acoustic data in the underwater operation equipment detection data, the optical image, electromagnetic frequency, eddy current response signal amplitude and sonar echo intensity in each grid of the detection pipeline CAD model are obtained respectively; The pipe wall thickness in each grid is calculated by combining the electromagnetic frequency and eddy current response signal amplitude in each grid with the metal thickness-signal attenuation relationship formula; Based on the optical image within each grid, the U-Net model is used to identify the rust area and coating peeling area in the optical image, and the volume of the rust area and coating peeling volume within each grid is obtained; The rust area volume, coating peeling volume, pipe wall thickness, and sonar echo intensity in each grid are homogenized and then imported into the corrosion risk index model. Output the corrosion risk index within each grid, where 、 、 、 are respectively represented by the volume of the rust spot area, the volume of the coating peeling, the pipe wall thickness, and the sonar echo intensity in the i-th grid after homogenization. 、 、 are the confidence factors of the optical data, electromagnetic data, and acoustic data in the i-th grid under the environmental characteristics. The confidence factors under the environmental characteristics specifically reflect the impact of environmental characteristics on the confidence of optical data, electromagnetic data, and acoustic data during detection by underwater operating equipment.

9. The underwater operation intelligent auxiliary decision-making method based on multimodal data fusion according to claim 8 is characterized in that: The metal thickness-signal attenuation formula is specifically established as follows: When the metal thickness When , the eddy current is only distributed on the surface, and the signal attenuation is saturated, where is the penetration depth of eddy current in the conductor, ,in 、 and are angular frequency, material magnetic permeability, and material electrical conductivity respectively; when When the thickness decreases, the signal attenuation is exponential, which can be approximated by the following model: ,in is the detection signal voltage when the metal thickness is D, and V0 is the reference signal voltage when there is no thickness limit; Establish the metal thickness-signal attenuation relationship formula: , introduce temperature compensation and correct conductivity ,in is the conductivity temperature coefficient, update the pipeline penetration depth The pipe wall thickness is then recalculated. The temperature is specifically collected by a temperature sensor carried by the underwater detection equipment.

Images