Deepwater rov intelligent electromagnetic detection robot visual servo scanning trajectory control system

Abstract

The application relates to the technical field of underwater robot control, and particularly discloses a deep-water ROV intelligent electromagnetic detection robot visual servo scanning track control system, which synchronously collects visual, acoustic and inertial motion data of an underwater robot and prior geometric data of a structure to be detected; further extracts visual sparse light spot features, acoustic profiles and abnormal area features, and performs credibility evaluation; through establishment of a unified state vector, fusion of multi-source data and geometric constraints, real-time estimation of accurate pose, motion state and perception uncertainty measurement of the robot relative to the underwater structure; according to the uncertainty measurement, adaptive generation of a local desired track for precise tracking or pose stabilization; through a control law capable of automatically adjusting gain according to the uncertainty, calculation of a motion control instruction, and realization of closed-loop control; and the application realizes continuous, adaptive and high-precision execution of detection operation under complex working conditions.

Description

Technical Field

[0001] This invention relates to the field of underwater robot control technology, specifically to a vision servo scanning trajectory control system for a deep-sea ROV intelligent electromagnetic inspection robot. Background Technology

[0002] The development of offshore oil and gas resources is an important component of the national energy strategy. With the advancement of the "Maritime Power" strategy, the scale and complexity of offshore oil and gas equipment, such as deep-water jackets and subsea pipelines, are increasing daily. These devices operate for extended periods in harsh marine environments characterized by high pressure, corrosion, ocean currents, and biofouling; their structural health directly impacts production safety and environmental protection. Therefore, regular high-precision non-destructive testing of underwater structures to promptly detect defects such as cracks and corrosion is crucial for ensuring their safe operation. Currently, underwater structure inspection primarily relies on remotely operated vehicles (ROVs) equipped with detection sensors (such as electromagnetic probes). To achieve automated inspection, the industry commonly employs a vision-servo-based trajectory control method, which uses cameras to acquire images of the structure and guides the ROV along a preset path. Existing typical technical solutions usually follow the following process: First, before the inspection task begins, a three-dimensional model of the underwater structure is obtained through laser scanning or design drawings, and a fixed inspection and scanning trajectory is planned offline based on this. During operation, the ROV attempts to track the preset trajectory. Its vision system is mainly used to identify predefined feature points (such as weld edges), calculates the pose deviation through classic image servo algorithms, and drives the thrusters to make position corrections.

[0003] Current deep-sea ROV inspection operations rely on pre-planned offline fixed trajectories and single / master-slave sensing feedback, which fails to address the core technical challenge of robust, continuous, and high-precision adaptive trajectory tracking in dynamic, unknown underwater environments characterized by strong disturbances, low visibility, and unstructured features. Specifically, when encountering internal wave impacts, suspended objects obscuring vision, or structural surfaces that significantly deviate from prior models, traditional methods, lacking real-time quantification of sensing quality and deep fusion of multi-source information, cause their fixed trajectories to fail. This results in the robot either losing the target and interrupting the mission, or colliding while attempting to track, making it impossible to reliably complete the inspection task while ensuring equipment safety. Summary of the Invention

[0004] The purpose of this invention is to provide a visual servo scanning trajectory control system for a deep-sea ROV intelligent electromagnetic inspection robot, in order to solve the problems mentioned above.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] The deep-sea ROV intelligent electromagnetic inspection robot visual servo scanning trajectory control system includes:

[0007] The multi-source synchronous acquisition module is used to simultaneously acquire visual image data, acoustic image data, inertial motion data, and prior geometric data of the underwater robot to be inspected.

[0008] The multimodal feature extraction module extracts sparse light spots with regular geometric shapes and stable optical properties as visual feature points based on visual image data, and assigns a confidence value to each visual feature point; based on acoustic image data, it extracts macroscopic structural contour lines and local areas with abnormal reflection intensity as acoustic features.

[0009] The fusion state estimation module inputs visual feature points, confidence assignments, acoustic features, inertial motion data, and prior geometric data into the extended state estimation framework for fusion calculation. By establishing a unified state vector that includes the underwater robot's spatial pose, motion state, and relative geometric relationship with the underwater structure, and by using the geometric and physical constraints between different data sources for iterative optimization, the module outputs in real time the underwater robot's accurate relative pose, relative motion state, and the uncertainty measure of the current estimate relative to the underwater structure.

[0010] The adaptive trajectory generation module takes the output relative pose, relative motion state and uncertainty measure as input, and combines them with preset scanning task rules to calculate and generate a local desired trajectory with spatiotemporal continuity in real time.

[0011] The variable gain closed-loop control module takes the generated local desired trajectory, relative pose, and relative motion state as input, calculates the response characteristics by automatically adjusting the control law based on the uncertainty metric, and outputs the final motion control command to the underwater robot's propulsion actuator, thus forming a closed-loop control.

[0012] As a further aspect of the present invention: the assignment of a confidence value to each visual feature point specifically includes:

[0013] The visual image data is binarized to obtain candidate light spot regions;

[0014] Calculate the contour compactness and area roundness of each candidate spot region, and determine the regions that simultaneously meet the compactness threshold and roundness threshold as candidate visual feature points with regular geometric shapes.

[0015] Calculate the average gray value and gray standard deviation of each candidate visual feature point in multiple consecutive frames of visual images. Points with an average gray value higher than the brightness threshold and a gray standard deviation lower than the stability threshold are determined as the final visual feature points.

[0016] Based on the contour tightness, area roundness, and grayscale standard deviation, a comprehensive credibility value is calculated and assigned to each final visual feature point through normalization and linear combination.

[0017] As a further aspect of the present invention: the extraction of macroscopic structural contour lines and local areas of abnormal reflection intensity as acoustic features specifically includes:

[0018] Apply Gaussian smoothing filter to the acoustic image data and calculate the gradient magnitude image of the acoustic image data;

[0019] In the gradient magnitude image, extract the arc segments that meet the preset radius range, and combine the interconnected arc segments with continuous curvature to form the macroscopic structural outline.

[0020] Within a preset neighborhood of the extracted macroscopic structural contour, connected regions with echo intensity higher than adjacent background regions are marked as regions with abnormal local reflection intensity.

[0021] The macroscopic structural outline, along with the geometric center coordinates and circumscribed rectangle of the local abnormal reflection intensity region, are collectively encapsulated into acoustic features.

[0022] As a further aspect of the present invention: the fusion calculation specifically includes:

[0023] Based on inertial motion data, the spatial pose and motion state of the underwater robot are predicted to obtain the predicted pose and predicted motion state.

[0024] Based on the predicted pose, visual feature points are spatially matched with the nominal features at corresponding positions in the prior geometric data. Different fusion weights are assigned to the successfully matched visual feature points according to the confidence level. The weighted matching results are used to perform the initial correction on the predicted pose and predicted motion state. When the number of available visual feature points is lower than the number threshold, the macroscopic structural contour lines in the acoustic features are fitted and aligned with the contours of the corresponding structures in the prior geometric data. The coverage overlap between the local abnormal reflection intensity region and the possible defect region in the prior geometric data is calculated. Based on the alignment and overlap calculation results, the pose and motion state after the initial correction are further corrected.

[0025] The statistical variance of the observation residuals during the initial and supplementary correction processes is calculated in real time. The statistical variance is compared with a preset sensor noise benchmark, and an uncertainty measure characterizing the reliability of the current estimate is output.

[0026] As a further aspect of the present invention: the supplementary correction of the pose and motion state after the initial correction based on the alignment and overlap calculation results specifically includes:

[0027] Calculate the geometric center of the local reflection intensity anomaly region, and calculate the major axis direction angle of the rectangle circumscribed by the geometric center;

[0028] The geometric center and orientation angle are compared with the preset center and preset orientation of the corresponding defect possibility area in the prior geometric data to obtain the center position deviation and orientation angle deviation respectively.

[0029] Based on the overlap ratio, the center position deviation and orientation angle deviation are scaled proportionally to generate the pose adjustment amount;

[0030] The pose adjustment is applied to the pose and motion state after the first correction to complete the supplementary correction, and the iterative residual of this supplementary correction is recorded to update the uncertainty metric.

[0031] As a further aspect of the present invention: the generation of a locally desired trajectory with spatiotemporal continuity specifically includes:

[0032] Determine whether the uncertainty measure exceeds a predetermined first threshold;

[0033] When the uncertainty measure does not exceed the first threshold, the first type of predetermined motion unit is selected for construction. The first type of predetermined motion unit is generated with the trajectory endpoint of the previous moment as the starting point and guided by the target detection point sequence determined by the scanning task rules.

[0034] When the uncertainty measure exceeds the first threshold, a second type of predetermined motion unit is selected for construction. The second type of predetermined motion unit is generated with the trajectory endpoint of the previous moment as the starting point and with the goal of maintaining the current relative pose stability.

[0035] The selected predetermined motion unit is spatiotemporally stitched with the current historical trajectory, and the stitching points are smoothed to form and output the local desired trajectory.

[0036] As a further aspect of the present invention: the output process of the motion control command is as follows:

[0037] Determine whether the uncertainty measure exceeds a predetermined second threshold;

[0038] When the uncertainty measure does not exceed the second threshold, the first type of control law is selected for calculation. The first type of control law uses the difference between the local desired trajectory and the relative pose as the trajectory tracking error, and performs proportional-integral-differential calculation on the trajectory tracking error using the first set of preset control gains.

[0039] When the uncertainty measure exceeds the second threshold, the second type of control law is selected for calculation. The second type of control law takes maintaining relative pose stability as its main objective, replaces the trajectory tracking error with the difference between the relative motion state and the zero vector, and uses the second set of preset control gains for calculation.

[0040] Based on the selected control law type and corresponding control gain, motion control commands are calculated and output in real time.

[0041] As a further aspect of the present invention: the real-time calculation to obtain and output motion control commands specifically includes:

[0042] When the uncertainty measure remains stable or decreases over three consecutive control cycles, the control gain corresponding to the selected control law is multiplied by an enhancement factor greater than 1 but less than 1.5.

[0043] When the uncertainty metric increases over two consecutive control cycles, a second-order low-pass filter is applied to the currently calculated motion control command. The cutoff frequency of the low-pass filter is inversely proportional to the magnitude of the increase in the uncertainty metric.

[0044] The calculation results after enhancement coefficient adjustment or low-pass filtering are compared with the maximum thrust constraint of the underwater robot's propulsion actuator.

[0045] The command components that exceed the maximum thrust constraint are proportionally scaled and limited to generate and output the actual executable motion control commands.

[0046] The beneficial effects of this invention are:

[0047] (1) By introducing an "uncertainty metric" as the core state feedback, the system can evaluate the reliability of the current perception information in real time and quantitatively. This drives the adaptive switching of trajectory generation and control strategies: high-precision active tracking and scanning are performed when perception is clear, and when visual occlusion or strong water flow disturbances cause perception deterioration, it automatically switches to a conservative dwell mode with pose stability as the core. This "intelligent degradation" mechanism effectively avoids trajectory loss, collision or task interruption caused by fixed trajectory planning in existing technologies when the environment changes suddenly, ensuring the safety of the robot body and the continued execution of the detection task under harsh working conditions.

[0048] (2) This invention does not simply list multiple sensors in parallel, but rather constructs a unified state estimation framework. It deeply integrates the reliability of visual features, the geometric properties of acoustic features, the kinematic model of inertial data, and prior structural knowledge through a weighted optimization method. This fusion mechanism enables the system to automatically enhance the dependence weight on other sensors (such as sonar and IMU) when a single sensor fails (e.g., visual failure due to turbid water), thereby maintaining centimeter-level accuracy in relative pose estimation even under conditions of partial sensor loss. This solves the problem of the sharp performance decline of traditional methods after key sensors are interfered with, providing a stable and reliable input benchmark for subsequent trajectory tracking. Attached Figure Description

[0049] The invention will now be further described with reference to the accompanying drawings.

[0050] Figure 1This is a system block diagram of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Please see Figure 1 As shown, this invention is a vision servo scanning trajectory control system for a deep-sea ROV intelligent electromagnetic inspection robot, comprising:

[0053] The multi-source synchronous acquisition module is used to simultaneously acquire visual image data, acoustic image data, inertial motion data, and prior geometric data of the underwater robot to be inspected.

[0054] The multimodal feature extraction module extracts sparse light spots with regular geometric shapes and stable optical properties as visual feature points based on visual image data, and assigns a confidence value to each visual feature point; based on acoustic image data, it extracts macroscopic structural contour lines and local areas with abnormal reflection intensity as acoustic features.

[0055] The fusion state estimation module inputs visual feature points, confidence assignments, acoustic features, inertial motion data, and prior geometric data into the extended state estimation framework for fusion calculation. By establishing a unified state vector that includes the underwater robot's spatial pose, motion state, and relative geometric relationship with the underwater structure, and by using the geometric and physical constraints between different data sources for iterative optimization, the module outputs in real time the underwater robot's accurate relative pose, relative motion state, and the uncertainty measure of the current estimate relative to the underwater structure.

[0056] The adaptive trajectory generation module takes the output relative pose, relative motion state and uncertainty measure as input, and combines them with preset scanning task rules to calculate and generate a local desired trajectory with spatiotemporal continuity in real time.

[0057] The variable gain closed-loop control module takes the generated local desired trajectory, relative pose, and relative motion state as input, calculates the response characteristics by automatically adjusting the control law based on the uncertainty metric, and outputs the final motion control command to the underwater robot's propulsion actuator, thus forming a closed-loop control.

[0058] In this invention, the underwater robot is an ROV intelligent electromagnetic detection robot.

[0059] In the multi-source synchronous acquisition module, visual image data is acquired through a high-definition waterproof camera component mounted on the front gimbal of the underwater robot. During the inspection operation, this component continuously captures images of the underwater environment directly in front of the robot at a fixed frequency, obtaining a raw image sequence containing optical information of the surface of the structure to be inspected.

[0060] Acoustic image data acquisition is accomplished using a scanning forward-looking sonar mounted on the bottom of the underwater robot. This sonar actively emits sound pulses and receives echoes from the surface of underwater structures, generating two-dimensional or three-dimensional acoustic images that reflect the structure's contours and surface conditions through signal processing.

[0061] The acquisition of inertial motion data is accomplished through an inertial measurement unit (IMU) fixedly installed at the core of the underwater robot's body. This unit measures and outputs the robot's raw triaxial acceleration and triaxial angular velocity data in three-dimensional space in real time.

[0062] The prior geometric data of the underwater structure to be inspected refers to the data file that has been pre-stored in the underwater robot's control computer or loaded in real time via an underwater acoustic communication link before the inspection task begins. This data file contains the design 3D model of the pipeline to be inspected, the theoretical spatial coordinates, geometric dimensions, and topological relationship information of key features (such as welds, flanges, and supports).

[0063] All four types of data are timestamped using a unified system clock to ensure strict time synchronization when entering subsequent processing flows.

[0064] In the multimodal feature extraction module, the visual image data processing begins with image segmentation. A global threshold-based binarization method is used to transform the original visual image into a binary image containing only black and white pixels. Pixels with brightness exceeding a set segmentation threshold are classified as targets (white), while the rest are considered background (black). Subsequently, through connected component analysis, all independent white connected regions are identified from the binary image; these regions are initially considered as candidate light spot regions.

[0065] For each candidate light spot region, its geometric morphology parameters are calculated to filter for regular shapes. The first parameter is contour compactness, which is calculated as: the square of the actual perimeter of the region, divided by the actual area of ​​the region, and then multiplied by four times pi. The second parameter is area roundness, which is calculated as: four times the actual area of ​​the region, divided by the area of ​​the smallest circumcircle of the region. Numerical thresholds are set for each parameter. Only when the calculated contour compactness of a candidate light spot region is less than or equal to the first numerical threshold, and the calculated area roundness is greater than or equal to the second numerical threshold, is the region determined to be a candidate visual feature point with regular geometric shape.

[0066] For each candidate visual feature point selected above, its optical stability is further analyzed. The grayscale value of the corresponding pixel in a five-frame visual image sequence is extracted. The arithmetic mean of these five grayscale values ​​is calculated as the average grayscale value; simultaneously, the standard deviation of these five grayscale values ​​is calculated as the grayscale standard deviation. A brightness threshold and a stability threshold are set. A candidate visual feature point is only determined as the final visual feature point if its average grayscale value is greater than or equal to the brightness threshold and its grayscale standard deviation is less than or equal to the stability threshold. Next, a comprehensive credibility value is calculated for each final visual feature point. The process is as follows: the three original parameters of the candidate stage, contour tightness, area roundness, and gray standard deviation, are normalized so that each parameter value is mapped to the range of zero to one. Then, the normalized contour tightness value is multiplied by the first weight coefficient, the normalized area roundness value is multiplied by the second weight coefficient, and the normalized gray standard deviation value is multiplied by the third weight coefficient. Finally, the three weighted values ​​are added together, and the sum is the comprehensive credibility value of the visual feature point.

[0067] The processing of acoustic image data begins with noise suppression. A Gaussian smoothing filter is applied to the raw acoustic image data; that is, the new gray value of each pixel is calculated by weighting the original gray values ​​of itself and its neighboring pixels according to a Gaussian distribution. Gradient magnitude images are then calculated for the smoothed image, where the value of each pixel represents the square root of the sum of the squares of the gray-level changes in intensity in the horizontal and vertical directions at that point.

[0068] In the obtained gradient magnitude image, macroscopic structural contour lines are extracted. A cumulative array-based method is used to detect all arc-shaped edge segments in the image that satisfy preset lower and upper radius conditions. The detected arc-shaped segments that are spatially connected and have continuous curvature changes in the tangential direction are combined to form a continuous macroscopic structural contour line representing the shape of structures such as underwater pipelines.

[0069] Within a strip-shaped neighborhood of a certain pixel width on both sides of the extracted macroscopic structural contour line, regions with abnormal local reflection intensity are identified. First, the average and standard deviation of the grayscale values ​​of all pixels within this neighborhood are calculated. Pixels with grayscale values ​​higher than "average plus twice the standard deviation" are initially marked as foreground points. Next, connected component analysis is performed on these foreground points, aggregating adjacent foreground points into independent connected regions. Finally, connected regions with an area greater than a preset minimum area threshold are formally marked as regions with abnormal local reflection intensity.

[0070] The results obtained from the above steps are encapsulated into acoustic features. The encapsulation includes: a sequence of spatial coordinates of an ordered series of points along the macroscopic structural contour; the coordinates of the geometric center point of each local region with abnormal reflection intensity, and the length, width, and orientation angle of its smallest bounding rectangle. These data together constitute the acoustic features used for subsequent processing.

[0071] In the fusion state estimation module, firstly, a unified state vector is defined to describe the underwater robot and its relationship with underwater structures. This state vector is a column vector containing multiple components, including the underwater robot's three-dimensional position coordinates in the global coordinate system. and three-dimensional attitude angle ; Three-dimensional linear velocity of underwater robots and three-dimensional angular velocity ; and auxiliary geometric parameters used to implicitly correlate the currently observed local structural features with the prior model, such as the tangent direction vector of the currently tracked local structure. and curvature estimate This state vector is denoted as... It unifies motion state, spatial pose, and local environmental geometric relationships within a single mathematical framework.

[0072] State prediction steps based on inertial motion data. Inertial motion data includes acceleration measured by triaxial accelerometers. Angular velocity measured by a three-axis gyroscope Based on Newton's laws of motion and rigid body kinematics, the current state is predicted through numerical integration using the optimally estimated state vector X_{k-1} from the previous moment and the inertial measurement value at the current moment. The core formula of the prediction process is the state prediction equation: ;

[0073] In this formula express The predicted state vector at time t. It is a nonlinear state transition function that depends on the state at the previous time step. The theoretical prediction of the current state is calculated, and its specific form is determined by the kinematic model of the underwater robot. yes In this embodiment, the control input vector at any given time is composed of acceleration and angular velocity data directly provided by the inertial measurement unit. It is the control input matrix, which maps the raw measurements to the state space. It is a process noise vector used to model uncertainties in the prediction process, and its covariance matrix is... The predicted pose and predicted motion state are calculated using this equation, based on pre-set sensor calibration data.

[0074] Visual feature matching and initial correction steps. Based on the predicted pose, the currently extracted visual feature points (coordinates are...) are... The camera projection model is back-projected into 3D space using a pre-calibrated camera projection model to obtain the 3D ray corresponding to the feature point under the current predicted pose assumption. This ray is then compared with the nominal features at the corresponding location in the prior geometric data (e.g., the known 3D coordinates of weld or flange corner points). Spatial matching is performed to find the closest pair that is oriented in the same direction. For each successfully matched visual feature point, a confidence score is assigned. Used as fusion weights The fusion weight is assigned a value based on credibility. The ratio of the confidence scores assigned to all successfully matched feature points. Initial correction using weighted matching results essentially solves a weighted least squares problem. The optimization objective is to minimize the weighted sum of reprojection errors of all matched feature points, thereby improving the predicted state. Make corrections to obtain the state after the initial correction. .

[0075] Acoustic feature alignment and supplementary correction steps. This step is initiated when the number of successfully matched visual feature points is less than a preset threshold (e.g., 3). First, the macroscopic structural contour lines extracted from the acoustic features (consisting of a series of ordered three-dimensional point sets) are aligned and supplemented. The rigid body transformation (representation) is fitted and aligned with the theoretical contour of the corresponding structure (e.g., a pipe section) in the prior geometric data. The alignment process finds an optimal rigid body transformation (rotation matrix) through an iterative nearest-point method. Translation vector ), making The average distance from the theoretical contour line is minimized after transformation. Secondly, the overlap between the local reflection intensity anomaly region and the marked defect potential region in the prior geometric data is calculated. . Coincidence The calculation method is as follows: the intersection area of ​​the two-dimensional projected area of ​​the acoustic anomaly region and the two-dimensional projected area of ​​the a priori possible defect region is divided by the two-dimensional projected area of ​​the a priori possible defect region. Then, supplementary corrections are performed based on the alignment results and the overlap calculation results. Specifically, the geometric center coordinates of the local reflection intensity anomaly region are calculated. The major axis angle of its smallest circumscribed rectangle in the horizontal plane Compare it with the preset center of the corresponding potential defect region in the prior geometric data. and preset direction By comparison, the center position deviation vector is obtained. and direction angle deviation Based on coverage overlap These two deviations are scaled proportionally to generate the pose adjustment amount. Among them, the position adjustment amount The calculation expression is: Direction adjustment amount The calculation expression is: Finally, Acting on the state after the first correction After completing the supplementary correction, the final optimized state estimate is obtained. .

[0076] Uncertainty metric calculation steps. The observation residuals generated during the initial and supplementary corrections are calculated in real time. For visual matching, the observation residual is the norm of the reprojection error vector for each matched feature point. For acoustic alignment, the observation residual is the average distance after the contour point set is aligned. The statistical variance of these residual values ​​is calculated. The calculated statistical variance Compared with the preset sensor noise reference variance Comparison. Uncertainty measure. Calculated using the following formula: ;

[0077] In this formula, This is the measure of uncertainty in the output. It is the statistical variance of the current observation residuals. It is a baseline variance obtained by statistical analysis of long-term test data of each sensor in the calibration environment, and is a preset constant. It represents the number of visual feature points that were successfully matched in the current frame. It is a preset reference value for the expected number of matching points under normal working conditions, such as 10. It is an exponential decay factor, which significantly increases the uncertainty measure when effective visual features are scarce. This formula incorporates observational consistency (…). ) and effective information content ( Two factors are considered. Finally, the mean and variance of the recorded supplementary correction iteration residuals are also used to update the relevant elements in the process noise covariance matrix Q online, achieving adaptive adjustment. The entire fusion calculation process is executed once in each control cycle (e.g., 100 milliseconds), with real-time output. and .

[0078] In the adaptive trajectory generation module, the trajectory generation process uses the underwater robot's relative pose, relative motion state, and uncertainty metric, output in real time by the fusion state estimation module, as core inputs. First, the received uncertainty metric value is compared with a predetermined first threshold. This first threshold, obtained through extensive historical data statistics and experimental calibration, typically has a value of 0.5 and is used to distinguish between reliable and unreliable perceived states. This judgment determines which strategy will be used to construct the robot's next motion intention.

[0079] When the uncertainty metric does not exceed the first threshold, it indicates that the current perception of itself and the environment is relatively reliable. At this point, a first type of predetermined motion unit is selected and constructed. The construction of this motion unit uses the endpoint of the historical local expected trajectory generated in the previous control cycle as the sole starting point for this motion. The target and form of the motion unit are determined by preset scanning task rules: the task rules predefine a series of target detection points that need to be reached sequentially and electromagnetic detection operations performed. These points are typically located above welds or critical parts of the structure to be inspected, and are accompanied by the desired detection posture. The construction process involves using the current starting point as the initial state and the position and posture of the next target detection point to be visited in the sequence as the final state, performing motion planning between the two. The planned path is a series of control points arranged in chronological order, containing three-dimensional position and three-dimensional posture information. This path satisfies the continuous velocity and acceleration constraints of robot kinematics, thus forming a predetermined motion unit guided by the execution of the detection task.

[0080] When the uncertainty metric exceeds the first threshold, it indicates a decline in current perception quality, possibly due to visual occlusion or severe water flow interference. At this point, a second type of predetermined motion unit is selected and constructed. This motion unit also starts from the endpoint of the previous historical trajectory, but its construction objective is entirely different from the first type. The core objective of the second type of motion unit is to "maintain the current relative pose stability," that is, to minimize the robot's pose change relative to the underwater structure within a short future time window. The construction method is as follows: using the current input relative pose and relative motion state as the desired constant values, a set of position and velocity commands is generated, ensuring that the robot's desired position remains unchanged or undergoes only minor drift compensation within the planned time period, while the desired angular velocity approaches zero. The resulting motion unit represents a conservative motion intent primarily aimed at stable dwell and waiting for perception recovery.

[0081] After selecting and constructing the corresponding predetermined motion unit (type 1 or type 2) based on the uncertainty metric, this new motion unit needs to be integrated with the existing historical trajectory. This integration process is called spatiotemporal stitching. Specifically, the last effective control point of the historical trajectory in time is obtained and connected to the first control point of the newly created motion unit in time. Since the motion states (such as velocity and acceleration) at the connection point may not be completely consistent, direct connection can lead to an unsmooth trajectory, potentially causing jitter during robot execution. Therefore, the local trajectory segments covered by the stitching point and the two control points before and after it must be smoothed. The smoothing process uses cubic polynomial interpolation, performed independently in each dimension of position and orientation. By solving a mathematical problem constrained by the position / or orientation of the trajectory points and minimizing the sum of squared jerkes of the entire trajectory, a new trajectory curve is obtained that passes through all key control points and is continuous in position, velocity, and acceleration at each point. This new curve is the final generated local expected trajectory with spatiotemporal continuity. The entire trajectory generation process is repeated once in each control cycle.

[0082] In the variable gain closed-loop control module, the closed-loop control process uses the locally desired trajectory output in real time from the adaptive trajectory generation process, and the underwater robot's relative pose and relative motion state relative to the underwater structure output in real time from the fusion state estimation process, as core inputs. First, the uncertainty metric of the input is compared with a predetermined second threshold. This second threshold is obtained through joint calibration using dynamic simulation and tank testing, and its value is typically slightly higher than the first threshold used in the trajectory generation process, for example, set to 0.7. This threshold is used to distinguish whether the control system should be in precise tracking mode or robust maintenance mode.

[0083] When the uncertainty metric does not exceed the second threshold, it indicates that the quality of the perceived information is sufficient to support accurate tracking. At this point, the first type of control law is selected for calculation. The core of this control law is calculating the trajectory tracking error: that is, subtracting the actual relative position and actual relative attitude obtained in real-time from the expected position and expected attitude of the local expected trajectory at the current moment, respectively, to obtain the position error vector and attitude error vector. Subsequently, proportional-integral-differential operations are performed on this error. The proportional term is the current error value multiplied by a proportional gain coefficient; the integral term is the historical cumulative error value multiplied by an integral gain coefficient; and the differential term is the difference between the current error and the error at the previous moment (i.e., the rate of change of error) multiplied by a differential gain coefficient. The three results are added together to obtain a preliminary control quantity. The first set of preset control gains used are a set of parameters calibrated in calm water to achieve the best tracking performance of the system, for example, a proportional gain coefficient of 2.5, an integral gain coefficient of 0.1, and a differential gain coefficient of 1.2.

[0084] When the uncertainty metric exceeds the second threshold, it indicates insufficient perception reliability, and continued forced trajectory tracking may lead to instability. In this case, a second type of control law is selected for calculation. This control law aims to maintain the stability of the current relative pose. Its calculation method replaces the trajectory tracking error with the "difference between the relative motion state and the zero vector." Specifically, the real-time input relative linear velocity and relative angular velocity are directly used as the controlled error. The control law also performs proportional-integral-derivative (PI) operations on this velocity error, but uses a different, preset second set of control gains. The parameters of this set of gains are specially designed; its proportional and integral gains are lower than the first set, while its derivative gain is higher. For example, the proportional gain coefficient is 1.0, the integral gain coefficient is 0.05, and the derivative gain coefficient is 2.0, aiming to provide a stronger damping effect, suppress motion drift, and prioritize system stability over tracking accuracy.

[0085] After selecting the control law and calculating the initial control command according to the above rules, further refined post-processing of the command is required based on the short-term trend of the uncertainty metric. The first step of post-processing is adaptive gain adjustment: continuously monitoring the value of the uncertainty metric over the most recent three consecutive control cycles. If it remains constant (fluctuation range less than 5%) or shows a decreasing trend, the environment is considered to be improving or stabilizing. At this point, all control gains corresponding to the currently used control law are uniformly multiplied by an enhancement coefficient greater than 1 but less than 1.5, such as 1.2, to improve the response speed of the control system. The second step of post-processing is filtering and smoothing: if the uncertainty metric continues to rise over the most recent two consecutive control cycles, the disturbance is considered to be intensifying. At this point, a one-time second-order low-pass filter is applied to the currently calculated initial motion control command vector. The cutoff frequency of this filter is set inversely proportional to the rise in uncertainty metric, specifically: the cutoff frequency equals a reference frequency value (e.g., 2 Hz) divided by (1 plus the rise). The rise is defined as the absolute value of the increment of uncertainty metric over the most recent two cycles. This operation can filter out high-frequency components in the command, making the control output smoother and enhancing anti-interference capabilities.

[0086] Finally, the control command vector, after gain adjustment or filtering, is compared with the maximum thrust or torque constraints provided by each thruster of the underwater robot. For each component in the command vector, if its absolute value exceeds the physical upper limit of the corresponding propulsion actuator, the component is proportionally scaled. The scaling method is as follows: find the component that exceeds the constraint by the largest absolute value, use this ratio as the global scaling factor, and multiply the entire control command vector by the reciprocal of this factor to ensure that all components are proportionally compressed within the physical constraints. The command after this scaling process is the final generated, practically safe-to-execute six-degree-of-freedom motion control command. This command is sent in real time to the thruster drive unit of the underwater robot, driving the thrusters to generate corresponding thrust and torque, thereby achieving the tracking of the desired local trajectory or the stable maintenance of the current pose, completing the entire closed-loop control process from perception to decision-making to execution.

[0087] The working principle of this invention is as follows: First, the underwater robot's sensors simultaneously acquire visual images, acoustic images, and inertial motion data, combined with a prior geometric model of the structure under inspection. Then, sparse light spots with regular shapes and stable optical properties are extracted from the visual images as visual feature points and assigned credibility. Simultaneously, structural contour lines and areas of abnormal reflection intensity are extracted from the acoustic images as acoustic features. Next, the above features, inertial data, and prior model are fused and calculated. A unified state vector containing the robot's pose, motion state, and environmental geometric relationships is established. Iterative optimization is performed using geometric and physical constraints between multi-source data, outputting in real-time the robot's precise relative pose, motion state, and an uncertainty measure representing the reliability of perception. Based on this, the system adaptively generates a local desired trajectory according to the uncertainty measure: when perception is reliable, a detection-oriented tracking trajectory is planned; when perception is unreliable, a conservative trajectory is generated with the goal of maintaining the stability of the current pose. By using a control law that can automatically switch and adjust the gain based on uncertainty metrics, motion control commands are calculated and output to drive the thruster to execute, thereby achieving high-precision tracking of the detection trajectory or robust maintenance of its own posture in complex underwater environments, forming a complete closed loop of perception, decision-making and control.

[0088] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A vision servo scanning trajectory control system for a deep-sea ROV intelligent electromagnetic inspection robot, characterized in that, include: The multi-source synchronous acquisition module is used to simultaneously acquire visual image data, acoustic image data, inertial motion data, and prior geometric data of the underwater robot to be inspected. The multimodal feature extraction module extracts sparse light spots with regular geometric shapes and stable optical properties as visual feature points based on visual image data, and assigns a confidence value to each visual feature point; based on acoustic image data, it extracts macroscopic structural contour lines and local areas with abnormal reflection intensity as acoustic features. The fusion state estimation module inputs visual feature points, confidence assignments, acoustic features, inertial motion data, and prior geometric data into the extended state estimation framework for fusion calculation. By establishing a unified state vector that includes the underwater robot's spatial pose, motion state, and relative geometric relationship with the underwater structure, and by using the geometric and physical constraints between different data sources for iterative optimization, the module outputs in real time the underwater robot's accurate relative pose, relative motion state, and the uncertainty measure of the current estimate relative to the underwater structure. The adaptive trajectory generation module takes the output relative pose, relative motion state and uncertainty measure as input, and combines them with preset scanning task rules to calculate and generate a local desired trajectory with spatiotemporal continuity in real time. The variable gain closed-loop control module takes the generated local desired trajectory, relative pose, and relative motion state as input, calculates the response characteristics by automatically adjusting the control law based on the uncertainty metric, and outputs the final motion control command to the underwater robot's propulsion actuator, thus forming a closed-loop control.

2. The deep-sea ROV intelligent electromagnetic inspection robot visual servo scanning trajectory control system according to claim 1, characterized in that, Assigning a confidence value to each visual feature point specifically includes: The visual image data is binarized to obtain candidate light spot regions; Calculate the contour compactness and area roundness of each candidate spot region, and determine the regions that simultaneously meet the compactness threshold and roundness threshold as candidate visual feature points with regular geometric shapes. Calculate the average gray value and gray standard deviation of each candidate visual feature point in multiple consecutive frames of visual images. Points with an average gray value higher than the brightness threshold and a gray standard deviation lower than the stability threshold are determined as the final visual feature points. Based on the contour tightness, area roundness, and grayscale standard deviation, a comprehensive credibility value is calculated and assigned to each final visual feature point through normalization and linear combination.

3. The deep-sea ROV intelligent electromagnetic inspection robot visual servo scanning trajectory control system according to claim 1, characterized in that, The extraction of macroscopic structural contour lines and local areas of abnormal reflection intensity as acoustic features specifically includes: Gaussian smoothing filter is applied to the acoustic image data, and the gradient magnitude image of the acoustic image data is calculated; In the gradient magnitude image, extract the arc segments that meet the preset radius range, and combine the interconnected arc segments with continuous curvature to form the macroscopic structural outline. Within a preset neighborhood of the extracted macroscopic structural contour lines, connected regions with echo intensity higher than adjacent background regions are marked as regions with abnormal local reflection intensity. The macroscopic structural outline, along with the geometric center coordinates and circumscribed rectangle of the local abnormal reflection intensity region, are collectively encapsulated into acoustic features.

4. The deep-sea ROV intelligent electromagnetic inspection robot visual servo scanning trajectory control system according to claim 1, characterized in that, The fusion computing specifically includes: Based on inertial motion data, the spatial pose and motion state of the underwater robot are predicted to obtain the predicted pose and predicted motion state. Based on the predicted pose, the visual feature points are spatially matched with the nominal features at the corresponding positions in the prior geometric data. Different fusion weights are assigned to the successfully matched visual feature points according to the confidence level. The weighted matching results are used to perform the first correction on the predicted pose and the predicted motion state. When the number of available visual feature points for matching is less than the number threshold, the macroscopic structural contour line in the acoustic features is fitted and aligned with the contour of the corresponding structure in the prior geometric data, and the coverage overlap of the local abnormal reflection intensity region with the possible defect region in the prior geometric data is calculated. Based on the alignment and overlap calculation results, the pose and motion state after the first correction are supplemented and corrected. The statistical variance of the observation residuals during the initial and supplementary correction processes is calculated in real time. The statistical variance is compared with a preset sensor noise benchmark, and an uncertainty measure characterizing the reliability of the current estimate is output.

5. The deep-sea ROV intelligent electromagnetic inspection robot visual servo scanning trajectory control system according to claim 4, characterized in that, The supplementary correction of the pose and motion state after the initial correction based on the alignment and overlap calculation results specifically includes: Calculate the geometric center of the local reflection intensity anomaly region, and calculate the major axis direction angle of the rectangle circumscribed by the geometric center; The geometric center and orientation angle are compared with the preset center and preset orientation of the corresponding defect possibility area in the prior geometric data to obtain the center position deviation and orientation angle deviation respectively. Based on the overlap ratio, the center position deviation and orientation angle deviation are scaled proportionally to generate the pose adjustment amount; The pose adjustment is applied to the pose and motion state after the first correction to complete the supplementary correction, and the iterative residual of this supplementary correction is recorded to update the uncertainty metric.

6. The deep-sea ROV intelligent electromagnetic inspection robot visual servo scanning trajectory control system according to claim 1, characterized in that, The generation of a locally desired trajectory with spatiotemporal continuity specifically includes: Determine whether the uncertainty measure exceeds a predetermined first threshold; When the uncertainty measure does not exceed the first threshold, the first type of predetermined motion unit is selected for construction. The first type of predetermined motion unit is generated with the trajectory endpoint of the previous moment as the starting point and guided by the target detection point sequence determined by the scanning task rules. When the uncertainty measure exceeds the first threshold, a second type of predetermined motion unit is selected for construction. The second type of predetermined motion unit is generated with the trajectory endpoint of the previous moment as the starting point and with the goal of maintaining the current relative pose stability. The selected predetermined motion unit is spatiotemporally stitched with the current historical trajectory, and the stitching points are smoothed to form and output the local desired trajectory.

7. The deep-sea ROV intelligent electromagnetic inspection robot visual servo scanning trajectory control system according to claim 1, characterized in that, The output process of the motion control command is as follows: Determine whether the uncertainty measure exceeds a predetermined second threshold; When the uncertainty measure does not exceed the second threshold, the first type of control law is selected for calculation. The first type of control law uses the difference between the local desired trajectory and the relative pose as the trajectory tracking error, and performs proportional-integral-differential calculation on the trajectory tracking error using the first set of preset control gains. When the uncertainty measure exceeds the second threshold, the second type of control law is selected for calculation. The second type of control law aims to maintain relative pose stability, replaces the trajectory tracking error with the difference between the relative motion state and the zero vector, and uses the second set of preset control gains for calculation. Based on the selected control law type and corresponding control gain, motion control commands are calculated in real time and output.

8. The deep-sea ROV intelligent electromagnetic inspection robot visual servo scanning trajectory control system according to claim 7, characterized in that, The real-time calculation to obtain and output motion control commands specifically includes: When the uncertainty measure remains stable or decreases over three consecutive control cycles, the control gain corresponding to the selected control law is multiplied by an enhancement factor greater than 1 but less than 1.

5. When the uncertainty metric increases over two consecutive control cycles, a second-order low-pass filter is applied to the currently calculated motion control command. The cutoff frequency of the low-pass filter is inversely proportional to the magnitude of the increase in the uncertainty metric. The calculation results after enhancement coefficient adjustment or low-pass filtering are compared with the maximum thrust constraint of the underwater robot's propulsion actuator. The command components that exceed the maximum thrust constraint are proportionally scaled and limited to generate and output the actual executable motion control commands.

Images