Underwater robot inspection and obstacle avoidance control method and system

Abstract

The present invention discloses an underwater robot inspection and obstacle avoidance control method and system, which relates to the field of robotics. By introducing an event camera to construct a two-dimensional event response distribution map, fusing local feature vectors obtained by multiple sensors to form a semantic point cloud, and combining with a historical environment map to construct a triple optimization mechanism of geometry, semantics and dynamic residuals, the current state and the historical map are deeply aligned, which further improves the accuracy of dynamic obstacle recognition and the adaptive ability of obstacle avoidance path planning. Through the calculation of dynamic confidence C and the color coding mechanism, the updated environment map can be partitioned into risk levels, distinguishing high dynamic risk, medium risk and low risk areas, effectively enhancing the intelligent decision-making ability of path selection. On this basis, combined with the return priority scheduling mechanism, data in red areas are uploaded first, achieving consistent matching between task risk level and data upload priority, and reducing redundant data transmission.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to an underwater robot inspection and obstacle avoidance control method and system. Background Art

[0002] In underwater application scenarios, underwater robots are widely used in tasks such as aquaculture monitoring, seabed facility maintenance, and energy pipeline inspections. They have to face complex and changeable environmental conditions with limited vision and significant sensor interference. Under this technical requirement, the control strategy with "underwater robots' recognition of dynamic obstacles during autonomous inspections" as the core has become a research focus, especially the ability to detect abnormal disturbances and adjust paths in real time in dynamic environments, which is directly related to the robot's safety and task completion efficiency.

[0003] The current mainstream obstacle avoidance control methods for underwater robots mostly rely on sonar, structured light or traditional visual sensors to obtain static environmental information. When faced with highly dynamic disturbance environments, such as surging water, temporary moving objects or biological disturbances, they often lack high responsiveness and adaptive update capabilities, and it is difficult to accurately identify areas of abnormal changes. Such defects may cause a series of system anomalies during the execution of tasks. For example, the robot may misjudge a high-disturbance area as a passable area, resulting in an increased risk of collision. Even in the process of multi-area mission feedback, important information may be delayed in uploading due to risk judgment errors, affecting the overall task scheduling and the efficiency of the human-machine collaboration mechanism, and ultimately damaging the stability and accuracy reliability of the operation. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the present invention provides an underwater robot inspection and obstacle avoidance control method and system, which solves the problems in the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: An underwater robot inspection and obstacle avoidance control method comprises the following steps:

[0006] S1: Use several sets of sensors and event cameras to monitor the environmental conditions of the underwater scene in real time to generate a two-dimensional event response distribution map. Based on the two-dimensional event response distribution map, determine the fused strong feature vector , and identify areas of abnormal dynamic changes;

[0007] S2: Obtain the environment map constructed at the previous moment in advance, combine it with the two-dimensional event response distribution map, determine several groups of point pairs, and analyze the positional relationship, semantic relationship and dynamic change between each point pair to construct the global objective function , after optimization, the optimal state variables are obtained , based on the optimal state variables , update the environment map constructed at the previous moment and generate an updated environment map;

[0008] S3: Determine the dynamic situation of each location in the updated environment map to generate a dynamic confidence C of each sub-area in the updated environment map to provide the underwater robot with obstacle avoidance path selection;

[0009] S4: Based on the dynamic confidence C value of each sub-area, the backhaul priority scheduling mechanism is triggered.

[0010] Preferably, S11: pre-capture the event that occurs when the brightness of each pixel in the underwater scene changes using an event camera to obtain a timestamp of the event. and pixel coordinates;

[0011] S12: Based on the parameters obtained in S11, simulate the real dynamic change information in the underwater scene and consider noise interference to calculate the event signal intensity at different pixel coordinates , specifically:

[0012] ;

[0013] Where, is the event signal intensity at the pixel coordinate (x, y) at time t, where t is the current time point, is the time offset index of multiple events that occurred, is the time decay constant, is the position weight;

[0014] S13: Traverse all pixel coordinates on the image plane involved in the event camera to form a A two-dimensional event response distribution graph where the values ​​are pixel intensities.

[0015] Preferably, S14: using several groups of sensors to monitor the environmental conditions of the underwater scene in real time, and dividing the two-dimensional event response distribution map into several groups of regions to generate local feature vectors h acquired by different sensors, and fusing the local feature vectors h acquired by each sensor to obtain a fused strong feature vector ;

[0016] S15: The fused strong feature vector Associated with the monitoring positions of the corresponding sensors to form a representation corresponding to each local area in the underwater scene to generate a semantic point cloud ;

[0017] S16: Based on the event signal intensity at different pixel coordinates obtained in S12 , the threshold segmentation method is used to identify the dynamic target area to form a binary dynamic obstacle mask , dynamic obstacle mask based on binarization , identifying the abnormal dynamic change area.

[0018] Preferably, S21: pre-acquire the environment map constructed at the previous moment, and use the nearest neighbor search algorithm to find the semantic point cloud in each local area. Establish a preliminary correspondence with the environment map constructed at the previous moment to obtain several groups of point pairs;

[0019] S22: Pre-set state variables to be optimized , and according to each semantic point cloud in several groups of point pairs The position of and the corresponding matching point in the environment map constructed at the previous moment, and obtain the geometric residual of each point pair , specifically: ;in, is the geometric residual of the kth semantic point cloud, is the position of the kth semantic point cloud, is the position that matches the position of the kth semantic point cloud in the environment map constructed at the previous moment; is the rigid transformation matrix to be optimized;

[0020] S23: According to the point pairs determined in S21, extract the corresponding semantic point cloud in each point pair And the position that matches the position of the corresponding semantic point cloud in the environment map constructed at the previous moment, the corresponding fused strong feature vector , by analyzing the feature differences within each point pair to obtain the semantic residual , specifically: Where, is the semantic residual of the kth semantic point cloud, is the fused strong feature vector in the kth semantic point cloud, The fused strong feature vector corresponding to the position that matches the position of the corresponding semantic point cloud in the environment map constructed at the previous moment ;

[0021] S24: Based on the abnormal dynamic change area identified in S16, determine the number of abnormal dynamic changes that occurred in the abnormal dynamic change area in the past, and obtain the dynamic residual of each point pair by considering the interference of the abnormal dynamic change area on the update of the environment map constructed at the previous moment , specifically: Where, is the dynamic residual of the kth semantic point cloud, is the dynamic weight, where ;in, is a dynamic obstacle mask based on the binary Calculate dynamic probability.

[0022] Preferably, S25: construct a global objective function , to minimize the geometric residual by , semantic residual and dynamic residuals Perform weighted square sum so that the state variables to be optimized obtained by optimization are As the optimal state variable , through the optimal state variables , the current fused strong feature vector Projected into the coordinate system defined by the environment map constructed at the previous moment to achieve alignment, specifically: ;in, The new position coordinates are obtained by converting the position of the kth semantic point cloud in the two-dimensional event response distribution graph to the coordinate system defined by the environment map constructed at the previous moment.

[0023] Preferably, S31: based on the environment map constructed at the previous moment, the position of the kth semantic point cloud in the two-dimensional event response distribution map is converted to the new position coordinates after the coordinate system defined by the environment map constructed at the previous moment is converted. , get the updated environment map.

[0024] Preferably, S32: dividing the updated environment map into several groups of sub-areas;

[0025] S33: In the underwater scene, in order to consider the dynamic degree of each location in the underwater scene, the dynamic trajectory is presented in the updated environment map. The updated environment map includes the dynamic confidence C of each sub-area, specifically:

[0026] ;

[0027] Where, is the dynamic confidence of the Ath sub-area in the updated environment map, is the number of static obstacle points in the Ath sub-area, is the semantic point cloud corresponding to the Ath sub-area The total number of positions, is the variance of the event signal intensity change rate at the a-th position in the corresponding sub-region, and a is the position number in the corresponding sub-region.

[0028] Preferably, S34: pre-set a confidence range, if the dynamic confidence C of each sub-region exceeds the confidence range, the corresponding sub-region in the updated environment map is rendered green; if the dynamic confidence C of each sub-region falls within the confidence range, the corresponding sub-region in the updated environment map is rendered orange; if the dynamic confidence C of each sub-region does not exceed the confidence threshold and does not fall within the confidence range, the corresponding sub-region in the updated environment map is rendered red;

[0029] S35: According to the current position of the underwater robot in the updated environment map and in combination with the sub-area displayed in red in S34, an obstacle avoidance path selection is provided for the underwater robot.

[0030] Preferably, S41: performing color presentation on the corresponding sub-areas in the updated environment map in S34 and executing the return priority scheduling mechanism, the specific execution steps are:

[0031] S411: extracting the sub-areas in red in the updated environment map and uploading them first;

[0032] S412: Extract the orange sub-area and the green sub-area in the updated environmental map, compress the data of the orange sub-area and the green sub-area, and then upload them. The orange sub-area is uploaded after the red sub-area, and the green sub-area is uploaded after the orange sub-area.

[0033] An underwater robot inspection and obstacle avoidance control system, including a graph construction module, a graph update module, a presentation module, and a feedback module;

[0034] The graph construction module is used to use several sets of sensors and event cameras to monitor the environmental conditions of the underwater scene in real time to generate a two-dimensional event response distribution map. Based on the two-dimensional event response distribution map, the fused strong feature vector is determined. , and identify areas of abnormal dynamic changes;

[0035] The graph update module is used to pre-acquire the environment map constructed at the previous moment, and combine it with the two-dimensional event response distribution map to determine several groups of point pairs. The positional relationship, semantic relationship and dynamic change between each point pair are analyzed to construct the global objective function. , after optimization, the optimal state variables are obtained , based on the optimal state variables , update the environment map constructed at the previous moment and generate an updated environment map;

[0036] The presentation module is used to determine the dynamic situation of each location in the updated environment map to generate a dynamic confidence C of each sub-area in the updated environment map to provide the underwater robot with obstacle avoidance path selection;

[0037] The feedback module is used to trigger the feedback priority scheduling mechanism based on the value of the dynamic confidence C of each sub-area.

[0038] The present invention provides an underwater robot inspection and obstacle avoidance control method and system, which has the following beneficial effects:

[0039] By introducing an event camera to construct a two-dimensional event response distribution map, fusing local feature vectors obtained by multiple sensors to form a semantic point cloud, and combining it with the historical environment map to construct a triple optimization mechanism of geometry, semantics, and dynamic residuals, the current state is deeply aligned with the historical map, further improving the accuracy of dynamic obstacle recognition and the adaptive ability of obstacle avoidance path planning. Through the calculation of dynamic confidence C and the color coding mechanism, the updated environment map can be divided into risk levels, distinguishing high dynamic risk, medium risk, and low risk areas, effectively enhancing the intelligent decision-making ability of path selection. On this basis, combined with the return priority scheduling mechanism, data in red areas is uploaded first, while orange and green areas are compressed and uploaded in sequence, achieving consistent matching between task risk level and data upload priority. This layered upload mechanism further optimizes the data return efficiency under the limited bandwidth of the underwater robot, reduces redundant data transmission, and prioritizes the timely processing and response of data in high-risk areas, effectively improving the system's perception agility and operational safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of the underwater robot inspection and obstacle avoidance control method of the present invention;

[0041] Figure 2 This is the overall logic diagram of the underwater robot inspection and obstacle avoidance control method of the present invention;

[0042] Figure 3 This is a partial logic diagram of the underwater robot inspection and obstacle avoidance control method of the present invention;

[0043] Figure 4 This is a block diagram of the underwater robot inspection and obstacle avoidance control system of the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] Example 1

[0046] See also Figures 1 to 3 The present invention provides an underwater robot inspection and obstacle avoidance control method, comprising the following steps:

[0047] S1: Use several sets of sensors and event cameras to monitor the environmental conditions of the underwater scene in real time to generate a two-dimensional event response distribution map. Based on the two-dimensional event response distribution map, determine the fused strong feature vector , and identify areas of abnormal dynamic changes;

[0048] S2: Obtain the environment map constructed at the previous moment in advance, combine it with the two-dimensional event response distribution map, determine several groups of point pairs, and analyze the positional relationship, semantic relationship and dynamic change between each point pair to construct the global objective function , after optimization, the optimal state variables are obtained , based on the optimal state variables , update the environment map constructed at the previous moment and generate an updated environment map;

[0049] S3: Determine the dynamic situation of each location in the updated environment map to generate a dynamic confidence C of each sub-area in the updated environment map to provide the underwater robot with obstacle avoidance path selection;

[0050] S4: Based on the dynamic confidence C value of each sub-area, the backhaul priority scheduling mechanism is triggered.

[0051] In this embodiment, a dynamic perception mechanism based on the fusion of event cameras and multimodal sensors is used to achieve real-time recognition of disturbed areas in the underwater environment and dynamic map updates. By introducing semantic point clouds, fusion residual optimization, dynamic confidence metric evaluation, and color-coded feedback scheduling mechanisms, the robot can autonomously judge the path passability and risk level, thereby realizing a closed-loop control link of "environmental modeling-dynamic recognition-obstacle avoidance path-data scheduling."

[0052] In particular, the dynamic confidence level C and color-coded partitioning mechanism introduced in steps S34-S41 enable hierarchical management and intelligent scheduling of inspection data: high-risk areas (red) are identified as dynamically active and prioritized for upload, ensuring timely response to sudden risks; medium-risk areas (orange) are compressed and uploaded later; and low-risk areas (green) are also compressed and uploaded last, saving bandwidth and computing resources. For example, during a submarine pipeline inspection, an underwater robot detected a sudden, high-speed moving target (such as a deep-sea organism or a drifting device) while passing through a certain area. The event signal intensity fluctuated dramatically, and the corresponding area was marked red. The system immediately transmitted data from this area back to the control platform for analysis and adjustment of the robot's path. Adjacent areas with milder dynamic disturbances were classified as orange or green and only compressed and transmitted later to avoid system congestion. This method effectively improves task response speed and data utilization efficiency, enhancing the robot's operational autonomy and information perception reliability in complex underwater environments.

[0053] Example 2

[0054] Please refer to Figures 1 to 3 ,Specifically: S11: Use the event camera to capture the event that occurs when the brightness of each pixel in the underwater scene changes in advance to obtain the timestamp of the event and pixel coordinates;

[0055] An event camera is a sensor designed based on biological vision mechanisms. Unlike traditional frame-based cameras, it operates on an asynchronous event-driven principle. While traditional cameras capture full frames at fixed intervals, event cameras respond immediately to changes in pixel brightness, generating an "event." Rather than capturing static images, event cameras record pixel-level events instead of capturing them frame by frame. When brightness changes occur in the environment, the sensor generates an event at the corresponding pixel location.

[0056] S12: Get parameters based on S11 (timestamp of event occurrence) and pixel coordinates), simulating the real dynamic change information in the underwater scene and taking into account noise interference to calculate the event signal intensity at different pixel coordinates , specifically:

[0057] ;

[0058] Where, is the event signal intensity at the pixel coordinate (x, y) at time t, where t is the current time point, is the time offset index of multiple events that occurred, is the time decay constant; The position weight represents the importance of each pixel position in the event response. This item can be obtained through pre-collected underwater environment sample data and neural network training. The training data includes a large number of underwater scene images marked with real obstacles and background noise, so that the model can automatically learn to have a lower response to noise, which is used to suppress noise interference caused by water particles, light scattering, etc.

[0059] Time decay constant It is used to control the rate of decay over time, which can be obtained through sensor calibration experiments; in underwater environments, reasonable value to ensure the distinction between noise and real dynamic events.

[0060] is the Dirac function, at the time of the event Activation, directly output by the event camera, when a pixel is When the brightness changes, the term is 1, otherwise it is 0. Using the Dirac function It is only activated when an event occurs, and the event is weighted in the time domain through a subsequent exponential decay function to reflect its impact over time. is a time exponential decay function, which is used to smoothly weight the event signal to ensure that the signal contribution is smaller as it is farther away from the event moment;

[0061] Due to the introduction of exponential decay and position weight , It can condense information such as the frequency and intensity of recent events at the pixel coordinates (x, y), reflecting whether dynamic changes are active.

[0062] S13: Traverse all pixel coordinates on the image plane involved in the event camera to form a A two-dimensional event response distribution graph where the values ​​are pixel intensities.

[0063] The pixel position refers to the specific pixel coordinates on the event camera sensor, which clearly indicates the specific area where the brightness change occurs;

[0064] In this embodiment, the present invention uses an event camera to capture the brightness changes of each pixel in an underwater environment in real time, obtaining the event timestamp and coordinates of each pixel. Subsequently, based on the S11 internal parameter, the event signal intensity of each pixel is calculated through exponential decay and spatial weighting. This intensity comprehensively reflects the frequency and intensity of events occurring in the pixel area in the recent period, while effectively suppressing short-term interference introduced by noise.

[0065] Finally, the system traverses all pixel coordinates on the image plane, forming a two-dimensional event response distribution map with event signal intensity as the pixel value. This method has the beneficial effect of quickly and accurately condensing the real dynamic change information in the underwater environment, providing high-quality data support for subsequent obstacle detection and obstacle avoidance control. For example, in actual applications, when a school of fish suddenly enters the robot's monitoring area, the event signal intensity of the pixels in this area will increase sharply. After the above processing, this area will appear as a clear high-intensity area in the event response distribution map, and the system will identify it as an abnormal dynamic change area, prompting the robot to adjust its obstacle avoidance strategy to ensure safe passage.

[0066] Example 3

[0067] Please refer to Figures 1 to 3 Specifically: S14: Use several groups of sensors to monitor the environmental conditions of the underwater scene in real time, and divide the two-dimensional event response distribution map into several groups of regions to generate local feature vectors h collected by different sensors, and fuse the local feature vectors h collected by each sensor to obtain a strong feature vector after fusion. , specifically:

[0068] ;

[0069] Where, is the strong feature vector after fusion in the i-th region, i and j are the region numbers, is the neighborhood set corresponding to the i-th region. The neighborhood set represents the regions adjacent to the corresponding region. By aggregating information on these nodes, feature alignment and enhancement across modalities and data sources can be achieved. is the attention weight, which reflects the importance of the association between regions i and j in the cross-modal fusion process. It is calculated using the input features by setting a gating network or attention mechanism (such as dot product attention). Its value is normalized to ensure that the sum is 1;

[0070] and is the local eigenvector of the i-th and j-th regions, In order to concatenate the local feature vectors of the i-th and j-th regions, the two vectors are directly concatenated to form a higher-dimensional feature representation. is the activation function (Sigmoid function), which is used to introduce nonlinear mapping; The feature fusion matrix is ​​obtained by passing each local feature vector h through the back propagation algorithm. It is used to perform linear transformation on the concatenated feature vector so that information from different modalities can be uniformly represented in the same feature space, thereby ensuring that the features of each modality maintain valid semantic information after fusion.

[0071] Specifically, underwater environments often integrate multi-source sensor data such as sonar, event cameras, and IMUs. Each sensor has a different signal-to-noise ratio and spatial resolution. The main goal of cross-modal fusion is to align the feature representations of each sensor in the same coordinate system to avoid the limitations of single-modal information.

[0072] Strong eigenvector after fusion It is used to fully integrate local information from different sensors and has stronger robustness and discrimination ability.

[0073] S15: The fused strong feature vector Associated with the monitoring positions of the corresponding sensors to form a representation corresponding to each local area in the underwater scene to generate a semantic point cloud ;Semantic point cloud is a strong feature vector containing position information and the corresponding position ;

[0074] S16: Based on the event signal intensity at different pixel coordinates obtained in S12 , the threshold segmentation method is used to identify the dynamic target area to form a binary dynamic obstacle mask , dynamic obstacle mask based on binarization , identify abnormal dynamic change areas, binary dynamic obstacle mask It means that those pixels judged as "dynamically changing" are marked as 1, and the rest of the pixels are marked as 0, thus forming a binary image that can represent the distribution of dynamic targets.

[0075] In this embodiment, the present invention achieves accurate perception and modeling of the underwater environment by the underwater robot through steps S14 to S16, further improving the ability to identify dynamic obstacles and the efficiency of obstacle avoidance decisions. Specifically, step S14 utilizes multiple sets of sensors to monitor the environmental conditions of the underwater scene in real time, and fuses local feature vectors from different sensors to generate high-dimensional strong feature vectors. By weighting and splicing feature vectors from multiple regions, and introducing activation functions and feature fusion matrices, this process effectively expands the ability to express information, allowing data from different modalities to be uniformly represented in the same feature space, thereby achieving accurate analysis and dynamic perception of the underwater scene.

[0076] Taking dynamic obstacle recognition as an example, when an underwater robot navigates an underwater tunnel, the surrounding water flow and countercurrents may cause changes in the surrounding environment. The semantic point cloud generated in step S15 allows the system to accurately reflect the dynamic changes in different areas, helping the robot determine which areas are static and which contain dynamic obstacles. Then, in step S16, the system uses threshold segmentation to identify dynamic target areas and generates a binary dynamic obstacle mask. This mechanism effectively reduces false positives and false negatives caused by dynamic obstacles. Through this processing, the system not only enhances its sensitivity to local dynamic obstacle areas but also extracts useful feature information from high-resolution images while removing noise, thereby improving the robot's autonomous navigation and obstacle avoidance capabilities in complex underwater environments. For example, when strong currents affect the structure of an underwater tunnel, the system can dynamically distinguish between the interference signals of the current and actual obstacles, planning a safer path for the robot and avoiding collisions. This solution effectively improves the system's adaptability to dynamic environments and ensures efficient and stable task execution. For example, during an underwater pipeline inspection, the system divides the pipeline area into blocks. When the sensor in a certain area detects a large number of event signals and the fused local features indicate that the edge of the area is a pipeline, the semantic point cloud generated by the fused features will accurately depict the pipeline outline; at the same time, the binary obstacle mask extracted by threshold segmentation identifies the dynamic interference area, ensuring that in areas with serious obstacles, the system prioritizes returning complete image data to support precise decision-making, thereby significantly reducing data redundancy and communication load.

[0077] Example 4

[0078] Please refer to Figures 1 to 3 , specifically: S21: pre-acquire the environment map constructed at the previous moment, and use the nearest neighbor search algorithm to find the semantic point cloud for each local area Establish a preliminary correspondence with the environment map constructed at the previous moment to obtain several groups of point pairs;

[0079] S22: Pre-set state variables to be optimized , and according to each semantic point cloud in several groups of point pairs The position of and the corresponding matching point in the environment map constructed at the previous moment, and obtain the geometric residual of each point pair , specifically: ;in, is the geometric residual of the kth semantic point cloud, is the position of the kth semantic point cloud, is the position that matches the position of the kth semantic point cloud in the environment map constructed at the previous moment; is the rigid transformation matrix to be optimized, which is the state variable to be optimized The obtained transformation matrix, the rigid transformation matrix to be optimized Is the transformation matrix from the current coordinate system to the coordinate system of the environment map constructed at the previous moment (belongs to the state variable to be optimized );

[0080] Geometric residual After the rigid transformation matrix T to be optimized, the mismatch between the positions of the two sets of corresponding points in space reflects the geometric alignment error of the point pair.

[0081] By using the ICP algorithm to find the nearest neighbor matching point in the environment map constructed at the previous moment, the nearest neighbor matching point is the corresponding semantic point cloud location;

[0082] Among them, the number of point pairs and semantic point cloud The number of points is the same, and each semantic point cloud The numbers of correspond to the numbers of the point pairs one by one;

[0083] S23: According to the point pairs determined in S21, extract the corresponding semantic point cloud in each point pair And the position that matches the position of the corresponding semantic point cloud in the environment map constructed at the previous moment, the corresponding fused strong feature vector , by analyzing the feature differences within each point pair to obtain the semantic residual , specifically: Where, is the semantic residual of the kth semantic point cloud, is the fused strong feature vector in the kth semantic point cloud, The fused strong feature vector corresponding to the position that matches the position of the corresponding semantic point cloud in the environment map constructed at the previous moment ;

[0084] Semantic Residual Reflects the matching degree of semantic features;

[0085] S24: Based on the abnormal dynamic change area identified in S16, determine the number of abnormal dynamic changes that occurred in the abnormal dynamic change area in the past, and obtain the dynamic residual of each point pair by considering the interference of the abnormal dynamic change area on the update of the environment map constructed at the previous moment , specifically: Where, is the dynamic residual of the kth semantic point cloud, is the dynamic weight, whose value is calculated based on the probability of being identified as dynamic at that point, where ;in, is a dynamic obstacle mask based on the binary The calculated dynamic probability reflects the possibility of the point being identified as dynamic. The bigger, The smaller it is, the weaker the influence of the dynamic area on the overall residual is, thus avoiding the large interference of dynamic points in global optimization. Reflecting the uncertainty of dynamic areas, the residuals of these areas are adjusted downward through dynamic weights, so that the points that have large errors due to motion or noise have relatively low contributions to the overall optimization goal.

[0086] S25: Constructing a global objective function , to minimize the geometric residual by , semantic residual and dynamic residuals Perform weighted square sum so that the state variables to be optimized obtained by optimization are As the optimal state variable , through the optimal state variables , the current fused strong feature vector Projected into the coordinate system defined by the environment map constructed at the previous moment to achieve alignment, specifically: ;in, The new position coordinates after converting the position of the kth semantic point cloud in the two-dimensional event response distribution map to the coordinate system defined by the environment map constructed at the previous moment;

[0087] Global objective function The specific expression is:

[0088] ;

[0089] Where k is the number of the semantic point cloud, For all state variables to be optimized Among the values ​​of , find the optimal state variable that makes the global objective function reach the minimum value , that is: among all the state variables to be optimized In the combination of The parameter combination that achieves the minimum value; and are weights, Indicates the influence of control semantic information on the overall optimization, To adjust this part ( ) constraint contribution, and The optimal configuration of these two parameters is obtained through offline experiments. represents the Euclidean distance norm;

[0090] State variables to be optimized Refers to the local scanning data (semantic point cloud) currently collected by the underwater robot. ) is a key parameter of the spatial relationship between the local coordinate system and the environment map constructed at the previous moment. Its main function is to map the data of the current local coordinate system to the coordinate system of the environment map constructed at the previous moment through rotation and translation, thereby achieving alignment and fusion of the two. State variables to be optimized It consists of a 3*3 rotation matrix and a 3*1 translation vector;

[0091] Solve the state variables to be optimized through optimization , it can use the latest information to update and correct the previous map in each data collection, thereby effectively reducing the cumulative error. If the direct mapping is not optimized, it will be difficult to correct the positioning error caused by sensor noise and dynamic interference.

[0092] Specifically, the purpose of aligning the previous map with the current data is to merge the new data with the historical data and dynamically update the map to make it more consistent with the actual situation of the current environment.

[0093] In this embodiment, the present invention realizes the precise alignment and fusion of the current collected data and the previous environmental map through steps S21-S25, thereby dynamically updating the map to make it closer to the real underwater environment. For example, a preliminary correspondence is established between each semantic point cloud in each local area and the previous map using the nearest neighbor search, and then the geometric residuals are calculated based on the matching point pairs to reflect the spatial position deviation, the semantic residuals to reflect the consistency of the feature description, and the dynamic residuals to reflect the interference of abnormal dynamic areas; finally, a global objective function is constructed, and the optimal state variable, that is, the optimal rigid transformation matrix T, is obtained by minimizing the weighted square sum of the residuals, and the strong feature vector after the current fusion is projected into the coordinate system of the previous environmental map. The beneficial effect of this method is that it can effectively fuse the real-time collected data with the historical map, correct the accumulated errors, and take into account the interference of dynamic information, so that the updated environmental map can more accurately reflect the current environmental state.

[0094] Example 5

[0095] Please refer to Figure 1 and Figure 3 Specifically: S31: Based on the environment map constructed at the previous moment, the new position coordinates after the position of the kth semantic point cloud in the two-dimensional event response distribution map is converted to the coordinate system defined by the environment map constructed at the previous moment , obtain the updated environment map to provide the robot with a reference benchmark for global navigation and obstacle avoidance.

[0096] S32: Divide the updated environment map into several groups of sub-areas;

[0097] S33: In the underwater scene, in order to consider the dynamic degree of each location in the underwater scene, the dynamic trajectory is presented in the updated environment map. The updated environment map includes the dynamic confidence C of each sub-area, specifically:

[0098] ;

[0099] Where, is the dynamic confidence of the Ath sub-area in the updated environment map, is the number of static obstacle points in the Ath sub-area, is the semantic point cloud corresponding to the Ath sub-area The total number of positions, is the variance of the event signal intensity change rate at the a-th position in the corresponding sub-region, a is the position number in the corresponding sub-region; exp(*) is the exponential function; is the proportion of static points, To quantify dynamic information, an exponential function is used as a decay term to suppress areas with obvious motion. The smaller the value, the more drastic the dynamic changes in the corresponding sub-region and the higher the risk;

[0100] In this embodiment, first, step S31 uses the new position coordinates after converting the kth semantic point cloud in the two-dimensional event response distribution map to the previous map coordinate system to accurately update the global map so that it reflects the current environmental status in real time, providing accurate navigation and obstacle avoidance benchmarks for the underwater robot.

[0101] Next, step S32 divides the updated map into several sub-areas, enabling detailed management of local risks. Step S33 calculates the static proportion within each sub-area based on the static obstacle count and the total number of points. This is combined with the variance of the rate of change of event signal strength at the corresponding location, adjusted by an exponential decay function, to generate a dynamic confidence factor, C, a quantitative indicator of dynamic information. Through zoning management and dynamic risk quantification, the system can intelligently allocate path selection and decision-making criteria globally, further improving obstacle avoidance efficiency and navigation safety, while reducing the risk of misjudgment due to dynamic environmental changes.

[0102] Example 6

[0103] Please refer to Figure 1Specifically: S34: pre-set a confidence range. If the dynamic confidence C of each sub-area exceeds the confidence range, it indicates that most of the positions in the corresponding sub-area are judged to be static, and the movement amplitude is small, and the risk is low. At this time, the corresponding sub-area in the updated environment map is green; if the dynamic confidence C of each sub-area falls within the confidence range, it indicates that some positions in the corresponding sub-area are judged to be static, and the movement amplitude is relatively small, and the risk is medium. At this time, the corresponding sub-area in the updated environment map is orange; if the dynamic confidence C of each sub-area does not exceed the confidence threshold and does not fall within the confidence range, it indicates that there are many moving points in the corresponding sub-area and the risk is high. At this time, the corresponding sub-area in the updated environment map is red;

[0104] This map can be color-coded internally by risk level (e.g. green, orange, red areas), reflecting the dynamic risk and feasible obstacle avoidance paths in each area.

[0105] S35: According to the current position of the underwater robot in the updated environment map and in combination with the sub-area displayed in red in S34, an obstacle avoidance path selection is provided for the underwater robot.

[0106] S41: Rendering the corresponding sub-areas in the updated environment map in color according to S34, and executing the return priority scheduling mechanism. The specific execution steps are as follows:

[0107] S411: extracting the sub-areas in red in the updated environment map and uploading them first;

[0108] S412: Extract the orange sub-areas and green sub-areas in the updated environmental map, compress the data of the orange sub-areas and the green sub-areas first, and then upload them. The orange sub-areas are uploaded after the red sub-areas, and the green sub-areas are uploaded after the orange sub-areas. This priority scheduling allows for gradual uploading in batches, regions, and priorities even if the entire map is large, rather than uploading all the data at once.

[0109] Because underwater communication systems typically have very limited bandwidth, directly transmitting large amounts of high-resolution data can easily fill up bandwidth resources, leading to delays and packet loss. Therefore, appropriate data compression is necessary. At critical moments, such as when encountering high-risk areas, the system requires rapid access to complete information to support decision-making. However, transmitting all details is not necessary at other times, so compressing data helps improve transmission efficiency and response speed.

[0110] The backhaul priority scheduling mechanism is a process of classifying the generated data by priority and arranging the transmission order. This mechanism dynamically allocates upload tasks under limited bandwidth resources based on the importance and real-time requirements of different data.

[0111] In this embodiment, the updated environmental map is first color-coded using a preset confidence range. This color coding intuitively reflects the dynamic risk of each sub-area, allowing the underwater robot to prioritize green and orange areas and avoid red high-risk areas when planning obstacle avoidance paths. For example, if a sub-area in the environmental map obtained by the robot during a seabed inspection is displayed as red, this area indicates frequent dynamic obstacle activity and a high risk. In step S35, the robot will avoid this area as a route and select green or orange areas as safe passages.

[0112] At the same time, in S41, the system executes a data backhaul scheduling mechanism based on the map color information: first, detailed sensor data from the red area is uploaded first to ensure that complete data is obtained in high-risk areas to support obstacle avoidance decisions; while data from the orange and green areas is uploaded after data compression, with the orange area uploaded before the green area, thereby ensuring the timely transmission of key decision-making data while effectively saving communication bandwidth and processing resources. This hierarchical upload strategy significantly reduces data redundancy and system load, and improves overall communication efficiency and obstacle avoidance safety.

[0113] Example 7

[0114] Please refer to Figure 4 ,Specifically: the underwater robot inspection and obstacle avoidance control system includes a ,graph building module, a graph updating module, a presentation module and a ,feedback module;

[0115] The graph construction module is used to use several sets of sensors and event cameras to monitor the environmental conditions of the underwater scene in real time to generate a two-dimensional event response distribution map. Based on the two-dimensional event response distribution map, the fused strong feature vector is determined. , and identify areas of abnormal dynamic changes;

[0116] The graph update module is used to pre-acquire the environment map constructed at the previous moment, and combine it with the two-dimensional event response distribution map to determine several groups of point pairs. The positional relationship, semantic relationship and dynamic change between each point pair are analyzed to construct the global objective function. , after optimization, the optimal state variables are obtained , based on the optimal state variables , update the environment map constructed at the previous moment and generate an updated environment map;

[0117] The presentation module is used to determine the dynamic situation of each location in the updated environment map to generate a dynamic confidence C of each sub-area in the updated environment map to provide the underwater robot with obstacle avoidance path selection;

[0118] The feedback module is used to trigger the feedback priority scheduling mechanism based on the value of the dynamic confidence C of each sub-area.

[0119] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An underwater robot inspection and obstacle avoidance control method, characterized by: The following steps are included: S1: Use several sets of sensors and event cameras to monitor the environmental conditions of the underwater scene in real time to generate a two-dimensional event response distribution map. Based on the two-dimensional event response distribution map, determine the fused strong feature vector , and identify areas of abnormal dynamic changes; S2: Obtain the environment map constructed at the previous moment in advance, combine it with the two-dimensional event response distribution map, determine several groups of point pairs, analyze the position relationship, semantic relationship and dynamic change between each point pair, and obtain the geometric residuals respectively. , semantic residual and dynamic residuals , to construct the global objective function , after optimization, the optimal state variables are obtained , based on the optimal state variables , update the environment map constructed at the previous moment and generate an updated environment map; S2 includes S25, specifically, constructing the global objective function , to minimize the geometric residual by , semantic residual and dynamic residuals Perform weighted square sum so that the state variables to be optimized obtained by optimization are As the optimal state variable , through the optimal state variables , the current fused strong feature vector Projected into the coordinate system defined by the environment map constructed at the previous moment to achieve alignment, specifically: ;in, To convert the position of the kth semantic point cloud in the two-dimensional event response distribution map to the new position coordinates defined by the coordinate system of the environment map constructed at the previous moment, is the position of the kth semantic point cloud, where the state variable to be optimized It consists of a 3*3 rotation matrix and a 3*1 translation vector; S3: Determine the dynamic situation of each location in the updated environment map to generate a dynamic confidence C of each sub-area in the updated environment map to provide the underwater robot with obstacle avoidance path selection; S4: Based on the dynamic confidence C value of each sub-area, the backhaul priority scheduling mechanism is triggered.

2. The underwater robot inspection and obstacle avoidance control method according to claim 1, characterized in that: S11: Use an event camera to capture the events that occur when the brightness of each pixel in the underwater scene changes in advance to obtain the timestamp of the event. and pixel coordinates; S12: Based on the parameters obtained in S11, simulate the real dynamic change information in the underwater scene and consider noise interference to calculate the event signal intensity at different pixel coordinates , specifically: ； Where, is the event signal intensity at the pixel coordinate (x, y) at time t, where t is the current time point, is the time offset index of multiple events that occurred, is the time decay constant, is the position weight; S13: Traverse all pixel coordinates on the image plane involved in the event camera to form a A two-dimensional event response distribution graph where the values ​​are pixel intensities.

3. The underwater robot inspection and obstacle avoidance control method according to claim 2, characterized in that: S14: Use several groups of sensors to monitor the environmental conditions of the underwater scene in real time, and divide the two-dimensional event response distribution map into several groups of regions to generate local feature vectors h collected by different sensors, and fuse the local feature vectors h collected by each sensor to obtain a strong feature vector after fusion. ; S15: The fused strong feature vector Associated with the monitoring positions of the corresponding sensors to form a representation corresponding to each local area in the underwater scene to generate a semantic point cloud ; S16: Based on the event signal intensity at different pixel coordinates obtained in S12 , the threshold segmentation method is used to identify the dynamic target area to form a binary dynamic obstacle mask , dynamic obstacle mask based on binarization , identifying the abnormal dynamic change area.

4. The underwater robot inspection and obstacle avoidance control method according to claim 3, characterized in that: S21: Pre-acquire the environment map constructed at the previous moment, and use the nearest neighbor search algorithm to find the semantic point cloud in each local area. Establish a preliminary correspondence with the environment map constructed at the previous moment to obtain several groups of point pairs; S22: Pre-set state variables to be optimized , and according to each semantic point cloud in several groups of point pairs The position of the point and the corresponding matching point in the environment map constructed at the previous moment, and the geometric residual of each point pair is obtained , specifically: ;in, is the geometric residual of the kth semantic point cloud, is the position of the kth semantic point cloud, is the position that matches the position of the kth semantic point cloud in the environment map constructed at the previous moment; is the rigid transformation matrix to be optimized; S23: According to the point pairs determined in S21, extract the corresponding semantic point cloud in each point pair And the position that matches the position of the corresponding semantic point cloud in the environment map constructed at the previous moment, the corresponding fused strong feature vector , by analyzing the feature differences within each point pair to obtain the semantic residual , specifically: Where, is the semantic residual of the kth semantic point cloud, is the fused strong feature vector in the kth semantic point cloud, The fused strong feature vector corresponding to the position that matches the position of the corresponding semantic point cloud in the environment map constructed at the previous moment ; S24: Based on the abnormal dynamic change area identified in S16, determine the number of abnormal dynamic changes that occurred in the abnormal dynamic change area in the past, and obtain the dynamic residual of each point pair by considering the interference of the abnormal dynamic change area on the update of the environment map constructed at the previous moment , specifically: Where, is the dynamic residual of the kth semantic point cloud, is the dynamic weight, where ;in, is a dynamic obstacle mask based on the binary Calculate dynamic probability.

5. The underwater robot inspection and obstacle avoidance control method according to claim 4, characterized in that: S31: Based on the environment map constructed at the previous moment, the new position coordinates after the position of the kth semantic point cloud in the two-dimensional event response distribution map is converted to the coordinate system defined by the environment map constructed at the previous moment , get the updated environment map.

6. The underwater robot inspection and obstacle avoidance control method according to claim 5, characterized in that: S32: Divide the updated environment map into several groups of sub-areas; S33: In the underwater scene, in order to consider the dynamic degree of each location in the underwater scene, the dynamic trajectory is presented in the updated environment map. The updated environment map includes the dynamic confidence C of each sub-area, specifically: ； Where, is the dynamic confidence of the Ath sub-area in the updated environment map, is the number of static obstacle points in the Ath sub-area, is the semantic point cloud corresponding to the Ath sub-area The total number of positions, is the variance of the event signal intensity change rate at the a-th position in the corresponding sub-region, and a is the position number in the corresponding sub-region.

7. The underwater robot inspection and obstacle avoidance control method according to claim 6, characterized in that: S34: A confidence range is preset. If the dynamic confidence C of each sub-region exceeds the confidence range, the corresponding sub-region in the updated environment map is colored green. If the dynamic confidence C of each sub-region falls within the confidence range, the corresponding sub-region in the updated environment map is colored orange. If the dynamic confidence C of each sub-region does not exceed the confidence threshold and does not fall within the confidence range, the corresponding sub-region in the updated environment map is colored red. S35: According to the current position of the underwater robot in the updated environment map and in combination with the sub-area displayed in red in S34, an obstacle avoidance path selection is provided for the underwater robot.

8. The underwater robot inspection and obstacle avoidance control method according to claim 7, characterized in that: S41: Rendering the corresponding sub-areas in the updated environment map in color according to S34, and executing the return priority scheduling mechanism. The specific execution steps are as follows: S411: extracting the sub-areas in red in the updated environment map and uploading them first; S412: Extract the orange sub-area and the green sub-area in the updated environmental map, compress the data of the orange sub-area and the green sub-area, and then upload them. The orange sub-area is uploaded after the red sub-area, and the green sub-area is uploaded after the orange sub-area.

9. An underwater robot inspection and obstacle avoidance control system, used to implement the underwater robot inspection and obstacle avoidance control method according to any one of claims 1 to 8, characterized in that: It includes graph construction module, graph update module, presentation module and return module; The graph construction module is used to use several sets of sensors and event cameras to monitor the environmental conditions of the underwater scene in real time to generate a two-dimensional event response distribution map. Based on the two-dimensional event response distribution map, the fused strong feature vector is determined. , and identify areas of abnormal dynamic changes; The graph update module is used to pre-acquire the environment map constructed at the previous moment, and combine it with the two-dimensional event response distribution map to determine several groups of point pairs. The positional relationship, semantic relationship and dynamic change between each point pair are analyzed to construct the global objective function. , after optimization, the optimal state variables are obtained , based on the optimal state variables , update the environment map constructed at the previous moment and generate an updated environment map; The presentation module is used to determine the dynamic situation of each location in the updated environment map to generate a dynamic confidence C of each sub-area in the updated environment map to provide the underwater robot with obstacle avoidance path selection; The feedback module is used to trigger the feedback priority scheduling mechanism based on the value of the dynamic confidence C of each sub-area.

Images