A marine inspection map optimization method and system based on robot collaborative perception

Abstract

The application discloses a marine inspection map optimization method and system based on robot collaborative perception, and belongs to the technical field of marine inspection. The method is as follows: data collection and fusion of a target area are simultaneously performed through the head and trunk of a robot to obtain multi-modal data and an initial map of the target area; a map missing degree of the initial map is identified and data supplementary collection is performed to construct a complete map of the target area; interference sources of a marine environment are identified based on the multi-modal data and disturbance intensity of each interference source is calculated to obtain a credibility weight of the multi-modal data; and the contribution weight of each multi-modal data in the map construction process is adjusted according to the credibility weight to reconstruct the complete map and obtain a final inspection map. Therefore, through implementation of the application, the problem that marine data collection in the prior art is easily affected by interference sources, resulting in large errors in the data and incomplete collection of part of the data and poor mapping accuracy can be solved.

Description

Technical Field

[0001] This application belongs to the field of marine inspection technology, specifically involving a method and system for optimizing marine inspection maps based on robot collaborative perception. Background Technology

[0002] Robots are commonly used for the inspection and maintenance of offshore infrastructure. With the rapid development of robotics technology, offshore inspection tasks place higher demands on the adaptability of robots in complex environments. Especially in special environments such as offshore platforms, traditional perception and mapping technologies face many challenges in inspection path planning due to interference from factors such as water mist, steam, gas leaks, and metal reflections.

[0003] Existing SLAM (Simultaneous Localization and Mapping) technology is a core technology for robots to synchronously estimate their pose and build environmental maps in unknown environments using sensor data, providing complete inspection maps for maritime patrol missions. However, this technology is susceptible to various interference sources (such as water mist, steam, and strong reflections) when collecting data in complex maritime scenarios, leading to significant errors in the collected data. Furthermore, some areas are obscured by obstacles during data collection, resulting in missing areas in the constructed map and affecting its accuracy. Summary of the Invention

[0004] This application proposes a method and system for optimizing maritime inspection maps based on robot collaborative perception. It can solve the problem that existing maritime data collection is easily affected by interference sources, resulting in large errors in the data and incomplete data collection, leading to poor mapping accuracy.

[0005] The first aspect of this application provides a method for optimizing maritime inspection maps based on robot cooperative perception, the method comprising: The robot simultaneously collects and fuses data from the target area using its head and torso to obtain multimodal data and an initial map of the target area; wherein, the multimodal data includes head visual images, multi-source environmental data, and robot pose data; The map missing degree of the initial map is identified by multimodal data, and data supplementation is carried out based on the map missing degree to construct a complete map of the target area; Based on multimodal data, the sources of interference in the marine environment are identified, and the credibility weights of the multimodal data are obtained by calculating the disturbance intensity of each source. Based on the aforementioned credibility weights, the contribution weights of each multimodal data in the map construction process are adjusted, and the complete map is reconstructed based on the adjustment results to obtain the final inspection map of the target area.

[0006] The aforementioned scheme utilizes the robot's head and torso to collaboratively acquire and fuse multimodal data, fully leveraging the complementary perspectives of sensors from different parts of the robot to overcome the limitations of single-sensor information and obtain richer multimodal data. By employing multimodal data to identify map gaps and proactively triggering supplementary data acquisition based on the degree of loss, compared to traditional fixed-path mapping methods, it can specifically fill in missing map information areas, significantly improving map integrity and usability. Various interference sources in the marine environment are identified and their intensity quantified, and the contribution weight of each modal data in map construction is dynamically adjusted accordingly. This effectively suppresses the negative impact of interference data on the mapping results, ensuring the final inspection map maintains high reliability even in harsh marine environments. Finally, the complete map is reconstructed based on the aforementioned reliability weights, completing the dynamic optimization of the map construction process, reducing data acquisition errors caused by interference sources, and improving mapping accuracy.

[0007] In one possible implementation of the first aspect, data is simultaneously collected and fused from the target area using the robot's head and torso to obtain multimodal data and an initial map of the target area, specifically as follows: A visual sensor deployed on the robot's head is used to acquire images of the target area where the robot is located, thus obtaining a visual image of the head. Various sensors deployed on the robot's head and torso are used to simultaneously collect environmental data and robot posture data of the target area, thereby obtaining multi-source environmental data and robot pose data. The multi-source environmental data includes three-dimensional point cloud and infrared temperature data of the target area; the robot pose data includes joint rotation angles, torso posture and gait phase information. The initial map is constructed by fusing the head visual image, the multi-source environmental data, and the robot pose data.

[0008] The aforementioned scheme utilizes sensors located on the head and torso to work collaboratively, leveraging the robot's flexible observation capabilities in complex terrain and enabling data acquisition from a superior perspective. It completes the acquisition of visual images, 3D point clouds, infrared temperature, and the robot's own pose data in a single operation, avoiding the spatiotemporal misalignment issues caused by time-division acquisition and laying a data foundation for subsequent high-precision fusion mapping. Additionally, it acquires joint rotation angles, torso posture, and gait phase information, accurately describing the robot's instantaneous pose changes during movement and providing key parameter support for eliminating motion distortion and improving point cloud registration accuracy.

[0009] In one possible implementation of the first aspect, data fusion is performed on the head visual image, the multi-source environmental data, and the robot pose data to construct multimodal data and the initial map, specifically as follows: The robot pose data is converted into motion state parameters of the robot in the torso laser point cloud coordinate system; The head visual image and other data collected by the robot's head sensors are subjected to coordinate transformation, and the coordinate transformation result is offset corrected using the motion state parameters to obtain head modal data in the torso laser point cloud coordinate system. By combining the 3D point cloud and the head modal data, the spatial grid positions and point cloud occupancy rates of each object and path in the target area are identified to obtain the initial map.

[0010] The aforementioned scheme converts robot pose data into motion state parameters in the torso laser point cloud coordinate system, and uses these parameters to correct the offset of head sensor data, effectively compensating for acquisition deviations caused by robot walking and torso swaying, and significantly improving the geometric accuracy of the map. By unifying head visual images and other head sensor data into the torso laser point cloud coordinate system through coordinate transformation, the inconsistency in data space caused by different sensor installation positions and orientations is resolved, providing a prerequisite for joint processing of multimodal data within the same framework. Combining 3D point cloud and head modal data to identify the spatial grid positions and point cloud occupancy of objects and paths ensures that the initial map contains not only geometric structure information but also semantic attributes such as visual texture and temperature, facilitating subsequent region identification.

[0011] In one possible implementation of the first aspect, the map missing degree of the initial map is identified through multimodal data, and data supplementation is performed based on the map missing degree to construct a complete map of the target area, specifically as follows: Based on the 3D point cloud of the target area provided by the multi-source environmental data, determine the sparse point cloud region in the initial map; Visual features are extracted from the head visual image to identify occluded shadow areas, metallic reflection areas, densely populated equipment areas, and areas with missing features in the initial map; The map missingness of the initial map is calculated by combining the sparse point cloud regions, the occlusion shadow regions, the metallic reflection regions, the dense equipment regions, and the feature missing regions. When the map missingness exceeds a first threshold, the robot head is controlled to collect supplementary data on the target area. The supplementary data collection results are used to fill in the initial map to obtain the complete map.

[0012] The above-mentioned solution comprehensively assesses the incompleteness of maps by considering five typical defects: sparse point clouds, occlusion and shadows, metallic reflections, dense equipment, and missing features. This covers the core factors that lead to incomplete maps in the marine environment and avoids the one-sidedness of judging by a single indicator. By setting a missingness threshold to trigger supplementary data collection, the robot has a closed-loop capability of actively compensating for deficiencies, which greatly improves the fault tolerance and adaptability of map construction.

[0013] In one possible implementation of the first aspect, the map missing degree of the initial map is calculated by combining the sparse point cloud regions, the occlusion shadow regions, the metallic reflection regions, the dense equipment regions, and the feature missing regions, specifically as follows: Based on the initial map, the 3D point cloud, and the head visual image, the point cloud sparsity of the sparse point cloud region, the spatial proportion of the occluded shadow region, the metallic reflection region, and the dense equipment region, as well as the feature missing rate of the feature missing region, are obtained respectively. The point cloud sparsity, spatial proportion, and feature missing rate are weighted and calculated to obtain the corresponding map missing degree.

[0014] The above scheme incorporates heterogeneous indicators such as point cloud sparsity, spatial proportion of occlusion / reflection / dense equipment areas, and feature missing rate into a unified weighted calculation framework, quantifying abstract map quality issues into specific and comparable numerical indicators, and providing a clear basis for decision-making.

[0015] In one possible implementation of the first aspect, interference sources in the marine environment are identified based on multimodal data. The confidence weights corresponding to the multimodal data are obtained by calculating the disturbance intensity of each interference source. Specifically: Multimodal data is input into a preset marine environmental disturbance identification model to identify the disturbance sources, and the disturbance intensity corresponding to heat source, water mist, reflection and point cloud distortion is calculated respectively. When the disturbance intensity is greater than a preset disturbance threshold, a corresponding disturbance label is generated for the interference source; Based on the mapping relationship between the interference source and the multimodal data, the confidence weight of each multimodal data is set through the disturbance label; wherein, the mapping relationship is used to reflect the sensitivity of different modal data acquisition processes to the interference source.

[0016] The aforementioned scheme utilizes a marine environmental disturbance identification model to specifically identify four typical marine disturbance sources: heat sources, water mist, reflections, and point cloud distortion. This enables the system to perceive specific marine conditions. The disturbance intensity of each source is calculated, and a disturbance threshold is set. Labels are generated only for disturbance sources exceeding the threshold, avoiding overreaction to weak disturbances and achieving reasonable allocation of computing resources. Based on the sensitivity mapping relationship between disturbance sources and various modal data, reliability weights are dynamically set. This "downweighting" of severely disturbed modal data and "upweighting" slightly disturbed or undisturbed modal data suppresses the impact of disturbances at the data source, resulting in more reasonable and reliable marine inspection maps.

[0017] In one possible implementation of the first aspect, the specific formula for calculating the disturbance intensity of the water mist is as follows: ; in, The intensity of the water mist disturbance. H The height of the head visual image. W The width of the head visual image. For pixels ( i , j The corresponding atmospheric scattering characteristic value; The specific formula for calculating the intensity of reflective disturbance is as follows: ; in, The intensity of the reflection disturbance. This represents the number of pixels in the head visual image whose brightness values ​​exceed the brightness threshold.

[0018] One possible implementation of the first aspect further includes: optimizing the robot's pose and map building process when collecting data based on preset torso steady-state reference factors, head active compensation factors, disturbance confidence modulation factors, and body motion coupling factors.

[0019] The above-mentioned solution ensures the geometric stability and pose continuity of the robot by providing various factors, providing a stable framework for the data acquisition process, and can also be used to correct local map details, improve map integrity, and reduce the impact of interference sources.

[0020] In one possible implementation of the first aspect, the method further includes: adding a disturbance attachment layer to the final inspection map; the disturbance attachment layer records the type of interference source and the disturbance intensity at different locations on the final inspection map.

[0021] The added disturbance attachment layer in the above scheme can comprehensively reflect the environmental disturbance situation, mark the areas with severe disturbance, and provide guidance information for subsequent robot data collection and inspection.

[0022] The second aspect of this application provides a marine inspection map optimization system based on robot collaborative perception. The system includes: a data acquisition module, a missing area supplementation module, a data reliability calculation module, and a map optimization module. The data acquisition module is used to simultaneously acquire and fuse data from the target area through the robot's head and torso to obtain multimodal data and an initial map of the target area; wherein, the multimodal data includes head visual images, multi-source environmental data, and robot pose data; The missing area completion module is used to identify the map missing degree of the initial map through multimodal data, collect data to supplement the map based on the map missing degree, and construct a complete map of the target area; The data credibility calculation module is used to identify interference sources in the marine environment based on multimodal data. By calculating the disturbance intensity of each interference source, the credibility weight corresponding to the multimodal data is obtained. The map optimization module is used to adjust the contribution weight of each multimodal data in the map construction process according to the confidence weight, and reconstruct the complete map according to the adjustment result to obtain the final inspection map of the target area. Attached Figure Description

[0023] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram illustrating the specific process of a marine inspection map optimization method based on robot collaborative perception, provided in one embodiment of this application. Figure 2 This is a structural diagram of a marine inspection map optimization system based on robot collaborative perception, provided in one embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0026] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0027] First Embodiment The robot possesses strong flexibility and multimodal perception capabilities. Especially in dynamically changing environments, the coordinated perception of its head and torso can effectively enhance its perception capabilities in complex scenarios. Therefore, the robot's head-torso coordinated perception capability enables real-time acquisition of multimodal data, providing rich data for inspection map construction. During map construction, interference from water mist, metallic reflections, and heat sources in special environments such as offshore platforms is also considered. To address the impact of multiple disturbance sources, the reliability of the collected data is adjusted to improve mapping accuracy.

[0028] like Figure 1 As shown, to address the problem that existing technologies for maritime data acquisition are easily affected by interference sources, resulting in large errors in the data and incomplete data collection, leading to poor mapping accuracy, the first embodiment of this application provides a detailed flowchart of a maritime inspection map optimization method based on robot collaborative perception. This embodiment's maritime inspection map optimization method based on robot collaborative perception includes steps S1 to S4, detailed below: Step S1: Simultaneously collect and fuse data from the target area using the robot's head and torso to obtain multimodal data and an initial map of the target area.

[0029] First, based on the robot's head-to-body heterogeneous perception architecture, multimodal data such as head vision, infrared, gas sensing data, torso lidar data, robot joint angles, torso posture, and gait phase information are collected simultaneously.

[0030] The head-to-body heterogeneous perception architecture is an architecture design that integrates and coordinates different types of sensors on the robot's head and torso. It includes a vision sensor and an infrared sensor deployed in the head that move in sync with the neck joint's pitch and yaw motion, and a LiDAR deployed in the torso. The vision sensor is used to acquire images of environmental texture, device contours, and occlusion boundaries to obtain a head-view image; the infrared sensor is used to acquire temperature distribution and abnormal heat source areas. The LiDAR serves as the main steady-state spatial geometry reference for the entire robot, continuously acquiring 3D point clouds of the robot's surrounding area. Simultaneously, the head-to-body heterogeneous perception architecture also acquires the robot's pose.

[0031] In this embodiment, the visual sensor acquires images of the target area where the robot is located, obtaining a head visual image. An infrared sensor and a lidar are used to acquire environmental data and robot posture data of the target area, respectively obtaining multi-source environmental data and robot pose data.

[0032] The multi-source environmental data includes 3D point cloud and infrared temperature data of the target area; the robot pose data includes joint rotation angles, trunk posture, and gait phase information. The joint rotation angles reflect the motion state of the robot's joints and are crucial for determining the robot's limb position and posture, helping to understand the robot's movement patterns and interactions with the environment. The trunk posture describes the robot's trunk's orientation and position in space, reflecting information such as the robot's overall orientation and tilt. The gait phase information represents the stages of the robot's gait during walking, such as starting, stepping, and landing, and is of significant reference value for understanding the robot's movement rhythm and stability, as well as for environmental perception and mapping during walking.

[0033] The rotation angle of the head joints is expressed as follows: ; in, This refers to the rotation angle of the head's joints. The head tilt angle, This refers to the nose yaw angle.

[0034] Torso posture The specific expression is: ; in, For the torso rotation matrix, This is the torso translation vector.

[0035] By employing multiple sensors in a collaborative manner, the robot can comprehensively and accurately perceive the complex marine environment. The different sensors complement each other: visual and infrared sensors focus on local details, while lidar provides the overall spatial structure, offering a rich and reliable data foundation for subsequent mapping and localization, thus enhancing the robot's adaptability and reliability in complex marine environments.

[0036] The data obtained through sensors mentioned above are collectively referred to as multimodal data. Therefore, multimodal data includes head visual images, multi-source environmental data, and robot pose data.

[0037] Furthermore, the head visual image, the multi-source environmental data, and the robot pose data are fused to construct the initial map. First, the robot pose data is converted into motion state parameters of the robot in the torso laser point cloud coordinate system. Coordinate transformation is performed on the head visual image and other data collected by the robot's head sensors, and the motion state parameters are used to correct the offset of the coordinate transformation results, obtaining head modal data in the torso laser point cloud coordinate system. Finally, by combining the 3D point cloud and the head modal data, the spatial grid positions and point cloud occupancy rates of each object and path in the target area are identified to obtain the initial map.

[0038] Step S2: Identify the map missing degree of the initial map through multimodal data, collect supplementary data based on the map missing degree, and construct a complete map of the target area.

[0039] During the initial map construction process, due to limitations in sensor field of view, occlusion, and other reasons, some areas of the map may not have had their data fully collected, resulting in missing parts of the map. This application's embodiments address these missing map parts by collecting supplementary data to complete and improve the map information, thus obtaining a complete map.

[0040] Therefore, this application embodiment triggers data supplementation collection by setting a missing value threshold, enabling the robot to have a closed-loop capability of actively compensating for deficiencies, which greatly improves the fault tolerance and adaptability of map construction.

[0041] The regions prone to map gaps in this application include sparse point cloud areas, occluded shadow areas, metallic reflection areas, densely populated equipment areas, and areas lacking features. Therefore, these regions are identified and located using multimodal data, and the map gap degree of the initial map is determined by the proportion of these regions. When the map gap degree exceeds a first threshold, it indicates a mapping gap. The robot head is then controlled to collect supplementary data on these map gaps in the target area. The supplementary data collection results are used to fill in the initial map, resulting in a complete map.

[0042] First, based on the 3D point cloud of the target area provided by the multi-source environmental data, sparse areas of the point cloud in the initial map are determined. Simultaneously, visual features are extracted from the head visual image to identify occluded shadow areas, metallic reflection areas, densely populated equipment areas, and areas with missing features in the initial map. For example, if a large area of ​​black shadow exists in a certain region of the head visual image, it can be considered an occluded shadow area; if several consecutive frames of the acquired head visual image show partially missing areas, they are considered areas with missing features.

[0043] Then, by combining the sparse point cloud regions, the occlusion shadow regions, the metallic reflection regions, the dense equipment regions, and the feature-missing regions, the map missing degree of the initial map is calculated.

[0044] Specifically, based on the initial map, the 3D point cloud, and the head visual image, the point cloud sparsity of the sparse point cloud region, the spatial proportion of the occluded shadow region, the metallic reflection region, and the densely populated equipment region (spatial proportion in the initial map), and the feature missing rate of the feature missing region are obtained. The point cloud sparsity, the spatial proportion, and the feature missing rate are weighted according to preset weighting coefficients to obtain the corresponding map missing degree.

[0045] The formula for calculating the map missingness is: ; in, , , , These are all weighting coefficients, set according to the priority of different regions, and can be configured in some embodiments. , , , . For the point cloud sparsity, The spatial proportion of the occluded shadow area. The feature missing rate, The spatial proportion of reflective metal areas and areas with dense equipment.

[0046] When the map missing degree is greater than the first threshold When this occurs, a mapping gap is determined to exist. Optionally, in this embodiment, the first threshold is set to 0.25.

[0047] If there are gaps in the mapping, the robot's head is controlled to collect supplementary data on the target area. The supplementary data is then used to fill in the initial map, resulting in the complete map. The supplementary data collection process involves controlling the head to perform targeted micro-pitch, micro-yaw, or short pauses to perform gaze-filling actions to supplement the target viewpoint. Satisfy the following formula: ; By supplementing the view, the head sensor can form a high-resolution supplementary observation of the map missing areas of the target area (i.e., the sparse point cloud areas, the occluded shadow areas, the metallic reflection areas, the dense equipment areas, and the feature missing areas).

[0048] By employing the aforementioned data supplementation and collection strategies, weights can be determined based on the characteristics of different regions, key areas can be prioritized, mapping efficiency and accuracy can be improved, missing thresholds can be reasonably set, excessive supplementation can be avoided and resources can be wasted, and complete and accurate maps can be quickly constructed with limited resources to meet the requirements of maritime patrol for map real-time performance and completeness.

[0049] Step S3: Identify the sources of interference in the marine environment based on multimodal data, and obtain the confidence weights corresponding to the multimodal data by calculating the disturbance intensity of each source.

[0050] Because in the marine environment, natural factors such as wind, waves, and currents, as well as potential human interference, can affect a robot's perception and movement. These disturbances can cause anomalies or deviations in sensor data, affecting the accuracy of mapping and the robot's positioning precision. Therefore, this application provides a marine environmental disturbance identification model to analyze and process multimodal data collected by the robot, such as visual, infrared, and lidar data, to determine whether marine environmental disturbances exist in the data and to identify the type of disturbance.

[0051] First, multimodal data is input into a preset marine environmental disturbance identification model to identify the disturbance sources. In this embodiment, the categories of disturbance sources are limited to heat sources, water mist, reflections, and point cloud distortion; in other embodiments, gas leakage and low-texture degradation (referring to the phenomenon that images are difficult to extract visual features, have increased matching ambiguity, and fail geometric estimation due to a lack of high-frequency details, blurred edges, or uniform surfaces) may also be included. Then, the disturbance intensity corresponding to heat sources, water mist, reflections, and point cloud distortion is calculated respectively.

[0052] Specifically, based on infrared temperature data, local heat sources, abnormal temperature rise areas, and heat reflection areas are identified, and the disturbance intensity of the heat sources is calculated; based on the three-dimensional point cloud collected by the torso sensor, point cloud echo anomalies, sparse distortion, and structural reflection distortion are identified, and the disturbance intensity of point cloud distortion is calculated.

[0053] The specific formula for calculating the disturbance intensity of water mist is as follows: ; in, The intensity of the water mist disturbance. H The height of the head visual image. W The width of the head visual image. For pixels ( i , j The corresponding atmospheric scattering characteristic value.

[0054] The specific formula for calculating the intensity of reflective disturbance is as follows: ; in, The intensity of the reflection disturbance. This represents the number of pixels in the head visual image whose brightness values ​​exceed the brightness threshold.

[0055] The disturbance intensity is compared with a preset disturbance threshold. This threshold is statistically derived from measured data across multiple scenarios under typical operating conditions of offshore power platforms and is used to distinguish between normal operating conditions and abnormal disturbance ranges. A corresponding disturbance tag is generated only when the disturbance intensity exceeds the preset disturbance threshold. Therefore, disturbance tags for the four types of disturbance sources are not always generated; they are generated based on whether the disturbance intensity of each source exceeds the threshold.

[0056] The generated disturbance labels facilitate the classification and management of different disturbance conditions, enabling targeted measures to reduce the impact of disturbances on mapping and localization, and improving the stability and reliability of the robot in complex disturbance environments. Once a disturbance label is generated, it indicates that the interference source will cause disturbances to sensor data acquisition, requiring corresponding adjustments.

[0057] For example, the disturbance threshold for water mist is set at 1.5 times the average value of the dark passage when there is no steam or water mist in the cabin, based on the baseline. The disturbance threshold for reflection is determined based on the camera's dynamic range and the typical brightness of specular reflection from the metal surface, using the maximum brightness value of the image. As a perturbation threshold for reflectivity, it is used to quantify the degree of interference of strong reflectivity of metal equipment with visual observation.

[0058] Based on the mapping relationship between the interference source and the multimodal data, the confidence weight of each multimodal data is set through the perturbation label. The mapping relationship is used to reflect the sensitivity of different modal data acquisition processes to the interference source.

[0059] The credibility weight should satisfy the following formula: ; in, , where is the credibility weight; This is the disturbance attenuation coefficient (with a value of 0 to 1). The disturbance intensity of the disturbance label corresponding to the target area (the disturbance intensity ranges from 0 to 1).

[0060] The disturbance attenuation coefficient is related to the mapping relationship and is determined based on the sensitivity of different modal data to the interference source. For example, visual sensitivity to water mist / reflection is taken as... Laser point cloud is sensitive to point cloud attenuation. Infrared detection of heat source disturbance .

[0061] Step S4: Adjust the contribution weight of each multimodal data in the map construction process according to the confidence weight, and reconstruct the complete map according to the adjustment result to obtain the final inspection map of the target area.

[0062] The contribution weight of data collected by sensors such as vision, infrared, and laser in the SLAM pose estimation and map building process is dynamically adjusted according to the aforementioned confidence weight, so as to reduce the impact of unreliable observations in areas with strong disturbances on the accuracy of map building.

[0063] The adjustment method of the contribution weight includes: during the pose estimation process, scaling the data residuals of the corresponding modal data according to the confidence weight. The lower the confidence, the smaller the impact of the modal data on the pose increment solution; during the map construction process, attenuating the raster update probability, point cloud fusion weight or texture update weight of the corresponding modal data according to the confidence weight, so as to avoid modal data affected by water mist, reflection or point cloud attenuation from directly covering reliable map data.

[0064] For example, when the target area is obscured by water mist and the confidence weight of vision decreases from 1.00 to 0.35, while the confidence weight of laser remains at 0.90 and the confidence weight of infrared remains at 0.85, pose estimation preferentially uses laser point cloud and torso pose data.

[0065] Therefore, the confidence weights corresponding to the perturbation labels are essentially used to distinguish whether the data of each modality within the target area will cause mapping blind spots, local texture mismatches, or a decrease in geometric accuracy. Among them, water mist, steam, and strong reflections mainly affect visual observation, point cloud echo anomalies mainly affect laser geometric observation, and heat source anomalies mainly affect infrared observation.

[0066] Adjusting the map building process by using the aforementioned credibility weights can improve mapping accuracy, enabling robots to better adapt to complex and turbulent marine environments and ensuring the construction of accurate and reliable maps under various working conditions.

[0067] As an improvement to the above solution, this application embodiment also optimizes the robot's pose and map building process when collecting data based on preset torso steady-state reference factor, head active compensation factor, disturbance confidence modulation factor and body motion coupling factor, so as to improve the robot's geometric stability and pose continuity and collect more reliable multimodal data.

[0068] The trunk steady-state benchmark factor The head active compensation factor is used to ensure the robot's global geometric stability and pose stability. This is used to improve the resolution of the data supplementation and acquisition process, thereby correcting map details in areas with missing or occluded areas; the perturbation confidence modulation factor The body motion coupling factor is determined by the credibility weight. The accuracy of the robot's joint movements is calibrated to eliminate observational biases caused by head and torso movements.

[0069] The above factors can be used to construct a unified factor graph, the optimization objective of which is: ; in, X i For robots at all times i The position, L j The first in the map landmark set j Each element. This is the robot's head view.

[0070] Based on the optimization objective of the unified factor graph, reasonable values ​​are assigned to the aforementioned factors. Collaborative optimization of multiple factors, comprehensively considering various aspects, can significantly improve... The accuracy and robustness of the data enable the construction of more accurate and reliable 3D maps.

[0071] Furthermore, this embodiment of the application adds a disturbance attachment layer to the final output inspection map. The disturbance attachment layer records the type of interference source and its disturbance intensity at different locations on the final inspection map, as well as the disturbance duration and the observation reliability of the multimodal data.

[0072] The disturbance attachment layer can comprehensively reflect environmental disturbances, improve the accuracy and reliability of disturbance information, provide a basis for accurately distinguishing between instantaneous interference and continuous anomalies, and improve the inspection efficiency of robots based on the final inspection map.

[0073] Implementing the embodiments of this application has the following beneficial effects: This application embodiment utilizes the robot's head and torso to collaboratively acquire and fuse multimodal data, fully leveraging the complementary perspectives of sensors from different parts of the robot to overcome the limitations of single-sensor information and obtain richer multimodal data. Multimodal data is used to identify map gaps, and supplementary data acquisition is proactively triggered based on the degree of gaps. Compared to traditional fixed-path mapping methods, this approach can specifically fill in missing map information areas, significantly improving map integrity and usability. Various interference sources in the marine environment are identified and their intensity quantified. Based on this, the contribution weight of each modal data in map construction is dynamically adjusted, effectively suppressing the negative impact of interference data on the mapping results and ensuring the final inspection map maintains high reliability even in harsh marine environments. Finally, the complete map is reconstructed based on the aforementioned reliability weights, completing the dynamic optimization of the map construction process, reducing data acquisition errors caused by interference sources, and improving mapping accuracy.

[0074] Second Embodiment Furthermore, in order to implement the marine inspection map optimization system based on robot collaborative perception corresponding to the above method embodiments, and to achieve the corresponding functions and technical effects, Figure 2 A structural diagram of a maritime patrol map optimization system based on robot collaborative perception is provided. For ease of explanation, only the parts relevant to this embodiment are shown. The maritime patrol map optimization system based on robot collaborative perception provided in this application embodiment includes: The data acquisition module 201 is used to simultaneously acquire and fuse data from the target area through the robot's head and torso to obtain multimodal data and an initial map of the target area; wherein, the multimodal data includes head visual images, multi-source environmental data and robot pose data.

[0075] In this embodiment of the application, a vision sensor deployed on the robot's head is used to acquire images of the target area where the robot is located, thereby obtaining a head vision image; Various sensors deployed on the robot's head and torso are used to simultaneously collect environmental data and robot posture data of the target area, thereby obtaining multi-source environmental data and robot pose data. The multi-source environmental data includes three-dimensional point cloud and infrared temperature data of the target area; the robot pose data includes joint rotation angles, torso posture and gait phase information. The initial map is constructed by fusing the head visual image, the multi-source environmental data, and the robot pose data.

[0076] In some embodiments, the data acquisition module 201 specifically comprises: First, based on the robot's head-to-body heterogeneous perception architecture, multimodal data such as head vision, infrared, gas sensing data, torso lidar data, robot joint angles, torso posture, and gait phase information are collected simultaneously.

[0077] The head-to-body heterogeneous perception architecture is an architecture design that integrates and coordinates different types of sensors on the robot's head and torso. It includes a vision sensor and an infrared sensor deployed in the head that move in sync with the neck joint's pitch and yaw motion, and a LiDAR deployed in the torso. The vision sensor is used to acquire images of environmental texture, device contours, and occlusion boundaries to obtain a head-view image; the infrared sensor is used to acquire temperature distribution and abnormal heat source areas. The LiDAR serves as the main steady-state spatial geometry reference for the entire robot, continuously acquiring 3D point clouds of the robot's surrounding area. Simultaneously, the head-to-body heterogeneous perception architecture also acquires the robot's pose.

[0078] In this embodiment, the visual sensor acquires images of the target area where the robot is located, obtaining a head visual image. An infrared sensor and a lidar are used to acquire environmental data and robot posture data of the target area, respectively obtaining multi-source environmental data and robot pose data.

[0079] The multi-source environmental data includes 3D point cloud and infrared temperature data of the target area; the robot pose data includes joint rotation angles, trunk posture, and gait phase information. The joint rotation angles reflect the motion state of the robot's joints and are crucial for determining the robot's limb position and posture, helping to understand the robot's movement patterns and interactions with the environment. The trunk posture describes the robot's trunk's orientation and position in space, reflecting information such as the robot's overall orientation and tilt. The gait phase information represents the stages of the robot's gait during walking, such as starting, stepping, and landing, and is of significant reference value for understanding the robot's movement rhythm and stability, as well as for environmental perception and mapping during walking.

[0080] The rotation angle of the head joints is expressed as follows: ; in, This refers to the rotation angle of the head's joints. The head tilt angle, This refers to the nose yaw angle.

[0081] Torso posture The specific expression is: ; in, For the torso rotation matrix, This is the torso translation vector.

[0082] By employing multiple sensors in a collaborative manner, the robot can comprehensively and accurately perceive the complex marine environment. The different sensors complement each other: visual and infrared sensors focus on local details, while lidar provides the overall spatial structure, offering a rich and reliable data foundation for subsequent mapping and localization, thus enhancing the robot's adaptability and reliability in complex marine environments.

[0083] The data obtained through sensors mentioned above are collectively referred to as multimodal data. Therefore, multimodal data includes head visual images, multi-source environmental data, and robot pose data.

[0084] Furthermore, the head visual image, the multi-source environmental data, and the robot pose data are fused to construct the initial map. First, the robot pose data is converted into motion state parameters of the robot in the torso laser point cloud coordinate system. Coordinate transformation is performed on the head visual image and other data collected by the robot's head sensors, and the motion state parameters are used to correct the offset of the coordinate transformation results, obtaining head modal data in the torso laser point cloud coordinate system. Finally, by combining the 3D point cloud and the head modal data, the spatial grid positions and point cloud occupancy rates of each object and path in the target area are identified to obtain the initial map.

[0085] The missing area supplementation module 202 is used to identify the map missing degree of the initial map through multimodal data, collect supplementary data based on the map missing degree, and construct a complete map of the target area.

[0086] In this embodiment of the application, the sparse area of ​​the point cloud in the initial map is determined based on the three-dimensional point cloud of the target area provided by the multi-source environmental data. Visual features are extracted from the head visual image to identify occluded shadow areas, metallic reflection areas, densely populated equipment areas, and areas with missing features in the initial map; The map missingness of the initial map is calculated by combining the sparse point cloud regions, the occlusion shadow regions, the metallic reflection regions, the dense equipment regions, and the feature missing regions. When the map missingness exceeds a first threshold, the robot head is controlled to collect supplementary data on the target area. The supplementary data collection results are used to fill in the initial map to obtain the complete map.

[0087] In some embodiments, the missing region supplementation module 202 specifically comprises: During the initial map construction process, due to limitations in sensor field of view, occlusion, and other reasons, some areas of the map may not have had their data fully collected, resulting in missing parts of the map. This application's embodiments address these missing map parts by collecting supplementary data to complete and improve the map information, thus obtaining a complete map.

[0088] Therefore, this application embodiment triggers data supplementation collection by setting a missing value threshold, enabling the robot to have a closed-loop capability of actively compensating for deficiencies, which greatly improves the fault tolerance and adaptability of map construction.

[0089] The regions prone to map gaps in this application include sparse point cloud areas, occluded shadow areas, metallic reflection areas, densely populated equipment areas, and areas lacking features. Therefore, these regions are identified and located using multimodal data, and the map gap degree of the initial map is determined by the proportion of these regions. When the map gap degree exceeds a first threshold, it indicates a mapping gap. The robot head is then controlled to collect supplementary data on these map gaps in the target area. The supplementary data collection results are used to fill in the initial map, resulting in a complete map.

[0090] First, based on the 3D point cloud of the target area provided by the multi-source environmental data, sparse areas of the point cloud in the initial map are determined. Simultaneously, visual features are extracted from the head visual image to identify occluded shadow areas, metallic reflection areas, densely populated equipment areas, and areas with missing features in the initial map. For example, if a large area of ​​black shadow exists in a certain region of the head visual image, it can be considered an occluded shadow area; if several consecutive frames of the acquired head visual image show partially missing areas, they are considered areas with missing features.

[0091] Then, by combining the sparse point cloud regions, the occlusion shadow regions, the metallic reflection regions, the dense equipment regions, and the feature-missing regions, the map missing degree of the initial map is calculated.

[0092] Specifically, based on the initial map, the 3D point cloud, and the head visual image, the point cloud sparsity of the sparse point cloud region, the spatial proportion of the occluded shadow region, the metallic reflection region, and the densely populated equipment region (spatial proportion in the initial map), and the feature missing rate of the feature missing region are obtained. The point cloud sparsity, the spatial proportion, and the feature missing rate are weighted according to preset weighting coefficients to obtain the corresponding map missing degree.

[0093] The formula for calculating the map missingness is: ; in, , , , These are all weighting coefficients, set according to the priority of different regions, and can be configured in some embodiments. , , , . For the point cloud sparsity, The spatial proportion of the occluded shadow area. The feature missing rate, The spatial proportion of reflective metal areas and areas with dense equipment.

[0094] When the map missing degree is greater than the first threshold When this occurs, a mapping gap is determined to exist. Optionally, in this embodiment, the first threshold is set to 0.25.

[0095] If there are gaps in the mapping, the robot's head is controlled to collect supplementary data on the target area. The supplementary data is then used to fill in the initial map, resulting in the complete map. The supplementary data collection process involves controlling the head to perform targeted micro-pitch, micro-yaw, or short pauses to perform gaze-filling actions to supplement the target viewpoint. Satisfy the following formula: ; By supplementing the view, the head sensor can form a high-resolution supplementary observation of the map missing areas of the target area (i.e., the sparse point cloud areas, the occluded shadow areas, the metallic reflection areas, the dense equipment areas, and the feature missing areas).

[0096] By employing the aforementioned data supplementation and collection strategies, weights can be determined based on the characteristics of different regions, key areas can be prioritized, mapping efficiency and accuracy can be improved, missing thresholds can be reasonably set, excessive supplementation can be avoided and resources can be wasted, and complete and accurate maps can be quickly constructed with limited resources to meet the requirements of maritime patrol for map real-time performance and completeness.

[0097] The data credibility calculation module 203 is used to identify interference sources in the marine environment based on multimodal data. By calculating the disturbance intensity of each interference source, the credibility weight corresponding to the multimodal data is obtained.

[0098] In this embodiment of the application, multimodal data is input into a preset marine environmental disturbance identification model to identify the disturbance sources, and the disturbance intensity corresponding to heat source, water mist, reflection and point cloud distortion is calculated respectively. When the disturbance intensity is greater than a preset disturbance threshold, a corresponding disturbance label is generated for the interference source; Based on the mapping relationship between the interference source and the multimodal data, the confidence weight of each multimodal data is set through the disturbance label; wherein, the mapping relationship is used to reflect the sensitivity of different modal data acquisition processes to the interference source.

[0099] In some embodiments, the data credibility calculation module 203 specifically comprises: Because in the marine environment, natural factors such as wind, waves, and currents, as well as potential human interference, can affect a robot's perception and movement. These disturbances can cause anomalies or deviations in sensor data, affecting the accuracy of mapping and the robot's positioning precision. Therefore, this application provides a marine environmental disturbance identification model to analyze and process multimodal data collected by the robot, such as visual, infrared, and lidar data, to determine whether marine environmental disturbances exist in the data and to identify the type of disturbance.

[0100] First, multimodal data is input into a preset marine environmental disturbance identification model to identify the disturbance sources. In this embodiment, the categories of disturbance sources are limited to heat sources, water mist, reflections, and point cloud distortion; in other embodiments, gas leakage and low-texture degradation (referring to the phenomenon that images are difficult to extract visual features, have increased matching ambiguity, and fail geometric estimation due to a lack of high-frequency details, blurred edges, or uniform surfaces) may also be included. Then, the disturbance intensity corresponding to heat sources, water mist, reflections, and point cloud distortion is calculated respectively.

[0101] Specifically, based on infrared temperature data, local heat sources, abnormal temperature rise areas, and heat reflection areas are identified, and the disturbance intensity of the heat sources is calculated; based on the three-dimensional point cloud collected by the torso sensor, point cloud echo anomalies, sparse distortion, and structural reflection distortion are identified, and the disturbance intensity of point cloud distortion is calculated.

[0102] The specific formula for calculating the disturbance intensity of water mist is as follows: ; in, The intensity of the water mist disturbance. H The height of the head visual image. W The width of the head visual image. For pixels ( i , j The corresponding atmospheric scattering characteristic value.

[0103] The specific formula for calculating the intensity of reflective disturbance is as follows: ; in, The intensity of the reflection disturbance. This represents the number of pixels in the head visual image whose brightness values ​​exceed the brightness threshold.

[0104] The disturbance intensity is compared with a preset disturbance threshold. This threshold is statistically derived from measured data across multiple scenarios under typical operating conditions of offshore power platforms and is used to distinguish between normal operating conditions and abnormal disturbance ranges. A corresponding disturbance tag is generated only when the disturbance intensity exceeds the preset disturbance threshold. Therefore, disturbance tags for the four types of disturbance sources are not always generated; they are generated based on whether the disturbance intensity of each source exceeds the threshold.

[0105] The generated disturbance labels facilitate the classification and management of different disturbance conditions, enabling targeted measures to reduce the impact of disturbances on mapping and localization, and improving the stability and reliability of the robot in complex disturbance environments. Once a disturbance label is generated, it indicates that the interference source will cause disturbances to sensor data acquisition, requiring corresponding adjustments.

[0106] For example, the disturbance threshold for water mist is set at 1.5 times the average value of the dark passage when there is no steam or water mist in the cabin, based on the baseline. The disturbance threshold for reflection is determined based on the camera's dynamic range and the typical brightness of specular reflection from the metal surface, using the maximum brightness value of the image. As a perturbation threshold for reflectivity, it is used to quantify the degree of interference of strong reflectivity of metal equipment with visual observation.

[0107] Based on the mapping relationship between the interference source and the multimodal data, the confidence weight of each multimodal data is set through the perturbation label. The mapping relationship is used to reflect the sensitivity of different modal data acquisition processes to the interference source.

[0108] The credibility weight should satisfy the following formula: ; in, , where is the credibility weight; This is the disturbance attenuation coefficient (with a value of 0 to 1). The disturbance intensity of the disturbance label corresponding to the target area (the disturbance intensity ranges from 0 to 1).

[0109] The disturbance attenuation coefficient is related to the mapping relationship and is determined based on the sensitivity of different modal data to the interference source. For example, visual sensitivity to water mist / reflection is taken as... Laser point cloud is sensitive to point cloud attenuation. Infrared detection of heat source disturbance .

[0110] The map optimization module 204 is used to adjust the contribution weight of each multimodal data in the map construction process according to the credibility weight, and reconstruct the complete map according to the adjustment result to obtain the final inspection map of the target area.

[0111] In this embodiment of the application, the contribution weight of data collected by sensors such as vision, infrared and laser in the SLAM pose estimation and map building process is dynamically adjusted according to the confidence weight, so as to reduce the impact of unreliable observations in areas with strong disturbances on the accuracy of map building.

[0112] The adjustment method of the contribution weight includes: during the pose estimation process, scaling the data residuals of the corresponding modal data according to the confidence weight. The lower the confidence, the smaller the impact of the modal data on the pose increment solution; during the map construction process, attenuating the raster update probability, point cloud fusion weight or texture update weight of the corresponding modal data according to the confidence weight, so as to avoid modal data affected by water mist, reflection or point cloud attenuation from directly covering reliable map data.

[0113] For example, when the target area is obscured by water mist and the confidence weight of vision decreases from 1.00 to 0.35, while the confidence weight of laser remains at 0.90 and the confidence weight of infrared remains at 0.85, pose estimation preferentially uses laser point cloud and torso pose data.

[0114] Therefore, the confidence weights corresponding to the perturbation labels are essentially used to distinguish whether the data of each modality within the target area will cause mapping blind spots, local texture mismatches, or a decrease in geometric accuracy. Among them, water mist, steam, and strong reflections mainly affect visual observation, point cloud echo anomalies mainly affect laser geometric observation, and heat source anomalies mainly affect infrared observation.

[0115] Adjusting the map building process by using the aforementioned credibility weights can improve mapping accuracy, enabling robots to better adapt to complex and turbulent marine environments and ensuring the construction of accurate and reliable maps under various working conditions.

[0116] As an improvement to the above solution, this application embodiment also optimizes the robot's pose and map building process when collecting data based on preset torso steady-state reference factor, head active compensation factor, disturbance confidence modulation factor and body motion coupling factor, so as to improve the robot's geometric stability and pose continuity and collect more reliable multimodal data.

[0117] The trunk steady-state benchmark factor The head active compensation factor is used to ensure the robot's global geometric stability and pose stability. This is used to improve the resolution of the data supplementation and acquisition process, thereby correcting map details in areas with missing or occluded areas; the perturbation confidence modulation factor The body motion coupling factor is determined by the credibility weight. The accuracy of the robot's joint movements is calibrated to eliminate observational biases caused by head and torso movements.

[0118] The above factors can be used to construct a unified factor graph, the optimization objective of which is: ; in, X i For robots at all times i The position, L j The first in the map landmark set j Each element. This is the robot's head view.

[0119] Based on the optimization objective of the unified factor graph, reasonable values ​​are assigned to the aforementioned factors. Collaborative optimization of multiple factors, comprehensively considering various aspects, can significantly improve... The accuracy and robustness of the data enable the construction of more accurate and reliable 3D maps.

[0120] Furthermore, this embodiment of the application adds a disturbance attachment layer to the final output inspection map. The disturbance attachment layer records the type of interference source and its disturbance intensity at different locations on the final inspection map, as well as the disturbance duration and the observation reliability of the multimodal data.

[0121] The disturbance attachment layer can comprehensively reflect environmental disturbances, improve the accuracy and reliability of disturbance information, provide a basis for accurately distinguishing between instantaneous interference and continuous anomalies, and improve the inspection efficiency of robots based on the final inspection map.

[0122] Implementing the embodiments of this application has the following beneficial effects: This application embodiment utilizes the robot's head and torso to collaboratively acquire and fuse multimodal data, fully leveraging the complementary perspectives of sensors from different parts of the robot to overcome the limitations of single-sensor information and obtain richer multimodal data. Multimodal data is used to identify map gaps, and supplementary data acquisition is proactively triggered based on the degree of gaps. Compared to traditional fixed-path mapping methods, this approach can specifically fill in missing map information areas, significantly improving map integrity and usability. Various interference sources in the marine environment are identified and their intensity quantified. Based on this, the contribution weight of each modal data in map construction is dynamically adjusted, effectively suppressing the negative impact of interference data on the mapping results and ensuring the final inspection map maintains high reliability even in harsh marine environments. Finally, the complete map is reconstructed based on the aforementioned reliability weights, completing the dynamic optimization of the map construction process, reducing data acquisition errors caused by interference sources, and improving mapping accuracy.

[0123] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, or improvements made by those skilled in the art within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for optimizing maritime inspection maps based on robot collaborative perception, characterized in that, include: The robot simultaneously collects and fuses data from the target area using its head and torso to obtain multimodal data and an initial map of the target area; wherein, the multimodal data includes head visual images, multi-source environmental data, and robot pose data; The map missing degree of the initial map is identified by multimodal data, and data supplementation is carried out based on the map missing degree to construct a complete map of the target area; Based on multimodal data, the sources of interference in the marine environment are identified, and the credibility weights of the multimodal data are obtained by calculating the disturbance intensity of each source. Based on the aforementioned credibility weights, the contribution weights of each multimodal data in the map construction process are adjusted, and the complete map is reconstructed based on the adjustment results to obtain the final inspection map of the target area.

2. The method for optimizing maritime inspection maps based on robot collaborative perception according to claim 1, characterized in that, The robot simultaneously collects and fuses data from the target area using its head and torso to obtain multimodal data and an initial map of the target area, specifically: A visual sensor deployed on the robot's head is used to acquire images of the target area where the robot is located, thus obtaining a visual image of the head. Various sensors deployed on the robot's head and torso are used to simultaneously collect environmental data and robot posture data of the target area, thereby obtaining multi-source environmental data and robot pose data. The multi-source environmental data includes three-dimensional point cloud and infrared temperature data of the target area; the robot pose data includes joint rotation angles, torso posture and gait phase information. The initial map is constructed by fusing the head visual image, the multi-source environmental data, and the robot pose data.

3. The method for optimizing maritime inspection maps based on robot collaborative perception according to claim 2, characterized in that, The head visual image, the multi-source environmental data, and the robot pose data are fused to construct multimodal data and the initial map, specifically as follows: The robot pose data is converted into motion state parameters of the robot in the torso laser point cloud coordinate system; The head visual image and other data collected by the robot's head sensors are transformed into coordinates, and the motion state parameters are used to correct the offset of the coordinate transformation result to obtain head modal data in the torso laser point cloud coordinate system. By combining the 3D point cloud and the head modal data, the spatial grid positions and point cloud occupancy rates of each object and path in the target area are identified to obtain the initial map.

4. The method for optimizing maritime inspection maps based on robot collaborative perception according to claim 1, characterized in that, The map missingness of the initial map is identified using multimodal data. Based on the map missingness, data is collected to supplement the missing data and construct a complete map of the target area. Specifically: Based on the 3D point cloud of the target area provided by the multi-source environmental data, determine the sparse point cloud region in the initial map; Visual features are extracted from the head visual image to identify occluded shadow areas, metallic reflection areas, densely populated equipment areas, and areas with missing features in the initial map; The map missingness of the initial map is calculated by combining the sparse point cloud regions, the occlusion shadow regions, the metallic reflection regions, the dense equipment regions, and the feature missing regions. When the map missingness exceeds a first threshold, the robot head is controlled to collect supplementary data on the target area. The supplementary data collection results are used to fill in the initial map to obtain the complete map.

5. The method for optimizing maritime inspection maps based on robot collaborative perception according to claim 4, characterized in that, By combining the sparse point cloud regions, the occlusion and shadow regions, the metallic reflection regions, the dense equipment regions, and the feature-missing regions, the map missing degree of the initial map is calculated, specifically as follows: Based on the initial map, the 3D point cloud, and the head visual image, the point cloud sparsity of the sparse point cloud region, the spatial proportion of the occluded shadow region, the metallic reflection region, and the dense equipment region, as well as the feature missing rate of the feature missing region, are obtained respectively. The point cloud sparsity, spatial proportion, and feature missing rate are weighted and calculated to obtain the corresponding map missing degree.

6. The method for optimizing maritime inspection maps based on robot collaborative perception according to claim 1, characterized in that, Interference sources in the marine environment are identified based on multimodal data. By calculating the disturbance intensity of each interference source, the confidence weights corresponding to the multimodal data are obtained, specifically: Multimodal data is input into a preset marine environmental disturbance identification model to identify the disturbance sources, and the disturbance intensity corresponding to heat source, water mist, reflection and point cloud distortion is calculated respectively. When the disturbance intensity is greater than a preset disturbance threshold, a corresponding disturbance label is generated for the interference source; Based on the mapping relationship between the interference source and the multimodal data, the confidence weight of each multimodal data is set through the disturbance label; wherein, the mapping relationship is used to reflect the sensitivity of different modal data acquisition processes to the interference source.

7. The method for optimizing maritime inspection maps based on robot collaborative perception according to claim 6, characterized in that, The specific formula for calculating the disturbance intensity of water mist is as follows: ； in, The intensity of the water mist disturbance. H The height of the head visual image. W The width of the head visual image. For pixels ( i , j The corresponding atmospheric scattering characteristic value; The specific formula for calculating the intensity of reflective disturbance is as follows: ； in, The intensity of the reflection disturbance. This represents the number of pixels in the head visual image whose brightness values ​​exceed the brightness threshold.

8. The method for optimizing maritime inspection maps based on robot collaborative perception according to claim 1, characterized in that, Also includes: Based on preset torso steady-state reference factors, head active compensation factors, disturbance confidence modulation factors, and body motion coupling factors, the robot's pose and map building process during data acquisition are optimized.

9. The method for optimizing maritime inspection maps based on robot collaborative perception according to claim 1, characterized in that, Also includes: A disturbance attachment layer is added to the final inspection map; the disturbance attachment layer records the types of interference sources and their disturbance intensities at different locations on the final inspection map.

10. A maritime inspection map optimization system based on robot collaborative perception, characterized in that, include: Data acquisition module, missing area supplementation module, data reliability calculation module, and map optimization module; The data acquisition module is used to simultaneously acquire and fuse data from the target area through the robot's head and torso to obtain multimodal data and an initial map of the target area; wherein the multimodal data includes head visual images, multi-source environmental data, and robot pose data; The missing area completion module is used to identify the map missing degree of the initial map through multimodal data, collect data to supplement the map based on the map missing degree, and construct a complete map of the target area; The data credibility calculation module is used to identify interference sources in the marine environment based on multimodal data. By calculating the disturbance intensity of each interference source, the credibility weight corresponding to the multimodal data is obtained. The map optimization module is used to adjust the contribution weight of each multimodal data in the map construction process according to the credibility weight, and reconstruct the complete map according to the adjustment result to obtain the final inspection map of the target area.

Images