A method for sensing and positioning in the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration

Abstract

The present invention discloses a method for sensing and positioning a bulk carrier tank cleaning robot in the cabin based on ship-engine collaboration. The method comprises: building a ship-engine collaborative sensing hardware system, including a blind-spot-free hardware sensing system on the cabin roof and an onboard in-cabin sensing system; using the blind-spot-free hardware sensing system, obtaining a blind-spot-free scene map of the bulk carrier in the cabin and global positioning information of the tank cleaning robot in the cabin through cabin roof multi-sensor calibration, target recognition, and mapping technology; using the onboard in-cabin sensing system, the tank cleaning robot obtains a high-precision local map of the cabin and local positioning information; and using cloud communication technology, integrating the cabin roof sensing and positioning information with the onboard sensing and positioning information, thereby improving the accuracy and robustness of the in-cabin sensing and positioning of the bulk carrier tank cleaning robot. The present invention overcomes the problems of the bulk carrier tank cleaning robot's limited sensing and positioning area and the insufficient ship-side sensing and positioning accuracy, and provides a foundation for collaborative tank cleaning operations between the tank cleaning robot and the ship unloader.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent ships, and in particular relates to a cabin sensing and positioning method for a bulk carrier tank cleaning robot based on ship-engine collaboration. Background Art

[0002] Bulk carriers are particularly important, accounting for approximately 50% of total seaborne cargo volume. In 2020 alone, the total weight of dry bulk cargoes such as coal, ore, and cement transshipped at major Chinese ports reached nearly 5 billion tons. During bulk carrier unloading, ships must occupy berths, and unloading efficiency directly impacts overall shipping efficiency. Bulk carrier unloading can be divided into two stages: first, the bulk cargo is unloaded via shore-based unloaders; second, when unloading is nearing completion and the unloaders are unable to continue, manual or human-operated tank cleaning machines must be deployed to perform the tank cleaning operations. Working within bulk carrier holds is dangerous and laborious, making automated tank cleaning a challenge. Perception and positioning are prerequisites for automated tank cleaning robots, which require information about their surroundings and their own status to carry out subsequent operations.

[0003] Research on autonomous tank cleaning robots is currently in its infancy. For example, the patent "Automatic Tank Cleaning System for Unmanned Pusher and Rake" describes how a pusher and rake perform actions within a tank after receiving instructions. The patents "Automated Tank Cleaning Method, Equipment, and Medium for Ship Cabins" and "Ship Unloader Positioning System" describe the use of flow machines and ship unloaders in automated tank cleaning but do not cover tank cleaning robot technology. The patent "Intelligent Tracked Tank Cleaning Machine and Tank Cleaning Method Thereof" describes the tank cleaning mechanism and its use. The patent "Path Planning Method for Unmanned Pusher and Rake for Ship Cabins" describes a method for planning a pusher and rake's path within a tank. Overall, research on tank cleaning robot technology for bulk carrier tank cleaning scenarios is severely insufficient, especially research on sensing and positioning technology for tank cleaning robots is virtually nonexistent. This severely restricts the application of autonomous tank cleaning robots in bulk carrier tank cleaning operations. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for sensing and positioning the interior of a bulk carrier tank cleaning robot based on ship-machine collaboration, which makes up for the problems of the limited sensing and positioning area of the bulk carrier tank cleaning robot itself and the insufficient sensing and positioning accuracy at the ship end, and provides a basis for the subsequent collaborative tank cleaning operations of the tank cleaning robot and the ship unloader.

[0005] To solve the above technical problems, the technical solution of the present invention is: a method for sensing and positioning the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration, comprising the following steps:

[0006] S1. Build a ship-engine collaborative perception hardware system, including at least: a cabin top non-blind-spot hardware perception system and an onboard cabin hardware perception system;

[0007] S2: Based on the blind-spot-free hardware perception system on the cabin roof, a blind-spot-free scene map of the bulk carrier inside the cabin and the global positioning information of the cabin cleaning robot are obtained through cabin roof multi-sensor calibration, target recognition and mapping technology;

[0008] S3. Based on the onboard cabin hardware perception system, the cabin cleaning robot obtains a high-precision local map and local positioning information inside the cabin;

[0009] S4. Based on cloud communication technology, the interactive integration of cabin top sensing and positioning information and airborne sensing and positioning information is realized to improve the cabin sensing and positioning accuracy and robustness of bulk carrier tank cleaning robots, and serve as the basis for the coordinated tank cleaning operations of tank cleaning robots and ship unloaders.

[0010] The specific settings of the hardware perception system with no blind spots on the cabin top in S1 are:

[0011] The blind-spot-free hardware perception system on the cabin top includes a cabin top, a plurality of cabin top sensors and a plurality of routers; wherein the plurality of cabin top sensors and the plurality of routers are arranged along the circumference of the cabin top.

[0012] The cabin roof sensors include at least visual sensors and lidars, and the number and layout of their settings are related to the observation coverage range; the router is a network router or an information wireless transmission device, and the number and layout of its settings are related to the network coverage range.

[0013] S1 also includes the construction of a cabin cleaning robot, which includes a frame and electronic control equipment, a cabin cleaning mechanism, a robot walking mechanism, a sensor bracket, an inertial navigation sensor, an ultrasonic sensor module, a visual sensor, a laser radar, and a gas detection sensor module; among them,

[0014] The frame serves as the supporting carrier of the cabin cleaning robot, the electronic control equipment is set inside the frame, the cabin cleaning mechanism is connected to the front end of the frame, the robot walking mechanism is set at the bottom or external side of the frame and the bottom of the robot walking mechanism exceeds the bottom of the frame, the ultrasonic sensor module is connected to the rear end of the frame, and the top of the frame is connected to the sensor bracket, and the visual sensor, lidar and gas detection sensor modules are respectively connected to the sensor bracket.

[0015] S2 is specifically:

[0016] S2.1. Calibrate the internal parameters of the cabin top sensors, and obtain sensor information for each cabin top sensor;

[0017] S2.2. Extract sensor features, perform feature extraction and segmentation on camera images or 3D point cloud information, and extract point cloud feature sets with depth information. Line Features Surface feature set in and are the i-th point feature, j-th line feature, and k-th surface feature in the n-th cabin top sensor, np, nl, and nP are the number of point, line, and surface features, respectively;

[0018] S2.3. Calibrate the external parameters of the cabin top sensor using a preset calibration strategy;

[0019] S2.4, according to the external parameters of each cabin top sensor, and Perform coordinate transformation to obtain the feature space position in the world coordinate system. This will provide a spliced cabin scene map without blind spots, and thus the distribution of bulk cargo will be obtained.

[0020] S2.5. Segment the mobile robot from the image or point cloud using a deep neural network and determine its rough global position in the scene map without blind spots in the cabin.

[0021] The preset calibration strategy in S2.3 is a two-layer "coarse-to-fine" calibration strategy, which is specifically as follows: the first-layer "coarse-to-fine" calibration strategy is to determine the coarse extrinsic parameters of the cabin roof sensor based on the rough installation position, and then use the optimization algorithm to solve the precise extrinsic parameters; the second-layer "coarse-to-fine" calibration strategy is to first construct a coarse loss function through line features {L} and surface features {PL} in the process of solving the precise extrinsic parameters, quickly obtain the initial value of the precise extrinsic parameters, and then obtain the precise extrinsic parameters of the cabin roof sensor by constructing a point feature loss function and graph optimization method.

[0022] S3 specifically:

[0023] S3.1. Construct a multi-sensor tightly coupled odometry for the cabin-clearing robot. Use orb corner points from continuously acquired images to construct a visual odometry, which serves as the initial value for the odometry and provides the initial value for the lidar odometry. The laser point cloud segmentation method used by the lidar odometry is characterized by first providing the initial value for the visual odometry, then pre-processing and segmenting the point cloud to obtain the bulkhead plane information, and then constructing an optimization function based on the point features and the bulkhead plane information to obtain the pose conversion between the previous and next lidar data. The robot's pose node at the current moment is obtained from the previous lidar pose node and the current pose conversion result.

[0024] S3.2. Use the Chow-Liu tree-based pose graph marginalization algorithm to process the elimination groups related to the old subgraph into a sparse tree structure;

[0025] S3.3. Use the g2o optimization library to process all pose nodes in the lidar odometry to obtain the optimal pose, and rearrange them together with the point features and bulkhead plane features contained in the pose nodes to form an accurate local map of the cabin;

[0026] S3.4. Using the laser point cloud segmentation method described in S3.1, generate point features and cabin wall features from the real-time collected lidar point cloud data, and match the above features with the local map of the cabin to obtain the spatial position of the robot in the local map.

[0027] S4 is specifically:

[0028] S4.1. Utilize cloud communication technology to centrally aggregate the cabin cleaning robot's information regarding the blind-spot-free hardware sensing system on the cabin roof and the onboard in-cabin hardware sensing system. The cabin cleaning robot's information regarding the blind-spot-free hardware sensing system on the cabin roof includes a map of the bulk carrier's blind-spot-free scene and the cleaning robot's global positioning information within the cabin. The onboard in-cabin hardware sensing system information includes a high-precision local map of the cabin and local positioning information.

[0029] S4.2. The scene map without blind spots is integrated with the accurate local map;

[0030] S4.3. Based on the particle filter framework, the global positioning information and local positioning information are fused to perform global high-precision robust positioning of the robot in the cabin.

[0031] S4.2 specifically states:

[0032] S4.2.1. Using the global positioning information of the cabin-clearing robot in the cabin as the initial pose value, match the local map of the cabin to the scene map without blind spots;

[0033] S4.2.2. Find the precise relative position relationship between the local map and the scene map without blind spots based on the closest point search method;

[0034] S4.2.3. Use the local map to replace the local information of the scene map without blind spots to obtain a more accurate scene map without blind spots.

[0035] S4.3 specifically states:

[0036] S4.3.1. The global positioning information of the cabin-clearing robot in the cabin provides an initial search space for the robot's cabin positioning, and particles are randomly seeded in the initial search space to represent the robot's possible positions and states;

[0037] S4.3.2. Construct the state transition equation p(s) that integrates the cabin and airborne sensor information. k |s k-1 ,u k ), where s k is the posture state at the current moment, s k-1 is the posture state at the previous moment, u k is the odometer information input at the current moment; where u kThe value of is determined by two parts. On the one hand, it is determined by the difference in global positioning information u between the previous and next moments. tk As the mileage value; on the other hand, the robot's movement u obtained by the robot's own encoder or inertial sensor before and after the moment rk As the odometer value; according to the odometer calculation accuracy of the cabin top and airborne sensors, the weight factors w1 and w2 are set, where w1+w2=1, which is expressed as

[0038] u k =w1u tk +w2u rk

[0039] S4.3.3. Obtain a new particle set from the state transition equation to represent the current state space of the robot;

[0040] S4.3.4, construct the in-cabin observation model p(o k |s k ,M), calculate the weight of each particle, where o k is the observation quantity at the current moment, M is the accurate cabin map without blind spots, s k is the particle’s posture state;

[0041] S4.3.5. Select the particle with the highest weight as the precise position of the robot, and then resample the particle set;

[0042] S4.3.6. Repeat S4.3.2 to S4.3.5 to continuously track the robot's global position.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] This invention proposes a method for sensing and positioning a bulk carrier tank-clearing robot in-cabin based on ship-engine collaboration. The method comprises a blind-spot-free hardware sensing system on the cabin roof and an onboard in-cabin sensing system, laying the hardware foundation for ship-engine collaborative sensing and positioning. Specifically, a blind-spot-free hardware sensing system based on the cabin roof utilizes cabin roof multi-sensor calibration, target recognition, and mapping techniques to obtain a blind-spot-free map of the bulk carrier inside the cabin and the global positioning information of the tank-clearing robot within the cabin. A high-precision localized map and local positioning information are also obtained by the onboard in-cabin sensing system. Finally, cloud communication technology is used to interactively integrate the cabin roof sensing and positioning information with the onboard sensing and positioning information, improving the accuracy and robustness of the in-cabin sensing and positioning of the bulk carrier tank-clearing robot. This invention is the first to provide a complete sensing and positioning system for a ship-engine collaborative tank-clearing robot. This system addresses the limited sensing and positioning area of the bulk carrier tank-clearing robot itself and the insufficient ship-side sensing and positioning accuracy, and provides a foundation for subsequent collaborative tank-clearing operations between the tank-clearing robot and the ship unloader. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention;

[0046] Figure 2 Schematic diagram of the ship-engine collaborative sensing hardware system in an embodiment of the present invention;

[0047] Figure 3 Schematic diagram of the structure of the tank cleaning robot in an embodiment of the present invention;

[0048] In the figure, 1-bulkhead; 2-cabin roof; 3-cabin bottom; 4-residual bulk cargo; 5-cabin cleaning robot; 6-cabin roof sensor; 7-router; 8-rack and electronic control equipment; 9-cabin cleaning mechanism; 10-robot walking mechanism; 11-sensor bracket; 12-inertial navigation sensor; 13-ultrasonic sensor module; 14-camera; 15-lidar; 16-gas detection sensor module. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0050] like Figure 1-3 As shown, the cabin perception method and system of the bulk carrier tank cleaning robot 5 based on ship-engine collaboration includes the following steps:

[0051] S1. Build a ship-machine collaborative perception hardware system, including two blind-spot-free hardware perception systems on the cabin roof and an onboard cabin hardware perception system.

[0052] S2. Based on the two blind-spot-free hardware perception systems on the cabin roof, the cabin roof two multi-sensor calibration, target recognition and mapping technology are used to obtain a blind-spot-free scene map of the bulk carrier in the cabin and the global positioning information of the cleaning robot 5 in the cabin.

[0053] S3. Based on the onboard cabin hardware perception system, the cabin cleaning robot 5 obtains a high-precision cabin local map and local positioning information.

[0054] S4. Based on cloud communication technology, the interactive fusion of the cabin top 2 perception and positioning information and the airborne perception and positioning information is realized to improve the perception and positioning accuracy and robustness of the bulk carrier tank cleaning robot 5 in the cabin, and provide a basis for the subsequent coordinated tank cleaning operations of the tank cleaning robot 5 and the ship unloader.

[0055] S1 is to build a ship-engine collaborative perception hardware system, including two blind-spot hardware perception systems on the cabin roof and an onboard cabin hardware perception system. The specific steps include the following:

[0056] S1.1. Build two blind-spot-free hardware sensing systems on the cabin roof. The two blind-spot-free hardware sensing systems on the cabin roof consist of the cabin roof 2, four cabin roof sensors 6, and four routers 7.

[0057] S1.1.1, the cabin top sensor 6 and the router 7 are arranged in a ring around the cabin top 2.

[0058] S1.1.2. The cabin top sensor 6 is a sensor or sensor combination such as a camera 14 or a lidar 15 that can observe the topographic features of the cabin bottom 3. The number and layout of the cabin top sensors 6 are related to the observation coverage range. The observation range of all the cabin top sensors must be sufficient to cover the entire cabin bottom 3.

[0059] S1.1.3. Router 7 is a network router or device capable of wireless information transmission. The number and layout of routers 7 are related to the network coverage, and should ensure that the cleaning robot 5 can transmit data with devices outside the cabin or on the cabin roof 2 through the routers 7.

[0060] S1.2. Build the typical structure and sensor configuration of the cleaning robot 5. The typical structure and sensor configuration of the cleaning robot 5 includes a frame and electronic control equipment 8, a cleaning mechanism 9, a robot travel mechanism 10, a sensor bracket 11, an inertial navigation sensor 12, an ultrasonic sensor module 13, a camera 14, a lidar 15, and a gas detection sensor module 16.

[0061] S1.2.1. The frame and electronic control equipment 8 are the main structure of the cabin cleaning robot 5 and are responsible for basic functions such as sensor data collection and processing, motor control, decision-making, and energy supply.

[0062] S1.2.2. The cleaning mechanism 9 can be divided into four types, namely, a bulldozer, a bucket, a rolling brush and a negative pressure adsorption mechanism, which are respectively used for cleaning heavy bulk cargo (iron ore, coal, etc.), light bulk cargo (grain), small bulk cargo (a small amount of bulk cargo remaining before the end of cleaning) and light bulk cargo (dust residues, etc., which are small in size and light in weight).

[0063] S1.2.3. The robot walking mechanism 10 can be divided into wheeled and tracked types. The wheeled type is used for bulk cargo clearing scenarios where the robot is not easy to sink, and the tracked type is used for bulk cargo clearing scenarios where the robot is easy to sink.

[0064] S1.2.4, inertial navigation sensor 12, ultrasonic sensor module 13, camera 14, lidar 15, gas detection sensor module 16, and other sensors can be selected based on actual perception and positioning requirements. The inertial navigation sensor 12 and ultrasonic sensor module 13 are primarily used in the robot's safety module to prevent rollover and avoid obstacles. The camera 14 and lidar 15 are used for the robot's environmental perception and positioning. The gas detection sensor module 16 can be used to monitor the gas status within the cabin to prevent poisoning and casualties after exiting the cabin.

[0065] S2. Using the two blind-spot-free hardware sensing systems on the cabin roof, cabin roof sensors 6 are used to perform blind-spot-free sensing within the cabin. This allows for the distribution of bulk cargo within the cabin without blind spots, providing cargo distribution information for the cabin cleaning robot 5 or ship unloader's operation planning. Furthermore, the cabin roof sensors 6 are used to estimate the global position of the cabin cleaning robot 5 within the cabin.

[0066] S2.1. Calibrate the internal parameters of the cabin top sensor 6. The cabin top sensor 6 obtains sensor information. If the cabin top sensor 6 is a camera 14, the Zhang Zhengyou calibration algorithm is used to obtain the camera 14 internal parameters. If it is a lidar 15, no internal parameter calibration is required.

[0067] S2.2. Extract sensor features, perform feature extraction and segmentation on camera 14 images or 3D point cloud information, and extract point cloud feature sets with depth information. Line Features Surface feature set in and are the i-th point feature, j-th line feature, and k-th surface feature of the n-th cabin top sensor 6. np, nl, and nP are the number of point, line, and surface features, respectively.

[0068] S2.3. Calibrate the external parameters of the cabin top sensor 6, using a two-layer "coarse to fine" calibration strategy.

[0069] S2.3.1. The first level, “from coarse to fine”, is to determine the rough external parameters of the cabin top sensor 6 from the rough installation position, and then use the optimization algorithm to solve the precise external parameters;

[0070] S2.3.2. The second layer, “from coarse to fine”, means that in the process of solving the precise extrinsic parameters, a rough loss function is first constructed through the line features {L} and the surface features {PL} to quickly obtain the initial value of the precise extrinsic parameters, and then the precise extrinsic parameters of the cabin roof sensor 6 are obtained by constructing the point feature loss function and the graph optimization method.

[0071] S2.4, according to the external parameters of each cabin top sensor 6, and Perform coordinate transformation to obtain the feature space position in the world coordinate system, and thus obtain a spliced cabin scene map without blind spots, that is, the distribution status of bulk cargo.

[0072] S2.5. Use a deep neural network such as YOLO or rangNet++ to segment the mobile robot from the image or point cloud and determine its rough global position in the scene map without blind spots in the cabin.

[0073] S3. Using the onboard cabin hardware perception system, the cabin-clearing robot 5 obtains a high-precision local map of the cabin and local positioning information. The robot constructs the local map using simultaneous localization and mapping technology based on graph optimization. It also determines the robot's position within the local map by matching the local map with its own sensor information.

[0074] S3.1. Construct a multi-sensor tightly coupled odometer for the cleaning robot 5. First, use the orb corner points in the continuous camera 14 images to construct the camera 14 odometer, and use this as the initial value of the odometer to provide an initial value for the laser radar 15 odometer.

[0075] S3.1.1. The characteristics of the laser radar 15 odometry are that, first, its initial value is provided by the visual odometry, second, the point cloud is preprocessed and segmented to obtain the plane information of the bulkhead 1, and then an optimization function is constructed based on the point features and the plane information of the bulkhead 1 to obtain the posture conversion between the front and rear laser radar 15 data, and the posture node of the robot at the current moment is obtained from the laser radar 15 posture node at the previous moment and the current posture conversion result.

[0076] S3.2. To quickly update the in-cabin map information, a pose graph marginalization algorithm based on the Chow-Liu tree is used to process the elimination clusters related to the old subgraph into a sparse tree structure, avoiding the introduction of new edges between all elimination cluster variable pairs. This will improve the sparsity of the pose graph and enhance the real-time performance of the in-cabin local map update.

[0077] S3.3. Use the g2o optimization library to process all pose nodes in the lidar 15 odometry to obtain the optimal pose, and rearrange them together with the point features and bulkhead 1 plane features contained in the pose nodes to form an accurate local map of the cabin.

[0078] S3.4. Use a lidar 15 point cloud segmentation method similar to that in S3.1.1 to generate point features and cabin wall 1 features from the real-time collected lidar 15 point cloud data, and match the above features with the local map of the cabin to obtain the spatial position of the robot in the local map.

[0079] S4. Based on cloud communication technology, the interactive fusion of the cabin roof 2 perception positioning information and the airborne perception positioning information is realized. First, the cloud communication technology is used to summarize the blind spot-free hardware perception system information of the cabin roof and the airborne cabin hardware perception system information. Secondly, the blind spot-free scene map is integrated with the precise local map to accurately describe the state of bulk cargo in the cabin. Then, the global positioning information and the local positioning information are integrated based on the particle filter framework to realize the global high-precision robust positioning of the robot in the cabin.

[0080] S4.1. Utilize cloud communication technology to aggregate the information from the two blind-spot-free hardware sensing systems on the cabin roof (a map of the bulk carrier's blind-spot-free scene and the global positioning information of the cleaning robot 5 within the cabin) and the information from the onboard cabin hardware sensing system (a high-precision local cabin map and local positioning information) on the cleaning robot 5.

[0081] S4.2. The scene map without blind spots is integrated with the accurate local map;

[0082] S4.2.1. Using the global positioning information of the cleaning robot 5 in the cabin as the initial pose value, match the local map of the cabin to the scene map without blind spots;

[0083] S4.2.2. Find the precise relative position relationship between the local map and the scene map without blind spots based on the closest point search method;

[0084] S4.2.3. Use the local map to replace the local information of the scene map without blind spots to obtain a more accurate scene map without blind spots.

[0085] S4.3. Based on the particle filter framework, the global positioning information and local positioning information are integrated to achieve the global high-precision robust positioning of the robot in the cabin.

[0086] S4.3.1. The global positioning information of the cleaning robot 5 in the cabin provides an initial search space for the robot's cabin positioning, and particles are randomly seeded in the initial search space to represent the robot's possible positions and states;

[0087] S4.3.2. Construct the state transition equation p(s) that integrates the cabin and airborne sensor information. k |s k-1 ,u k ), where s k is the posture state at the current moment, s k-1 is the posture state at the previous moment, u k The odometer information entered at the current moment.

[0088] S4.3.2.1、u k The value of is determined by two parts. On the one hand, it is determined by the difference in global positioning information u between the cabin top 2 at the previous and next moments. tkAs the mileage value; on the other hand, the robot's movement u obtained by the robot's own encoder or inertial sensor before and after the moment rk As the mileage value.

[0089] S4.3.2.2. Set weight factors w1 and w2 based on the odometer calculation accuracy of the cabin roof 2 and the onboard sensor, where w1+w2=1, expressed as

[0090] u k =w1u tk +w2u rk

[0091] S4.3.3. The new particle set obtained from the state transfer equation represents the current state space of the robot.

[0092] S4.3.4, construct the in-cabin observation model p(o k |s k ,M), calculate the weight of each particle, where o k is the observation quantity at the current moment, M is the accurate cabin map without blind spots, s k is the particle's posture state.

[0093] S4.3.5. Select the particle with the highest weight as the precise position of the robot, and then resample the particle set;

[0094] S4.3.6. Repeat S4.3.2 to S4.3.5 to achieve continuous tracking of the robot's global position.

[0095] In general, the present invention discloses a method for sensing and positioning the interior of a bulk carrier tank cleaning robot 5 based on ship-engine collaboration. First, a ship-engine collaborative sensing hardware system is constructed, including two blind-spot-free hardware sensing systems on the cabin roof and an onboard in-cabin hardware sensing system. Secondly, based on the two blind-spot-free hardware sensing systems on the cabin roof, the cabin roof 2 multi-sensor calibration, target recognition and mapping technology are used to obtain a blind-spot-free scene map of the bulk carrier in the cabin and the global positioning information of the tank cleaning robot 5 in the cabin. Then, based on the onboard in-cabin hardware sensing system, the tank cleaning robot 5 obtains a high-precision in-cabin local map and local positioning information. Finally, based on cloud communication technology, the interactive fusion of the cabin roof 2 sensing and positioning information and the onboard sensing and positioning information is realized, thereby improving the in-cabin sensing and positioning accuracy and robustness of the bulk carrier tank cleaning robot 5, and providing a basis for the subsequent collaborative cleaning operations of the tank cleaning robot 5 and the ship unloader.

[0096] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for sensing and positioning in the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration, characterized in that: The following steps are involved: S1. Build a ship-engine collaborative perception hardware system, including at least: a cabin top non-blind-spot hardware perception system and an onboard cabin hardware perception system; S2: Based on the blind-spot-free hardware perception system on the cabin roof, a blind-spot-free scene map of the bulk carrier inside the cabin and the global positioning information of the cabin cleaning robot are obtained through cabin roof multi-sensor calibration, target recognition and mapping technology; S3. Based on the onboard cabin hardware perception system, the cabin cleaning robot obtains a high-precision local map and local positioning information inside the cabin; S4. Based on cloud communication technology, the interactive integration of cabin top sensing and positioning information and airborne sensing and positioning information is realized to improve the cabin sensing and positioning accuracy and robustness of bulk carrier tank cleaning robots, and serve as the basis for the coordinated tank cleaning operations of tank cleaning robots and ship unloaders.

2. The method for sensing and positioning in the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration according to claim 1 is characterized in that: The specific settings of the hardware perception system with no blind spots on the cabin top in S1 are: The blind-spot-free hardware perception system on the cabin top includes a cabin top, a plurality of cabin top sensors and a plurality of routers; wherein the plurality of cabin top sensors and the plurality of routers are arranged along the circumference of the cabin top.

3. The method for sensing and positioning inside the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration according to claim 2 is characterized in that: The cabin roof sensors include at least visual sensors and lidars, and the number and layout of their settings are related to the observation coverage range; the router is a network router or an information wireless transmission device, and the number and layout of its settings are related to the network coverage range.

4. The method for sensing and positioning in the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration according to claim 1 is characterized in that: S1 also includes the construction of a cabin cleaning robot, which includes a frame and electronic control equipment, a cabin cleaning mechanism, a robot walking mechanism, a sensor bracket, an inertial navigation sensor, an ultrasonic sensor module, a visual sensor, a laser radar, and a gas detection sensor module; among them, The frame serves as the supporting carrier of the cabin cleaning robot, the electronic control equipment is set inside the frame, the cabin cleaning mechanism is connected to the front end of the frame, the robot walking mechanism is set at the bottom or external side of the frame and the bottom of the robot walking mechanism exceeds the bottom of the frame, the ultrasonic sensor module is connected to the rear end of the frame, and the top of the frame is connected to the sensor bracket, and the visual sensor, lidar and gas detection sensor modules are respectively connected to the sensor bracket.

5. The method for sensing and positioning inside the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration according to claim 1 is characterized in that: S2 is specifically: S2.

1. Calibrate the internal parameters of the cabin top sensors, and obtain sensor information for each cabin top sensor; S2.

2. Extract sensor features, perform feature extraction and segmentation on camera images or 3D point cloud information, and extract point cloud feature sets with depth information. Line Features Surface feature set in and are the i-th point feature, j-th line feature, and k-th surface feature in the n-th cabin top sensor, np, nl, and nP are the number of point, line, and surface features, respectively; S2.

3. Calibrate the external parameters of the cabin top sensor using a preset calibration strategy; S2.4, according to the external parameters of each cabin top sensor, and Perform coordinate transformation to obtain the feature space position in the world coordinate system. This will provide a spliced cabin scene map without blind spots, and thus the distribution of bulk cargo will be obtained. S2.

5. Segment the mobile robot from the image or point cloud using a deep neural network and determine its rough global position in the scene map without blind spots in the cabin.

6. The method for sensing and positioning in the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration according to claim 5 is characterized in that: The preset calibration strategy in S2.3 is a two-layer "coarse-to-fine" calibration strategy. Specifically, the first layer of the "coarse-to-fine" calibration strategy is to determine the rough extrinsic parameters of the cabin top sensor based on the rough installation position, and then use the optimization algorithm to solve the precise extrinsic parameters; the second layer of the "coarse-to-fine" calibration strategy is to first construct a rough loss function through the line features {L} and the surface features {PL} in the process of solving the precise extrinsic parameters, Quickly obtain the initial value of the accurate extrinsic parameters, and then obtain the accurate extrinsic parameters of the cabin top sensor by constructing a point feature loss function and graph optimization method.

7. The method for sensing and positioning in the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration according to claim 4 is characterized in that: S3 specifically: S3.

1. Construct a multi-sensor tightly coupled odometry for the cabin-clearing robot. Use orb corner points from continuously acquired images to construct a visual odometry, which serves as the initial value for the odometry and provides the initial value for the lidar odometry. The laser point cloud segmentation method used by the lidar odometry is characterized by first providing the initial value for the visual odometry, then pre-processing and segmenting the point cloud to obtain the bulkhead plane information, and then constructing an optimization function based on the point features and the bulkhead plane information to obtain the pose conversion between the previous and next lidar data. The robot's pose node at the current moment is obtained from the previous lidar pose node and the current pose conversion result. S3.

2. Use the Chow-Liu tree-based pose graph marginalization algorithm to process the elimination groups related to the old subgraph into a sparse tree structure; S3.

3. Use the g2o optimization library to process all pose nodes in the lidar odometry to obtain the optimal pose, and rearrange them together with the point features and bulkhead plane features contained in the pose nodes to form an accurate local map of the cabin; S3.

4. Using the laser point cloud segmentation method described in S3.1, generate point features and cabin wall features from the real-time collected lidar point cloud data, and match the above features with the local map of the cabin to obtain the spatial position of the robot in the local map.

8. The method for sensing and positioning in the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration according to claim 1 is characterized in that: S4 is specifically: S4.

1. Utilize cloud communication technology to centrally aggregate the cabin cleaning robot's information regarding the blind-spot-free hardware sensing system on the cabin roof and the onboard in-cabin hardware sensing system. The cabin cleaning robot's information regarding the blind-spot-free hardware sensing system on the cabin roof includes a map of the bulk carrier's blind-spot-free scene and the cleaning robot's global positioning information within the cabin. The onboard in-cabin hardware sensing system information includes a high-precision local map of the cabin and local positioning information. S4.

2. The scene map without blind spots is integrated with the accurate local map; S4.

3. Based on the particle filter framework, the global positioning information and local positioning information are fused to perform global high-precision robust positioning of the robot in the cabin.

9. The method for sensing and positioning in the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration according to claim 8 is characterized in that: S4.2 specifically states: S4.2.

1. Using the global positioning information of the cabin-clearing robot in the cabin as the initial pose value, match the local map of the cabin to the scene map without blind spots; S4.2.

2. Find the precise relative position relationship between the local map and the scene map without blind spots based on the closest point search method; S4.2.

3. Use the local map to replace the local information of the scene map without blind spots to obtain a more accurate scene map without blind spots.

10. The method for sensing and positioning in the cabin of a bulk carrier tank cleaning robot based on ship-engine collaboration according to claim 8, characterized in that: S4.3 specifically states: S4.3.

1. The global positioning information of the cabin-clearing robot in the cabin provides an initial search space for the robot's cabin positioning, and particles are randomly seeded in the initial search space to represent the robot's possible positions and states; S4.3.

2. Construct the state transition equation p(s) that integrates the cabin and airborne sensor information. k |s k-1 ,u k ), where s k is the posture state at the current moment, s k-1 is the posture state at the previous moment, u k is the odometer information input at the current moment; where u k The value of is determined by two parts. On the one hand, it is determined by the difference in global positioning information u between the previous and next moments. tk As the mileage value; on the other hand, the robot's movement u obtained by the robot's own encoder or inertial sensor before and after the moment rk As the odometer value; set the weight factors w1 and w2 according to the odometer calculation accuracy of the cabin roof and airborne sensors, where w1+w2=1, expressed as u k =w1u tk +w2u rk S4.3.

3. Obtain a new particle set from the state transition equation to represent the current state space of the robot; S4.3.4, construct the in-cabin observation model p(o k |s k ,M), calculate the weight of each particle, where o k is the observation quantity at the current moment, M is the accurate cabin map without blind spots, s k is the particle’s posture state; S4.3.

5. Select the particle with the highest weight as the precise position of the robot, and then resample the particle set; S4.3.

6. Repeat S4.3.2 to S4.3.5 to continuously track the robot's global position.

Images