Ship hatch cover plate material cleaning robot system based on multi-sensor fusion and working method

Abstract

The invention relates to a ship hatch cover plate material cleaning robot system based on multi-sensor fusion and a working method. The robot system comprises a sensing assembly, a control assembly, an execution assembly and a communication assembly. The sensing assembly is used for acquiring environment information, and the control assembly performs fusion processing on data sensed by the sensing assembly based on a ubuntu system and can determine the position and posture of the robot; an optimal material clearing path is planned, an obstacle avoidance instruction is generated, and the control assembly sends the instruction to an STM32 single-chip microcomputer system in the execution assembly; the execution assembly comprises an STM32 single-chip microcomputer system, a crawler-type chassis and a sweeping and pushing integrated material cleaning mechanism, the STM32 single-chip microcomputer system converts a control signal into a driving signal, the crawler-type chassis is driven to achieve movement of the robot, and the sweeping and pushing integrated material cleaning mechanism is driven to complete material cleaning operation; and the communication assembly adopts a 5G communication module, so that the robot is connected with a cloud platform. According to the device, automation and intelligence of ship hatch cover plate material cleaning operation can be achieved, and the port operation efficiency and safety are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robots, in particular to a ship hatch cover cleaning robot system and working method based on multi-sensor fusion. Background Art

[0002] In port operations, the cleaning work of ship hatch covers is crucial for port operation efficiency and safety. The traditional manual cleaning method has problems of low efficiency and high danger. Especially in bad weather conditions, the difficulty and risk of manual cleaning are further increased. With the advancement of the construction of smart ports, the demand for port operation automation and intelligence is growing day by day. Developing a robot system capable of automatically cleaning ship hatch covers has become a key research direction for improving port operation levels.

[0003] At present, although many achievements have been made in the field of intelligent robots, the research on special robots for ship hatch cover cleaning is still in its infancy. In a complex ship cabin environment, existing robot technologies have obvious deficiencies in path planning, positioning accuracy, and adapting to special working conditions, and it is difficult to meet the actual operation requirements. For example, path planning in a complex environment is prone to falling into a local optimal solution, resulting in low cleaning efficiency; the positioning accuracy is greatly affected by environmental interference, and it is impossible to ensure the precise operation of the robot; the cleaning structure design is unreasonable and cannot effectively clean various materials on the hatch cover. Summary of the Invention

[0004] The purpose of the present invention is to provide a ship hatch cover cleaning robot system and working method based on multi-sensor fusion, which can be applicable to the automated cleaning operation of port ship hatch covers, realize the automation and intelligence of ship hatch cover cleaning operations, and improve port operation efficiency and safety.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A ship hatch cover cleaning robot system based on multi-sensor fusion, characterized in that: the robot system includes a sensing component, a control component, an execution component, and a communication component;

[0006] The sensing component is used to obtain environmental information, and the sensing group includes an L1 radar, multiple laser range sensors, a camera, an electronic compass, and a GPS;

[0007] The control component performs fusion processing on the data sensed by the sensing component based on the ubuntu system, and can determine the position and attitude of the robot; plan the optimal cleaning path and generate an obstacle avoidance instruction, and the control component sends the instruction to the STM32 single-chip microcomputer system in the execution component;

[0008] The execution component includes an STM32 single-chip microcomputer system, a crawler chassis, and a sweeping and pushing integrated material cleaning mechanism. The STM32 single-chip microcomputer system converts control signals into driving signals to drive the crawler chassis to realize the movement of the robot and drive the sweeping and pushing integrated material cleaning mechanism to complete the material cleaning operation;

[0009] The communication component uses a 5G communication module to connect the robot to the cloud platform.

[0010] Furthermore, the L1 radar is used to detect long-distance environmental information, the laser range sensor is responsible for precisely sensing nearby objects, the camera provides visual recognition, the electronic compass is used to determine the orientation of the robot, and the GPS obtains spatial coordinates. The data collected by these sensors are transmitted to the Jetson orin nano in the control component.

[0011] Furthermore, the sweeping and pushing integrated material cleaning mechanism includes a cleaning component and a material pushing component. The cleaning component uses high-strength bristles. The material pushing component includes a mounting rod, a mounting block, a connecting piece, and an arc-shaped push plate. One end of the mounting rod is connected to the crawler chassis, and the other end of the mounting rod is fixed with the mounting block. The mounting block is connected with the arc-shaped push plate through a connecting piece. Both ends of the arc-shaped push plate are provided with L-shaped mounting plates, and the high-strength bristles are fixed on the L-shaped mounting plates. A push rod mounting bracket is fixed under the mounting block, and the push rod is fixed on the push rod mounting bracket. The power end of the push rod is fixed at the rear end of the arc-shaped push plate.

[0012] A method for a ship hatch cover material cleaning robot system based on multi-sensor fusion, characterized in that it includes the following steps:

[0013] Step S1: Use lidar and camera for joint positioning to accurately determine the position and attitude of the robot in the world coordinate system;

[0014] Step S2: Use ultrasonic sensors and lidar to cooperate in perceiving the environment, and construct a high-precision grid map based on multi-source data fusion;

[0015] [[ID=?]]Step S3: Use ultrasonic sensors to measure the distance to nearby obstacles through sound wave reflection;

[0016] Step S4: Use the Kalman filtering algorithm to fuse the data of two types of sensors, namely ultrasonic radar and lidar, effectively reducing the noise interference of a single sensor and realizing real-time detection of dynamic obstacles.

[0017] Further, in step S1, further: A lidar is used to obtain three-dimensional point cloud data of the environment by emitting laser beams. A camera on the ship unloader collects images of the ship's hatch cover area to form two-dimensional image data. The scale-invariant feature transform algorithm is used to extract feature points in the image. A matching algorithm based on ORB descriptors is used to find the corresponding relationship between the point cloud data after conversion by the lidar and the feature point data after conversion by the camera, and accurately determine the position and pose of the robot in the world coordinate system.

[0018] Further, in step S1, a lidar is used to obtain three-dimensional point cloud data of the environment by emitting laser beams, and a camera collects images of the ship's hatch cover area to form two-dimensional image data. Further, using the scale-invariant feature transform algorithm to extract feature points in the image is as follows: The lidar scans at a specific frequency, and each scan generates a large number of discrete distance data points. These points constitute the point cloud data, which is represented in matrix form as:

[0019]

[0020] where (x i , y i , z i ) represents the three-dimensional coordinates of the i-th point in the lidar coordinate system. The point cloud data obtained by the lidar is in its own coordinate system. To make the data consistent with the world coordinate system required for the robot's operation, coordinate transformation is needed; a transformation matrix T is constructed through the pre-acquired rotation matrix R and translation vector t l-w to achieve the transformation from the lidar coordinate system to the world coordinate system:

[0021]

[0022] The transformed point cloud data P W is:

[0023] P w = T l-w ·P lidar

[0024] In the positioning scenario of the material cleaning robot, the first frame scanned by the lidar after the robot is powered on is used as the initial frame, and its pose transformation is the identity transformation. At the same time, the point cloud obtained is used as the reference point cloud; then during the movement of the robot, the point cloud P current scanned by the current lidar is registered with the reference point cloud; in each iteration process, the ICP algorithm is divided into two main steps:

[0025] Step S11, corresponding point search: For each point in the transformed current point cloud P' current = T init ·P current in, in the reference point cloud Preference Find the point with the closest distance in it, and construct a set of corresponding point pairs \(C =\{(P', i , P j )\}, where \(P' i \in P' current p j \in P reference

[0026] Step S12. Transformation optimization: Based on the found set of corresponding point pairs \(C\), calculate a new transformation matrix \(T icp \) by minimizing the objective function \(E new :\) Solve for \(T\) through singular value decomposition new Update the transformation matrix; repeat the above two steps until the objective function converges to a smaller value. At this time, the obtained transformation matrix is the optimized lidar positioning transformation matrix, which can more accurately determine the position and orientation of the robot in the world coordinate system;

[0027] The camera on the ship unloader collects images of the ship's hatch cover area to form two-dimensional image data; use the scale-invariant feature transform algorithm to extract the feature points in the image to obtain a set of feature points \(F camera :\)

[0028] F camera =\{f 1, f_2,..., f n \}.

[0029] Furthermore, the finding of the corresponding relationship in the point cloud data after lidar conversion and the feature point data after camera conversion to accurately determine the position and orientation of the robot in the world coordinate system is further as follows: Find the corresponding relationship in the point cloud data \(P w \) after lidar conversion and the feature point data \(F w \) after camera conversion to obtain a set of matching point pairs \(M\), and use the least squares method to optimize the matching point pair error to construct an objective function \(E\):

[0030]

[0031] Minimize the objective function through iterative solution to obtain the optimal transformation matrix, and then accurately determine the position and orientation of the robot in the world coordinate system.

[0032] Furthermore, the step S4 is further as follows:

[0033] The fusion formula for the fused environmental information is as follows:

[0034] z k =K k \cdot z lidar +(1 - K k )zultrasonidar

[0035] where z k is the fused environmental information, K k is the Kalman gain coefficient, z lidar and z ultrasonidar are the measurement values of the ultrasonic radar and lidar respectively;

[0036] In global path planning, the heuristic function is optimized by combining the structural characteristics of the ship's hatch cover. The cost function of the traditional A* algorithm is:

[0037] f(n) = h(n) + g(n)

[0038] where g(n) is the actual cost from the starting point to the current node, and h(n) is the heuristic estimated cost from the current node to the target node. For the possible slopes and uneven areas on the surface of the hatch cover, a terrain adaptation factor α is introduced to adjust the heuristic function to preferentially select flat paths and reduce the energy consumption and movement risk of the robot:

[0039]

[0040] where: max(H) represents the maximum height difference of the entire hatch cover terrain, and the ratio of the two is used to quantify the relative undulation degree of the current node, normalize the height, and reflect the steepness of the node. h(n) is the original heuristic function, and h new (n) is multiplied by the terrain adaptation factor α, making the algorithm more sensitive to terrain undulations during path planning;

[0041] To deal with dynamic obstacles (such as residues of temporarily stacked goods), the dynamic window approach is used for local path adjustment. DWA generates candidate trajectories by evaluating the robot's velocity space (v, ω) and selects the optimal path based on the following evaluation function:

[0042] Score(v, w) = λ1·Headin(v, w) + λ2·Dist(v, w) + λ3·Velocity(v, w)

[0043] where Heading represents the alignment of the trajectory towards the target point, Dist is the distance to the nearest obstacle, Velocity is the weighted value of the current speed, and λ1, λ2, λ3 are weight coefficients. By integrating the global path and local obstacle avoidance strategies, the robot can achieve smooth and safe movement in a complex environment.

[0044] Further, the step S2 is further as follows: First, divide the hatch cover plate into grids of 0.1m×0.1m. In each grid, use the occupancy grid algorithm to convert the sensor measurement value into the occupancy probability of the grid cell. Each cell contains an occupancy probability value, which is used to characterize whether the area is occupied by an obstacle. The sensor measurement value is the data after the fusion of the lidar and ultrasonic radar through Kalman filtering.

[0045] The beneficial effects of the present invention are as follows: The robot obtains environmental information through the sensing component, determines its own position and attitude through the joint positioning technology; the control component uses the path planning method to plan the optimal path and adjusts the path in real time according to environmental changes; the execution component drives the tracked chassis to move according to the control instruction and performs the material cleaning operation through the sweeping and pushing integrated material cleaning mechanism; the communication component realizes the remote interaction between the robot and the operator to ensure the smooth and efficient progress of the entire material cleaning process. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a schematic diagram of the system of the present invention;

[0047] Figure 2 is a flowchart of the working method of the robot of the present invention;

[0048] Figure 3 is a schematic structural diagram of the sweeping and pushing integrated material cleaning mechanism of the present invention in the upward viewing direction;

[0049] Figure 4 is a schematic structural diagram of the sweeping and pushing integrated material cleaning mechanism of the present invention in the downward viewing direction.

[0050] Wherein: 1. Cleaning component, 2. Pushing component, 21. Mounting rod, 22. Mounting block, 23. Connecting piece, 24. Arc-shaped pushing plate, 25. Push rod mounting bracket, 26. Push rod, 27. L-shaped mounting plate, 3. Tracked chassis. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] The present invention will be further described below with reference to the accompanying drawings.

[0052] Please refer to Figure 1 , the present invention provides an embodiment: A ship hatch cover cleaning robot system based on multi-sensor fusion, characterized in that: the robot system includes a sensing component, a control component, an execution component and a communication component;

[0053] The sensing component is used to obtain environmental information. The sensing component includes an L1 radar, multiple laser range sensors, a camera, an electronic compass and a GPS; four laser range sensors can be provided.

[0054] The control component performs fusion processing on the data sensed by the sensing component based on the ubuntu system, and can determine the position and attitude of the robot; plan the optimal material cleaning path and generate obstacle avoidance instructions, and the control component sends the instructions to the STM32 single-chip microcomputer system in the execution component; the position and attitude of the robot are determined by using multi-sensor positioning technology, and the optimal material cleaning path is planned through path planning and obstacle avoidance algorithms, and obstacle avoidance instructions are generated.

[0055] The execution component includes an STM32 single-chip microcomputer system, a crawler chassis 3 and a sweeping and pushing integrated material cleaning mechanism. The STM32 single-chip microcomputer system converts the control signal into a driving signal to drive the crawler chassis 3 to realize the movement of the robot and drive the sweeping and pushing integrated material cleaning mechanism to complete the material cleaning operation; the sweeping and pushing integrated material cleaning mechanism adopts a special design and can adapt to various materials and shapes on the surface of the ship's hatch cover. This mechanism consists of a cleaning component and a pushing component 2. The cleaning component 1 uses high-strength bristles and can effectively clean the fine materials on the surface of the hatch cover; the pushing component 2 uses a push plate with a certain arc, and during the cleaning process, the materials are concentrated and pushed to a specified position for subsequent cleaning and collection.

[0056] The communication component uses a 5G communication module to connect the robot to the cloud platform. Operators can remotely monitor the working status of the robot through the cloud platform and remotely control it through the handle or upper computer programming when necessary.

[0057] During the operation process, the robot first obtains environmental information through the sensing component, the control component processes and analyzes the information, plans a reasonable path and generates control instructions, the execution component operates according to the instructions, and at the same time realizes the interaction with the operator through the communication component to ensure the smooth and efficient progress of the material cleaning operation. The system schematic diagram is as follows Figure 1 as shown.

[0058] Please continue to refer to Figure 1 as shown. In an embodiment of the present invention, the L1 radar is used to detect long-distance environmental information, the laser ranging sensor is responsible for accurately sensing close-range objects, the camera provides visual recognition, the electronic compass is used to determine the orientation of the robot, and the GPS obtains spatial coordinates. The data collected by these sensors are transmitted to the Jetson orin nano in the control component. Jetson orin nano is an edge AI

[0059] embedded device.

[0060] Please refer to Figure 1 、 Figure 3 、 Figure 4As shown in the figure, in an embodiment of the present invention, the sweeping and pushing integrated material cleaning mechanism includes a cleaning component 1 and a material pushing component 2. The material pushing component 2 includes a mounting rod 21, a mounting block 22, a connecting piece 23, and an arc-shaped pushing plate 24. One end of the mounting rod 21 is connected to the crawler chassis 3, and the other end of the mounting rod 21 is fixed with the mounting block 22. The mounting block 22 is connected with the arc-shaped pushing plate 24 through the connecting piece 23. Both ends of the arc-shaped pushing plate 24 are provided with L-shaped mounting plates 27, and the high-strength bristles are fixed on the L-shaped mounting plates 27. A push rod mounting bracket 25 is fixed below the mounting block 22, and a push rod 26 is fixed on the push rod mounting bracket 25. The power end of the push rod 26 is fixed at the rear end of the arc-shaped pushing plate 24.

[0061] Please refer to Figure 2 , the present invention provides another embodiment: a method for a ship hatch cover material cleaning robot system based on multi-sensor fusion, characterized in that it includes the following steps:

[0062] Step S1: Use a lidar and a camera for joint positioning to accurately determine the position and attitude of the robot in the world coordinate system;

[0063] Step S2: Use ultrasonic sensors and lidar to cooperate in perceiving the environment, and construct a high-precision grid map based on multi-source data fusion; the grid map divides the surface of the hatch cover into grid cells of 0.1m×0.1m, and each cell contains an occupancy probability value, which is used to characterize whether the area is occupied by obstacles [8]. By updating the grid map in real time, the robot can dynamically perceive environmental changes and provide a reliable basis for path planning.

[0064] Step S3: Use ultrasonic sensors to measure the distance to nearby obstacles through sound wave reflection;

[0065] Step S4: Use the Kalman filter algorithm to fuse the data of these two types of sensors, namely ultrasonic radar and lidar, effectively reducing the noise interference of a single sensor and realizing the real-time detection of dynamic obstacles.

[0066] Please continue to refer to Figure 2As shown in the figure, in an embodiment of the present invention, in step S1, further: a lidar is used to obtain three-dimensional point cloud data of the environment by emitting laser beams, a camera on the ship unloader collects images of the ship's hatch cover area to form two-dimensional image data, scale-invariant feature transform algorithm is used to extract feature points in the image, and a matching algorithm based on ORB descriptors is used to find the corresponding relationship between the point cloud data after conversion by the lidar and the feature point data after conversion by the camera, so as to accurately determine the position and pose of the robot in the world coordinate system. In the working scenario of the ship's hatch cover cleaning robot, the ship unloader serves as a carrying platform, and the camera installed on it is responsible for collecting two-dimensional image data of the ship's hatch cover area, which cooperates with the lidar data to assist the robot in completing environmental perception and positioning. The ship unloader operates at the dock shore. When the ship docks at the dock, the cargo hold area where the hatch cover is located is the operation target area of the ship unloader, and the material handling device of the ship unloader will extend into the cargo hold after the hatch cover is opened for material loading and unloading.

[0067] Please continue to refer to Figure 2 As shown in the figure, in an embodiment of the present invention, in step S1, the lidar is used to obtain three-dimensional point cloud data of the environment by emitting laser beams, and the camera collects images of the ship's hatch cover area to form two-dimensional image data. Further, using the scale-invariant feature transform algorithm to extract feature points in the image is as follows: the lidar scans at a specific frequency, and each scan will generate a large number of discrete distance data points, and these points constitute the point cloud data, which is represented in matrix form as:

[0068]

[0069] where (x i , y i , z i ) represents the three-dimensional coordinates of the i-th point in the lidar coordinate system. The point cloud data obtained by the lidar is in its own coordinate system. To make the data consistent with the world coordinate system required for the robot's operation, coordinate transformation is needed; a transformation matrix T is constructed through the pre-obtained rotation matrix R and translation vector t l-w to achieve the conversion from the lidar coordinate system to the world coordinate system:

[0070]

[0071] The converted point cloud data P W is:

[0072] p w = T l-w · T lidar

[0073] In the positioning scenario of the blanking robot, the first frame scanned by the lidar after the robot is powered on is used as the initial frame, and its pose transformation is the identity transformation. At the same time, the point cloud obtained is used as the reference point cloud. After that, during the movement of the robot, the point cloud P obtained by scanning the current lidar current is registered with the reference point cloud; in each iteration process, the ICP algorithm is divided into two main steps:

[0074] Step S11, corresponding point search: For each point in the transformed current point cloud P' current = T init ·P currrent , find the point with the closest distance in the reference point cloud P reference to construct a set of corresponding point pairs C = {(P' i , P j )}, where P' i ∈P' current p j ∈P reference

[0075] Step S12, transformation optimization: Based on the found set of corresponding point pairs C, calculate a new transformation matrix T icp by minimizing the objective function E new : Solve for T through singular value decomposition new to update the transformation matrix; repeat the above two steps until the objective function converges to a smaller value. At this time, the obtained transformation matrix is the optimized lidar positioning transformation matrix, which can more accurately determine the position and orientation of the robot in the world coordinate system;

[0076] The camera on the ship unloader collects image data of the ship's hatch cover area to form two-dimensional image data; the scale-invariant feature transform algorithm is used to extract the feature points in the image to obtain a set of feature points F camera :

[0077] F camera = {f 1, f2,..., f n}.

[0078] Please continue to refer to Figure 2 As shown, in an embodiment of the present invention, the method for finding the corresponding relationship between the point cloud data after lidar conversion and the feature point data after camera conversion to accurately determine the position and orientation of the robot in the world coordinate system is further as follows: Find the corresponding relationship between the point cloud data P w after lidar conversion and the feature point data F w after camera conversion to obtain a set of matching point pairs M, and use the least squares method to optimize the matching point pair error to construct an objective function E:

[0079]

[0080] By iteratively solving to minimize the objective function, the optimal transformation matrix is obtained, and then the position and attitude of the robot in the world coordinate system are accurately determined.

[0081] Please continue to refer to Figure 2 As shown, in one embodiment of the present invention, in step S4, it is further:

[0082] The fusion formula of the fused environmental information is as follows:

[0083] z k = K k ·z lidar +(1 - K k )z ultrasonidar

[0084] where z k is the fused environmental information, K k is the Kalman gain coefficient, z lidar and z ultrasonidar are the measurement values of the ultrasonic radar and the lidar respectively;

[0085] In global path planning, the heuristic function is optimized by combining the structural characteristics of the ship's hatch cover. The cost function of the traditional A* algorithm is:

[0086] f(n)=h(n)+g(n)

[0087] where g(n) is the actual cost from the starting point to the current node, and h(n) is the heuristic estimated cost from the current node to the target node. For the possible slopes and uneven areas on the surface of the hatch cover, a terrain adaptation factor α is introduced to adjust the heuristic function to preferentially select flat paths and reduce the energy consumption and movement risk of the robot:

[0088]

[0089] where: max(H) represents the maximum height difference of the entire hatch cover terrain, and the ratio of the two is used to quantify the relative undulation degree of the current node, normalize the height, and reflect the steepness of the node. h(n) is the original heuristic function, and h new (n) is multiplied by the terrain adaptation factor α, making the algorithm more sensitive to terrain undulations during path planning;

[0090] To deal with dynamic obstacles (such as residues of temporarily stacked goods), the dynamic window approach is used for local path adjustment. DWA generates candidate trajectories by evaluating the robot's velocity space (v, ω) and selects the optimal path based on the following evaluation function:

[0091] Score(v, w) = λ1·Heading(v, w) + λ2·Dist(v, w) + λ3·Velocity(v, w)

[0092] Among them, Heading represents the alignment degree of the trajectory towards the target point, Dist is the distance from the nearest obstacle, Velocity is the weighted value of the current speed, and λ1, λ2, and λ3 are weight coefficients. By fusing the global path and the local obstacle avoidance strategy, the robot can achieve smooth and safe movement in a complex environment. Among them, the global path planning refers to the path planning of the robot to the location where the materials are cleaned up.

[0093] Please continue to refer to Figure 2 As shown, in an embodiment of the present invention, the step S2 is further as follows: First, the cabin cover plate is divided into grids of 0.1m × 0.1m. In each grid, the occupancy grid algorithm is used to convert the sensor measurement value into the occupancy probability of the grid cell. Each cell contains an occupancy probability value, which is used to characterize whether the area is occupied by an obstacle. The sensor measurement value is the data after the fusion of the lidar and the ultrasonic radar through Kalman filtering.

[0094] In summary, the robot obtains environmental information through the sensing component, determines its own position and posture through the joint positioning technology; the control component uses the path planning method to plan the optimal path and adjusts the path in real time according to environmental changes; the execution component drives the tracked chassis 3 to move according to the control instruction and performs the material cleaning operation through the sweeping and pushing integrated material cleaning mechanism; the communication component realizes the remote interaction between the robot and the operator to ensure the smooth and efficient progress of the entire material cleaning process.

[0095] The above are only the preferred embodiments of the present invention and should not be construed as a limitation to this application. All equivalent changes and modifications made in accordance with the scope of the patent application of the present invention shall fall within the scope covered by the present invention.

Claims

1. A ship hatch cover cleaning robot system based on multi-sensor fusion, characterized in that: The robot system includes a sensing component, a control component, an execution component, and a communication component; The sensing component is used to obtain environmental information. The sensing component includes an L1 radar, multiple laser range sensors, a camera, an electronic compass, and a GPS; The control component performs fusion processing on the data sensed by the sensing component based on the ubuntu system and can determine the position and attitude of the robot; Plan the optimal material cleaning path and generate obstacle avoidance instructions. The control component sends the instructions to the STM32 single-chip microcomputer system in the execution component; The execution component includes an STM32 single-chip microcomputer system, a crawler chassis, and a sweeping and pushing integrated material cleaning mechanism. The STM32 single-chip microcomputer system converts the control signal into a driving signal to drive the crawler chassis to realize the movement of the robot and drive the sweeping and pushing integrated material cleaning mechanism to complete the material cleaning operation; The communication component uses a 5G communication module to connect the robot to the cloud platform.

2. The multi-sensor fusion-based ship hatch cover cleaning robot system according to claim 1, wherein: The L1 radar is used to detect long-distance environmental information. The laser range sensor is responsible for accurately sensing nearby objects. The camera provides visual recognition. The electronic compass is used to determine the orientation of the robot. The GPS obtains spatial coordinates. The data collected by these sensors are transmitted to the Jetson orin nano in the control component.

3. The ship hatch cover cleaning robot system based on multi-sensor fusion according to claim 1, characterized in that: The sweeping and pushing integrated material cleaning mechanism includes a cleaning component and a pushing component. The cleaning component uses high-strength bristles. The pushing component includes a mounting rod, a mounting block, a connecting piece, and an arc-shaped pushing plate. One end of the mounting rod is connected to the crawler chassis. The other end of the mounting rod is fixed with the mounting block. The mounting block is connected with the arc-shaped pushing plate through the connecting piece. Both ends of the arc-shaped pushing plate are provided with L-shaped mounting plates. The high-strength bristles are fixed on the L-shaped mounting plates. A push rod mounting bracket is fixed under the mounting block. The push rod is fixed on the push rod mounting bracket. The power end of the push rod is fixed at the rear end of the arc-shaped pushing plate.

4. The working method of the ship hatch cover cleaning robot based on multi-sensor fusion according to claim 1, characterized in that: It includes the following steps: Step S1: Use the laser radar and the camera for joint positioning to accurately determine the position and attitude of the robot in the world coordinate system; Step S2: Use the ultrasonic sensor and the laser radar to cooperate to sense the environment and construct a high-precision grid map based on multi-source data fusion; Step S3: Use the ultrasonic sensor to measure the distance to the nearby obstacle through sound wave reflection; Step S4: Use the Kalman filter algorithm to fuse the data of the ultrasonic radar and the laser radar, effectively reducing the noise interference of a single sensor and realizing the real-time detection of dynamic obstacles.

5. The working method of the ship hatch cover cleaning robot based on multi-sensor fusion according to claim 4, characterized in that: In step S1, further: Use the laser radar to obtain the three-dimensional point cloud data of the environment by emitting laser beams. The camera on the ship unloader collects images of the ship's hatch cover area to form two-dimensional image data. Use the scale-invariant feature transform algorithm to extract the feature points in the image. Use the matching algorithm based on the ORB descriptor to find the corresponding relationship in the point cloud data after the laser radar conversion and the feature point data after the camera conversion, and accurately determine the position and attitude of the robot in the world coordinate system.

6. The working method of the ship hatch cover cleaning robot based on multi-sensor fusion according to claim 5, characterized in that: In step S1, a lidar is used to obtain 3D point cloud data of the environment by emitting laser beams, and a camera is used to collect images of the ship's hatch cover area to form 2D image data. The feature points in the image are extracted using the scale-invariant feature transform algorithm. Further, the lidar scans at a specific frequency, and each scan generates a large number of discrete distance data points. These points constitute the point cloud data, which is represented in matrix form as: where (x i , y i , z i ) represents the three-dimensional coordinates of the i-th point in the lidar coordinate system. The point cloud data obtained by the lidar is in its own coordinate system. To make the data consistent with the world coordinate system required for the robot's operation, a coordinate system transformation is needed; a transformation matrix T is constructed through the pre-acquired rotation matrix R and translation vector t l-w to achieve the transformation from the lidar coordinate system to the world coordinate system: The converted point cloud data P W is as follows: P w = T l-w · P lidar In the positioning scenario of the material cleaning robot, the first frame scanned by the lidar after the robot is powered on is used as the initial frame, and its pose transformation is the identity transformation. At the same time, the point cloud obtained is used as the reference point cloud. After that, during the movement of the robot, the point cloud P obtained by the current lidar scan is current registered with the reference point cloud; in each iteration, the ICP algorithm consists of two main steps: Step S11, corresponding point search: For each point in the transformed current point cloud P' current = T init · P currrent , find the point with the closest distance in the reference point cloud P reference to construct a set of corresponding point pairs C = {(P' i , P j )}, where P' i ∈ P' current p j ∈ P reference Step S12, Transformation Optimization: Based on the found set of corresponding point pairs C, calculate a new transformation matrix T by minimizing the objective function E icp : new Solve for T through singular value decomposition Update the transformation matrix; repeat the above two steps until the objective function converges to a smaller value. At this time, the obtained transformation matrix is the optimized lidar positioning transformation matrix, which can more accurately determine the position and orientation of the robot in the world coordinate system; new ​ The camera on the ship unloader collects images of the ship's hatch cover area to form two-dimensional image data; the scale-invariant feature transform algorithm is used to extract the feature points in the image, and the feature point set F is obtained camera : F camera = {f 1, f2,..., f n}。 7. The working method of the ship hatch cover cleaning robot based on multi-sensor fusion according to claim 5, characterized in that: The process of finding the corresponding relationship between the point cloud data after lidar conversion and the feature point data after camera conversion to accurately determine the position and orientation of the robot in the world coordinate system is further as follows: Among the point cloud data P w after lidar conversion and the feature point data F w after camera conversion, find the corresponding relationship to obtain the set of matching point pairs M, and use the least squares method to optimize the error of the matching point pairs to construct the objective function E: By iteratively solving to minimize the objective function, the optimal transformation matrix is obtained, and then the position and pose of the robot in the world coordinate system are accurately determined.

8. The working method of the ship hatch cover cleaning robot based on multi-sensor fusion according to claim 5, characterized in that: In step S4, further: The fusion formula for the fused environmental information is as follows: z k = K k ·z lidar +(1 - K k )z ultrasonidar where z k is the fused environmental information, K k is the Kalman gain coefficient, z lidar and z ultrasonidar are the measurement values of the ultrasonic radar and the lidar respectively; In global path planning, the heuristic function is optimized in combination with the structural characteristics of the ship's hatch cover. The cost function of the traditional A* algorithm is: f(n) = h(n) + g(n) where g(n) is the actual cost from the starting point to the current node, and h(n) is the heuristic estimated cost from the current node to the target node. For the possible slopes and uneven areas on the surface of the hatch cover, a terrain adaptation factor α is introduced to adjust the heuristic function to preferentially select flat paths and reduce the energy consumption and movement risk of the robot. where: max(H) represents the maximum height difference of the entire hatch cover terrain, and the ratio of the two is used to quantify the relative undulation degree of the current node, normalize the height, and reflect the steepness of the node. h(n) is the original heuristic function, and h new (n) is multiplied by the terrain adaptation factor α, making the algorithm more sensitive to terrain undulations during path planning; To deal with dynamic obstacles (such as residual goods temporarily piled up), the dynamic window approach is used for local path adjustment. DWA generates candidate trajectories by evaluating the velocity space (v, ω) of the robot and selects the optimal path based on the following evaluation function: Score(v, w) = λ1·Heading(v, w) + λ2·Dist(v, w) + λ3·Velocity(v, w) where Heading represents the alignment of the trajectory towards the target point, Dist is the distance to the nearest obstacle, Velocity is the weighted value of the current speed, and λ1, λ2, λ3 are weight coefficients. By fusing the global path and local obstacle avoidance strategies, the robot can achieve smooth and safe movement in a complex environment.

9. The working method of the ship hatch cover cleaning robot based on multi-sensor fusion according to claim 4, characterized in that: In step S2, further: First, the hatch cover is divided into grids of 0.1m × 0.1m. In each grid, the occupancy grid algorithm is used to convert the sensor measurement values into the occupancy probabilities of the grid cells. Each cell contains an occupancy probability value, which is used to characterize whether the area is occupied by an obstacle. The sensor measurement values are the data after the lidar and ultrasonic radar are fused through Kalman filtering.

Images