A motion-based online rust identification method for a rust removal robot for ship hulls

Abstract

This invention discloses a motion-based online rust identification method for a ship hull rust removal robot. The robot includes a wall-climbing robot body, a jet module, a power module, and a vision module. The power module and jet module are both located below the robot body, and the vision module is embedded in the lower part of the robot body. The robot body is equipped with a microcomputer. The jet module includes a water pump, a water pipe, and a nozzle. The microcomputer includes an AI chip, a memory, a battery, and a control drive board. The power module includes a drive motor and magnetically attached track wheels. The vision module includes a retractable camera, a camera baffle, a laser spotlight, and a photosensitive element. This robot uses machine vision technology in conjunction with a photosensitive element to detect the rust removal effect in real time by utilizing the difference in reflection characteristics after light shines on rusted and non-rusted areas of the steel plate. Without contaminating the camera with rust, this effectively assists the ship hull wall-climbing robot in performing fully automated rust removal on the ship's outer hull.

Description

Technical Field

[0001] This invention relates to the field of rust removal and automation of ship hull climbing walls, specifically to a motion-based online rust identification method for a rust removal robot for ship hulls. Background Technology

[0002] Rust removal is a crucial procedure in ship repair and maintenance. Regular rust removal effectively prevents the ship's outer hull from corroding due to prolonged immersion in seawater. Previously, this work was primarily done manually with handheld spray guns. This method not only severely pollutes the marine and coastal ecosystems but also endangers the health of rust removal workers.

[0003] Current ship hull-climbing robots are not yet truly intelligent; they require manual, visual remote control to remove rust from rust spots on the ship's outer hull. Furthermore, in actual rust removal processes, rust debris removed by water jets can easily obscure the vision sensors, making such robots, relying solely on machine vision, unsuitable for practical engineering applications. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention provides a motion-based online rust identification method for a ship hull rust removal robot, thereby solving the aforementioned technical problems.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is: a motion-based online rust identification method for a rust removal robot for ship hulls, comprising a rust removal robot, which includes a wall-climbing robot body, a jet module, a power module, and a vision module. The power module and the jet module are both located below the robot body, and the vision module is embedded in the lower part of the robot body. The wall-climbing robot body is equipped with a microcomputer, which includes an AI chip, a memory, a battery, and a control drive board. The jet module includes a water pump, a water pipe, and a nozzle. The power module includes a drive motor and magnetically attached track wheels. The vision module includes a retractable camera, a camera baffle, a laser spotlight, and a photosensitive element. The photosensitive element is located near the laser spotlight and is used to receive the light reflected from the steel plate after the laser spotlight illuminates the steel plate.

[0006] Based on the working method of the intelligent wall-climbing rust removal robot for ship outer plating described above, the specific steps are as follows:

[0007] Step S1: First, place the robot at the apex of one edge of the entire ship's outer plating and start the robot control system;

[0008] Step S2: Initialize the state of all robot modules; control the drive board to transmit the rotation speed and steering signal of the magnetically adsorbed track wheel to the power module, the power module works, and drives the robot to move at a constant speed according to the visual detection path; at the same time, control the drive board to transmit the start signal to the camera, so that it extends from the robot body to capture the image of the outer plate in real time.

[0009] Step S3: The outer panel image captured by the camera is processed using a quantized real-time image semantic segmentation model to obtain the category of each pixel in the image; pixels are classified into three categories: rust on the outer panel, non-rust on the outer panel, and non-outer panel.

[0010] Step S4: When a pixel of the outer plate rust is identified, the control drive board controls the power module to stop moving, memorizes the vertical coordinates of the top and bottom pixels of the outer plate rust category in the image, and the horizontal coordinates of the leftmost and rightmost pixels. Based on these coordinates, a rectangular rust removal area is established and the length and width of the area are calculated. The time to remove rust is then determined.

[0011] Step S5: Control the drive board to control the cover plate moving motor, and hide the telescopic camera in the body to avoid rust residue contaminating the camera during rust removal. The robot starts to move at a uniform speed and rust removal path from the lower left corner of the rectangular rust removal area, and controls the water pump and brush to perform jet rust removal.

[0012] Step S6: While the robot is moving from the lower left corner to the upper right corner along the rust removal path, the control drive board controls the laser spotlight of the vision module to illuminate the rust spots. Utilizing the principle that rust-free steel plates reflect light differently from rusty steel plates, the rust removal effect is inspected in real time from behind the robot. When a rust spot is found to be below the rust removal standard, the robot returns to the inspection area and repeatedly removes rust from that area until it meets the rust removal standard. When a rust spot is found to meet the rust removal standard, the robot continues to move along the path to the upper right corner and remove rust.

[0013] Step S7: After all rust spots in the rust removal area have been removed, the control drive board controls the machine to return to the vision detection path and continue to perform real-time image recognition along the path at the detection speed; when pixels such as outer plate rust are identified, return to step S4; when pixels such as no outer plate rust are identified, continue to travel along the vision detection path until the opposite vertex of the entire ship's outer plate is reached.

[0014] Furthermore, the specific steps for real-time detection of rust areas during the inspection phase are as follows:

[0015] SA1: Images of every location on a large number of rust-containing hull plates are collected. Starting from the rust removal starting point of the wall-climbing robot, images are collected according to a regular travel path, ensuring that the length and width of each image on the hull plate are consistent and that the images do not overlap. The collected image data are labeled according to the order of collection and three types of pixels in the images: rust on the hull plate, no rust on the hull plate, and no hull plate. After labeling, a dataset D for real-time detection of rust on hull plates is constructed. Dataset D is used as the training set for the real-time image semantic segmentation BiSeNet model.

[0016] SA2: The real-time semantic segmentation BiSeNet model is trained using dataset D. Post-processing operations, such as Conditional Random Field (CRF), are introduced into the BiSeNet model to further optimize the smoothness and accuracy of the segmentation results and optimize the model.

[0017] SA3: During image detection, the camera captures a frontal image through the lens and transmits it to a microcomputer; the trained BiSeNet model outputs three types of pixels in the image: rust on the outer panel, non-rust on the outer panel, and non-outer panel.

[0018] SA4: When the output image contains pixels of the outer panel rust category, the position of these pixels in the image area is identified. A rectangular area to be rusted is established based on the ordinates of the top and bottom pixels and the x-coordinates of the left and right pixels of this category. The drive board moves the robot body to the lower left corner of this rectangular area. The rust removal time and path are determined based on the coordinates, and preparation for rust removal is made. At the same time, the drive board retracts the camera into the robot body and controls the cover plate motor to hide the camera and prevent it from being contaminated by rust during the rust removal process. When the output image contains pixels of the non-outer panel category, the position of the boundary line between these pixels and the other two categories of pixels in the image area is identified. The robot's movement is controlled based on this boundary line position to prevent the robot from going off the outer panel. When the output image contains only pixels of the non-rust category of the outer panel, the robot moves at a constant speed according to the detection speed and detection path.

[0019] Furthermore, the steps for rust removal control and real-time verification of the effect during the rust removal stage are as follows:

[0020] SB1: As the wall-climbing robot enters the rust removal stage, the laser spotlight is turned on to inspect the rust-removed steel plate in real time; the laser spotlight shines light onto the surface of the steel plate, and the surface of the steel plate reflects the light to the photosensitive element near the laser spotlight;

[0021] SB2: After receiving the reflected light, the photosensitive element outputs different electrical signals to the microcomputer according to the intensity of the reflected light;

[0022] SB3: When the electrical signal reaches a certain value, it means that the rust spot has been cleaned. The robot continues to perform jet rust removal on the rust area according to the rust removal path. When the electrical signal does not reach this value, it means that the rust spot needs to be removed again. The robot returns to the position and performs rust removal repeatedly.

[0023] The rust removal time in the robot's rust removal stage is determined based on the length and width of the rusted area: Let the brush radius be R0, and the cleaning time be πR0. 2 The area is processed in time t0, and the length of the rectangular area to be derusted in the k-th captured image of the outer plate is α. k Width is β k If the photosensitive element does not detect an area requiring repeated rust removal, the rust removal time for that area is as follows;

[0024]

[0025] The beneficial effects of this invention are as follows: This invention discloses a motion-based online rust identification method for a ship hull rust removal robot. This robot combines machine vision technology with photosensitive elements, utilizing the difference in reflection characteristics between rusted and non-rusted areas of a steel plate after light shines on them to detect the rust removal effect in real time. Provided the camera is not contaminated by rust, it effectively assists the ship hull climbing robot in performing fully automated rust removal on the ship's outer hull, eliminating the need for manual remote control during operation. This invention utilizes the principle that the brightness of the reflected light from a clean steel plate differs from the brightness of the light reflected from the rusted area, resulting in a significant difference in the corresponding signal value output by the photosensitive element. This provides a more automated improvement to existing wall-climbing rust removal robots that require manual remote control, truly freeing up human hands. Attached Figure Description

[0026] Figure 1 This is the visual detection path of the robot on the unfolded diagram of the ship's outer plating in this embodiment of the invention;

[0027] Figure 2 This embodiment of the invention includes a rectangular rust removal area and a rust removal driving route plan in a pixel image of the outer plate rust type.

[0028] Figure 3 This is a system diagram of a smart wall-climbing rust removal robot for ship outer plating, as described in an embodiment of the present invention.

[0029] Figure 4 This is a diagram of the improved BiSeNet model structure in an embodiment of the present invention;

[0030] Figure 5 This is a three-dimensional structural diagram of the intelligent wall-climbing rust removal robot for ship outer plates in an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of the invention.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0033] This invention proposes a smart wall-climbing rust removal robot for ship hulls, comprising a main body, a jet module, a power module, and a vision module. The power module and jet module are located below the main body, while the vision module is embedded in the lower part of the main body. The main body houses a microcomputer, which includes an AI chip, memory, a battery, and a control drive board. The jet module includes a water pump, water pipes, and nozzles. The power module includes a drive motor and magnetically attached track wheels. The vision module includes a retractable camera, a camera shield, a laser spotlight, and a photosensitive element. The photosensitive element is placed near the laser spotlight to receive the light reflected from the steel plate after the laser spotlight illuminates it. The photosensitive element can be used to measure the intensity of the reflected light, thereby indirectly reflecting the reflectivity of the steel plate. Rusty steel plates have extremely low reflectivity, while rust-free steel plates have extremely high reflectivity.

[0034] The working method of the intelligent wall-climbing rust removal robot for ship outer plating described above includes the robot's overall workflow steps, real-time rust area detection methods, rust removal control after rust area identification, and real-time effect verification methods. The robot's overall workflow steps include:

[0035] S1: The robot is positioned at the apex of one edge of the entire ship's outer plating, and the robot control system is activated;

[0036] S2: Initialize the state of all robot modules; control the drive board to transmit the rotation speed and steering signals of the magnetically adsorbed track wheels to the power module, the power module works, and drives the robot to move at a constant speed according to the visual detection path; at the same time, control the drive board to transmit the start signal to the camera, so that it extends from the robot body to capture the image of the outer plate in real time.

[0037] S3: Use a lightweight real-time image semantic segmentation model to process the outer panel image captured by the camera to obtain the category of each pixel in the image; and classify the pixels into three categories: rust on the outer panel, non-rust on the outer panel, and non-outer panel.

[0038] S4: When a pixel of the outer panel rust is identified, the control drive board controls the power module to stop moving, memorizes the vertical coordinates of the top and bottom pixels of the outer panel rust category in the image, and the horizontal coordinates of the leftmost and rightmost pixels. Based on these coordinates, a rectangular rust removal area is established and the length and width of the area are calculated, and the time to remove rust is determined.

[0039] S5: The control drive board controls the cover plate moving motor. The telescopic camera is hidden in the body to avoid rust residue contaminating the camera during rust removal. The robot starts moving at a uniform speed and rust removal path from the lower left corner of the rectangular rust removal area, and controls the water pump and brush to perform jet rust removal.

[0040] S6: While the robot is moving from the lower left corner to the upper right corner along the rust removal path, the control drive board controls the laser spotlight of the vision module to illuminate the rust spots. Utilizing the principle that rust-free steel plates reflect light differently from rusty steel plates, the rust removal effect is inspected in real time from behind the robot. When a rust spot is found to be below the rust removal standard, the robot returns to the inspection area and repeatedly removes rust from that area until it meets the rust removal standard. When a rust spot is found to meet the rust removal standard, the robot continues to move along the path to the upper right corner and remove rust.

[0041] S7: After all rust spots in the rust removal area have been removed, the control drive board controls the machine to return to the vision detection path and continue to perform real-time image recognition along the path at the detection speed; when a pixel such as outer plate rust is detected, return to step S4; when no pixel such as outer plate rust is detected, continue to travel along the vision detection path until the opposite vertex of the entire ship's outer plate is reached.

[0042] The steps for real-time detection of rust areas during the inspection phase include:

[0043] SA1: Images of every location on a large number of rust-containing hull plates are collected. Starting from the rust removal starting point of the wall-climbing robot, images are collected according to a regular travel path, ensuring that the length and width of each image on the hull plate are consistent and that the images do not overlap. The collected image data are labeled according to the order of collection and three types of pixels in the images: rust on the hull plate, no rust on the hull plate, and no hull plate. After labeling, a dataset D for real-time detection of rust on hull plates is constructed. Dataset D is used as the training set for the real-time image semantic segmentation BiSeNet model.

[0044] SA2: The real-time semantic segmentation BiSeNet model is trained using dataset D. Post-processing operations, such as Conditional Random Field (CRF), are introduced into the BiSeNet model to further optimize the smoothness and accuracy of the segmentation results and optimize the model.

[0045] SA3: During image detection, the camera captures a frontal image through the lens and transmits it to a microcomputer; the trained BiSeNet model outputs three types of pixels in the image: rust on the outer panel, non-rust on the outer panel, and non-outer panel.

[0046] SA4: When the output image contains pixels of the outer panel rust category, the position of these pixels in the image area is identified. A rectangular area to be rusted is established based on the ordinates of the top and bottom pixels and the x-coordinates of the left and right pixels of this category. The drive board moves the robot body to the lower left corner of this rectangular area. The rust removal time and path are determined based on the coordinates, and preparation for rust removal is made. At the same time, the drive board retracts the camera into the robot body and controls the cover plate motor to hide the camera and prevent it from being contaminated by rust during the rust removal process. When the output image contains pixels of the non-outer panel category, the position of the boundary line between these pixels and the other two categories of pixels in the image area is identified. The robot's movement is controlled based on this boundary line position to prevent the robot from going off the outer panel. When the output image contains only pixels of the non-rust category of the outer panel, the robot moves at a constant speed according to the detection speed and detection path.

[0047] Methods and steps for rust removal control and real-time inspection of rust removal effectiveness during the rust removal stage:

[0048] SB1: As the wall-climbing robot enters the rust removal stage, the laser spotlight is turned on to inspect the rust-removed steel plate in real time; the laser spotlight shines light onto the surface of the steel plate, and the surface of the steel plate reflects the light to the photosensitive element near the laser spotlight;

[0049] SB2: After receiving the reflected light, the photosensitive element outputs different electrical signals to the microcomputer according to the intensity of the reflected light;

[0050] SB3: When the electrical signal reaches a certain value, it indicates that the rust spot has been completely removed, and the robot continues to perform jet rust removal on the rust area according to the rust removal path; when the electrical signal does not reach this value, it indicates that the rust spot needs to be removed again. The robot returns to this position and repeats the rust removal process.

[0051] The rust removal time in the robot's rust removal stage is determined based on the length and width of the rusted area: Let the brush radius be R0, and the cleaning time be πR0. 2 The area is processed in time t0, and the length of the rectangular area to be derusted in the k-th captured image of the outer plate is α. k Width is β k If the photosensitive element does not detect an area requiring repeated rust removal, the rust removal time for that area is as follows.

[0052]

[0053] In this invention, the wall-climbing robot moves along the outer hull of a ship while capturing images in real time via a camera. A deep learning model for image semantic segmentation is used to identify rust areas in the images. The robot memorizes the ordinates of the top and bottom pixels, and the abscissas of the left and right pixels for each rust category on the outer hull. Based on these coordinates, a rectangular rust removal area is established, and its length and width are calculated to determine the area to be removed. The second step involves determining the pump's operating time based on the area to be removed. The third step, after determining the pump's operating time and the robot's path, involves the camera stopping and retracting into the robot's body, and the baffle moving. The first step involves blocking the camera to prevent contamination during rust removal. The second step, with the camera hidden, involves controlling the drive board to move the magnetically attached tracked wheels along a specific path while simultaneously controlling the water pump and laser spotlight. The laser spotlight illuminates the steel plate being cleaned, and the photosensitive element receives the light reflected from the steel plate surface, outputting a corresponding electrical signal. When the electrical signal reaches a certain standard value, it indicates that rust removal at that point is complete, thus controlling the magnetically attached track to move the robot to other irradiated points for rust removal. The third step, after all rust at all irradiated points in the rusted area has been removed, returns to the first step to continue rust removal according to the rusted area. This invention utilizes the principle that the brightness of the light reflected from a clean steel plate differs from the brightness reflected from the rusted area, resulting in a significant difference in the corresponding signal value output by the photosensitive element. This provides a more automated improvement to existing wall-climbing rust removal robots that require manual remote control, truly freeing up human hands.

[0054] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for motion-based online rust identification of a rust removal robot for ship hulls, characterized in that, The invention includes a rust removal robot, comprising a wall-climbing robot body, a jetting module, a power module, and a vision module. The power module and jetting module are located below the robot body, and the vision module is embedded in the lower part of the robot body. The wall-climbing robot body is equipped with a microcomputer, which includes an AI chip, memory, a battery, and a control drive board. The jetting module includes a water pump, a water pipe, and nozzles. The power module includes a drive motor and magnetically attached track wheels. The vision module includes a retractable camera, a camera baffle, a laser spotlight, and a photosensitive element. The photosensitive element is placed near the laser spotlight to receive light reflected from the steel plate after the laser spotlight illuminates it. Based on the above-mentioned motion-based online rust identification method for ship hull rust removal robots, the specific steps are as follows: Step S1: First, place the robot at the apex of one edge of the entire ship's outer plating and start the robot control system; Step S2: Initialize the state of all robot modules; control the drive board to transmit the rotation speed and steering signal of the magnetically adsorbed track wheel to the power module, the power module works, and drives the robot to move at a constant speed according to the visual detection path; at the same time, control the drive board to transmit the start signal to the camera, so that it extends from the robot body to capture the image of the outer plate in real time. Step S3: The outer panel image captured by the camera is processed using a quantized real-time image semantic segmentation model to obtain the category of each pixel in the image; pixels are classified into three categories: rust on the outer panel, non-rust on the outer panel, and non-outer panel. Step S4: When a pixel of the outer plate rust is identified, the control drive board controls the power module to stop moving, memorizes the vertical coordinates of the top and bottom pixels of the outer plate rust category in the image, and the horizontal coordinates of the leftmost and rightmost pixels. Based on these coordinates, a rectangular rust removal area is established and the length and width of the area are calculated. The time to remove rust is then determined. Step S5: Control the drive board to control the cover plate moving motor, and hide the telescopic camera in the body to avoid rust residue contaminating the camera during rust removal. The robot starts to move at a uniform speed and rust removal path from the lower left corner of the rectangular rust removal area, and controls the water pump and brush to perform jet rust removal. Step S6: While the robot is moving from the lower left corner to the upper right corner along the rust removal path, the control drive board controls the laser spotlight of the vision module to illuminate the rust spots. Utilizing the principle that rust-free steel plates reflect light differently from rusty steel plates, the rust removal effect is inspected in real time from behind the robot. When the rust spots are found to be below the rust removal standard, the robot returns to the inspection area to repeatedly remove rust from the rust removal area until it meets the rust removal standard. When the rust spots are found to meet the rust removal standard, the robot continues to move along the path to the upper right corner and remove rust. Step S7: After all rust spots in the rust removal area have been removed, the control drive board controls the machine to return to the vision detection path and continue to perform real-time image recognition along the path at the detection speed; when pixels such as outer plate rust are identified, return to step S4; when pixels such as no outer plate rust are identified, continue to travel along the vision detection path until the opposite vertex of the entire ship's outer plate is reached.

2. The method for motion-based online rust identification of a ship hull rust removal robot according to claim 1, characterized in that, The specific steps for real-time detection of rust areas during the inspection phase are as follows: SA1: Images of every location on a large number of rust-containing hull plates are collected. Starting from the rust removal starting point of the wall-climbing robot, images are collected according to a regular travel path, ensuring that the length and width of each image on the hull plate are consistent and that the images do not overlap. The collected image data are labeled according to the order of collection and three types of pixels in the images: rust on the hull plate, no rust on the hull plate, and no hull plate. After labeling, a dataset D for real-time detection of rust on hull plates is constructed. Dataset D is used as the training set for the real-time image semantic segmentation BiSeNet model. SA2: The real-time semantic segmentation BiSeNet model is trained using dataset D. Post-processing operations, such as Conditional Random Field (CRF), are introduced into the BiSeNet model to further optimize the smoothness and accuracy of the segmentation results and optimize the model. SA3: During image detection, the camera captures a frontal image through the lens and transmits it to a microcomputer; the trained BiSeNet model outputs three types of pixels in the image: rust on the outer panel, non-rust on the outer panel, and non-outer panel. SA4: When the output image contains pixels of the outer panel rust category, the position of these pixels in the image area is identified. A rectangular area to be rusted is established based on the ordinates of the top and bottom pixels and the x-coordinates of the left and right pixels of this category. The drive board moves the robot body to the lower left corner of this rectangular area. The rust removal time and path are determined based on the coordinates, and preparation for rust removal is made. At the same time, the drive board retracts the camera into the robot body and controls the cover plate motor to hide the camera and prevent it from being contaminated by rust during the rust removal process. When the output image contains pixels of the non-outer panel category, the position of the boundary line between these pixels and the other two categories of pixels in the image area is identified. The robot's movement is controlled based on this boundary line position to prevent the robot from going off the outer panel. When the output image contains only pixels of the non-rust category of the outer panel, the robot moves at a constant speed according to the detection speed and detection path.

3. The method for motion-based online rust identification of a ship hull rust removal robot according to claim 2, characterized in that, The steps for rust removal control and real-time effect verification during the rust removal stage are as follows: SB1: As the wall-climbing robot enters the rust removal stage, the laser spotlight is turned on to inspect the rust-removed steel plate in real time; the laser spotlight shines light onto the surface of the steel plate, and the surface of the steel plate reflects the light to the photosensitive element near the laser spotlight; SB2: After receiving the reflected light, the photosensitive element outputs different electrical signals to the microcomputer according to the intensity of the reflected light; SB3: When the electrical signal reaches a certain value, it proves that the rust spot has been cleaned, and the robot continues to perform jet rust removal on the rust area according to the rust removal path; when the electrical signal does not reach this value, it proves that the rust spot needs to be removed again. The fuselage was returned to that position and repeatedly derusted. The rust removal time during the robot's rust removal stage is determined based on the length and width of the rusted area: Let the brush radius be... R 0， Cleaning π R 0 2 Area and time t 0， The captured k The length of the rectangular area to be derusted in the image of the outer plate is... α k , width is β k， If the photosensitive element does not detect any areas requiring repeated rust removal, the rust removal time for those areas is as follows: .

Images