Ship hull cleaning robot based on visual recognition and power adaptive control method thereof

Abstract

The application discloses a hull cleaning robot based on visual recognition and a power self-adaptive control method thereof, and belongs to the technical field of cleaning robots. The hull cleaning robot comprises a robot body, a power control hub, a multi-dimensional sensing system, a multi-stage cooperative cleaning module and a closed-loop feedback system. The control method comprises the following steps: step 1, obtaining a three-dimensional point cloud model and image features of a region to be cleaned, calculating the average height features of pollutants, and taking the average height features as input data for pollution assessment; step 2, receiving the feature information extracted in step 1, identifying the types of pollutants and calculating a comprehensive cleaning index to output a pollution classification result; step 3, matching corresponding power parameters according to the pollution level classification result output in step 2, and driving cleaning mechanisms at all stages to perform adaptive cleaning; and step 4, after step 3 is performed, detecting the residual pollution ratio of the cleaning region, and feeding back the result to the control system to trigger dynamic compensation, so that complete closed-loop cleaning is formed.

Description

Technical Field

[0001] This invention belongs to the field of cleaning robot technology, specifically relating to a vision-based ship hull cleaning robot and its power adaptive control method. Background Technology

[0002] With the continuous expansion of the marine shipping industry and the extension of ship operating cycles, large amounts of marine organisms, such as barnacles, seaweed, shellfish, and microbial films, easily adhere to the surface of ships. These deposits not only significantly increase the roughness of the hull surface, increasing navigation resistance, leading to increased fuel consumption and decreased navigation efficiency, but may also accelerate the corrosion of hull materials and shorten the ship's maintenance cycle. Therefore, efficient and regular cleaning of these deposits has become a crucial aspect of ship maintenance management.

[0003] In recent years, ship cleaning robots have been increasingly applied in the field of ship maintenance, but existing equipment still has many shortcomings in practical applications. Firstly, regarding energy utilization, many traditional cleaning devices still operate with fixed power, maintaining a constant brush speed or jet pressure throughout the cleaning process. When facing lightly soiled areas, excessive power output leads to unnecessary energy waste; while when encountering heavily soiled areas such as areas with dense barnacles, fixed power may be insufficient to thoroughly remove stubborn deposits, resulting in incomplete cleaning and even requiring repeated operations, further reducing operational efficiency. Secondly, in terms of information perception and execution control, although some cleaning robots are equipped with visual recognition systems, these systems are mostly used only for auxiliary path planning or simple obstacle recognition, failing to fully utilize information such as the type, coverage area, and density of pollutants acquired by the visual system. The lack of a mechanism to convert visual recognition results into real-time control parameters for the actuators prevents the cleaning system from automatically adjusting brush speed, water jet pressure, or cleaning methods according to different types and degrees of deposits, resulting in a significant disconnect between the perception system and the mechanical execution system. Furthermore, in terms of mechanical structure design, existing cleaning devices mostly employ a single type of cleaning mechanism, such as a single brush or a fixed pressure jet device. However, marine deposits exhibit significant differences in physical properties. For instance, hard deposits like barnacles typically have strong adhesion and require high mechanical force or high-pressure water flow for effective removal; while soft deposits like seaweed are more suitable for cleaning with gentle brushing or low-pressure water flow. Using a single cleaning mechanism not only reduces overall cleaning efficiency but may also lead to accelerated wear and tear on critical components such as the brush head, increasing equipment maintenance costs.

[0004] Therefore, in order to improve the efficiency and intelligence of ship cleaning operations, it is urgent to design an intelligent cleaning system that can combine visual recognition technology to adjust the working parameters of multi-level cleaning mechanisms in real time according to the type and density information of pollutants. This would enable targeted treatment of different deposits, effectively reduce energy consumption and extend the service life of equipment while ensuring cleaning results. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention aims to provide a vision-based hull cleaning robot and its power adaptive control method.

[0006] On one hand, the present invention provides a vision-based hull cleaning robot, comprising: The robot body uses a lightweight aluminum alloy frame as the main chassis. Four permanent magnet adsorption wheel sets are symmetrically arranged under the main chassis, which are driven independently by four DC geared motors. The power control center integrates a host computer, a slave computer, a Raspberry Pi 5, and a power management module. The multi-dimensional sensing system includes a front-facing depth camera and a ring-shaped LED fill light array. The front-facing depth camera is connected to the Raspberry Pi Pi5, a data transfer and preprocessing node, via a USB data interface to transmit high-resolution color image signal data in real time. The multi-level collaborative cleaning modules are arranged sequentially from front to back along the central axis of the main chassis, forming a process chain of "crushing-peeling-polishing", including a coarse cleaning unit, a high-pressure rinsing unit, and a fine cleaning unit; The closed-loop feedback system includes a rear-mounted depth camera installed at the rear of the main chassis. The lens focuses vertically downward on the cleaned area. The rear-mounted depth camera is connected to the host computer via a USB interface to transmit real-time images after cleaning back to the host computer's evaluation model, which serves as the basis for deciding whether the robot needs to reduce its speed or perform reciprocating cleaning. The front-facing depth camera transmits data to the host computer, which runs the YOLO detection algorithm and power allocation algorithm. After calculating the power parameters for each stage, the host computer sends the command packet to the Raspberry Pi PI5 via the CAN bus. The Raspberry Pi PI5 transmits the information to the lower-level computer and parses the command packet, outputting multiple PWM / analog signals to the motor drivers and water pump inverters at each stage. The data from the rear-facing depth camera is transmitted back to the host computer. If the host computer determines that the cleanliness is unqualified, it sends a deceleration command to the moving motor, forming a complete visual feedback closed-loop control.

[0007] On the other hand, the present invention provides a power adaptive control method for a ship hull cleaning robot based on vision recognition, which is implemented based on the aforementioned vision recognition-based ship hull cleaning robot and includes the following steps: Step 1: Multidimensional Environmental Image Acquisition and Depth Feature Preprocessing: Obtain the 3D point cloud model and image features of the area to be cleaned, and calculate the average height features of the pollutants. And use it as input data for pollution assessment; Step 2: Construct a pollutant classification and evaluation model based on deep learning: Receive the feature information extracted in Step 1, identify the types of pollutants, calculate the Comprehensive Cleanliness Index (CII), and output the pollution classification results; Step 3: Power adaptive mapping and linkage control of multi-level actuators: Based on the pollution level classification results output in Step 2, match the corresponding power parameters and drive each level of cleaning mechanism to perform adaptive cleaning; Step 4, Cleaning quality closed-loop detection and compensation operation: After performing step 3, the residual contamination ratio of the cleaned area is detected, and the result is fed back to the control system to trigger dynamic compensation, forming a complete closed-loop cleaning.

[0008] Furthermore, step 1 specifically includes: Image acquisition: With the assistance of an LED fill light array, the depth camera at the front end of the robot acquires color images and depth information images of the area to be cleaned in real time; Preprocessing: Median filtering and illumination compensation correction are performed on the acquired images to eliminate the interference of the underwater environment on image contrast; Spatial mapping: Depth information is pixel-level aligned with RGB images to construct a 3D point cloud model of the surface to be cleaned. By calculating the normal displacement of each pixel in the 3D point cloud model relative to the ship's reference plane, the average height feature of the pollutants is extracted and calculated. .

[0009] Furthermore, the average height characteristics of the pollutants were calculated. The specific implementation steps are as follows: ① Preprocess point cloud data and align coordinates: Obtain the original depth map and color image of the area to be cleaned through the front-facing depth camera, and use the camera's intrinsic parameter matrix to convert the two-dimensional depth map into a set of point clouds P in three-dimensional space; ② Mathematical fitting of hull reference: The point cloud set P is fitted with a random sampling consensus algorithm to remove outlier points that belong to pollutants and extract the optimal fitting reference surface S representing the clean hull surface. ③ Mapping and extraction of pollutant target point clouds: An improved YOLOv11 network is used to perform semantic segmentation on RGB images, outputting a two-dimensional pixel mask for a specific pollutant. Based on spatial mapping relationships, the two-dimensional pixel mask is mapped to a three-dimensional point cloud set P, extracting a subset of target point clouds belonging only to the pollutant. ; ④ Calculation of point normal displacement: for a subset of the target point cloud Any point P in k ( , Calculate the vertical spatial distance from this point to the hull reference plane S, i.e., the normal displacement. ; ⑤ Average height characteristics Calculation: Traversing the target point cloud subset For all N pixels, sum the normal displacements of all points. The arithmetic mean was calculated to obtain the average height characteristics of this type of pollutant. .

[0010] Furthermore, step 2 specifically includes: Step 2.1, Target Detection and Semantic Segmentation: An improved YOLOv11 series network is used to perform instance segmentation on the input image, extract the contours and features of each independent pollutant in the field of view, and generate corresponding pixel-level masks, so as to identify the types of pollutants in the image in real time, including three types: hard attachments, soft attachments and biofilms. Step 2.2: Calculate the pollution coverage rate The proportion of pixels of various contaminants in the current detection field of view is calculated using a mask counting algorithm. Step 2.3: Calculate the Cleanliness Index: Based on the identification results, introduce weighting factors to calculate the Comprehensive Cleanliness Index (CII) for the current area; Step 2.4, Level Classification: Map the Comprehensive Cleanliness Index (CII) value to four operational levels: Level I, Level II, Level III, and Level IV.

[0011] Furthermore, in step 2.2, the pollution coverage rate The calculation process is as follows: ① Generation of the mask binarization matrix: Based on the output of the YOLOv11 instance segmentation network in step 2.1, a classification index matrix with the same resolution as the original image is generated; for the i-th type of pollutant, its corresponding semantic mask is extracted and converted into a binary mask matrix M. i (u, v): ② Cumulative statistics of the total number of pixels: Using a mask counting algorithm, spatial accumulation is performed on all elements in the binary matrix; ③ Determining the total number of pixels in the effective field of view: Determine the total number of pixels in the current detection field of view. If there are occluded areas or invalid edges in the image, invalid pixels will be removed in advance, and only the total number of pixels in the valid observation area will be counted. ④ Pollution coverage rate Normalization calculation: The total number of pixels N of the i-th type of pollutant i Total number of pixels in the field of view The ratio is calculated to obtain the coverage ratio of this type of pollutant on the current work surface. This value reflects the concentration of a specific type of pollutant, and its range is between [0, 1]. ⑤ Multi-class weighted composition: If multiple types of pollutants exist simultaneously in the field of view, then R is calculated separately for each class. c1 ,R c2 ,....., ; calculate R c1 , R c2 ,....., Multiply by their respective hardness weighting factors This is used for the weighted summation of the final Comprehensive Cleanliness Index (CII).

[0012] Furthermore, if the pixel with coordinates (u, v) belongs to the i-th type of pollutant, then M i (u, v) = 1; if the pixel with coordinates (u, v) does not belong to the i-th type of pollutant, then M i (u, v) = 0.

[0013] Furthermore, step 3 specifically includes: Step 3.1, Instruction Generation: The main control unit generates three sets of independent control parameters by looking up the preset power mapping table according to the Comprehensive Cleanliness Index (CII) level.

[0014] The steps for obtaining the preset power mapping table are as follows: ① Construct a standard pollution sample test library: In a controlled water tank or real dock environment, select ship hull surfaces with different species (barnacles, algae, biofilm), different coverage rates, and different thicknesses as test samples. Collect sample data using a front-mounted depth camera, and run an evaluation model on a host computer to calculate the Comprehensive Cleanliness Index (CII) for each test sample, and record the initial state.

[0015] ② Multi-gradient power traversal and cleaning test: For each sample with a specific CII value, the control system issues a traversal command, so that the coarse cleaning motor, high-pressure water pump and fine cleaning motor perform repeated cleaning tests within the safety threshold with different power combinations, different PWM duty cycles, different CAN communication control speeds and different inverter frequencies.

[0016] ③ Quantification of cleaning effect and calibration of optimal solution: After each test, the proportion of residual contamination is extracted using a rear depth camera. ; Search for what can make The set of hardware control parameters that is less than 5% below the acceptable threshold, has no paint damage, and has the lowest total energy consumption is defined as the optimal set of control parameters corresponding to the current CII value.

[0017] ④ Construct a non - linear mapping matrix and curve fitting to summarize the test data of a large number of samples, establish a three - dimensional scatter data set with the CII value as the independent variable and the rough - washing motor speed, high - pressure water pressure, and fine - washing motor speed as the dependent variables, and form a power mapping table.

[0018] Step 3.2, Hardware linkage execution: If the comprehensive cleaning index CII > threshold A, control the start of the rough - washing motor of the rough - washing unit, linearly adjust the rotation speed of the stainless - steel spiked roller brush according to the comprehensive cleaning index CII, and use high torque to break hard shells; according to the density of the rough - washing debris, adjust the output pressure of the high - pressure water pump of the high - pressure flushing unit in real - time, and wash the debris away from the wall through the bidirectional nozzle group; control the fine - washing motor of the fine - washing unit to keep the rotation speed of the nylon brush constant or fine - tune it to remove the residual biofilm.

[0019] Step 3.3, Communication protocol: The host computer sends parameters to the lower computer through the serial port / CAN bus. The lower computer uses a hardware timer to generate PWM waveforms to drive each module to work; specifically including: the rough - washing motor driver for driving the stainless - steel spiked roller brush of the rough - washing unit, the frequency converter for adjusting the water pump pressure of the high - pressure flushing unit, the secondary motor driver for driving the high - density nylon brush of the fine - washing unit, and the mobile motor driver for controlling the movement of the robot body.

[0020] Further, the specific content of step 4 includes: Post - inspection comparison: The rear - mounted depth camera real - time collects images of the cleaned area, and calculates the residual pollution ratio Rremain through the same semantic segmentation algorithm; Closed - loop logic: If the residual pollution ratio 5% < ≤0, it is judged as qualified, and the robot maintains the original speed and moves forward; if the residual pollution ratio ≥5%, it is judged that the cleaning is not thorough, and the self - compensation mechanism is triggered; the self - compensation mechanism is to automatically reduce the rotation speed of the mobile motor, increase the cleaning duration per unit area, or command the robot to retreat and perform "reciprocating" secondary cleaning until the detection is qualified.

[0021] Further, the specific algorithm steps of the residual pollution ratio are as follows: ① Rear - mounted image data acquisition: After the cleaning mechanism finishes its operation, use the rear - mounted depth camera at the tail of the robot to real - time collect RGB - D image data of the cleaned area; ② Residue semantic segmentation and extraction: Input the image collected by the rear - mounted depth camera into a pre - trained semantic segmentation model, identify "residual biological tissue after cleaning", and generate a binary mask (u, v); ③ Total number of residual pixels statistics: Statistic the binary mask The total number of pixels with a value of 1 in (u, v) is denoted as . ; ④ Residual pollution ratio Quantitative calculation: Calculate the proportion of residual contaminants in the cleaned area to the total area of ​​the detected area: ,in, This represents the total number of effective detection pixels for the rear camera.

[0022] Compared with the prior art, the present invention has the following advantages: (1) This invention constructs a three-level control architecture consisting of a host computer, a Raspberry Pi Pi5, and an ESP32, combined with a dual sensing unit consisting of a front-facing depth camera and a feedback depth camera, and a combined execution unit consisting of a fine washing roller brush, a coarse washing roller brush, and a rinsing device, and combined with a power system consisting of a unified lithium battery pack and a step-down module, thereby realizing the quantitative identification of the degree of pollution on the ship's hull.

[0023] (2) This invention trains a YOLOv11 target detection model based on a dedicated ship hull pollutant database, performs pixel-level analysis on the image data collected by the front depth camera, and accurately calculates the proportion of pollutant pixels; by establishing a dynamic mapping relationship between multi-gradient pollution levels and cleaning power, the output power of the motor drive system is dynamically adjusted according to the quantitative identification results, so that the cleaning intensity and pollution level are accurately matched; at the same time, the data collected after cleaning by the rear depth camera is analyzed by the model and compared with the qualified threshold, and the power adjustment and secondary cleaning action are automatically triggered, forming a cleaning quality feedback closed-loop control system of "quantitative identification-power adaptation-effect feedback-parameter correction".

[0024] (3) This invention not only significantly improves energy utilization efficiency and avoids energy waste in the traditional fixed power mode, but also greatly improves the uniformity and automation of hull cleaning through full-process automated control logic and full-area visual detection capability, effectively solving the technical problems of insufficient recognition accuracy, low energy utilization rate and difficulty in guaranteeing cleaning quality of existing hull cleaning robots. Attached Figure Description

[0025] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0027] Example 1 This embodiment provides a vision-based ship cleaning robot, including a robot body, a power control center, a multi-dimensional perception system, a multi-level collaborative cleaning module, and a closed-loop feedback system. The robot body uses a lightweight aluminum alloy frame as its main chassis, with four permanent magnet adsorption wheel sets symmetrically arranged below the chassis, driven independently by four DC geared motors. The power control center integrates a host computer, a slave computer, a Raspberry Pi 5, and a power management module. The multi-dimensional perception system includes a front-facing depth camera and a ring-shaped LED supplementary lighting array. The front-facing depth camera is connected to the Raspberry Pi 5, a data transfer and preprocessing node, via a USB data interface, transmitting high-resolution color image signal data in real time. The multi-level collaborative cleaning module is arranged sequentially from front to back along the central axis of the main chassis, forming a "crushing-peeling-polishing" process chain, including a rough cleaning unit, a high-pressure rinsing unit, and a fine cleaning unit. The closed-loop feedback system includes a rear-mounted depth camera installed at the rear of the main chassis. The lens focuses vertically downwards on the cleaned area. The rear-mounted depth camera is connected to the host computer via a USB interface to transmit real-time images after cleaning back to the host computer's evaluation model, which serves as the basis for deciding whether the robot needs to reduce its speed or perform reciprocating cleaning.

[0028] In the robot structure of this invention, the modules are interconnected through hierarchical communication and power distribution. At the perception layer, the front and rear depth cameras are connected to a Raspberry Pi Pi 5 via a high-speed USB data cable interface to acquire images and depth information of the hull surface. The Raspberry Pi Pi 5 acts as a data relay and preprocessing node, performing basic processing on the acquired image data before transmitting the raw image data stream to the host computer via Gigabit Ethernet or built-in WiFi. At the decision layer, the host computer, as the core computing unit, is responsible for running visual recognition and control decision algorithms. It establishes bidirectional communication with the Raspberry Pi Pi 5 via a USB-to-TTL communication link, sending the processed control logic instructions to the lower-level control system. At the execution layer, the Raspberry Pi Pi 5 further sends control signals to the ESP32 microcontroller via the GPIO interface or USB instruction set, and the ESP32 microcontroller completes the underlying hardware control tasks. The GPIO output interface of the ESP32 microcontroller is connected to the signal input terminals of the multi-channel motor drive module, and the high-voltage output terminals of the motor drive module are connected to multiple sets of motors. These motors drive the coarse cleaning rollers of the coarse cleaning unit, the fine cleaning rollers of the fine cleaning unit, and the four permanent magnet adsorption wheels on the bottom of the robot's main chassis, achieving mechanical cleaning and movement control of the hull surface. Simultaneously, the high-pressure rinsing unit integrates an electronically controlled high-pressure water pump, whose control terminal is also connected to the PWM control interface of the ESP32 microcontroller to achieve dynamic adjustment of the spray pressure. In terms of energy supply, the battery system provides power to the entire system through a power distribution board, providing a stable DC regulated power supply to the host computer and Raspberry Pi Pi 5, and a high-current power supply to the motor drive modules, thus ensuring the robot's stable operation during high-load cleaning operations.

[0029] In a preferred embodiment of the present invention, the hull cleaning robot achieves adaptive adjustment of cleaning power through multi-dimensional environmental perception, intelligent recognition and calculation, and multi-level actuator collaborative control. The robot is first deployed on the hull surface and the operating system is activated. It uses a front-mounted depth camera to perceive the environment of the area to be cleaned. This front-mounted depth camera has an IP68 protection rating, enabling stable operation in complex underwater environments. It also works in conjunction with a ring-shaped supplementary light to improve image clarity in low-light conditions. The robot collects RGB images and depth point cloud data of the area in front in real time. The image data is transmitted via USB to a Raspberry Pi Pi 5 for preliminary processing, including image denoising, pixel alignment, and data organization. The processed data is then sent to a host computer via Gigabit Ethernet for further intelligent recognition and analysis.

[0030] During the image data processing stage, the host computer loads a pre-trained improved YOLOv11 deep learning model to perform real-time recognition of the acquired images. This model can not only identify the types of attachments on the ship's hull surface, such as barnacles, seaweed, and shellfish, but also calculate the coverage ratio R of each type of attachment in the image. ci Simultaneously, by combining point cloud data acquired by a depth camera, the thickness information of the attached material is analyzed to extract the average thickness feature value H of the pollutants. avg Based on this, the host computer operates an adaptive evaluation algorithm for power consumption, which calculates the Comprehensive Cleanliness Index (CII) by comprehensively considering pollutant type weights, coverage ratios, and thickness characteristics. The calculation formula is as follows: Among them, W i The weighting coefficient for hard shell attachments is generally between 0.5 and 2.0; β is the thickness correction factor. Based on the calculated CII value, the system automatically classifies the area to be cleaned into four levels: lightly contaminated, moderately contaminated, heavily contaminated, and extremely contaminated, thus providing a basis for subsequent actuator control.

[0031] During the hardware execution phase, the host computer generates corresponding power control command packets based on the contamination level and sends them to the Raspberry Pi 5 via the communication link. The Raspberry Pi parses the commands and sends the control information to the underlying ESP32 microcontroller, which then performs the specific hardware control tasks. The ESP32 uses its internal hardware timer to output three independent control signals to coordinate the control of different cleaning mechanisms. In the rough cleaning phase, when a heavily contaminated area is identified, the ESP32 outputs a high-duty-cycle PWM signal via GPIO to drive the rough cleaning motor of the rough cleaning unit, causing the stainless steel barbed roller brush to rotate at a high speed of approximately 1200 rpm, thereby breaking down and removing hard deposits such as barnacles. In the rinsing phase, the ESP32 controls the inverter of the high-pressure water pump via the DAC interface, adjusting the nozzle pressure in real time according to the density and quantity of the removed material. The pressure range can vary between 20 MPa and 45 MPa to ensure that debris is effectively flushed away from the hull surface. In the fine cleaning phase, the rear nylon brush motor of the fine cleaning unit is further controlled to perform surface polishing and cleaning, restoring the hull surface to a relatively smooth state. Meanwhile, the four permanent magnet adsorption structure on the bottom of the robot dynamically adjusts its movement speed according to the level of pollution, realizing an intelligent movement strategy of "low-speed operation in heavily polluted areas and rapid passage through lightly polluted areas" to improve overall cleaning efficiency.

[0032] To ensure cleaning effectiveness, a closed-loop feedback and self-compensation control mechanism is introduced. After fine cleaning, the depth camera re-detects the cleaned area in real time and transmits the image data back to the host computer for identification and analysis. If the back-end identification model determines that the residual contamination ratio still exceeds the set threshold of 5%, the host computer will immediately trigger the compensation mechanism: instructing the ESP32 to reduce the operating speed of the robot's moving motor and simultaneously increase the output power of the coarse washing roller and high-pressure rinsing unit. If necessary, the robot is controlled to perform short-distance reciprocating movements to perform a second cleaning of the same area. This process continues until the contaminant ratio in the feedback image drops below the set threshold, thereby ensuring that the hull cleaning quality meets the expected standards. Through the above-mentioned multi-level perception, decision-making, and execution collaborative mechanism, this invention can significantly improve the operational efficiency and intelligence level of the hull cleaning robot.

[0033] In summary, this invention discloses a three-in-one cleaning device layout and its adaptive control logic that integrates "crushing-peeling-polishing". The CII index is mapped in real time to three independent control parameters: the speed of the coarse washing roller brush motor, the pressure of the high-pressure water pump, and the torque of the fine washing roller brush. Through the relay cooperation of various actuators in time and space, the invention solves the problems of difficulty in completely removing hard deposits (barnacles, etc.) in the fixed power mode of existing technologies, as well as the energy waste caused by a single mechanism in lightly polluted areas.

[0034] Example 2 like Figure 1 As shown, this embodiment provides a power adaptive control method for a ship hull cleaning robot based on vision recognition, implemented based on the vision recognition-based ship hull cleaning robot described in Embodiment 1, including the following steps: (I) Step 1: Multidimensional environmental image acquisition and depth feature preprocessing: Obtain the three-dimensional point cloud model and image features of the area to be cleaned, and calculate the average height features of pollutants. And use it as input data for pollution assessment.

[0035] Image acquisition: The RealSense D435i depth camera at the front of the robot, with the assistance of an LED fill light array, acquires color images and depth information images of the area to be cleaned in real time.

[0036] Preprocessing: Median filtering and illumination compensation correction are performed on the acquired images to eliminate the interference of the underwater environment on image contrast.

[0037] Spatial mapping: Depth information is pixel-level aligned with RGB images to construct a 3D point cloud model of the surface to be cleaned. By calculating the normal displacement of each pixel in this 3D point cloud model relative to the ship's reference plane, the average height feature of the pollutants is extracted and calculated. .

[0038] Among them, the average height characteristics of pollutants are calculated. The specific implementation steps are as follows: ① Preprocess point cloud data and align coordinates: Obtain the original depth map and color image (RGB) of the area to be cleaned using a front-facing depth camera; using the camera's intrinsic parameter matrix, the 2D depth map is converted into a set of point clouds P in 3D space; each point in the point cloud... Includes three-dimensional coordinates ( , , ).

[0039] ② Mathematical Fitting of Hull Reference Surface (Extraction of Reference Surface): Since the hull surface can be approximated as a plane from a local perspective, the Random Sample Consensus Algorithm (RANSAC) is used to fit the point cloud set P to a plane, removing outliers that belong to contaminants, and extracting the optimal fitting reference surface S representing the clean hull surface. The spatial geometric equation of the hull reference surface S can be expressed as: Ax + By + Cz + D = 0, where A, B, and C are the normal vector components of the reference surface, and D is the intercept constant of the plane, the value of which is related to the normal distance from the depth camera center to the hull reference surface.

[0040] ③ Mapping and Extraction of Pollutant Target Point Clouds: An improved YOLOv11 network is used to perform semantic segmentation on the RGB image, outputting a two-dimensional pixel mask for a specific pollutant (e.g., barnacles). Based on the spatial mapping relationship, this two-dimensional pixel mask is mapped to a three-dimensional point cloud set P, and a subset of target point clouds belonging only to that pollutant is extracted. Assume that the subset contains N valid points.

[0041] ④ Calculation of point normal displacement: for a subset of the target point cloud Any point P in k ( , ), calculate the spatial vertical distance (i.e., normal displacement) from this point to the hull reference plane S. The calculation formula is as follows: normal displacement This represents the absolute expansion thickness of the contaminant at the pixel location.

[0042] ⑤ Average height characteristics Calculation: Traversing the target point cloud subset For all N pixels, sum the normal displacements of all points. The arithmetic mean was calculated to obtain the average height (thickness) characteristics of this type of pollutant. The calculation formula is: Calculated It will be directly input into the graded evaluation model as a three-dimensional correction parameter for calculating the Comprehensive Cleanliness Index (CII).

[0043] (ii) Step 2: Construct a pollutant classification and evaluation model based on deep learning: Receive the feature information extracted in step 1, identify the types of pollutants and calculate the comprehensive cleanliness index (CII) to output the pollution classification results.

[0044] Step 2.1, Target Detection and Semantic Segmentation: An improved YOLOv11 series network is used to perform instance segmentation on the input image, extract the contours and features of each independent pollutant in the field of view, and generate corresponding pixel-level masks, thereby identifying the types of pollutants in the image in real time and classifying them into three categories: hard attachments, soft attachments, and biofilms.

[0045] Step 2.2: Calculate the pollution coverage rate The proportion of pixels of various pollutants in the current detection field of view is calculated using a mask counting algorithm.

[0046] Pollution coverage The detailed calculation process is as follows: ① Generation of the mask binarization matrix (data extraction): Based on the output of the YOLOv11 instance segmentation network in step 2.1, a classification index matrix with the same resolution as the original image is generated for the current field of view. For the i-th type of pollutant (e.g., hard deposits), its corresponding semantic mask is extracted and converted into a binary mask matrix M. i (u, v).

[0047] If the pixel with coordinates (u, v) belongs to the i-th type of pollutant, then M i (u, v) = 1; if the pixel with coordinates (u, v) does not belong to the i-th type of pollutant, then M i (u, v) = 0.

[0048] ② Cumulative statistics of the total number of pixels (mask counting): Using a mask counting algorithm, the binary matrix M is counted. i All elements are spatially summed. The total number of pixels N for the i-th type of pollutant is calculated. i Its mathematical expression is: , where W and H represent the width and height pixel values ​​of the current detection field of view image, respectively.

[0049] ③ Determining the total number of pixels in the effective field of view: Determine the total number of pixels in the current detection field of view. Under normal circumstances, = W×H. If there are occluded areas or invalid edges in the image, invalid pixels will be pre-removed, and only the total number of pixels within the valid observation area will be counted.

[0050] ④ Pollution coverage rate Normalization calculation: The total number of pixels N of the i-th type of pollutant i Total number of pixels in the field of view By performing a ratio calculation, the pollution coverage rate of this type of pollutant on the current work surface can be obtained. , This value reflects the concentration of a specific type of pollutant and ranges from [0, 1].

[0051] ⑤ Multi-class weighting (preparing for subsequent CII calculation): If multiple types of pollutants exist simultaneously in the field of view (e.g., i=1 represents barnacles, i=2 represents seaweed), then calculate R for each type. c1 , R c2 ,....., These ratios will be multiplied by their respective hardness weighting factors. This is used for the weighted summation of the final Comprehensive Cleanliness Index (CII).

[0052] Step 2.3: Calculate the Cleanliness Index: Based on the identification results, a weighting factor is introduced, and the overall cleanliness index of the current area is calculated using the following formula: ,in, The hardness weighting coefficients for different types of pollutants; The coverage ratio of pollutant type i; β represents the average height of the pollutants calculated from the depth image; β is the height correction parameter.

[0053] Step 2.4, Classification: Map the Comprehensive Cleanliness Index (CII) value to four operating levels: Level I (light, only fine cleaning required), Level II (moderate, rinsing + fine cleaning), Level III (heavy, rough cleaning + rinsing + fine cleaning), and Level IV (extreme, full power and slow down).

[0054] (III) Step 3, Power Adaptive Mapping and Linkage Control of Multi-level Actuators: Based on the pollution level classification results output in Step 2, match the corresponding power parameters and drive each level of cleaning mechanism to perform adaptive cleaning.

[0055] Step 3.1, Instruction Generation: The main control unit generates three sets of independent control parameters by looking up the preset power mapping table according to the Comprehensive Cleanliness Index (CII) level.

[0056] The steps for obtaining the preset power mapping table are as follows: ① Construct a standard pollution sample test library: In a controlled water tank or real dock environment, select ship hull surfaces with different species (barnacles, algae, biofilm), different coverage rates, and different thicknesses as test samples. Collect sample data using a front-mounted depth camera, and run an evaluation model on a host computer to calculate the Comprehensive Cleanliness Index (CII) for each test sample, and record the initial state.

[0057] ② Multi-gradient power traversal and cleaning test: For each sample with a specific CII value, the control system issues a traversal command, so that the coarse cleaning motor, high-pressure water pump and fine cleaning motor perform repeated cleaning tests within the safety threshold with different power combinations, different PWM duty cycles, different CAN communication control speeds and different inverter frequencies.

[0058] ③ Quantification of cleaning effect and calibration of optimal solution: After each test, the proportion of residual contamination is extracted using a rear depth camera. Finding ways to make The set of hardware control parameters that results in a value less than 5% below the acceptable threshold, with no paint damage and the lowest total energy consumption, is defined as the optimal control parameter set corresponding to the current CII value.

[0059] ④ Construct a nonlinear mapping matrix and curve fitting. Summarize a large number of sample test data, establish a three-dimensional scatter dataset with CII value as the independent variable and coarse wash motor speed, high pressure water pressure, and fine wash motor speed as the dependent variables, and form a power mapping table.

[0060] Step 3.2, Hardware-linked execution: If the Comprehensive Cleanliness Index (CII) > threshold A, control the start of the coarse cleaning motor in the coarse cleaning unit, and linearly adjust the speed of the stainless steel spiked roller brush according to the CII to break up hard shells with high torque; adjust the output pressure of the high-pressure water pump in the high-pressure rinsing unit in real time according to the density of the coarse cleaning sludge, and flush the debris away from the wall through the bidirectional nozzle group; control the fine cleaning motor in the fine cleaning unit to keep it constant or finely adjust the speed of the nylon brush to ensure that residual biofilm is removed without damaging the hull paint.

[0061] Step 3.3, Communication Protocol: The host computer sends parameters to the slave computer via serial port / CAN bus. The slave computer uses a hardware timer to generate PWM waveforms to drive each module to work. Specifically, this includes: a coarse washing motor driver that drives the stainless steel spiked roller brush of the coarse washing unit, a frequency converter that adjusts the water pump pressure of the high-pressure rinsing unit, a secondary motor driver that drives the high-density nylon brush of the fine washing unit, and a motion motor driver that controls the movement of the robot body.

[0062] (iv) Step 4, Cleaning quality closed-loop detection and compensation operation: After executing step 3, detect the residual contamination ratio of the cleaned area and feed the result back to the control system to trigger dynamic compensation, forming a complete closed-loop cleaning.

[0063] Post - inspection comparison: The rear - mounted depth camera captures images of the cleaned area in real - time, and calculates the residual pollution ratio through the same semantic segmentation algorithm. .

[0064] Residual pollution ratio The specific algorithm steps are as follows: ① Rear - mounted image data acquisition: After the cleaning mechanism (rotating brush, high - pressure water gun) finishes its operation, the rear - mounted depth camera at the robot's tail captures RGB - D image data of the cleaned area in real - time.

[0065] ② Residue semantic segmentation and extraction: Input the images captured by the rear - mounted camera into a pre - trained semantic segmentation model. This model is specifically for identifying "residual biological tissues after cleaning" and generates a binary mask (u, v) of the residual pollutants.

[0066] ③ Total number of residual pixels statistics: Count the total number of pixel points with a value of 1 in the binary mask (u, v), denoted as : = CountPixels( (u, v) == 1).

[0067] ④ Quantification calculation of the residual ratio : Calculate the ratio of the residual pollutants in the cleaned area to the total area of the detection area: , where is the total number of effective detection pixels of the rear - mounted camera.

[0068] Closed - loop logic: If the residual pollution ratio 5% < ≤ 0, it is judged as qualified, and the robot keeps moving forward at the original speed; if the residual pollution ratio [[ID=4?]]≥5%, it is judged that the cleaning is not thorough, and the self - compensation mechanism is triggered; the self - compensation mechanism is to automatically reduce the rotational speed of the moving motor, increase the cleaning duration per unit area, or command the robot to reverse and perform "reciprocating" secondary cleaning until the detection is qualified.

[0069] The present invention breaks through the limitation of single - area - ratio judgment, and proposes a comprehensive cleaning index (CII) calculation method that combines pollutant category, coverage area and thickness characteristics. Through the depth information obtained by the depth camera, three - dimensional feature compensation is performed on the image pixel ratio, a hardness weight matrix of pollutants is established, and more accurate quantitative characterization of the hull surface fouling condition is achieved. As the only decision - making source for subsequent power distribution, this evaluation system significantly improves the recognition dimension of the system for complex marine attachments.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A ship hull cleaning robot based on vision recognition, characterized in that, include: The robot body includes a main chassis, with four permanent magnet adsorption wheel sets symmetrically arranged below the main chassis, which are independently driven by four DC geared motors. The power control center integrates a host computer, a slave computer, a Raspberry Pi 5, and a power management module. The multi-dimensional sensing system includes a front-facing depth camera and a ring-shaped LED fill light array. The front-facing depth camera is connected to the Raspberry Pi Pi5, a data transfer and preprocessing node, via a USB data interface to transmit high-resolution color image signal data in real time. The multi-level collaborative cleaning modules are arranged sequentially from front to back along the central axis of the main chassis, forming a "crush-peel-polish" process chain, including a coarse cleaning unit, a high-pressure rinsing unit, and a fine cleaning unit; The closed-loop feedback system includes a rear-mounted depth camera installed at the rear of the main chassis. The lens focuses vertically downward on the cleaned area. The rear-mounted depth camera is connected to the host computer via a USB interface to transmit real-time images after cleaning back to the host computer's evaluation model, which serves as the basis for deciding whether the robot needs to reduce its speed or perform reciprocating cleaning. The front-facing depth camera transmits data to the host computer, which runs the YOLO detection algorithm and power allocation algorithm. After calculating the power parameters for each stage, the host computer sends the command packet to the Raspberry Pi PI5 via the CAN bus. The Raspberry Pi PI5 transmits the information to the lower-level computer and parses the command packet, outputting multiple PWM / analog signals to the motor drivers and water pump inverters at each stage. The data from the rear-facing depth camera is transmitted back to the host computer. If the host computer determines that the cleanliness is unqualified, it sends a deceleration command to the moving motor, forming a complete visual feedback closed-loop control.

2. A power adaptive control method for a ship hull cleaning robot based on vision recognition, implemented based on the ship hull cleaning robot based on vision recognition as described in claim 1, characterized in that, Includes the following steps: Step 1: Multidimensional Environmental Image Acquisition and Depth Feature Preprocessing: Obtain the 3D point cloud model and image features of the area to be cleaned, and calculate the average height features of the pollutants. And use it as input data for pollution assessment; Step 2: Construct a pollutant classification and evaluation model based on deep learning: Receive the feature information extracted in Step 1, identify the types of pollutants, calculate the Comprehensive Cleanliness Index (CII), and output the pollution classification results; Step 3: Power adaptive mapping and linkage control of multi-level actuators: Based on the pollution level classification results output in Step 2, match the corresponding power parameters and drive each level of cleaning mechanism to perform adaptive cleaning; Step 4, Cleaning quality closed-loop detection and compensation operation: After performing step 3, the residual contamination ratio of the cleaned area is detected, and the result is fed back to the control system to trigger dynamic compensation, forming a complete closed-loop cleaning.

3. The power adaptive control method for a ship hull cleaning robot based on vision recognition according to claim 2, characterized in that, Step 1 specifically includes: Image acquisition: With the assistance of an LED fill light array, the depth camera at the front end of the robot acquires color images and depth information images of the area to be cleaned in real time; Preprocessing: Median filtering and illumination compensation correction are performed on the acquired images to eliminate the interference of the underwater environment on image contrast; Spatial mapping: Depth information is pixel-level aligned with RGB images to construct a 3D point cloud model of the surface to be cleaned. By calculating the normal displacement of each pixel in the 3D point cloud model relative to the ship's reference plane, the average height feature of the pollutants is extracted and calculated. .

4. The power adaptive control method for a ship hull cleaning robot based on visual recognition according to claim 3, characterized in that, Calculate the average height characteristics of pollutants The specific implementation steps are as follows: ① Preprocess point cloud data and align coordinates: Obtain the original depth map and color image of the area to be cleaned through the front-facing depth camera, and use the camera's intrinsic parameter matrix to convert the two-dimensional depth map into a set of point clouds P in three-dimensional space; ② Mathematical fitting of hull reference: The point cloud set P is fitted with a random sampling consensus algorithm to remove outlier points that belong to pollutants and extract the optimal fitting reference surface S representing the clean hull surface. ③ Mapping and extraction of pollutant target point clouds: The improved YOLOv11 network is used to perform semantic segmentation on the RGB image, outputting a two-dimensional pixel mask for a specific pollutant; based on the spatial mapping relationship, the two-dimensional pixel mask is mapped to the three-dimensional point cloud set P, and the target point cloud subset belonging only to the pollutant is extracted. ; ④ Calculation of point normal displacement: for a subset of the target point cloud Any point P in k ( , Calculate the vertical spatial distance from this point to the hull reference plane S, i.e., the normal displacement. ; ⑤ Average height characteristics Calculation: Traversing the target point cloud subset For all N pixels, sum the normal displacements of all points. The arithmetic mean was calculated to obtain the average height characteristics of this type of pollutant. .

5. The power adaptive control method for a ship hull cleaning robot based on vision recognition according to claim 2, characterized in that, Step 2 specifically includes: Step 2.1, Target Detection and Semantic Segmentation: An improved YOLOv11 series network is used to perform instance segmentation on the input image, extract the contours and features of each independent pollutant in the field of view, and generate corresponding pixel-level masks, so as to identify the types of pollutants in the image in real time, including three types: hard attachments, soft attachments and biofilms. Step 2.2: Calculate the pollution coverage rate The proportion of pixels of various contaminants in the current detection field of view is calculated using a mask counting algorithm. Step 2.3: Calculate the Cleanliness Index: Based on the identification results, introduce weighting factors to calculate the Comprehensive Cleanliness Index (CII) for the current area; Step 2.4, Level Classification: Map the Comprehensive Cleanliness Index (CII) value to four operational levels: Level I, Level II, Level III, and Level IV.

6. The power adaptive control method for a ship hull cleaning robot based on vision recognition according to claim 5, characterized in that, In step 2.2, the pollution coverage rate The calculation process is as follows: ① Generation of mask binarization matrix: Based on the output of the YOLOv11 instance segmentation network in step 2.1, a classification index matrix with the same resolution as the original image is generated; For the i-th type of pollutant, extract its corresponding semantic mask and convert it into a binary mask matrix M. i (u, v): ② Cumulative statistics of the total number of pixels: Using a mask counting algorithm, spatial accumulation is performed on all elements in the binary matrix; ③ Determining the total number of pixels in the effective field of view: Determine the total number of pixels in the current detection field of view. If there are occluded areas or invalid edges in the image, invalid pixels will be removed in advance, and only the total number of pixels in the valid observation area will be counted. ④ Pollution coverage rate Normalization calculation: The total number of pixels N of the i-th type of pollutant i Total number of pixels in the field of view The ratio is calculated to obtain the coverage ratio of this type of pollutant on the current work surface. This value reflects the concentration of a specific type of pollutant, and its range is between [0, 1]. ⑤ Multi-class weighted composition: If multiple types of pollutants exist simultaneously in the field of view, then R is calculated separately for each class. c1 , R c2 ,....., ; calculate R c1 , R c2 ,....., Multiply by their respective hardness weighting factors This is used for the weighted summation of the final Comprehensive Cleanliness Index (CII).

7. The power adaptive control method for a ship hull cleaning robot based on vision recognition according to claim 6, characterized in that, If the pixel with coordinates (u, v) belongs to the i-th type of pollutant, then M i (u, v) = 1; if the pixel with coordinates (u, v) does not belong to the i-th type of pollutant, then M i (u, v) = 0.

8. The power adaptive control method for a ship hull cleaning robot based on visual recognition according to claim 2, characterized in that, Step 3 specifically includes: Step 3.1, Instruction Generation: The main control unit generates three sets of independent control parameters by looking up the preset power mapping table according to the Comprehensive Cleanliness Index (CII) level. Step 3.2, Hardware Linkage Execution: If the Comprehensive Cleanliness Index (CII) > Threshold A, control the coarse washing motor of the coarse washing unit to start, and linearly adjust the speed of the stainless steel spiked roller brush according to the CII to break up hard shells with high torque; according to the density of the coarse washing sludge, adjust the output pressure of the high-pressure water pump of the high-pressure rinsing unit in real time to flush the debris away from the wall through the bidirectional nozzle group; control the fine washing motor of the fine washing unit to keep it constant or finely adjust the speed of the nylon brush to remove residual biofilm; Step 3.3, Communication Protocol: The host computer sends parameters to the slave computer via serial port / CAN bus. The slave computer uses a hardware timer to generate PWM waveforms to drive each module to work. Specifically, this includes: a coarse washing motor driver that drives the stainless steel spiked roller brush of the coarse washing unit, a frequency converter that adjusts the water pump pressure of the high-pressure rinsing unit, a secondary motor driver that drives the high-density nylon brush of the fine washing unit, and a motion motor driver that controls the movement of the robot body.

9. The power adaptive control method for a ship hull cleaning robot based on vision recognition according to claim 2, characterized in that, Step 4 specifically includes: Post-inspection comparison: The rear depth camera acquires images of the cleaned area in real time, and the proportion of residual contamination is calculated using the same semantic segmentation algorithm. ; Closed-loop logic: If the residual pollution ratio is 5% < ≤ 0, it is judged as qualified, and the robot maintains its original speed; if the residual pollution ratio ≥ 5%, it is judged that the cleaning is incomplete, and the self-compensation mechanism is triggered; the self-compensation mechanism is to automatically reduce the speed of the moving motor, increase the cleaning duration per unit area, or command the robot to retreat and perform "reciprocating" secondary cleaning until the detection is qualified.

10. The power adaptive control method for a ship hull cleaning robot based on vision recognition according to claim 9, characterized in that, residual pollution ratio The specific algorithm steps are as follows: ① Rear image data acquisition: After the cleaning mechanism has finished its work, the rear depth camera at the tail of the robot is used to acquire RGB-D image data of the cleaned area in real time; ② Semantic segmentation and extraction of residues: The images acquired by the rear depth camera are input into the pre-trained semantic segmentation model to identify "residual biological tissue after cleaning" and generate a binary mask of residual pollutants. (u, v); ③Statistical analysis of the total number of residual pixels: statistical analysis of the binarized mask. The total number of pixels with a value of 1 in (u, v) is denoted as . ; ④ Residual pollution ratio Quantitative calculation: Calculate the proportion of residual contaminants in the cleaned area to the total area of ​​the detected area: ,in, This represents the total number of effective detection pixels for the rear camera.

Images