Ship cleaning robot path guiding method and system based on binocular vision

Abstract

The invention discloses a ship cleaning robot path guiding method and system based on binocular vision, and the method comprises the steps: installing an underwater binocular camera on a ship cleaning robot, collecting an underwater image, uploading the underwater image to a shore-based display control unit through an umbilical cable, carrying out the preprocessing, carrying out the operation of the preprocessed image through two threads, pixel coordinates of the fitting straight line and three-dimensional point cloud data of the surface of the ship body are obtained; and obtaining three-dimensional space coordinates of the fitting straight line and the reference straight line relative to the robot by combining the pixel coordinates of the fitting straight line and the reference straight line and the three-dimensional point cloud data of the hull surface, obtaining a deviation angle and distance of the robot relative to the reference straight line through coordinate transformation, and controlling a left track and a right track of the robot to generate a speed difference for steering. And completing path guidance of the robot. Aiming at the problems existing in manual operation, the low-cost binocular camera is used for guiding the robot to advance along the boundary of the cleaning area and the uncleaned area, and full-coverage cleaning of the surface of the ship is conveniently completed.

Description

Technical Field

[0001] The present invention relates to the technical field of ship cleaning robot control, and more specifically, to a path guidance method and system for a ship cleaning robot based on binocular vision. Background Art

[0002] A ship cleaning robot is an automated machine designed to solve the problems of low efficiency, high labor intensity, and high danger in traditional manual ship cleaning. The purpose is to give full play to the advantages of the robot and completely liberate people from this heavy work. Currently, it is being increasingly applied in practice and has broad market prospects. How to achieve automatic cleaning of ships is one of the research focuses.

[0003] Due to the complex underwater environment and the influence of factors such as tides, waves, and undercurrents, it is very difficult for a ship cleaning robot to maintain its own posture during underwater operations. It is often pushed away from its original position or changes direction during movement, resulting in discontinuous and incomplete cleaning areas.

[0004] Existing ship cleaning robots usually adopt a manual control mode when performing cleaning operations, that is, an operator controls the robot to perform cleaning operations through a console on the shore. Although this cleaning mode can ensure the cleaning rate, it requires the operator to spend a great deal of energy and the cleaning effect depends on the proficiency of the operator, which cannot give full play to the advantages of the cleaning robot and greatly restricts the cleaning efficiency and large-scale application of the cleaning robot.

[0005] Therefore, aiming at the problem that a ship cleaning robot cannot navigate automatically during the cleaning process, it is urgent to provide a path guidance method and system to guide the path of the ship cleaning robot. Summary of the Invention

[0006] In view of the above technical problems, the present invention provides a path guidance method and system for a ship cleaning robot based on binocular vision, which is used to guide the path of the ship cleaning robot so that it can automatically adjust after deviating from its original posture, thereby achieving the purpose of automatically cleaning the entire surface of the ship.

[0007] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] In the first aspect, the present invention provides a path guidance method for a ship cleaning robot based on binocular vision, and the method includes the following steps:

[0009] S1: Install an underwater binocular camera on the ship cleaning robot, collect underwater images and upload them to the shore-based display and control unit through an umbilical cable;

[0010] S2: The shore-based display and control unit preprocesses the image, and operates on the preprocessed image in two threads to respectively obtain the pixel coordinates of the fitted straight line and the three-dimensional point cloud data of the hull surface;

[0011] S3: Combine the pixel coordinates of the fitted straight line and the reference straight line and the three-dimensional point cloud data of the hull surface to obtain the three-dimensional space coordinates of the fitted straight line and the reference straight line relative to the ship cleaning robot, and obtain the offset angle and distance of the ship cleaning robot relative to the reference straight line through coordinate transformation, and send them to the controller of the ship cleaning robot;

[0012] S4: The controller of the ship cleaning robot performs calculations based on the offset angle and distance, controls the left and right tracks to generate a speed difference for steering, and completes the path guidance of the ship cleaning robot.

[0013] Further, in S1, the underwater binocular camera is sealed in a waterproof housing and installed in the middle position of the ship cleaning robot; the underwater binocular camera consists of two color cameras, with a resolution of 720×480 and a frame rate of 15FPS, and obtains synchronized images in a hard trigger mode.

[0014] Further, in S2, the preprocessing process includes color restoration and defogging of the underwater image, and then the respectively processed images are fused. The specific method includes:

[0015] For color restoration, a gray-world-based white balance algorithm is used. First, the color image is separated into R, G, and B three channels, and the average value of the pixels of the image to be processed is calculated as:

[0016]

[0017] where K is the average value of the pixels of the image to be processed, R avg 、G avg 、B avg are the average values of the pixels of the three channels;

[0018] The gains of the R, G, and B three color channels relative to the average value K of the pixels of the image to be processed are respectively:

[0019]

[0020]

[0021] Then, multiply the pixel values of each of the three-channel images by the relative gains of each channel to obtain a new three-channel image, and then fuse the new three-channel images to obtain the processed color image;

[0022] For the dehazing process, an improved dehazing algorithm based on the dark channel prior theory is used to perform dehazing operations on the image. When calculating the dark channel of the image, only the blue and green channels are used for calculation, and its expression is:

[0023]

[0024] In the formula, J is the image channel, Ω(x) represents the region centered on pixel point x, and y represents any pixel point within Ω(x);

[0025]

[0026] In the formula, t(x) is the transmittance, I is the image obtained by the camera, and A is the ambient background light;

[0027] The dehazed underwater image J(x) is:

[0028]

[0029] After image processing, the Laplacian pyramid algorithm is used to decompose the image to obtain the Laplacian pyramid of the image. Suppose there are N layers of images in total, and the L-th layer image is expressed as:

[0030]

[0031] Among them, P L represents the L-th layer image of the Laplacian pyramid of the image, and G L is the L-th layer image of the Gaussian pyramid of the image, which is expressed as:

[0032] G L = w(m,n)Down(G L-1 )

[0033] Among them, w(m,n) represents the Gaussian kernel convolution, and Down(G L-1 ) represents downsampling the image G L-1 , and the length and width of the sampled image are reduced to half of the original;

[0034] G' L+1 represents the image after upsampling the image G L , and its expression is:

[0035] G’ L+1 = Up(G L )w(m,n)

[0036] Among them, Up(G L ) represents upsampling the image G L ;

[0037] When calculating the Laplacian pyramid of an image, the saturation weight maps W' 1 and W' 2 are extracted from the images processed by the two algorithms simultaneously, the color weight maps W'' 1 and W'' 2 , and the saliency weight maps W'' 1 and W'' 2 . Then, the three weight maps are normalized to obtain the normalized images The calculation method is as follows:

[0038]

[0039] For the normalized images Gaussian filtering and downsampling are performed to obtain a Gaussian pyramid with the same number of layers as the Laplacian pyramid. The L-th layer is denoted as

[0040] After obtaining the two pyramids of the image, each layer of the image is fused to obtain the fused image. The fused image M of the L-th layer L is denoted as:

[0041]

[0042] Finally, the image pyramid obtained by layer-by-layer fusion is inversely deduced and reconstructed layer by layer to obtain the final image.

[0043] Furthermore, in S2, the preprocessed image is operated on in two threads to respectively obtain the pixel coordinates of the fitted straight line and the three-dimensional point cloud data of the hull surface, where:

[0044] One thread takes the left-eye image as the input, extracts the boundary between the cleaned area and the uncleaned area through edge detection, then fits the boundary pixels into a straight line through straight line fitting, and records the pixel coordinates of this fitted straight line;

[0045] The other thread takes the left-eye and right-eye images as the input simultaneously, performs stereo rectification on the images using the underwater binocular camera parameters calibrated by the equivalent focal length method, and uses the SGBM stereo matching algorithm for stereo matching to obtain the depth map of the hull surface, and then performs three-dimensional reconstruction on the hull surface to obtain the three-dimensional point cloud data of the hull surface.

[0046] Furthermore, the underwater camera calibration work is divided into two steps. First, the camera is placed in a waterproof housing for on-water calibration work to obtain the camera internal parameter matrix as:

[0047]

[0048] The external parameter matrix is:

[0049]

[0050] In the formula, f x , f y are the equivalent focal lengths; (u 0 , v 0 ) are the principal point coordinates; R and t are respectively the rotation matrix and translation matrix from the right-eye camera to the left-eye camera;

[0051] Then, the refraction generated by the light at the water-air interface is equivalent to the change in the camera focal length. Multiply the camera focal length in air by the refractive index ratio of water to air, 1.33, to obtain the underwater camera focal length. Thus, replace the camera focal length obtained in the first step to get the underwater camera internal parameter matrix as:

[0052]

[0053] That is, the parameters of the underwater camera are obtained.

[0054] In a second aspect, the present invention also provides a path guidance system for a ship cleaning robot based on binocular vision. The system includes: an underwater binocular camera, an umbilical cable, and a shore-based display and control unit; and applies the above-mentioned path guidance method for a ship cleaning robot based on binocular vision to guide the path of the ship cleaning robot.

[0055] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0056] The present invention identifies and locates the boundary between the cleaned area and the uncleaned area through an underwater binocular camera installed on the ship cleaning robot, and then calculates the deviation during the movement of the ship cleaning robot, guides the path of the ship cleaning robot, enables it to automatically adjust after deviating from the original posture, and makes the ship cleaning robot move along the boundary to achieve full-coverage cleaning of the ship surface.

[0057] Other features and advantages of the present invention will be described in the subsequent specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in the written specification and the drawings.

[0058] Next, through the drawings and embodiments, the technical solutions of the present invention will be further described in detail. Description of the Drawings

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0060] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention.

[0061] Figure 1 Schematic flow diagram of a path guidance method for a ship cleaning robot based on binocular vision provided by an embodiment of the present invention.

[0062] Figure 2 Schematic diagram of the cleaning operation of a ship cleaning robot provided by an embodiment of the present invention.

[0063] Figure 3 Schematic flow diagram of binocular guidance provided by an embodiment of the present invention.

[0064] Figure 4 Schematic diagram of image preprocessing provided by an embodiment of the present invention.

[0065] Figure 5 Schematic diagram of the boundary straight line fitting effect provided by an embodiment of the present invention.

[0066] In the figure, 1 is the shore-based display and control unit; 2 is the umbilical cable; 3 is the ship cleaning robot; 4 is the underwater binocular camera; 5 is the ship surface. Detailed implementation manners

[0067] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention.

[0068] In the description of the present invention, it should be noted that in some processes described in the specification and drawings of this application, there are multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. In addition, various serial numbers, etc. are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0069] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.

[0070] See Figure 1As shown in the figure, in view of the problems existing in manual operation, an embodiment of the present invention provides a path guidance method for a ship cleaning robot based on binocular vision. The method mainly includes the following steps:

[0071] S1: Install an underwater binocular camera on the ship cleaning robot, collect underwater images and upload them to the shore-based display and control unit through the umbilical cable;

[0072] S2: The shore-based display and control unit preprocesses the images, and operates on the preprocessed images in two threads, respectively obtaining the pixel coordinates of the fitted straight line and the three-dimensional point cloud data of the hull surface;

[0073] S3: Combine the coordinates of the fitted straight line pixels, the reference straight line pixels, and the three-dimensional point cloud data of the hull surface to obtain the three-dimensional space coordinates of the fitted straight line and the reference straight line relative to the ship cleaning robot, and obtain the offset angle and distance of the ship cleaning robot relative to the reference straight line through coordinate transformation, and send them to the controller of the ship cleaning robot;

[0074] S4: The controller of the ship cleaning robot performs calculations based on the offset angle and distance, controls the left and right tracks to generate a speed difference for steering, and completes the path guidance of the ship cleaning robot.

[0075] The following combines Figures 1 - 5 As shown in the figure, the working principle and implementation method of the present invention are introduced in detail:

[0076] In the embodiment of the present invention, as Figure 2 shown in the figure, the underwater binocular camera 4 is sealed in a waterproof housing and installed in the middle position of the ship cleaning robot 3. The ship cleaning robot 3 performs cleaning operations on the ship surface 5. Specifically, the underwater binocular camera 4 is composed of two color cameras, with a resolution of 720×480 and a frame rate of 15FPS. It obtains synchronized images in a hard trigger mode, and uploads the obtained images to the upper computer software of the shore-based display and control unit 1 through the network communication of the umbilical cable 2. The CPU model of the shore-based display and control unit 1 is AMD R7-7435H, and the operating system is ubuntu22.04.

[0077] See Figure 3 and Figure 4 As shown in the figure, after receiving the images, the shore-based display and control unit first preprocesses the images, including color restoration and dehazing processing of underwater images, which are used to restore the true colors of underwater images and eliminate impurities diffused into the water during the cleaning process; then fuses the two processed images respectively to improve the image quality and facilitate subsequent binocular matching and three-dimensional reconstruction work. Among them:

[0078] For color restoration, a white balance algorithm based on the gray world is used. First, the color image is separated into R, G, and B three channels, and then the pixel average values of the three channels are calculated as Ravg , G avg , B avg . After obtaining the average pixel values of each channel, the average pixel value of the image to be processed can be calculated as:

[0079]

[0080] Furthermore, the gains of the R, G, and B color channels relative to the average pixel value K of the image to be processed are calculated as:

[0081]

[0082] Then, multiply the pixel values of each of the three-channel images by their respective relative gains, and take 255 for pixel values greater than 255, so as to obtain new three-channel images, and then fuse the new three-channel images to obtain the processed color image.

[0083] For the dehazing process, an improved dehazing algorithm based on the dark channel prior theory is used to perform dehazing operations on the image to remove the impurities diffused into the water during the cleaning process. Different from the air, since water has the strongest absorption effect on red light and the attenuation of red light underwater is the most serious, in most cases, the dark channel calculated directly using the dark channel algorithm in the air is the red channel, and the dehazing effect is not good. Therefore, in the present invention, only the blue and green channels are used to calculate the dark channel of the image, and its expression is:

[0084]

[0085] In the formula, J is the image channel, and Ω(x) represents the region centered on the pixel point x.

[0086] The transmittance t(x) is:

[0087]

[0088] In the formula, I is the image obtained by the camera, and A is the ambient background light.

[0089] Thus, the dehazed underwater image J(x) is obtained:

[0090]

[0091] Furthermore, after image processing, the Laplacian pyramid algorithm is used to decompose the image to obtain the Laplacian pyramid of the image. Assuming there are N layers of images, the L-th layer of the image is expressed as:

[0092]

[0093] Among them, P L represents the L-th layer of the Laplacian pyramid of the image; G LIt is the L-th layer image of the Gaussian pyramid of the image, expressed as:

[0094] G L = w(m,n)Down(G L-1 )

[0095] where w(m,n) represents the Gaussian kernel convolution, and Down(G L-1 ) represents downsampling the image G L-1 , and after sampling, the length and width of the image are reduced to half of the original.

[0096] G' L+1 represents the image after upsampling the image G L , and the expression is:

[0097] G' L+1 = Up(G L )w(m,n)

[0098] where Up(G L ) represents upsampling the image G L .

[0099] In this embodiment, when calculating the Laplacian pyramid of the image, the saturation weight maps W' 1 , W' 2 , the color weight maps W” 1 , W” 2 , and the saliency weight maps W”’ 1 , W”' 2 are extracted from the images processed by the two algorithms at the same time, and then the three weight maps are normalized to obtain the normalized images The calculation method is:

[0100]

[0101] Furthermore, the normalized images are subjected to Gaussian filtering and downsampling to obtain a Gaussian pyramid with the same number of layers as the Laplacian pyramid. The L-th layer is expressed as

[0102] After obtaining the two pyramids of the image, each layer of the image is fused to obtain the fused image. The fused image M of the L-th layer L can be expressed as:

[0103]

[0104] Finally, the image pyramid obtained by layer-by-layer fusion is inversely deduced and reconstructed layer by layer to obtain the final image.

[0105] In a specific embodiment, after preprocessing, the image is processed synchronously in two threads. One thread takes the left-eye image as input, and the second thread takes both the left-eye and right-eye images as input simultaneously.

[0106] The first thread first performs boundary detection on the left-eye image to obtain the boundary between the cleaned area and the uncleaned area, then performs linear fitting on the obtained boundary to obtain the fitted line of the boundary, and the effect diagram is as Figure 5 shown. Finally, the pixel coordinates of the fitted line in the left-eye image are found and recorded.

[0107] After receiving the left and right images, the second thread performs stereo rectification on the underwater images according to the pre-calibrated camera parameters to eliminate image distortion. After stereo rectification, the SGBM stereo matching algorithm is used to perform stereo matching on the left and right images, and then three-dimensional reconstruction is performed on the images with the left-eye image as the reference to obtain the three-dimensional point cloud of the hull surface.

[0108] Among them, the underwater camera calibration work is divided into two steps. First, the camera is placed in a waterproof housing for on-water calibration work, and the camera internal parameter matrix obtained is:

[0109]

[0110] The external parameter matrix is:

[0111]

[0112] In the formula, f x , f y are called the equivalent focal lengths; (u 0 , v 0 ) are the principal point coordinates; R and t are the rotation matrix and translation matrix of the right-eye camera relative to the left-eye camera respectively. At the same time, the third-order radial distortion coefficients k 1 , k 2 , k 3 and the second-order tangential distortion coefficients p 1 , p 1 are obtained through camera calibration to correct image distortion.

[0113] Then, the refraction of light at the water-air interface is equivalent to the change in the camera focal length. The camera focal length in air is multiplied by the refractive index ratio of water to air, 1.33, to obtain the underwater camera focal length. Thus, the camera focal length obtained in the first step is replaced to obtain the underwater camera internal parameter matrix as:

[0114]

[0115] In this way, all the parameters of the underwater camera are obtained.

[0116] Combining the pixel coordinates of the boundary line (fitted line) and the reference line (the center line in the image, representing the traveling direction of the robot) in the left-eye image and the 3D point cloud data of the image, the 3D spatial points of the boundary line and the reference line in space relative to the camera can be obtained.

[0117] Through the pre-measured relative position of the binocular camera and the robot, coordinate transformation is performed to obtain the true positions of the boundary line and the reference line relative to the robot, thereby calculating the offset angle and offset distance between the boundary line and the forward direction of the robot.

[0118] After obtaining the offset, it is sent to the robot controller through the umbilical cable for operation through the PID algorithm, thereby controlling the speed difference between the left and right tracks of the robot to perform steering, so that the robot moves along the boundary direction all the time, completing the automatic navigation of the cleaning robot.

[0119] From the description of the above embodiments, those skilled in the art can know that: the embodiments of the present invention provide a path guidance method for a ship cleaning robot based on binocular vision, which is used to guide the path of the ship cleaning robot so that it can automatically adjust after deviating from the original posture; in this method, a low-cost underwater binocular camera is installed on the ship cleaning robot to identify and locate the boundary between the cleaned area and the uncleaned area, and then calculate the deviation during the traveling process of the ship cleaning robot, so that it can automatically adjust after deviating from the original posture, and make the ship cleaning robot travel along the boundary to achieve full-coverage cleaning of the ship surface.

[0120] Furthermore, the embodiments of the present invention also provide a path guidance system for a ship cleaning robot based on binocular vision. This system mainly includes: an underwater binocular camera, an umbilical cable, and a shore-based display and control unit; applying the path guidance method for a ship cleaning robot based on binocular vision in the above embodiments to guide the path of the ship cleaning robot, so that the robot moves along the boundary direction all the time, completing the automatic navigation of the cleaning robot.

[0121] For the path guidance system for a ship cleaning robot based on binocular vision provided by the embodiments of the present invention, its implementation principle and the technical effects generated are the same as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in this embodiment, reference can be made to the corresponding content in the foregoing method embodiments, and details will not be repeated here.

[0122] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems or computer program products, etc. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0123] It should be noted that the word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware including several different components and by means of a suitably programmed computer.

[0124] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other.

[0125] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein.

[0126] Rather, it is to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A path guidance method for a ship cleaning robot based on binocular vision, characterized in that: The method comprises the following steps: S1: Install an underwater binocular camera on the ship cleaning robot to collect underwater images and upload them to the shore-based display and control unit via an umbilical cable; S2: The shore-based display and control unit preprocesses the image, and operates the preprocessed image in two threads to obtain the pixel coordinates of the fitting line and the three-dimensional point cloud data of the hull surface respectively; S3: combining the pixel coordinates of the fitting straight line and the reference straight line and the three-dimensional point cloud data of the hull surface, obtaining the three-dimensional spatial coordinates of the fitting straight line and the reference straight line relative to the ship cleaning robot, and obtaining the offset angle and distance of the ship cleaning robot relative to the reference straight line through coordinate transformation, and sending them to the controller of the ship cleaning robot; S4: The controller of the ship cleaning robot performs calculations based on the offset angle and the distance, controls the left and right crawlers to generate a speed difference for steering, and completes the path guidance of the ship cleaning robot.

2. A path guidance method for a ship cleaning robot based on binocular vision according to claim 1, characterized in that: In S1, the underwater binocular camera is sealed in a waterproof housing and installed in the middle of the ship cleaning robot; the underwater binocular camera consists of two color cameras with a resolution of 720×480 and a frame rate of 15FPS, and a hard trigger method is used to obtain synchronized images.

3. A path guidance method for a ship cleaning robot based on binocular vision according to claim 1, characterized in that: In S2, the preprocessing process includes color restoration and defogging of underwater images, and then the processed images are fused. The specific method includes: Color restoration uses a white balance algorithm based on the grayscale world. First, the color image is separated into three channels: R, G, and B. The average pixel value of the image to be processed is calculated as: Where K is the average pixel value of the image to be processed, R avg , G avg , B avg is the pixel average of the three channels; The gains of the three color channels R, G, and B relative to the average pixel value K of the image to be processed are: Then, the pixel values ​​of each of the three-channel images are multiplied by the relative gain of each channel to obtain a new three-channel image, and then the new three-channel image is fused to obtain a processed color image; The defogging process uses an improved defogging algorithm based on dark channel prior theory to perform defogging operations on the image. When calculating the dark channel of the image, only the blue and green channels are used for calculation. The expression is: Where J is the image channel, Ω(x) represents the area centered on pixel x, and y represents any pixel in Ω(x); Where t(x) is the transmittance, I is the image obtained by the camera, and A is the ambient background light; The underwater image J(x) after dehazing is: After image processing, the Laplacian pyramid algorithm is used to decompose the image to obtain the image Laplacian pyramid; suppose there are N layers of images, and the Lth layer image is expressed as: Among them, P L represents the Lth layer image of the image Laplacian pyramid; G L is the L-th layer image of the image Gaussian pyramid, expressed as: G L =w(m,n)Down(G L-1 ) Among them, w(m,n) represents Gaussian kernel convolution, Down(G L-1 ) represents the image G L-1 Downsampling is performed, and the length and width of the image are reduced to half of the original after sampling; G' L+1 Represents the image G L The expression of the up-sampled image is: G' L+1 =Up(G L )w(m,n) Among them, Up(G L ) represents the image G L Perform upsampling; When calculating the image Laplacian pyramid, the saturation weight map W'1, W'2, the color weight map W"1, W"2, and the saliency weight map W"'1, W"'2 are extracted from the images processed by the two algorithms at the same time, and then the three weight maps are normalized to obtain the normalized image The calculation method is: After normalization, the image Gaussian filtering and downsampling are performed to obtain a Gaussian pyramid with the same number of layers as the Laplace pyramid. The Lth layer is represented as After obtaining the two image pyramids, fuse each layer of images to obtain the fused image. The L-th layer fused image M L It is expressed as: Finally, the image pyramid obtained after layer-by-layer fusion is reversely deduced and reconstructed layer by layer to obtain the final image.

4. A path guidance method for a ship cleaning robot based on binocular vision according to claim 1, characterized in that: In S2, the preprocessed image is operated in two threads to obtain the pixel coordinates of the fitting line and the three-dimensional point cloud data of the hull surface respectively, wherein: One thread takes the left image as input, extracts the boundary between the cleaned area and the uncleaned area through boundary detection, then fits the boundary pixels into a straight line through straight line fitting, and records the pixel coordinates of the fitted line; The two threads take the left and right images as input at the same time, perform stereo correction on the images through the underwater binocular camera parameters calibrated by the equivalent focal length method, and use the SGBM stereo matching algorithm for stereo matching to obtain the depth map of the hull surface, and then reconstruct the hull surface in three dimensions to obtain the three-dimensional point cloud data of the hull surface.

5. A path guidance method for a ship cleaning robot based on binocular vision according to claim 4, characterized in that: The underwater camera calibration work is divided into two steps. First, the camera is placed in a waterproof housing for water calibration, and the camera internal parameter matrix is ​​obtained as: The external parameter matrix is: In the formula, f x 、f y is the equivalent focal length; (u0, v0) is the principal point coordinate; R and t are the rotation matrix and translation matrix from the right camera to the left camera respectively; Then the refraction of light at the water-air interface is equivalent to the change in the camera focal length. The camera focal length in the air is multiplied by the ratio of the refractive index of water to air 1.33 to obtain the underwater focal length of the camera. The camera focal length obtained in the first step is replaced to obtain the internal parameter matrix of the underwater camera: That is, obtain the parameters of the underwater camera.

6. A ship cleaning robot path guidance system based on binocular vision, characterized in that: The system comprises: an underwater binocular camera, an umbilical cable and a shore-based display and control unit; and uses a binocular vision-based ship cleaning robot path guidance method as described in any one of claims 1 to 5 to guide the path of the ship cleaning robot.

Images