Real-time panoramic imaging method and device for underwater cleaning robot

By real-time detection of the posture of the cleaning robot on the marine catheter and adjusting the texture mapping relationship, the problem of poor panoramic imaging of the existing underwater cleaning robot is solved, more efficient and accurate panoramic imaging is achieved, and the robot's obstacle crossing and crossing performance is improved.

CN114742941BActive Publication Date: 2025-06-20ZHEJIANG UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210264457.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-17
Publication Date
2025-06-20
Estimated Expiration
2042-03-17

AI Technical Summary

Technical Problem

Existing underwater cleaning robots have poor panoramic imaging effects in marine catheter-type scenarios, small field of view and easy to fall off in case of surges, and limited obstacle crossing and cross-tube performance.

Method used

The real-time panoramic imaging method is adopted to detect the posture of the cleaning robot on the pipeline in real time, and adjust the texture mapping relationship in real time according to the posture, and correctly map the image texture to the projection surface to achieve panoramic imaging.

Benefits of technology

It improves the panoramic imaging efficiency and accuracy of marine catheter frame scenes, enhances the obstacle and transpiration performance of cleaning robots, and reduces the risk in surge situations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114742941B_ABST
    Figure CN114742941B_ABST
Patent Text Reader

Abstract

The present invention discloses a real-time panoramic imaging method and device for an underwater cleaning robot, including: constructing a three-dimensional projection surface for panoramic imaging based on the working scenario of the cleaning robot on an ocean conduit; acquiring real-time multi-angle images collected by a camera on the cleaning robot, and stitching the multi-angle images to obtain a stitched panoramic image; performing conduit edge detection on the stitched panoramic image to obtain a real-time conduit edge line; constructing a transformation equation based on the pixel points on the real-time conduit edge line, the homography matrix, and the pixel points on the reference conduit edge line, and solving the transformation equation to obtain the real-time rotation angle included in the homography matrix; retrieving a texture mapping matrix corresponding to the real-time rotation angle from a texture mapping library according to the real-time rotation angle, and mapping the stitched image onto the three-dimensional projection surface based on the texture mapping matrix to achieve panoramic imaging. This method and device can improve the efficiency and accuracy of panoramic imaging in scenarios such as ocean jacket structures.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of panoramic imaging, and particularly relates to a real-time panoramic imaging method and device for an underwater cleaning robot. Background Art

[0002] Offshore oil platforms have seen significant development as important equipment for exploiting offshore oil and gas resources. Jacket platforms are an indispensable support structure for most offshore oil platforms. Being long-term immersed in seawater, they become a breeding ground for marine organisms. A large number of marine organisms such as oysters and barnacles grow on the surface of the jacket platforms. These marine organisms attach to the surface of the jacket platforms and, during their metabolism, produce various chemical substances, among which acidic substances are highly corrosive to the jacket platforms, reducing their service life. To mitigate the adverse effects of marine organisms on the jacket platforms, it is crucial to regularly clean the marine organisms attached to the surface of the jacket platforms of offshore oil platforms.

[0003] Underwater cleaning robots can achieve regular cleaning of the marine organisms attached to the surface of the jacket platforms of offshore oil platforms. However, the existing underwater cleaning robots adopt a reflective panoramic imaging system, which obtains image information such as the pose of the robot in the pipeline by installing a mirror on the top of the cleaning robot. But this method has encountered great problems in practice. Firstly, the field of view that the robot can see through the mirror is very small, and it can only observe the image information within a range of 20 cm around the machine. Secondly, the method of installing a mirror on the top of the cleaning robot will increase the overall height of the machine. In the case of surges, the cleaning robot is subjected to a large impact force, easily causing dangerous situations such as the robot detaching from the wall surface and falling into the sea. The obstacle-crossing performance and pipe-crossing performance of the robot also have great limitations.

[0004] Three-dimensional panoramic imaging technology is adopted to replace the mirror method to solve the above problems. Three-dimensional panoramic imaging technology takes multiple-angle photos of the existing scene, stitches and fuses multiple pictures, and finally projects them onto a curved surface to form an effect similar to the real world. Three-dimensional panoramic imaging technology is mainly divided into cylindrical panoramic and spherical 360° panoramic. Cylindrical panoramic projects the panoramic pictures taken along the horizontal direction onto a cylindrical plane, and then places the viewing angle inside the cylinder to achieve the purpose of horizontal panoramic view. Spherical 360° panoramic takes multiple-angle panoramic photos along both the horizontal and vertical directions, and the final imaging curved surface is a sphere, that is, a projection curved surface with countless axes of symmetry, expanding the imaging viewing angle.

[0005] However, in the scenario of offshore jacket platforms, the projection plane is only symmetric with respect to the x-axis and y-axis. When the cleaning robot is in different postures on the round pipe, the viewing angles of the cameras on the cleaning robot are different, and the mapping relationship between the cameras and the 3D model is also different. Simply directly using the established mapping relationship between the projection surface and the panoramic image is not sufficient to correctly map the image texture to the projection surface to obtain the correct panoramic image. Summary of the Invention

[0006] In view of the above technical problems, the purpose of the present invention is to provide a real-time panoramic imaging method and device for an underwater cleaning robot, which can detect the posture of the cleaning robot on the pipeline in real time and adjust the texture mapping relationship in real time according to the posture, so as to correctly map the image texture to the projection surface and realize panoramic imaging in the scenario of offshore jacket platforms.

[0007] To achieve the above invention purpose, an embodiment provides a real-time panoramic imaging method for an underwater cleaning robot, including the following steps:

[0008] Construct a three-dimensional projection surface for panoramic imaging based on the working scenario of the cleaning robot on the offshore pipeline;

[0009] Obtain multiple images collected by the cameras on the cleaning robot in real time, and splice the multiple images to obtain a spliced panoramic image;

[0010] Perform pipeline edge detection on the spliced panoramic image to obtain a real-time pipeline edge line;

[0011] Construct a transformation equation based on the pixel points on the real-time pipeline edge line, the homography matrix, and the pixel points on the reference pipeline edge line, and solve the transformation equation to obtain the real-time rotation angle included in the homography matrix;

[0012] Retrieve the texture mapping matrix corresponding to the real-time rotation angle from the texture mapping library according to the real-time rotation angle, and map the spliced image to the three-dimensional projection surface based on the texture mapping matrix to achieve panoramic imaging.

[0013] In one embodiment, the constructed projection surface is a surface formed by using a cylinder to intercept a hemispherical surface with the hemispherical surface as the reference. Preferably, when intercepting, the height of the cylinder is parallel to the diameter of the hemispherical surface.

[0014] In one embodiment, the EDLines algorithm is used to perform pipeline edge detection on the spliced panoramic image, and a threshold detection method is adopted for the detected line segments to screen out long line segments as the real-time pipeline edge line.

[0015] In one embodiment, images in four directions collected by four cameras are obtained in real time, and the four images are spliced to obtain a spliced panoramic image.

[0016] In one embodiment, a cleaning robot model is also built within the three-dimensional projection surface. After obtaining the real-time rotation angle, the pose of the cleaning robot model is adjusted according to the real-time rotation angle to obtain a complete real-time panoramic view.

[0017] To achieve the above-mentioned invention object, another embodiment provides a real-time panoramic imaging device for an underwater cleaning robot, including:

[0018] A three-dimensional projection surface construction module, configured to construct a three-dimensional projection surface for panoramic imaging based on the working scenario of the cleaning robot on the marine conduit;

[0019] A stitched panoramic view construction module, configured to obtain multi-angle images collected by a camera on the cleaning robot in real time, and stitch the multi-angle images to obtain a stitched panoramic view;

[0020] An edge detection module, configured to perform conduit edge detection on the stitched panoramic view to obtain a real-time conduit edge line;

[0021] A real-time rotation angle calculation module, configured to construct a transformation equation based on the pixel points on the real-time conduit edge line, the homography matrix, and the pixel points on the reference conduit edge line, and solve the transformation equation to obtain the real-time rotation angle included in the homography matrix;

[0022] A panoramic imaging module, configured to retrieve a texture mapping matrix corresponding to the real-time rotation angle from a texture mapping library according to the real-time rotation angle, and map the stitched image to the three-dimensional projection surface based on the texture mapping matrix to achieve panoramic imaging.

[0023] Compared with the prior art, the beneficial effects of the present invention at least include:

[0024] After stitching multi-angle images collected in real time to obtain a stitched panoramic view, edge detection is performed to obtain a real-time conduit edge line. Based on the mapping relationship of the pixel points between the real-time conduit edge line and the reference conduit edge line with respect to the homography matrix, a transformation equation is constructed. By solving the transformation equation, the real-time rotation angle is obtained, and the corresponding texture mapping matrix is called from a pre-constructed texture mapping library for panoramic imaging, which can improve the efficiency and accuracy of panoramic imaging in scenarios such as marine jacket structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0026] Figure 1It is a flowchart of the real-time panoramic imaging method of the underwater cleaning robot provided by the embodiment;

[0027] Figure 2 It is the three-dimensional projection surface provided by the embodiment;

[0028] Figure 3 It is a schematic diagram of the catheter edge detection provided by the embodiment, where (a) is the stitched panoramic image, (b) is the edge detection result image, (c) is the threshold screening to detect the catheter straight line, and (d) is the real-time catheter edge line;

[0029] Figure 4 It is the relationship between the robot sizes in different states provided by the embodiment;

[0030] Figure 5 It is a schematic structural diagram of the real-time panoramic imaging device of the underwater cleaning robot provided by the embodiment. Detailed implementation manners

[0031] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific implementation manners described herein are only used to explain the present invention and do not limit the protection scope of the present invention.

[0032] To solve the problem that the panoramic imaging of the cleaning robot in the ocean round pipe scenario is not real-time, resulting in inaccurate panoramic imaging. The embodiment provides a real-time panoramic imaging method for an underwater cleaning robot.

[0033] Figure 1 It is a flowchart of the real-time panoramic imaging method of the underwater cleaning robot provided by the embodiment. As Figure 1 shown, the real-time panoramic imaging method of the underwater cleaning robot provided by the embodiment includes the following steps:

[0034] Step 1, construct a three-dimensional projection surface for panoramic imaging based on the working scenario of the cleaning robot on the ocean catheter.

[0035] In the embodiment, the cleaning robot is equipped with multiple cameras for image acquisition, specifically, cameras can be distributed in the front, back, left, and right directions to collect images from different angles. The cleaning robot crawls on the ocean catheter, and the ocean catheter is generally cylindrical. Therefore, the constructed projection surface is a surface formed by using a cylindrical surface to intercept a hemispherical surface with the hemispherical surface as the reference, as Figure 2 shown. Specifically, the construction process is as follows:

[0036] The surface equation of the cylinder is: x1 2 +y1 2 = r, and the surface equation of the spherical surface is:

[0037] When y1 > y2, the projection surface consists of the following equations for drawing cylindrical textures:

[0038]

[0039] In the above equations, i and j are the sequences of surface nodes, (x, y, z) are the spatial coordinates of the projection surface, r is the radius of the cylinder, h is the height of the cylinder, m1 is the number of axial vertices, and n1 is the number of circumferential vertices.

[0040] When y1 > y2, the projection surface consists of the following equations for drawing spherical textures:

[0041]

[0042] In the above equations, R is the radius of the sphere, r1 is the radius of the circular surface of the sphere at a certain height, which is calculated by and m is the number of meridians, n2 is the number of latitudes.

[0043] Step 2: Real-time obtain multiple images collected by the camera on the cleaning robot, and splice the multiple images to obtain a spliced panoramic image.

[0044] In the embodiment, multiple cameras are controlled by multi-threading to synchronously collect multiple images. It should be noted that during the actual collection process, the angles between the multiple cameras and the catheter are the same to ensure the accuracy of the spliced panoramic image. This angle can be set to 45°, 60°, 120°, and 135°. The spliced panoramic images obtained at different angles are different.

[0045] In the embodiment, two-dimensional panoramic imaging technology is used to splice multiple images to form a spliced panoramic image. Specifically, it includes: performing camera distortion correction, local feature point detection, feature point matching, image splicing, and seam melting processing on the multiple images to obtain the spliced panoramic image.

[0046] Step 3: Perform catheter edge detection on the spliced panoramic image to obtain a real-time catheter edge line.

[0047] In the embodiment, first, the EDLines algorithm is used to perform catheter edge detection on the spliced panoramic image. After detecting the edge points, the straightness criterion and the least squares line fitting method are used to extract line segments from the generated pixel chain. Then, a threshold detection method is used for the detected line segments to screen out long line segments as the real-time catheter edge line, as shown in Figure 3 . This edge detection method is fast and accurate.

[0048] Step 4: Construct a transformation equation based on the pixel points on the real-time catheter edge line, the homography matrix, and the pixel points on the reference catheter edge line, and solve the transformation equation to obtain the real-time rotation angle included in the homography matrix.

[0049] In the embodiment, after edge detection of the catheter, after extracting the catheter edge line, the included angle between the camera and the catheter can be obtained through homography transformation, and the pose of the cleaning robot on the catheter and the texture mapping matrix in the three-dimensional imaging are adjusted, so as to achieve the purpose of real-time displaying a panoramic image.

[0050] In the embodiment, the constructed transformation equation is:

[0051] q b = H ba q a

[0052] Among them, q b represents the pixel point coordinates on the real-time catheter edge line, q a represents the pixel point coordinates on the reference edge line, and H ba represents the homography matrix, which is composed of the rotation matrix R ba and the translation matrix t ab of two cameras corresponding to the pixel plane b and the pixel plane a, the two camera intrinsic parameter matrices K a and K b , and the pixel plane parameters ( d a ). Among them, represents the transpose of the normal vector of the pixel plane a, d a represents the distance to the pole in polar coordinates, and the homography matrix H ba is:

[0053]

[0054] During the movement of the cleaning robot on the catheter, there is no relative movement between the coordinate space of the cleaning robot and the coordinate space of the catheter in the x and y directions, that is, the center of the cleaning robot and the center of the catheter always remain aligned. The cleaning robot only rotates and moves along the radial direction of the catheter due to the transformation of the pose of the cleaning robot. As Figure 4 shown, where (a) is a schematic diagram of the vertical state, (b) is a diagram of the dimensional relationship in the vertical state, (c) is a schematic diagram of the horizontal state, and (d) is a diagram of the dimensional relationship in the horizontal state.

[0055] When the robot is in the vertical state and the horizontal state respectively, it corresponds to the maximum and minimum limits of the distance between the robot and the circular tube in the radial direction. From Figure 4 it can be known that:

[0056]

[0057]

[0058] Among them, R cLet \(R\) be the radius of the catheter, \(r\) be the wheel diameter of the cleaning robot, \(W\) be the axial distance between the left and right wheels of the cleaning robot, and \(L\) be the distance between the centers of the front and rear wheels of the cleaning robot. Therefore, there is a translation vector \(t\). ab = [0 0 z].

[0059]

[0060] When running the panoramic imaging algorithm in different catheter scenarios, \(z\) max The only variable in the formula is the catheter radius \(R\). c .

[0061] The parameters of the cleaning robot are shown in Table 1:

[0062] Table 1

[0063]

[0064] Substitute the parameters of the cleaning robot into \(z\) max formula and take the derivative with respect to \(R\). It is found that \(z\) max is monotonically decreasing with respect to \(R\). Therefore, when \(R\) is the smallest, \(z\) max is the largest. In this scenario, the only maximum in the radial direction of the cleaning robot is the largest, and the influence on the solution of the rotation angle of the cleaning robot is also the largest. In the panoramic imaging system, the distance between the viewing point of the cleaning robot and the circular tube is greater than 1000 mm. Compared with the maximum displacement, the movement of the cleaning robot is ignored. Therefore, the translation vector \(t\) ab = (0, 0, 0). The above formula can be simplified as:

[0065]

[0066] After simplification, the rotation matrix \(R\) ba is:

[0067] \(R\) ba = \(K\) b -1 \(H\) ba \(K\) a

[0068] \(K\) a and \(K\) b are the internal parameter matrices of the camera, expressed as:

[0069]

[0070] \(f\) xa , \(f\) xb , \(f\) ya , \(f\) yb are the focal length parameters of the camera, \(u\) 0a , \(v\) 0a , \(u\) 0b , \(v\)0b is the main point displacement vector. When performing the projection transformation matrix, the focal length parameters f of the two cameras x , f y are both equal to 1, and the displacement vectors u 0a , v 0a , u 0b , v 0b are half of the length and width of the stitched panoramic image;

[0071] It is also known that the expression of the rotation matrix is:

[0072]

[0073] Then the solution is obtained:

[0074]

[0075] Among them, α, β, and γ respectively represent the rotation angles in the x, y, and z directions.

[0076] Step 5: Retrieve the texture mapping matrix corresponding to the real-time rotation angle from the texture mapping library according to the real-time rotation angle, and map the stitched image to the three-dimensional projection surface based on this texture mapping matrix to achieve panoramic imaging.

[0077] When the cleaning robot is in different postures on the catheter, the viewing angles of each camera of the cleaning robot are different, and the mapping relationship between the camera and the three-dimensional projection surface is also different. Therefore, it is necessary to be able to detect the posture of the robot on the round pipe in real time and adjust the texture mapping relationship in real time according to the posture to achieve panoramic imaging in scenarios such as offshore jacket platforms.

[0078] During real-time three-dimensional imaging, retrieve the texture mapping matrix corresponding to the real-time rotation angle from the texture mapping library according to the real-time rotation angle, and map the stitched image to the three-dimensional projection surface based on this texture mapping matrix to achieve panoramic imaging.

[0079] It should be noted that the texture mapping matrix in the texture mapping library is obtained by mapping through the OpenGL texture mapping principle. Specifically, the texture mapping process includes: before performing texture mapping, it is necessary to establish a mapping relationship between three-dimensional points and pixels, and their mapping relationship is achieved through texture coordinates. By specifying the texture coordinates of several key points on a three-dimensional model, the one-to-one correspondence between three-dimensional points and two-dimensional pixels is achieved, and the remaining points are interpolated through the fragment shader to achieve the complete mapping of the image. The value range of texture coordinates is 0 to 1, and the lower left corner of the texture image is the origin of the texture coordinates.

[0080] In the embodiment, a cleaning robot model is also built inside the three-dimensional projection surface. After obtaining the real-time rotation angle, the pose of the cleaning robot model is adjusted according to the real-time rotation angle to obtain a complete real-time panoramic image.

[0081] The real-time panoramic imaging method of the underwater cleaning robot provided by the above embodiment detects the pose of the cleaning robot on the pipeline in real time, and adjusts the texture mapping relationship in real time according to the pose, so as to correctly map the image texture onto the projection surface, realizing panoramic imaging in the ocean jacket type scene.

[0082] Figure 5 It is a schematic structural diagram of the real-time panoramic imaging device of the underwater cleaning robot provided by the embodiment. As Figure 5 shown, the real-time panoramic imaging device provided by the embodiment includes:

[0083] A three-dimensional projection surface construction module, configured to construct a three-dimensional projection surface for panoramic imaging based on the working scene of the cleaning robot on the ocean pipeline;

[0084] A stitched panoramic image construction module, configured to obtain multi-angle images collected by a camera on the cleaning robot in real time, and stitch the multi-angle images to obtain a stitched panoramic image;

[0085] An edge detection module, configured to perform duct edge detection on the stitched panoramic image to obtain a real-time duct edge line;

[0086] A real-time rotation angle calculation module, configured to construct a transformation equation based on the pixel points on the real-time duct edge line, the homography matrix, and the pixel points on the reference duct edge line, and solve the transformation equation to obtain the real-time rotation angle included in the homography matrix;

[0087] A panoramic imaging module, configured to retrieve a texture mapping matrix corresponding to the real-time rotation angle from the texture mapping library according to the real-time rotation angle, and map the stitched image onto the three-dimensional projection surface based on the texture mapping matrix to realize panoramic imaging.

[0088] It should be noted that when the real-time panoramic imaging device of the underwater cleaning robot provided by the above embodiment performs real-time panoramic imaging, the above-mentioned division of each functional module should be used for illustration. The above functions can be allocated to different functional modules according to needs, that is, the internal structure of the terminal or server is divided into different functional modules to complete all or part of the functions described above. In addition, the real-time panoramic imaging device of the underwater cleaning robot provided by the above embodiment and the embodiment of the real-time panoramic imaging method of the underwater cleaning robot belong to the same concept. For the specific implementation process, please refer to the embodiment of the real-time panoramic imaging method of the underwater cleaning robot, which will not be elaborated here.

[0089] The specific embodiments described above have elaborated in detail the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, supplements, equivalent replacements, etc. made within the scope of the principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A real-time panoramic imaging method for an underwater cleaning robot, characterized in that, It includes the following steps: Construct a three-dimensional projection surface for panoramic imaging based on the working scenario of the cleaning robot on the marine conduit; Obtain multiple images collected by the camera on the cleaning robot in real time, and splice the multiple images to obtain a spliced panoramic image; Perform conduit edge detection on the spliced panoramic image to obtain a real-time conduit edge line; Construct a transformation equation based on the pixel points on the real-time conduit edge line, the homography matrix, and the pixel points on the reference conduit edge line, and solve the transformation equation to obtain the real-time rotation angle included in the homography matrix. The constructed transformation equation is: q b = H ba q a Among them, q b represents the pixel point coordinates on the real-time catheter edge line, and q a represents the pixel point coordinates on the reference edge line. H ba represents the homography matrix, which consists of the rotation matrix R ba and the translation matrix t ab of two cameras corresponding to pixel plane b and pixel plane a, the two camera intrinsic parameter matrices K a and K b , and the pixel plane parameters . Among them, represents the transpose of the normal vector of pixel plane a, and d a represents the distance to the pole in polar coordinates. The homography matrix H ba is as follows: Retrieve the texture mapping matrix corresponding to the real-time rotation angle from the texture mapping library according to the real-time rotation angle, and map the spliced image to the three-dimensional projection surface based on the texture mapping matrix to achieve panoramic imaging.

2. The real-time panoramic imaging method for an underwater cleaning robot according to claim 1, characterized in that, The constructed projection surface is a surface formed by intercepting a hemispherical surface with a cylinder based on the hemispherical surface. When intercepting, the height of the cylinder is parallel to the diameter of the hemispherical surface.

3. The real-time panoramic imaging method for an underwater cleaning robot according to claim 1, characterized in that, Use the EDLines algorithm to perform conduit edge detection on the spliced panoramic image, and adopt a threshold detection method for the detected line segments to screen out long line segments as the real-time conduit edge line.

4. The real-time panoramic imaging method for an underwater cleaning robot according to claim 1, characterized in that, When solving the transformation equation, let t ab =(0, 0, 0), then the homography matrix H ba is simplified to: After simplification, the rotation matrix R ba is as follows: K a associated with K b is the internal parameter matrix of the camera, expressed as: f xa , f xb , f ya , f yb is the focal length parameter of the camera, u 0a , v 0a , u 0b , v 0b is the principal point displacement vector. When performing the projection transformation matrix, the two camera focal length parameters f x , f y are both equal to 1, and the displacement vectors u 0a , v 0a , u 0b , v 0b are half of the length and width of the stitched panoramic image; Also, the expression of the rotation matrix is known as: Then the solution is obtained as: Where α, β, and γ respectively represent the rotation angles in the x, y, and z directions.

5. The real-time panoramic imaging method for an underwater cleaning robot according to claim 1, characterized in that, Obtain images in four directions collected by four cameras in real time, and splice the four images to obtain a spliced panoramic image.

6. The real-time panoramic imaging method for an underwater cleaning robot according to claim 1, characterized in that, A cleaning robot model is also built inside the three-dimensional projection surface. After obtaining the real-time rotation angle, the pose of the cleaning robot model is adjusted according to the real-time rotation angle to obtain a complete real-time panoramic image.

7. A real-time panoramic imaging device for an underwater cleaning robot, characterized in that, It includes: A three-dimensional projection surface construction module for constructing a three-dimensional projection surface for panoramic imaging based on the working scenario of the cleaning robot on the marine conduit; A spliced panoramic image construction module for obtaining multi-angle images collected by the camera on the cleaning robot in real time and splicing the multi-angle images to obtain a spliced panoramic image; An edge detection module for performing conduit edge detection on the spliced panoramic image to obtain a real-time conduit edge line; A real-time rotation angle calculation module for constructing a transformation equation based on the pixel points on the real-time conduit edge line, the homography matrix, and the pixel points on the reference conduit edge line, and solving the transformation equation to obtain the real-time rotation angle included in the homography matrix. The constructed transformation equation is: q b = H ba q a Among them, q b represents the pixel point coordinates on the real-time catheter edge line, and q a represents the pixel point coordinates on the reference edge line. H ba represents the homography matrix, which is composed of the rotation matrix R ba and the translation matrix t ab of two cameras corresponding to the pixel plane b and the pixel plane a, the two camera intrinsic parameter matrices K a and K b , and the pixel plane parameter . Among them, represents the transpose of the normal vector of the pixel plane a, and d a represents the distance to the pole in polar coordinates. The homography matrix H ba is: A panoramic imaging module for retrieving the texture mapping matrix corresponding to the real-time rotation angle from the texture mapping library according to the real-time rotation angle, and mapping the spliced image to the three-dimensional projection surface based on the texture mapping matrix to achieve panoramic imaging.

Citation Information

Patent Citations

  • Real-time image stitching method based on circular tube operation robot

    CN107507133A

  • Marine culture region underwater scene panorama generation method

    CN113538702A