Eye plate three-dimensional reconstruction and welding groove guiding and positioning method and device

Abstract

The invention provides an eye plate three-dimensional reconstruction and welding groove guide positioning method which comprises the following steps: S1, a welding robot moves around a salvage eye plate, and a depth camera acquires multi-view data and local point cloud data of the salvage eye plate; s2, the multi-view data and the local point cloud data are transmitted to an upper computer, the upper computer performs three-dimensional reconstruction on the salvage eye plate by using a neural radiation field algorithm, and a three-dimensional reconstruction model of the salvage eye plate is generated; s3, performing point cloud registration on the local point cloud data and the three-dimensional reconstruction model through an ICP algorithm, and converting the three-dimensional reconstruction data of the salvage eye plate into a coordinate system of a welding robot; s4, the welding robot positions the welding groove through the optimal path track, the point cloud information of the welding groove is obtained again, a PROSAC algorithm is used for extracting a welding guide track, and a welding guide starting point and a welding guide ending point are obtained through the spatial position relation of the welding guide track; and S5, the welding robot is guided to the welding starting point and the welding ending point, and welding operation of the fishing eye plate is completed.

Description

Technical Field

[0001] The present invention relates to the technical field of shipping, and in particular to a method and device for three-dimensional reconstruction of eyebolts and guiding and positioning of welding grooves. Background Art

[0002] The shipping industry is a pillar industry of the national economy. The trade between China and important foreign economic entities has developed rapidly. Accidents often occur during the transportation of ships. The accident ships not only affect the local water environment, but may also cause traffic jams in the transportation channels.

[0003] The existing sunken ship salvage technologies mainly include external buoyancy salvage method, internal buoyancy salvage method, crane ship salvage method and hybrid salvage method. The crane ship salvage method has become the main method for sunken ship salvage in China, and the sunken ship is lifted by using the jack steel cables passing through the bottom of the sunken ship. However, the traditional crane ship salvage method is limited by the geological conditions of the sunken ship water area and the depth of the sunken ship water area. In an environment where the geology at the bottom of the sunken ship is hard and the water area is deep, it is difficult for the crane ship salvage method to carry out the salvage operation. To cope with a wider range of deep water areas and more complex sunken ship geological conditions, a more advanced salvage method is urgently needed.

[0004] To cope with a wider range of deep water areas and more complex sunken ship geological conditions, a more advanced salvage method is urgently needed. The Shanghai Salvage Bureau has publicly announced a salvage method based on underwater welding. A stress-bearing eyebolt is welded on the hull of the sunken ship underwater, the lifting wire is connected to the stress-bearing eyebolt, and the surface crane lifts the sunken ship out of the water by pulling the lifting wire. At the same time, during the actual operation process, affected by the water depth, generally an underwater unmanned high-pressure dry cabin is used to carry welding equipment and dive to the sunken ship area with the salvage eyebolt and attach to the surface of the sunken ship to complete the welding of the salvage eyebolt.

[0005] However, there are still the following technical problems: During the underwater sunken ship salvage operation, an underwater high-pressure dry operation cabin is used to create a dry environment for convenient welding operation. However, the volume of the underwater high-pressure dry cabin is limited and it cannot be manually operated on site, and welding robots need to be relied on for welding operations. The position of the salvage eyebolt will be dynamically adjusted according to the condition of the sunken ship's flank plates. This requires that before the welding robot performs the welding operation, it is necessary to perform three-dimensional reconstruction on the salvage eyebolt to obtain the spatial three-dimensional information and specific position of the salvage eyebolt, and then identify and position the welding groove to guide the welding robot to perform precise welding operations.

[0006] Therefore, a method for quickly and stably obtaining three-dimensional reconstruction data of the salvage eyebolt in the underwater high-pressure dry cabin and performing welding groove identification and positioning is needed. Summary of the Invention

[0007] To solve the above technical problems, the object of the present invention is to provide a method and device for guiding and positioning the welding groove of a salvage eye plate in an underwater high-pressure dry cabin. The many technical effects that can be produced by the preferred technical solutions provided by the present invention are described in detail below.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] The method for three-dimensional reconstruction and welding groove guiding and positioning of the eye plate provided by the present invention includes the following steps:

[0010] S1: The welding robot moves around the salvage eye plate, and the depth camera acquires multi-view data and local point cloud data of the salvage eye plate;

[0011] S2: Transmit the multi-view data and the local point cloud data to the upper computer, and the upper computer uses the neural radiance field algorithm to perform three-dimensional reconstruction on the salvage eye plate to generate a three-dimensional reconstruction model of the salvage eye plate;

[0012] S3: Perform point cloud registration on the local point cloud data and the three-dimensional reconstruction model through the ICP algorithm, and convert the three-dimensional reconstruction data of the salvage eye plate to the coordinate system of the welding robot;

[0013] S4: The welding robot locates the welding groove through the optimal path trajectory, acquires the point cloud information of the welding groove again, uses the PROSAC algorithm to extract the welding guiding trajectory, and obtains the starting and ending points of the welding guide through the spatial position relationship of the welding guiding trajectory;

[0014] S5: Guide the welding robot to the starting and ending points of welding to complete the welding operation of the salvage eye plate.

[0015] Preferably, in the S1, the welding robot takes the central axis of the salvage eye plate as a reference, adjusts the end pose of the depth camera during horizontal movement, so that the line of sight of the depth camera is perpendicular to the central axis of the salvage eye plate, and after the welding robot completes one horizontal movement, it then performs vertical movement, so that the depth camera acquires multi-view data and local point cloud data of the salvage eye plate.

[0016] Preferably, in the S2, the neural radiance field algorithm uses a fully connected network structure, and maps the feature point coordinates and directions inferred from the input multi-view data to a high-dimensional space; among them, substitute the coordinate x and the direction view angle d into formula (1) for position encoding; when inputting the coordinate x, L takes 10; when inputting the direction view angle d, L takes 4;

[0017] γ(p)=(sin(2 0 πp),cos(2 0 πp),...,sin(2 L-1πp), cos(2 L-1 πp)) (1).

[0018] Preferably, the neural radiance field algorithm uses a voxel rendering algorithm to model color and density. For the points on the observation ray that are continuous, the color of the corresponding pixel on the camera imaging plane can be calculated through an integral formula. Among them, the integral formula is shown in formula (2):

[0019]

[0020] Among them, the voxel density σ(x) represents the particle density of the ray at a certain place, c(x) represents the particle color at a certain place along the ray direction, and T(x) represents the cumulative transparency of the ray r(t) = o + td from t n to t f which is shown as in formula (3):

[0021]

[0022] Preferably, in the step S3, the ICP algorithm is used to perform point cloud registration on the local feature point cloud of the depth camera and the three-dimensional reconstruction point cloud of the fishing eye plate. By calculating the rigid transformation between the two groups of point clouds, the alignment of the point clouds is achieved. Furthermore, the overall point cloud of the fishing eye plate is transformed into the coordinate system of the depth camera. Finally, using the hand-eye matrix obtained by hand-eye calibration, the three-dimensional point cloud of the fishing eye plate is transformed into the robot coordinate system.

[0023] Preferably, in the step S4, the specific steps of using the PROSAC algorithm to obtain the point cloud data of both sides of the groove on the best fitting plane by segmenting the groove point cloud are as follows:[[]]

[0024] S401: Input the groove point cloud data;

[0025] S402: Sort the points according to their characteristics for priority;

[0026] S403: Randomly select 3 points from the sorted points for plane fitting;

[0027] S404: Screen the inliers according to the distance threshold;

[0028] S405: Iterate and expand the sampling point range according to the priority sorting;

[0029] S406: Iteratively update the plane with the most inliers as the best plane;

[0030] S407: Determine whether the number of inliers in the fitting plane reaches the set quantity; if it reaches, enter S408; otherwise, return to S402 to reselect points for plane fitting;

[0031] S408: Segment the inliers of the fitted plane from the groove point cloud.

[0032] Preferably, the mathematical equation of the fitted plane is:

[0033] Ax + By + Cz + D = 0 (4)

[0034] where (A, B, C) is the normal vector, and D can be obtained by substituting the parameters of the inliers of the fitted plane;

[0035] By calculating the intersection line of the fitted planes M on both sides of the groove 1 The plane equation and the fitted plane M 2 The intersection line of the plane equations is the welding guidance trajectory. Among them, the mathematical equation of the guidance trajectory is:

[0036]

[0037] For the obtained welding trajectory 10, the guiding straight line L1 and the adjacent guiding straight line L2; by calculating the intersection points of the two straight lines within a set threshold range, the starting point or the ending point of the welding guidance can be obtained; and through the extracted welding trajectory, its intersection points are obtained by projecting onto a two-dimensional plane, and the intersection points are used as the starting and ending points of the welding.

[0038] Preferably, a device for three-dimensional reconstruction of an eye plate and welding groove guiding and positioning includes a depth camera, a welding robot, a control cabinet, and a host computer. The depth camera is arranged at the end of the welding robot and is used to obtain multi-views of the salvaged eye plate as the welding robot moves; the welding robot is communicatively connected to the control cabinet and is used to weld the salvaged eye plate on the sunken ship surface; the host computer is communicatively connected to the depth camera and the control cabinet respectively and is used to process the three-dimensional reconstruction model of the underwater salvaged eye plate, extract the welding trajectory information, and provide it to the welding robot; the method for three-dimensional reconstruction of the salvaged eye plate and guiding and positioning the welding groove of the welding robot is the aforementioned method for three-dimensional reconstruction of the eye plate and welding groove guiding and positioning.

[0039] The preferred technical solution of the present invention can at least further produce the following technical effects: The present invention provides a method for three-dimensional reconstruction of an eye plate and guiding and positioning of a welding groove, including the following steps: S1: The welding robot moves around the salvaged eye plate, and the depth camera obtains multi-view data and local point cloud data of the salvaged eye plate; S2: The multi-view data and local point cloud data are transmitted to the upper computer, and the upper computer uses the neural radiance field algorithm to perform three-dimensional reconstruction on the salvaged eye plate to generate a three-dimensional reconstruction model of the salvaged eye plate; S3: The local point cloud data is registered with the three-dimensional reconstruction model through the ICP algorithm, and the three-dimensional reconstruction data of the salvaged eye plate is converted into the coordinate system of the welding robot; S4: The welding robot locates the welding groove through the optimal path trajectory, obtains the point cloud information of the welding groove again, uses the PROSAC algorithm to extract the welding guiding trajectory, and obtains the starting and ending points of the welding guide through the spatial position relationship of the welding guiding trajectory; S5: Guide the welding robot to the welding starting and ending points to complete the welding operation of the salvaged eye plate.

[0040] The present invention uses an improved neural radiance field algorithm and the coordinated cooperation of a welding robot and a depth camera to perform path planning during the process of the welding robot equipped with a depth camera to obtain multi-views of the salvaged eye plate, effectively completing the acquisition of data in the blind area of the salvaged eye plate's field of view, and effectively solving the problem of incomplete or inaccurate reconstruction caused by the limitation of the view blind area in traditional view-based three-dimensional reconstruction methods. Moreover, the improved neural radiance field algorithm makes full use of the characteristics of deep learning algorithms, and can achieve fast and robust three-dimensional reconstruction in an underwater high-pressure dry cabin.

[0041] In addition, the present invention completes the conversion of the three-dimensional reconstruction data of the salvaged eye plate to the robot coordinate system in one point cloud registration process, avoiding the cumulative error that may be brought by multiple registrations, and realizing the real-time perception of the operation environment in the underwater high-pressure dry cabin quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0043] Figure 1 is a flowchart of the method for three-dimensional reconstruction of an eye plate and guiding and positioning of a welding groove provided by the present invention;

[0044] Figure 2 is a flowchart of S4 of the method for three-dimensional reconstruction of an eye plate and guiding and positioning of a welding groove provided by the present invention;

[0045] Figure 3It is a schematic diagram of the fully connected network structure of the neural radiance field in S2 of the method for three-dimensional reconstruction of the eye plate and guiding and positioning of the welding groove provided by the present invention;

[0046] Figure 4 It is a schematic diagram of the structure of the device for three-dimensional reconstruction of the eye plate and guiding and positioning of the welding groove provided by the present invention;

[0047] Figure 5 It is a schematic diagram of the moving trajectory of the welding robot of the device for three-dimensional reconstruction of the eye plate and guiding and positioning of the welding groove provided by the present invention.

[0048] In the figure:

[0049] 1. Depth camera; 2. Welding robot; 3. Control cabinet; 4. Host computer; 5. Salvage eye plate; 51. Central axis. Specific implementation manner

[0050] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described in detail below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other implementation manners obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope protected by the present invention.

[0051] As Figures 1-5 shown, the present invention provides a method for three-dimensional reconstruction of the eye plate and guiding and positioning of the welding groove, including the following steps:

[0052] S1: The welding robot 2 moves around the salvage eye plate 5, and the depth camera 1 acquires multi-view data and local point cloud data of the salvage eye plate 5;

[0053] S2: The multi-view data and local point cloud data are transmitted to the host computer 4, and the host computer 4 uses the neural radiance field algorithm to perform three-dimensional reconstruction on the salvage eye plate 5 to generate a three-dimensional reconstruction model of the salvage eye plate 5;

[0054] S3: The local point cloud data and the three-dimensional reconstruction model are subjected to point cloud registration through the ICP algorithm, and the three-dimensional reconstruction data of the salvage eye plate 5 is converted into the coordinate system of the welding robot 2;

[0055] S4: The welding robot 2 locates the welding groove through the optimal path trajectory, acquires the point cloud information of the welding groove again, uses the PROSAC algorithm to extract the welding guiding trajectory, and obtains the starting and ending points of the welding guide through the spatial position relationship of the welding guiding trajectory;

[0056] S5: Guide the welding robot 2 to the starting and ending points of welding to complete the welding operation of the salvage eye plate.

[0057] As Figure 5As shown in the figure, in S1, the host computer 4 controls the welding robot 2 to move around the salvage eye plate 5. The welding robot 2 adjusts the end attitude of the depth camera 1 during horizontal movement with the central axis 51 of the salvage eye plate 5 as a reference, ensuring that the line of sight of the depth camera 1 is perpendicular to the central axis 51 of the salvage eye plate 5. After the welding robot 2 completes one horizontal movement, it then makes a vertical movement so that the depth camera 1 can obtain multi-view data and local point cloud data of the salvage eye plate 5. Among them, the distance of one horizontal movement is 1.5 m, the distance of one vertical movement is 0.3 m, and the maximum included angle between the line of sight of the depth camera 1 and the central axis 51 of the salvage eye plate 5 is θ.

[0058] In S2, as Figure 3 shown, the neural radiance field algorithm uses a fully connected network structure to map the feature point coordinates and directions inferred from the input multi-view data into a high-dimensional space; among them, the coordinates x and the direction view d are substituted into formula (1) for position encoding; when inputting the coordinates x, L takes 10; when inputting the direction view d, L takes 4;

[0059] γ(p) = (sin(2 0 πp), cos(2 0 πp),..., sin(2 L-1 πp), cos(2 L-1 πp)) (1).

[0060] The neural radiance field algorithm uses the volume rendering algorithm to model color and density. The welding robot 2 is equipped with a camera to observe along a specific direction, and the points on the observation ray are continuous. The color of the corresponding pixel on the camera imaging plane is obtained by integrating the colors of the points passed by the corresponding ray, that is, the color of the corresponding pixel on the camera imaging plane is calculated through the integral formula; among them, the integral formula is as shown in formula (2):

[0061]

[0062] Among them, the voxel density σ(x) represents the particle density of the ray at a certain place, c(x) represents the particle color at a certain place along the ray direction, and T(x) represents the cumulative transparency of the ray r(t) = o + td from t n to t f and is expressed as shown in formula (3):

[0063]

[0064] The neural radiance field algorithm optimizes the continuous points on the ray that are difficult to estimate by using a piecewise approximation rendering method. During the rendering process, multi-level voxel sampling is used, and more sampling points can be obtained for the area containing more visible content. The improved neural radiance field network proposed in the present invention adds a multi-resolution hash encoding technique on the basis of the original neural network. Its principle is as follows: Given the input coordinate x, the voxels surrounding it are enclosed by frames with different resolutions, and the vertices of these voxels are assigned indices by means of hash mapping of integer coordinates. For the generated vertex indices, the corresponding F-dimensional feature vectors are found in the hash table. Linear interpolation is performed according to the relative position of x in the corresponding voxel. The interpolation results of x are combined and input into a fully connected network for prediction. The multi-resolution hash encoding is applied to the position encoding of the parameters input into the network, improving the input of the neural network and enhancing the network operation efficiency. When specifically using the multi-view data of the fishing eye plate 5 for three-dimensional reconstruction, using the improved neural radiance field algorithm can greatly improve the three-dimensional reconstruction speed of the fishing eye plate 5, and can stably, quickly, and accurately perform three-dimensional reconstruction on the underwater fishing eye plate 5.

[0065] The neural radiance field algorithm determines the feature points of the fishing eye plate 5 in the multi-view data through the multi-view data of the fishing eye plate 5, and infers the specific pose of the depth camera 1 through calculation. Introducing the hash encoding technique into the neural radiance field algorithm significantly improves the inference speed of the neural radiance field algorithm and can quickly complete the three-dimensional reconstruction of the fishing.

[0066] In S3, the ICP algorithm is used to perform point cloud registration on the local feature point cloud of the depth camera 1 and the three-dimensional reconstruction point cloud of the fishing eye plate 5. By calculating the rigid transformation between the two groups of point clouds, the alignment of the point clouds is achieved, and then the overall point cloud of the fishing eye plate 5 is transformed into the coordinate system of the depth camera 1. Finally, using the hand-eye matrix obtained by performing hand-eye calibration on the depth camera 1 and the welding robot 2, the three-dimensional point cloud of the fishing eye plate 5 is transformed into the robot coordinate system.

[0067] ICP (Iterative Closest Point) is a classic point cloud registration algorithm used to calculate the rigid transformation (rotation and translation) between two groups of point clouds, so that the source point cloud can be aligned with the target point cloud as much as possible. Its core idea is to iteratively minimize the error between the two groups of point sets through nearest neighbor matching and rigid transformation optimization. By using the ICP algorithm, the spatial pose relationship between the local point cloud of the fishing eye plate 5 and the complete point cloud of the eye plate obtained by three-dimensional reconstruction is calculated, and the overall point cloud of the eye plate is transformed into the coordinate system of the depth camera 1. Through the hand-eye matrix obtained by performing hand-eye calibration on the depth camera 1 and the welding robot 2, the three-dimensional point cloud of the fishing eye plate 5 is transformed into the coordinate system of the welding robot 2.

[0068] In S4, the welding robot 2 locates the welding groove through the optimal trajectory, obtains the point cloud information of the welding groove again, and extracts the welding trajectory through the improved PROSAC algorithm. The implementation principle of the PROSAC algorithm for extracting the welding trajectory is as follows: The PROSAC algorithm is an improved random model fitting method, aiming to reduce the number of iterations and improve the model fitting accuracy. The core idea of the random model fitting method is to repeatedly randomly extract subsets from the dataset, fit the model using the subsets, and judge the number of data points supported by the model based on the model. Therefore, when the proportion of outliers in the data is relatively large, the random sampling strategy of the random model fitting method often requires a large number of iterations to find the optimal model. The PROSAC algorithm, through the progressive sampling method, sorts the data according to some prior information (such as the quality of feature matching, the distance from the point to the model, etc.). It preferentially samples points that may be inliers from the points with higher rankings. As the number of iterations increases, the sampling range gradually expands to include more data points, accelerating the convergence speed and reducing the number of iterations. To improve the speed of fitting the point cloud plane, the progressive sampling of the PROSAC algorithm assumes that the high-priority points are concentrated in the inlier region. Therefore, by preferentially sampling from these points, a suitable model can be found earlier. Assuming there are m inliers in the dataset, and the priority distribution among these inliers conforms to a certain specific distribution (such as Gaussian distribution), the probability expectation P of the progressive sampling can be expressed as:

[0069] P = 1 - (1 - P i ) t (6)

[0070] In the formula: P i represents the probability of finding all inliers among the first i points, and t is the current number of iterations.

[0071] During the iteration process of the PROSAC algorithm, by gradually expanding the size of the sampling subset, concentrating on the points with higher priorities in the early stage and gradually adding the points with lower priorities, the effect of progressive optimization is ensured. Its sampling strategy can be summarized as:

[0072] n t = min(t, N) (7)

[0073] In the formula, n t is the number of sampling points in the t-th iteration, and N is the total number of data points. At the beginning, n t is small and gradually increases as t increases. In this way, the PROSAC algorithm can quickly find the inlier set of the correct model with hysteresis in fewer iterations and gradually eliminate the influence of outliers.

[0074] For example Figure 2As shown in the figure, the specific steps for the present invention to use the PROSAC algorithm to obtain the point cloud data of both sides of the groove on the best fitting plane by segmenting the groove point cloud are as follows:

[0075] S401: Input the groove point cloud data;

[0076] S402: Sort the priorities according to the characteristics of the points;

[0077] S403: Randomly select 3 points from the sorted points for plane fitting;

[0078] S404: Screen the inliers according to the distance threshold;

[0079] S405: Iterate and expand the sampling point range according to the priority sorting;

[0080] S406: Iteratively update the plane with the most inliers as the best plane;

[0081] S407: Determine whether the number of inliers in the fitting plane reaches the set quantity; if so, go to S408; otherwise, return to S402 to re - extract points for plane fitting;

[0082] S408: Segment the inliers of the fitting plane from the groove point cloud.

[0083] The mathematical equation of the fitting plane is:

[0084] Ax + By + Cz + D = 0 (4)

[0085] where (A, B, C) is the normal vector, and D can be obtained by substituting the parameters of the inliers of the fitting plane;

[0086] By calculating the intersection line of the fitting planes M on both sides of the groove 1 The plane equation and the fitting plane M 2 The intersection line of the plane equations is the welding guidance trajectory, where the mathematical equation of the guidance trajectory is:

[0087]

[0088] For the obtained welding trajectory guiding straight line L1 and the adjacent guiding straight line L2; by calculating the intersection points of the two straight lines within the set threshold range, the starting point or the ending point of the welding guidance can be obtained; and through the extracted welding trajectory, by projecting it onto the two - dimensional plane to obtain its intersection points, and taking the intersection points as the welding start and end points.

[0089] Such as Figures 4-5As shown, the present invention provides a device for three-dimensional reconstruction of a salvaged eye plate 5 and guiding and positioning a welding groove, comprising a depth camera 1, a welding robot 2, a control cabinet 3 and a host computer 4, wherein the depth camera 1 is used as a visual sensor and is arranged at the end of the welding robot 2 through an adjustment component, so as to obtain multi-view data of the underwater salvaged eye plate 5 and a local feature point cloud of the salvaged eye plate 5 in a primary camera coordinate system as the welding robot 2 moves; the welding robot 2 is used as a welding device in an underwater high-pressure dry cabin and is connected to the control cabinet 3 for communication, so as to weld the salvaged eye plate 5 on the surface of a sunken ship; the control cabinet 3 is used to control the welding robot 2 to perform welding operations and to communicate with the host computer 4 to obtain spatial position information of the salvaged eye plate 5 provided by the host computer 4; the host computer 4 is connected to the depth camera 1 and the control cabinet 3 for communication, so as to process a three-dimensional reconstruction model of the salvaged eye plate 5 and extract welding trajectory information to provide it to the welding robot 2; the method for three-dimensional reconstruction of the salvaged eye plate 5 and guiding and positioning the welding groove by the welding robot 2 is the aforementioned three-dimensional reconstruction and welding groove guiding and positioning method of the salvaged eye plate 5.

[0090] Furthermore, during the operation, the depth camera 1, the welding robot 2, the control cabinet 3 and the salvage eye plate 5 are located in an underwater high-pressure dry operation cabin.

[0091] It should be noted that the underwater salvage eye plate 5 is carried in an underwater high-pressure dry cabin and is welded to the side surface of the sunken ship by a welding robot 2 to serve as a fulcrum for the lifting cable for salvaging the sunken ship.

[0092] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0093] In the description of the present invention, it should be noted that, unless otherwise specified, the meaning of "plurality" is two or more; the orientations or positional relationships indicated by the terms "upper", "lower", "left", "right", "inner", "outer", "front end", "rear end", "head", "tail", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0094] In the description of the present invention, it should also be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0095] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "an example" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0096] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for three-dimensional reconstruction of eye plate and guiding and positioning welding groove, characterized in that: The steps include: S1: The welding robot moves around the salvage eye plate, and the depth camera obtains multi-view data and local point cloud data of the salvage eye plate; S2: transmitting the multi-view data and the local point cloud data to a host computer, and the host computer uses a neural radiation field algorithm to perform three-dimensional reconstruction on the salvage eye plate to generate a three-dimensional reconstruction model of the salvage eye plate; S3: performing point cloud registration on the local point cloud data and the three-dimensional reconstruction model by using an ICP algorithm, and converting the three-dimensional reconstruction data of the salvage eye plate into a coordinate system of the welding robot; S4: The welding robot locates the welding groove through the optimal path trajectory, obtains the point cloud information of the welding groove again, extracts the welding guide trajectory using the PROSAC algorithm, and obtains the welding guide start and end points through the spatial position relationship of the welding guide trajectory; S5: Guide the welding robot to the welding start and end points to complete the welding operation of the salvage eye plate.

2. The eye plate three-dimensional reconstruction and welding groove guidance positioning method according to claim 1 is characterized in that: In S1, the welding robot takes the central axis of the salvage eye plate as a reference and adjusts the end posture of the depth camera during horizontal movement so that the line of sight of the depth camera is perpendicular to the central axis of the salvage eye plate. After completing a horizontal movement, the welding robot moves in the vertical direction so that the depth camera obtains multi-view data and local point cloud data of the salvage eye plate.

3. The eye plate three-dimensional reconstruction and welding groove guidance positioning method according to claim 1 is characterized in that: In S2, the neural radiation field algorithm uses a fully connected network structure to map the coordinates and directions of the feature points inferred from the input multi-view data to a high-dimensional space; wherein the coordinate x and the direction viewing angle d are substituted into formula (1) for position encoding; when the coordinate x is input, L is 10; when the direction viewing angle d is input, L is 4; γ(p)=(sin(2 0 πp),cos(2 0 πp),...,sin(2 L-1 πp),cos(2 L-1 (1).

4. The eye plate three-dimensional reconstruction and welding groove guidance positioning method according to claim 2 is characterized in that: The neural radiation field algorithm uses a voxel rendering algorithm to model color and density. For the continuity of points on the observation ray, the color of the corresponding pixel on the camera imaging plane can be calculated by an integral formula; wherein the integral formula is shown in formula (2): Among them, the voxel density σ(x) represents the particle density at a certain point of the ray, c(x) represents the particle color at a certain point along the ray direction, and T(x) represents the ray r(t)=o+td from t n to f The cumulative transparency of is expressed as shown in formula (3):

5. The eye plate three-dimensional reconstruction and welding groove guidance positioning method according to claim 1 is characterized in that: In S3, the ICP algorithm is used to perform point cloud registration between the local feature point cloud of the depth camera and the three-dimensional reconstructed point cloud of the salvage eye plate. The point cloud alignment is achieved by calculating the rigid transformation between the two sets of point clouds, and then the overall point cloud of the salvage eye plate is converted into the coordinate system of the depth camera. Finally, the hand-eye matrix obtained by hand-eye calibration between the depth camera and the welding robot is used to convert the three-dimensional point cloud of the salvage eye plate into the robot coordinate system.

6. The eye plate three-dimensional reconstruction and welding groove guidance positioning method according to claim 1 is characterized in that: In S4, the specific steps of using the PROSAC algorithm to segment the groove point cloud to obtain the point cloud data on both sides of the groove on the best fitting plane are: S401: Input the groove point cloud data; S402: Prioritize points according to their features; S403: randomly selecting 3 points from the sorted points for plane fitting; S404: Filtering inliers according to a distance threshold; S405: Iterate and expand the range of sampling points according to priority sorting; S406: Iteratively update the plane with the largest number of internal points as the optimal plane; S407: Determine whether the number of points in the fitting plane reaches the set number; If it is reached, then go to S408; otherwise, return to S402 to re-extract points for plane fitting; S408: Segment the inner points of the fitting plane from the groove point cloud.

7. The eye plate three-dimensional reconstruction and welding groove guidance positioning method according to claim 6 is characterized in that: The mathematical equation of the fitting plane is: Ax+By+Cz+D=0 (4) Where (A, B, C) is the normal vector, and D can be obtained by substituting the parameters of the interior point of the fitting plane; The intersection of the plane equation of the fitting plane M1 and the plane equation of the fitting plane M2 on both sides of the groove is calculated to obtain the welding guide trajectory, where the mathematical equation of the guide trajectory is: The obtained welding trajectory guiding straight line L1 and the adjacent guiding straight line L2 are calculated by the intersection of the two straight lines within the set threshold range to obtain the starting point or the ending point of the welding guidance; and the extracted welding trajectory is projected onto a two-dimensional plane to obtain its intersection, and the intersection is used as the starting and ending points of the welding.

8. A device for three-dimensional reconstruction of eye plate and guiding and positioning of welding groove, characterized in that: It includes a depth camera, a welding robot, a control cabinet and a host computer, wherein the depth camera is arranged at the end of the welding robot, and is used to obtain multiple views of the salvaged eye plate as the welding robot moves; the welding robot is communicatively connected to the control cabinet, and is used to weld the salvaged eye plate on the surface of the sunken ship; the host computer is communicatively connected to the depth camera and the control cabinet, respectively, and is used to process the three-dimensional reconstructed model of the underwater salvaged eye plate, and extract welding trajectory information, and provide it to the welding robot; the method for three-dimensionally reconstructing the salvaged eye plate and guiding and positioning the welding groove by the welding robot is the eye plate three-dimensional reconstruction and welding groove guiding and positioning method according to any one of claims 1 to 7.

Images