A method, device, equipment and storage medium for recovering an underwater robot

Abstract

The present application discloses a method, device, equipment and storage medium for recovering an underwater robot, and relates to the field of computer technology. The method comprises: a mother boat comprises a first mechanical arm and a second mechanical arm, a camera array is deployed on the first mechanical arm and the second mechanical arm, at least four marks are determined on the robot, the mother boat obtains an image set through the camera array, determines a target image, obtains a first coordinate and a second coordinate, determines a third coordinate of the mark in the camera coordinate system according to the first coordinate and the second coordinate, determines a fourth coordinate of the mark in the mother boat coordinate system according to the third coordinate and the rotation angle corresponding to the mark, determines the posture of the robot using the fourth coordinate corresponding to at least four marks and the fifth coordinate in the robot coordinate system, and recovers the robot according to the posture of the robot. The method can obtain a more accurate posture of the underwater robot, reduce damage caused by collision with the mother boat, and reduce the risk of recovery.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for recovering an underwater robot. Background Art

[0002] An underwater robot is an underwater vehicle that does not require human control and performs navigation tasks underwater through autonomous or remote control means. Application scenarios may include underwater environmental detection, life-saving and rescue, and military missions. As the application of underwater robots becomes more and more extensive, the demand for the recovery of underwater robots is also becoming stronger.

[0003] At present, during the automatic recovery of underwater robots, the position and posture of underwater robots can be determined by inertial navigation and other sensors. However, due to the poor positioning accuracy of inertial navigation and other sensors, the accuracy of the position and posture of underwater robots (meter level or worse) is also poor, which makes it difficult to meet the requirements of automatic docking with the mother ship. If docking with the mother ship in this position, it is very easy to cause a collision with the mother ship, causing damage to the underwater robot or the mother ship, and the recovery risk is high. Summary of the invention

[0004] The present application provides a method, device, equipment and storage medium for recovering an underwater robot, which can obtain a more accurate position and posture of the underwater robot, reduce damage caused by collision with a mother boat, and reduce the risk of recovery.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a method for recovering an underwater robot, which is applied to a mother boat, wherein the mother boat comprises a first mechanical arm and a second mechanical arm, wherein the first mechanical arm is perpendicular to the second mechanical arm, wherein a first end of the first mechanical arm is connected to a first end of the second mechanical arm, and a second end of the second mechanical arm is connected to a side fixed end of the mother boat, wherein the second mechanical arm can rotate around an axis direction of the side fixed end, wherein a first camera array is deployed on the first mechanical arm, and a second camera array is deployed on the second mechanical arm, wherein the method comprises:

[0007] Acquire a first image set captured by the first camera array on the robot, and acquire a second image set captured by the second camera array on the robot; the robot includes at least 4 marks on its body;

[0008] Determine respectively from the first image set the first target image corresponding to each mark, and respectively determine from the second image set the second target image corresponding to each mark;

[0009] Determine the first coordinates marked in the first target image in the first camera coordinate system according to the pixel coordinates marked in the first target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the first camera coordinate system; determine the second coordinates marked in the second target image in the second camera coordinate system according to the pixel coordinates marked in the second target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the second camera coordinate system;

[0010] Determine the third coordinate of each marker in the camera coordinate system according to the first coordinate and the second coordinate corresponding to each marker;

[0011] Determine the fourth coordinate of each marker in the mother ship coordinate system according to the third coordinate corresponding to each marker and the rotation angle corresponding to each marker;

[0012] Determine the position and posture of the robot using a fourth coordinate corresponding to at least four markers and a fifth coordinate in the robot coordinate system;

[0013] The robot is recovered according to the position and posture of the robot.

[0014] Among some possible implementations,

[0015] The first coordinate includes a first direction and a second direction, and the second coordinate includes a first direction and a third direction; the first direction of the first coordinate is the same as the first direction of the second coordinate, and the second direction of the first coordinate is different from the third direction of the second coordinate; the third coordinate is determined by:

[0016] The average value of the coordinate value in the first direction of the first coordinate and the coordinate value in the first direction of the second coordinate is used as the coordinate value in the first direction of the third coordinate; the coordinate value in the second direction of the first coordinate is used as the coordinate value in the second direction of the third coordinate; and the coordinate value in the third direction of the second coordinate is used as the coordinate value in the third direction of the third coordinate.

[0017] In some possible implementations, the first coordinate includes a first direction and a second direction, and the second coordinate includes a first direction and a third direction; the first direction of the first coordinate is the same as the first direction of the second coordinate, and the second direction of the first coordinate is different from the third direction of the second coordinate; the third coordinate is determined by:

[0018] Determine whether t is greater than or equal to a threshold of the number of iterations, and obtain a first determination result;

[0019] If the first judgment result indicates that t is less than the iteration number threshold, then according to the t-th round coordinate value of the second direction and the pixel coordinates of the target mark in the second target image, determine the t+1-th round coordinate value of the third direction of the second coordinate and the t+1-th round coordinate value of the first direction, and according to the t+1-th round coordinate value of the third direction and the pixel coordinates of the target mark in the first target image, determine the t+1-th round coordinate value of the first direction and the t+1-th round coordinate value of the second direction of the first coordinate;

[0020] If the first judgment result represents that t is greater than or equal to the iteration number threshold, the t-th round coordinate value in the second direction of the first coordinate is used as the coordinate value in the second direction of the third coordinate, the t-th round coordinate value in the third direction of the third coordinate is used as the coordinate value in the third direction of the third coordinate, and the average of the t+1-th round coordinate value in the first direction of the first coordinate and the t+1-th round coordinate value in the first direction of the second coordinate is used as the coordinate value in the first direction of the third coordinate.

[0021] In some possible implementations, determining the fourth coordinate of each marker in the mother ship coordinate system according to the third coordinate corresponding to each marker and the rotation angle corresponding to each marker includes:

[0022]

[0023] in, Indicates i The fourth coordinate of the marker in the mother ship's coordinate system, Indicates i The rotation angle corresponding to the marker is Indicates i The third coordinate corresponding to the marker, i Is a positive integer.

[0024] In some possible implementations, the posture of the robot includes a position and an attitude angle, and the posture of the robot is determined by:

[0025]

[0026] The robot's posture includes position and attitude angle. represents the robot position, represents the posture of the robot, Indicates i a fifth coordinate in the robot coordinate system, Indicates i The fourth coordinate of a mark in the mother ship's coordinate system.

[0027] In some possible implementations, the method further includes:

[0028] Determine whether the posture of the robot meets the recovery condition, and obtain a second determination result;

[0029] The step of recovering the robot according to the position and posture of the robot comprises:

[0030] If the second judgment result indicates that the position and posture of the robot meets the recycling condition, the robot is recycled according to the position and posture of the robot.

[0031] In some possible implementations, the method further includes:

[0032] If the second judgment result indicates that the posture of the robot does not meet the recovery condition, a control instruction is sent to the robot; the control instruction is used to instruct the robot to adjust its posture.

[0033] In a second aspect, the present application provides a recovery device for an underwater robot, the device being a mother boat, the mother boat comprising a first mechanical arm and a second mechanical arm, the first mechanical arm being perpendicular to the second mechanical arm, the first end of the first mechanical arm being connected to the first end of the second mechanical arm, the second end of the second mechanical arm being connected to a side fixed end of the mother boat, the second mechanical arm being rotatable around an axis direction of the side fixed end, a first camera array being deployed on the first mechanical arm, and a second camera array being deployed on the second mechanical arm; the mother boat comprising:

[0034] an acquisition module, used to acquire a first image set acquired by the first camera array on the robot, and acquire a second image set acquired by the second camera array on the robot; the robot includes at least 4 marks on its body;

[0035] A screening module, used to respectively determine the first target image corresponding to each mark from the first image set, and respectively determine the second target image corresponding to each mark from the second image set;

[0036] A determination module, used to determine the first coordinates of the marks in the first target image in the first camera coordinate system according to the pixel coordinates of the marks in the first target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the first camera coordinate system; determine the second coordinates of the marks in the second target image in the second camera coordinate system according to the pixel coordinates of the marks in the second target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the second camera coordinate system; determine the third coordinates of each mark in the camera coordinate system according to the first coordinates and the second coordinates corresponding to each mark; determine the fourth coordinates of each mark in the mother ship coordinate system according to the third coordinates corresponding to each mark and the rotation angle corresponding to each mark; determine the position and posture of the robot using the fourth coordinates corresponding to at least 4 marks and the fifth coordinate in the robot coordinate system;

[0037] The recovery module is used to recover the robot according to the position and posture of the robot.

[0038] In a third aspect, the present application provides a computing device, including a memory and a processor;

[0039] One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as described in any one of the first aspects.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method as described in any one of the first aspects.

[0041] It can be seen from the above technical solution that the present application has at least the following beneficial effects:

[0042] In the present application, the mother boat includes a first robotic arm and a second robotic arm, the first end of the first robotic arm is connected to the first end of the second robotic arm, the second end of the second robotic arm is connected to the side fixed end of the mother boat, the first robotic arm is perpendicular to the second robotic arm, and the second robotic arm can rotate around the axis direction of the side fixed end, the first robotic arm is deployed with a first camera array, the second robotic arm is deployed with a second camera array, at least four marks are determined on the robot, the mother boat obtains a first image set and a second image set through the first camera array and the second camera array, and determines a first target image and a second target image corresponding to each mark according to the first image set and the second image set, respectively, and determines a first target image and a second target image corresponding to each mark according to the first camera array and the second image set. The pixel coordinates of the markers in the target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the first camera coordinate system are used to determine the first coordinates of the markers in the first target image in the first camera coordinate system. Similarly, the second coordinates are determined according to the first coordinates and the second coordinates corresponding to each marker, and the third coordinates of each marker in the camera coordinate system are determined according to the third coordinates corresponding to each marker and the rotation angle corresponding to each marker. The fourth coordinates of each marker in the mother ship coordinate system are determined by using the fourth coordinates corresponding to at least 4 markers and the fifth coordinates in the robot coordinate system. The robot is recovered according to the robot's posture. At present, in the process of automatic recovery of underwater robots, the posture of underwater robots can be determined by sensors such as inertial navigation. However, due to the poor positioning accuracy of sensors such as inertial navigation, the accuracy of the posture of underwater robots (meter level or worse) is also poor, which makes it difficult to meet the requirements of automatic docking with the mother ship. If docking with the mother ship in this posture, it is very easy to cause a collision with the mother ship, causing damage to the underwater robot or the mother ship, and the recovery risk is high. It can be seen that the present application uses the first robotic arm and the second robotic arm to more accurately position the robot, which can obtain a more accurate posture of the underwater robot, reduce damage caused by collision with the mother boat, and reduce the risk of recovery.

[0043] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of features or beneficial effects means that specific technical features, technical solutions or beneficial effects are included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the present embodiment can also be combined in any appropriate manner. Those skilled in the art will understand that the embodiment can be realized without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in a specific embodiment that does not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of an application scenario of the recovery of an underwater robot provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of the recovery of an underwater robot provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a marking of an underwater robot provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a mapping relationship between pixel coordinates and coordinates in a first camera coordinate system provided in an embodiment of the present application;

[0048] Figure 5 A schematic diagram of a mapping relationship between pixel coordinates and coordinates in a second camera coordinate system provided in an embodiment of the present application;

[0049] Figure 6 A schematic diagram of a coordinate system of a third coordinate provided in an embodiment of the present application;

[0050] Figure 7 A schematic diagram of the position of a robot relative to a mother boat provided in an embodiment of the present application;

[0051] Figure 8 A schematic diagram of a recovery device for an underwater robot provided in an embodiment of the present application;

[0052] Fig. 9 A schematic diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The terms "first", "second", "third", etc. in the specification of this application and the accompanying drawings are used to distinguish different objects rather than to limit a specific order.

[0054] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0055] The present application embodiment provides an application scenario of underwater robot recovery, such as Figure 1 As shown, this figure is a schematic diagram of an application scenario of the recovery of an underwater robot provided in an embodiment of the present application.

[0056] exist Figure 1In the scene shown, it can be seen that the first robotic arm 101 is perpendicular to the second robotic arm 103, the first end of the first robotic arm 101 is connected to the second end of the second robotic arm 103, the second end of the second robotic arm 103 is connected to the side fixed end of the mother boat 105, the first robotic arm 101 is deployed with a first camera array 102, the second robotic arm 103 is deployed with a second camera array 104, the mother boat 105 controls the underwater robot 106 to approach the mother boat, and the underwater robot 106 drives into the first camera array 102 and the second camera array 104 on the first robotic arm 101. Within the shooting range of the second camera array 104 of the second manipulator 103, the mother boat 105 determines the position of the underwater robot 106 through the images shot by the first camera array 102 and the second camera array 104, u is the first direction in the mother boat 105 coordinate system, v is the second direction in the mother boat 105 coordinate system, w is the third direction in the mother boat 105 coordinate system, x is the first direction in the underwater robot 106 coordinate system, y is the second direction in the underwater robot 106 coordinate system, and z is the third direction in the underwater robot 106 coordinate system. In combination with the above scenario, the underwater robot recovery method provided by the embodiment of the present application is introduced below.

[0057] The present application embodiment provides a method for recovering an underwater robot, such as Figure 2 As shown, this figure is a flowchart of the recovery of an underwater robot provided by an embodiment of the present application. The method is applied to a mother boat, the mother boat includes a first mechanical arm and a second mechanical arm, the first mechanical arm is perpendicular to the second mechanical arm, the first end of the first mechanical arm is connected to the first end of the second mechanical arm, the second end of the second mechanical arm is connected to the side fixed end of the mother boat, the second mechanical arm can rotate around the axis direction of the side fixed end, the first mechanical arm is deployed with a first camera array, and the second mechanical arm is deployed with a second camera array, the method includes:

[0058] S201: The mother boat obtains a first image set captured by a first camera array of the underwater robot.

[0059] When the robot approaches the mother boat, the robot is controlled to enter a hovering state, and the first camera array on the first robotic arm shoots the underwater robot to obtain multiple groups of first images. The first images with the same coding are integrated into a first image set. The mother boat obtains the first image set to provide a sufficient basis for the subsequent position and posture of the underwater robot.

[0060] The number of cameras in the first camera array is selected based on the parameters of the cameras themselves and the size of the underwater robot, and the length of the first camera array is greater than or equal to the axial length of the underwater robot.

[0061] S202: The mother boat obtains a second image set captured by a second camera array of the underwater robot.

[0062] When the robot approaches the mother boat, the robot is controlled to enter a hovering state, and the second camera array on the second robotic arm shoots the underwater robot to obtain multiple groups of second images. The second images with the same coding are integrated into a second image set. The mother boat obtains the second image set to provide a sufficient basis for the subsequent position and posture of the underwater robot.

[0063] The number of cameras in the second camera array is selected based on the parameters of the cameras themselves and the size of the underwater robot, and the length of the second camera array is greater than or equal to the axial length of the underwater robot.

[0064] In order to better determine the location of the underwater robot, at least four marks should be identified on the body of the underwater robot before the underwater recovery operation, such as Figure 3 As shown, the figure is a schematic diagram of a mark of an underwater robot provided by an embodiment of the present application. The figure exemplarily shows the positions of the first mark 301, the second mark 302, the third mark 303 and the fourth mark 304 on the underwater robot. The purpose of determining the four marks is to better determine the position and posture of the underwater robot. Only four marks of the underwater robot are shown in this application. In practice, different numbers of marks can be determined as needed. The mark can be composed of a group of light-emitting indication devices with light sources. The illumination of the light can be set according to the design requirements, and the light source dot matrix on the light-emitting indication device is encoded to distinguish different positions on the robot. When the robot approaches the mother boat, the coded light-emitting indication device is automatically turned on, and the first camera array and the second camera array start working.

[0065] S203: The mother ship determines the first target images corresponding to the respective marks from the first image set.

[0066] In a laboratory simulation environment, sample images taken by the first camera array and the second camera array at different locations and under different lighting conditions are obtained as training data for the deep learning model of encoded image recognition and classification. The sample images are used to train the model to output sample images with markers and the pixel coordinates of the markers. After calculating the distance between the marker and the center point of the image, the sample image with the marker with the shortest distance is determined as the target image.

[0067] The following is a detailed introduction to the acquisition of the first target image.

[0068] The mother boat first determines the pixel coordinates of the marker in the image from the first image set, calculates the distance between the pixel coordinates and the center point of the image, obtains an image in which the marker in the camera is closest to the center point of the camera image, and determines this image as the first target image, which is the image with the smallest error in determining the marker position.

[0069] S204: The mother ship determines the second target images corresponding to the respective marks from the second image set.

[0070] The following is a detailed introduction to the acquisition of the second target image.

[0071] The mother boat selects an image in which the image of the mark in the camera is closest to the center point of the camera image from the second image set, and determines this image as the second target image, which is the image with the smallest error in determining the position of the mark.

[0072] S205 , determining first coordinates of the mark in the first target image in the first camera coordinate system according to the pixel coordinates of the mark in the first target image and a mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the first camera coordinate system.

[0073] like Figure 4 As shown in FIG. 1 , the figure is a schematic diagram of a mapping relationship between pixel coordinates and coordinates in a first camera coordinate system provided by an embodiment of the present application. In the figure, the center point C (J / 2, K / 2) of the pixel coordinates corresponds to the center point C (J / 2, K / 2) in the first coordinate system. , one of the labels (j, k) of the pixel coordinates corresponds to the first coordinate system , u' is the first direction of the first camera coordinate system, and v' is the second direction of the first camera coordinate system.

[0074] Each mark on the underwater robot consists of three groups of directions, among which the first coordinates formed by the first direction and the second direction are obtained by photographing the first camera array. The first mechanical arm where the first camera array is located photographs the robot when the robot is hovering, and then the first mechanical arm rotates. Each time it rotates an angle, the first camera array takes a group of images. The rotation angle Can be set freely, assuming the rotation angle is 60 degrees, then the first camera array on the first robotic arm needs to capture times, thereby determining the first coordinate.

[0075] S206 , determining second coordinates of the mark in the second target image in the second camera coordinate system according to the pixel coordinates of the mark in the second target image and a mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the second camera coordinate system.

[0076] like Figure 5 As shown in FIG. 1 , the figure is a schematic diagram of a mapping relationship between pixel coordinates and coordinates in a second camera coordinate system provided by an embodiment of the present application. In the figure, the center point E (J / 2, L / 2) of the pixel coordinates corresponds to the coordinates in the second coordinate system. , one of the labels (j, l) of the pixel coordinates corresponds to the second coordinate system , u' is the first direction of the second camera coordinate system, and w' is the third direction of the second camera coordinate system.

[0077] The second coordinates formed by the first direction and the third direction are obtained by taking pictures by the second camera array. The second robot arm where the second camera array is located takes pictures of the robot when the robot is hovering. Then the second robot arm rotates. Every time it rotates an angle, the second camera array takes a group of images. Can be set freely, assuming the rotation angle is 60 degrees, the second camera array on the second robotic arm needs to capture times, thereby determining the second coordinate.

[0078] S207 . Determine the third coordinate of each marker in the camera coordinate system according to the first coordinate and the second coordinate corresponding to each marker.

[0079] like Figure 6 As shown, this figure is a schematic diagram of a coordinate system of a third coordinate provided in an embodiment of the present application. In the figure, u is the first direction in the mother ship coordinate system, v is the second direction in the mother ship coordinate system, w is the third direction in the mother ship coordinate system, u' is the first direction in the third coordinate system, v' is the second direction in the third coordinate system, and w' is the third direction in the third coordinate system. is the rotation angle.

[0080] The first coordinate includes a first direction and a second direction, and the second coordinate includes a first direction and a third direction. The first direction in the first coordinate is the same as the first direction in the second coordinate, and the second direction in the first coordinate is different from the third direction in the second coordinate. According to the first coordinate and the second coordinate corresponding to each mark, the third coordinate can be determined in the following manner:

[0081] The average value of the coordinate value in the first direction of the first coordinate and the coordinate value in the first direction of the second coordinate is used as the coordinate value in the first direction of the third coordinate; the coordinate value in the second direction of the first coordinate is used as the coordinate value in the second direction of the third coordinate; the coordinate value in the third direction of the second coordinate is used as the coordinate value in the third direction of the third coordinate, therefore, the first direction, second direction and third direction of the third coordinate are all determined.

[0082] The third coordinate can also be determined in the following way:

[0083] Determine whether t is greater than or equal to a threshold of the number of iterations, and obtain a first determination result;

[0084] If the first judgment result indicates that t is less than the iteration number threshold, then according to the t-th round coordinate value of the second direction and the pixel coordinates of the target mark in the second target image, determine the t+1-th round coordinate value of the third direction of the second coordinate and the t+1-th round coordinate value of the first direction, and according to the t+1-th round coordinate value of the third direction and the pixel coordinates of the target mark in the first target image, determine the t+1-th round coordinate value of the first direction and the t+1-th round coordinate value of the second direction of the first coordinate;

[0085] If the first judgment result indicates that t is greater than or equal to the iteration number threshold, the t-th coordinate value in the second direction of the first coordinate is used as the coordinate value in the second direction of the third coordinate, the t-th coordinate value in the third direction of the third coordinate is used as the coordinate value in the third direction of the third coordinate, and the average of the t+1-th coordinate value in the first direction of the first coordinate and the t+1-th coordinate value in the first direction of the second coordinate is used as the coordinate value in the first direction of the third coordinate. Therefore, the first direction, second direction and third direction of the third coordinate are all determined.

[0086] According to the above method, a schematic diagram is obtained, such as Figure 7 As shown, the figure is a schematic diagram of the position of a robot relative to a mother boat provided by an embodiment of the present application. In the figure, it is assumed that the camera A in the first camera array photographs the target mark, the camera B in the second camera array photographs the target mark, the distance between camera A and the second robotic arm is V0, the distance between camera B and the first robotic arm is W0, the coordinate value of the second direction is composed of the pixel coordinates m and V0 of the target mark in the first target image, and the coordinate value of the third direction is composed of the pixel coordinates n and W0 of the target mark in the second target image; assuming that the iteration number threshold is 3, t=2, then t is less than the iteration number threshold, then according to the second round coordinate value of the second direction and the pixel coordinate n of the target mark in the second target image, determine the third round coordinate value of the third direction , if the second round coordinate value of the third direction is directly used The position determined as the target marker is not accurate, and depth information must be considered, so the third round coordinate value of the third direction is and the pixel coordinate m of the target mark in the first target image, determine the third round coordinate value of the second direction . Thus, the iteration number threshold is reached, and the third round coordinate value of the second direction of the first coordinate can be As the coordinate value of the second direction of the third coordinate, the third wheel coordinate value of the third direction of the second coordinate As the coordinate value of the third direction of the third coordinate, the average of the third wheel coordinate value of the first direction of the first coordinate and the third wheel coordinate value of the first direction of the second coordinate is taken as the coordinate value of the first direction of the third coordinate.

[0087] Assuming the iteration threshold is 3, t=3, then t is equal to the iteration threshold, then the third coordinate value of the second direction of the first coordinate is As the coordinate value of the second direction of the third coordinate, the third wheel coordinate value of the third direction of the third coordinate As the coordinate value of the third direction of the third coordinate, the average of the t+1th round coordinate value of the first direction of the first coordinate and the t+1th round coordinate value of the first direction of the second coordinate is taken as the coordinate value of the first direction of the third coordinate. Therefore, the first direction, second direction and third direction of the third coordinate are all determined.

[0088] S208. Determine the fourth coordinate of each mark in the mother ship coordinate system according to the third coordinate corresponding to each mark and the rotation angle corresponding to each mark.

[0089] The fourth coordinate of each marker in the mother ship coordinate system is determined by the following method:

[0090]

[0091] in, Indicates i The fourth coordinate of the marker in the mother ship's coordinate system, Indicates i The rotation angle corresponding to the marker is Indicates i The third coordinate corresponding to the marker, i Is a positive integer.

[0092] Exemplarily, the fourth coordinate of the first marker 301 in the mother ship coordinate system is determined according to the formula:

[0093]

[0094] The fourth coordinate of the first marker 301 in the mother ship coordinate system can be obtained: .

[0095] S209: Determine the position and posture of the robot using the fourth coordinates corresponding to the at least four markers and the fifth coordinate in the robot coordinate system.

[0096] The robot's posture includes position and attitude angle. The robot's posture is determined by:

[0097]

[0098] Among them, the robot's posture includes position and attitude angle. represents the robot position, represents the posture of the robot, is the yaw angle of the underwater robot, is the pitch angle of the underwater robot, is the rolling angle of the underwater robot, Indicates i a fifth coordinate in the robot coordinate system, Indicates i The fourth coordinate of a mark in the mother ship's coordinate system.

[0099] In some examples, you can choose to use 6 markers to determine the robot's posture. For example, you can substitute the values ​​of the 6 markers into the formula to calculate , .

[0100] Exemplarily, the first marker 301 is selected and brought into the formula:

[0101] Substituting the values ​​of the six markers into the formula will result in six sets of equations. Through these six sets of equations, the six unknowns mentioned above, i.e., the position and posture of the robot, can be solved.

[0102] The robot's position can also be determined by:

[0103] Exemplarily, the positions of the four markers on the robot are calculated according to the following formula.

[0104]

[0105]

[0106]

[0107]

[0108]

[0109]

[0110] Among them, the robot contains 4 marks, set the mark , , and are four markers on the robot, and the distances between the four markers are known. and The distance between , and The distance between , and The distance between , and The distance between , and The distance between , and The distance between In the formula Can be achieved through , To express it in the formula, we can solve , .

[0111] In order to make the calculation results more accurate, multiple sets of marked position coordinates are selected in practice and brought into the formula for calculation and solution. The average and variance of the multiple sets of results are calculated to improve the accuracy of the calculation results.

[0112] S210: Recover the robot according to the position and posture of the robot.

[0113] Determine whether the robot's posture meets the recovery condition, and obtain a second determination result;

[0114] If the second judgment result indicates that the robot's posture meets the recovery conditions, the robot is recovered according to the robot's posture; if the second judgment result indicates that the robot's posture does not meet the recovery conditions, a control instruction is sent to the robot; the mother boat instructs the robot to adjust its posture to meet the recovery conditions and recovers the robot.

[0115] In the present application, the mother boat includes a first robotic arm and a second robotic arm, the first end of the first robotic arm is connected to the first end of the second robotic arm, the second end of the second robotic arm is connected to the side fixed end of the mother boat, the first robotic arm is perpendicular to the second robotic arm, and the second robotic arm can rotate around the axis direction of the side fixed end, the first robotic arm is deployed with a first camera array, the second robotic arm is deployed with a second camera array, at least four marks are determined on the robot, the mother boat obtains a first image set and a second image set through the first camera array and the second camera array, and determines a first target image and a second target image corresponding to each mark according to the first image set and the second image set, respectively, and determines a first target image and a second target image corresponding to each mark according to the first camera array and the second image set. The pixel coordinates of the markers in the target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the first camera coordinate system are used to determine the first coordinates of the markers in the first target image in the first camera coordinate system. Similarly, the second coordinates are determined according to the first coordinates and the second coordinates corresponding to each marker, and the third coordinates of each marker in the camera coordinate system are determined according to the third coordinates corresponding to each marker and the rotation angle corresponding to each marker. The fourth coordinates of each marker in the mother ship coordinate system are determined by using the fourth coordinates corresponding to at least 4 markers and the fifth coordinates in the robot coordinate system. The robot is recovered according to the robot's posture. At present, in the process of automatic recovery of underwater robots, the posture of underwater robots can be determined by sensors such as inertial navigation. However, due to the poor positioning accuracy of sensors such as inertial navigation, the accuracy of the posture of underwater robots (meter level or worse) is also poor, which makes it difficult to meet the requirements of automatic docking with the mother ship. If docking with the mother ship in this posture, it is very easy to cause a collision with the mother ship, causing damage to the underwater robot or the mother ship, and the recovery risk is high. It can be seen that the present application uses the first robotic arm and the second robotic arm to more accurately position the robot, which can obtain a more accurate posture of the underwater robot, reduce damage caused by collision with the mother boat, and reduce the risk of recovery.

[0116] Combination of the above Figures 1 to 6 A method for recovering an underwater robot provided in an embodiment of the present application is introduced in detail. The following will introduce the device and equipment provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0117] like Figure 8 As shown, the figure is a schematic diagram of a recovery device of an underwater robot provided in an embodiment of the present application, the device includes a mother boat and an underwater robot, the mother boat includes a first mechanical arm and a second mechanical arm, the first mechanical arm is perpendicular to the second mechanical arm, the first end of the first mechanical arm is connected to the first end of the second mechanical arm, the second end of the second mechanical arm is connected to the side fixed end of the mother boat, the second mechanical arm can rotate around the axis direction of the side fixed end, the first mechanical arm is deployed with a first camera array, and the second mechanical arm is deployed with a second camera array; the mother boat includes:

[0118] An acquisition module 701 is used to acquire a first image set captured by the first camera array on the robot, and acquire a second image set captured by the second camera array on the robot; the robot includes at least 4 marks on its body;

[0119] A screening module 702, configured to respectively determine the first target images corresponding to the respective marks from the first image set, and respectively determine the second target images corresponding to the respective marks from the second image set;

[0120] The determination module 703 is used to determine the first coordinates of the marks in the first target image in the first camera coordinate system according to the pixel coordinates of the marks in the first target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the first camera coordinate system; determine the second coordinates of the marks in the second target image in the second camera coordinate system according to the pixel coordinates of the marks in the second target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the second camera coordinate system; determine the third coordinates of each mark in the camera coordinate system according to the first coordinates and the second coordinates corresponding to each mark; determine the fourth coordinates of each mark in the mother ship coordinate system according to the third coordinates corresponding to each mark and the rotation angle corresponding to each mark; determine the position and posture of the robot using the fourth coordinates corresponding to at least four marks and the fifth coordinates in the robot coordinate system;

[0121] The recovery module 704 is used to recover the robot according to the position and posture of the robot.

[0122] Optionally, the first coordinate includes a first direction and a second direction, and the second coordinate includes a first direction and a third direction; the first direction of the first coordinate is the same as the first direction of the second coordinate, and the second direction of the first coordinate is different from the third direction of the second coordinate; the third coordinate is determined by:

[0123] The average value of the coordinate value in the first direction of the first coordinate and the coordinate value in the first direction of the second coordinate is used as the coordinate value in the first direction of the third coordinate; the coordinate value in the second direction of the first coordinate is used as the coordinate value in the second direction of the third coordinate; and the coordinate value in the third direction of the second coordinate is used as the coordinate value in the third direction of the third coordinate.

[0124] Optionally, the first coordinate includes a first direction and a second direction, and the second coordinate includes a first direction and a third direction; the first direction of the first coordinate is the same as the first direction of the second coordinate, and the second direction of the first coordinate is different from the third direction of the second coordinate; the third coordinate is determined by:

[0125] Determine whether t is greater than or equal to a threshold of the number of iterations, and obtain a first determination result;

[0126] If the first judgment result indicates that t is less than the iteration number threshold, then according to the t-th round coordinate value of the second direction and the pixel coordinates of the target mark in the second target image, determine the t+1-th round coordinate value of the third direction of the second coordinate and the t+1-th round coordinate value of the first direction, and according to the t+1-th round coordinate value of the third direction and the pixel coordinates of the target mark in the first target image, determine the t+1-th round coordinate value of the first direction and the t+1-th round coordinate value of the second direction of the first coordinate;

[0127] If the first judgment result represents that t is greater than or equal to the iteration number threshold, the t-th round coordinate value in the second direction of the first coordinate is used as the coordinate value in the second direction of the third coordinate, the t-th round coordinate value in the third direction of the third coordinate is used as the coordinate value in the third direction of the third coordinate, and the average of the t+1-th round coordinate value in the first direction of the first coordinate and the t+1-th round coordinate value in the first direction of the second coordinate is used as the coordinate value in the first direction of the third coordinate.

[0128] Optionally, determining the fourth coordinate of each marker in the mother ship coordinate system according to the third coordinate corresponding to each marker and the rotation angle corresponding to each marker includes:

[0129]

[0130] in, Indicates i The fourth coordinate of the marker in the mother ship's coordinate system, Indicates i The rotation angle corresponding to the marker is Indicates i The third coordinate corresponding to the marker, i Is a positive integer.

[0131] Optionally, the posture of the robot includes a position and an attitude angle, and the posture of the robot is determined by:

[0132]

[0133] The robot's posture includes position and attitude angle. represents the robot position, represents the posture of the robot, Indicates i a fifth coordinate in the robot coordinate system, Indicates i The fourth coordinate of a mark in the mother ship's coordinate system.

[0134] Optionally, the method further includes:

[0135] Determine whether the posture of the robot meets the recovery condition, and obtain a second determination result;

[0136] The step of recovering the robot according to the position and posture of the robot comprises:

[0137] If the second judgment result indicates that the position and posture of the robot meets the recycling condition, the robot is recycled according to the position and posture of the robot.

[0138] Optionally, the method further includes:

[0139] If the second judgment result indicates that the posture of the robot does not meet the recovery condition, a control instruction is sent to the robot; the control instruction is used to instruct the robot to adjust its posture.

[0140] The present application also provides a computing device. Fig. 9 As shown, this figure is a schematic diagram of a computing device provided by an embodiment of the present application, and the computing device 800 includes a bus 801, a processor 802, a communication interface 803 and a memory 804. The processor 802, the memory 804 and the communication interface 803 communicate through the bus 801.

[0141] The bus 801 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0142] The processor 802 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0143] The communication interface 803 is used for communicating with the outside.

[0144] The memory 804 may include a volatile memory, such as a random access memory (RAM). The memory 804 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0145] The memory 804 stores executable codes, and the processor 802 executes the executable codes to perform a method for recovering an underwater robot.

[0146] Specifically, in implementing Figure 8 In the case of the embodiment shown, and Figure 8 When each module or unit of the underwater robot recovery device described in the embodiment is implemented by software, the execution Figure 8 The software or program code required for the functions of each module / unit in the system may be partially or completely stored in the memory 804. The processor 802 executes the program code corresponding to each unit stored in the memory 804 to execute a method for recovering an underwater robot.

[0147] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute a method for recovering an underwater robot.

[0148] The embodiment of the present application further provides a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part.

[0149] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer or data center to another website, computer or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0150] When the computer program product is executed by a computer, the computer executes any method of a method for recovering an underwater robot. The computer program product may be a software installation package, and when any method of a method for recovering an underwater robot is required, the computer program product may be downloaded and executed on a computer.

[0151] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0152] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A method for recovering an underwater robot, characterized in that: Applied to a mother boat, the mother boat comprises a first mechanical arm and a second mechanical arm, the first mechanical arm is perpendicular to the second mechanical arm, a first end of the first mechanical arm is connected to a first end of the second mechanical arm, a second end of the second mechanical arm is connected to a side fixed end of the mother boat, the second mechanical arm can rotate around the axis direction of the side fixed end, a first camera array is deployed on the first mechanical arm, and a second camera array is deployed on the second mechanical arm, the method comprises: Acquire a first image set captured by the first camera array on the robot, and acquire a second image set captured by the second camera array on the robot; the robot includes at least 4 marks on its body; Determine respectively from the first image set the first target image corresponding to each mark, and respectively determine from the second image set the second target image corresponding to each mark; Determine the first coordinates marked in the first target image in the first camera coordinate system according to the pixel coordinates marked in the first target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the first camera coordinate system; determine the second coordinates marked in the second target image in the second camera coordinate system according to the pixel coordinates marked in the second target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the second camera coordinate system; The first coordinate includes a first direction and a second direction, and the second coordinate includes a first direction and a third direction; the first direction of the first coordinate is the same as the first direction of the second coordinate, and the second direction of the first coordinate is different from the third direction of the second coordinate; the third coordinate is determined by: Determine whether t is greater than or equal to a threshold of the number of iterations, and obtain a first determination result; If the first judgment result indicates that t is less than the iteration number threshold, then according to the t-th round coordinate value of the second direction and the pixel coordinates of the target mark in the second target image, determine the t+1-th round coordinate value of the third direction of the second coordinate and the t+1-th round coordinate value of the first direction, and according to the t+1-th round coordinate value of the third direction and the pixel coordinates of the target mark in the first target image, determine the t+1-th round coordinate value of the first direction and the t+1-th round coordinate value of the second direction of the first coordinate; If the first judgment result indicates that t is greater than or equal to the iteration number threshold, the t-th round coordinate value in the second direction of the first coordinate is used as the coordinate value in the second direction of the third coordinate, the t-th round coordinate value in the third direction of the third coordinate is used as the coordinate value in the third direction of the third coordinate, and the average of the t+1-th round coordinate value in the first direction of the first coordinate and the t+1-th round coordinate value in the first direction of the second coordinate is used as the coordinate value in the first direction of the third coordinate; Determine the third coordinate of each marker in the camera coordinate system according to the first coordinate and the second coordinate corresponding to each marker; According to the third coordinates corresponding to each mark and the rotation angle corresponding to each mark, the fourth coordinates of each mark in the mother ship coordinate system are determined, including: in, Indicates i The fourth coordinate of the marker in the mother ship's coordinate system, Indicates i The rotation angle corresponding to the marker is Indicates i The third coordinate corresponding to the marker, i is a positive integer; The robot's posture is determined by using the fourth coordinates corresponding to at least four markers and the fifth coordinate in the robot coordinate system, wherein the robot's posture includes a position and an attitude angle, and the robot's posture is determined by: The robot's posture includes position and attitude angle. represents the robot position, represents the posture of the robot, Indicates i a fifth coordinate in the robot coordinate system, Indicates i The fourth coordinate of the marker in the mother ship's coordinate system; The robot is recovered according to the position and posture of the robot.

2. The method according to claim 1, characterized in that The first coordinate includes a first direction and a second direction, and the second coordinate includes a first direction and a third direction; the first direction of the first coordinate is the same as the first direction of the second coordinate, and the second direction of the first coordinate is different from the third direction of the second coordinate; the third coordinate is determined by: The average value of the coordinate value in the first direction of the first coordinate and the coordinate value in the first direction of the second coordinate is used as the coordinate value in the first direction of the third coordinate; the coordinate value in the second direction of the first coordinate is used as the coordinate value in the second direction of the third coordinate; and the coordinate value in the third direction of the second coordinate is used as the coordinate value in the third direction of the third coordinate.

3. The method according to claim 1, characterized in that The method further comprises: Determine whether the posture of the robot meets the recovery condition, and obtain a second determination result; The step of recovering the robot according to the position and posture of the robot comprises: If the second judgment result indicates that the position and posture of the robot meets the recycling condition, the robot is recycled according to the position and posture of the robot.

4. The method according to claim 3, characterized in that The method further comprises: If the second judgment result indicates that the posture of the robot does not meet the recycling condition, a control instruction is sent to the robot; the control instruction is used to instruct the robot to adjust its posture.

5. A recovery device for an underwater robot, characterized in that: The device is a mother boat, the mother boat comprises a first mechanical arm and a second mechanical arm, the first mechanical arm is perpendicular to the second mechanical arm, the first end of the first mechanical arm is connected to the first end of the second mechanical arm, the second end of the second mechanical arm is connected to the side fixed end of the mother boat, the second mechanical arm can rotate around the axis direction of the side fixed end, the first mechanical arm is deployed with a first camera array, and the second mechanical arm is deployed with a second camera array; the mother boat comprises: an acquisition module, used to acquire a first image set acquired by the first camera array on the robot, and acquire a second image set acquired by the second camera array on the robot; the robot includes at least 4 marks on its body; A screening module, used to respectively determine the first target image corresponding to each mark from the first image set, and respectively determine the second target image corresponding to each mark from the second image set; A determination module, used to determine the first coordinates marked in the first target image in the first camera coordinate system according to the pixel coordinates marked in the first target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the first camera coordinate system; determine the second coordinates marked in the second target image in the second camera coordinate system according to the pixel coordinates marked in the second target image and the mapping relationship between the pixel coordinates in the pre-fitted image and the coordinates in the second camera coordinate system, wherein the first coordinates include a first direction and a second direction, and the second coordinates include a first direction and a third direction; the first direction of the first coordinates is the same as the first direction of the second coordinates, and the second direction of the first coordinates is different from the third direction of the second coordinates; and the third coordinates are determined in the following manner: Determine whether t is greater than or equal to a threshold of the number of iterations, and obtain a first determination result; If the first judgment result indicates that t is less than the iteration number threshold, then according to the t-th round coordinate value of the second direction and the pixel coordinates of the target mark in the second target image, determine the t+1-th round coordinate value of the third direction of the second coordinate and the t+1-th round coordinate value of the first direction, and according to the t+1-th round coordinate value of the third direction and the pixel coordinates of the target mark in the first target image, determine the t+1-th round coordinate value of the first direction and the t+1-th round coordinate value of the second direction of the first coordinate; If the first judgment result indicates that t is greater than or equal to the iteration number threshold, the t-th round coordinate value in the second direction of the first coordinate is used as the coordinate value in the second direction of the third coordinate, the t-th round coordinate value in the third direction of the third coordinate is used as the coordinate value in the third direction of the third coordinate, and the average of the t+1-th round coordinate value in the first direction of the first coordinate and the t+1-th round coordinate value in the first direction of the second coordinate is used as the coordinate value in the first direction of the third coordinate; Determine the third coordinate of each marker in the camera coordinate system according to the first coordinate and the second coordinate corresponding to each marker; determine the fourth coordinate of each marker in the mother ship coordinate system according to the third coordinate corresponding to each marker and the rotation angle corresponding to each marker, including: in, Indicates i The fourth coordinate of the marker in the mother ship's coordinate system, Indicates i The rotation angle corresponding to the marker is Indicates i The third coordinate corresponding to the marker, i is a positive integer; The robot's posture is determined by using the fourth coordinates corresponding to at least four markers and the fifth coordinate in the robot coordinate system, wherein the robot's posture includes a position and an attitude angle, and the robot's posture is determined by: The robot's posture includes position and attitude angle. represents the robot position, represents the posture of the robot, Indicates i a fifth coordinate in the robot coordinate system, Indicates i The fourth coordinate of the marker in the mother ship's coordinate system; The recovery module is used to recover the robot according to the position and posture of the robot.

6. A computing device, characterized in that including memory and processor; One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as claimed in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 4.

Images