Optical sensor, pose control method of optical sensor, and related device

Abstract

Embodiments of the present application provide an optical sensor, a pose control method of the optical sensor and related equipment, which are used to solve the problem of pose control of an underwater robot in a closed environment. The method of the embodiments of the present application comprises: after receiving a first captured image and a second captured image sent by an image capturing module, extracting first position data of an underwater robot from the first captured image; determining displacement change data of the underwater robot according to the first captured image and the second captured image; determining relative displacement data of the underwater robot according to the first position data and the displacement change data; and controlling the underwater robot to move to target position data according to the relative position data.

Description

Technical Field

[0001] This application relates to the field of underwater robot technology, and in particular to a pose control method and related equipment. Background Technology

[0002] Remotely operated underwater vehicles (ROVs) have the need for attitude hovering control, such as performing tasks like fixed-point reconnaissance and photography in the water. However, the influence of water currents can cause the ROV's position to drift. In this case, there must be a feedback source of position change information to the controller to keep the ROV in position hovering control.

[0003] Currently, a complete spatial attitude description is mainly obtained by integrating a depth gauge and an inertial measurement unit (IMU) with a Doppler velocity log (DVL). However, since the Doppler velocity log is a sensor composed of multiple acoustic transceivers, its use is limited to open water environments such as lakes and oceans in structurally enclosed environments. In enclosed scenarios such as swimming pools and water tanks, sound waves are interfered with by multiple reflections in the narrow space, rendering it unusable. Summary of the Invention

[0004] This application provides an optical sensor, a pose control method for the optical sensor, and related equipment to solve the attitude control problem of underwater robots in a closed environment.

[0005] The first aspect of this application provides a pose control method applied to an underwater robot, including:

[0006] Upon receiving the first and second acquired images from the image acquisition module, the first position data of the underwater robot is extracted from the first acquired image.

[0007] The displacement change data of the underwater robot are determined based on the first acquired image and the second acquired image;

[0008] The relative displacement data of the underwater robot is determined based on the first position data and the displacement change data;

[0009] Based on the relative position data, the underwater robot is controlled to move to the target position data.

[0010] Optionally, extracting the first position data of the underwater robot from the first acquired image includes:

[0011] The first image feature of the first acquired image is set as the image coordinate system;

[0012] The first image coordinate data corresponding to the first acquired image of the underwater robot is determined according to the image coordinate system;

[0013] Convert the first image coordinate data into the first position data.

[0014] Optionally, determining the displacement change data of the underwater robot based on the first acquired image and the second acquired image includes:

[0015] Set initial displacement data; wherein, the initial displacement data is used to describe the displacement data generated by the image acquisition module during the process of generating the first acquired image to the second acquired image;

[0016] Set the second image coordinate data of the second acquired image, and determine the displacement change data based on the second image coordinate data, the first position data, and the initial displacement data.

[0017] Optionally, determining the relative displacement data of the underwater robot based on the first position data and the displacement change data includes:

[0018] Based on the first image coordinate data and the second image coordinate data, an intermediate coefficient function is set;

[0019] Substitute the first position data and the displacement change data into the intermediate coefficient function to obtain the relative displacement data.

[0020] Optionally, the data for controlling the underwater robot to move to the target location includes:

[0021] The second position data of the underwater robot is extracted from the second acquired image based on the displacement change data;

[0022] Based on the second position data and the relative position data, the underwater robot is controlled to move to the target position data.

[0023] Optionally, the optical sensor further includes a pose hovering feedback device, a motion controller, and a vector thruster, and the method further includes:

[0024] Set the desired attitude data and desired depth data of the underwater robot;

[0025] The current posture data and current depth data of the underwater robot are acquired by the posture hovering feedback device.

[0026] Determine the absolute attitude data based on the desired attitude data, the desired depth data, the current attitude data, and the current depth data;

[0027] Based on the absolute attitude data, obtain the attitude control command generated by the motion controller;

[0028] The vector thruster controls the current attitude data and the current depth data based on the attitude control commands, so that the attitude data of the underwater robot is the target attitude data.

[0029] A second aspect of this application provides an optical sensor, including:

[0030] The pose hovering feedback device is used to collect image data and attitude data of the underwater robot, so as to determine the relative displacement data and absolute attitude data based on the target attitude data and target position data.

[0031] A motion controller is used to generate attitude control commands based on the relative displacement data and the absolute attitude data;

[0032] Vector thruster, used to adjust the position data and attitude data of the underwater robot according to the attitude control command, so that the underwater robot meets the target attitude data and the target position data;

[0033] The underwater robot is used to perform the pose control method as described in the first aspect.

[0034] Optionally, the pose hovering feedback device includes:

[0035] The image acquisition module is used to acquire images of the underwater robot.

[0036] The height measurement module is used to acquire the current water pressure data of the underwater robot, so as to obtain the current height data of the underwater robot; wherein, the current height data is used to describe the distance data between the underwater robot and the water surface;

[0037] A distance measurement module is used to acquire the current depth data of the underwater robot; wherein, the current depth data is used to describe the distance between the underwater robot and the seabed;

[0038] An attitude measurement module is used to acquire the current attitude data of the underwater robot; wherein the current attitude data is used to describe the attitude angle data of the underwater robot.

[0039] A third aspect of this application provides a pose control system for an underwater robot, comprising:

[0040] The extraction unit is used to extract the first position data of the underwater robot from the first acquired image after receiving the first acquired image and the second acquired image sent by the image acquisition module.

[0041] The determining unit is used to determine the displacement change data of the underwater robot based on the first acquired image and the second acquired image;

[0042] The determining unit is further configured to determine the relative displacement data of the underwater robot based on the first position data and the displacement change data;

[0043] The control unit is used to control the underwater robot to move to the target position data based on the relative position data.

[0044] The third aspect of this application provides a method for performing the pose control method described in the first aspect.

[0045] A fourth aspect of this application provides a pose control device, including:

[0046] Central processing unit, memory, input / output interfaces, wired or wireless network interfaces, and power supply;

[0047] The memory is either a short-term storage memory or a persistent storage memory;

[0048] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the pose control method described in the first aspect.

[0049] A fifth aspect of this application provides a computer-readable storage medium, characterized in that the computer-readable storage medium includes instructions that, when executed on a computer, cause the computer to perform the pose control method described in the first aspect.

[0050] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: Through the pose control method disclosed in this application, after receiving the first and second acquired images sent by the image acquisition module, the first position data of the underwater robot is extracted from the first acquired image; then, the displacement change data of the underwater robot is determined based on the first and second acquired images; next, the relative displacement data of the underwater robot is determined based on the first position data and the displacement change data; finally, based on the relative position data, the underwater robot is controlled to move to the target position data. Thus, the optical tracking method replaces the DVL multi-sound transceiver method, thereby solving the problem that DVL cannot be used in enclosed environments with multiple sound wave reflections. Simultaneously, by combining optical motion estimation of the underwater robot's displacement data, motion rules are determined, thereby adjusting the position of the underwater robot and achieving hovering control in space. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0052] Figure 1 This is a schematic diagram of the structure of an optical sensor disclosed in an embodiment of this application;

[0053] Figure 2 This is a flowchart illustrating a pose control method disclosed in an embodiment of this application;

[0054] Figure 3 This is a flowchart illustrating another pose control method disclosed in an embodiment of this application;

[0055] Figure 4 This is a flowchart illustrating another pose control method disclosed in an embodiment of this application;

[0056] Figure 5 This is a block diagram of a pose control logic disclosed in an embodiment of this application;

[0057] Figure 6 This is a schematic diagram of the structure of a pose control system disclosed in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of the structure of a posture control device disclosed in an embodiment of this application. Detailed Implementation

[0059] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0060] It should be noted that the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0061] ROVs require attitude hovering control, but current attitude feedback sensors primarily use depth gauges and inertial measurement units (IMUs) integrated with dynamic range sensors (DVLs) to obtain a complete spatial attitude description. However, since DVLs use multi-acoustic transceiver arrays, multiple sound echoes in enclosed environments can easily affect their normal operation, making them susceptible to environmental interference. Specifically, DVLs are limited to use in open water environments such as lakes and oceans. In enclosed spaces like swimming pools and tanks, sound waves are affected by multiple reflections within the confined space, rendering them unusable. Furthermore, DVLs are expensive and costly.

[0062] Therefore, the technical solution of this application uses a camera as the main sensor to estimate the motion relationship of the ROV relative to the bottom of the water using optical methods, and integrates a single-point depth-penetrating sonar and an inertial measurement unit to make up for the shortcomings of optical methods in the underwater environment. It can also output displacement signals such as DVL. However, the attitude hovering device of this invention can overcome the limitations of acoustic sensors in closed scenarios, and at the same time has a competitive cost advantage.

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of an optical sensor disclosed in an embodiment of this application.

[0065] Depend on Figure 1As can be seen, the sensor in this application's technical solution can be specifically understood as an optical sensor. It is equipped with one or more of a pose hovering feedback device, a motion controller, or a vector thruster. The pose hovering feedback device is used to acquire image data and attitude data of the underwater robot, and to determine relative displacement data and absolute attitude data based on the target attitude data and target position data. The motion controller is used to generate attitude control commands based on the relative displacement data and absolute attitude data. The vector thruster is used to adjust the underwater robot's position and attitude data according to the attitude control commands, so that the underwater robot meets the target attitude and target position data.

[0066] In one specific embodiment, the pose hovering feedback device further includes: an image acquisition module for acquiring image data of the underwater robot; a height measurement module for acquiring current water pressure data of the underwater robot to obtain current height data of the underwater robot; wherein the current height data is used to describe the distance data between the underwater robot and the water surface; a distance measurement module for acquiring current depth data of the underwater robot; wherein the current depth data is used to describe the distance data between the underwater robot and the bottom of the water; and an attitude measurement module for acquiring current attitude data of the underwater robot; wherein the current attitude data is used to describe the attitude angle data of the underwater robot. It is easy to understand that the image acquisition module is... Figure 1 The camera in the middle. The height measurement module is... Figure 1 The pressure sensor in the middle. The distance measurement module is... Figure 1 The single-beam ranging sonar in the system. The attitude measurement module is... Figure 1 The inertial measurement unit (IMU) is used in this system. Correspondingly, a pressure sensor detects water pressure to determine altitude, a camera acquires image information and obtains partial attitude through optical tracking, a single-beam ranging sonar corrects for distance scale, and the IMU obtains attitude angle values. For ease of understanding, this will not be elaborated upon further. In another implementation, the single-beam sonar can be replaced with other distance sensors, or a binocular vision sensor can be used instead of a distance sensor. The specific sensors used are not limited here.

[0067] Furthermore, by Figure 1 As can be seen, the optical sensor is communicatively connected to the ROV. The optical sensor can be part of the ROV or a separate device; details will not be elaborated here. It should also be noted that there are two states when the ROV is placed in water: fully submerged and partially submerged. The physical calculation formulas for water pressure or buoyancy in these different states, and the corresponding methods for obtaining height or depth, will not be detailed here.

[0068] For a detailed description of the application of the optical sensors described above, please refer to [link / reference]. Figure 2 , Figure 2 This is a flowchart illustrating a pose control method disclosed in an embodiment of this application. It includes steps 201-204.

[0069] 201. After receiving the first and second acquired images sent by the image acquisition module, extract the first position data of the underwater robot from the first acquired image.

[0070] After the optical sensor receives the first and second acquired images from the image acquisition module, it can extract the first position data of the underwater robot from the first acquired image.

[0071] In one specific embodiment, regarding optical motion estimation, the technical solution of this application can acquire images through an image acquisition module (camera) that captures images of the ROV facing the seabed. The image acquisition module can capture an optical image of the ROV at any given time point, and then capture another optical image at a different time point. The optical image at any given time point is the first acquired image described above, and the other optical image is the second acquired image. It is easy to understand that the first and second acquired images are temporally continuous images.

[0072] Therefore, based on the first acquired image, the image in the first acquired image can be converted into coordinate positions in a coordinate system, thereby determining the first position data of the underwater robot. Specifically, the first acquired image consists of multiple pixels. By analyzing the object features of the underwater robot, the image plane coordinates can be determined as the coordinates of the underwater robot's object features on the first acquired image.

[0073] 202. Determine the displacement change data of the underwater robot based on the first and second acquired images.

[0074] Once the first and second images are acquired, the displacement change data of the underwater robot can be determined based on the first and second images.

[0075] In one specific embodiment, since the first and second acquired images are two consecutively captured images, the displacement change data of the underwater robot can be determined based on its position in the first and second acquired images. In other words, the second acquired image can be obtained after the displacement change in the first acquired image. For ease of understanding, this will not be elaborated further below.

[0076] 203. Determine the relative displacement data of the underwater robot based on the first position data and displacement change data.

[0077] After determining the displacement change data, it is necessary to determine the relative displacement data corresponding to each pixel. Specifically, the relative displacement data of the underwater robot can be calculated by differentiating the first position data and the displacement change data.

[0078] In one specific embodiment, the horizontal and vertical coordinates and displacement change data in the coordinate system of the first position data are transformed, and then the derivative of the transformed function expression is calculated to obtain the relative displacement data. It is easy to understand that since the first position data is related to the acquired image, and the acquired image is composed of multiple pixels, in one specific embodiment, the relative displacement data can be understood as the relative displacement data of the corresponding pixels. In other words, it can also be understood as an average displacement change based on the image coordinate system.

[0079] 204. Based on the relative position data, control the underwater robot to move to the target position data.

[0080] After determining the relative position data, since the current position of the underwater robot can be understood as the position corresponding to the second acquired image, the underwater robot can be controlled to move from the position corresponding to the second acquired image to the target position data.

[0081] It should be noted that the above is only one specific implementation method. In another specific embodiment, the camera can collect images in real time to determine the current position data of the underwater robot. Thus, the underwater robot can be controlled to move from the current position data to the position corresponding to the target position data.

[0082] The pose control method disclosed in this embodiment involves extracting first position data of an underwater robot from the first image after receiving a first and second acquired image from the image acquisition module; determining displacement change data of the underwater robot based on the first and second acquired images; determining relative displacement data of the underwater robot based on the first position data and displacement change data; and finally controlling the underwater robot to move to the target position data based on the relative position data. This replaces the DVL (Digital Vectoring) multi-sound transceiver method with an optical tracking method, thus solving the problem that DVL cannot be used in enclosed environments with multiple sound wave reflections. Simultaneously, by combining optical motion estimation of the underwater robot's displacement data, motion rules are determined, thereby adjusting the underwater robot's position and achieving hovering control in space.

[0083] To facilitate Figure 2 For a detailed description of the pose control method, please refer to [link / reference]. Figure 3 , Figure 3 This is a flowchart illustrating another pose control method disclosed in an embodiment of this application. It includes steps 301-305.

[0084] 301. Set the first image feature of the first acquired image as the image coordinate system, and determine the first image coordinate data corresponding to the first acquired image of the underwater robot according to the image coordinate system.

[0085] Since the camera can only capture images of the underwater robot hovering in the water, it cannot directly determine the robot's relative position within the water. Therefore, it is necessary to set the image plane coordinates of the underwater robot in the captured images as image features. It's easy to understand that the captured images are obtained by the image acquisition device (optical camera) on the underwater robot. It should also be noted that the image plane coordinates are the coordinates of relevant feature points of the object on the captured image, which can also be understood as the pixel points corresponding to the object's feature points on the captured image; details will not be elaborated here. In another description, the object's feature points can also be understood as the geometric center of the object. Specifically, by setting the first image feature of the first captured image as the image coordinate system, the first image coordinate data of the underwater robot can be determined based on the image coordinate system.

[0086] In one specific embodiment, based on the Farneback dense optical flow method, the first image coordinate data corresponding to the first acquired image can be determined as x and y. Here, x represents the horizontal coordinate of the underwater robot, and y represents the vertical coordinate of the underwater robot.

[0087] 302. Convert the first image coordinate data into the first position data.

[0088] After determining the initial image coordinate data, it is necessary to convert this coordinate data into positional data for subsequent calculations. Specifically, the elements within the image are first represented using quadratic polynomials.

[0089] In one specific embodiment, specifically,

[0090] f(x,y)~p(x,y)=r1+r2x+r3y+r4x 2 +r5y 2 +r6xy (1),

[0091] Here, x and y are the coordinates in the image coordinate system, and r1 to r6 are the coefficients of the quadratic polynomial expansion. f() represents the acquired image as described above, and p() represents the image representation method after transformation by the quadratic polynomial expansion.

[0092] Then, formula (1) can be rewritten as f(x) = p(x) = x T Ax+b T x+c (2),

[0093] in,

[0094] It's easy to understand that T is the matrix transpose operator, and A, b, and c are the simplified coefficients of the quadratic polynomial expansions from r1 to r6. None of these have specific numerical values; they are simplifications and symbols used in the calculation process. Therefore, the data in the first position can be determined based on f(x) = p(x).

[0095] 303. Set initial displacement data and set second image coordinate data for the second acquired image, so as to determine displacement change data based on the second image coordinate data, the first position data and the initial displacement data.

[0096] In this embodiment, step 303 is the same as described above. Figure 2 Step 202 is similar and will not be elaborated here. However, it should be noted that in this embodiment, to determine the actual change value of the displacement change data, the function sign of the displacement change data can be preset, i.e., initial unique data. It is easy to understand that the initial displacement data is used to describe the displacement data generated during the process from the generation of the first acquired image to the generation of the second acquired image by the image acquisition module. Therefore, the second image coordinate data of the second acquired image can be set, thereby determining the displacement change data based on the second image coordinate data, the first position data, and the initial displacement data.

[0097] In one specific embodiment, based on the displacement change d generated by image f1, the time-continuous image f2 can be represented as follows:

[0098]

[0099] It is not difficult to understand that, among them, and This represents the corresponding change in coefficients after a displacement change in f1. f1 and f2 are two consecutive images, where f1 is replaced by the image f2. Therefore, the corresponding f2(x) = p(xd) can be understood as the coordinate data of the second image.

[0100] Therefore, the displacement change d can be achieved by... Obtain, specifically

[0101]

[0102] Therefore, the specific displacement change data can be determined based on the functional expression of d.

[0103] 304. Based on the first image coordinate data and the second image coordinate data, set an intermediate coefficient function, and substitute the first position data and displacement change data into the intermediate coefficient function to obtain the relative displacement data.

[0104] To determine the relative displacement data corresponding to each pixel, an intermediate coefficient function can be set based on the first image coordinate data and the second image coordinate data. Then, by substituting the first position data and displacement change data into the intermediate coefficient function, the relative displacement data can be obtained.

[0105] In one specific embodiment, the corresponding relative displacement data can be determined by differentiating the above formula (5). First, an intermediate coefficient function is set; specifically, the intermediate processes A(x,y) and Δb(x,y) are rewritten as formula (5).

[0106] A(x,y)d(x,y)=Δb(x,y) (6),

[0107] As shown in formula (5), A(x,y) and Δb(x,y) are respectively,

[0108]

[0109]

[0110] It is not difficult to understand that A1 and A2 are A(x,y) corresponding to f1 and f2, respectively.

[0111] Based on the assumption that the flow field change is smooth, d can be solved by addressing the intermediate variable A. T A and A T Δb is obtained, specifically.

[0112] d(x,y)=(∑wA T A) -1 (∑wA T Δb) (9),

[0113] Where w is the Gaussian smoothing weight, which is Gaussian noise used in the algorithm calculation and is unrelated to the changes in x or y. Finally, the average displacement is calculated based on dense optical flow, resulting in... Therefore, the relative displacement data can be determined.

[0114] 305. Extract the second position data of the underwater robot from the second acquired image based on the displacement change data, so as to control the underwater robot to move to the target position data based on the second position data and the relative position data.

[0115] In this embodiment, step 305 is the same as described above. Figure 2 Step 204 is similar and will not be described in detail here. However, it should be noted that in this embodiment, the second position data corresponding to the second acquired image is used as the current position data of the underwater robot. Therefore, the position of the underwater robot corresponding to the winter solstice target position data can be controlled based on the second position data and the relative position data.

[0116] The pose control method disclosed in this embodiment uses the optical principle of a camera to achieve optical tracking, thereby replacing the sensor frame based on acoustic principles.

[0117] Furthermore, in another feasible technical solution, optical sensors can also control the attitude data of underwater robots. For details, please refer to [link / reference needed]. Figure 4 , Figure 4 This is a flowchart illustrating another pose control method disclosed in an embodiment of this application. It includes steps 401-404.

[0118] 401. Set the desired attitude data and desired depth data of the underwater robot, and acquire the current attitude data and current depth data of the underwater robot collected by the pose hovering feedback device.

[0119] Due to the above Figure 3 The basic operation involves adjusting the underwater robot's horizontal and vertical height in the water. Therefore, other methods are needed to control its attitude angle and depth. Specifically, the desired attitude and depth data of the underwater robot can be set in advance. Then, in actual application, the current attitude and depth data of the underwater robot are acquired from the pose hovering feedback device. It is easy to understand that the attitude data can be understood as the current deflection or rotation angle of the underwater robot, and the depth data can be understood as the current water depth of the underwater robot.

[0120] In one specific embodiment, since the underwater robot's degrees of freedom include depth and attitude angle, these can be obtained by a pressure sensor and an inertial measurement unit, respectively. Thus, the underwater robot's current attitude data and current depth data can be acquired.

[0121] In another specific embodiment, based on Figure 3 The illustrated embodiment, due to Since the displacement changes are based on the image coordinate system, they do not have the scale significance of actual distance. Therefore, it is necessary to rely on single-beam sonar to probe the depth and obtain the actual distance scale between the underwater robot and the bottom of the water. Based on this actual distance scale, the displacement of the underwater robot in space in the image coordinate system can be deduced, thereby obtaining the distance r between the ROV and the bottom of the water.

[0122] It should also be noted that, furthermore, r, And the camera intrinsic parameter K as the regression function E p The input is used to complete the distance scale correction, at which point the relative displacements dX and dY of the two images can be obtained.

[0123] 402. Determine the absolute attitude data based on the desired attitude data, desired depth data, current attitude data, and current depth data.

[0124] Based on step 401, the expected pose data, expected depth data, current pose data, and current depth data can be used to determine the absolute pose data.

[0125] In one specific embodiment, the absolute attitude variables [Z, Roll, Pitch, Yaw] are obtained by a pressure sensor and an inertial measurement unit. Here, Z, Roll, Pitch, and Yaw represent the robot's attitude in space, and the attitude description can express the complete degrees of freedom using X, Y, Z, Roll, Pitch, and Yaw. Correspondingly, X, Y, and Z represent the XYZ axes in the image coordinate system. Roll, Pitch, and Yaw are the Euler angles corresponding to X, Y, and Z. Yaw represents the yaw angle about the Z-axis, Pitch represents the pitch angle about the X-axis, and Roll represents the roll angle about the Y-axis. Thus, the absolute attitude data of the underwater robot can be determined.

[0126] 403. Based on the absolute attitude data, obtain the attitude control commands generated by the motion controller.

[0127] Once the absolute attitude data is determined, the attitude control commands generated by the motion controller can be obtained.

[0128] In one specific embodiment, based on the desired attitude data and desired depth data, and after comparing them with the absolute attitude data, the motion controller can generate control signals to the underwater robot, namely the attitude control commands described above.

[0129] 404. Using vector thrusters, the current attitude data and current depth data are controlled based on attitude control commands to make the underwater robot's attitude data the target attitude data.

[0130] Therefore, by using vector thrusters and under the control of attitude control commands, the current sub-attitude data and current depth data of the underwater robot can be controlled, so that the attitude data of the underwater robot is the target attitude data.

[0131] In one specific embodiment, the current attitude angle and depth of the underwater robot are adjusted by attitude control commands, thereby controlling the underwater robot to move and rotate, thereby adjusting the attitude of the underwater robot, and finally reaching the previously set attitude or depth.

[0132] Based on the above embodiments, the target pose data can be set as follows: Here, ref represents the attitude to be maintained and the reference attitude input of the control system.

[0133] The pose control method disclosed in this embodiment uses a camera as the main sensor to estimate the motion relationship of the ROV relative to the seabed using optical methods. It integrates a single-point depth-penetrating sonar and an inertial measurement unit to compensate for the shortcomings of optical methods in underwater environments. It can output displacement signals similar to DVL (Depth-Voltage Lift) signals, but the attitude hovering device of this invention overcomes the limitations of acoustic sensors in enclosed environments and has a competitive cost advantage. Furthermore, using single-beam sonar as a distance-scale-free correction method for monocular vision sensors effectively improves algorithm reliability and avoids scale initialization failures.

[0134] For the convenience of understanding the above Figures 2 to 4 To understand the described pose control method in practical applications, please refer to [link / reference needed]. Figure 5 , Figure 5 This is a block diagram of a pose control logic disclosed in an embodiment of this application.

[0135] Specifically, by Figure 5 As can be seen, an underwater robot may include an attitude hovering feedback device, a motion controller, and a vector thruster in one of its components. The attitude hovering feedback device can be understood as the optical sensors described above, including an optical camera, a single-beam sonar, an inertial measurement unit, and a pressure sensor. Specific descriptions of the attitude hovering feedback device, motion controller, and vector thruster are not provided here; please refer to the corresponding descriptions above. It should also be noted that "+", "-", and... This refers to the feedback method used to control signals entering the system, represented by a block diagram. "+" indicates positive feedback, and "-" indicates negative feedback. This refers to a node in the signal processing, indicating that signal computation will be performed; it does not inherently represent any computational meaning. Generally, a stable system uses negative feedback signals for error compensation. Figure 5 The target is the hovering attitude, with positive input (hovering attitude). The relative and absolute attitude feedback to the system is negative, which is the source of error and must be compensated by the motion controller. Therefore, the motion controller constructs an error with the input variables, determines the attitude control command, and adjusts the ROV attitude in real time.

[0136] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0137] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a pose control system disclosed in an embodiment of this application.

[0138] The extraction unit 601 is used to extract the first position data of the underwater robot from the first acquired image after receiving the first acquired image and the second acquired image sent by the image acquisition module.

[0139] The determining unit 602 is used to determine the displacement change data of the underwater robot based on the first acquired image and the second acquired image;

[0140] The determining unit 602 is also used to determine the relative displacement data of the underwater robot based on the first position data and displacement change data;

[0141] The control unit 603 is used to control the underwater robot to move to the target position data based on the relative position data.

[0142] For example, the system further includes: a setting unit 604 and a conversion unit 605;

[0143] Setting unit 604 is used to set the first image feature of the first acquired image as an image coordinate system;

[0144] The determining unit 602 is specifically used to determine the first image coordinate data corresponding to the first acquired image of the underwater robot according to the image coordinate system;

[0145] The conversion unit 605 is used to convert the first image coordinate data into first position data.

[0146] For example, the system includes:

[0147] The setting unit 604 is specifically used to set initial displacement data; wherein, the initial displacement data is used to describe the displacement data generated by the image acquisition module during the process from generating the first acquired image to the second acquired image;

[0148] The determining unit 602 is specifically used to set the second image coordinate data of the second acquired image, and to determine the displacement change data based on the second image coordinate data, the first position data and the initial displacement data.

[0149] For example, the system further includes: an acquisition unit 606;

[0150] Setting unit 604 is specifically used to set intermediate coefficient functions based on the first image coordinate data and the second image coordinate data;

[0151] The acquisition unit 606 is used to substitute the first position data and displacement change data into the intermediate coefficient function to obtain the relative displacement data.

[0152] For example, the system includes:

[0153] The extraction unit 601 is specifically used to extract the second position data of the underwater robot from the second acquired image based on the displacement change data.

[0154] The control unit 603 is specifically used to control the underwater robot to move to the target position data based on the second position data and the relative position data.

[0155] For example, the optical sensor also includes a pose hovering feedback device, a motion controller, and a vector thruster, and the system also includes:

[0156] The setting unit 604 is also used to set the desired attitude data and desired depth data of the underwater robot;

[0157] The acquisition unit 606 is also used to acquire the current attitude data and current depth data of the underwater robot collected by the pose hovering feedback device;

[0158] The determining unit 602 is also used to determine absolute attitude data based on the desired attitude data, desired depth data, current attitude data, and current depth data;

[0159] The acquisition unit 606 is also used to acquire the attitude control command generated by the motion controller based on the absolute attitude data;

[0160] The control unit 603 is also used to control the current attitude data and the current depth data respectively through the vector thruster based on attitude control commands, so that the attitude data of the underwater robot is the target attitude data.

[0161] Please refer to the following: Figure 7 The schematic diagram of a pose control device disclosed in this application includes:

[0162] Central processing unit 701, memory 705, input / output interface 704, wired or wireless network interface 703, and power supply 702;

[0163] Memory 705 is either a short-term storage memory or a persistent storage memory;

[0164] The central processing unit 701 is configured to communicate with the memory 705 and execute instructions stored in the memory 705 to perform the aforementioned operations. Figures 2 to 4 The pose control method in any of the embodiments shown.

[0165] This application also provides a chip system, characterized in that the chip system includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected via a circuit, and the at least one processor is used to run computer programs or instructions to perform the aforementioned... Figures 2 to 4 The pose control method in any of the embodiments shown.

[0166] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0167] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0168] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0170] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A pose control method, characterized in that, Applied to optical sensors, the method includes: Upon receiving the first and second acquired images from the image acquisition module, the first position data of the underwater robot is extracted from the first acquired image. The displacement change data of the underwater robot are determined based on the first acquired image and the second acquired image; The relative displacement data of the underwater robot is determined based on the first position data and the displacement change data; Based on the relative displacement data, control the underwater robot to move to the target position data; The step of extracting the first position data of the underwater robot from the first acquired image includes: The first image feature of the first acquired image is set as the image coordinate system; The first image coordinate data corresponding to the first acquired image of the underwater robot is determined according to the image coordinate system; Convert the first image coordinate data into the first position data; The optical sensor also includes a pose hovering feedback device, a motion controller, and a vector thruster; the method further includes: Set the desired attitude data and desired depth data of the underwater robot; The current posture data and current depth data of the underwater robot are acquired by the posture hovering feedback device. Determine the absolute attitude data based on the desired attitude data, the desired depth data, the current attitude data, and the current depth data; Based on the absolute attitude data, obtain the attitude control command generated by the motion controller; The vector thruster controls the current attitude data and the current depth data based on the attitude control commands, so that the attitude data of the underwater robot is the target attitude data.

2. The pose control method according to claim 1, characterized in that, The step of determining the displacement change data of the underwater robot based on the first acquired image and the second acquired image includes: Set initial displacement data; wherein, the initial displacement data is used to describe the displacement data generated by the image acquisition module during the process of generating the first acquired image to the second acquired image; Set the second image coordinate data of the second acquired image, and determine the displacement change data based on the second image coordinate data, the first position data, and the initial displacement data.

3. The pose control method according to claim 2, characterized in that, Determining the relative displacement data of the underwater robot based on the first position data and the displacement change data includes: Based on the first image coordinate data and the second image coordinate data, an intermediate coefficient function is set; Substitute the first position data and the displacement change data into the intermediate coefficient function to obtain the relative displacement data.

4. The pose control method according to claim 1, characterized in that, The data for controlling the underwater robot to move to the target location includes: The second position data of the underwater robot is extracted from the second acquired image based on the displacement change data; Based on the second position data and the relative displacement data, the underwater robot is controlled to move to the target position data.

5. An optical sensor, characterized in that, include: The pose hovering feedback device is used to collect image data and attitude data of the underwater robot, so as to determine the relative displacement data and absolute attitude data based on the target attitude data and target position data. A motion controller is used to generate attitude control commands based on the relative displacement data and the absolute attitude data; Vector thruster, used to adjust the position data and attitude data of the underwater robot according to the attitude control command, so that the underwater robot meets the target attitude data and the target position data; The optical sensor is used to perform the pose control method as described in any one of claims 1 to 4.

6. The optical sensor according to claim 5, characterized in that, The pose hovering feedback device includes: The image acquisition module is used to acquire image data of the underwater robot; The height measurement module is used to acquire the current water pressure data of the underwater robot, so as to obtain the current height data of the underwater robot; wherein, the current height data is used to describe the distance data between the underwater robot and the water surface; A distance measurement module is used to acquire the current depth data of the underwater robot; wherein, the current depth data is used to describe the distance between the underwater robot and the seabed; An attitude measurement module is used to acquire the current attitude data of the underwater robot; wherein the current attitude data is used to describe the attitude angle data of the underwater robot.

7. A pose control system for executing the pose control method according to any one of claims 1 to 4, characterized in that, The system, applied to underwater robots, includes: The extraction unit is used to extract the first position data of the underwater robot from the first acquired image after receiving the first acquired image and the second acquired image sent by the image acquisition module. The determining unit is used to determine the displacement change data of the underwater robot based on the first acquired image and the second acquired image; The determining unit is further configured to determine the relative displacement data of the underwater robot based on the first position data and the displacement change data; The control unit is used to control the underwater robot to move to the target position data based on the relative displacement data.

8. A posture control device, characterized in that, The device includes: Central processing unit, memory, input / output interfaces, wired or wireless network interfaces, and power supply; The memory is either a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the pose control method according to any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on a computer, cause the computer to perform the pose control method as described in any one of claims 1 to 4.

Images