Fixed-time adaptive depth estimation method, device and storage medium for monocular camera

Abstract

The embodiment of the present application discloses a fixed-time adaptive depth estimation method for a monocular camera, a computer device, and a computer-readable storage medium. The method includes the following steps: obtaining the first coordinate system information obtained by collecting feature points with a monocular camera, the first coordinate system information being the coordinates of the feature points in the monocular camera coordinate system; constructing a hand-eye robot system model based on the first coordinate system information, the robot's forward kinematics, and the internal and external parameter information of the monocular camera, the hand-eye robot system model being used to represent the correlation between the coordinate information and depth under the base coordinates of the feature points; obtaining the regression matrix and known vectors obtained by linearizing the hand-eye robot system model, and designing auxiliary variables based on the regression matrix and the known vectors; constructing an adaptive law that converges in a fixed time based on the auxiliary vectors; and obtaining depth information within a fixed time using the adaptive law. Therefore, the present application can quickly determine the depth information of the feature position for controlling the hand-eye robot.

Description

Technical Field

[0001] The present application belongs to the field of robot control technology, and in particular relates to a fixed-time adaptive depth estimation method for a monocular camera, a computer device, and a computer-readable storage medium. Background Art

[0002] Visual servoing is an important control method for robotic systems. It uses visual information as feedback to perform non-contact measurements of the environment, which can greatly improve the flexibility and accuracy of robotic systems. It is widely used in the control of wheeled robots, robotic arms, and drones. The relationship between Euclidean space and image space is a significant issue affecting visual servo control. One factor affecting this relationship is that image space is a two-dimensional projection of three-dimensional Euclidean space. To compensate for the lack of depth information in two-dimensional image data, existing technologies focus on visual servoing strategies based on monocular cameras. For feature points detected by a monocular camera, without prior geometric knowledge of the observed object, it is difficult to measure the time-varying depth that exhibits nonlinearity in the image Jacobian matrix. Existing technologies use methods such as adaptive estimation strategies and composite learning laws to solve the problem of time-varying depth estimation of feature points in visual servoing.

[0003] However, traditional adaptive estimation methods driven by control errors are less than ideal in terms of parameter convergence to the true value and convergence speed. While the composite learning law method can accurately converge the parameters to the true value, its convergence speed is less than satisfactory. Improving the speed of acquiring depth information is a technical problem that those skilled in the art urgently need to solve.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] Based on this, it is necessary to address the above problems and propose a monocular camera fixed-time adaptive depth estimation method, computer equipment and computer-readable storage medium, which can solve the problem that the robot cannot quickly obtain depth information.

[0006] The present application solves the technical problem by adopting the following technical solutions:

[0007] The present application provides a fixed-time adaptive depth estimation method for a monocular camera, comprising the following steps: obtaining first coordinate system information obtained by collecting feature points with a monocular camera, the first coordinate system information being the coordinates of the feature points in the monocular camera coordinate system; constructing a hand-eye robot system model based on the first coordinate system information, the robot's forward kinematics, and the internal and external parameter information of the monocular camera, the hand-eye robot system model being used to represent the correlation between the coordinate information under the base coordinates of the feature points and the depth; obtaining a regression matrix and known vectors obtained by linearizing the hand-eye robot system model, and designing auxiliary variables based on the regression matrix and the known vectors; constructing an adaptive law that converges in a fixed time based on the auxiliary vectors; and obtaining depth information within a fixed time using the adaptive law.

[0008] In an optional embodiment of the present application, obtaining first coordinate system information obtained by capturing feature points with a monocular camera includes: obtaining second coordinate system information of the feature points, where the second coordinate system information is the two-dimensional pixel coordinates of the feature points in the image plane of the monocular camera coordinate system; obtaining internal and external parameter information, and determining the first coordinate system information based on the internal and external parameter information and the second coordinate system information.

[0009] In an optional embodiment of the present application, a hand-eye robot system model is constructed based on the first coordinate system information, the robot forward kinematics, and the internal and external parameter information of the monocular camera, including: constructing the hand-eye robot system model based on the first coordinate system information, the robot forward kinematics, and the internal and external parameter information of the monocular camera. The hand-eye robot system model can be expressed as:

[0010]

[0011] Among them, z i is the depth of the feature point; Represents the third row of the monocular camera intrinsic parameter matrix; c R b = c R e e R b Represents the rotation matrix between the monocular camera coordinate system and the base coordinate system; c t b = c t e e R b + e t b Represents the displacement vector between the monocular camera coordinate system and the base coordinate system, c R e and c t e represents the external parameters of the monocular camera, e R b and e t bThey represent the rotation and displacement of the base coordinate system relative to the end of the manipulator, respectively, and are obtained by forward kinematics; b x i Indicates the coordinates of the feature point in the base coordinate system, which is determined by the first coordinate system information.

[0012] In an optional embodiment of the present application, obtaining the auxiliary vector after linearizing the hand-eye robot system model includes linearly parameterizing the hand-eye robot system model, and the result is:

[0013]

[0014] Among them, Φ i (p i ) is independent of The regression matrix is ​​expressed as y i (p i ) is a known vector, and the known vector is expressed as And there is

[0015] In an optional embodiment of the present application, designing auxiliary variables based on the regression matrix and the known vector includes: obtaining adjustment parameters r, l, where: is a preset positive constant, is a preset constant; construct auxiliary variables P, Q, G according to the auxiliary vector, where, Specifically, it can be expressed as:

[0016]

[0017] Define the initial condition G0 = G -1 (0) = ηI, η>0.

[0018] In an optional embodiment of the present application, constructing an adaptive law with fixed-time convergence based on the auxiliary vector includes: solving the auxiliary variable to obtain:

[0019]

[0020] Obtain the residual error and define the auxiliary variable R according to the correlation between the residual error and the first-order derivative of the auxiliary variable, which can be expressed as: Where E is the residual error, and I is the identity matrix; To estimate the error; obtain auxiliary parameters, including learning gain and adjustment parameters; construct an adaptive law based on the auxiliary variable R and auxiliary parameters, which can be expressed as:

[0021]

[0022] in, represents learning gain; a, b, and c are adjustment parameters.

[0023] In an optional embodiment of the present application, an adaptive law is used to obtain depth information within a fixed time, including: continuously acquiring image data captured by a monocular camera, processing the image data into first coordinate system information, and then entering the adaptive law for solution; marking the result obtained by continuously solving the adaptive law as estimated depth information, and marking the estimated depth information that converges after a fixed time as depth information.

[0024] In an optional embodiment of the present application, after obtaining the depth information within a fixed time using the adaptive law, the method also includes: constructing a controller model; determining the displacement error and rotation error based on the homography matrix, wherein the displacement error is the difference between the current pixel coordinates of the feature point and the expected pixel coordinates, and the rotation error is represented by the rotation axis and the rotation angle; substituting the displacement error, rotation error and depth information into the controller model to obtain control parameters to control the robot to complete the designed control task.

[0025] The present application also provides a computer device, comprising a processor and a memory: the processor is configured to execute a computer program stored in the memory to implement the aforementioned method.

[0026] The present application also provides a computer-readable storage medium storing a computer program, which implements the aforementioned method when the computer program is executed by a processor.

[0027] The embodiments of the present application have the following beneficial effects:

[0028] This application can be driven by estimation error and design a fixed-time adaptive law for estimating feature positions and depth parameters to improve the slow convergence of compound learning rate parameter estimation. It solves the problem of difficulty in measuring the nonlinear time-varying depth that appears in the image Jacobian matrix without prior geometric knowledge of the observed object in monocular camera visual servoing, and the problem of unsatisfactory control effect of traditional adaptive methods applied to robots, achieving rapid convergence of pixel tracking error and estimation error in hand-eye robot systems.

[0029] The above description is only an overview of the technical solution of this application. In order to more clearly understand the technical means of this application, which can be implemented in accordance with the contents of the description, and to make the above and other purposes, features and advantages of this application more obvious and easy to understand, the following preferred embodiments are specifically described in detail with reference to the accompanying drawings. It should be understood that the above general description and the detailed description below are only exemplary and explanatory and do not limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0031] in:

[0032] Figure 1 A flowchart of a fixed-time adaptive depth estimation method for a monocular camera provided by an embodiment is provided;

[0033] Figure 2 A schematic diagram of the convergence effect of depth estimation provided by an embodiment;

[0034] Figure 3 A schematic diagram of the control effect convergence effect provided by an embodiment;

[0035] Figure 4 A schematic diagram of the convergence effect of the estimation error provided by an embodiment;

[0036] Figure 5 A schematic diagram of the convergence effect of pixel error provided by an embodiment;

[0037] Figure 6 A schematic diagram of the convergence effect of rotation error provided by an embodiment;

[0038] Figure 7 The present invention is a schematic block diagram of the structure of a computer device provided by an embodiment. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0040] Visual servoing, as an important control method for robotic systems, is widely used in robotic control. The relationship between Euclidean space and image space is a significant issue affecting visual servo control. One factor influencing this relationship is that image space is a two-dimensional projection of three-dimensional Euclidean space. To compensate for the lack of depth information in two-dimensional image data, existing technologies have used alternative sensors such as laser and acoustic ranging technology, combined monocular camera-based vision systems with other sensors, or used binocular cameras to triangulate the corresponding images to obtain depth information.

[0041] However, the above existing technologies all involve additional hardware devices. The practical disadvantages caused by integrating additional sensors include, but are not limited to, increased cost, increased complexity, reduced reliability, and increased processing burden. Due to these practical limitations, some existing technologies focus on visual servoing strategies based on monocular cameras, which rely on analytical techniques to address the lack of depth information. However, for feature points detected by a monocular camera, without prior geometric knowledge of the observed object, it is difficult to measure the time-varying depth that exhibits nonlinearity in the image Jacobian matrix. Therefore, some existing technologies use methods such as adaptive estimation strategies and composite learning laws to solve the problem of time-varying depth estimation of feature points in visual servoing.

[0042] However, the traditional adaptive estimation method driven by control error in the prior art is not very effective in terms of parameter convergence to the true value and convergence speed. The composite learning law method can accurately make the parameters converge to the true value, but its effect on parameter convergence speed still needs improvement. Since the slow convergence process increases the time cost and computer consumption cost, how to improve the speed of acquiring depth information is solved by this application. A fixed-time adaptive depth estimation method for a monocular camera is proposed to solve this problem. In order to clearly describe the fixed-time adaptive depth estimation method for a monocular camera provided in this embodiment, please refer to Figures 1 to 6 .

[0043] Step S110: Obtain the first coordinate system information obtained by collecting feature points with a monocular camera. The first coordinate system information is the coordinates of the feature points in the monocular camera coordinate system. Construct a hand-eye robot system model based on the first coordinate system information, the robot's forward kinematics, and the internal and external parameter information of the monocular camera. The hand-eye robot system model is used to represent the correlation between the coordinate information of the feature points in the base coordinate system and the depth.

[0044] In one embodiment, it is worth noting that the method adopted in the present application can be applied to a hand-eye robot equipped with a monocular camera in a preferred embodiment, so as to achieve the same or similar effects with simpler equipment, reduce the manufacturing cost of the equipment, and facilitate the promotion of the solution.

[0045] In one embodiment, step S110: obtain first coordinate system information obtained by capturing feature points with a monocular camera, where the first coordinate system information is the coordinates of the feature points in the monocular camera coordinate system; construct a hand-eye robot system model based on the first coordinate system information, the robot's forward kinematics, and the internal and external parameter information of the monocular camera, where the hand-eye robot system model is used to represent the correlation between the coordinate information of the feature points in the base coordinates and the depth.

[0046] In one embodiment, when estimating the depth, it is necessary to obtain the coordinate information of the feature points. Specifically, the first thing that needs to be collected may be the coordinate information of the feature points. The feature points may be preset, and may be several points on the collected object. For example, if the collected object is a square, its 8 vertices may be feature points; if the collected object is a QR code, the 4 fixed points of the QR code may be feature points. The specific feature points are pre-set, and are only used as examples here, not as limitations on the scheme. In a preferred embodiment, there are multiple feature points, of which 4 pairs of points are preferred. It can be understood that during the acquisition process, the image captured by the monocular camera is a planar image, so the initial coordinate information of the feature point is the pixel coordinate, that is, the second coordinate information p i , Second coordinate information p i It is a two-dimensional information, which only includes u horizontal pixel coordinates and v vertical pixel coordinates, i = 0, 1, 2, ..., N represents the i+1th feature point of the target.

[0047] The coordinate information actually needed should be the coordinate information of the feature points in the base coordinate system. The second coordinate information p i The first coordinate system information is converted into the Cartesian coordinates of the feature points in the base coordinate system. The conversion process requires obtaining the internal and external parameter information of the monocular camera to achieve it. Specifically, in the monocular camera coordinate system, the image plane coordinates and second coordinate information The relationship between can be referred to formula (1):

[0048] p i =K c x i (1)

[0049] Among them, K is the internal parameter of the monocular camera.

[0050] Feature points in the monocular camera coordinate system c x i and the image plane p i The projection relationship between them can be expressed as formula (2):

[0051]

[0052] in, c x i It is obtained through the following relationship:

[0053] c x i = c R b b x i + c tb (3)

[0054] in, c R b = c R e e R b Represents the rotation matrix between the monocular camera coordinate system and the base coordinate system, c t b = c t e e R b + e t b Represents the displacement vector between the monocular camera coordinate system and the base coordinate system, where c R e , c t e is the external parameter of the monocular camera in the hand-eye system, that is Based on the above processing, the second coordinate information of the feature point collected by the monocular camera can be converted into first coordinate information that can be used for processing.

[0055] In one embodiment, step S110: constructing a hand-eye robot system model based on the first coordinate system information, the robot forward kinematics, and the internal and external parameter information of the monocular camera, including: constructing a hand-eye robot system model based on the first coordinate system information, the robot forward kinematics, and the internal and external parameter information of the monocular camera.

[0056] In one embodiment, the first coordinate system information is the current position of the collected feature point. If the system knows the depth, then after controlling the movement of the robot, the current position of the feature point and the target position are the same.

[0057] By using equations (1) to (3) to solve the first coordinate system information, the hand-eye robot system model can be derived. The hand-eye robot system model can be used to obtain the feature point depth z i , specifically for the hand-eye robot system model, it can be expressed as follows:

[0058]

[0059] Among them, z i is the depth of the feature point; Represents the third row of the monocular camera intrinsic parameter matrix; c R b = c R e e R b Represents the rotation matrix between the monocular camera coordinate system and the base coordinate system; c t b =c t e e R b + e t b Represents the displacement vector between the monocular camera coordinate system and the base coordinate system, c R e and c t e represents the external parameters of the monocular camera, e R b and e t b They represent the rotation and displacement of the base coordinate system relative to the end of the manipulator, respectively, and are obtained by forward kinematics; b x i The coordinates of the feature points in the base coordinate system are determined by the first coordinate system information. What the monocular camera cannot determine is the depth information, that is, the purpose is to determine the estimated depth, and the depth can be used b x i So, as long as we estimate b x i , we can get the depth z i .

[0060] in, is the i-th row of the monocular camera intrinsic parameter matrix K, Record b x i The estimated value of depth z i Estimated value of It can be expressed as:

[0061]

[0062] In one embodiment, obtaining the auxiliary vector after linearizing the hand-eye robot system model includes linearly parameterizing the hand-eye robot system model, and the result is:

[0063]

[0064] That is to say, by linearly parameterizing equation (5), we can get equation (6). In equation (6), is independent of b x i The regression vector of Defined as a known vector, and

[0065] Step S120: Obtain the regression matrix and known vectors obtained by linearizing the hand-eye robot system model, and design auxiliary variables based on the regression matrix and known vectors.

[0066] In one embodiment, step S120: designing auxiliary variables based on the regression matrix and the known vector includes: obtaining adjustment parameters r, l, where is a preset positive constant, is a preset positive constant; construct auxiliary variables P, Q, G according to the auxiliary vector, where,

[0067] In one embodiment, to obtain depth information, it is necessary to estimate the first coordinate system information. During the estimation process, since some parameters in the robot system are unknown, an adaptive law with fixed-time convergence can be constructed to assist in estimating the target depth. Specifically, the vector obtained by linearizing the hand-eye robot system model can be used to construct auxiliary variables P, Q, and G, where: Auxiliary variables P, Q, G can be expressed as follows:

[0068]

[0069] Among them, ф(Φ) and y are the regression vector and known vector obtained in the previous article. The specific solution process can be referred to as shown in the previous article and will not be repeated here. is a positive constant used to adjust the incentive level; Used to ensure that the auxiliary variables P, Q, and G are bounded, and define the initial condition G0 = G -1 (0) = ηI, η> 0. The constant l plays the role of a forgetting factor and can be used as a small constant. The smaller l is, the more historical information ф is contained in the matrix G.

[0070] Step S130: constructing an adaptive law with fixed-time convergence according to the auxiliary vector; and obtaining depth information within a fixed time using the adaptive law.

[0071] In one embodiment, constructing a fixed-time convergence adaptive law based on the auxiliary vector includes: solving the first-order derivative of the auxiliary variable shown in equation (7) to obtain:

[0072]

[0073] Obtain the residual error and define the auxiliary variable R according to the correlation between the residual error and the first-order derivative of the auxiliary variable, which can be expressed as:

[0074]

[0075] In formula (9), E is the residual error, and I is the identity matrix; To estimate the error, the estimated error is determined by the first coordinate system information.

[0076] Obtain auxiliary parameters, including learning gain and adjustment parameters; construct an adaptive law based on the auxiliary variable R and auxiliary parameters. The adaptive law can be expressed as shown in formula (11):

[0077]

[0078] in, represents learning gain; a, b, and c are adjustment parameters, and the specific settings can be a, b>0, c>2. express The first-order derivative of is usually used to express the first-order derivative of the parameter to be estimated when designing the adaptive law. = indicates that only one feature point is used here, and the subscript i is written because four pairs of points are required to calculate the homography matrix H. As can be seen from Equation (10), all parameters except the first coordinate information are known, so the adaptive law is only related to the first coordinate information.

[0079] In one embodiment, an adaptive law that converges in a fixed time is constructed based on an auxiliary vector; depth information is obtained within a fixed time using the adaptive law, including: continuously acquiring image data captured by a monocular camera, processing the image data into first coordinate system information, and then entering the adaptive law for solution; marking the result obtained by continuously solving the adaptive law as estimated depth information, and marking the estimated depth information that converges after a fixed time as depth information.

[0080] Therefore, the first coordinate information can be correctly estimated, the first coordinate information can be converged, and finally brought into formula (5), and the depth information can be obtained. The method provides continuous excitation conditions, specifically, continuous input of image data. After the above processing, the depth information can be quickly converged. And when the test accuracy meets the engineering requirements, the estimated value is By bringing in the robot controller, accurate control of the robot can be achieved.

[0081] The method provided in this application enables rapid depth estimation, resolving the difficulty in measuring nonlinear, time-varying depth in the image Jacobian matrix in monocular visual servoing without prior geometric knowledge of the observed object. This method also addresses the suboptimal parameter identification performance and convergence speed of traditional error-driven adaptive estimation methods, improving the parameter convergence speed of the composite learning law approach.

[0082] In one embodiment, step S130: after obtaining the depth information within a fixed time using the adaptive law, the method further includes: constructing a controller model; determining a displacement error and a rotation error based on the homography matrix, wherein the displacement error is the difference between the current pixel coordinates of the feature point and the expected pixel coordinates, and the rotation error is represented by the rotation axis and the rotation angle; substituting the displacement error, rotation error and depth information into the controller model to obtain control parameters to control the robot to complete the designed control task.

[0083] According to the previous discussion, the relationship between the actual position and the expected position of the feature points in the camera coordinate system collected by the monocular camera is as shown in formula (11). The specific relationship determination process requires the depth ratio and homography matrix. The above relationship can be specifically expressed as:

[0084]

[0085] Among them, x i is the current position, x ri is the target position; z i , z ri >0 represents the current depth and target depth of the target feature point respectively; α>0 is the depth ratio; is the homography matrix; is the normal vector of the reference image plane, d r >0 represents along n r Direction, the distance from the target to the reference image plane, both belong to the relevant information between the feature point and the reference image plane. The reference image plane is the image plane predetermined when the machine captures the image. Therefore, the relevant information between the feature point and the reference image plane can be obtained when the feature point is captured; and Representing the rotation matrix and displacement vector between the current and desired poses, respectively. The information related to the feature points and the reference image plane, the depth ratio, and the internal and external parameters including the homography matrix, rotation matrix, and displacement vector obtained when building a hand-eye robotic system model with depth information can all be directly obtained. The acquisition process is well-known and will not be elaborated on here.

[0086] In one embodiment, the robot's dynamic information and torque information are obtained to construct a controller model. Specifically, the controller is designed based on the n-DOF robot dynamic model:

[0087]

[0088] In formula (12), Represent the displacement, velocity and acceleration of the robot respectively, which belong to dynamic information; is the moment of inertia, is the centripetal torque, is the gravity moment, the above belongs to the moment information; the output and calculation is the control torque of the robot.

[0089] Define displacement error e v and the rotation error e w . Displacement error e v It can be calculated as follows:

[0090] e v =p e (t)-p er (13)

[0091] Among them, p e (t), p er are the current pixel coordinates and expected pixel coordinates of the feature point respectively.

[0092] e ω (t)=u(t)θ(t) (14)

[0093] Where u(t) is the rotation axis and θ(t) is the rotation angle. Both can be determined after obtaining the depth information. The specific calculation process belongs to the existing technology and will not be described in detail.

[0094] In one embodiment, if a conventional PD controller is used, the controller design may be as follows:

[0095]

[0096] in, and is the control gain; is the estimated value of A(p), J(q) is the robot Jacobian matrix. e (p) is a Jacobian-like matrix, and its specific definition can be found in:

[0097]

[0098] And there is: definition is the displacement vector of the end effector in the base coordinates, K e is a reversible matrix. Therefore, depth information is obtained based on the adaptive law and substituted into the controller model to obtain control parameters, which can specifically be the torque that controls the robot's displacement. In reality, the existing technology for robot control suffers from a lack of accuracy, which in turn is due to a lack of depth information. However, the method provided in this application enables rapid determination of depth information, ultimately enabling more precise and rapid robot control.

[0099] In one embodiment, the present application uses an improved fixed-time adaptive law driven by estimation error to rapidly obtain depth information for robot control. Furthermore, the present application verifies the effectiveness of this method by building a 7-DOF robot model in MATLAB / Simulink. Experimental results demonstrate that the method provided by this application addresses the difficulty in measuring nonlinear time-varying depth in the image Jacobian matrix without prior geometric knowledge of the observed object in monocular camera visual servoing, as well as the unsatisfactory control effects of traditional adaptive methods applied to robots. This method achieves rapid convergence of pixel tracking error and estimation error in a hand-eye robot system.

[0100] Specifically, in Simulink simulation, the commonly used composite learning law method in the prior art is compared with the method described in the present invention. The simulation results are as follows: Figure 2 It shows the depth estimation values ​​of the two methods. It can be seen from the figure that the depth estimation value of the method provided by the present invention converges to the true value faster than the composite learning law method; Figure 3 The control effects of the two methods are shown. Since the estimated value of the method of the present invention converges faster, the control effect is better than that of the composite learning law method when a reference pixel trajectory is given. Figure 4-Figure 6 They represent estimation error, pixel error and rotation error respectively. It can be seen from the figure that the error convergence speed of the method of the present invention is better than that of the composite learning law method.

[0101] Therefore, this application can use estimation error as a driver to design a fixed-time convergence adaptive law for estimating feature positions and depth parameters, thereby improving the slow convergence of compound learning rate parameter estimation. This solves the problem of difficulty in measuring the nonlinear time-varying depth that appears in the image Jacobian matrix without prior geometric knowledge of the observed object in monocular camera visual servoing, and the problem of unsatisfactory control effects of traditional adaptive methods applied to robots, achieving rapid convergence of pixel tracking error and estimation error in hand-eye robotic systems.

[0102] Figure 7 FIG1 shows an internal structure diagram of a computer device in an embodiment. The computer device can be a terminal or a server. Figure 7 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor may implement a fixed-time adaptive depth estimation method for a monocular camera. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor may implement a fixed-time adaptive depth estimation method for a monocular camera. Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0103] In one embodiment, the present application further proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the aforementioned method.

[0104] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0105] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A fixed-time adaptive depth estimation method for a monocular camera, characterized in that: The steps include: Acquire first coordinate system information obtained by capturing feature points with a monocular camera, where the first coordinate system information is the coordinates of the feature points in the monocular camera coordinate system; construct a hand-eye robot system model based on the first coordinate system information, the robot's forward kinematics, and the monocular camera's internal and external parameter information, where the hand-eye robot system model is used to represent the correlation between the coordinate information of the feature points in the base coordinates and the depth; Obtaining a regression matrix and known vectors obtained by linearizing the hand-eye robot system model, and designing auxiliary variables based on the regression matrix and the known vectors; constructing an adaptive law with fixed-time convergence based on the auxiliary variables; Obtaining depth information within a fixed time using the adaptive law; The step of designing auxiliary variables according to the regression matrix and the known vectors includes: Get the adjustment parameters r, l, where is a preset positive constant, is a preset positive constant; Construct auxiliary variables P, Q, and G according to the known vector, where: Specifically, it can be expressed as: Define the initial condition G0 = G -1 (0) = ηI, η>0; The step of constructing a fixed-time convergent adaptive law based on the auxiliary variables includes: Solving for the auxiliary variables yields: Obtain the residual error, and define the auxiliary variable R according to the correlation between the residual error and the first-order derivative of the auxiliary variable, which can be expressed as: Where E is the residual error, and I is the identity matrix; is the estimation error; Obtain auxiliary parameters, including learning gain and adjustment parameters; construct the adaptive law based on the auxiliary variable R and the auxiliary parameters, expressed as: in, represents the learning gain; a, b and c are the adjustment parameters.

2. The monocular camera fixed time adaptive depth estimation method according to claim 1, characterized in that The obtaining of first coordinate system information obtained by collecting feature points with a monocular camera includes: Acquire second coordinate system information of the feature point, where the second coordinate system information is the two-dimensional pixel coordinates of the feature point in the image plane of the monocular camera coordinate system; Acquire internal and external parameter information, and determine the first coordinate system information according to the internal and external parameter information and the second coordinate system information.

3. The monocular camera fixed time adaptive depth estimation method according to claim 1, characterized in that The step of constructing a hand-eye robot system model based on the first coordinate system information, the robot's forward kinematics, and the internal and external parameter information of the monocular camera includes: A hand-eye robot system model is constructed based on the first coordinate system information, the robot forward kinematics, and the internal and external parameter information of the monocular camera. The hand-eye robot system model can be expressed as: Among them, z i is the depth of the feature point; Represents the third row of the monocular camera intrinsic parameter matrix; c R b = c R e e R b Represents the rotation matrix between the monocular camera coordinate system and the base coordinate system; c t b = c t e e R b + e t b Represents the displacement vector between the monocular camera coordinate system and the base coordinate system, c R e and c t e represents the external parameters of the monocular camera, e R b and e t b They represent the rotation and displacement of the base coordinate system relative to the end of the manipulator, respectively, and are obtained by forward kinematics; b x i The coordinates of the feature point in the base coordinate system are determined by the first coordinate system information.

4. The fixed-time adaptive depth estimation method for a monocular camera according to claim 3, wherein: The acquisition linearizes the hand-eye robot system model to obtain an auxiliary vector, including The hand-eye robot system model is linearly parameterized, and the result is: Among them, Φ i (p i ) is independent of b x i The regression matrix is ​​expressed as y i (p i ) is the known vector, and the known vector is expressed as And there is 5. The monocular camera fixed time adaptive depth estimation method according to claim 1, characterized in that The obtaining of depth information within a fixed time by using the adaptive law includes: Continuously acquiring image data captured by a monocular camera, processing the image data into first coordinate system information, and then applying the information to the adaptive law for solving; A result obtained by continuously solving the adaptive law is marked as estimated depth information, and the estimated depth information that converges after a fixed time is marked as the depth information.

6. The monocular camera fixed time adaptive depth estimation method according to claim 1, characterized in that After obtaining the depth information within a fixed time by using the adaptive law, the method further includes: Build the controller model; Determine a displacement error and a rotation error according to the homography matrix, wherein the displacement error is the difference between the current pixel coordinates of the feature point and the expected pixel coordinates, and the rotation error is represented by a rotation axis and a rotation angle; The displacement error, rotation error and depth information are substituted into the controller model to obtain control parameters, so as to control the robot to complete the designed control task.

7. A computer device, characterized in that: including processor and memory; The processor is configured to execute the computer program stored in the memory to implement the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images