Image vision-based mobile robot unknown static target surrounding control method

Abstract

The application provides a kind of unknown static target surrounding control method of mobile robot based on image vision, comprising: collecting the image containing unknown static target, extracting the pixel coordinates of target feature point in image plane;Based on monocular camera internal parameter, pixel coordinates are converted into normalized image plane coordinates, with the expected steady-state coordinates of corresponding expected surrounding radius, image error signal is constructed;The forward linear velocity control variable and the steering angular velocity control variable of mobile robot are calculated;Control variable is output through a closed-loop control structure, to synchronously achieve two control targets: drive image error signal to converge to expected steady-state coordinates, and ensure that target feature point is always within the effective field of view of monocular camera during the whole process of surrounding control;The calculated forward linear velocity control variable and steering angular velocity control variable are output to the motion execution mechanism of mobile robot, to drive mobile robot to perform fixed-distance surrounding control on unknown static target with expected surrounding radius.

Description

Technical Field

[0001] This invention belongs to the field of mobile robot control technology, specifically relating to a mobile robot's control method for orbiting unknown static targets based on image vision. Background Technology

[0002] Target orbit control refers to controlling a mobile robot to perform continuous orbiting motion around a target at a preset distance. It is a core foundational technology for mobile robots to perform tasks such as scene monitoring, emergency rescue, target tracking, and track maintenance, and has broad application value and research significance in the fields of civilian industry, public safety, and special operations.

[0003] Currently, the mainstream solutions for target orbit control of mobile robots mostly rely on Global Navigation Satellite Systems (GNSS) to obtain the global position information of the robot and the target, and construct control laws based on the global relative positions to complete orbit control. This type of solution depends on stable satellite positioning signals, and cannot achieve stable and effective orbit control in scenarios where GNSS signals are unavailable, such as indoors, underground, under building obstruction, or with electromagnetic shielding, thus greatly limiting its applicable scenarios.

[0004] For scenarios without global positioning due to GNSS signal rejection, several alternative solutions have been developed in the existing technology: One type of solution uses ranging sensors such as lidar and ultra-wideband (UWB) to obtain the distance and orientation information between the robot and the target to construct the control law. However, this type of solution relies on stable and reliable distance measurement data. In confined environments such as narrow indoor spaces and deep seas, distance measurement is easily affected by environmental interference, resulting in deviations or even failures. In addition, the hardware cost of ranging sensors is high, and the system deployment complexity is high. Another type of solution uses a lower-cost and simpler-to-deploy monocular vision sensor to obtain the pixel coordinates of the target through visual detection and reconstruct the azimuth information of the target relative to the robot. It designs the orbital control law based solely on the azimuth measurement value, without relying on distance measurement data and global position information, and has become the mainstream research direction in scenarios without global positioning.

[0005] Existing target orbit control schemes based on monocular vision azimuth angle reconstruction still have a number of technical limitations that need to be optimized in practical applications. First, these schemes need to reconstruct azimuth angle information from the target pixel coordinates. The accuracy of azimuth angle calculation is highly dependent on the precise calibration results of the camera. Camera calibration errors will be directly transmitted to the control loop, easily leading to a decrease in control performance or even system instability, and are highly sensitive to camera calibration. Second, most existing schemes do not fully consider the boundedness constraint of the camera's field of view. During orbit control, there is a risk that the target will leave the camera's field of view. Once the target is lost, the orbit control task will be interrupted directly. In addition, some schemes still need to obtain the robot's global position and heading angle information, which further increases the hardware cost and information processing load of the system. Finally, in terms of stability, some of these schemes can only achieve bounded error convergence of the controlled target, and cannot achieve asymptotic convergence without steady-state residue, which is difficult to meet the application requirements of high-precision orbit control.

[0006] Existing solutions are all designed for forward tracking scenarios and cannot adapt to the kinematic characteristics of tangential orbiting of static targets by underactuated robots, which can easily lead to target loss; and they cannot achieve distance control in physical space solely through image feedback when the depth is unknown.

[0007] In summary, how to reduce the sensitivity of the orbit control scheme to camera calibration in the absence of global positioning information and additional ranging sensors, while ensuring that the target remains within the camera's field of view during orbiting, and improving control convergence accuracy and system environmental adaptability, is a core technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] To address the shortcomings and deficiencies of existing technologies, this invention provides a method and system for controlling the orbit of a mobile robot around an unknown static target based on image vision. This method is applicable to incomplete wheeled mobile robots equipped with a monocular camera. This invention abandons the conventional approach of relying on spatial geometry reconstruction in existing vision-based orbit control schemes. It transforms the spatial domain fixed-distance orbit control problem into a coordinate adjustment problem of the normalized image plane of the monocular camera. The image error corresponding to the normalized image coordinates is directly used as the direct feedback quantity for closed-loop control. Throughout the process, there is no need to explicitly solve for the relative position, relative distance, and azimuth angle between the robot and the target, thus fundamentally reducing the sensitivity of the orbit control to camera calibration accuracy. The integrated closed-loop control structure designed in this invention embeds a field-of-view constraint mechanism based on the invariant set of the image plane, eliminating the need for an additional independent field-of-view compensation stage. It can simultaneously achieve asymptotic convergence of image errors and maintain the target field of view throughout the entire orbiting process. The scheme relies solely on image data acquired by a monocular camera, requiring no global position, vehicle orientation information, or additional ranging data. Furthermore, it designs adaptive control schemes for two typical scenarios: known and unknown relative heights between the camera and the target. In the scenario where the relative height is unknown, an adaptive sliding mode control scheme with boundary constraints is used to solve for unknown parameters online. Based on Lyapunov stability theory, the control architecture design of this invention can achieve convergence of orbiting distance without steady-state residual errors. While ensuring control accuracy and system stability, it significantly reduces the system hardware threshold and deployment complexity, demonstrating excellent engineering applicability.

[0009] The specific technical solution adopted by this invention to solve its technical problem is as follows:

[0010] A mobile robot unknown static target surround control method based on image vision is applicable to non-holonomic wheeled mobile robots equipped with a monocular camera, and is used to perform fixed-distance surround control on unknown static targets that are stationary and have no prior pose information.

[0011] Considering existing conventional target-following schemes, the robot's linear velocity points towards the target, and the camera can maintain its field of view by being mounted upright. When performing orbiting around a static target, the underactuated robot's linear velocity is in the tangential direction of the circle. If the camera is still mounted upright, the robot will lose sight of the target once it initiates tangential motion. Therefore, this application must include the camera's rotational offset angle.

[0012] Therefore, as the basic configuration for implementing the present invention, a monocular camera is mounted on the robot body. By fixing or adjusting the mounting position, the optical axis of the camera (i.e., the normal to the image plane) is made perpendicular to the forward direction of the robot, and the optical axis points to the side surrounding the target in the horizontal plane; including:

[0013] The monocular camera acquires images containing unknown static targets in real time, and extracts the pixel coordinates of the target feature points in the image plane.

[0014] Based on the intrinsic parameters of the monocular camera, the pixel coordinates are converted into normalized image plane coordinates;

[0015] Based on the normalized image plane coordinates and the expected steady-state coordinates of the corresponding expected orbital radius, an image error signal is constructed;

[0016] Based solely on image data acquired by a monocular camera, the forward linear velocity control quantity and the turning angular velocity control quantity of the mobile robot are calculated using the image error signal as the direct feedback quantity. The forward linear velocity control quantity and the turning angular velocity control quantity are output through a closed-loop control structure. This control structure embeds a constraint mechanism based on the image plane invariant set, eliminating the need for an additional independent field-of-view compensation control loop. This achieves two control objectives simultaneously: driving the image error signal to converge to the desired steady-state coordinates, and ensuring that the target feature points remain within the effective field of view of the monocular camera throughout the entire orbit control process.

[0017] The calculated forward linear velocity control quantity and steering angular velocity control quantity are output to the motion actuator of the mobile robot, driving the mobile robot to perform fixed-distance orbiting control around the unknown static target with the desired orbiting radius.

[0018] Furthermore, the normalized image plane coordinates include horizontal coordinates p and vertical coordinates q. The conversion method is to use the pixel coordinates of the principal point of the monocular camera as the origin, and combine the scaling factors of the monocular camera in the horizontal and vertical directions to convert the pixel coordinates of the target feature points into the corresponding horizontal coordinates p and vertical coordinates q, respectively. The image error signal includes a horizontal error component and a vertical error component, wherein the horizontal error component is the horizontal coordinate p of the normalized image plane, and the vertical error component is the difference between the vertical coordinate q of the normalized image plane and the desired steady-state vertical coordinate. The desired steady-state vertical coordinate is determined by the relative height between the optical center of the monocular camera and the target feature point, and the desired orbital radius.

[0019] Furthermore, in the calculation of the forward linear velocity control quantity and the steering angular velocity control quantity, it is not necessary to use the relative position, relative distance, azimuth angle between the robot and the target, as well as the robot's global pose and vehicle heading information as intermediate control variables. The control quantity is solved only by normalizing the image plane coordinates.

[0020] Furthermore, existing image-bounded tracking schemes, when the target's depth scale is unknown, can only guarantee that the target does not leave the image, and objectively cannot specify and maintain a fixed following distance in three-dimensional physical space. In contrast, the solution of this invention, without relying on distance sensors or using the robot's global coordinates, achieves pre-specified distance control directly at the image level through pure image error feedback and embedded invariant set constraints. Its core implementation mechanism is as follows:

[0021] The specific implementation of the embedded constraint mechanism based on the image plane invariant set is as follows: Based on the field of view constraint boundary conditions of the monocular camera, a rectangular feasible region matching the image error signal is constructed; based on the boundary of the rectangular feasible region, an elliptical invariant set completely contained within it is designed; the initial value of the image error signal is within the elliptical invariant set to ensure that the image error signal is always within the elliptical invariant set throughout the entire orbit control process.

[0022] Furthermore, when the relative height between the optical center of the monocular camera and the target feature point is known, the forward linear velocity control quantity and the steering angular velocity control quantity are calculated through a closed-loop feedback control law, the expression of which is:

[0023]

[0024] In the formula, w is the steering angular velocity control value, v is the forward linear velocity control value, p and q are the horizontal and vertical coordinates of the normalized image plane, h is the relative height between the optical center of the monocular camera and the target feature point, and d is the desired orbital radius. , The preset positive control gain, This is a predefined bounded steady-state linear velocity function.

[0025] Furthermore, when the relative height between the optical center of the monocular camera and the target feature point is unknown, the forward linear velocity control quantity and the steering angular velocity control quantity are calculated using an adaptive sliding mode control law. At the same time, the unknown relative height parameter is estimated online based on the normalized image plane coordinate data and control quantity data within a preset sampling period using a parameter adaptive law with a projection operator.

[0026] Furthermore, the parameter adaptive law with projection operator is implemented using a parallel learning method, completing parameter iterative updates based on multiple sets of sampled data during the learning phase; the projection operator is used to constrain the upper and lower bounds of the parameter estimates, ensuring the boundedness and convergence of the parameter estimation process, and its execution rule is as follows:

[0027] Based on the known relative height sign and the preset lower and upper bounds of the absolute value of the relative height, the upper and lower boundary thresholds of the parameter estimate are determined.

[0028] When the parameter estimate reaches any of the aforementioned boundary thresholds, and the calculated value of this adaptive update causes the parameter estimate to exceed the boundary threshold, the projection operator outputs 0, and the parameter update stops.

[0029] Except for the cases mentioned above, the projection operator directly outputs the calculated value of this adaptive update and performs parameter iterative update.

[0030] Furthermore, the closed-loop control structure is designed based on Lyapunov stability theory, driving the image error signal to converge asymptotically to the desired steady-state coordinates over time, ultimately enabling the relative distance between the mobile robot and the target to converge to the desired orbital radius without any steady-state residual error.

[0031] Furthermore, the predefined bounded steady-state linear velocity function Satisfy: When time approaches infinity, The absolute value of the lower limit is greater than 0 to ensure that the robot maintains continuous orbital motion throughout the entire process.

[0032] In addition, a mobile robot unknown static target surround control system based on image vision, used to execute the method described above, the system is mounted on a non-complete wheeled mobile robot, including a monocular camera module, an image processing module, a closed-loop control module and a motion execution module;

[0033] The monocular camera module is used to acquire images containing unknown static targets in real time;

[0034] The image processing module is used to extract the pixel coordinates of the target feature points, complete the conversion of pixel coordinates to normalized image plane coordinates, and construct an image error signal;

[0035] The closed-loop control module has an embedded closed-loop control structure with image plane invariant set constraints, which is used to calculate the forward linear velocity control quantity and the turning angular velocity control quantity of the mobile robot with the image error signal as the direct feedback quantity.

[0036] The motion execution module is used to receive control signals and drive the mobile robot to perform a fixed-distance orbital motion.

[0037] And a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described above.

[0038] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0039] Compared to existing technologies, this invention and its preferred solution abandon the conventional approach of relying on spatial geometry reconstruction in existing vision-based surround control schemes. It eliminates the need for explicit solutions to the relative position, distance, and azimuth angle between the robot and the target throughout the entire process, fundamentally avoiding the propagation and accumulation of camera calibration errors to the control system. This effectively reduces the dependence of surround control on camera calibration accuracy and improves the robustness of the control system. This invention employs an integrated closed-loop control architecture, simultaneously achieving control error convergence and target field-of-view maintenance within the same control structure. It eliminates the need for additional independent field-of-view compensation control, ensuring that the target remains within the camera's effective field of view throughout the surround control process. This prevents control task interruption due to target loss and improves the continuity and stability of the surround control process. The surround control of this invention relies solely on image data acquired by a monocular camera, eliminating the need for global position information of the robot and target, robot heading information, or additional ranging sensors. This significantly simplifies the system hardware architecture, reduces deployment costs and environmental dependence, and makes it stably applicable to various restricted application scenarios where satellite positioning signals are denied. Meanwhile, this invention covers two typical application scenarios: known and unknown relative height between the camera and the target. It designs adaptive control schemes for different prior information conditions, effectively broadening the applicable scenarios of the scheme and improving its engineering practicality. Based on stability theory, this invention completes the control architecture design, which can achieve asymptotic convergence of control error. It solves the limitation of existing similar schemes that can only achieve convergence of bounded error, and effectively improves the accuracy and stability of orbit control. Attached Figure Description

[0040] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0041] Figure 1 This is a schematic diagram illustrating the robot kinematics model and coordinate system definition in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of the monocular camera imaging model and coordinate mapping relationship in an embodiment of the present invention;

[0043] Figure 3 This is a diagram of the fixed-distance orbital motion trajectory of a mobile robot in a scenario where the relative height is known, according to an embodiment of the present invention.

[0044] Figure 4 This is a simulation diagram of the control effect in a scenario where the relative altitude of the present invention is known.

[0045] Figure 5 This is a diagram of the fixed-distance orbital motion trajectory of a mobile robot in a scenario with unknown relative height, according to an embodiment of the present invention.

[0046] Figure 6 This is a simulation diagram of the control effect and parameter estimation in a scenario where the relative height is unknown, according to an embodiment of the present invention. Detailed Implementation

[0047] To make the features and advantages of the present invention more apparent and understandable, specific embodiments are described below in detail:

[0048] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0049] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0050] This invention addresses the need for orbiting unknown static targets in incomplete wheeled mobile robots. It abandons the existing approach of pixel coordinate-azimuth angle reconstruction-control law design, directly transforming the spatial domain orbiting control problem into an image plane adjustment problem. It constructs a fully closed-loop control system solely based on image information acquired by a monocular camera, eliminating the need for global robot pose information and additional ranging sensors. Simultaneously, it solves the problems of existing solutions, such as sensitivity to calibration, lack of field-of-view constraints, and poor scene adaptability. The overall implementation process of this solution is as follows:

[0051] The first step is relative position modeling: establish the relative position model of the stationary unknown target in the body coordinate system of the nonholonomic wheeled mobile robot, and combine the nonholonomic kinematic equations of the robot to clarify the core control target that orbits the target at a preset distance;

[0052] The second step is image coordinate mapping: Based on the pinhole camera imaging model, the mapping relationship between the target's spatial relative position and the camera pixel coordinates and normalized image plane coordinates is constructed, the boundary conditions of the camera's field of view constraint are clarified, and the direct correlation between spatial orbital motion and image information is established.

[0053] The third step is to construct visual dynamics: Based on the robot's kinematic model and coordinate mapping relationship, the normalized image coordinates are differentiated to derive the image visual open-loop dynamics system with the robot's linear velocity and angular velocity as control inputs, thus completing the core transformation from the spatial orbit control problem to the image plane adjustment problem.

[0054] The fourth step is to propose reasonable assumptions: clarify the preconditions applicable to the design of the control law and the proof of stability of this scheme, including the initial visibility of the target, the relative height sign and the upper and lower bounds being known, etc., to provide theoretical boundaries for the closed-loop control design;

[0055] Step 5, control design for scenes with known altitude: For scenes where the relative altitude between the camera and the target is measurable and known, a dual-input closed-loop control law based on direct feedback of normalized image error is designed. The camera's field of view constraint is maintained throughout the entire process through the design of elliptic invariant sets, and the asymptotic convergence control of fixed-distance orbit is completed.

[0056] Step 6, Adaptive control design for scenes with unknown altitude: For scenes where the relative altitude between the camera and the target cannot be accurately obtained, a parallel learning adaptive law with projection operator is designed to estimate the unknown relative altitude parameters online. Combined with an adaptive sliding mode controller, target fixed-distance orbit control and full-range field of view maintenance are achieved without prior altitude information.

[0057] The following detailed explanation of the implementation principles, formula derivation, and parameter design of each step is provided through specific examples.

[0058] first step:

[0059] This embodiment first presents the modeling of the target orbit control problem using an airborne monocular camera. The kinematic model of the wheeled mobile robot is as follows:

[0060]

[0061] in, They represent The derivative with respect to time. Indicates the robot's position in the global coordinate system middle Axis position information, For robots relative to The heading angle of the axis, such as Figure 1 As shown. Among them, and These are the robot's forward linear velocity and turning angular velocity, respectively, and are considered as the robot's control inputs.

[0062] Assuming the target to be orbited is stationary but its position is unknown beforehand, for convenience, we will use a coordinate system relative to the global coordinate system. In coordinates of the axis This is indicated by the assumption that the robot cannot obtain global position information for itself and the target, i.e., it is operating in an environment where GNSS signals are unavailable.

[0063] The robot's relative position to the target is defined as... Its coordinates are relative to the robot's body coordinate system. of The axis is given and defined as follows:

[0064]

[0065] because It is an orthogonal matrix, therefore That is, the norm represents the relative distance between the robot and the target. Therefore, for a robot system, the target orbiting problem discussed in this embodiment is defined as: designing control inputs based on the visual servo angle of the image using a monocular camera mounted on the robot, without any mutual communication. and This enables the robot to move around a target at a specified distance. In other words, it is necessary to make... and ,in It is a pre-set positive number.

[0066] Step Two:

[0067] The dynamics of relative position are reconstructed using image visual servoing, and then the target orbiting problem is transformed into an accommodation problem within the image plane depicted by a single constrained field-of-view camera. First, the image plane under the camera's visual frame is rigorously described, such as... Figure 2 As shown.

[0068] Consider a feature point of the target Its coordinate system in the camera coordinate system It has coordinates, where and This represents the spatial coordinates of the point relative to the camera's optical coordinate system. From... point Projected onto the image plane pixel coordinates In the pinhole camera model, the following relationship can be used to describe it:

[0069] in and These represent the scaling factors in the horizontal and vertical directions, respectively. Furthermore, Represents the pixel coordinates of the camera's principal point.

[0070] Due to the camera's field of view constraints, only when the feature points... It can only be detected when it lies within the image plane. In other words, its corresponding pixel coordinates. Subject to the following boundary constraints:

[0071] in and They represent and Maximum pixel coordinates in the direction.

[0072] It is important to note that the monocular camera is rigidly mounted in the robot's fixed coordinate system. of On the axis, the camera's optical axis is parallel to the robot's left and right wheel axles. Simultaneously, the normal to the camera's image plane points to the left side of the robot. The relative height difference between the feature point and the camera's optical center is denoted as... To avoid strange occurrences, it should be ensured that Therefore, feature points In the camera coordinate system The position in can be represented as:

[0073] To facilitate design control, pixel coordinates Normalized coordinates mapped to the normalized image plane ,in satisfy:

[0074] Weighing If the view constraint is satisfied, then the point Staying within the field of vision constraints, i.e., satisfying Specifically, on the one hand, according to the formula Japanese style ,Mode China The constraint is equivalent to the following restriction:

[0075] On the other hand, note that as long as you click Stay within the field of vision constraints, It always holds true, and its sign is the same as... To remain consistent. Therefore, according to the formula... ,Mode Japanese style When the feature point is located in the camera coordinate system Below the optical center (i.e. )hour, It is positive, and Conversely, when the feature point is located above the optical center (i.e., )hour, It is negative, and In other words, China The constraint is equivalent to the following restriction:

[0076] Step 3:

[0077] Next, use coordinates Give a precise definition of the kinematic equations. Since the target is stationary, therefore According to the formula The time derivative of the position error satisfies .on the other hand, Remain unchanged, that is According to the formula After some calculations, the open-loop visual kinematics, expressed in terms of linear velocity and angular velocity, can be obtained as follows:

[0078]

[0079] Then, consider the formula The adjustment problem: Design based on image feedback signal control input and , making ,and Satisfying the field of view constraint, that is:

[0080]

[0081]

[0082] For all ,and ,in To satisfy the formula The constant. Proposition 1: Assume Satisfaction Then the formula The regulation problem can be solved, i.e., equation The target orbiting problem can also be solved.

[0083] Proof: According to ,have ;

[0084] therefore,

[0085] According to the proposition To ensure that the mobile robot reaches the specified distance from the stationary target Simply put the formula status Adjust to a specific point ,in , This represents the relative height between the camera's optical center and the feature point. When When it is measurable and known, It can be known in advance, and therefore can be used directly in the control design. Conversely, when When it cannot be accurately obtained, i.e. If it is unpredictable, then additional adaptive control algorithms need to be designed to identify it. value.

[0086] Step 4:

[0087] To ensure the feasibility of the closed-loop control law, the convergence of parameter estimation, and the effectiveness of the vision constraints, this embodiment proposes the following rational assumptions, the engineering significance and function of each assumption being as follows:

[0088] Assumption Feature points projection Initially, it is located within the constrained field of view camera range (ensuring the validity of the initial conditions of the control loop, providing a premise for subsequent invariant set constraints).

[0089] Assumption : Known The sign of the symbol, and the existence of a positive constant. , making (Ensure that the adaptive parameter estimation process is bounded, avoid singularity in division operations, and ensure that the constraints of the projection operator are effective).

[0090] Assumption 3: A positive constant Meet the conditions (Guarantee the desired steady-state coordinates) Located within the camera's feasible field of view, it provides a basis for steady-state convergence of fixed-distance orbiting.

[0091] Step 5:

[0092] Given the given conditions, the control law is designed as follows:

[0093] in You can choose any option. To be determined later. For a predefined function, satisfying , For an essentially bounded function space, it represents All are bounded functions.

[0094] Define image error as ,in:

[0095]

[0096] To ensure It is valid, as long as the image error is guaranteed. For any Established, among which It is a rectangular field that satisfies:

[0097]

[0098]

[0099] in, These are general placeholder variables within the image error plane, corresponding to the horizontal and vertical components of the image error, respectively, and used to describe the value range of the rectangular feasible region.

[0100] Combined and Image error The closed-loop dynamics are:

[0101]

[0102] This invention constructs an elliptic invariant set based on the Lyapunov function of the closed-loop system. Its design must meet two core requirements: first, the boundary of the elliptic invariant set must be completely within the rectangular feasible region corresponding to the camera's field-of-view constraint, theoretically eliminating the risk of the target going out of the field of view; second, the initial value of the image error must be within this elliptic invariant set to ensure that the trajectory of the closed-loop system does not exceed the set boundary throughout the entire process. The Lyapunov function selected in this embodiment is:

[0103]

[0104] Based on the isosurface of the Lyapunov function, the following elliptic field is defined:

[0105]

[0106] in:

[0107]

[0108] and ,in addition:

[0109]

[0110] in, The convergence margin parameter designed for the invariant set is used to reserve a safe boundary between the elliptical invariant set and the rectangular feasible region, ensuring that image errors do not reach the camera's field of view boundary, while providing sufficient conditions for the system's asymptotic convergence. For any given real number satisfy .

[0111] Solvability of the target orbit problem:

[0112] To prove the stability and invariant set properties of the closed-loop system, the derivative of the selected Lyapunov function along the trajectory of the closed-loop system is taken. Combining this with the closed-loop dynamic equation of equation (13), we can obtain:

[0113]

[0114] This proves that the Lyapunov function is non-increasing along the system trajectory, and the closed-loop system trajectory always remains within the elliptic invariant set corresponding to the initial value, ensuring that the target remains within the camera's field of view throughout the entire process; when the control gain... Furthermore, based on Barbara's principle, image errors can be further demonstrated. The asymptotic convergence to 0 enables fixed-distance orbital control.

[0115] In the assumption Below, for any Assuming ,in It is given by equation (14). Then the equation... Each solution satisfies For any Therefore, during the target orbiting process, the feature points remain within the visual field (i.e., (Established). Furthermore, further assumptions... .in From the formula Given, and Then the image error satisfy Therefore, the formula The target orbiting problem is thus solved; based on the above solvability theorem, in engineering implementation, it is only necessary to ensure that the initial image error lies within the elliptic invariant set. Within the range, and by selecting a control gain that satisfies the constraints, and applying the closed-loop feedback control law of equation (10), the mobile robot can achieve a fixed-distance orbit around an unknown static target, ensuring that the target does not leave the effective field of view of the camera throughout the entire process. The pseudocode for the implementation process is shown below:

[0116] 1. Begin;

[0117] 2. Assumptions 1-2 are true;

[0118] 3. Conditions: ,in From the formula Give;

[0119] 4. Given ;

[0120] : Represents the gain parameter, satisfying ;

[0121] Relative height;

[0122] The distance maintained while orbiting the target;

[0123] The robot's steady-state linear velocity satisfies... ;

[0124] 5. Input: Initial value According to the formula Get control input .Mode Pixel coordinates of the target feature center and camera parameters Acquired by computer vision techniques (e.g., ORB feature detection, SIFT feature detection, KLT optical flow tracking, etc.);

[0125] 6. Controller: in the form of a... As shown;

[0126] 7. Update control input ;

[0127] 8. End.

[0128] For an adaptive closed-loop system, the extended Lyapunov function is chosen as follows:

[0129]

[0130] in The parameter estimation error is given. Differentiating the Lyapunov function along the system trajectory, and combining the closed-loop dynamics of equation (21) with the adaptive update law with the projection operator, it can be proved that... All signals in the closed-loop system are bounded, and the image error and parameter estimation error can asymptotically converge to 0.

[0131] Step 6: Surround control under unknown relative characteristic height.

[0132] In practical applications, accurately determining the relative height between the camera's optical center and the feature point is crucial. It might be quite difficult, especially for various things that cannot be directly measured. In an unknown environment, to address this challenge, this embodiment introduces an adaptive law to estimate... Thus, without directly obtaining precise The feedback control law is obtained under the condition of the value.

[0133] For convenience, please note and order for The estimated value. Replacing the previous value in the formula. The actual value used Using the estimated value Consider the following adaptive control law:

[0134]

[0135]

[0136] in with formula Same as in To be determined in subsequent steps, Represents the sign function. Estimated value. This will be achieved through a parallel learning method, as described below. It is assumed that the learning process and the control process occur simultaneously, but only within a specific time interval. The process will be carried out internally, including This indicates the termination time of the learning process. The learning phase uses a sampling method and will employ... as well as Learning is performed using the sampled data. Sampling time interval The sampling is divided into several finite sampling intervals, each interval corresponding to the time difference between two adjacent sampling times. Let the total number of sampling times be... To enable the control system to start after If the number of samples within a finite positive integer is a given index, then... The value range is 0 to Now, consider the following adaptive update law. :

[0137]

[0138]

[0139] Among them, the initial conditions satisfy ,in and Assuming As shown. Take any one. To be determined later;

[0140] As a preferred embodiment of the present invention, the parallel learning mechanism of the above parameter adaptive law is executed synchronously with the orbital control process. The learning phase and the control phase share the same sampling period of image data and control quantity data, without the need to set up a separate offline learning stage. The adaptive law estimates the unknown relative height parameter online through the correlation analysis of multiple sets of historical sampling data, which can effectively suppress the parameter estimation fluctuation caused by sampling noise at a single moment and ensure the convergence and stability of the estimation process.

[0141] It's about adaptive laws. The projection operator is defined as: for any ,have:

[0142]

[0143] Consider according to formula Defined image error Furthermore, the estimation error is defined. for:

[0144] The formula with formula - By combining these methods, image errors can be obtained. With estimation error The closed-loop dynamics are as follows:

[0145]

[0146]

[0147]

[0148] based on Define the following ellipsoidal region:

[0149]

[0150] in:

[0151]

[0152] in ,and:

[0153]

[0154] Among them, the convergence margin parameter of the invariant set design For any given positive number, satisfying .

[0155] Solvability of the target orbit problem: under the assumption Under the condition that, for any ,like ,and ,in From the formula Given. Then the formula. All solutions satisfy For any Therefore, during the target orbiting process, the feature points always remain within the field of view constraint (i.e., equation...). (This is valid). Furthermore, if we further assume... , ,in Depend on In addition, ,and Then the image error With estimation error satisfy Therefore, the formula The target orbiting problem is thus solved. Based on the aforementioned solvability theorem, in engineering implementation, it is only necessary to ensure that the initial image error lies within the elliptic invariant set. By selecting the control gain that satisfies the constraints and applying the closed-loop feedback control law of formula (10), the mobile robot can achieve a fixed-distance orbit around an unknown static target, and ensure that the target does not leave the effective field of view of the camera throughout the entire process.

[0156] As a further preferred embodiment, whether the initial image error is located within the elliptic invariant set can be verified by the following method: converting the feature point pixel coordinates extracted from the initial frame image into normalized image coordinates, calculating the corresponding image error, and substituting it into the inequality constraints of the elliptic invariant set for verification. If the inequality is satisfied, the initial condition is valid.

[0157] The pseudocode for the implementation process is shown below:

[0158] 1. Begin;

[0159] 2. Assumptions 1-3 are true;

[0160] 3. Conditions: ,in From the formula Give;

[0161] 4. Given ;

[0162] : Represents the gain parameter, satisfying ,

[0163] ;

[0164] Relative height;

[0165] The distance maintained while orbiting the target;

[0166] The robot's steady-state linear velocity satisfies... ;

[0167] 5. Input: Initial value According to the formula Get control input .Mode Pixel coordinates of the center of the target feature and camera parameters Acquired by computer vision technology;

[0168] 6. Controller: in the form of a... As shown;

[0169] 7. Update control input ;

[0170] 8. End.

[0171] This embodiment uses Matlab for simulation verification. Two sets of simulation experiments are given to verify the effectiveness of the proposed target orbit control strategy. The relative height is assumed to be... The camera's intrinsic parameters are set as follows: , and Therefore, from the equation ,have and The expected normalized coordinates are set to... The corresponding relative distance is Therefore, from the formula ,have and For ease of visual demonstration, we assume the target is in the global coordinate system. Central .

[0172] Simulation 1: Relative Height Consider the given information. In this case, the control law formula is used. The design gain is selected as The predefined function is set as follows: For any This holds true. Assume the robot's initial pose is... According to the formula The relative positions of the robot and the target are ,correspond Simulation results can be found in [link / reference]. Figure 3 , Figure 4 .picture The robot's orbital trajectory around a stationary target is demonstrated, clearly depicting the trajectory determined by the proposed control law. The overall motion process is dominated. Furthermore, the specified distance between the robot and the target converges rapidly to the desired value. (Figure) In the first and second subplots, the image error remains within the field of view constraint (marked by dashed lines) and converges rapidly to zero. The third subplot demonstrates the controller's excellent transient response characteristics.

[0173] Simulation-II: Relative Height It is an unknown quantity.

[0174] In this case, assuming Adopting the formula The given control law, in which the designed control gain is set as follows: and order For any This holds true. Assume the robot's initial position is... , ,and According to the formula The relative positions between the robot and the target are , corresponding to .

[0175] Simulation results are as follows Figure 5 , Figure 6 As shown in the figure. This demonstrates the trajectory of a robot performing a circular motion around a stationary target when the relative altitude is unknown, with its steady-state distance remaining within the desired value. (Figure) In the first and second subplots, the image error remains within the field-of-view constraints (shown by dashed lines) and converges rapidly to zero. The third subplot illustrates the controller-based... The graph shows good transient performance, and the last subgraph demonstrates the accurate estimation effect of the adaptive law on unknown parameters.

[0176] In summary, the image vision-based mobile robot unknown static target orbit control method provided by the embodiments of the present invention addresses the technical defects of existing mobile robot target orbit control schemes. Through the design of a visual servo control architecture with direct closed loop in the pixel domain, it achieves high-precision and robust target orbit control without global positioning or additional ranging sensors. At the same time, the system solves the core problems of existing schemes, such as sensitivity to camera calibration, easy loss of target from the camera's field of view, and insufficient scene adaptability.

[0177] This invention breaks through the conventional technical path of existing monocular vision-based surround control schemes that rely on azimuth angle reconstruction. It directly transforms the target surround control problem in the spatial domain into an image plane adjustment problem. Using image visual error as the core control feedback quantity, it constructs a fully closed-loop control system. This eliminates the need to reconstruct explicit spatial geometric quantities such as azimuth angles through camera imaging models, fundamentally avoiding the transmission and accumulation of camera calibration errors to the control loop, and significantly reducing the sensitivity of surround control to camera calibration accuracy. Simultaneously, this control architecture does not rely on the global position information or heading angle information of the robot and target, nor does it require additional ranging equipment such as lidar or ultra-wideband radar. It can complete the entire surround control task solely based on the image pixel values ​​acquired by the monocular camera, effectively simplifying the system hardware architecture, reducing deployment costs and information processing load, and allowing the solution to be stably applied to various restricted scenarios where GNSS signals are denied.

[0178] To address the problem that existing orbit control schemes do not fully consider the boundedness of the camera's field of view and are prone to target loss leading to control task interruption, this invention embeds an image field of view constraint invariance preservation mechanism in the control law design. By constructing an image plane feasible region and elliptical invariant set that adapt to the camera's imaging boundary, and combining constraint-type control term design with Lyapunov stability analysis, it theoretically and rigorously guarantees that the target feature point is always within the camera's effective imaging area throughout the entire orbit control process. This fundamentally avoids the risk of the target leaving the field of view and significantly improves the continuity and stability of the orbit control process.

[0179] To further enhance the scenario adaptability of the solution, this invention designs adaptive closed-loop control schemes for two typical application scenarios: one where the relative height between the camera optical center and the target feature points is known and the other where it is unknown. In scenarios where the relative height is known, a dual-input control law based on direct feedback from normalized image coordinates is designed, enabling asymptotic convergence of the fixed-distance loop error. In scenarios where the relative height cannot be accurately obtained, an adaptive-sliding mode controller based on image visual feedback is designed, coupled with a parallel learning adaptive law with a projection operator. This allows for accurate online estimation of unknown relative height parameters using only image measurement data, eliminating the need for additional height measurement sensors. This overcomes the shortcomings of existing similar solutions that cannot simultaneously cover both scenarios, allowing the solution to be widely adaptable to various application environments with different prior information.

[0180] In summary, this invention forms a complete surround control system for unknown static targets that relies solely on monocular vision through a pixel-domain direct closed-loop control architecture, an embedded field-of-view constraint preservation mechanism, and a dual-scene adaptive control scheme design. Its core innovations include a surround control method directly based on image visual errors, an invariant set construction and controller design adapted to camera field-of-view constraints, image control laws and adaptive parameter estimation schemes for scenarios with known and unknown relative heights, and a fully closed-loop surround control implementation architecture that relies solely on monocular image measurements. While ensuring control accuracy and stability, this invention effectively reduces the system's hardware threshold and environmental dependence, possessing strong engineering application value and promising prospects for widespread adoption.

[0181] Based on the same inventive concept, the present invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the computer programs include program instructions, and the processor is configured to execute the program instructions to implement the image vision-based mobile robot unknown static target surround control method described in the above embodiments, specifically including:

[0182] 1. Control the monocular camera to acquire images containing unknown static targets, extract the pixel coordinates of target feature points and convert them into normalized image plane coordinates;

[0183] 2. Construct an image error signal based on normalized image plane coordinates, and use the image error signal as a direct feedback quantity to calculate the forward linear velocity control quantity and the turning angular velocity control quantity of the mobile robot;

[0184] 3. Execute closed-loop control logic with embedded image plane invariant set constraints to simultaneously achieve image error convergence and target field of view preservation;

[0185] 4. Output the control signal to the motion actuator of the mobile robot to drive the robot to complete the fixed-distance orbit control.

[0186] The processor may be a central processing unit (CPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or other programmable logic devices. A general-purpose processor may be a microprocessor or any conventional processor.

[0187] Based on the same inventive concept, this invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it performs the image vision-based mobile robot unknown static target surround control method described in the above embodiments. The computer-readable storage medium can be an electrical, magnetic, optical, or semiconductor system, device, or instrument, such as a portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), flash memory, fiber optic medium, portable compact disk read-only memory (CD-ROM), etc., and is applicable to various tangible storage media required for patent examination.

[0188] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0189] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

[0190] This invention is not limited to the preferred embodiment described above. Anyone inspired by this invention can derive various other forms of image vision-based mobile robot unknown static target orbit control methods. All equivalent variations and modifications made within the scope of the claims of this invention should be included within the scope of this invention.

Claims

1. A method for controlling the orbit of an unknown static target in a mobile robot based on image vision, applicable to non-holonomic wheeled mobile robots equipped with a monocular camera, used to perform fixed-distance orbit control on an unknown static target that is stationary and has no prior pose information; characterized in that, The monocular camera is mounted on the robot body, with its optical axis perpendicular to the robot's forward direction and pointing horizontally towards the side surrounding the target. The monocular camera acquires images containing unknown static targets in real time, and extracts the pixel coordinates of the target feature points in the image plane. Based on the intrinsic parameters of the monocular camera, the pixel coordinates are converted into normalized image plane coordinates; Based on the normalized image plane coordinates and the expected steady-state coordinates of the corresponding expected orbital radius, an image error signal is constructed; Based solely on image data acquired by a monocular camera, the forward linear velocity control and steering angular velocity control of the mobile robot are calculated using the image error signal as direct feedback. These control quantities are output through a closed-loop control structure, which embeds a constraint mechanism based on an image plane invariant set: a rectangular feasible region matching the image error signal is constructed based on the monocular camera's field-of-view constraint boundary conditions; an elliptical invariant set completely contained within the boundary of this rectangular feasible region is designed; the initial value of the image error signal lies within the elliptical invariant set to ensure that the image error signal remains within the elliptical invariant set throughout the entire orbiting control process; the elliptical invariant set is defined by the isosurface of a Lyapunov function selected in the closed-loop control structure, and its boundary lies entirely within the rectangular feasible region; no additional independent field-of-view compensation control is required to simultaneously achieve two control objectives: driving the image error signal to converge to the desired steady-state coordinates, and ensuring that the target feature points remain within the effective field of view of the monocular camera throughout the entire orbiting control process. The calculated forward linear velocity control quantity and steering angular velocity control quantity are output to the motion actuator of the mobile robot, driving the mobile robot to perform fixed-distance orbiting control around the unknown static target with the desired orbiting radius.

2. The method for controlling the orbit of an unknown static target in a mobile robot based on image vision according to claim 1, characterized in that: The normalized image plane coordinates include horizontal coordinates p and vertical coordinates q. The conversion method is to use the pixel coordinates of the principal point of the monocular camera as the origin, and combine the scaling factors of the monocular camera in the horizontal and vertical directions to convert the pixel coordinates of the target feature points into the corresponding horizontal coordinates p and vertical coordinates q, respectively. The image error signal includes a horizontal error component and a vertical error component. The horizontal error component is the horizontal coordinate p of the normalized image plane, and the vertical error component is the difference between the vertical coordinate q of the normalized image plane and the desired steady-state vertical coordinate. The desired steady-state vertical coordinate is determined by the relative height between the optical center of the monocular camera and the target feature point, and the desired orbital radius.

3. The image vision-based mobile robot unknown static target orbit control method according to claim 1, characterized in that: In the calculation of the forward linear velocity control quantity and the steering angular velocity control quantity, it is not necessary to use the relative position, relative distance, azimuth angle between the robot and the target, as well as the robot's global pose and vehicle heading information as intermediate control variables. The control quantity is solved only by normalizing the image plane coordinates.

4. The image vision-based mobile robot unknown static target orbit control method according to claim 1, characterized in that: When the relative height between the optical center of the monocular camera and the target feature point is known, the forward linear velocity control quantity and the steering angular velocity control quantity are calculated through a closed-loop feedback control law. The expression of the closed-loop feedback control law is as follows: In the formula, w is the steering angular velocity control value, v is the forward linear velocity control value, p and q are the horizontal and vertical coordinates of the normalized image plane, h is the relative height between the optical center of the monocular camera and the target feature point, and d is the desired orbital radius. , The preset positive control gain, This is a predefined bounded steady-state linear velocity function.

5. The image vision-based mobile robot unknown static target orbit control method according to claim 1, characterized in that: When the relative height between the optical center of the monocular camera and the target feature point is unknown, the forward linear velocity control quantity and the steering angular velocity control quantity are calculated using an adaptive sliding mode control law. At the same time, the unknown relative height parameter is estimated online based on the normalized image plane coordinate data and control quantity data within a preset sampling period using a parameter adaptive law with a projection operator.

6. The image vision-based mobile robot unknown static target orbit control method according to claim 5, characterized in that: The parameter adaptive law with projection operator is implemented using a parallel learning method, completing parameter iterative updates based on multiple sets of sampled data during the learning phase; the projection operator is used to constrain the upper and lower bounds of the parameter estimates, ensuring the boundedness and convergence of the parameter estimation process, and its execution rule is as follows: Based on the known relative height sign and the preset lower and upper bounds of the absolute value of the relative height, the upper and lower boundary thresholds of the parameter estimate are determined. When the parameter estimate reaches any of the aforementioned boundary thresholds, and the calculated value of this adaptive update causes the parameter estimate to exceed the boundary threshold, the projection operator outputs 0, and the parameter update stops. Except for the cases mentioned above, the projection operator directly outputs the calculated value of this adaptive update and performs parameter iterative update.

7. The image vision-based mobile robot unknown static target orbit control method according to claim 1, characterized in that: The closed-loop control structure is designed based on Lyapunov stability theory, which drives the image error signal to converge asymptotically to the desired steady-state coordinates over time, ultimately enabling the relative distance between the mobile robot and the target to converge to the desired orbital radius without any steady-state residual error.

8. The image vision-based mobile robot unknown static target orbit control method according to claim 4, characterized in that: The predefined bounded steady-state linear velocity function Satisfy: When time approaches infinity, The absolute value of the lower limit is greater than 0 to ensure that the robot maintains continuous orbital motion throughout the entire process.

9. A mobile robot unknown static target surround control system based on image vision, used to execute the method according to any one of claims 1-8, characterized in that, The system is mounted on a non-complete wheeled mobile robot and includes a monocular camera module, an image processing module, a closed-loop control module, and a motion execution module. The monocular camera module is used to acquire images containing unknown static targets in real time; The image processing module is used to extract the pixel coordinates of the target feature points, complete the conversion of pixel coordinates to normalized image plane coordinates, and construct an image error signal; The closed-loop control module has an embedded closed-loop control structure with image plane invariant set constraints, which is used to calculate the forward linear velocity control quantity and the turning angular velocity control quantity of the mobile robot with the image error signal as the direct feedback quantity. The motion execution module is used to receive control signals and drive the mobile robot to perform a fixed-distance orbital motion.

Images