Vision-based mobile robot navigation-following formation control method and system

Abstract

The invention discloses a vision-based mobile robot navigation-following formation control method and system. The method comprises the steps of S1, installing vision cameras on other mobile robots except a first mobile robot in a multi-robot system; s2, establishing a dipole vector field, and generating and maintaining a required formation shape; s3, converting the visibility constraint of the visual camera into an angular velocity input constraint and a linear velocity input constraint, respectively constructing an angular velocity control scheme and a linear velocity control scheme, and then respectively designing an angular velocity auxiliary system and a linear velocity auxiliary system by using the angular velocity control scheme and the linear velocity control scheme; and S4, estimating the pose of the navigator, and realizing formation control under the condition of no communication according to the pose of the navigator in combination with the angular velocity, the state of the linear velocity auxiliary system and the dipole vector field. According to the method, the communication-free autonomous operation capability of the navigation-following formation mode is promoted, and the collaboration and stability of the formation are ensured by designing the dipole vector field.

Description

Technical Field

[0001] The present invention relates to the field of robot motion control, and in particular to a vision-based mobile robot pilot-follower formation control method and system. Background Art

[0002] Visual positioning technology is a positioning method based on image acquisition and analysis. It obtains image information of the environment or target object through the device, and uses computer vision algorithms to process and extract features of the image, so as to achieve accurate estimation of the robot's position and posture. This technology generates position information by analyzing visual data in real time, and provides control instructions to the robot based on the positioning results, so as to effectively adjust the robot's positioning to reduce position deviation. The core of formation control using Kinect camera is to estimate the robot's posture relative to the reference coordinate system. Many solutions rely on the comprehensive application of multiple sensors to meet performance standards, but in scenarios where there is a lack of information sharing, at least three landmark points are required to achieve positioning, which poses a significant challenge. In addition, existing research has insufficient discussion on ensuring visibility constraints between members, especially in maintaining spatial relationships with dynamic objects (such as navigators). Accurately estimating the relative posture and motion state of moving targets is crucial to maintaining visibility and specific displacement relationships, but the dynamic model of the robot system usually does not directly contain bearing angle information, which makes the robot formation have poor control performance without direct feedback to the controller and cannot be strictly formed.

[0003] Based on this, the problem of how to achieve formation tracking control between multiple robots without direct feedback to the controller needs to be solved urgently. Summary of the invention

[0004] The present invention provides a vision-based mobile robot pilot-follower formation control method and system to solve the technical problems mentioned in the background technology.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] The present invention provides a vision-based mobile robot pilot-follower formation control method, comprising the following steps:

[0007] S1. Installing visual cameras on mobile robots other than the first mobile robot in the multi-robot system;

[0008] S2, establish a dipole vector field to generate and maintain the desired formation shape based on the desired position difference between the robots;

[0009] S3, converting the visibility constraint of the visual camera into an angular velocity input constraint and a linear velocity input constraint, and constructing an angular velocity control scheme and a linear velocity control scheme according to the angular velocity input constraint and the linear velocity input constraint, and then using the angular velocity control scheme and the linear velocity control scheme to design an angular velocity auxiliary system and a linear velocity auxiliary system respectively;

[0010] S4. Use the visual camera to capture Kinect images, and estimate the position and posture of the navigator through the Kinect images. According to the position and posture of the navigator and in combination with the state of the angular velocity auxiliary system, the state of the linear velocity auxiliary system and the dipole vector field, formation control is realized without communication.

[0011] Furthermore, the S1 is specifically:

[0012] A Kinect camera is installed on the follower robot in the multi-robot system V to estimate the posture of the local navigator, which includes the relative direction and relative position; wherein the multi-robot system V = {V1, ...V i ....V j ..., V N} includes multiple mobile robots, and the multiple mobile robots are divided into multiple echelons; N represents the total number of mobile robots;

[0013] When the number of echelons is two, the mobile robot in the first echelon is the local navigator, that is, the navigator V j , the next echelon is the follower V i ;

[0014] When the number of echelons is three or more, the mobile robot in the first echelon is the global leader. Starting from the second echelon, the next echelon and the previous echelon are followers V respectively. i and local navigator, global navigator and local navigator are collectively referred to as Navigator V j .

[0015] Furthermore, the robot equipped with a Kinect camera has a limited visual detection range, and the specific visibility constraints are as follows:

[0016]

[0017] Among them, γ i Indicates follower V i The azimuth in the local navigator coordinate system J, where the local navigator coordinate system J is a coordinate system established with the local navigator as the origin; r i Indicates follower V i The relative distance to the local navigator; γ max Indicates the maximum azimuth angle; r maxis the equilateral length of the viewing cone; ||.|| represents the Euclidean vector norm;

[0018] In the above formula, r i The following relationship is satisfied:

[0019]

[0020] Among them, x i and i Followers V i The x-axis coordinate value and the y-axis coordinate value in the local navigator coordinate system J;

[0021] Furthermore, the S2 specifically includes the following steps:

[0022] S21. Establish a dipole vector field for each follower. The expression of the dipole vector field is as follows:

[0023]

[0024] Among them, ε i =[ε xi ε yi ] T Is a follower V i The formation error vector relative to the local leader coordinate system J is: xi and ε yi Respectively represent the followers V i The components of the formation error vector on the x-axis and y-axis relative to the local leader coordinate system J; F i (ε i ) indicates the follower V i The dipole vector field of T , p≠0; T represents the transpose of the matrix;

[0025] S22. For each follower V i An initial controller is established, and then the desired formation with respect to the local leader coordinate system J is achieved based on the dipole vector field and with the help of the initial controller.

[0026] Furthermore, the follower V i The initial controller u 0i =[v 0i ω 0i ] T :v 0i and ω 0i Respectively represent the followers V i The linear velocity and angular velocity of v 0i and ω 0i The calculation formula is as follows:

[0027]

[0028]

[0029] Among them, k1 and k2 are two positive scalars; β i Indicates follower V i With Navigator V j The direction difference relative to the global coordinate system G, which is a coordinate system with a fixed point in the space where the mobile robot is located as reference; v max Indicates the maximum linear speed of the mobile robot; express The derivative of represents the dipole vector field at point (x i ,y i ) direction;

[0030] Direction difference β i The following relationship is satisfied:

[0031] β i =θ i -θ j =ω i -ω j

[0032] Among them, θ i Indicates follower V i Relative to the direction of the global coordinate system G, ω i Indicates follower V i Angular velocity; ω j Indicates Navigator V j Angular velocity;

[0033] The following relationship is satisfied:

[0034]

[0035] Among them, F yi , F xi Followers V i The components of the dipole vector field on the y-axis and x-axis; and F yi , F xi They satisfy the following relationships respectively:

[0036] F yi =3ε xi ε yi

[0037]

[0038] Furthermore, the S3 specifically includes the following steps:

[0039] S31, define the visibility constraints of the Kinect camera in the actual motion scene. The visibility constraints of the Kinect camera include azimuth constraints and distance constraints. In order to ensure that the follower V i Effectiveness under visual constraints, convert the above visibility constraints into actual control input constraints; specifically, convert the azimuth constraint into an angular velocity input constraint, and convert the distance constraint into a linear velocity input constraint;

[0040] S32, constructing the angular velocity input saturation function S(ω) according to the angular velocity input constraint and the linear velocity input constraint i ) and the linear velocity input saturation function S(v i ) to avoid saturation of mobile robot input;

[0041] S33, according to the angular velocity input saturation function S(ω i )Define the follower V i The angular velocity difference Δω i , Δω i =S(ω i )-ω i , then by Δω i =S(ω i )-ω i Get the control scheme of angular velocity;

[0042] Similarly, according to the linear velocity input saturation function S(v i )Define the follower V i Linear velocity difference Δv i , Δv i =S(v i )-V i , then by Δv i =S(v i )-v i Get the control scheme of line speed;

[0043] The control scheme of angular velocity and the control scheme of linear velocity constitute the optimized controller;

[0044] S34. Then, the angular velocity auxiliary system and the linear velocity auxiliary system are designed respectively through the optimized controller, that is, the angular velocity control scheme and the linear velocity control scheme, and the formation error is gradually approached to zero by adjusting the angular velocity auxiliary system and the linear velocity auxiliary system.

[0045] Furthermore, the azimuth constraint is expressed by a formula, which is as follows:

[0046] -γ max ≤γ i ≤γ max

[0047] The distance constraint is expressed by a formula, which is as follows:

[0048]

[0049] in, Indicates follower V i The maximum allowed distance to the target location, and

[0050] The angular velocity input constraint is expressed by a formula, which is as follows:

[0051]

[0052] Among them, k3 is the third positive scalar; Indicates the lower limit of the angle range, Indicates the upper limit of the angle range;

[0053] The linear velocity input constraint is expressed by a formula, which is as follows:

[0054]

[0055] in, Indicates follower V i The derivative of the maximum allowed distance from the target position, k5 represents the fifth positive scalar; α i Indicates follower V i the polar angle of the current position relative to the target position, and

[0056] Furthermore, the angular velocity input saturation function in S32 is specifically as follows:

[0057]

[0058] Among them, S(ω i ) represents the angular velocity input saturation function: represents the lower limit of angular velocity, Indicates the upper limit of angular velocity; and The following relationship is satisfied:

[0059]

[0060] in, Indicates the angle offset;

[0061] The linear velocity input saturation function in S32 is specifically as follows:

[0062]

[0063] in, Indicates the lower limit of line speed; The following relationship is satisfied:

[0064]

[0065] Furthermore, the control scheme of the angular velocity in S33 is as follows:

[0066]

[0067] Among them, 1i Indicates the status of the angular velocity assist system;

[0068] The control scheme of the line speed is as follows:

[0069]

[0070] Among them, 2i Indicates the status of the Line Speed ​​Assist system.

[0071] Furthermore, the angular velocity auxiliary system in S34 is specifically as follows:

[0072]

[0073] Among them, Δα i Indicates the angle difference between the current angle and the target angle, and χ1 indicates a very small positive value set;

[0074] The line speed auxiliary system is specifically as follows:

[0075]

[0076] Among them, k6 represents the sixth positive scalar, and χ2 represents another set extremely small positive value.

[0077] On the other hand, the present invention also provides a vision-based mobile robot pilot-follower formation system, including a multi-robot system V, wherein the multiple mobile robots in the multi-robot system V are controlled in formation according to the above mobile robot pilot-follower formation control method.

[0078] Beneficial effects of the present invention:

[0079] 1. The present invention proposes a vision-based mobile robot pilot-follower formation control method, which is a low-cost, fully autonomous method that conforms to the development trend of multi-robot formation control. Relatively speaking, the Kinect camera is a powerful and reasonably priced sensor, and the image-based calculation is also relatively low in computational cost. Each follower is equipped with a camera, and the Kinect camera is used as the only sensor. It is a sensor for detecting the posture of the navigator, responsible for capturing the position and direction information of the navigator; at the same time, it serves as a direction indicator for the follower, guiding the movement of the follower through visual feedback, thereby replacing the gyroscope or acceleration sensor that may be relied on in the traditional formation.

[0080] 2. The present invention can determine the follower V without communication between mobile robots and without laser rangefinders. i and Navigator V j The distance between. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 Schematic diagram of visibility constraints of the Kinect camera in the present invention; wherein (a) is a schematic diagram of positioning based on vision; (b) is a schematic diagram of visual detection range;

[0082] Figure 2 Schematic diagram of visual tracking results in Gazebo in an embodiment of the present invention;

[0083] Figure 3 Schematic diagram of some variables in the embodiments of the present invention;

[0084] Figure 4 is a schematic diagram of some parameters between a leader and a follower in an embodiment of the present invention;

[0085] Figure 5 It is a schematic diagram of the structure of the leader and the follower in the present invention;

[0086] Figure 6 This is an example diagram of the formation shape of multiple mobile robots in an embodiment of the present invention. DETAILED DESCRIPTION

[0087] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many other different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0088] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0089] Reference Figure 1 The present application embodiment provides a vision-based mobile robot pilot-follower formation control method, comprising the following steps:

[0090] S1. Installing visual cameras on mobile robots other than the first mobile robot in the multi-robot system;

[0091] Here, the first mobile robot cannot be simply understood as the frontmost mobile robot. Specifically, if the multi-robot system has only two echelons, a visual camera is installed on the follower. If the multi-robot system has three or more echelons, a visual camera is installed on other mobile robots except the global leader.

[0092] S2, establish a dipole vector field to generate and maintain the desired formation shape based on the desired position difference between the robots;

[0093] S3, converting the visibility constraint of the visual camera into an angular velocity input constraint and a linear velocity input constraint, and constructing an angular velocity control scheme and a linear velocity control scheme according to the angular velocity input constraint and the linear velocity input constraint, and then using the angular velocity control scheme and the linear velocity control scheme to design an angular velocity auxiliary system and a linear velocity auxiliary system respectively;

[0094] S4. Use the visual camera to capture Kinect images, and estimate the position and posture of the navigator through the Kinect images. According to the position and posture of the navigator and in combination with the state of the angular velocity auxiliary system, the state of the linear velocity auxiliary system and the dipole vector field, formation control is realized without communication.

[0095] In some embodiments, S1 specifically includes:

[0096] Kinect cameras are installed on the mobile robots other than the first mobile robot in the multi-robot system V to estimate the posture of the local navigator; the posture of the local navigator includes relative direction and relative position; wherein the multi-robot system V = {V1, ...V i ....V j ..., V N} includes multiple mobile robots, and the multiple mobile robots are divided into multiple echelons; N represents the total number of mobile robots;

[0097] When the number of echelons is two, the mobile robot in the first echelon is the local navigator, that is, the navigator V j , the next echelon is the follower V i ;

[0098] When the number of echelons is three or more, the mobile robot in the first echelon is the global leader. Starting from the second echelon, the next echelon and the previous echelon are followers V respectively. i and local navigator, global navigator and local navigator are collectively referred to as Navigator V j .

[0099] Reference Figure 6 , Figure 6 is an example diagram of a formation shape of multiple mobile robots in an embodiment of the present invention, Figure 6 The multiple mobile robots are divided into three echelons, among which the mobile robot in Level 1 is the global leader, the two mobile robots in Level 2 are followers relative to the mobile robot in Level 1, the multiple mobile robots in Level 3 are also followers relative to the two mobile robots in Level 2, and the two mobile robots in Level 2 are local leaders relative to the multiple mobile robots in Level 3.

[0100] In some embodiments, the Kinect camera has visibility constraints in the actual motion scene, and the field of view of the Kinect camera is represented as an isosceles triangle cone (i.e. Figure 1 Its limited viewing angle is 2γ max <π. Follower V i Positioning is performed relative to its local navigator, and the displacement of the x-coordinate and y-coordinate as well as the relative orientation β in the local navigator coordinate system J are measured. i and the azimuth angle γ i ∈[-γ max , γ max ],like Figure 1 The relative distance is defined as x i and i Followers V i The x-axis coordinate value and y-axis coordinate value in the local leader coordinate system J. Since the mobile robot has a limited visual detection range (i.e., visibility constraints), the target can only be detected if the following visibility constraints are met: The target includes the follower v i The target and leader v j The moving target point;

[0101] The robot equipped with a Kinect camera has a limited visual detection range. The specific visibility constraints are as follows:

[0102]

[0103] Among them, γ i Indicates follower V i The azimuth in the local navigator coordinate system J, r i Indicates follower V i The distance relative to the local pilot coordinate system J; the local pilot coordinate system J is based on the local pilot V j The coordinate system established for the origin; γ max Indicates the maximum azimuth angle; r max is the equilateral length of the viewing cone; ||.|| represents the Euclidean vector norm.

[0104] In some embodiments, S2 specifically includes the following steps:

[0105] S21. Establish a dipole vector field for each follower. The expression of the dipole vector field is as follows:

[0106]

[0107] Among them, ε i =[ε xi ε yi ] T Is a follower V i The formation error vector relative to the local leader coordinate system J is: xi and ε yi Respectively represent the followers V i The components of the formation error vector on the x-axis and y-axis relative to the local leader coordinate system J; F i (ε i ) indicates the follower V i The dipole vector field of T , p≠0; T represents the transpose of the matrix;

[0108] S22. For each follower V i An initial controller is established, and then the desired formation with respect to the local leader coordinate system J is achieved based on the dipole vector field and with the help of the initial controller.

[0109] In some embodiments, the follower V i The initial controller u 0i =[v 0i ω zi ] T :v 0i and ω 0iRespectively represent the followers V i The linear velocity and angular velocity of v 0i and ω 0i The calculation formula is as follows:

[0110]

[0111]

[0112] Among them, k1 and k2 are two positive scalars; β i Indicates follower V i The direction difference with the local navigator relative to the global coordinate system G, which is a coordinate system with a fixed point in the space where the mobile robot is located as reference; v max Indicates the maximum linear speed of the mobile robot; express The derivative of represents the dipole vector field at point (x i ,y i ), (xi, yi) represents the direction of the follower V i Position coordinates relative to its local navigator coordinate system J;

[0113] Direction difference β i The following relationship is satisfied:

[0114] β i =θ i -θ j =ω i -ω j

[0115] Among them, θ i Indicates follower V i Relative to the direction of the global coordinate system G, ω i Indicates follower V i Angular velocity; ω j Indicates Navigator V j Angular velocity;

[0116] The following relationship is satisfied:

[0117]

[0118] Among them, F yi , F xi Followers V i The components of the dipole vector field on the y-axis and x-axis; and F yi , F xi They satisfy the following relationships respectively:

[0119] F yi=3ε xi ε yi

[0120]

[0121] In some embodiments, S3 specifically includes the following steps:

[0122] S31, defining the visibility constraints of the Kinect camera in the actual motion scene, the visibility constraints of the Kinect camera include azimuth constraints and distance constraints; in order to ensure the effectiveness of the mobile robot under visual constraints, the above visibility constraints are converted into actual control input constraints; specifically, the azimuth constraints are converted into angular velocity input constraints, and the distance constraints are converted into linear velocity input constraints;

[0123] S32, constructing the angular velocity input saturation function S(ω) according to the angular velocity input constraint and the linear velocity input constraint i ) and the linear velocity input saturation function S(v i ) to avoid saturation of mobile robot input;

[0124] S33, according to the angular velocity input saturation function S(ω i )Define the follower V i The angular velocity difference Δω i , Δω i =S(ω i )-ω i , then by Δω i =S(ω i )-ω i Get the control scheme of angular velocity;

[0125] Similarly, according to the linear velocity input saturation function S(v i )Define the follower V i Linear velocity difference Δv i , Δv i =S(vi)-V i , then by Δv i =S(vi)-vi to obtain the control scheme of line speed;

[0126] The control scheme of angular velocity and the control scheme of linear velocity constitute the optimized controller;

[0127] S34. Then, the angular velocity auxiliary system and the linear velocity auxiliary system are designed respectively through the optimized controller, that is, the angular velocity control scheme and the linear velocity control scheme, and the formation error is gradually approached to zero by adjusting the angular velocity auxiliary system and the linear velocity auxiliary system.

[0128] In some embodiments, the azimuth constraint is expressed by a formula, as follows:

[0129] -γ max ≤γ i ≤γ max

[0130] The distance constraint is expressed by a formula, which is as follows:

[0131]

[0132] in, Indicates follower V i The maximum allowed distance to the target location, and

[0133] The angular velocity input constraint is expressed by a formula, which is as follows:

[0134]

[0135] Among them, k3 is the third positive scalar; Indicates the lower limit of the angle range, Indicates the upper limit of the angle range;

[0136] The linear velocity input constraint is expressed by a formula, which is as follows:

[0137]

[0138] in, Indicates follower V i The derivative of the maximum allowed distance from the target position, k5 represents the fifth positive scalar; α i Indicates follower V i the polar angle of the current position relative to the target position, and

[0139] In some embodiments, the angular velocity input saturation function in S32 is specifically as follows:

[0140]

[0141] Among them, S(ω i ) represents the angular velocity input saturation function; represents the lower limit of angular velocity, represents the upper limit of angular velocity, and The following relationship is satisfied:

[0142]

[0143] in, Indicates the angle offset;

[0144] The linear velocity input saturation function in S32 is specifically as follows:

[0145]

[0146] in, Indicates the lower limit of line speed; The following relationship is satisfied:

[0147]

[0148] In some embodiments, the control scheme of the angular velocity in S33 is as follows:

[0149]

[0150] Among them, 1i Indicates the status of the angular velocity assist system;

[0151] The control scheme of the line speed is as follows:

[0152]

[0153] Among them, 2i Indicates the status of the Line Speed ​​Assist system.

[0154] In some embodiments, the angular velocity auxiliary system in S34 is specifically as follows:

[0155]

[0156] Among them, Δα i Indicates the angle difference between the current angle and the target angle; χ1 indicates a very small positive value set;

[0157] The line speed auxiliary system is specifically as follows:

[0158]

[0159] Among them, k6 represents the sixth positive scalar, and χ2 represents another set extremely small positive value.

[0160] In some embodiments, the S4 specifically includes the following steps:

[0161] S41, Kinect camera uses KCF algorithm to track the local navigator in real time; and obtains the position information of the local navigator in real time;

[0162] S42. In order to further obtain the information required to estimate the local navigator's position and posture, the Kinect board area in the navigator device is selected as the region of interest (ROI) of the visual tracking algorithm. Figure 2 Figure (a) shows the visual tracking result in Gazebo, where the box surrounds the ROI and the dot at the other end of the arrow marks the center of mass of the ROI;

[0163] S43, obtain the position information of the left boundary midpoint, the centroid and the right boundary midpoint of the tracking area from the RGBD image of the target tracking algorithm, the specific information is as follows: Figure 2 As shown in the figure, rL=[xL, yL, dL], rM=[xM, yM, dM] and rR=[xR, yR, dR], corresponding to the left boundary midpoint position [xL, yL], the centroid position [xM, yM] and the right boundary midpoint position [xR, yR] of the tracking area on the depth image, and the depth values ​​dL, dM and dR corresponding to the three target points;

[0164] S44. Use a directed graph to simulate the information exchange between robots through forward-looking vision. Each robot is represented as a node in the directed graph, and the information exchange is represented as directed edges connecting these nodes. The direction of the edge represents the direction in which information flows from one robot to another. Define l r is the actual length of the local leader's kinect on the axis where the long side of the follower's kinect is located, l f For followers V i Local Navigator V from perspective j The length of kinect;

[0165] These variables are already in Figure 3 and Figure 4 Mark out. Then there is l f = xR-xL and l r =κdMl f , where κ>0 is the scaling factor mapping the length measured in pixels to the length measured in meters; the direction β relative to the local navigator coordinate system J i The estimate of can be expressed as β i =arctan((d L -d R ) / L r ); define rO = [xO, yO] as the centroid of the depth image, lf as the horizontal offset in the depth image, that is, lf = xM-xO. Let lir be the navigator V j Kinect center to follower v i The offset of the kinect center on the axis;

[0166] S45, Follower V i The long side of the kinect center is located in the global coordinate system and can be estimated as From this we can infer that γi =arctan(-κl f );

[0167] S46, Solve the follower V i The distance between the local navigator and i , Then calculate the follower V i Relative to the coordinates of the local navigator, the calculation formula is as follows:

[0168] x i =-r i cosα i &y i =-r i sinα i

[0169] S47, infer the follower V i The formation error is as follows:

[0170] ε xi =-r i cosα i -δ xi &ε yi =-r i sinα i -δ yi

[0171] At this point, the position of the local navigator is obtained;

[0172] S48. Realize formation control without communication based on the position and posture of the local navigator and in combination with the state of the angular velocity auxiliary system, the state of the linear velocity auxiliary system and the dipole vector field.

[0173] The present invention proposes a vision-based mobile robot pilot-follower formation control method, which is a low-cost, fully autonomous method and conforms to the development trend of multi-robot formation control. Relatively speaking, the Kinect camera is a powerful and reasonably priced sensor, and the image-based calculation is selected, so the calculation cost is relatively low. Each follower is equipped with a camera, and the Kinect camera is used as the only sensor, which is not only a sensor for detecting the posture of the pilot, but also responsible for capturing the position and direction information of the pilot; at the same time, it serves as a direction indicator for the follower, guiding the movement of the follower through visual feedback, thereby replacing the gyroscope or acceleration sensor that may be relied on in the traditional formation.

[0174] The present invention can determine the follower V without communication between mobile robots and without laser rangefinders. i and Navigator V jThe distance between the two. The Kinect image captured by the Kinect camera can be used to estimate the navigator's position. Microsoft's Kinect camera combines 3D depth map information with traditional RGB color camera data. Compared with stereo cameras, it can obtain RGB images and depth images at the same time faster.

[0175] The second aspect of the present invention also provides a vision-based mobile robot pilot-follower formation system, including a multi-robot system V, wherein the multiple mobile robots in the multi-robot system V are controlled in formation according to the above mobile robot pilot-follower formation control method.

[0176] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention. In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A vision-based mobile robot pilot-follower formation control method, characterized in that: The steps include: S1. Installing visual cameras on other mobile robots except the first mobile robot in the multi-robot system; S2, establish a dipole vector field to generate and maintain the desired formation shape based on the desired position difference between the robots; S3, converting the visibility constraint of the visual camera into an angular velocity input constraint and a linear velocity input constraint, and constructing an angular velocity control scheme and a linear velocity control scheme according to the angular velocity input constraint and the linear velocity input constraint, and then using the angular velocity control scheme and the linear velocity control scheme to design an angular velocity auxiliary system and a linear velocity auxiliary system respectively; S4. Use the visual camera to capture Kinect images, and estimate the position and posture of the navigator through the Kinect images. According to the position and posture of the navigator and in combination with the state of the angular velocity auxiliary system, the state of the linear velocity auxiliary system and the dipole vector field, formation control is realized without communication.

2. The mobile robot pilot-follower formation control method according to claim 1, characterized in that: The S1 is specifically: A Kinect camera is installed on each mobile robot other than the first mobile robot in the multi-robot system V to estimate the posture of the local navigator; the posture of the local navigator includes a relative direction and a relative position; wherein the multi-robot system V = {V1, ...V i ....V j ..., V N } includes multiple mobile robots, and the multiple mobile robots are divided into multiple echelons; N represents the total number of mobile robots; When the number of echelons is two, the mobile robot in the first echelon is the local navigator, that is, the navigator V j , the next echelon is the follower V i ; When the number of echelons is three or more, the mobile robot in the first echelon is the global navigator. Starting from the second echelon, the next echelon and the previous echelon are followers Vi and local navigators respectively. The global navigator and the local navigator are collectively referred to as the navigator V. j .

3. The mobile robot pilot-follower formation control method according to claim 2, characterized in that: The robot equipped with a Kinect camera has a limited visual detection range. The specific visibility constraints are as follows: Among them, γ i Indicates follower V i The azimuth in the local navigator coordinate system J, where the local navigator coordinate system J is a coordinate system established with the local navigator as the origin; r i Indicates follower V i The relative distance to the local navigator; γ max Indicates the maximum azimuth angle; r max is the equilateral length of the viewing cone, ||.|| represents the Euclidean vector norm; In the above formula, r i The following relationship is satisfied: Among them, x i and i Followers V i The x-axis coordinate value and y-axis coordinate value in the local navigator coordinate system J.

4. The mobile robot pilot-follower formation control method according to claim 3, characterized in that: The S2 specifically includes the following steps: S21. Establish a dipole vector field for each follower Vi. The expression of the dipole vector field is as follows: Among them, ε i =[ε xi ε yi ] T Is a follower V i The formation error vector relative to the local leader coordinate system J is: xi and ε yi Respectively represent the followers V i The components of the formation error vector on the x-axis and y-axis relative to the local leader coordinate system J; F i (ε i ) indicates the follower V i The dipole vector field of T , p≠0; T represents the transpose of the matrix; S22. For each follower V i An initial controller is established, and then the desired formation with respect to the local leader coordinate system J is achieved based on the dipole vector field and with the help of the initial controller.

5. The mobile robot pilot-follower formation control method according to claim 4, characterized in that: The follower V i The initial controller u 0i =[v 0i ω 0i ] T :v 0i and ω 0i Respectively represent the followers V i The linear velocity and angular velocity of v 0i and ω 0i The calculation formula is as follows: Among them, k1 and k2 are two positive scalars; β i Indicates follower V i With Navigator V j The direction difference relative to the global coordinate system G, which is a coordinate system with a fixed point in the space where the mobile robot is located as reference; v max Indicates the maximum linear speed of the mobile robot; express The derivative of represents the dipole vector field at point (x i ,y i ) direction; Direction difference β i The following relationship is satisfied: b i =θ i -θ j =ω i -oh j Among them, θ i Indicates follower V i Relative to the direction of the global coordinate system G, ω i Indicates follower V i Angular velocity; ω j Indicates Navigator V j Angular velocity; The following relationship is satisfied: Among them, F yi , F xi Followers V i The components of the dipole vector field on the y-axis and x-axis; and F yi , F xi They satisfy the following relationships respectively: F yi =3e xi e yi 6. The mobile robot pilot-follower formation control method according to claim 5, characterized in that: The S3 specifically includes the following steps: S31, defining the visibility constraints of the Kinect camera in the actual motion scene, the visibility constraints of the Kinect camera include azimuth constraints and distance constraints; in order to ensure the effectiveness of the follower Vi under the visual constraints, the above visibility constraints are converted into actual control input constraints; specifically, the azimuth constraints are converted into angular velocity input constraints, and the distance constraints are converted into linear velocity input constraints; S32, constructing the angular velocity input saturation function S(ω) according to the angular velocity input constraint and the linear velocity input constraint. i ) and the linear velocity input saturation function S(v i ) to avoid saturation of mobile robot input; S33, according to the angular velocity input saturation function S(ω i )Define the follower V i The angular velocity difference Δω i , Δω i =S(ω i )-ω i , then by Δω i =S(ω i )-ω i Get the control scheme of angular velocity; Similarly, according to the linear velocity input saturation function S(v i )Define the follower V i Linear velocity difference Δv i , Δv i =S(v i )-v i , then by Δv i =S(v i )-v i Get the control scheme of line speed; The control scheme of angular velocity and the control scheme of linear velocity constitute the optimized controller; S34, then designing an angular velocity auxiliary system and a linear velocity auxiliary system respectively through the optimized controller, that is, the angular velocity control scheme and the linear velocity control scheme, and adjusting the angular velocity auxiliary system and the linear velocity auxiliary system so that the formation error gradually approaches zero; The angular velocity auxiliary system is specifically as follows: Among them, Δα i Indicates the angle difference between the current angle and the target angle, and χ1 indicates a very small positive value set; The line speed auxiliary system is specifically as follows: Among them, k6 represents the sixth positive scalar, and χ2 represents another set extremely small positive value.

7. The mobile robot pilot-follower formation control method according to claim 6, characterized in that: The azimuth constraint is expressed by a formula, which is as follows: -c max ≤γ i ≤γ max The distance constraint is expressed by a formula, which is as follows: in, represents the maximum allowed distance between the follower Vi and the target position, and The angular velocity input constraint is expressed by a formula, which is as follows: Among them, k3 is the third positive scalar; Indicates the lower limit of the angle range, Indicates the upper limit of the angle range; The linear velocity input constraint is expressed by a formula, which is as follows: in, Indicates follower V i The derivative of the maximum allowed distance from the target position, k5 represents the fifth positive scalar; α i Indicates follower V i the polar angle of the current position relative to the target position, and 8. The mobile robot pilot-follower formation control method according to claim 7, characterized in that: The angular velocity input saturation function in S32 is specifically as follows: Among them, S(ω i ) represents the angular velocity input saturation function; represents the lower limit of angular velocity, Indicates the upper limit of angular velocity; and The following relationship is satisfied: in, Indicates the angle offset; The linear velocity input saturation function in S32 is specifically as follows: in, Indicates the lower limit of line speed, The following relationship is satisfied:

9. The mobile robot pilot-follower formation control method according to claim 8, characterized in that: The control scheme of the angular velocity in S33 is as follows: Among them, 1i Indicates the status of the angular velocity assist system; The control scheme of the line speed is as follows: Among them, 2i Indicates the status of the Line Speed ​​Assist system.

10. A vision-based mobile robot pilot-follower formation system, comprising a multi-robot system V, wherein a plurality of mobile robots in the multi-robot system V are subjected to formation control according to the mobile robot pilot-follower formation control method according to any one of claims 1 to 9.

Images