Visual control system of electric climbing robot

Abstract

The invention discloses an electric power climbing robot visual control system, which comprises an image acquisition module, an image processing module, a collision prediction module and a control module, and is characterized in that the image acquisition module is used for acquiring an environment image in real time, and the image processing module is used for performing stabilization processing on the acquired image; then foreground segmentation, optical flow analysis and motion detection are carried out, an obstacle moving track is extracted, the collision prediction module is used for judging whether the robot will collide with an obstacle according to the robot moving track and the obstacle moving state, and the control module is used for determining whether the robot executes an obstacle avoidance strategy according to the judgment result of the collision prediction module; the method has the advantages that accurate detection of foreground obstacles is guaranteed by stabilizing the images, background modeling and an optical flow method are combined, the detection and analysis capability of dynamic obstacles is improved, collision prediction is performed based on the real-time updated obstacle motion state and the robot track, and the effectiveness of obstacle avoidance decision making is improved.

Description

Technical Field

[0001] The invention relates to the technical field of electric climbing robots, and in particular to a visual control system of an electric climbing robot. Background Art

[0002] With the development of robotics technology, especially in the field of power climbing robots, the application demand is increasing. These robots need to operate safely and efficiently in complex environments. Power climbing robots are usually used in scenes such as high-voltage wires and communication towers to perform inspection, maintenance and other tasks. In these application scenarios, the robot must be able to automatically avoid obstacles under various environmental conditions to ensure its safe operation and efficient work.

[0003] At present, many existing robot vision control systems mainly focus on the detection and avoidance of static obstacles. These systems usually rely on static image data of the environment and perform collision prediction and path planning by identifying fixed obstacles. The detection of static obstacles is relatively simple because their positions do not change, and the system can make collision predictions more accurately. However, for dynamic obstacles, the effect of this method is significantly reduced. The position and speed of dynamic obstacles are constantly changing during movement, which makes it difficult for traditional prediction technologies based on static obstacles to accurately judge the risk of collision. The collision prediction algorithms in the existing technology often show poor performance when dealing with dynamic obstacles and cannot track the movement trajectory of the obstacles in real time. This results in the robot being unable to make effective obstacle avoidance decisions in a timely manner when facing dynamic obstacles, increasing the risk during operation. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] The present invention has been proposed in view of the above-mentioned problems.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a visual control system of an electric climbing robot, comprising an image acquisition module: used for real-time acquisition of environmental images;

[0007] Image processing module: used to stabilize the collected images first, then perform foreground segmentation, optical flow analysis and motion detection, and extract the movement trajectory of obstacles;

[0008] Collision prediction module: used to determine whether the robot will collide with an obstacle based on the robot's movement trajectory and the obstacle's motion state;

[0009] Control module: used to decide whether the robot executes the obstacle avoidance strategy based on the judgment result of the collision prediction module;

[0010] The system operation steps include:

[0011] S1, real-time collection of environmental images;

[0012] S2, stabilizing the acquired images to eliminate the image displacement caused by the robot movement;

[0013] S3, combining background modeling and optical flow method, identifies and analyzes the motion state of foreground obstacles and extracts the movement trajectory of obstacles;

[0014] S4, judging whether the robot will collide with the obstacle according to the robot movement trajectory and the obstacle movement trajectory;

[0015] S5: decide whether to execute the obstacle avoidance strategy based on the collision judgment result.

[0016] As a preferred solution of the visual control system of an electric climbing robot of the present invention, S2 specifically includes:

[0017] S21, using the ORB algorithm to detect feature points in adjacent frame images;

[0018] S22, use brute force matcher to perform feature point matching;

[0019] S23, extracting the positions of the matching points and calculating the affine transformation matrix;

[0020] S24, performing geometric transformation on the image according to the affine transformation matrix, adjusting the position, angle and scale of the image, and eliminating the image displacement caused by the robot movement.

[0021] As a preferred solution of the visual control system of an electric climbing robot of the present invention, S3 specifically includes:

[0022] S31, initializing and applying a background modeler: using a Gaussian mixture model to establish a background model, applying the current frame image to the background modeler, and generating a foreground mask;

[0023] S32, binarization processing: binarizing the foreground mask to highlight the foreground area;

[0024] S33, calculating optical flow: using an optical flow algorithm to calculate optical flow in continuous image frames to obtain a motion vector for each pixel;

[0025] S34, apply mask: apply the foreground mask to the optical flow result to filter out background noise;

[0026] S35, extracting motion vector: extracting the motion vector of the foreground object from the optical flow calculation;

[0027] S36, trajectory estimation: estimate the motion trajectory of the foreground object in the image according to the motion vector.

[0028] As a preferred solution of the visual control system of an electric climbing robot of the present invention, S4 specifically includes:

[0029] S41, judging whether the obstacle's motion trajectory intersects with the robot's motion trajectory according to the obstacle's moving direction;

[0030] S42, calculating the time taken for the obstacle to reach the intersection of the trajectories;

[0031] S43, calculating the time taken by the robot to reach the intersection of the trajectories;

[0032] S44, judging whether the robot will collide with the obstacle according to the time when the obstacle and the robot reach the intersection of the trajectory.

[0033] As a preferred solution of the visual control system of an electric climbing robot of the present invention, the calculation formula of the time taken for the obstacle to reach the intersection of the track is:

[0034]

[0035] Among them, (x i ,y i ) is the coordinate of the trajectory intersection point, (x o ,y o ) is the current position coordinate of the obstacle, (v x1 ,v y1 ) is the velocity vector of the obstacle.

[0036] As a preferred solution of the visual control system of an electric climbing robot of the present invention, the calculation formula of the time taken by the robot to reach the intersection of the trajectories is:

[0037]

[0038] Among them, (x i ,y i ) is the coordinate of the trajectory intersection point, (x n ,y n ) is the current position coordinate of the obstacle, (v x2 ,v y2 ) is the robot's motion velocity vector.

[0039] As a preferred solution of a visual control system of an electric climbing robot of the present invention, the method for judging whether the robot will collide with an obstacle in S44 is: if t1-T≤t2≤t1+T, then it is judged that the robot will collide with the obstacle; otherwise, it is judged that the robot will not collide with the obstacle, wherein T is an error value for preventing the robot from colliding with the obstacle when the obstacle passes through the intersection.

[0040] As a preferred solution of the visual control system of an electric climbing robot of the present invention, the collision judgment result in S5 includes whether a collision will occur and the time and location where the collision may occur.

[0041] As a preferred solution of the visual control system of an electric climbing robot of the present invention, the obstacle avoidance strategy includes: if a collision is about to occur, the robot stops moving immediately; if there is enough time, the robot's moving speed is adjusted.

[0042] The beneficial effects of the present invention are as follows: by stabilizing the image, the image displacement caused by the robot movement is eliminated, and the accurate detection of foreground obstacles is ensured. In combination with background modeling and optical flow method, the motion state of the foreground obstacle can be accurately identified and analyzed, and the precise motion trajectory can be extracted, thereby improving the detection and analysis capabilities of dynamic obstacles. Collision prediction is performed based on the real-time updated obstacle motion state and robot trajectory, providing higher real-time performance and accuracy, and improving the effectiveness of obstacle avoidance decisions. It is intelligently determined whether to execute the obstacle avoidance strategy based on the collision prediction result, and can flexibly adapt to dynamic environmental changes, thereby optimizing the robot's path planning and obstacle avoidance decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0044] Figure 1 This is an architectural diagram of a visual control system of an electric climbing robot according to the present invention.

[0045] Figure 2 The present invention is an operation flow chart of a visual control system of an electric climbing robot. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0047] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0048] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0049] Secondly, the present invention is described in detail with reference to the schematic diagram. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0050] Example 1

[0051] A visual control system for an electric climbing robot, comprising:

[0052] Image acquisition module: used to collect environmental images in real time;

[0053] Image processing module: used to stabilize the collected images first, then perform foreground segmentation, optical flow analysis and motion detection, and extract the movement trajectory of obstacles;

[0054] Collision prediction module: used to determine whether the robot will collide with an obstacle based on the robot's movement trajectory and the obstacle's motion state;

[0055] Control module: used to decide whether the robot executes the obstacle avoidance strategy based on the judgment result of the collision prediction module;

[0056] The system operation steps include:

[0057] S1, real-time collection of environmental images.

[0058] S2, performs stabilization processing on the acquired images to eliminate the image displacement caused by the robot movement.

[0059] Specifically, S2 includes:

[0060] S21, using the ORB algorithm to detect feature points in adjacent frame images;

[0061] S22, use brute force matcher to perform feature point matching;

[0062] S23, extracting the positions of the matching points and calculating the affine transformation matrix;

[0063] S24, performing geometric transformation on the image according to the affine transformation matrix, adjusting the position, angle and scale of the image, and eliminating the image displacement caused by the robot movement.

[0064] S3, combines background modeling and optical flow method to identify and analyze the motion state of foreground obstacles and extract the movement trajectory of obstacles.

[0065] Specifically, S3 includes:

[0066] S31, initializing and applying a background modeler: using a Gaussian mixture model to establish a background model, applying the current frame image to the background modeler, and generating a foreground mask;

[0067] S32, binarization processing: binarizing the foreground mask to highlight the foreground area;

[0068] S33, calculating optical flow: using an optical flow algorithm to calculate optical flow in continuous image frames to obtain a motion vector for each pixel;

[0069] S34, apply mask: apply the foreground mask to the optical flow result to filter out background noise;

[0070] S35, extracting motion vector: extracting the motion vector of the foreground object from the optical flow calculation;

[0071] S36, trajectory estimation: estimate the motion trajectory of the foreground object in the image according to the motion vector.

[0072] S4, judging whether the robot will collide with the obstacle according to the robot movement trajectory and the obstacle movement trajectory.

[0073] S5: decide whether to execute the obstacle avoidance strategy based on the collision judgment result.

[0074] Specifically, the collision judgment result in S5 includes whether a collision will occur and the time and location where the collision may occur. The obstacle avoidance strategy includes: if a collision is about to occur, the robot stops moving immediately; if there is enough time, the robot's movement speed is adjusted.

[0075] Example 2

[0076] A visual control system for an electric climbing robot, based on Embodiment 1, S4 specifically includes:

[0077] S41, judging whether the obstacle's moving trajectory intersects with the robot's moving trajectory according to the obstacle's moving direction.

[0078] S42, calculating the time taken for the obstacle to reach the intersection of the trajectories. The calculation formula for the time taken for the obstacle to reach the intersection of the trajectories is:

[0079]

[0080] Among them, (x i ,y i ) is the coordinate of the trajectory intersection point, (x o ,y o ) is the current position coordinate of the obstacle, (v x1 ,v y1 ) is the velocity vector of the obstacle.

[0081] S43, calculating the time taken by the robot to reach the intersection of the trajectories. The calculation formula for the time taken by the robot to reach the intersection of the trajectories is:

[0082]

[0083] Among them, (x i ,y i ) is the coordinate of the trajectory intersection point, (x n ,y n ) is the current position coordinate of the obstacle, (v x2 ,v y2 ) is the robot's motion velocity vector.

[0084] S44, judging whether the robot will collide with the obstacle according to the time when the obstacle and the robot reach the intersection of the trajectory.

[0085] Specifically, the method for judging whether the robot will collide with the obstacle in S44 is: if t1-T≤t2≤t1+T, then it is judged that the robot will collide with the obstacle; otherwise, it is judged that the robot will not collide with the obstacle, wherein T is an error value for preventing the robot from colliding with the obstacle when the obstacle passes through the intersection.

[0086] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A visual control system for an electric climbing robot, characterized in that: include: Image acquisition module: used to collect environmental images in real time; Image processing module: used to stabilize the collected images first, then perform foreground segmentation, optical flow analysis and motion detection, and extract the movement trajectory of obstacles; Collision prediction module: used to determine whether the robot will collide with an obstacle based on the robot's movement trajectory and the obstacle's motion state; Control module: used to decide whether the robot executes the obstacle avoidance strategy based on the judgment result of the collision prediction module; The system operation steps include: S1, real-time collection of environmental images; S2, stabilizing the acquired images to eliminate the image displacement caused by the robot movement; S3, combining background modeling and optical flow method, identifies and analyzes the motion state of foreground obstacles and extracts the movement trajectory of obstacles; S4, judging whether the robot will collide with the obstacle according to the robot movement trajectory and the obstacle movement trajectory; S5: decide whether to execute the obstacle avoidance strategy based on the collision judgment result.

2. The electric climbing robot visual control system according to claim 1, characterized in that: S2 specifically includes: S21, using the ORB algorithm to detect feature points in adjacent frame images; S22, use brute force matcher to perform feature point matching; S23, extracting the positions of the matching points and calculating the affine transformation matrix; S24, performing geometric transformation on the image according to the affine transformation matrix, adjusting the position, angle and scale of the image, and eliminating the image displacement caused by the robot movement.

3. The electric climbing robot visual control system according to claim 2, characterized in that: S3 specifically includes: S31, initializing and applying a background modeler: using a Gaussian mixture model to establish a background model, applying the current frame image to the background modeler, and generating a foreground mask; S32, binarization processing: binarizing the foreground mask to highlight the foreground area; S33, calculating optical flow: using an optical flow algorithm to calculate optical flow in continuous image frames to obtain a motion vector for each pixel; S34, apply mask: apply the foreground mask to the optical flow result to filter out background noise; S35, extracting motion vector: extracting the motion vector of the foreground object from the optical flow calculation; S36, trajectory estimation: estimate the motion trajectory of the foreground object in the image according to the motion vector.

4. The electric climbing robot visual control system according to claim 3, characterized in that: S4 specifically includes: S41, judging whether the obstacle's motion trajectory intersects with the robot's motion trajectory according to the obstacle's moving direction; S42, calculating the time taken for the obstacle to reach the intersection of the trajectories; S43, calculating the time taken by the robot to reach the intersection of the trajectories; S44, judging whether the robot will collide with the obstacle according to the time when the obstacle and the robot reach the intersection of the trajectory.

5. The electric climbing robot visual control system according to claim 4, characterized in that: The calculation formula for the time it takes for the obstacle to reach the intersection of the trajectories is: Among them, (x i ,y i ) is the coordinate of the trajectory intersection point, (x o ,y o ) is the current position coordinate of the obstacle, (v x1 ,v y1 ) is the velocity vector of the obstacle.

6. The electric climbing robot visual control system according to claim 5, characterized in that: The calculation formula for the time it takes the robot to reach the intersection of the trajectories is: Among them, (x i ,y i ) is the coordinate of the trajectory intersection point, (x n ,y n ) is the current position coordinate of the obstacle, (v x2 ,v y2 ) is the robot's motion velocity vector.

7. The electric climbing robot visual control system according to claim 6, characterized in that: The method for judging whether the robot will collide with the obstacle in S44 is: if t1-T≤t2≤t1+T, then it is judged that the robot will collide with the obstacle; otherwise, it is judged that the robot will not collide with the obstacle, wherein T is an error value for preventing the robot from colliding with the obstacle when the obstacle passes through the intersection.

8. The electric climbing robot visual control system according to claim 7, characterized in that: The collision judgment result in S5 includes whether a collision will occur and the time and location where the collision may occur.

9. The electric climbing robot visual control system according to claim 8, characterized in that: The obstacle avoidance strategy includes: if a collision is about to occur, the robot stops moving immediately; if there is enough time, Then adjust the robot's moving speed.

Images