Inspection robot navigation method, system and equipment based on Beidou and binocular vision fusion and storage medium

Abstract

The invention discloses an inspection robot navigation method, system and device based on Beidou and binocular vision fusion and a storage medium, and relates to the field of robot navigation and obstacle avoidance, and the method comprises the following steps: carrying out multi-modal data fusion processing based on an initial coordinate of an inspection robot and an image captured by a binocular vision system to obtain an environment sensing result; carrying out global planning and local optimization based on an environment perception result, and carrying out cooperation to generate a trajectory planning scheme; based on the environment sensing result and the trajectory planning scheme, intelligent navigation cooperative regulation and control are carried out, and an inspection robot motion control strategy is obtained; by improving the environmental perception accuracy, optimizing the path planning and performing intelligent navigation regulation and control, the navigation problem of the inspection robot in complex environments such as a transformer substation can be effectively solved, the inspection safety and efficiency are improved, and the method has remarkable practical application value.

Description

Technical Field

[0001] This invention relates to the field of robot navigation and obstacle avoidance, and in particular to a navigation method, system, device and storage medium for inspection robots based on the fusion of Beidou and binocular vision. Background Technology

[0002] With the rapid development of smart grids and automation technologies, substation inspection robots have gradually replaced manual inspections and are widely used in the daily inspection and maintenance of power equipment. As key functions, autonomous navigation and obstacle avoidance technologies directly affect the robot's task completion efficiency in complex environments. However, the complex obstacle distribution, strong electromagnetic interference, and the obstruction of navigation signals by metal structures within substations pose significant challenges to the accurate positioning and efficient obstacle avoidance of inspection robots. In particular, traditional navigation technologies often exhibit insufficient accuracy and low efficiency when dealing with the dynamic environment and multi-source interference of substations, requiring further improvement.

[0003] Currently, several classic methods have been proposed for the navigation and obstacle avoidance problems of inspection robots. Among them, the binocular vision path planning method achieves path planning and obstacle avoidance capabilities for automated guided vehicles (AGVs) through binocular vision depth calculation and obstacle recognition, exhibiting high reliability, especially in static environments. However, this method relies too heavily on binocular vision, resulting in insufficient real-time performance in dynamic and complex environments. The vector field histogram navigation method acquires environmental depth information through a binocular vision system and combines it with a vector field histogram algorithm to optimize the robot's obstacle avoidance path, solving the obstacle avoidance problem for dynamic obstacles to some extent. However, this method suffers from local optima in path planning, and its obstacle recognition accuracy is heavily dependent on visual information, making it difficult to cope with environmental constraints such as signal interference or changes in lighting. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is: how to solve the problems of insufficient accuracy and low efficiency of traditional navigation technology in the complex environment of substations, as well as the poor real-time performance, limited path planning, and weak ability to cope with environmental changes of existing classic methods, so as to improve the navigation efficiency and obstacle avoidance performance of inspection robots.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a navigation method for an inspection robot based on the fusion of BeiDou and binocular vision, comprising:

[0008] Based on the initial coordinates of the inspection robot and the images captured by the binocular vision system, multimodal data fusion processing is performed to obtain the environmental perception results;

[0009] Based on the environmental perception results, global planning and local optimization are performed and coordinated to generate a trajectory planning scheme.

[0010] Based on the environmental perception results and trajectory planning scheme, intelligent navigation and collaborative regulation are carried out to obtain the motion control strategy of the inspection robot.

[0011] As a preferred solution for the navigation method of inspection robots based on the fusion of Beidou and binocular vision, the following is provided:

[0012] The environmental perception results obtained by performing multimodal data fusion processing based on the initial coordinates of the inspection robot and the images captured by the binocular vision system include:

[0013] The robot's initial coordinates in three-dimensional space are output through the BeiDou system. The error introduced by multipath effect is obtained by modeling environmental features. Kalman filtering is applied to dynamically optimize the data during real-time operation. Parallax is calculated using images captured by left and right cameras to determine the distance between obstacles and the robot, thus achieving accurate positioning of obstacles in three-dimensional space. Features of input images are extracted through convolutional neural networks to identify obstacle types and dynamic behaviors, generate three-dimensional point cloud data of the local environment, and construct a local environment map.

[0014] As a preferred solution for the navigation method of inspection robots based on the fusion of Beidou and binocular vision, the following is provided:

[0015] The process of performing multimodal data fusion processing based on the initial coordinates of the inspection robot and the images captured by the binocular vision system to obtain the environmental perception results also includes:

[0016] A weighted fusion-based multimodal information processing model is introduced to combine the global positioning of BeiDou navigation with the local perception of binocular vision. The location information is optimized by weighted fusion, the priority of BeiDou and visual data is dynamically adjusted, and the fused location information is further optimized by a filter to smooth the final estimate and obtain the environmental perception result.

[0017] The beneficial effects of this preferred technical solution are as follows: the weighted fusion multimodal information processing model can give full play to the global positioning advantages of Beidou navigation and the local perception advantages of binocular vision, and dynamically adjust the priority of the two data according to different situations, so that the fused position information is more accurate and reliable. The application of filters further smooths the estimated value and improves the stability of environmental perception.

[0018] As a preferred solution for the navigation method of inspection robots based on the fusion of Beidou and binocular vision, the following is provided:

[0019] The process of generating a trajectory planning scheme based on environmental perception results, involving global planning and local optimization in a coordinated manner, includes:

[0020] A global path search is performed by calculating the total cost function of each node. The actual path cost from the starting point to the current node and the heuristic estimate from the current node to the target point are considered together, and nodes closer to the target point are selected first. The global path is then optimized by the robot acquiring state information from the environment, exploring the optimal path through trial actions, receiving rewards based on the current state and the selected actions, and updating the value function of future paths to obtain the optimal global path planning strategy.

[0021] As a preferred solution for the navigation method of inspection robots based on the fusion of Beidou and binocular vision, the following is provided:

[0022] The process of generating a trajectory planning scheme based on environmental perception results, involving global planning and local optimization in a coordinated manner, also includes:

[0023] A deep learning-based motion prediction model is used to predict the future trajectory of dynamic obstacles, calculate their future positions in real time, and adjust the local path accordingly.

[0024] As a preferred solution for the navigation method of inspection robots based on the fusion of Beidou and binocular vision, the following is provided:

[0025] The process of generating a trajectory planning scheme based on environmental perception results, involving global planning and local optimization in a coordinated manner, also includes:

[0026] The system achieves coordinated optimization of global path planning and local path optimization through weighting factors, and dynamically adjusts the weighting factors according to the complexity of the environment.

[0027] The beneficial effects of this preferred technical solution are as follows: dynamically adjusting the weight factors of global path planning and local path optimization according to the complexity of the environment enables the robot to flexibly balance global goals and local obstacle avoidance requirements in different environments. In complex environments, local obstacle avoidance is prioritized, and in open environments, global planning is prioritized, thereby improving the adaptability and flexibility of path planning.

[0028] As a preferred solution for the navigation method of inspection robots based on the fusion of Beidou and binocular vision, the following is provided:

[0029] The intelligent navigation and collaborative control strategy for the inspection robot, based on environmental perception results and trajectory planning schemes, includes the following:

[0030] By combining visual perception data from environmental perception results with trajectory planning schemes, obstacle movement is predicted and avoided in real time. The position, velocity, and acceleration of obstacles are monitored in real time, and the robot's movement path is adjusted accordingly. A safe zone is constructed on the predicted trajectory of dynamic obstacles. When the robot is about to enter the safe zone, the obstacle avoidance mechanism is triggered. An obstacle avoidance path is generated through an optimization algorithm, taking into account both distance cost and path smoothness cost. The priority of global and local paths is dynamically adjusted according to the complexity of the environment.

[0031] The beneficial effects of this preferred technical solution are as follows: dynamic obstacle avoidance by integrating visual perception data and trajectory planning schemes enables timely and accurate responses to changes in obstacles; the construction of safe zones and the triggering of obstacle avoidance mechanisms ensure that the robot can react promptly when encountering obstacles; the generation of obstacle avoidance paths by comprehensively considering distance costs and path smoothness costs makes the robot's obstacle avoidance actions more reasonable and efficient; and the dynamic adjustment of the priority of global and local paths according to environmental complexity ensures that the robot can complete navigation tasks efficiently and safely in different environments.

[0032] Secondly, the present invention provides a navigation system for an inspection robot based on the fusion of BeiDou and binocular vision, comprising:

[0033] The multimodal data fusion module is used to perform multimodal data fusion processing based on the initial coordinates of the inspection robot and the images captured by the binocular vision system to obtain environmental perception results;

[0034] The trajectory planning module is used to perform global planning and local optimization based on environmental perception results, and to coordinate these processes to generate a trajectory planning scheme.

[0035] The inspection robot motion control module is used to carry out intelligent navigation and collaborative regulation based on environmental perception results and trajectory planning schemes, so as to obtain the inspection robot motion control strategy.

[0036] Thirdly, the present invention provides an electronic device, comprising:

[0037] Memory and processor;

[0038] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the one or more programs are executed by the one or more processors, the one or more processors implement the inspection robot navigation method based on Beidou and binocular vision fusion as described in this invention.

[0039] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the aforementioned navigation method for an inspection robot based on the fusion of BeiDou and binocular vision.

[0040] The beneficial effects of this invention are as follows: In complex environments such as substations, there are numerous electrical devices, lines, and potential temporary obstacles. This invention outputs the robot's initial coordinates in three-dimensional space through the BeiDou system and utilizes environmental feature modeling and Kalman filtering optimization to effectively reduce errors introduced by multipath effects and improve BeiDou positioning accuracy. Simultaneously, the binocular vision system uses images captured by left and right cameras to calculate parallax, accurately determining the distance between obstacles and the robot. Combined with convolutional neural networks to identify obstacle types and dynamic behaviors, a local environmental map is constructed. During inspections, it can quickly identify suddenly appearing obstacles such as construction tools and temporarily piled-up debris, providing a reliable basis for the robot to adjust its path in a timely manner and avoid collisions, greatly improving the safety of the inspection process. This invention also introduces a weighted fusion-based multimodal information processing model, combining the global positioning of BeiDou navigation with the local perception of binocular vision. Within the large area of ​​a substation, the BeiDou Navigation Satellite System provides global location information, helping robots determine their approximate location within the substation. Meanwhile, the binocular vision system enables detailed perception of the surrounding environment in localized areas. When a robot approaches electrical equipment for inspection, binocular vision clearly captures details such as the equipment's appearance and connecting lines. Combined with BeiDou positioning information, it accurately determines the equipment's specific location, providing comprehensive and accurate environmental information for subsequent inspection operations. During global path search, by calculating the total cost function of each node, and comprehensively considering the actual path cost from the starting point to the current node and the heuristic estimate from the current node to the target point, nodes closer to the target point are prioritized. This allows the robot to quickly plan a relatively optimal global path in large-scale substation inspection tasks, reducing unnecessary detours and time waste. Furthermore, by using reinforcement learning to optimize the global path, the robot can acquire state information from the environment and continuously explore the optimal path through trial actions. Employing a deep learning-based motion prediction model, it can predict the future trajectory of dynamic obstacles, calculate their future positions in real time, and adjust local paths. Within a substation, there may be dynamic obstacles such as workers and vehicles. The robot can predict the movement direction and speed of these dynamic obstacles in advance, adjust its local path in time, avoid collisions, and ensure the continuity and efficiency of the inspection process. Simultaneously, it achieves coordinated optimization of global path planning and local path optimization through weighting factors, dynamically adjusting the weighting factors according to environmental complexity. In open areas with few obstacles, the priority of the global path is increased, enabling the robot to quickly reach the target point; in areas with dense obstacles, the priority of local path optimization is increased, ensuring the robot can flexibly avoid obstacles and successfully complete the inspection task. Based on environmental perception results and trajectory planning schemes, combined with visual perception data, the robot performs real-time prediction and avoidance of obstacle movement.A safe zone is constructed on the predicted trajectory of dynamic obstacles. When the robot is about to enter the safe zone, the obstacle avoidance mechanism is triggered. An obstacle avoidance path is generated by an optimization algorithm, taking into account both distance cost and path smoothness cost. The priority of global and local paths is dynamically adjusted according to the environmental complexity. In the daily inspection of substations, the environmental complexity of different areas is different. This dynamic adjustment mechanism enables the robot to better adapt to the complex and ever-changing environment, improving the robot's environmental adaptability and inspection efficiency. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is an overall flowchart of the inspection robot navigation method based on the fusion of Beidou and binocular vision provided by the present invention;

[0043] Figure 2 This is a schematic diagram of obstacle recognition and classification in the inspection robot navigation method based on the fusion of Beidou and binocular vision provided by the present invention;

[0044] Figure 3 This is a schematic diagram of the deep reinforcement learning (DRL) technology used in the inspection robot navigation method based on the fusion of Beidou and binocular vision provided by this invention. Detailed Implementation

[0045] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0046] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a navigation method for an inspection robot based on the fusion of BeiDou and binocular vision, including:

[0047] S1: Based on the initial coordinates of the inspection robot and the images captured by the binocular vision system, multimodal data fusion processing is performed to obtain the environmental perception results;

[0048] S2: Based on the environmental perception results, perform global planning and local optimization and coordinate them to generate a trajectory planning scheme;

[0049] S3: Based on the environmental perception results and trajectory planning scheme, intelligent navigation collaborative regulation is carried out to obtain the motion control strategy of the inspection robot.

[0050] It should be noted that through steps S1-S3, the Beidou high-precision navigation is combined with binocular artificial intelligence visual perception technology. Multimodal information fusion solves the limitations of single perception technology in complex dynamic environments. By using deep learning-driven trajectory planning and dynamic obstacle avoidance strategies, the collaborative work of global path planning and local path optimization is realized, which effectively improves the robot's navigation accuracy, obstacle avoidance efficiency and dynamic adaptability in complex substation environments.

[0051] Example 2, refer to Figures 1-3 As one embodiment of the present invention, based on the previous embodiment, a navigation method for an inspection robot based on the fusion of BeiDou and binocular vision is provided, including:

[0052] In this embodiment, the environmental perception results obtained in step S1 above, based on the initial coordinates of the inspection robot and the images captured by the binocular vision system, include the following:

[0053] The location information provided by the BeiDou system is optimized through error compensation and filtering algorithms;

[0054] Specifically, the initial coordinates (x, y, y) of the robot in three-dimensional space are output through the BeiDou system. B y B , z B Since signal interference may introduce errors, the data is corrected as follows:

[0055] x B ′=x B +Δx, y B ′=y B +Δy,z B ′=z B +Δz

[0056] Where Δx, Δy, and Δz represent the error introduced by multipath effects, obtained through environmental feature modeling, and dynamically optimized using Kalman filtering during real-time operation. Kalman filtering is used to smooth BeiDou navigation data, and the formula is as follows:

[0057]

[0058] in, Let z be the estimated position at the current time k. k This is the noise data currently being observed. Here, H represents the estimated value from the previous time step, and K represents the measurement matrix. k The filter gain, which controls the weight ratio between observed and predicted values, is defined as:

[0059]

[0060] Among them, P k-1 To estimate the error covariance, R is the measurement noise covariance.

[0061] It should be noted that the role of Kalman filtering is to iteratively optimize and adjust the interference-affected positioning information of the BeiDou system to a more accurate estimate, thereby providing a stable and reliable global positioning capability. Through the above process, the BeiDou system provides the robot with precise global position information, enabling it to determine its spatial position relative to the entire substation environment. This provides an important reference coordinate system for path planning and environmental perception.

[0062] In another possible implementation, particle filtering can be used instead of Kalman filtering when processing BeiDou data. Particle filtering is a filtering algorithm based on the Monte Carlo method, which has good performance in handling nonlinear and non-Gaussian systems. In BeiDou positioning, errors caused by factors such as signal interference may be nonlinear, and particle filtering can more accurately estimate the positioning information of the BeiDou system affected by interference.

[0063] Specifically, based on the BeiDou system's observation and motion models, a set of random particles is generated, each representing a possible position estimate. Then, based on the current observations, the weight of each particle is calculated; the weight indicates the degree of match between the particle and the observation. Next, through a resampling process, particles with smaller weights are removed, and particles with larger weights are duplicated, resulting in a new set of particles. Finally, the position estimate for the current moment is calculated based on the new particle set.

[0064] The core of a binocular vision system lies in using left and right cameras to simulate the visual function of human eyes. It obtains depth information of obstacles in the environment through parallax calculation and combines this with artificial intelligence technology to classify and identify obstacles. This part mainly solves the problem of dynamic perception for robots in local environments, especially in areas where BeiDou signals may be blocked. Binocular vision can independently complete detailed modeling of the surrounding environment. Binocular vision relies on images captured by the left and right cameras and determines the distance between obstacles and the robot through parallax calculation.

[0065] Specifically, let the horizontal pixel coordinates of the obstacle on the imaging planes of the left and right cameras be x. L and x R The horizontal pixel offset (parallax) between the two is d = |x L -x R According to the binocular ranging formula, the distance to the obstacle is expressed as:

[0066]

[0067] Where Z represents the distance from the obstacle to the camera; f is the focal length of the camera, which determines the image size; is the camera baseline distance, i.e., the actual distance between the optical centers of the two cameras; and d is the parallax value, representing the magnitude of the obstacle's offset in the left and right images. By calculating Z, the robot can accurately locate the position of the obstacle in three-dimensional space.

[0068] Furthermore, in addition to depth information, the binocular system also utilizes artificial intelligence algorithms to analyze the features of objects in the environment, identifying obstacle types and dynamic behaviors. The input image is processed by a convolutional neural network (CNN) to extract features, and the output is the probability that each target is an obstacle.

[0069] P(O) = Softmax(W·F + b)

[0070] Where P(O) represents the probability that the target is an obstacle, W is the network weight matrix, F is the feature vector of the input image, and b is the bias term.

[0071] Through model training, the system can accurately identify different types of obstacles (such as static obstacles and dynamic targets), providing important references for obstacle avoidance strategies, such as... Figure 2 The diagram shown is a schematic of obstacle recognition and classification.

[0072] By utilizing depth calculations and obstacle recognition results from binocular vision, 3D point cloud data of the local environment can be generated. This point cloud data is used to construct a local environment map, and the point cloud formula is:

[0073] P = {(x, y, Z) | x = Xx} C y = Yy C Z = Z}

[0074] Where P is a point cloud set, X, Y, Z represent the world coordinates of the obstacle, and x... C y C The coordinates of the image center are the coordinates of the camera.

[0075] In another possible implementation, when processing binocular vision data, a stereo matching algorithm can be used to optimize disparity calculation: in addition to traditional disparity calculation methods, a semi-global matching (SGM) algorithm can be used to improve the accuracy and efficiency of disparity calculation.

[0076] Specifically, the SGM algorithm considers both local and global image information by aggregating costs in multiple directions. First, it calculates the matching cost for each pixel in the left and right images. Then, it aggregates these matching costs in multiple directions to obtain the aggregated cost for each pixel. Finally, it selects the disparity value corresponding to the minimum aggregated cost as the disparity of that pixel.

[0077] To comprehensively utilize the global positioning capability of BeiDou navigation and the local perception capability of binocular vision, a multimodal information processing model based on weighted fusion is introduced. By combining data from the two perception modes, more comprehensive and reliable environmental perception results can be generated.

[0078] Specifically, BeiDou provides global position coordinates, while binocular vision provides local obstacle information. The two are then weighted and fused to optimize the position information.

[0079]

[0080] in, For the merged robot position information, and Position data are provided by BeiDou navigation and binocular vision, respectively, with α serving as a weighting factor to dynamically adjust the priority of BeiDou and visual data. For example, in environments with significant signal interference, the value of α is reduced to increase the weight of visual data. The fused position information may still be affected by environmental noise; therefore, a filter is used to further optimize the final position estimate.

[0081]

[0082] This filter can smooth the final estimate based on the temporal changes of BeiDou and visual data, ensuring that the robot's perception of the environment is stable and reliable.

[0083] It should be noted that multimodal environmental perception and information fusion solve the shortcomings of BeiDou single positioning in complex environments, while enhancing the real-time perception capability of binocular vision, providing a precise environmental cognition foundation for subsequent path planning and navigation.

[0084] In this embodiment, step S2 above, based on the environmental perception results, performs global planning and local optimization in a coordinated manner to generate a trajectory planning scheme, including:

[0085] The task of global path planning is to find an optimal path for the robot from the starting point to the target point. This path needs to avoid known obstacles as much as possible and minimize path length or other costs. First, a global path search is performed based on the classic A* algorithm. This algorithm finds the optimal path by calculating the total cost function of the nodes, which is expressed as:

[0086] f(n) = g(n) + h(n)

[0087] Where f(n) is the total cost, representing the comprehensive evaluation value of the current node n; g(n) is the actual path cost from the starting point to the current node n, usually the path length or other physical quantities related to robot navigation; h(n) is a heuristic estimate used to evaluate the cost from the current node n to the target point, usually calculated using Euclidean distance, and its calculation formula is:

[0088]

[0089] Where, x n y n It is the coordinate of the current node n, x t y t These are the coordinates of the target point. This heuristic function ensures that the algorithm prioritizes nodes closer to the target point when searching for a path, thus improving the efficiency of path searching.

[0090] To avoid path planning getting trapped in local optima, deep reinforcement learning (DRL) techniques are introduced to optimize the global path. For example... Figure 3 The diagram shown is a schematic of DRL.

[0091] In a reinforcement learning framework, the robot acquires state information (including position, obstacle distribution, etc.) from the environment and explores the optimal path through trial-and-error actions. In each action, the robot adjusts its actions based on the current state s. t and the chosen action a t Receive reward r t And update the value function of future paths:

[0092]

[0093] Where Q(s) t a t ) represents the current state s t Choose action a t value; r t α is the current reward value, used to measure the quality of the current action; α is the learning rate, which determines the magnitude of value updates; γ is the discount factor, used to balance the weight of current rewards and future rewards.

[0094] Through continuous trial and learning, the robot gradually mastered the optimal path planning strategy from the starting point to the target point in complex environments.

[0095] It should be noted that the introduction of reinforcement learning enables global path planning to dynamically adapt to changes in the environment, avoiding the local optima problem caused by the limitations of static maps in the traditional A* algorithm.

[0096] Global path planning provides the robot with a general direction of motion, but in dynamic environments, real-time local path optimization is crucial. By combining the Bézier curve method with dynamic obstacle prediction technology, we can ensure that the robot can generate smooth and efficient local paths while avoiding dynamic obstacles.

[0097] Specifically, Bézier curves are a mathematical tool for path smoothing. They generate smooth curves by defining control points, avoiding sharp turns or discontinuities in the path. The specific formula is as follows:

[0098] C(t)=(1-t) 2 P0+2(1-t)tP1+t 2 P2, t∈[0,1]

[0099] Where C(t) is the position of the curve at time t; P0 is the starting point, representing the robot's current position; P1 is the control point, used to adjust the shape of the curve to ensure that the path avoids obstacles; and P2 is the target point, representing the position that the robot expects to reach.

[0100] By dynamically adjusting the position of control point P1, Bezier curves can generate smooth paths that avoid obstacles, making robot movement more efficient and safer in local environments.

[0101] In another possible implementation, a local path can be generated using the artificial potential field method: the artificial potential field method is a path planning method based on a virtual force field, which treats obstacles as repulsive sources and target points as attractive sources, and the robot moves under the action of attraction and repulsion.

[0102] Specifically, the algorithm calculates the attractive and repulsive forces acting on the robot at its current position. The attractive force points towards the target point, while the repulsive force is generated by obstacles. The robot's direction of movement is determined by the direction of the resultant force of these forces. By adjusting the magnitude and direction of these forces, the robot can avoid obstacles and move towards the target point. The artificial potential field method is computationally simple and has good real-time performance, but it may encounter local minima, causing the robot to get stuck. Local minima can be addressed by combining this method with other approaches (such as random search).

[0103] In addition, to cope with the interference of dynamic obstacles, a motion prediction model based on deep learning is adopted to predict the future motion trajectory of the obstacles.

[0104] Specifically, let the initial position of the obstacle be P0 and its velocity be v. o The acceleration is a o Then its position at time t is calculated using the following formula:

[0105]

[0106] By calculating the future positions of dynamic obstacles in real time, robots can anticipate potential collision risks and adjust their local paths accordingly. For example, when an obstacle moves from the left to the right, the predictive model calculates the obstacle's future path area and marks these areas as impassable in the local path planning, prompting the robot to choose an alternative, safer path.

[0107] Global path planning and local path optimization are two complementary techniques: the global path provides the robot's overall direction from the starting point to the target point, while the local path adjusts according to real-time environmental changes. Co-optimization of both is achieved through weighting factors, and the final path calculation formula is as follows:

[0108] P opt (t)=β·P global (t)+(1-β)·P local (t)

[0109] Among them, P opt (t) is the robot's final path at time t; P global (t) is the result of global path planning; P local (t) represents the result of local path optimization; β is a dynamically adjusted weighting factor. β changes dynamically depending on the environmental complexity. For example, in areas with fewer obstacles, the global path has a higher weight, while in areas with dense obstacles, the weight of the local path increases. This collaborative optimization mechanism ensures that the robot can balance global objectives and local obstacle avoidance needs, enabling it to complete navigation tasks efficiently and safely even in highly dynamic environments.

[0110] It should be noted that the aforementioned trajectory planning technology effectively solves the problems of low path planning efficiency and insufficient dynamic obstacle avoidance performance of traditional methods in complex environments. The collaborative optimization of global and local methods not only improves the accuracy of robot navigation but also gives it greater adaptability and robustness, providing strong support for efficient inspection in complex substation environments.

[0111] In this embodiment, the intelligent navigation collaborative control strategy for the inspection robot, based on the environmental perception results and trajectory planning scheme in step S3 above, includes:

[0112] After completing global path planning and local path optimization, the robot needs to translate the planned path into specific motion control strategies during actual operation in order to achieve efficient autonomous navigation and dynamic obstacle avoidance.

[0113] By combining global path planning, local path optimization, and dynamic obstacle avoidance prediction through an intelligent collaborative navigation mechanism, the robot's motion state is dynamically adjusted, enabling it to adapt to changes in complex environments and complete navigation tasks.

[0114] Specifically, autonomous navigation control is implemented: the core task of autonomous navigation is to generate control commands in real time based on the planned path and guide the robot to perform specific movements. Through comprehensive path data provided by multimodal environmental perception, combined with intelligent navigation algorithms, the robot's speed and direction are precisely controlled.

[0115] The motion of a robot on a two-dimensional plane is determined by two parameters: velocity and direction. Its kinematic formula is described as follows:

[0116]

[0117] Where v represents the robot's velocity, and θ represents the robot's forward direction angle. and These represent the robot's velocity components in the x and y directions, respectively. Path planning result P opt (t) provides the robot's target position at each moment. The robot dynamically adjusts its speed and direction by calculating the difference between its current position and the target position.

[0118]

[0119] Where, x t y t The coordinates of the target point, x c y c The current coordinates of the robot, k v This is the proportional gain, used to adjust the magnitude of speed changes. While performing navigation, the robot simultaneously monitors changes in the surrounding environment to ensure consistency between path adjustments and execution. For example, when the local path planning updates the target point, the navigation controller recalculates the speed and direction in real time based on the new target position, thereby avoiding path deviation and the accumulation of target errors.

[0120] Setting up a dynamic obstacle avoidance strategy: In complex environments, the dynamic characteristics of obstacles are a significant factor affecting robot navigation efficiency. To address this issue, a dynamic obstacle avoidance strategy is employed, combining visual perception data with a deep learning prediction model to predict and avoid obstacle movements in real time.

[0121] The dynamic behavior of the obstacle is described by kinematic equations, and its position changes over time as follows:

[0122]

[0123] Where, x t y t Let x0 be the predicted position of the obstacle at time t; y0 be the initial position of the obstacle; v x v y a is the velocity component of the obstacle; x a yLet represent the acceleration component of the obstacle. By monitoring the position, velocity, and acceleration of the obstacle in real time, the robot can predict the obstacle's future trajectory and adjust its own path accordingly. To ensure the efficiency of obstacle avoidance, the robot constructs a safe zone on the predicted trajectory of the dynamic obstacle. When the robot is about to enter this zone, the obstacle avoidance mechanism is triggered, adjusting its path based on the obstacle's position and direction of motion. The obstacle avoidance path is generated through an optimization algorithm, whose cost function considers distance cost (avoiding the robot approaching the obstacle) and path smoothness cost (ensuring the robot's smoothness during movement). The optimization formula for the obstacle avoidance path is:

[0124]

[0125] Among them, P robot (i) is the i-th position on the robot's path, P obstacle (i) is the i-th position on the obstacle path, and λ1 and λ2 are weight coefficients used to balance obstacle avoidance and path smoothness.

[0126] In another possible implementation, a method combining reinforcement learning and behavior trees can be introduced: reinforcement learning allows the robot to find the optimal obstacle avoidance strategy through continuous trial and learning, while behavior trees can manage the robot's behavior in a hierarchical manner, improving the maintainability and flexibility of the system.

[0127] Specifically, the robot's obstacle avoidance behavior is abstracted as nodes in a behavior tree, with each node representing a specific behavior (such as turning left, turning right, accelerating, or decelerating). During reinforcement learning, the robot selects a behavior to execute based on the current environmental state and receives a reward or penalty based on the execution result. Through continuous iteration, the robot learns the optimal behavior to choose under different environmental states. The behavior tree can dynamically adjust the priority of behaviors according to the robot's task and environmental conditions, ensuring that the robot can efficiently avoid obstacles.

[0128] Adaptive cooperative navigation: In actual operation, global path planning and local path optimization need to work together to adapt to dynamic changes in the environment. The robot needs to dynamically adjust the priorities of global and local paths at different stages. Through an adaptive cooperative navigation mechanism, the weights of global and local paths are adjusted according to the complexity of the environment.

[0129] Specifically, assuming the obstacle density in the current environment is ρ, the dynamic weight allocation between the global path and the local path is expressed as:

[0130]

[0131] Where β represents the weight of the global path, ρ0 is the threshold for obstacle density, and κ is an adjustment factor. When the obstacle density in the environment is low, the value of β is large, and the priority of the global path is higher; when the obstacle density is high, the value of β decreases, and the priority of the local path increases.

[0132] The final path is calculated using the following formula:

[0133] P final (t)=β·P global (t)+(1-β)·P local (t)

[0134] It should be noted that this collaborative mechanism ensures that the robot can flexibly adjust its navigation strategy in complex environments, prioritizing local obstacle avoidance in areas with dense obstacles, and prioritizing global planning in open areas.

[0135] Real-time environmental feedback and dynamic optimization: During navigation, the robot collects environmental feedback information in real time and optimizes its navigation strategy based on the results. For example, when obstacle avoidance fails and a collision occurs, the system records the current environmental information and uses it as training data for online updates of the reinforcement learning model. Through this mechanism, the robot can continuously adapt to new scenarios and changing environmental conditions, improving the intelligence of its navigation.

[0136] It should be noted that through intelligent cooperative navigation and obstacle avoidance mechanisms, a seamless integration of global path planning and local obstacle avoidance is achieved, enabling the robot to smoothly and efficiently complete autonomous navigation tasks in the dynamic and complex substation environment. This design significantly improves the robustness and safety of robot navigation, while also providing crucial support for the efficient integration of path planning and environmental perception.

[0137] Example 3: The above is an illustrative scheme of the inspection robot navigation method based on BeiDou and binocular vision fusion in this embodiment. It should be noted that the technical solution of the inspection robot navigation system based on BeiDou and binocular vision fusion belongs to the same concept as the technical solution of the inspection robot navigation method based on BeiDou and binocular vision fusion described above. Details not described in detail in the technical solution of the inspection robot navigation system based on BeiDou and binocular vision fusion in this embodiment can be found in the description of the technical solution of the inspection robot navigation method based on BeiDou and binocular vision fusion described above.

[0138] This embodiment also provides a navigation system for an inspection robot based on the fusion of BeiDou and binocular vision, including:

[0139] The multimodal data fusion module is used to perform multimodal data fusion processing based on the initial coordinates of the inspection robot and the images captured by the binocular vision system to obtain environmental perception results;

[0140] The trajectory planning module is used to perform global planning and local optimization based on environmental perception results, and to coordinate these processes to generate a trajectory planning scheme.

[0141] The inspection robot motion control module is used to carry out intelligent navigation and collaborative regulation based on environmental perception results and trajectory planning schemes, so as to obtain the inspection robot motion control strategy.

[0142] This embodiment also provides an electronic device applicable to the navigation method of inspection robots based on the fusion of Beidou and binocular vision, including:

[0143] The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement the inspection robot navigation method based on the fusion of BeiDou and binocular vision as proposed in the above embodiments.

[0144] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the inspection robot navigation method based on the fusion of Beidou and binocular vision as proposed in the above embodiment.

[0145] The storage medium proposed in this embodiment belongs to the same inventive concept as the inspection robot navigation method based on the fusion of Beidou and binocular vision proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0146] Example 4, referring to Tables 1-3, is an embodiment of the present invention, providing a navigation method for an inspection robot based on the fusion of Beidou and binocular vision. To verify the beneficial effects of the present invention, a simulation experiment is conducted for scientific demonstration.

[0147] To verify the superiority of the intelligent cooperative navigation and obstacle avoidance method proposed in this invention, a systematic comparison was conducted with two existing methods. The experiment selected a simulated substation scenario as the test object, and the equipment included core components such as a BeiDou high-precision navigation module, a binocular vision system, and an embedded deep learning processing unit. The experiment was conducted in a simulated substation scenario, and the specific experimental environment configuration is shown in Table 1.

[0148] Table 1 Experimental Environment Configuration

[0149]

[0150] The comparative experiment was conducted in a dynamic obstacle avoidance scenario, covering multiple indicators such as navigation accuracy, obstacle avoidance efficiency, success rate, and obstacle avoidance time. Each indicator and its description are shown in Table 2.

[0151] Table 2 Experimental Indicators

[0152]

[0153] The specific results of the experiment are shown in Table 3.

[0154] Table 3. Experimental results data for each indicator.

[0155]

[0156] Through the above experimental comparisons, it is clear that the method of the present invention is significantly superior to the classic method in several key indicators such as navigation accuracy, obstacle avoidance efficiency, success rate, and obstacle avoidance response speed.

[0157] In terms of navigation accuracy, the average deviation of the method in this invention is 5.3 cm, which is an improvement of 66.5% and 57.9% compared to binocular vision path planning and vector field histogram navigation, respectively. This advantage is attributed to the synergistic fusion of Beidou high-precision navigation and binocular artificial intelligence vision, enabling the robot to maintain accurate positioning in complex environments. Regarding obstacle avoidance efficiency, the obstacle avoidance path length of the method in this invention is 12.7 m, significantly shorter than binocular vision path planning and vector field histogram navigation. The deep learning-driven trajectory optimization algorithm can adjust the path in real time according to the dynamic environment, thereby reducing unnecessary detours. In terms of success rate, the method in this invention achieves 98%, demonstrating higher robustness and adaptability in complex dynamic environments, accurately completing navigation tasks even in areas with signal interference or dense dynamic obstacles. Obstacle avoidance time is a key indicator of a robot's dynamic response capability. The average obstacle avoidance time of the method in this invention is 1.5 s, mainly due to the introduction of dynamic obstacle prediction technology, which allows the robot to adjust its path in advance, thus making a rapid obstacle avoidance response.

[0158] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A navigation method for an inspection robot based on the fusion of BeiDou navigation and binocular vision, characterized in that, include: Based on the initial coordinates of the inspection robot and the images captured by the binocular vision system, multimodal data fusion processing is performed to obtain the environmental perception results; Based on the environmental perception results, global planning and local optimization are performed and coordinated to generate a trajectory planning scheme. Based on the environmental perception results and trajectory planning scheme, intelligent navigation and collaborative regulation are carried out to obtain the motion control strategy of the inspection robot.

2. The navigation method for an inspection robot based on the fusion of BeiDou and binocular vision as described in claim 1, characterized in that, The environmental perception results obtained by performing multimodal data fusion processing based on the initial coordinates of the inspection robot and the images captured by the binocular vision system include: The robot's initial coordinates in three-dimensional space are output through the BeiDou system. The error introduced by multipath effect is obtained by modeling environmental features. Kalman filtering is applied to dynamically optimize the data during real-time operation. Parallax is calculated using images captured by left and right cameras to determine the distance between obstacles and the robot, thus achieving accurate positioning of obstacles in three-dimensional space. Features of input images are extracted through convolutional neural networks to identify obstacle types and dynamic behaviors, generate three-dimensional point cloud data of the local environment, and construct a local environment map.

3. The navigation method for an inspection robot based on the fusion of BeiDou and binocular vision as described in claim 2, characterized in that, The process of performing multimodal data fusion processing based on the initial coordinates of the inspection robot and the images captured by the binocular vision system to obtain the environmental perception results also includes: A weighted fusion-based multimodal information processing model is introduced to combine the global positioning of BeiDou navigation with the local perception of binocular vision. The location information is optimized by weighted fusion, the priority of BeiDou and visual data is dynamically adjusted, and the fused location information is further optimized by a filter to smooth the final estimate and obtain the environmental perception result.

4. The navigation method for an inspection robot based on the fusion of BeiDou and binocular vision as described in claim 3, characterized in that, The process of generating a trajectory planning scheme based on environmental perception results, involving global planning and local optimization in a coordinated manner, includes: A global path search is performed by calculating the total cost function of each node. The actual path cost from the starting point to the current node and the heuristic estimate from the current node to the target point are considered together, and nodes closer to the target point are selected first. The global path is then optimized by the robot acquiring state information from the environment, exploring the optimal path through trial actions, receiving rewards based on the current state and the selected actions, and updating the value function of future paths to obtain the optimal global path planning strategy.

5. The navigation method for an inspection robot based on the fusion of BeiDou and binocular vision as described in claim 4, characterized in that, The process of generating a trajectory planning scheme based on environmental perception results, involving global planning and local optimization in a coordinated manner, also includes: A deep learning-based motion prediction model is used to predict the future trajectory of dynamic obstacles, calculate their future positions in real time, and adjust the local path accordingly.

6. The navigation method for an inspection robot based on the fusion of BeiDou and binocular vision as described in claim 5, characterized in that, The process of generating a trajectory planning scheme based on environmental perception results, involving global planning and local optimization in a coordinated manner, also includes: The system achieves coordinated optimization of global path planning and local path optimization through weighting factors, and dynamically adjusts the weighting factors according to the complexity of the environment.

7. The navigation method for an inspection robot based on the fusion of BeiDou and binocular vision as described in claim 6, characterized in that, The intelligent navigation and collaborative control strategy for the inspection robot, based on environmental perception results and trajectory planning schemes, includes the following: By combining visual perception data from environmental perception results with trajectory planning schemes, obstacle movement is predicted and avoided in real time. The position, velocity, and acceleration of obstacles are monitored in real time, and the robot's movement path is adjusted accordingly. A safe zone is constructed on the predicted trajectory of dynamic obstacles. When the robot is about to enter the safe zone, the obstacle avoidance mechanism is triggered. An obstacle avoidance path is generated through an optimization algorithm, taking into account both distance cost and path smoothness cost. The priority of global and local paths is dynamically adjusted according to the complexity of the environment.

8. A navigation system for an inspection robot based on the fusion of BeiDou and binocular vision, employing the method described in any one of claims 1 to 7, characterized in that, include: The multimodal data fusion module is used to perform multimodal data fusion processing based on the initial coordinates of the inspection robot and the images captured by the binocular vision system to obtain environmental perception results; The trajectory planning module is used to perform global planning and local optimization based on environmental perception results, and to coordinate these processes to generate a trajectory planning scheme. The inspection robot motion control module is used to carry out intelligent navigation and collaborative regulation based on environmental perception results and trajectory planning schemes, so as to obtain the inspection robot motion control strategy.

9. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Images