Robot positioning method, device and equipment, readable storage medium and program product

Abstract

The invention relates to a robot positioning method, device and equipment, a readable storage medium and a program product, and relates to the technical field of robot control. Comprising the following steps: acquiring environment observation data in a motion process of a target robot, first motion observation data representing an inertial motion increment of the target robot and second motion observation data representing a displacement increment of the target robot; performing joint initialization processing on the environment observation data with the first motion observation data and the second motion observation data to obtain a first environment scale parameter and a second environment scale parameter; fusing the first environment scale parameter and the second environment scale parameter to obtain a physical environment scale parameter; based on the physical environment scale parameters, converting the environment observation data into target environment observation data with physical size attributes; and fusing the target environment observation data, the first motion observation data and the second motion observation data to obtain a robot positioning result. The method can improve the positioning precision of the electric power inspection robot.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a robot positioning method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] The safety of electrical equipment is crucial for maintaining daily power supply, therefore power companies have established inspection and patrol systems. Traditional manual inspection methods not only require significant manpower, are time-consuming and labor-intensive, but are also prone to misjudgments, have low efficiency, and are subject to stringent environmental requirements. Therefore, power companies have introduced power inspection robots to replace manual inspections.

[0003] However, power line inspection robots still suffer from shortcomings in positioning and navigation in practical applications. For example, the commonly used magnetic track following method lacks flexibility and is difficult to change routes; visual positioning is easily affected by factors such as light intensity, texture, and moving objects; GPS (Global Positioning System) has low accuracy in strong electromagnetic environments such as substations or power plants; and while inertial navigation and odometer positioning have acceptable short-term accuracy, they suffer from cumulative errors and cannot meet the needs of long-distance, precise inspections. Therefore, current power line inspection robots suffer from low positioning accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide a robot positioning method, device, computer equipment, computer-readable storage medium, and computer program product that can improve the positioning accuracy of power inspection robots in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a robot localization method, including:

[0006] Acquire environmental observation data and motion observation data of the target robot during its movement. The motion observation data includes first motion observation data characterizing the inertial motion increment of the target robot and second motion observation data characterizing the displacement increment of the target robot.

[0007] The environmental observation data is jointly initialized with the first motion observation data and the second motion observation data to obtain the first environmental scale parameter and the second environmental scale parameter. The first environmental scale parameter is obtained based on the environmental scale estimation of inertial motion, and the second environmental scale parameter is obtained based on the environmental scale estimation of displacement.

[0008] The first environmental scale parameter and the second environmental scale parameter are fused to obtain the physical environmental scale parameter;

[0009] Based on physical environmental scale parameters, environmental observation data is converted into target environmental observation data with physical size attributes;

[0010] The localization result of the target robot is obtained by fusing the target environment observation data, the first motion observation data, and the second motion observation data.

[0011] In one embodiment, the localization result of the target robot is obtained by fusing target environment observation data, first motion observation data, and second motion observation data, including:

[0012] Environmental adaptability analysis was performed on the target environment observation data, the first motion observation data, and the second motion observation data respectively, and the environmental adaptability analysis results of the target environment observation data, the first motion observation data, and the second motion observation data were obtained respectively.

[0013] Based on the results of each environmental adaptability analysis, the fusion weights of the target environment observation data, the first motion observation data, and the second motion observation data are determined.

[0014] Based on the fusion weights, the target environment observation data, the first motion observation data, and the second motion observation data are fused.

[0015] In one embodiment, the robot localization method further includes:

[0016] Based on the first motion observation data and the second motion observation data, the pose of the target robot is predicted, and the pose prediction result of the target robot is obtained.

[0017] Extract data features from the target environment observation data, perform target robot pose matching based on the data features, and obtain the target robot pose observation results;

[0018] Based on the fusion weights, the target environment observation data, the first motion observation data, and the second motion observation data are fused, including:

[0019] Based on the fusion weights, the pose prediction results and pose observation results are weighted and fused to obtain the localization results of the target robot.

[0020] In one embodiment, the robot localization method further includes:

[0021] Error analysis is performed on the pose prediction results and pose observation results to obtain the error analysis results between the pose prediction results and pose observation results;

[0022] Based on the error analysis results, error compensation is performed on the pose prediction results and pose observation results to obtain the compensated pose prediction results and compensated pose observation results.

[0023] Based on the fusion weights, the pose prediction results and pose observation results are weighted and fused to obtain the localization results of the target robot, including:

[0024] Based on the fusion weights, the compensated pose prediction results and the compensated pose observation results are weighted and fused to obtain the localization results of the target robot.

[0025] In one embodiment, the robot localization method further includes:

[0026] Based on the target environment observation data and the target robot's localization results, an environmental map of the target robot is constructed;

[0027] In the environmental map, loop closure detection is performed on the target robot to obtain the loop closure detection results;

[0028] Based on the loop closure detection results, the localization results of the target robot are optimized to obtain the optimized localization results.

[0029] In one embodiment, based on target environment observation data and the target robot's localization results, an environmental map of the target robot is constructed, including:

[0030] Based on the target environment observation data and the target robot's localization results, a preliminary environmental map of the target robot is constructed.

[0031] Obtain environmental observation data at the current moment;

[0032] If the environmental observation data at the current moment meets the map construction conditions, the preliminary environmental map is optimized based on the environmental observation data at the current moment to obtain the optimized environmental map.

[0033] Secondly, this application also provides a robot positioning device, comprising:

[0034] The data acquisition module is used to acquire environmental observation data and motion observation data of the target robot during its movement. The motion observation data includes first motion observation data characterizing the inertial motion increment of the target robot and second motion observation data characterizing the displacement increment of the target robot.

[0035] The joint initialization processing module is used to perform joint initialization processing on environmental observation data with first motion observation data and second motion observation data respectively to obtain first environmental scale parameters and second environmental scale parameters. The first environmental scale parameters are obtained based on the environmental scale estimation of inertial motion, and the second environmental scale parameters are obtained based on the environmental scale estimation of displacement.

[0036] The scale parameter acquisition module is used to fuse the first environmental scale parameter and the second environmental scale parameter to obtain the physical environmental scale parameter;

[0037] The data conversion module is used to convert environmental observation data into target environmental observation data with physical size attributes based on physical environmental scale parameters.

[0038] The data fusion module is used to obtain the localization result of the target robot by fusing the target environment observation data, the first motion observation data, and the second motion observation data.

[0039] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0040] Acquire environmental observation data and motion observation data of the target robot during its movement. The motion observation data includes first motion observation data characterizing the inertial motion increment of the target robot and second motion observation data characterizing the displacement increment of the target robot.

[0041] The environmental observation data is jointly initialized with the first motion observation data and the second motion observation data to obtain the first environmental scale parameter and the second environmental scale parameter. The first environmental scale parameter is obtained based on the environmental scale estimation of inertial motion, and the second environmental scale parameter is obtained based on the environmental scale estimation of displacement.

[0042] The first environmental scale parameter and the second environmental scale parameter are fused to obtain the physical environmental scale parameter;

[0043] Based on physical environmental scale parameters, environmental observation data is converted into target environmental observation data with physical size attributes;

[0044] The localization result of the target robot is obtained by fusing the target environment observation data, the first motion observation data, and the second motion observation data.

[0045] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0046] Acquire environmental observation data and motion observation data of the target robot during its movement. The motion observation data includes first motion observation data characterizing the inertial motion increment of the target robot and second motion observation data characterizing the displacement increment of the target robot.

[0047] The environmental observation data is jointly initialized with the first motion observation data and the second motion observation data to obtain the first environmental scale parameter and the second environmental scale parameter. The first environmental scale parameter is obtained based on the environmental scale estimation of inertial motion, and the second environmental scale parameter is obtained based on the environmental scale estimation of displacement.

[0048] The first environmental scale parameter and the second environmental scale parameter are fused to obtain the physical environmental scale parameter;

[0049] Based on physical environmental scale parameters, environmental observation data is converted into target environmental observation data with physical size attributes;

[0050] The localization result of the target robot is obtained by fusing the target environment observation data, the first motion observation data, and the second motion observation data.

[0051] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0052] Acquire environmental observation data and motion observation data of the target robot during its movement. The motion observation data includes first motion observation data characterizing the inertial motion increment of the target robot and second motion observation data characterizing the displacement increment of the target robot.

[0053] The environmental observation data is jointly initialized with the first motion observation data and the second motion observation data to obtain the first environmental scale parameter and the second environmental scale parameter. The first environmental scale parameter is obtained based on the environmental scale estimation of inertial motion, and the second environmental scale parameter is obtained based on the environmental scale estimation of displacement.

[0054] The first environmental scale parameter and the second environmental scale parameter are fused to obtain the physical environmental scale parameter;

[0055] Based on physical environmental scale parameters, environmental observation data is converted into target environmental observation data with physical size attributes;

[0056] The localization result of the target robot is obtained by fusing the target environment observation data, the first motion observation data, and the second motion observation data.

[0057] The aforementioned robot localization method, device, computer equipment, computer-readable storage medium, and computer program product, through multi-sensor data fusion, solve the problem of low localization accuracy of power inspection robots in complex environments. Specifically, it acquires environmental observation data during the target robot's movement, first motion observation data representing the target robot's inertial motion increment, and second motion observation data representing the target robot's displacement increment, providing a data foundation for subsequent fusion localization. Subsequently, the environmental observation data is jointly initialized with the two types of motion observation data to obtain their respective independent environmental scale parameters. This operation effectively overcomes the inherent scale uncertainty problem of monocular vision. Then, the two scale parameters are fused to generate a unified physical environmental scale parameter, achieving cross-validation and optimal integration between scale information obtained from different motion sources, minimizing scale estimation bias caused by sensor noise or instantaneous anomalies. Based on this, the original environmental observation data is converted into target environmental observation data with physical size attributes, completing the accurate mapping of the entire visual map and robot trajectory to the real physical world, laying a unified spatiotemporal benchmark for subsequent fusion. Finally, by tightly coupling and fusing the target environment observation data with the two types of motion observation data to solve the localization result, the global consistency of the visual sensor, the high-frequency dynamic characteristics of the IMU, and the planar motion accuracy of the odometer can be fully coordinated. This not only effectively suppresses the cumulative drift of the IMU and the odometer, but also overcomes the shortcomings of pure vision localization under the conditions of missing texture or changing lighting. Thus, in specific scenarios with strong electromagnetic interference and complex structures, such as substations, high-precision localization of power inspection robots is achieved, significantly improving the automation level of power inspection. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a diagram illustrating the application environment of a robot localization method in one embodiment;

[0060] Figure 2 This is a flowchart illustrating a robot localization method in one embodiment;

[0061] Figure 3 This is a schematic diagram of data alignment in one embodiment;

[0062] Figure 4 Here is a confidence weight allocation process in one embodiment;

[0063] Figure 5 This is a schematic diagram of constraint factors in one embodiment;

[0064] Figure 6 This is a schematic diagram of the architecture of a power inspection robot positioning system in a specific embodiment.

[0065] Figure 7 This is a flowchart illustrating a robot localization method in a specific embodiment;

[0066] Figure 8 This is a schematic diagram of the operation flow of an optical motion capture system in a specific embodiment;

[0067] Figure 9 This is a structural block diagram of a robot positioning device in one embodiment;

[0068] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0070] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0071] The safety of electrical equipment is crucial for maintaining daily power supply. Therefore, power companies have established inspection systems. Regular inspections by on-duty personnel allow them to understand the operational status of various electrical equipment within power plants or substations, such as main transformers, high-voltage circuit breakers, capacitors, and other auxiliary electrical equipment. By monitoring equipment operation, they can prevent malfunctions caused by prolonged high-load operation. Traditional manual inspection methods rely on staff who observe and record equipment readings and judge operating status based on experience. This method is not only labor-intensive and time-consuming but also prone to misjudgments and has low efficiency. Furthermore, it is impossible to conduct inspections in certain weather conditions, such as heavy rain or snow, making it difficult to detect equipment defects in a timely manner and thus affecting the normal operation of the entire power grid system. In recent years, with the rapid development of the social economy and electrification technology, the demand for electricity supply has been continuously increasing, leading to a rise in the number of electrical equipment built and put into use. The types of new equipment are also increasing, and the systems are becoming more complex. This makes traditional manual inspection methods increasingly inadequate for the growing workload and difficulty of inspections. To overcome the shortcomings of traditional manual inspection methods, power grid companies have introduced robotic inspection methods. Power inspection robots can periodically and autonomously inspect the interior of substations or power plants. By collecting data on equipment inside the station through sensors such as infrared and vision, they can judge the operating status of the equipment. This not only reduces the workload of manual inspections but also ensures the real-time and accuracy of inspection work, playing an important role in ensuring the safe and stable operation of power generation and transmission equipment and even the entire power grid system.

[0072] However, power line inspection robots still suffer from shortcomings in positioning and navigation in practical applications. The commonly used magnetic track following method lacks flexibility and is difficult to change routes. Other single-sensor positioning methods also have limitations. For example, visual positioning is easily affected by factors such as light intensity, texture, and moving objects; GPS accuracy is low in strong electromagnetic environments such as substations or power plants; and while inertial navigation and odometer positioning offer acceptable short-term accuracy, they suffer from cumulative errors, making them unsuitable for long-distance, precise inspections. Therefore, current power line inspection robots suffer from low positioning accuracy.

[0073] To address the aforementioned problems, this application provides a robot localization method that can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. Terminal 102 can be, but is not limited to, various sensors mounted on the power inspection robot, such as cameras, inertial measurement units (IMUs), odometers, lidar, GPS, etc., used to collect real-time data on robot operation and environmental perception. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Its deployment methods include, but are not limited to, being integrated into the power inspection robot as a built-in computing unit for efficient local processing, or acting as an external independent computing node for remote data interaction with the robot via a wireless network. Server 104 is mainly used to provide services such as robot positioning calculation, multi-sensor data fusion, inspection map construction and optimization. The data storage system is used to store the data that server 104 needs to process, including but not limited to raw sensor data, robot positioning results, inspection maps, etc. The data storage system can be integrated on server 104 or deployed in the cloud or other network servers.

[0074] In one exemplary embodiment, such as Figure 2 As shown, a robot localization method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the following steps are included:

[0075] Step S202: Obtain environmental observation data and motion observation data of the target robot during its movement. The motion observation data includes first motion observation data characterizing the inertial motion increment of the target robot and second motion observation data characterizing the displacement increment of the target robot.

[0076] The target robot refers to a robot that performs power line inspection tasks. Its working environment is typically a substation or power plant, a scenario with strong electromagnetic interference and complex structures. The robot is equipped with various sensors to perceive the environment and its operational status. Environmental observation data refers to sensor data obtained from perceiving the robot's external environment, primarily used to construct environmental maps and calculate the robot's pose relative to the environment. In this embodiment, environmental observation data is mainly acquired through visual sensors such as binocular cameras and LiDAR. The visual sensors passively capture RGB or grayscale image sequences of the environment, while the LiDAR acquires two-dimensional or three-dimensional point cloud data of the surrounding environment by emitting and receiving laser beams. These point clouds consist of a large number of points with three-dimensional coordinates, accurately describing the geometric contours of the environment.

[0077] Motion observation data refers to sensor data reflecting the robot's own operating state and changes. It is further divided into first motion observation data and second motion observation data. First motion observation data refers to the motion increment information measured by the inertial measurement unit (IMU) within a continuous time interval. It characterizes the inertial motion increment of the target robot and reflects the changes in the robot's angular velocity and linear acceleration in three-dimensional space. IMU sensors typically include a three-axis gyroscope and a three-axis accelerometer. The gyroscope outputs the angular velocity in three axes in radians per second, and the accelerometer outputs the linear acceleration in three axes in meters per square second. By pre-integrating these raw data within a sampling period, the relative motion increment of the robot within that period can be obtained. Pre-integration processing refers to accumulating multiple IMU measurements into a single relative motion constraint, avoiding repeated integration during the back-end optimization process, thereby improving computational efficiency. The second motion observation data refers to the motion increment information measured by the wheel odometry within continuous time intervals. It characterizes the displacement increment of the target robot in a two-dimensional plane. To achieve alignment and collaborative optimization with IMU pre-integration on the time scale and to avoid repeated motion integration in the back-end optimization, this embodiment also performs pre-integration processing on the raw odometry data. Specifically, the photoelectric encoder installed on the robot's drive wheel generates pulses as the wheel rotates. By counting the number of pulses received within a fixed time interval, such as a sampling period, and combining this with the known wheel circumference and the number of pulses per revolution, the straight-line distance rolled by the wheel in that period can be calculated. By accumulating the displacement in multiple consecutive periods, an odometry pre-integration constraint matching the IMU pre-integration period is formed. This constraint, as an observation, is input into the server along with the IMU pre-integration and environmental observation data for back-end optimization.

[0078] For example, during the movement of the target robot, its onboard vision sensor and lidar simultaneously acquire images and 3D point clouds of the environment through passive optical imaging and active laser ranging, respectively, forming environmental observation data. Simultaneously, the robot's onboard IMU continuously measures the robot's angular velocity and linear acceleration through its built-in gyroscope and accelerometer, and outputs high-frequency inertial motion increment information, i.e., the first motion observation data, after pre-integration processing. The robot's onboard wheel odometry calculates the displacement increment by measuring the number of pulses in the wheels through encoders, and outputs the robot's relative displacement change on a two-dimensional plane, i.e., the second motion observation data, after pre-integration processing. Finally, all sensors upload their collected observation data to the server for subsequent multi-sensor data fusion and pose estimation.

[0079] Step S204: Perform joint initialization processing on the environmental observation data with the first motion observation data and the second motion observation data to obtain the first environmental scale parameter and the second environmental scale parameter. The first environmental scale parameter is obtained based on the environmental scale estimation of inertial motion, and the second environmental scale parameter is obtained based on the environmental scale estimation of displacement.

[0080] It should be noted that vision-based environmental observation data lacks a true physical scale when reconstructing robot motion trajectories. The robot displacement and environmental structure estimated by the vision sensor from image sequences are based on arbitrary scale references and cannot directly reflect true metric distances. Moreover, the motion sensors used to provide physical scale references have inherent limitations, such as random bias in IMUs and susceptibility to noise and wheel slippage interference in odometers. If a precise scale reference cannot be restored for visual observations and the coordinate systems of each sensor cannot be unified to a stable initial state, reliable fusion of the three heterogeneous observation data—vision, IMU, and odometer—cannot be achieved. To address this, this embodiment employs a loosely coupled joint initialization method. Within a sliding window, the scale-less environmental observation information is aligned and optimized with two types of motion observation data with physical units. Specifically, it is aligned with the IMU pre-integrated data reflecting inertial motion to solve for the first environmental scale parameter, and aligned with the odometer pre-integrated data reflecting planar displacement to solve for the second environmental scale parameter. In this way, the server can independently recover the true scale of the environment from sensors based on different physical principles, thereby providing accurate initial values ​​for subsequent tightly coupled optimization.

[0081] Joint initialization processing refers to a data alignment process based on optimization algorithms. Its goal is to find an optimal set of initial states, including scale, velocity, gravity direction, and sensor bias, under a unified spatiotemporal reference, so that the robot trajectory predicted based on motion observation data matches the visual trajectory reconstructed based on environmental observation data. The first environmental scale parameter is an environmental scale factor obtained by aligning the environmental observation data with the first motion observation data; it represents the multiplier required to scale the visually reconstructed model to the real world. The second environmental scale parameter is another environmental scale factor obtained by aligning the environmental observation data with the second motion observation data.

[0082] For example, the server maintains a sliding window containing several consecutive image keyframes. Within this window, environmental observation data is first processed using visual motion estimation techniques to calculate the scale-free relative poses between the camera keyframes, thus forming a visual motion trajectory. Next, alignment optimization with the first motion observation data, i.e., IMU pre-integrated data, is performed. An optimization variable containing scale parameters, velocity, and gravity direction is first constructed, such as... x I = [ v b 0 b 0 , v b 0 b 0 ,..., v b 0 b 0 , g c 0 , s 1 ] T ,in, It is a jointly initialized vector to be optimized, which integrates the target robot's running speed, the gravity direction in the camera coordinate system, and the scale parameters. This represents the instantaneous velocity of the target robot body at the nth keyframe. Indicates the initial camera coordinate system The direction of gravity downwards, Let represent the first environmental scale parameter to be solved. The first environmental scale parameter is solved by minimizing the difference between the visual relative pose transformation and the IMU pre-integration. Specifically, the visually estimated relative pose is multiplied by the candidate scale parameter, and this, along with the displacement and rotation obtained from the IMU pre-integration, forms the residual term. A nonlinear optimization method, such as the Gauss-Newton method, is then used to iteratively adjust the scale parameter. To minimize the residual, the corresponding scale parameter is the first environmental scale parameter. Simultaneously, alignment optimization is performed between the visual motion trajectory and the odometry pre-integration data to construct a second environmental scale parameter containing only the parameters to be solved. The optimization variables are determined by minimizing the difference between the visual plane displacement and the odometer-measured displacement to solve for the second environmental scale parameter. Specifically, the visually estimated inter-frame two-dimensional plane displacement is multiplied by the candidate scale parameter, and then combined with the displacement measured by the odometer pre-integration to form a residual term. The scale parameter is then adjusted iteratively through nonlinear optimization. To minimize the residual, the corresponding scale parameter is the second environmental scale parameter.

[0083] In some embodiments, Figure 3 A schematic diagram of data alignment is shown. Figure 3 The process mainly consists of three parts: camera motion, IMU pre-integration, and odometry pre-integration. The camera is the visual sensor mounted on the target robot. Camera motion can be understood as calculating the relative pose of the camera between different keyframes using visual motion estimation technology, thereby obtaining a visual motion trajectory. In the joint initialization phase, the visual motion trajectory provided by the camera motion is loosely coupled and aligned with the IMU pre-integration and odometry pre-integration respectively, jointly recovering the robot's velocity, gravity direction, and key scale parameters, laying the foundation for subsequent tight-coupling optimization and precise positioning in the backend.

[0084] Step S206: The first environmental scale parameter and the second environmental scale parameter are fused to obtain the physical environmental scale parameter.

[0085] Among them, the physical environmental scale parameter refers to the true environmental scale factor obtained by weighted fusion of the first environmental scale parameter and the second environmental scale parameter.

[0086] For example, the server assigns appropriate dynamic confidence weights to the first and second environmental scale parameters, and then merges the two scale parameters into a more accurate and stable physical environmental scale parameter through weighted average calculation. This physical environmental scale parameter will serve as a unified scale benchmark to initialize the state variables in the subsequent tightly coupled back-end optimization, thereby effectively overcoming the possible bias or failure problems of single sensor scale estimation, and significantly improving the success rate of system initialization and the robustness of overall positioning.

[0087] In some embodiments, Figure 4 The confidence weight allocation process for the first and second environmental scale parameters is illustrated. First, i represents the iteration counter, used to record the number of loops in the process, and its initial value is 1. Represents the first environmental scale parameter The weight, Indicates the second environmental scale parameter The weights are initially set to 0.5, meaning that initially, the scale estimates of the two sensors (IMU and odometry) are assumed to have the same confidence level. In each iteration, the server will... , Calculate the first environmental scale parameters and second environmental scale parameters Next, a validity check is performed. The physical scale factor should theoretically be positive. If a negative scale is calculated, it indicates that the estimation is invalid. In this case, the other scale parameter is fully trusted, and its weight is set to 1, while the current scale parameter's weight is set to 0. That is, a judgment is made... or Is it less than 0? <0, >0, then Set to 0, Set it to 1; if >0, <0, then Set to 1, Set to 0; if <0, If the value is less than 0, it indicates that both scale estimates are invalid, so the current weights are not updated, and the weight values ​​from the previous iteration are retained; if... >0, A value greater than 0 indicates that both scale estimates are effective. At this point, the process enters the error comparison and weight fine-tuning stage, which involves comparing the errors corresponding to the two scale estimates. , ,like This indicates that the IMU's estimate is more accurate, and therefore... The value was increased. The value is reduced, that is , ,like This indicates that the odometer's estimate is more accurate, therefore... The value was increased. The value is reduced, that is , Repeat the above steps until the specified number of iterations, such as 5, is completed, and then output the final weights. and weight These two weights are used to weight and fuse the first and second environmental scale parameters to obtain the final physical environmental scale parameters.

[0088] Step S208: Based on the physical environment scale parameters, convert the environmental observation data into target environmental observation data with physical size attributes.

[0089] Among them, target environment observation data refers to environmental observation data that has real size attributes after being calibrated by physical environment scale parameters, including but not limited to three-dimensional road sign point clouds and robot motion trajectories with physical dimensions.

[0090] For example, the server obtains the coordinates of all scale-free 3D landmark points and camera poses from the visually estimated motion trajectory, and then multiplies them by the physical environment scale parameter to correctly scale the entire visually reconstructed model to real-world metric units. This conversion ensures that the environment map reconstructed based on visual observations corresponds perfectly to the physical space where the robot actually moves, laying an accurate data foundation for subsequent tightly coupled optimization, path planning, and equipment status analysis that rely on precise physical dimensions.

[0091] Step S210: By fusing the target environment observation data, the first motion observation data, and the second motion observation data, the localization result of the target robot is obtained.

[0092] The localization result of the target robot refers to the final output of the target robot's pose information in the physical world, including but not limited to at least one of the target robot's three-dimensional position coordinates, three-dimensional pose, and timestamp confidence.

[0093] For example, after obtaining a unified physical scale benchmark, the server can achieve final high-precision positioning within a tightly coupled graph optimization framework. This framework is a multi-sensor fusion architecture that directly represents the observation data from different sensors as constraints between state variables, and estimates the most probable values ​​of all variables by optimizing a unified graphical model. Specifically, within a sliding window of local optimization, the server constructs a state variable to be optimized, containing the robot's pose, velocity, sensor bias, and coordinates of 3D landmark points. This variable can be represented as:

[0094] χ=[ x 0 , x 1 ,..., x n , λ 0 , λ 1 ,..., λ k ]

[0095] in, A set of state variables to be optimized is defined, which contains two types of state changes: robot state sequences, i.e. and environmental road sign set . This represents the robot state at n+1 consecutive keyframes within the sliding window, where each robot state... Defined as:

[0096] x i =[ p b j w , v b j w , q b j w , b ω ],i ∈ [0,n]

[0097] in, Represents the robot's body coordinate system at time i. The three-dimensional position coordinates of the origin in the world coordinate system w can also be understood as the robot's position on the global map. Let represent the velocity vector of the robot in the world coordinate system w at time i. This indicates that at time i, the coordinates of the robot's body are... The rotation relationship to the world coordinate system w can also be understood as the robot's orientation or posture in the global map. For IMU bias, The bias represents the inherent systematic error in the sensor readings. Represents angular velocity. Environmental landmark set, i.e. This represents the three-dimensional coordinates of k+1 three-dimensional landmarks in the world coordinate system w. These landmarks are extracted from visual observation and transformed using physical environment scale parameters.

[0098] Simultaneously, the server uses a graph model to uniformly model multi-source observation data into geometric and motion constraints on state variables. For example... Figure 5 As shown, the constraints include feature point observation, keyframe pose, IMU pre-integration constraints, odometry pre-integration constraints, and prior information constraints. Feature point observation and keyframe pose jointly define the visual reprojection error constraint, whose physical source is the target environment observation data, used to constrain the robot's pose and map point position. IMU pre-integration constraints define the robot's inertial motion error constraint, whose physical source is the first motion observation data, used to constrain the relative motion between consecutive keyframes. Odometry pre-integration constraints define the robot's planar motion error constraint, whose physical source is the second motion observation data, providing supplementary constraints for the robot's planar motion. Prior information constraints are obtained by marginalizing historical states outside the sliding window; that is, the information contained in the marginalized states is converted into prior constraints to ensure historical consistency in optimization. The sum of squares of all error terms is minimized using a nonlinear optimization algorithm such as the Gauss-Newton method, thereby achieving the maximum a posteriori estimate of the state variables, and finally outputting the precise pose of the target robot in the world coordinate system as the localization result.

[0099] In this embodiment, the low positioning accuracy of power inspection robots in complex environments is addressed through multi-sensor data fusion. Specifically, environmental observation data during the robot's movement, first motion observation data representing the robot's inertial motion increment, and second motion observation data representing the robot's displacement increment are acquired, providing a data foundation for subsequent fusion positioning. Subsequently, the environmental observation data is jointly initialized with the two types of motion observation data to obtain their respective independent environmental scale parameters. This operation effectively overcomes the inherent scale uncertainty problem of monocular vision. The two scale parameters are then fused to generate a unified physical environmental scale parameter, achieving cross-validation and optimal integration of scale information obtained from different motion sources, minimizing scale estimation bias caused by sensor noise or instantaneous anomalies. Based on this, the original environmental observation data is converted into target environmental observation data with physical size attributes, completing the accurate mapping of the entire visual map and robot trajectory to the real physical world, laying a unified spatiotemporal benchmark for subsequent fusion. Finally, by tightly coupling and fusing the target environment observation data with the two types of motion observation data to solve the localization result, the global consistency of the visual sensor, the high-frequency dynamic characteristics of the IMU, and the planar motion accuracy of the odometer can be fully coordinated. This not only effectively suppresses the cumulative drift of the IMU and the odometer, but also overcomes the shortcomings of pure vision localization under the conditions of missing texture or changing lighting. Thus, in specific scenarios with strong electromagnetic interference and complex structures, such as substations, high-precision localization of power inspection robots is achieved, significantly improving the automation level of power inspection.

[0100] In an exemplary embodiment, the localization result of the target robot is obtained by fusing target environment observation data, first motion observation data, and second motion observation data, including: performing environmental adaptability analysis on the target environment observation data, first motion observation data, and second motion observation data respectively to obtain the environmental adaptability analysis results of the target environment observation data, first motion observation data, and second motion observation data; determining the fusion weights of the target environment observation data, first motion observation data, and second motion observation data according to the environmental adaptability analysis results; and fusing the target environment observation data, first motion observation data, and second motion observation data based on the fusion weights.

[0101] Environmental adaptability analysis refers to the process of real-time evaluation of the working status and data quality of various observation data or sensors under current environmental conditions. The results of environmental adaptability analysis are reliability assessment indicators derived from the analysis of sensor data quality and environmental characteristics. Specifically, these may include visual reliability indicators, IMU reliability indicators, and odometer reliability indicators. Visual reliability indicators are scores derived from a comprehensive analysis of the number of image feature points, their distribution uniformity, and the proportion of dynamic object occlusion; these are used to assess whether the current environment is suitable for visual positioning. IMU reliability indicators are scores derived from a comprehensive analysis of IMU bias and bias stability; these are used to assess the reliability of inertial navigation data. Odometer reliability indicators are scores derived from a comprehensive analysis of wheel slippage detection results, displacement consistency, and wheel speed symmetry; these are used to assess the accuracy of wheeled odometer data. Fusion weights are weight coefficients dynamically assigned based on the results of each environmental adaptability analysis. These coefficients determine the contribution of various observation data to the tightly coupled optimization process; higher weights indicate greater importance of the corresponding observation data in the fusion process.

[0102] For example, when performing multi-sensor data fusion, the server first performs quality assessments on various types of observation data separately. It should be noted that in addition to visual sensors, IMUs, and odometers, GPS sensor data and other sensor data can also be fused. For target environment observation data, the server counts the number of feature points in the current frame, assesses the uniformity of feature point distribution, and detects the proportion of dynamic objects. When there are too few feature points, their distribution is concentrated, or the proportion of dynamic objects is too high, the visual reliability is considered low. For the first motion observation data, the server can calculate the variance of IMU measurements, monitor the rate of bias change, and assess the sufficiency of motion excitation. When vibration is severe, the bias is unstable, or the motion is uniform, the IMU reliability is considered low. For the second motion observation data, the system can compare the difference in left and right wheel speeds, detect abrupt displacement changes, and analyze motion consistency. When asymmetrical motion, abnormal displacement, or slippage characteristics occur, the odometer reliability is considered limited.

[0103] Next, the server executes a dynamic confidence weighting strategy to determine the fusion weights based on the environmental adaptability analysis results of each sensor. For example, in outdoor scenes with strong and unobstructed GPS signals, the weight of GPS sensor data can be increased to reduce cumulative drift, while in environments where GPS signals are weak, such as tunnels or mountainous areas, its weight can be automatically reduced, while simultaneously increasing the contribution ratio of the IMU and visual sensors. As another example, for target environment observation data, its weight can be increased in textured static environments (i.e., increasing visual weight), while its weight can be reduced in scenes with missing features or dynamic interference. For first motion observation data, its weight can be increased during high-speed motion, while its weight can be reduced in low-speed, constant-speed states. For second motion observation data, its weight can be increased when traveling in a straight line on flat ground, while its weight can be adaptively reduced under abnormal conditions such as slippage. The percentage increase or decrease in weight can be determined according to the actual situation, and this embodiment does not impose any restrictions on this. Finally, the server introduces the determined weight coefficients into the tightly coupled graph optimization framework. When constructing the nonlinear least squares problem, the various observation residual terms are multiplied by the corresponding fusion weights to form a weighted cost function. The weighted optimization problem is solved iteratively using the Gauss-Newton method, so that the reliable sensor data under different environmental conditions plays a dominant role in state estimation, thereby achieving seamless switching and continuous accurate positioning.

[0104] In an exemplary embodiment, the robot localization method further includes: performing pose prediction on the target robot based on first motion observation data and second motion observation data to obtain the pose prediction result of the target robot; extracting data features from the target environment observation data, and performing pose matching on the target robot based on the data features to obtain the pose observation result of the target robot.

[0105] The pose prediction result refers to the prior estimate of the robot's pose calculated based on the first and second motion observation data, representing the current pose prediction value derived from the state at the previous moment. The pose observation result refers to the robot's pose measurement value obtained through geometric calculation based on the target environment observation data. Specifically, it can be calculated by matching the current visual feature points with existing map features and using the PnP (Perspective-n-Point) algorithm, representing the pose observation value directly obtained from environmental perception. Data features refer to the visual information extracted from the target environment observation data for pose calculation, including each feature descriptor and its two-dimensional coordinates in the image, as well as the coordinates of the three-dimensional map points corresponding to these features.

[0106] For example, firstly, the server performs pose prediction based on motion observation data. That is, using the first motion observation data, it obtains high-frequency pose prediction through inertial navigation calculation, and simultaneously uses the second motion observation data to obtain planar motion pose prediction through the kinematic model of a wheeled robot. These two prediction results constitute the IMU pre-integration constraint and odometry pre-integration constraint in the graph optimization framework, respectively. Simultaneously, the server performs pose observation calculation, that is, it extracts visual feature points from the target environment observation data, establishes the correspondence between the current observation and map points through feature matching, and thus calculates the current camera pose, obtaining the pose observation result. This observation result constitutes the visual reprojection constraint in the graph optimization framework.

[0107] In some embodiments, the target environment observation data, the first motion observation data, and the second motion observation data are fused based on each fusion weight, including: weighted fusion of the pose prediction result and the pose observation result based on each fusion weight to obtain the localization result of the target robot.

[0108] For example, in the weighted fusion stage, the server, based on the aforementioned environmental adaptability analysis results and the fusion weight allocation mechanism, transforms the dynamic fusion weights into covariance matrix weights for each factor in graph optimization. That is, for sensor data with high reliability, a smaller covariance (higher weight) is assigned, while for data with low reliability, a larger covariance (lower weight) is assigned. When constructing the nonlinear least squares problem, weighted fusion is achieved by adjusting the information matrices corresponding to the constraints of each factor, ultimately obtaining a globally optimized robot localization result.

[0109] In an exemplary embodiment, the robot localization method further includes: performing error analysis on the pose prediction result and the pose observation result to obtain the error analysis result between the pose prediction result and the pose observation result; and performing error compensation on the pose prediction result and the pose observation result based on the error analysis result to obtain the compensated pose prediction result and the compensated pose observation result.

[0110] Error analysis results refer to the systematic bias assessment obtained by comparing the differences between pose prediction results and pose observation results. This includes, but is not limited to, at least one of the following: position error vector, pose error quaternion, and error covariance statistics for each degree of freedom. These metrics characterize the degree of inconsistency between motion prediction and environmental observation. Error compensation refers to the process of correcting pose prediction and pose observation results based on error analysis results. By modeling and eliminating systematic biases, the accuracy of motion prediction is improved, and the difference between prediction and observation is reduced. The compensated pose prediction result refers to the motion prediction value after error compensation processing, and the compensated pose observation result refers to the visual observation value after error compensation processing.

[0111] For example, to further improve positioning accuracy, this embodiment also introduces an error prediction model based on deep learning, such as Long Short-Term Memory (LSTM) networks. The core advantage of LSTM is that it captures the dynamic trends of time-series data. It learns the variation patterns of IMU and visual deviations in historical moments, such as the increase in IMU deviation during strong vibrations and the sharp increase in visual deviation during sudden changes in illumination. Based on these patterns, it predicts the error distribution for the next moment. The key to error compensation is to weaken the impact of errors by dynamically adjusting filtering parameters. Filtering parameters refer to the noise covariance matrix in the core of multi-sensor fusion, such as the Extended Kalman Filter (EKF). This matrix includes the process noise covariance of the IMU and the observation noise covariance of the vision. If LSTM predicts that the IMU deviation will increase in the next moment, it automatically increases the process noise covariance of the IMU and reduces its weight in fusion to avoid the IMU data with large deviations dominating the positioning. If it predicts that the visual deviation will increase in the next moment due to occlusion, it increases the observation noise covariance of the vision and simultaneously increases the weight of the IMU or LiDAR, using more reliable sensor data to offset visual errors. By dynamically adjusting the filtering parameters, the multi-sensor data fusion rules are optimized, actively avoiding the negative impact of error sources, thereby compensating for pose prediction and pose observation results.

[0112] In some embodiments, the pose prediction result and the pose observation result are weighted and fused based on each fusion weight to obtain the localization result of the target robot, including: weighting and fusing the compensated pose prediction result and the compensated pose observation result based on each fusion weight to obtain the localization result of the target robot.

[0113] For example, in the weighted fusion stage, the server fuses the compensated pose prediction result with the compensated pose observation result to obtain the localization result of the target robot. Experiments show that the error compensation mechanism based on deep learning in this embodiment reduces the robot pose estimation error by about 35% under conditions of strong vibration, complex lighting, and partial occlusion. This significantly improves the localization stability and accuracy in complex environments.

[0114] In an exemplary embodiment, the robot localization method further includes: constructing an environmental map of the target robot based on target environment observation data and the localization result of the target robot; performing loop closure detection on the target robot in the environmental map to obtain loop closure detection results; and optimizing the localization result of the target robot based on the loop closure detection results to obtain optimized localization results.

[0115] The environment map refers to a physically scaled 3D environment model constructed based on target environment observation data. This map includes a feature map composed of visual feature points and a pose map composed of keyframe poses. Each visual feature point contains its 3D spatial coordinates and feature descriptors, and the pose map records the robot's motion trajectory. Loop closure detection is a technical process for identifying whether the target robot has returned to a previously visited scene. By comparing the similarity between the current observation and historical map data, a loop closure is considered to have occurred when the similarity exceeds a set threshold. The loop closure detection result refers to the specific output of the loop closure detection process, which may include at least one of the following: the matching relationship between the current frame and historical keyframes, the relative pose transformation matrix, and the loop closure confidence score. The optimized localization result refers to the corrected pose estimate obtained by globally optimizing the target robot's historical motion trajectory using the spatial constraints provided by loop closure detection.

[0116] For example, this embodiment effectively solves the problem of accumulated errors in long-term localization through environmental map construction and loop closure detection mechanisms. Specifically, during robot movement, the server correlates target environment observation data with localization results in real time. For each keyframe, the system extracts visual feature points and transforms them from the camera coordinate system to the world coordinate system, while recording the robot pose at that moment. New feature points are added to the global map after screening, forming an incrementally constructed sparse feature map. Each feature point in the map has real physical size attributes, consistent with the scale benchmark of the target environment observation data. Next, the server performs loop closure detection. When a new keyframe is generated, the server calculates the similarity score between the current frame and historical keyframes in the map. If the score exceeds a set threshold, further geometric verification is performed. The PnP algorithm is used to calculate the relative pose transformation between the current frame and the matching historical frame, forming loop closure candidates. The server adopts a consistency check strategy to ensure the reliability of loop closure detection, and finally outputs loop closure detection results containing matching frame pairs and relative pose constraints. Finally, the server performs global optimization based on the loop closure detection results. That is, once a valid loop closure is confirmed, the system adds the loop closure constraint as a new factor to the tightly coupled graph optimization framework. This constraint connects the current pose node with historical pose nodes, forming an additional spatial closure constraint. The system then initiates a global optimization process, resolving the maximum a posteriori estimate of the robot's complete trajectory by minimizing the residuals of all factors, including the loop closure constraint. This optimization process effectively corrects the accumulated error caused by sensor drift, significantly improving the long-term positioning accuracy and ground motion of the power inspection robot in a wide range of environments. Figure 1 To the point of being responsive.

[0117] In an exemplary embodiment, constructing an environmental map of the target robot based on target environment observation data and the target robot's localization results includes: constructing a preliminary environmental map of the target robot based on target environment observation data and the target robot's localization results; acquiring environmental observation data at the current moment; and optimizing the preliminary environmental map based on the environmental observation data at the current moment, provided that the environmental observation data at the current moment meets the map construction conditions, to obtain an optimized environmental map.

[0118] The initial environment map refers to the physically scaled 3D environment model initially constructed during robot movement. Map construction conditions refer to the decision criteria for using current observation data for map optimization, including keyframe quality assessment, feature point richness detection, and viewpoint change angle thresholds, among other technical indicators. The optimized environment map is an environment model obtained through global optimization by fusing environmental observation data from multiple time points, exhibiting higher consistency and accuracy.

[0119] For example, this embodiment achieves continuous improvement and accuracy enhancement of the environmental map through a phased map construction and optimization mechanism. Specifically, during the target robot's movement, the server correlates the target environment observation data with the positioning results in real time. For each keyframe that passes the quality assessment, the system extracts feature points and calculates their feature descriptors. Simultaneously, based on the current accurate positioning results, these feature points are transformed from the camera coordinate system to the world coordinate system. Newly added 3D landmarks are added to the preliminary environmental map after triangulation verification and redundancy detection, forming a local map structure based on a sliding window. Next, the server acquires the environmental observation data at the current moment and evaluates the map construction conditions. When a new image frame arrives, the system evaluates its suitability for map optimization from multiple dimensions, such as calculating the viewpoint change angle between the current frame and the nearest keyframe, counting the number of feature points in the current frame, and detecting the uniformity of feature point distribution. If the above evaluation conditions are met simultaneously, the current frame is used as a new keyframe for map optimization. Finally, the server performs environmental map optimization. Once the map construction conditions are confirmed to be met, the server adds new keyframes and their observation data to the graph optimization framework. The optimization process includes two levels: local optimization targets the most recent specified number of keyframes and their observed landmarks within the sliding window, adjusting the state of these variables through bundle adjustments; global optimization is initiated when a loop closure is detected, performing global optimization on the poses of all keyframes and the coordinates of landmarks. Through this incremental optimization strategy, the system continuously improves the accuracy and consistency of the environmental map while ensuring real-time performance, providing a reliable environmental model for long-term accurate positioning.

[0120] In one specific embodiment, the server may be the backend server of the power inspection robot positioning system, such as... Figure 6As shown, the system mainly comprises a multi-sensor data acquisition layer, a multi-sensor data fusion layer, a localization and modeling layer, and a control and communication layer. The multi-sensor data acquisition layer primarily uses the vision sensors, LiDAR, IMU, odometer, and GPS mounted on the power inspection robot to synchronously acquire environmental images, 3D point clouds, inertial data, wheel displacement, and absolute position information in real time, and performs spatiotemporal alignment of the multi-source sensor data. The multi-sensor data fusion layer mainly employs a multimodal fusion algorithm based on extended Kalman filtering to dynamically weight and fuse multi-source sensor data and compensate for errors, generating robot pose estimates with a unified physical scale. The localization and modeling layer is mainly used to construct and optimize an environmental map with physical size attributes, while correcting long-term drift through a loop closure detection mechanism to achieve globally consistent localization and modeling. The control and communication layer mainly uses a CAN bus and 5G or Wi-Fi network to achieve real-time data interaction between the power inspection robot and the server, upload pose information, and schedule inspection tasks. When a localization anomaly is detected, it triggers safety intervention strategies, including but not limited to at least one strategy such as deceleration, attitude re-evaluation, and remote intervention requests, to ensure the safe and reliable execution of inspection tasks.

[0121] like Figure 7The diagram shows a detailed flowchart of the robot localization method. First, in the data initialization phase, the raw sensor data acquired by the odometry, IMU, and camera are processed in parallel, including odometry pre-integration, IMU pre-integration, and visual feature extraction and tracking. The preprocessed data enters the joint initialization phase. The system first determines whether initialization is complete. If not, joint initialization processing is performed, including Structure from Motion (SFM) based on environmental observation data and visual-inertial odometry alignment. Visual SFM recovers the relative pose and 3D feature points of the camera through feature matching of multiple frames, initially establishing a scale-free visual map. Visual-inertial odometry alignment aligns the environmental observation data with the first motion observation data (IMU pre-integration) and the second running observation data (odometry pre-integration), thereby solving for the first and second environmental scale parameters. The first and second environmental scale parameters are then fused to obtain a unified physical environmental scale parameter, which is used to calibrate the environmental observation data. After calibration, initialization is considered complete, and the system enters the tightly coupled backend optimization phase. In this phase, the system maintains a sliding window, performing local optimization only on the data within the window to balance computational efficiency and accuracy, and outputs the robot pose. During this process, the system determines whether to insert a keyframe. If the current frame contributes significantly to localization and mapping, it is inserted into the keyframe database as the basis for subsequent loop closure detection and global optimization. If no keyframe insertion is needed, subsequent frames are processed within the sliding window until a new frame that meets the conditions appears. The keyframe database stores all inserted keyframes and their feature information, serving as comparison samples for loop closure detection. Next, the system determines whether a loop closure is detected, i.e., by matching the features of the current keyframe with historical keyframes to determine if the robot has returned to a previously visited position. If a loop closure is detected, the feature correspondence of the loop closure frame is extracted to confirm the validity of the loop closure. The loop closure detection status is the result identifier for loop closure detection, used to indicate whether the robot has returned to an explored area. If feature recovery is successful, the loop closure detection status is marked as valid loop closure; if feature recovery fails, the loop closure detection status is marked as no loop closure. If a local loop is detected, such as when the robot moves back and forth within a small area, and the looped frame is a recent keyframe within the sliding window, then only the keyframes within the sliding window are optimized locally to quickly correct the accumulated error in a small area. If a global loop is detected, such as when the robot returns to a distant historical area, and the looped frame is an early frame in the keyframe database, then all keyframes and global 3D feature points are optimized all at once to eliminate long-term accumulated positioning errors. It can be understood that keyframes within the sliding window also participate in loop detection; that is, the system matches the features of the latest keyframe within the sliding window with the features of historical keyframes in the keyframe database to determine if a loop exists.Global optimization adjusts the poses of all keyframes, and these optimized poses are synchronously updated to the keyframe database to ensure that the keyframe information in the database is globally optimal. This updated keyframe information will then continue to participate in loop closure detection.

[0122] In practical implementation, the hardware of the power inspection robot positioning system adopts a lightweight carbon fiber body structure and integrates modular sensor mounting interfaces, supporting rapid disassembly and maintenance. The vision camera and LiDAR are mounted on the top of the robot to obtain optimal field of view and scanning range, while the IMU is close to the center mass point to reduce vibration interference. The software is developed using the ROS (Robot Operating System) framework, supporting real-time multi-threaded parallel computing. The system has automatic startup, data calibration, self-recovery, and remote upgrade functions, ensuring long-term autonomous operation in complex outdoor environments.

[0123] To evaluate the positioning results of this embodiment, a Vicon optical motion capture system was built in a laboratory environment as a true reference system. The system setup and data acquisition process are as follows: Figure 8 As shown, the Vicon camera and its support were first installed and secured. Then, the camera parameters were individually adjusted to ensure coverage of the entire robot's motion area. After camera calibration and removal of environmental interference reflections, a precise rigid body model of the robot was established in the system. Once preparation was complete, data acquisition began. The system transmitted millimeter-precision rigid body motion data to the ROS system in real time via the Vicon bridging module. Simultaneously, the ROS data logging package recorded the robot's own sensor data and positioning results, providing synchronous ground truth data for subsequent trajectory comparison and error analysis.

[0124] In a real-world experiment at a 500kV substation, the inspection robot autonomously navigated a pre-defined inspection path approximately 1200 meters long, including straight sections, curves, and obstacle avoidance. The results showed that the average positioning error of the power inspection robot's positioning system was 4.7 cm, with an attitude angle error of less than 0.5°, representing an improvement of approximately 63% in accuracy compared to single-vision positioning. In a mountainous transmission line scenario, even when GPS signals were completely lost, the system maintained continuous and stable positioning, with trajectory drift controlled at 0.8%, superior to the 1.9% of traditional fusion solutions. Furthermore, after 72 hours of continuous operation, the attitude estimation drift was less than 1.2%, demonstrating excellent long-term stability.

[0125] This embodiment achieves high-precision, autonomous, and stable positioning capabilities for power inspection robots in complex environments, demonstrating promising application prospects. It can be widely used in power inspection, transmission line maintenance, substation monitoring, and other industrial automation fields requiring precise positioning.

[0126] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0127] Based on the same inventive concept, this application also provides a robot positioning device for implementing the robot positioning method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more robot positioning device embodiments provided below can be found in the limitations of the robot positioning method described above, and will not be repeated here.

[0128] In one exemplary embodiment, such as Figure 9 As shown, a robot positioning device is provided, comprising:

[0129] The data acquisition module 902 is used to acquire environmental observation data and motion observation data of the target robot during its movement. The motion observation data includes first motion observation data characterizing the inertial motion increment of the target robot and second motion observation data characterizing the displacement increment of the target robot.

[0130] The joint initialization processing module 904 is used to perform joint initialization processing on the environmental observation data and the first motion observation data and the second motion observation data respectively to obtain the first environmental scale parameter and the second environmental scale parameter. The first environmental scale parameter is obtained based on the environmental scale estimation of inertial motion, and the second environmental scale parameter is obtained based on the environmental scale estimation of displacement.

[0131] The scale parameter acquisition module 906 is used to fuse the first environmental scale parameter and the second environmental scale parameter to obtain the physical environmental scale parameter;

[0132] The data conversion module 908 is used to convert environmental observation data into target environmental observation data with physical size attributes based on physical environmental scale parameters.

[0133] The data fusion module 910 is used to obtain the positioning result of the target robot by fusing the target environment observation data, the first motion observation data and the second motion observation data.

[0134] In one embodiment, the data fusion module 910 is further configured to: perform environmental adaptability analysis on the target environment observation data, the first motion observation data, and the second motion observation data respectively, to obtain the environmental adaptability analysis results of the target environment observation data, the first motion observation data, and the second motion observation data respectively; determine the fusion weights of the target environment observation data, the first motion observation data, and the second motion observation data respectively based on the environmental adaptability analysis results; and fuse the target environment observation data, the first motion observation data, and the second motion observation data based on the fusion weights.

[0135] In one embodiment, the robot localization device is further configured to: predict the pose of the target robot based on the first motion observation data and the second motion observation data to obtain the pose prediction result of the target robot; extract the data features of the target environment observation data, and perform pose matching of the target robot based on the data features to obtain the pose observation result of the target robot; the data fusion module 910 is further configured to: perform weighted fusion of the pose prediction result and the pose observation result based on each fusion weight to obtain the localization result of the target robot.

[0136] In one embodiment, the robot localization device is further configured to: perform error analysis on the pose prediction result and the pose observation result to obtain the error analysis result between the pose prediction result and the pose observation result; based on the error analysis result, perform error compensation on the pose prediction result and the pose observation result to obtain the compensated pose prediction result and the compensated pose observation result; the data fusion module 910 is further configured to: perform weighted fusion of the compensated pose prediction result and the compensated pose observation result based on each fusion weight to obtain the localization result of the target robot.

[0137] In one embodiment, the robot localization device is further configured to: construct an environmental map of the target robot based on target environment observation data and the localization result of the target robot; perform loop closure detection on the target robot in the environmental map to obtain loop closure detection results; and optimize the localization result of the target robot based on the loop closure detection results to obtain optimized localization results.

[0138] In one embodiment, the robot localization device is further configured to: construct a preliminary environmental map of the target robot based on the target environment observation data and the localization result of the target robot; acquire the environmental observation data at the current moment; and optimize the preliminary environmental map based on the environmental observation data at the current moment, provided that the environmental observation data at the current moment meets the map construction conditions, to obtain an optimized environmental map.

[0139] Each module in the aforementioned robot positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0140] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores robot localization data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a robot localization method.

[0141] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0143] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0144] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0148] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A robot localization method, characterized in that, The method includes: The environmental observation data and motion observation data of the target robot during its movement are acquired. The motion observation data includes first motion observation data characterizing the inertial motion increment of the target robot and second motion observation data characterizing the displacement increment of the target robot. The environmental observation data is jointly initialized with the first motion observation data and the second motion observation data to obtain a first environmental scale parameter and a second environmental scale parameter. The first environmental scale parameter is obtained based on the environmental scale estimation of inertial motion, and the second environmental scale parameter is obtained based on the environmental scale estimation of displacement. The first environmental scale parameter and the second environmental scale parameter are fused to obtain the physical environmental scale parameter; Based on the physical environment scale parameters, the environmental observation data is converted into target environmental observation data with physical size attributes; The localization result of the target robot is obtained by fusing the target environment observation data, the first motion observation data, and the second motion observation data.

2. The method according to claim 1, characterized in that, The step of fusing the target environment observation data, the first motion observation data, and the second motion observation data to obtain the localization result of the target robot includes: Environmental adaptability analysis is performed on the target environment observation data, the first motion observation data, and the second motion observation data respectively to obtain the environmental adaptability analysis results of the target environment observation data, the first motion observation data, and the second motion observation data. Based on the environmental adaptability analysis results, the fusion weights of the target environment observation data, the first motion observation data, and the second motion observation data are determined. Based on the fusion weights, the target environment observation data, the first motion observation data, and the second motion observation data are fused.

3. The method according to claim 2, characterized in that, The method further includes: Based on the first motion observation data and the second motion observation data, the pose of the target robot is predicted to obtain the pose prediction result of the target robot. Extract the data features from the target environment observation data, perform target robot pose matching based on the data features, and obtain the pose observation results of the target robot; The process of fusing the target environment observation data, the first motion observation data, and the second motion observation data based on the respective fusion weights includes: Based on the fusion weights, the pose prediction result and the pose observation result are weighted and fused to obtain the localization result of the target robot.

4. The method according to claim 3, characterized in that, The method further includes: An error analysis is performed on the pose prediction result and the pose observation result to obtain the error analysis result between the pose prediction result and the pose observation result; Based on the error analysis results, error compensation is performed on the pose prediction results and the pose observation results to obtain the compensated pose prediction results and the compensated pose observation results. The step of weightedly fusing the pose prediction result and the pose observation result based on the fusion weights to obtain the localization result of the target robot includes: Based on the fusion weights, the compensated pose prediction result and the compensated pose observation result are weighted and fused to obtain the localization result of the target robot.

5. The method according to claim 1, characterized in that, The method further includes: Based on the target environment observation data and the target robot's localization results, an environmental map of the target robot is constructed; In the environmental map, loop closure detection is performed on the target robot to obtain the loop closure detection results; Based on the loop closure detection results, the localization results of the target robot are optimized to obtain optimized localization results.

6. The method according to claim 5, characterized in that, The step of constructing an environmental map of the target robot based on the target environment observation data and the target robot's localization results includes: Based on the target environment observation data and the target robot's localization results, a preliminary environmental map of the target robot is constructed; Obtain environmental observation data at the current moment; If the environmental observation data at the current moment meets the map construction conditions, the preliminary environmental map is optimized based on the environmental observation data at the current moment to obtain an optimized environmental map.

7. A robot positioning device, characterized in that, The device includes: The data acquisition module is used to acquire environmental observation data and motion observation data of the target robot during its movement. The motion observation data includes first motion observation data characterizing the inertial motion increment of the target robot and second motion observation data characterizing the displacement increment of the target robot. The joint initialization processing module is used to perform joint initialization processing on the environmental observation data with the first motion observation data and the second motion observation data respectively to obtain a first environmental scale parameter and a second environmental scale parameter, wherein the first environmental scale parameter is obtained based on the environmental scale estimation of inertial motion, and the second environmental scale parameter is obtained based on the environmental scale estimation of displacement. The scale parameter acquisition module is used to fuse the first environmental scale parameter and the second environmental scale parameter to obtain the physical environmental scale parameter; The data conversion module is used to convert the environmental observation data into target environmental observation data with physical size attributes based on the physical environment scale parameters. The data fusion module is used to fuse the target environment observation data, the first motion observation data, and the second motion observation data to obtain the positioning result of the target robot.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images