Boundary-free intelligent mowing robot based on visual fusion positioning

Abstract

The invention discloses a boundary-free intelligent mowing robot based on visual fusion positioning, and the robot comprises a multi-sensor fusion unit which fuses RTK data, visual information, and IMU inertial data; the RTK positioning module is used for acquiring RTK data through GNSS observation data and a factor graph optimization method; the visual SLAM module is used for realizing real-time positioning and map construction in a dynamic scene on the basis of the introduced RTK data in combination with visual information and IMU inertial data; the path planning module is used for generating a full-coverage mowing path according to RTK global positioning and a visual SLAM local map; and the cloud control interface supports remote setting of the mowing period, real-time monitoring of the state and exception handling through an APP. According to the invention, centimeter-level high-precision positioning can be realized, the positioning requirement in a complex environment is met, all-around spatial perception and mapping are realized, the mowing efficiency is improved, and the mowing requirements of different users are met.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent lawn mowing robots, and particularly relates to a borderless intelligent lawn mowing robot based on visual fusion positioning. Background Art

[0002] An intelligent lawn mowing robot is an outdoor service robot integrating functions such as environmental perception, autonomous navigation, path planning, and automatic obstacle avoidance, and is mainly used for lawn trimming and maintenance in scenarios such as family courtyards and public green spaces. Compared with traditional lawn mowers, it can achieve automatic weeding without manual control.

[0003] Although existing intelligent lawn mowing robots can perform automatic mowing, their positioning effect is not good, which often leads to incomplete mowing in the designated area, or mowing operations in non-designated areas, and the reliability of their mowing is questionable. Summary of the Invention

[0004] The purpose of the present invention is to provide a borderless intelligent lawn mowing robot based on visual fusion positioning to solve the problems in the background art.

[0005] To achieve the above purpose, the present invention provides the following technical solutions: A borderless intelligent lawn mowing robot based on visual fusion positioning, comprising: A multi-sensor fusion unit that fuses RTK data, visual information, and IMU inertial data; An RTK positioning module for obtaining RTK data through GNSS observation data and factor graph optimization methods; A visual SLAM module for achieving real-time positioning and map construction in a dynamic scene based on the introduced RTK data, combined with visual information and IMU inertial data; A path planning module that generates a full-coverage mowing path according to RTK global positioning and visual SLAM local maps; A cloud control interface that supports remotely setting the mowing cycle, real-time monitoring of the status, and exception handling through an APP.

[0006] Preferably, the factor graph optimization method separates speed and position estimation and performs RTK positioning using a two-stage processing method.

[0007] Preferably, the RTK positioning based on the factor graph optimization method includes a speed estimation stage and a position estimation stage. Combining the observation data and navigation data of the base station to correct the pseudorange in the GNSS observation data; Using the Doppler effect to perform a preliminary estimation of the speed, and correcting the speed in combination with the receiver clock drift, including outlier rejection and interpolation processing; Use the corrected speed as a loose constraint condition, and combine the corrected pseudorange and the carrier phase data in the GNSS observation data for position estimation.

[0008] Preferably, in the position estimation, an M-estimator based on the Huber function is used to optimize the parameter factors.

[0009] Preferably, Akima interpolation method is used for the interpolation of the speed.

[0010] Preferably, in the RTK positioning, an error model based on the satellite elevation angle is used as a factor.

[0011] Preferably, the visual SLAM module includes: a rotation-invariant feature point extraction unit for stable feature matching in dynamic scenes; an IMU pre-integration unit for pre-integrating inertial data on a manifold and jointly optimizing with the visual reprojection error; a semantic segmentation unit for distinguishing the lawn boundary and obstacles through LSM.

[0012] Preferably, the visual SLAM module enhances global positioning in the following ways: adding RTK data as an absolute position factor to the factor graph of visual-inertial SLAM; fusing the global position constraint of RTK in key frames to correct the cumulative error.

[0013] Preferably, the path planning module supports the multi-map reuse function: automatically loading and stitching maps of different mowing areas based on the RTK positioning result; correcting the map drift error through loop detection and RTK absolute position; dynamically adjusting the density of the mowing path according to the lawn growth rate.

[0014] Preferably, the positioning method of the intelligent mowing robot includes the steps of: obtaining the original pseudorange, pseudorange rate and ADR data through a general GNSS module; optimizing and fusing the Doppler velocity estimation and pseudorange position correction using a factor graph; tightly coupling and optimizing the RTK positioning result and the visual SLAM pose; outputting a global-local joint positioning result with centimeter-level accuracy.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The mowing robot of the present invention can achieve centimeter-level high-precision positioning, meet the positioning requirements in complex environments, realize all-round spatial perception and mapping, improve the mowing efficiency, and meet the mowing needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the frame structure of the present invention.

[0017] Figure 2 It is a flow chart of RTK positioning based on the factor graph optimization method in the present invention.

[0018] Figure 3 A factor graph for speed estimation in RTK.

[0019] Figure 4 A factor graph for position estimation in RTK.

[0020] Figure 5 An overall framework diagram of a visual SLAM model. Specific implementation manners

[0021] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts fall within the protection scope of the present invention.

[0022] Referring to Figure 1 , the present invention provides a borderless intelligent lawn mowing robot based on visual fusion positioning, including: A multi-sensor fusion unit for data fusion, including RTK data, visual information, and IMU inertial data; An RTK positioning module for obtaining RTK data through GNSS observation data and factor graph optimization methods to achieve centimeter-level RTK global positioning; A visual SLAM module for realizing real-time positioning and map construction in a dynamic scene based on the introduced RTK data in combination with visual information and IMU inertial data; A path planning module for generating a full-coverage mowing path according to RTK global positioning and the visual SLAM local map; A cloud control interface that supports remotely setting the mowing cycle, real-time monitoring of the status, and exception handling through an APP.

[0023] In this multi-sensor fusion unit, visual data provides relative position information of the local environment, IMU data provides motion state estimation, and RTK data provides global position information. By combining these data, the intelligent lawn mower can achieve robust high-precision positioning in different environments (such as indoor and outdoor, dynamically changing scenes). Considering the requirement of low cost, when combining RTK data, it is necessary to balance real-time performance and computational load. It is possible to use RTK data at a lower frequency or correct RTK information at key frames to reduce computational complexity.

[0024] In the present invention, the RTK positioning module includes general GNSS positioning. However, the observation data of consumer-grade GNSS modules are often affected by significant noise, and there are often data missing and outliers, resulting in difficult-to-guarantee positioning accuracy. For DTU devices, their observation data fluctuates greatly. Especially when integrating with RTK technology to achieve precise positioning, they face multiple challenges: (1) The pseudorange noise is large and the GNSS observation quality is poor; (2) Problems such as data missing, duplication, and time jumps frequently occur, increasing the processing difficulty. Therefore, the present invention provides a new method based on factor graph optimization (FGO, Factor Graph Optimization) for RTK position estimation.

[0025] Factor graph optimization is an optimization method based on graph theory. By constructing a factor graph model, the observation data and prior knowledge are transformed into nodes and edges in the graph, and then the most likely state is solved through an optimization algorithm. The factor graph optimization method can apply various complex non-linear constraints and optimize all state variables (the entire movement trajectory) simultaneously. There are many outliers among various constraint conditions (edges in the graph), but robust optimization techniques do not require manual setting of outlier threshold parameters.

[0026] In this embodiment, referring to Figure 3 as shown, the RTK position estimation method based on factor graph optimization includes a speed estimation stage and a position estimation stage; in the speed estimation stage, the speed is globally optimized through Doppler factors and acceleration constraints, and in the position estimation stage, the base station corrected pseudorange, ADR time difference, and speed estimation results are fused, and the position is robustly optimized through the M-estimator of the Huber function.

[0027] (Speed Estimation Stage) Referring to Figure 3 , the Doppler factor is introduced here to construct a factor graph containing speed state variables, with the motion acceleration as a constraint. A global optimization method based on the factor graph (Georgia Tech Smoothing and Mapping, GTSAM) is used for speed estimation. The estimated speed is corrected by outlier rejection and interpolation processing in combination with the receiver clock drift. The Akima interpolation method is used here for interpolation processing to avoid overshoot problems.

[0028] (Position Estimation Stage) Referring to Figure 4, define state nodes {x1, x2, …, xn} based on time series; construct multi-source factors, including pseudorange factors, Doppler factors, ADR factors, etc.; and use the corrected velocity in the velocity estimation stage as a loose constraint condition in position estimation, that is, between each state (position), the estimated velocity is used to constrain the relative position, and a high-precision relative position constraint is added when ADR is effective; at the same time, use the pseudorange corrected by the base station as an absolute position constraint, and use the M-estimator with the Huber function to perform global optimization and solution based on the factor graph to obtain the position estimation result. This position estimation does not require post-processing, and the optimized estimated position will be used as the final estimated value. In this position estimation stage, use the error model based on satellite elevation angle as the error function of the factor, with higher accuracy.

[0029] In the present invention, although the accurate relative position (velocity) can be calculated by the time difference of the Accumulated Doppler Range (ADR), the preliminary estimation of the velocity using the Doppler effect has higher reliability.

[0030] In the present invention, this factor graph optimization method separates the velocity and position estimations and adopts a two-stage processing method, thereby significantly improving the positioning accuracy. The velocity estimation method can also be used to determine whether the lawn mowing robot is in a stopped state. If the velocity and position are simultaneously incorporated into the optimization process, the determination of interpolation and stop positions may become complicated, especially in the case of a large number of missing values, outliers, altitude fluctuations, or satellite occlusions in the data. Therefore, estimating and correcting the velocity in a separate stage is an effective strategy to improve the overall optimization effect.

[0031] In the present invention, with reference to Figure 5 , in order to improve the real-time perception ability of the lawn mowing robot in dynamic scenarios, combine SLAM (Simultaneous Localization and Mapping) and lightweight visual recognition technology. Specifically, this visual SLAM module includes: A rotation-invariant feature point extraction unit for stable feature matching in dynamic scenarios; An IMU pre-integration unit for pre-integrating inertial data on the manifold and jointly optimizing with the visual reprojection error; A semantic segmentation unit for distinguishing the lawn boundary and obstacles through the LSM (Lawn Semantic Model).

[0032] In this visual SLAM module, by introducing rotation-invariant feature points and descriptors, real-time localization and map construction in complex dynamic environments are achieved. This method is compatible with monocular cameras, binocular cameras, and RGB-D cameras, facilitating the expansion and upgrade of the system. It performs particularly well in SLAM systems that combine a monocular camera with an IMU (Inertial Measurement Unit) or a binocular camera with an IMU.

[0033] In this visual SLAM module, regarding the tight coupling of visual information and the IMU, the IMU pre-integration method is adopted. By performing pre-integration on the manifold and then combining it with the reprojection model of VSLAM for graph optimization, the pre-integration residual and the reprojection error are jointly minimized, thereby achieving more accurate pose estimation.

[0034] The present invention combines VSLAM (Visual SLAM) and LSM (Lawn Segment Model) for semantic modeling of courtyard lawns to address problems such as poor robustness and low pose estimation accuracy of traditional SLAM algorithms in dynamic scenarios.

[0035] Furthermore, the borderless intelligent lawn mowing robot of the present invention also integrates a fast initialization method and uses an inertial navigation system to achieve fast re-localization after losing tracking, ensuring continuous and stable operation in dynamic scenarios.

[0036] In this visual SLAM module, based on the factor graph optimization framework, different sensor data are fused. At the same time, the absolute position information in the RTK positioning module is introduced, and the absolute position factor is added to the factor graph of visual-inertial SLAM. The global position constraint of RTK is fused in the key frames to correct the cumulative error, so as to achieve more accurate position estimation. During the factor graph optimization process, the RTK factor helps correct the overall position estimation. Especially when the visual data is sparse or unreliable, the role of the RTK factor is particularly important.

[0037] The path planning module supports the function of multi-map reuse: Based on the RTK positioning result, automatic loading and stitching of maps of different mowing areas are realized; The map drift error is corrected through loop detection and RTK absolute position; The density of the mowing path is dynamically adjusted according to the lawn growth rate.

[0038] By introducing high-precision RTK data into the multi-map fusion process, the intelligent lawn mower can more accurately align its pose between different maps. This pose alignment not only enhances the system's adaptability in large-scale and complex environments but also ensures that the lawn mower can use the same set of precise environmental models for navigation and decision-making during multiple operations. In addition, RTK data also provides important reference for loop closure and map correction processes. By introducing RTK data, the intelligent lawn mower can more reliably identify loop closure opportunities and use the absolute position information provided by RTK to correct the cumulative errors in the map. This process makes the finally generated map more accurate and consistent, ensuring more stable and reliable navigation of the lawn mower in complex environments. At the same time, the intelligent lawn mower robot can seamlessly fuse maps constructed at different times and different locations, which means that maps constructed in different operation tasks can be reused and integrated, thus avoiding repeated map construction, saving computing resources, and improving operation efficiency.

Claims

1. A borderless intelligent lawn mowing robot based on visual fusion positioning, characterized in that: Multi-sensor fusion unit, integrating RTK data, visual information and IMU inertial data; RTK positioning module, used to obtain RTK data through GNSS observation data and factor graph optimization method; Visual SLAM module, which is used to realize real-time positioning and map construction in dynamic scenes based on the imported RTK data and combined with visual information and IMU inertial data; Path planning module, which generates a full-coverage mowing path based on RTK global positioning and visual SLAM local map; Cloud control interface supports remote setting of mowing cycle, real-time monitoring status and exception handling through APP.

2. The borderless intelligent lawn mowing robot based on visual fusion positioning as claimed in claim 1, characterized in that: The factor graph optimization method separates velocity and position estimation and adopts a two-stage processing approach for RTK positioning.

3. The borderless intelligent lawn mowing robot based on visual fusion positioning as claimed in claim 2, characterized in that: The RTK positioning based on the factor graph optimization method includes a velocity estimation stage and a position estimation stage. Correct the pseudo-range in GNSS observation data by combining the observation data and navigation data of the base station; The Doppler effect is used to make a preliminary estimate of the velocity, and the velocity is corrected in combination with the receiver clock drift, including outlier removal and interpolation processing; The corrected speed is taken as a loose constraint, and the position is estimated by combining the corrected pseudorange and the carrier phase data in the GNSS observation data.

4. The borderless intelligent lawn mowing robot based on visual fusion positioning as claimed in claim 1, characterized in that: In the position estimation, an M estimator based on the Huber function is used to optimize parameter factors.

5. The borderless intelligent lawn mowing robot based on visual fusion positioning as claimed in claim 3, characterized in that: The velocity is interpolated using the Akima interpolation method.

6. The borderless intelligent lawn mowing robot based on visual fusion positioning as claimed in claim 3, characterized in that: In the RTK positioning, an error model based on satellite elevation angle is used as an error model of factors.

7. The borderless intelligent lawn mowing robot based on visual fusion positioning as claimed in claim 1, characterized in that: The visual SLAM module includes: Rotation-invariant feature point extraction unit for stable feature matching in dynamic scenes; IMU pre-integration unit, which pre-integrates inertial data on the manifold and jointly optimizes it with the visual reprojection error; The semantic segmentation unit distinguishes lawn boundaries and obstacles through LSM.

8. The borderless intelligent lawn mowing robot based on visual fusion positioning as claimed in claim 1, characterized in that: The visual SLAM module enhances global positioning by: Add RTK data as absolute position factor to the factor graph of visual inertial SLAM; The global position constraints of RTK are fused in the keyframes to correct the accumulated errors.

9. The borderless intelligent lawn mowing robot based on visual fusion positioning as claimed in claim 1, characterized in that: The path planning module supports multi-map multiplexing function: Automatically load and stitch maps of different mowing areas based on RTK positioning results; Correct map drift errors through closed-loop detection and RTK absolute position; Dynamically adjust mowing path density based on lawn growth rate.

10. A borderless intelligent lawn mowing robot based on visual fusion positioning as claimed in any one of claims 1 to 9, characterized in that: The positioning step of the intelligent lawn mowing robot includes: Obtain raw pseudorange, pseudorange rate and ADR data through the universal GNSS module; Utilize factor graph to optimize the fusion of Doppler velocity estimation and pseudorange position correction; Tightly couple and optimize the RTK positioning results with the visual SLAM pose; Output global-local joint positioning results with centimeter-level accuracy.

Images