Mower automatic cruise path planning method based on IMU and camera

Through the automatic cruise path planning method of IMU and camera, the problem of insufficient navigation accuracy and obstacle avoidance capabilities of traditional intelligent lawn mowers in complex environments is solved, high-precision navigation and obstacle avoidance are achieved, and the safety of lawn mowers is improved and user experience and optimize navigation strategies through machine learning.

CN118938907APending Publication Date: 2024-11-12ZHEJIANG YAT ELECTRICAL APPLIANCE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410982934.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Traditional smart lawn mowers lack navigation accuracy and obstacle avoidance capabilities in complex environments, especially in weak GPS signal areas and dense obstacles, which are prone to collisions and shutdown accidents, and insufficient data collection, making it difficult to generate high-precision thermal maps for use in Optimize navigation.

Method used

Using an automatic cruise path planning method based on IMU and camera, through initial data acquisition, real-time obstacle avoidance, data upload and processing, thermal map generation, path planning and real-time path adjustment, combined with machine learning algorithms to optimize navigation strategies, and use cameras and IMUs for Obstacle detection and path adjustment to generate high-precision thermal maps for optimized navigation.

Benefits of technology

It realizes high-precision navigation and obstacle avoidance in complex environments, improves the safety and flexibility of lawn mowers, and continuously optimizes path planning through machine learning, improving work efficiency and user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118938907A_ABST
    Figure CN118938907A_ABST
Patent Text Reader

Abstract

The invention discloses a lawn mower automatic cruise path planning method based on an IMU and a camera, and the method comprises the steps: enabling a lawn mower to randomly move for multiple times in a lawn region, and collecting the GPS signal intensity, position coordinates and environment data in real time; a camera and an IMU unit are used for obstacle detection and obstacle avoidance; generating a signal strength thermodynamic diagram according to the acquired GPS signal strength data; dividing a lawn area into a plurality of grid units, and setting priorities according to the signal intensity and terrain complexity of the thermodynamic diagram; by adopting a path generation algorithm, signal weak areas and obstacles are avoided as much as possible on the basis of ensuring lawn coverage; in the running process of the mower, the current GPS signal intensity and the surrounding environment are monitored in real time, and a path is dynamically adjusted according to signal changes and obstacle conditions. According to the invention, data can be collected efficiently, path planning is carried out according to the GPS signal intensity thermodynamic diagram, weak signal areas and obstacles are avoided, and normal work of the mower is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of lawn mowers, and in particular to an automatic cruising path planning method for a lawn mower based on an IMU and a camera. Background Art

[0002] With the continuous development of smart home and automation technology, smart lawn mowers have gradually become an important tool for modern garden maintenance. However, there are still many challenges in the navigation and positioning of traditional smart lawn mowers in complex environments. Especially in areas with weak GPS signals and dense obstacles, the navigation accuracy and obstacle avoidance capabilities of existing technologies are often insufficient.

[0003] Traditional smart lawn mowers are highly dependent on signals and GPS signals. The navigation accuracy will decrease in areas with weak signals or blind spots (such as shadows of buildings and dense forests). Especially in areas with dense obstacles and complex terrain, the obstacle avoidance and path adjustment capabilities of the lawn mower are limited. The lawn mower is prone to collisions, shutdowns and other accidents, which have an impact. At the same time, traditional smart lawn mowers do not collect sufficient data and lack comprehensive environmental data collection and analysis, making it difficult to generate high-precision heat maps for optimized navigation. In existing solutions, users have limited real-time monitoring and path adjustment capabilities for lawn mowers, which affects the user experience. Summary of the invention

[0004] In order to solve the above problems, the present invention provides an automatic cruising path planning method for a lawn mower based on IMU and camera, which can comprehensively collect data and avoid obstacles in real time during the initial operation, and generate high-precision thermal maps to optimize navigation.

[0005] To this end, the technical solution of the present invention is: a lawn mower automatic cruising path planning method based on IMU and camera, comprising the following steps:

[0006] S1. Initial data collection:

[0007] S1.1, the lawn mower moves randomly multiple times in the lawn area, collecting GPS signal strength, location coordinates and environmental data in real time;

[0008] S1.2, during data collection, use cameras and IMU units for obstacle detection and avoidance;

[0009] S2, data upload and processing: after the lawn mower completes data collection, it uploads the data to the cloud server or local storage device; the collected data is cleaned and preprocessed to generate a preliminary signal strength distribution map;

[0010] S3, generating a heat map: generating a signal strength heat map according to the collected GPS signal strength data;

[0011] S4. Generate path planning:

[0012] S4.1. Divide the lawn area into several grid cells and evaluate each grid cell according to the signal strength and terrain complexity of the heat map;

[0013] S4.2. Set priorities based on the signal strength of the grid cells, giving priority to areas with higher signal strength;

[0014] S4.3. Use a path generation algorithm to avoid weak signal areas and obstacles as much as possible while ensuring lawn coverage;

[0015] S5. Real-time path adjustment:

[0016] S5.1. The lawn mower moves along the path planned in step S4. During operation, the current GPS signal strength and surrounding environment are monitored in real time, and the path is dynamically adjusted according to signal changes and obstacles.

[0017] S5.2: If the lawn mower enters a weak signal area or encounters a new obstacle, local path replanning is triggered and the process returns to step S4.3 to recalculate and adjust the mowing path.

[0018] On the basis of the above solution and as a preferred solution of the above solution: it also includes step S6, creating a path optimization model:

[0019] S6.1. Collect data during the operation of the lawn mower, including path, signal strength, and obstacle avoidance records, and establish a data set;

[0020] S6.2. Use machine learning algorithms to train models and learn optimal path planning and obstacle avoidance strategies;

[0021] S6.3. Apply the trained model to the path planning and navigation of the lawn mower to continuously optimize its performance.

[0022] On the basis of the above scheme and as the preferred scheme of the above scheme: an operation interface is set on the mobile terminal to view the location, status and environmental data of the lawn mower in real time, and display the generated GPS signal strength heat map. Users can provide feedback and suggestions through the operation interface to adjust the mowing path and navigation strategy.

[0023] On the basis of the above scheme and as a preferred scheme of the above scheme: in step S1.1, the random movement step is as follows:

[0024] S1.1.1. Use a pseudo-random number generation algorithm to randomly select a direction between 0° and 360° and record the selected direction;

[0025] S1.1.2, move the preset step length in the selected direction;

[0026] S1.1.3. At the end of each movement, record the current position, GPS signal strength, IMU data, and camera image.

[0027] On the basis of the above scheme and as a preferred scheme of the above scheme: in step S1.2, the steps of obstacle detection and obstacle avoidance are as follows:

[0028] S1.2.1. Use the camera to collect the front image, and perform denoising and smoothing preprocessing operations on the collected image;

[0029] S1.2.2. Use computer vision algorithms to detect obstacles in front of you and estimate the distance to the obstacles based on the camera’s viewing angle and the position of the objects in the image.

[0030] S1.2.3, using the IMU data to estimate the current posture of the lawn mower, and combining the IMU data with the GPS data to estimate the current speed and acceleration of the lawn mower;

[0031] S1.2.4, combine the image processing results of the camera and the IMU data to determine whether there is an obstacle;

[0032] S1.2.5. Adjust the path of the mower according to the location and distance of obstacles to avoid collision.

[0033] On the basis of the above scheme and as a preferred scheme of the above scheme: in step S2, data uploading and processing includes the following steps:

[0034] S2.1, storing the collected GPS signal strength, location coordinates, IMU data and camera images in a local storage device;

[0035] S2.2, upload the collected data to the cloud server or local computer through the communication module;

[0036] S2.3. Clean the collected data, remove redundant and invalid data, and handle missing data and outliers;

[0037] S2.4. Preprocess the collected data and perform normalization and smoothing operations.

[0038] On the basis of the above solution and as a preferred solution of the above solution: in step S3, the steps of generating the thermal map are as follows:

[0039] S3.1. Grid division: divide the lawn area into several grid units;

[0040] S3.2, Signal strength statistics: Calculate the average GPS signal strength in each grid cell;

[0041] S3.3. Heat map drawing: Based on the signal strength of each grid cell, a visual heat map is generated to show the signal strength distribution.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. It can collect data efficiently, and fully cover the lawn area through initial random movement and real-time obstacle avoidance to ensure the comprehensiveness and reliability of data collection; based on the collected GPS signal strength data, it generates a signal strength heat map, and plans the path based on the strength heat map. On the basis of ensuring lawn coverage, it tries to avoid weak signal areas and obstacles to ensure the normal operation of the lawn mower.

[0044] 2. It can avoid obstacles and adjust the path in real time, use the camera and IMU for dynamic obstacle avoidance and path correction, monitor the current GPS signal strength and surrounding environment in real time, and dynamically adjust the path according to signal changes and obstacles. When the lawn mower enters a weak signal area or encounters a new obstacle, it can trigger local path replanning, recalculate and adjust the mowing path, greatly improving the safety and flexibility of the lawn mower.

[0045] 3. Use machine learning algorithms to train models based on the collected data to learn the best path planning and obstacle avoidance strategies. Through continuous operation and data accumulation, the lawn mower can gradually learn and adapt to different environments and improve work efficiency.

[0046] 4. A mobile operating interface is set up, such as a mobile phone APP. Users can view the location, status and environmental data of the lawn mower in real time through the App, and display the generated GPS signal strength heat map in the App. Users can also provide feedback and suggestions through the App to adjust the mowing path and navigation strategy to improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0048] See the attached drawings. The automatic cruising path planning method for a lawn mower based on IMU and camera described in this embodiment is run on an intelligent lawn mower, which at least needs to include the following hardware:

[0049] Lawn mower body: Installs the motor, wheels, power system and control system for mowing;

[0050] IMU unit: used to detect the attitude and motion state of the lawn mower, including accelerometer, gyroscope and magnetometer;

[0051] Camera: used for real-time visual data collection, environmental perception and obstacle detection;

[0052] GPS module: used to obtain the location information of the lawn mower;

[0053] Computing unit: used to process sensor data and make decisions, including obstacle avoidance and path planning. Embedded computing platforms (such as Raspberry Pi, NVIDIA Jetson, etc.) can be used.

[0054] Communication module: used for data transmission and remote control (such as Wi-Fi, Bluetooth, 4G / 5G, etc.);

[0055] Power management system: Provides power to each component to ensure stable system operation.

[0056] Path planning specifically includes the following steps:

[0057] S1. Initial data collection:

[0058] S1.1. Initialize the lawn mower, including:

[0059] Initialize the system: Start the mower and perform a system self-check to ensure that all sensors and modules are working properly;

[0060] Initial position: Set the starting position of the mower;

[0061] Initial direction: Set the initial moving direction of the lawn mower;

[0062] Step size setting: Set the step size of each random move (such as 0.5 meters).

[0063] After initialization, the mower performs multiple random moves in the lawn area, collecting GPS signal strength, location coordinates, and environmental data in real time; the random move steps are as follows:

[0064] S1.1.1. Use a pseudo-random number generation algorithm to randomly select a direction between 0° and 360° and record the selected direction;

[0065] S1.1.2, move the preset step length in the selected direction;

[0066] S1.1.3. At the end of each movement, record the current position, GPS signal strength, IMU data, and camera image.

[0067] S1.2. During data collection, use the camera and IMU unit to detect and avoid obstacles. The steps of obstacle detection and avoidance are as follows:

[0068] S1.2.1. Use the camera to collect the front image, and perform pre-processing operations such as denoising and smoothing on the collected image;

[0069] S1.2.2. Use computer vision algorithms (such as YOLO, SSD, etc.) to detect obstacles in front and estimate the distance of obstacles based on the camera’s viewing angle and the position of objects in the image.

[0070] YOLO is a single-stage object detection algorithm known for its fast and accurate features. The YOLO algorithm converts the object detection task into a regression problem, which achieves fast object detection by dividing the image into grids, predicting a fixed number of bounding boxes for each grid, and outputting the detection results directly in the convolutional neural network.

[0071] SSD is also a representative of single-stage target detection algorithms. Similar to YOLO, SSD performs target detection directly on the feature map of the convolutional neural network, without the need to generate candidate regions and then classify and adjust positions like the two-stage algorithm. SSD can effectively handle targets of different sizes by making predictions on feature maps of different scales, thereby improving the accuracy and speed of detection.

[0072] S1.2.3, use the IMU data to estimate the current posture (pitch angle, roll angle, yaw angle) of the lawn mower, and combine the IMU data with the GPS data to estimate the current speed and acceleration of the lawn mower;

[0073] S1.2.4, combine the image processing results of the camera and the IMU data to determine whether there is an obstacle;

[0074] S1.2.5. Adjust the path of the mower to avoid collisions based on the location and distance of obstacles. That is, if an obstacle is detected, stop the current movement, reselect a random direction, ensure that the new direction avoids the obstacle, adjust the path and continue moving.

[0075] S2, data upload and processing: after the lawn mower completes data collection, it uploads the data to the cloud server or local storage device; the collected data is cleaned and preprocessed to generate a preliminary signal strength distribution map;

[0076] Data uploading and processing includes the following steps:

[0077] S2.1, storing the collected GPS signal strength, location coordinates, IMU data and camera images in a local storage device;

[0078] S2.2, upload the collected data to the cloud server or local computer through the communication module;

[0079] S2.3. Clean the collected data, remove redundant and invalid data, and handle missing data and outliers;

[0080] S2.4. Preprocess the collected data by performing normalization, smoothing and other operations to prepare for subsequent analysis.

[0081] S3, generating a heat map: generating a signal strength heat map based on the collected GPS signal strength data; the steps for generating the heat map are as follows:

[0082] S3.1. Grid division: divide the lawn area into several grid units. The size of the grid can be set according to the size of the lawn area.

[0083] S3.2, Signal strength statistics: Calculate the average GPS signal strength in each grid cell;

[0084] S3.3. Heat map drawing: Based on the signal strength of each grid cell, a visual heat map is generated to show the signal strength distribution.

[0085] S4. Generate path planning: After obtaining the heat map, you can analyze the heat map data:

[0086] Identify weak signal areas: Use heat maps to identify areas with weak GPS signal strength, which may affect the navigation accuracy of the lawn mower. Specifically, the heat map color distribution can be used to identify areas where the signal strength is below the set threshold.

[0087] Evaluate signal coverage: Check the heat map for areas of uneven signal coverage to ensure the mower has good signal reception across the lawn area so that weak signal areas do not affect the main mowing path.

[0088] According to the above analysis conclusions, the path planning strategy is generated:

[0089] S4.1. Grid division: The lawn area is divided into several grid cells, and each grid cell is evaluated according to the signal strength and terrain complexity of the heat map;

[0090] S4.2, Priority setting: Set the priority according to the signal strength of the grid unit, and give priority to the area with higher signal strength;

[0091] S4.3. Path generation: Use a path generation algorithm (such as A* algorithm, Dijkstra algorithm, etc.) to avoid signal weak areas and obstacles as much as possible while ensuring lawn coverage.

[0092] Dijkstra's algorithm is a typical shortest path algorithm, which is used to find the shortest path from the starting point to all other points. Dijkstra's algorithm prioritizes which paths to explore. Instead of exploring all possible paths equally, it prefers low-cost paths. The algorithm can assign lower costs to some paths (such as obstacle-free paths, etc.) and higher costs to some paths (such as obstacle paths, etc.). When the movement costs of different paths are different, use this search instead of breadth-first search.

[0093] The A* algorithm is a modification of Dijkstra's algorithm that optimizes for a single goal. Dijkstra's algorithm finds paths to all locations; A* finds paths to a location or the closest of several locations. It prioritizes paths that appear to be closer to the goal.

[0094] A* algorithm and Dijkstra algorithm are both mature existing technologies and will not be described in detail here.

[0095] S5. Real-time path adjustment:

[0096] S5.1. The lawn mower moves along the path planned in step S4. During operation, the current GPS signal strength and surrounding environment are monitored in real time, and the path is dynamically adjusted according to signal changes and obstacles.

[0097] S5.2: If the lawn mower enters a weak signal area or encounters a new obstacle, local path replanning is triggered, and the process returns to step S4 to recalculate and adjust the mowing path.

[0098] Step S6: Create a path optimization model:

[0099] S6.1. Collect data during the operation of the lawn mower, including path, signal strength, obstacle avoidance records, etc., and establish a data set;

[0100] S6.2. Use machine learning algorithms (such as reinforcement learning, supervised learning, etc.) to train the model and learn the best path planning and obstacle avoidance strategies;

[0101] S6.3. Apply the trained model to the path planning and navigation of the lawn mower to continuously optimize its performance.

[0102] S7. Set up a mobile phone APP associated with the lawn mower. On the mobile phone APP, you can view the location, status and environmental data of the lawn mower in real time, and display the generated GPS signal strength heat map. Users can provide feedback and suggestions through the mobile phone APP to adjust the mowing path and navigation strategy.

[0103] User feedback and improvements:

[0104] Collect user feedback on the operating performance of the lawn mower through the mobile app, record the questions and improvement suggestions raised by users; analyze user feedback, identify common problems and improvement points; adjust path planning and navigation strategies based on feedback, and continuously optimize user experience and mowing results.

[0105] Through the above steps, this embodiment uses GPS heat map data to optimize the path planning and navigation strategy of the lawn mower, which can effectively improve the working efficiency and navigation accuracy of the lawn mower in complex environments. At the same time, through continuous optimization of machine learning and user feedback, the performance and user experience of the lawn mower are further improved.

[0106] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. Automatic cruising path planning method for lawn mower based on IMU and camera, Features: The following steps are involved: S1. Initial data collection: S1.1, the lawn mower moves randomly multiple times in the lawn area, collecting GPS signal strength, location coordinates and environmental data in real time; S1.2, during data collection, use cameras and IMU units for obstacle detection and avoidance; S2, data upload and processing: After the lawn mower completes data collection, it uploads the data to the cloud server or local storage device; Clean and preprocess the collected data to generate a preliminary signal intensity distribution map; S3, generating a heat map: generating a signal strength heat map according to the collected GPS signal strength data; S4. Generate path planning: S4.

1. Divide the lawn area into several grid cells and evaluate each grid cell according to the signal strength and terrain complexity of the heat map; S4.

2. Set priorities based on the signal strength of the grid cells, giving priority to areas with higher signal strength; S4.

3. Use a path generation algorithm to avoid weak signal areas and obstacles as much as possible while ensuring lawn coverage; S5. Real-time path adjustment: S5.

1. The lawn mower moves along the path planned in step S4. During operation, the current GPS signal strength and surrounding environment are monitored in real time, and the path is dynamically adjusted according to signal changes and obstacles. S5.2: If the lawn mower enters a weak signal area or encounters a new obstacle, local path replanning is triggered and the process returns to step S4.3 to recalculate and adjust the mowing path.

2. The automatic cruising path planning method for a lawn mower based on IMU and camera as claimed in claim 1, Features: The method also includes step S6, creating a path optimization model: S6.

1. Collect data during the operation of the lawn mower, including path, signal strength, and obstacle avoidance records, and establish a data set; S6.

2. Use machine learning algorithms to train models and learn optimal path planning and obstacle avoidance strategies; S6.

3. Apply the trained model to the path planning and navigation of the lawn mower to continuously optimize its performance.

3. The automatic cruising path planning method for a lawn mower based on IMU and camera as claimed in claim 1, Features: By setting up an operation interface on the mobile terminal, the location, status and environmental data of the lawn mower can be viewed in real time, and the generated GPS signal strength heat map can be displayed. Users can provide feedback and suggestions through the operation interface to adjust the mowing path and navigation strategy.

4. The automatic cruising path planning method for a lawn mower based on IMU and camera as claimed in claim 1, Features: In step S1.1, the random movement step is as follows: S1.1.

1. Use a pseudo-random number generation algorithm to randomly select a direction between 0° and 360° and record the selected direction; S1.1.2, move the preset step length in the selected direction; S1.1.

3. At the end of each movement, record the current position, GPS signal strength, IMU data, and camera image.

5. The automatic cruising path planning method for a lawn mower based on IMU and camera as claimed in claim 1, Features: In step S1.2, the steps of obstacle detection and obstacle avoidance are as follows: S1.2.

1. Use the camera to collect the front image, and perform denoising and smoothing preprocessing operations on the collected image; S1.2.

2. Use computer vision algorithms to detect obstacles in front of you and estimate the distance to the obstacles based on the camera’s viewing angle and the position of the objects in the image. S1.2.3, using the IMU data to estimate the current posture of the lawn mower, and combining the IMU data with the GPS data to estimate the current speed and acceleration of the lawn mower; S1.2.4, combine the image processing results of the camera and the IMU data to determine whether there is an obstacle; S1.2.

5. Adjust the path of the mower according to the location and distance of obstacles to avoid collision.

6. The automatic cruising path planning method for a lawn mower based on IMU and camera as claimed in claim 1, Features: In step S2, data uploading and processing includes the following steps: S2.1, storing the collected GPS signal strength, location coordinates, IMU data and camera images in a local storage device; S2.2, upload the collected data to the cloud server or local computer through the communication module; S2.

3. Clean the collected data, remove redundant and invalid data, and handle missing data and outliers; S2.

4. Preprocess the collected data and perform normalization and smoothing operations.

7. The automatic cruising path planning method for a lawn mower based on IMU and camera as claimed in claim 1, Features: In step S3, the steps of generating a heat map are as follows: S3.

1. Grid division: divide the lawn area into several grid units; S3.2, Signal strength statistics: Calculate the average GPS signal strength in each grid cell; S3.

3. Heat map drawing: Based on the signal strength of each grid cell, a visual heat map is generated to show the signal strength distribution.