Smart lawnmower and smart lawnmowing system

Abstract

This invention discloses an intelligent lawnmower system, comprising an intelligent lawnmower and a mobile terminal. The mobile terminal includes: an interface for connecting to the intelligent lawnmower for data transmission; multiple sensors for collecting environmental data around the intelligent lawnmower; and a processor configured to perform sensor fusion on the environmental data collected by the multiple sensors, generate control navigation and mowing commands, and send them to the intelligent lawnmower via the interface. The intelligent lawnmower includes: a main body; a fixing device disposed on the main body for fixing the mobile terminal to the intelligent lawnmower; an interface for connecting to the mobile terminal for data transmission; and a controller electrically connected to the interface, which controls the behavior of the intelligent lawnmower according to the commands from the mobile terminal when the interface of the intelligent lawnmower is connected to the interface of the mobile terminal.

Description

Technical Field

[0001] This invention relates to a lawnmower and a lawnmower system, and more particularly, to an intelligent lawnmower and an intelligent lawnmower system. Background Technology

[0002] With the rise and popularization of smart homes, the technology of smart lawnmowers is advancing rapidly, and their acceptance in households is gradually increasing. Because they don't require manual pushing or following, they greatly reduce the labor and time of users. Current smart lawnmowers generally use GPS with ordinary positioning accuracy for area identification, and use boundary line signals and inertial measurement units (IMUs) to calculate accurate locations. However, this approach usually has low positioning accuracy, cannot achieve real-time positioning and navigation, and struggles to achieve efficient path planning and complete area coverage. High-precision positioning solutions, such as RTK based on satellite signals or UWB based on radio signals, have been limited by hardware costs and system reliability. Furthermore, even with high-precision positioning achieved regardless of cost, it is far from sufficient for autonomous smart lawnmowers. Due to a lack of deep understanding of the surrounding environment, lawnmowers cannot effectively handle complex situations such as road surfaces, obstacles, and lighting conditions. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the main objective of this disclosure is to provide a low-cost intelligent lawnmower with higher positioning accuracy and a deeper understanding of the surrounding environment.

[0004] To achieve the above objectives, the present disclosure adopts the following technical solution:

[0005] A smart lawnmower system includes a smart lawnmower and a mobile terminal. The mobile terminal includes: an interface for connecting to the smart lawnmower for data transmission; a sensor for collecting environmental data around the smart lawnmower; and a processor configured to generate control navigation and mowing commands based on the environmental data collected by the sensor and send them to the smart lawnmower via a wired connection. The smart lawnmower includes: a main body; a fixing device disposed on the main body for fixing the mobile terminal to the smart lawnmower; an interface for connecting to the mobile terminal for data transmission; and a controller electrically connected to the interface, which controls the behavior of the smart lawnmower according to the commands from the mobile terminal when the interface of the smart lawnmower is connected to the interface of the mobile terminal.

[0006] Optionally, the fixing device includes a resilient clamping mechanism capable of clamping a mobile terminal with a size of 4 inches to 12 inches.

[0007] Optionally, the fixing device is disposed on the upper surface of the main body.

[0008] Optionally, the interface of the intelligent lawnmower is located on a fixed device and directly mates with the interface of the mobile device.

[0009] Optionally, the interface of the smart lawnmower is located on the upper surface of the main body and is connected to the interface of the mobile terminal via a data cable.

[0010] Optionally, the intelligent lawnmower also includes a receiving cavity disposed in the main body, and the fixing device includes a retractable support structure. When the support structure is in a first state, the fixing device is entirely located inside the receiving cavity, and when the support structure is in a second state, part of the fixing device is located outside the receiving cavity.

[0011] Optionally, the top of the receiving cavity has a cover plate for waterproofing and dustproofing, and the cover plate has a closed state and an open state; when the cover plate is in the closed state, the support structure is in a first state; when the support structure is in a second state, the cover plate is in the open state.

[0012] Optionally, the interface of the smart lawnmower is located in the receiving cavity and is connected to the interface of the mobile terminal via a data cable.

[0013] Optionally, the smart lawnmower has a USB interface and is connected to the interface of the mobile terminal via a data cable.

[0014] Optionally, when the interface of the smart lawnmower is connected to the interface of the mobile terminal, it can charge the mobile terminal.

[0015] Optionally, the smart lawnmower also includes a first power threshold, wherein the remaining power of the smart lawnmower is used to charge the mobile terminal only when the remaining power of the smart lawnmower is greater than the first power threshold.

[0016] Optionally, the smart lawnmower uses a short-range wireless communication interface, and the mobile terminal also uses a short-range wireless communication interface.

[0017] Optionally, the smart lawnmower uses Bluetooth as its interface, as does the mobile terminal.

[0018] Optionally, the smart lawnmower uses a ZigBee interface, and the mobile terminal also uses a ZigBee interface.

[0019] Optionally, the multiple sensors of the mobile terminal include at least a camera and an inertial measurement unit. The processor fuses the visual data acquired by the camera and the pose data acquired by the inertial measurement unit to perform real-time localization and map construction of the smart lawnmower, and generates navigation and mowing action commands.

[0020] Optionally, the intelligent lawn mowing system further includes a cloud server, and the mobile terminal further includes a wireless communication device. The mobile terminal uploads the environmental data collected by the sensor to the cloud server for processing through the wireless communication device, and then generates control navigation and lawn mowing instructions based on the processing results returned by the cloud server and sends them to the intelligent lawn mower.

[0021] A smart lawnmower system includes a smart lawnmower, a mobile terminal, and a cloud server. The mobile terminal includes: a wireless connection device for connecting to the cloud server for data transmission; multiple sensors for collecting environmental data around the smart lawnmower; and a processor configured to upload the environmental data collected by the multiple sensors to the cloud server via the wireless connection device. The cloud server is configured to pair the mobile terminal and the smart lawnmower, perform sensor fusion calculations on the environmental data collected by the multiple sensors uploaded by the mobile terminal, generate control navigation and mowing instructions, and send them to the smart lawnmower via a wired connection. The smart lawnmower includes: a main body; a fixing device disposed on the main body for fixing the mobile terminal to the smart lawnmower; a wireless connection device for connecting to the cloud server for data transmission; and a controller configured to receive instructions from the cloud server via the wireless connection device to control the behavior of the smart lawnmower.

[0022] Optionally, the multiple sensors of the mobile terminal include at least a camera and an inertial measurement unit. The cloud server fuses the visual data acquired by the camera and the pose data acquired by the inertial measurement unit to perform real-time positioning and map construction of the smart lawnmower, and generates control commands for navigation and mowing.

[0023] Optionally, the mobile terminal can move up and down relative to the main body.

[0024] The advantage of this invention is that by fixing and connecting the mobile terminal to the smart lawnmower, the production cost of the smart lawnmower is reduced, while the advantages of the mobile terminal's multiple sensors and processor in positioning, environmental understanding, and computing power are fully utilized by the smart lawnmower. Attached Figure Description

[0025] Figure 1 This is a side view of an embodiment of a smart lawnmower according to this application;

[0026] Figure 2 This is a side view of an embodiment of a smart lawnmower according to this application;

[0027] Figure 3A yes Figure 2 A 3D view of the extendable bracket of the camera on the smart lawnmower shown.

[0028] Figure 3B yes Figure 3A A cross-sectional view of the retractable bracket of the camera of the smart lawnmower shown.

[0029] Figure 3C yes Figure 3A The diagram shows a cross-sectional view of the retractable bracket of the camera on the smart lawnmower during its extension and retraction transformation.

[0030] Figure 4A This is a side view of an intelligent lawnmower according to an embodiment of this application in a non-working state;

[0031] Figure 4B yes Figure 4A The image shows a side view of the smart lawnmower in operation.

[0032] Figure 5A This is a side view of an intelligent lawnmower according to an embodiment of this application in a non-working state;

[0033] Figure 5B yes Figure 5A The image shows a side view of the smart lawnmower in operation.

[0034] Figure 6 yes Figure 1 A schematic diagram of the inertial measurement unit of the intelligent lawnmower shown;

[0035] Figure 7 This is a schematic diagram of the dual inertial measurement unit of an intelligent lawnmower according to an embodiment of this application;

[0036] Figure 8 This is a system schematic diagram of an intelligent lawnmower according to an embodiment of this application;

[0037] Figure 9 This is a flowchart of a Simultaneous Localization and Mapping (SLAM) algorithm according to an embodiment of this application;

[0038] Figure 10 This is a flowchart of a sensor fusion algorithm according to an embodiment of this application;

[0039] Figure 11A This is a display interface in a boundary recognition mode according to an embodiment of this application;

[0040] Figure 11B This is a display interface under another boundary recognition mode according to an embodiment of this application;

[0041] Figure 12 This is a schematic diagram of the road surface recognition and selection function according to an embodiment of this application;

[0042] Figure 13A This is a schematic diagram of an obstacle recognition function according to an embodiment of this application;

[0043] Figure 13BThis is another schematic diagram of the obstacle recognition function according to an embodiment of this application;

[0044] Figure 14 This is a flowchart of an obstacle avoidance algorithm according to an embodiment of this application;

[0045] Figure 15 This is a display interface for setting virtual obstacles according to an embodiment of this application;

[0046] Figure 16 This is a schematic diagram of a smart lawnmower and a camera set in the scene, according to another embodiment of this application;

[0047] Figure 17A yes Figure 16 The diagram shows a data transmission architecture between a smart lawnmower and a camera set up in the scene.

[0048] Figure 17B yes Figure 16 The diagram shows another data transmission architecture between the smart lawnmower and the camera set up in the scene;

[0049] Figure 17C yes Figure 16 The diagram shows the data transmission architecture of the smart lawnmower, the cameras installed in the scene, and the cloud server.

[0050] Figure 18 This is a side view of an intelligent lawn mowing system according to another embodiment of this application;

[0051] Figure 19A yes Figure 18 Side view of the mounting device for the smart lawnmower shown;

[0052] Figure 19B yes Figure 19A The side view of the clamp of the fixing device of the smart lawnmower shown in retraction;

[0053] Figure 19C yes Figure 19A The side view of the clamp of the fixing device of the smart lawnmower shown in the figure when extended;

[0054] Figure 20 This is a side view of an intelligent lawnmower in an intelligent lawnmower system according to another embodiment of this application;

[0055] Figure 21A This is a schematic diagram of the inertial measurement unit of a mobile terminal in an intelligent lawn mowing system according to another embodiment of this application;

[0056] Figure 21B This is a schematic diagram of the camera of a mobile terminal in an intelligent lawn mowing system according to another embodiment of this application;

[0057] Figure 21CThis is a schematic diagram of the interface of a mobile terminal in a smart lawn mowing system according to another embodiment of this application;

[0058] Figure 22A This is a first data transmission architecture diagram of an intelligent lawn mowing system according to another embodiment of this application;

[0059] Figure 22B This is a second data transmission architecture diagram of an intelligent lawn mowing system according to another embodiment of this application;

[0060] Figure 22C This is a third data transmission architecture diagram of an intelligent lawn mowing system according to another embodiment of this application;

[0061] Figure 22D This is a fourth data transmission architecture diagram of an intelligent lawn mowing system according to another embodiment of this application;

[0062] Figure 22E This is a fifth data transmission architecture diagram of an intelligent lawn mowing system according to another embodiment of this application. Detailed Implementation

[0063] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0064] like Figure 1 As shown, this application proposes an intelligent lawnmower 110, including: a cutting blade 112 for cutting grass; a main body 113 for mounting the cutting blade 112; wheels 114 that can rotate and support the main body 113; a light 119 for illumination; a camera assembly 120 for acquiring image information of the lawnmower's surrounding environment; an inertial measurement unit (IMU) 122 for acquiring the lawnmower's pose information; and a processor ( Figure 1 (Not shown in the image), electrically connected to the camera assembly 120 and the inertial measurement unit 122, used for calculating and processing information acquired by the camera assembly 120 and the inertial measurement unit 122; memory ( Figure 1 (Not shown in the image) is used to store the control program 145 for controlling the operation of the intelligent lawnmower 110. The processor can call the control program 145 to fuse the image information of the lawnmower's surrounding environment collected by the camera component 120 and the lawnmower's pose information data collected by the inertial measurement unit 122 to realize Simultaneous Localization and Mapping (SLAM) of the lawnmower, and generate corresponding navigation and mowing commands according to preset logic and real-time data to control the behavior of the intelligent lawnmower 110.

[0065] Optionally, the camera assembly 120 can be mounted on the front of the smart lawnmower 110, see [reference]. Figure 1 The camera assembly 120 installed at the front of the lawnmower 110 can effectively capture image information of the environment in front of the intelligent lawnmower 110. Compared to image information from the sides or rear of the lawnmower, the image information from the front of the lawnmower is more valuable for navigation and obstacle avoidance. Optionally, the camera assembly 120 can also be mounted on the upper front of the lawnmower via a bracket 123, such as... Figure 2 As shown. By raising the bracket 123, the vertical distance between the camera assembly 120 and the ground increases, thereby increasing the field of view of the camera assembly 120 and making the line of sight less likely to be obstructed by near-ground obstacles such as weeds.

[0066] Optionally, the bracket 123 is a telescopic device. For example... Figures 3A-3CThe bracket 123 shown is composed of a telescopic sleeve with a pin 392. The telescopic sleeve with pin 392 consists of two hollow tubes, an inner and an outer one, through which the wires of the camera assembly 120 pass. The outer tube 394 has multiple holes 395 arranged sequentially along its length. The inner tube 391 has a hole, and inside the inner tube 391, perpendicular to the hole, is a pin 392 with a rounded head. The pin 392 is connected to a spring 393, one end of which is fixed to the inner wall of the inner tube 391, and the other end is connected to the bottom of the pin 392, providing an outward force to the pin 392 so that the head of the pin 392 can extend outward through the hole in the inner tube 391 when not pushed by other external forces. When the outer tube 394 is fitted onto the inner tube 391, and one of the multiple sequentially arranged holes 395 on the outer tube 394 is aligned with the hole in the inner tube 391, without external force pushing, the head of the pin 392 will sequentially pass through the hole in the inner tube 391 and the hole 395 on the outer tube 394 that is aligned with the hole in the inner tube 391 and extend outward, thus fixing the outer tube 394 relative to the inner tube 391 in a latching manner. The length adjustment of the bracket 123 is achieved by changing the position of the outer tube 394 of the telescopic sleeve of the pin 392 relative to the inner tube 391: First, overcome the force of the spring 393 itself and press the head of the pin 392 into the inner tube 391. When the head of the pin 392 is approximately on the same plane as the hole 395 on the outer tube 394, quickly slide the outer tube 394 to the ideal position, realign the other hole 395 of the outer tube 394 with the hole of the inner tube 391, and then allow the pin 392 to naturally release to the point where its head protrudes from the hole of the inner tube 391 and the other hole 395 on the outer tube 394 that is aligned with the hole of the inner tube 391. At this point, the pin 392 fixes the outer tube 394 in the new position relative to the inner tube 391. The telescopic bracket 123 makes the position adjustment of the camera assembly 120 more convenient, while also enhancing the protection of the camera assembly 120 and extending its service life. The bracket 123 can also be telescopic through other structures. Alternatively, the telescopic structure may not be purely mechanical but rather an electromechanical combination, electrically connected to the processor of the intelligent lawnmower 110. The processor can autonomously adjust the length of the bracket 123 to adjust the height of the camera assembly 120 based on the image information collected by the camera assembly 120. This application does not limit the specific implementation method; as long as the bracket 123 of the camera assembly 120 can be telescopic, it falls within the protection scope of this application.

[0067] Furthermore, in conjunction with the retractable bracket 123, the main body 113 of the intelligent lawnmower 110 can be provided with an inwardly recessed receiving cavity 115, see [reference]. Figures 4A-4BThe top opening of the receiving cavity 115 is located on the upper surface of the lawnmower body 113. The bracket 123 is fixed inside the receiving cavity 115 by fastening mechanisms such as screws and nuts. A cover plate 118 is located on the top of the receiving cavity 115, and the cover plate 118 can be opened and closed. For example, the cover plate 118 is hinged to one side of the top opening of the receiving cavity 115, including a first position when open. Figure 4B ) and the second position when closed ( Figure 4A Alternatively, the cover 118 consists of a sliding cover and a sliding cover guide that can slide back and forth, including a first position covering the top opening of the receiving cavity 115 and a second position exposing the opening of the receiving cavity 115. The advantage of the receiving cavity 115 and the cover 118 in conjunction with the retractable bracket 123 is that when the smart lawnmower 110 is not in use, the bracket 123 can be shortened and the cover 118 closed, allowing the camera assembly 120 to be hidden and stored inside the lawnmower body 113. This is not only neater and more aesthetically pleasing, but also waterproof, dustproof, and lightproof, reducing the frequency of camera cleaning and delaying aging. Before the smart lawnmower 110 is in operation, the cover 118 is opened and the bracket 123 is extended, allowing the camera assembly 120 to extend out of the receiving cavity 115 of the smart lawnmower 110 to capture images around the smart lawnmower 110. The specific shapes of the receiving cavity 115 and the cover plate 118 are not limited in this application; in addition, the specific position of the receiving cavity 115 can be determined according to the position of the motor, PCB board and other devices of the smart lawnmower 110, so as to facilitate the acquisition of image information around the smart lawnmower 110 and minimize the impact on the arrangement of the components inside the main body 113 of the smart lawnmower 110. There are no restrictions in this application, and Figure 4 is just an exemplary illustration.

[0068] Furthermore, the bracket 123 can also be configured as a foldable structure. Referring to Figures 5a-5b, a groove 117 is provided on the upper surface of the main body 113 of the smart lawnmower 110 to accommodate the bracket 123 and the camera assembly 120. The bracket 123 is hinged to a point on the top surface of the main body 113 of the smart lawnmower 110, allowing the bracket 123 to rotate around the hinge point, overcoming friction when moved by hand. When not in use, the bracket 123 can be rotated around the hinge point to lie flat and stored in the groove 117 on the top surface of the main body 113 of the smart lawnmower 110. Figure 5A This improves aesthetics and tidiness, reduces the space required for storing the smart lawnmower 110, and enhances camera protection, extending its service life. During operation, the bracket 123 is erected, as... Figure 5BThe bracket's standing angle can be adjusted as needed. Furthermore, a rotatable connection mechanism, such as a damped pivot structure or a ball bearing structure, can be used between the bracket 123 and the camera assembly 120. This allows the user to freely adjust the angle of the camera assembly 120 before turning on the smart lawnmower 110. Alternatively, the rotatable connection mechanism may not be purely mechanical but rather electromechanical, electrically connected to the processor of the smart lawnmower 110. The processor can then autonomously adjust the angle of the camera assembly 120 based on the image information captured by the camera assembly 120. It should be noted that the telescopic, folding, and rotating designs of the bracket 123 for the camera assembly 120 described above are merely examples and are not limited to the specific implementation methods described. The scope of protection of this application should not be limited based on these examples.

[0069] The camera assembly 120 may include a single or dual (multiple) cameras. In terms of ranging principles, monocular cameras and dual (multiple) cameras differ significantly. Dual (multiple) cameras, similar to human eyes, primarily determine distance by calculating the parallax of two (multiple) images captured simultaneously by two (multiple) cameras. Therefore, dual (multiple) cameras can estimate depth without relying on other sensing devices when stationary. However, their depth range and accuracy are limited by the baseline (distance between the optical centers of the two cameras) and resolution. Furthermore, parallax calculation is quite resource-intensive, resulting in complex configuration, high computational load, and high energy consumption. Monocular cameras, on the other hand, capture image frames that are two-dimensional projections of three-dimensional space, losing environmental depth information. Distance can only be calculated by calculating the parallax created by the motion of objects in the image when the camera is moved. This drawback can be mitigated to some extent by fusing pose data acquired by an inertial measurement unit. For example, the algorithm of a monocular vision-inertial measurement system (VINS-Mono) is widely used in positioning-dependent devices such as robots and drones due to its low cost, small size, and low power consumption. VINS-Mono can calculate the camera's movement and rotation by fusing the offset of feature points between consecutive frames captured by the camera and IMU data, and unlike GPS sensors, it is not limited by signal interference. Therefore, this application does not impose a strict limit on the specific number of cameras included in the camera assembly 120.

[0070] In addition to ordinary single or dual (multi) cameras, the camera assembly 120 can also include a depth camera, also known as an RGB-D camera. The biggest advantage of an RGB-D camera is that it can measure the distance between an object and the camera by actively emitting light towards the object and receiving the reflected light, similar to a laser sensor, using infrared structured light or Time-of-Flight (ToF) principles. Compared to dual (multi) cameras that rely on software calculations, RGB-D cameras obtain depth through physical measurement, saving significant computational effort. Commonly used RGB-D cameras include Microsoft's Kinect and Intel's RealSense. However, due to limitations in sensor accuracy and measurement range, depth cameras suffer from narrow measurement range, high noise, small field of view, susceptibility to sunlight interference, and inability to measure transmissive materials. Therefore, they are typically used more indoors than outdoors. Applying an RGB-D camera to the smart lawnmower 110 requires integration with other sensors and is best suited for use in conditions where sunlight is not intense.

[0071] The inertial measurement unit 122 includes at least an accelerometer and a gyroscope. An accelerometer is a sensor used to measure linear acceleration. When a rigid body is at rest relative to the Earth, its linear acceleration is 0. However, due to the influence of gravity, when measuring the linear acceleration of a rigid body using an accelerometer, a reading of approximately 9.81 m / s² will be obtained on the vertically downward axis pointing towards the Earth's center. 2 Similarly, under the influence of gravity, when the accelerometer reading on the rigid body is 0, the rigid body is in free fall, and its actual downward velocity is 9.81 m / s². 2The actual acceleration. Micro-Electro-Mechanical System (MEMS) sensors are widely used in smart home appliances. The internal structure of a MEMS accelerometer is a spring-mass microstructure; when acceleration occurs along the deformation axis of the micro-spring-mass, the micro-spring deforms. By measuring the deformation of the micro-spring using microelectronics, the acceleration along the axis can be obtained. Due to this structure, MEMS accelerometers cannot measure the actual acceleration of a rigid body; they can only provide acceleration measurements along its measurement axis. In practical applications, three MEMS measurement systems are typically used to form an orthogonal triaxial measurement system. The actual acceleration is measured along the three orthogonal measurement axes, and the actual acceleration is calculated from these components. A gyroscope is a sensor used to measure the angular velocity of a rigid body. Similar to MEMS accelerometers, MEMS gyroscopes can only measure the angular velocity component of rotation around a single measurement axis. Therefore, they are integrated and packaged as a three-axis gyroscope with three orthogonal measurement axes. The angular velocity components of the rigid body's rotation are measured separately along the three axes, and finally synthesized to obtain the actual angular velocity of the rigid body. In the typical xyz coordinate system, the angle of rotation around the x-axis of the reference coordinate system is defined as the roll angle, the angle of rotation around the y-axis as the pitch angle, and the angle of rotation around the z-axis as the yaw angle.

[0072] Typically, an inertial measurement unit 122 includes three single-axis accelerometers and three single-axis gyroscopes to measure the angular velocity and acceleration of an object in three-dimensional space, and calculate the object's attitude accordingly. Further, the inertial measurement unit 122 may also include a magnetometer. A magnetometer, also called a geomagnetic sensor, can be used to test the strength and direction of a magnetic field to determine the orientation of a device. The principle of a magnetometer is similar to that of a compass, measuring the angle between the current device and the four cardinal directions. Six-axis or nine-axis sensors, as integrated sensor modules, reduce circuit board space and overall footprint. The accuracy of integrated sensor data depends not only on the precision of the components themselves but also on post-assembly correction and the appropriate algorithms for different applications. A suitable algorithm can fuse data from multiple sensors, compensating for the shortcomings of a single sensor in calculating accurate position and orientation. Generally, the IMU sensor is best positioned at the object's center of gravity; therefore, preferably, the inertial measurement unit 122 can be positioned at the center of gravity G of the intelligent lawnmower 110, such as... Figure 6 As shown. Because the inertial measurement unit 122 is inexpensive, in one embodiment, dual inertial measurement units 122 can also be used to improve the accuracy and stability of the IMU data, such as... Figure 7As shown. On the one hand, the relative angular velocity and relative acceleration between the target object and the moving reference frame can be obtained based on the difference in output from the two inertial measurement units 122; on the other hand, the redundant design of the dual inertial measurement units 122 ensures the stability of positioning by monitoring the status of the two inertial measurement units 122 in real time and immediately switching to the other inertial measurement unit 122 when one inertial measurement unit 122 malfunctions.

[0073] The system diagram of the Smart Lawn Mower 110 is as follows: Figure 8 As shown, the system includes a power module 701, a sensor module 702, a control module 703, a drive module 704, and an actuator 705. The power module 701 supplies power to the drive module 704, control module 703, and sensor module 702. To meet the autonomous movement requirements of the intelligent lawnmower 110, the power module 701 preferably includes a battery pack providing direct current. The sensor module 702 includes at least a camera assembly 120 and an inertial measurement unit 122. The intelligent lawnmower 110 may also be equipped with other sensors such as GPS sensors, collision sensors, and drop sensors; information collected by these other sensors can also be comprehensively referenced during processing. The control module 703 includes: an input module 141 for receiving various raw data collected or detected by the sensor module 702; a processor 142 for logic operations, which can be a CPU or a microcontroller with high data processing speed; a memory 144 for storing various data and control programs 145; and an output module 143 for converting control commands into motor drive commands and sending them to the drive controller 161 of the motor drive switch. The drive module 704 includes a motor drive switch circuit 162, a drive controller 161, and a motor 163. Figure 8 The motor drive switching circuit 162 shown uses the most common MOSFET switch. The drive controller 161 controls the switching of the MOSFET switch by applying a voltage to its gate. The orderly switching of the MOSFET switch causes the motor windings to conduct in an orderly manner, thereby driving the motor 163 to rotate. Figure 8This disclosure only illustrates one common motor drive circuit and does not limit the specific implementation of the motor drive circuit. The rotation of motor 163 then directly or indirectly drives actuator 705 through a transmission mechanism. The actuator 705 of the intelligent lawnmower 110 mainly includes blades 112 and wheels 114. Optionally, blades 112 and wheels 114 are driven by independent motors 163. Optionally, the left and right rear wheels 114 can also be driven by independent motors 163, thereby achieving more flexible turning and attitude adjustment. The control program 145 stored in memory 144 mainly consists of two modules: a positioning and mapping module 146 and a function application module 147, where the positioning and mapping module 146 is the basis of the function application module 147. The localization and mapping module 146 addresses the fundamental questions of where the intelligent lawnmower 110 is, what the map is, and what the surrounding environment is like. It tracks the location of the intelligent lawnmower 110 as it moves and builds an understanding of the real world—this is called Simultaneous Localization and Mapping (SLAM). Based on the resolution of these fundamental questions, the functional application module 147 can then implement specific functions such as lawn mowing area boundary delineation, intelligent obstacle avoidance, road surface recognition and selection, navigation integration, and intelligent lighting. Of course, this classification is mainly for ease of understanding and explanation. In actual implementation, the localization and mapping module 146 and the functional application module 147 are not completely separate parts. The process of implementing the functional application module 147 itself deepens the understanding of the real world, and its results are fed back to the localization and mapping module 146, thereby continuously improving the map.

[0074] For the intelligent lawnmower 110, the implementation of Simultaneous Localization and Mapping (SLAM) requires the fusion of image data from the camera component 120 and pose data from the inertial measurement unit 122 (also known as sensor fusion). This is because visual sensors, such as cameras, perform well in most textured scenes, but are largely ineffective in scenes with fewer features, such as glass or white walls. While the inertial measurement unit can measure angular velocity and acceleration, it must be integrated over time to obtain the object's position or orientation. Furthermore, inertial components based on microelectromechanical systems (MEMS) inevitably have system biases. The combination of these two factors results in a very large cumulative error / drift over a long period. However, for rapid movements within a short time, its relative displacement data has high accuracy. During rapid movement, the camera may experience motion blur, or the overlap between two frames may be too small for feature matching. With the inertial measurement unit, a good pose estimate can be obtained even during periods when camera data is invalid. If the camera remains stationary, the pose estimate obtained from the visual information will also remain fixed. Therefore, during slow motion, visual data can effectively estimate and correct drift in inertial measurement unit (IMU) readings, ensuring that pose estimation remains effective even after slow motion. This demonstrates the strong complementarity between visual data and IMU data; fusing data from both the camera assembly 120 and the inertial measurement unit 122 can improve the accuracy and stability of positioning and mapping.

[0075] Because the data types measured by the camera assembly 120 and the inertial measurement unit 122 (visual measurement measures the coordinates of an object projected onto a pixel plane, while the inertial measurement unit measures the object's three-dimensional acceleration and rotational angular velocity) and measurement rates (visual measurement is limited by frame rate and image processing speed; the camera sampling rate can only reach tens of frames per second, while the inertial measurement unit can easily reach hundreds or even thousands of frames per second), there are significant differences when fusing the two data sets. Whether converting the motion measured by the inertial measurement unit into object coordinates (accumulating bias during integration) or converting the visual data into motion data (causing large fluctuations in calculated acceleration due to positioning errors during differentiation), additional errors are introduced. Therefore, detection and optimization are required during data fusion. Generally, instead of differentiating the visual data, fusion typically chooses to integrate the motion detected by the inertial measurement unit into object coordinates and then fuse it with the visual data. For example... Figure 9 As shown, the key modules in the entire flowchart can be broken down into the following parts: image and IMU data preprocessing, initialization, local optimization, mapping, keyframe extraction, loop closure detection, and global optimization. The main functions of each module are:

[0076] Image and IMU data preprocessing: Feature points are extracted from image frames acquired by camera component 120, and optical flow tracking is performed using KLT pyramids to prepare for subsequent visual initialization to solve the pose of the intelligent lawnmower 110. IMU data acquired by inertial measurement unit 122 is pre-integrated to obtain the pose, velocity, and rotation angle at the current moment. Simultaneously, the pre-integration increment between adjacent frames, which will be used in the back-end optimization, as well as the pre-integration covariance matrix and Jacobian matrix, are calculated.

[0077] Initialization: In the initialization, visual initialization is performed first to calculate the relative pose of the smart lawnmower 110; then the initialization parameters are solved by aligning with the IMU pre-integration.

[0078] Local optimization: Visual-inertial local optimization is performed on the sliding window, that is, visual constraints and IMU constraints are placed in a large objective function for nonlinear optimization; the local optimization here only optimizes the variables in the window of the current frame and the previous n frames (e.g., n is 4), and the local optimization outputs a more accurate pose of the smart lawnmower 110.

[0079] Mapping: Using the obtained pose, the depth of corresponding feature points is calculated using triangulation, and the current environment map is reconstructed simultaneously. In the SLAM model, the map refers to the set of all landmarks. Once the locations of the landmarks are determined, the mapping can be considered complete.

[0080] Keyframe extraction: Keyframes are selected image frames that can be recorded but avoid redundancy. The selection criteria for keyframes are that the displacement between the current frame and the previous frame exceeds a certain threshold or the number of matching feature points is less than a certain threshold.

[0081] Loop closure detection: Loop closure detection, also known as loop closure detection, saves the key frames of the previously detected image. When the smart lawnmower 110 returns to the same place it has passed through before, it can determine whether it has been there before by matching the feature points.

[0082] Global optimization: Global optimization is performed when loop closure is detected, using visual constraints, IMU constraints, and loop closure detection constraints to perform nonlinear optimization. Global optimization is based on local optimization, outputting a more accurate pose of the intelligent lawnmower 110 and updating the map.

[0083] In the above algorithms, local optimization optimizes the image frames within the sliding window, while global optimization optimizes all keyframes. Local optimization alone has low accuracy and poor global consistency, but it is fast and has high IMU utilization; global optimization alone has high accuracy and good global consistency, but it is slow and has low IMU utilization. Combining the two can complement each other's advantages, making the positioning results more accurate. The output pose is a 6-DOF pose, which refers to the three-dimensional motion (translation) of the smart lawnmower 110 in the xyz direction plus pitch / yaw / roll (rotation). During the fusion process, by aligning the pose sequence estimated by the IMU and the pose sequence estimated by vision, the true scale of the trajectory of the smart lawnmower 110 can be estimated. Moreover, the IMU can predict the pose of the image frame and the position of the feature point in the next frame image at the previous moment, improving the matching speed of the feature tracking algorithm and the robustness of the algorithm to deal with rapid rotation. Finally, the gravity vector provided by the accelerometer in the IMU can convert the estimated position into the world coordinate system required for actual navigation.

[0084] Compared to the relatively low accuracy (in meters) of 2D / 3D position output by the Global Positioning System (GPS), SLAM provides a higher accuracy (in centimeters) of pose with six degrees of freedom, and is independent of satellite signal strength and unaffected by interference from other electromagnetic signals. However, compared to the low computational and low-power GPS positioning, SLAM is more energy-intensive. Furthermore, since the smart lawnmower 110 operates outdoors, its camera sensors require frequent cleaning. If cleaning is not timely, the acquired image frames may become blurry, failing to provide effective visual data. Moreover, to accurately solve the SLAM problem, the smart lawnmower 110 needs to repeatedly observe the same area, achieving closed-loop motion. Therefore, system uncertainty accumulates until closed-loop motion occurs. Especially when the lawn is vast, the surrounding area is open, and there are few feature references, the system uncertainty during large closed-loop motions by the smart lawnmower 110 may lead to the failure of loop closure checks, resulting in the failure of global SLAM optimization and large positioning deviations. In environments with wide lawns and open surroundings, there is less interference with satellite signals, and GPS positioning results are usually more stable and accurate. Moreover, GPS is now widely used and inexpensive. Therefore, the Smart Lawn Mower 110 can also be equipped with a GPS sensor and use GPS+SLAM combined navigation.

[0085] The combined positioning method consisting of camera assembly 120, inertial measurement unit 122, and GPS can be found in [reference needed]. Figure 10First, the reliability of data from each sensor is assessed. If all sensors fail, navigation stops and a maintenance alert is issued. If two sensors fail, the remaining sensor is used for positioning and navigation for a short period, such as 3 seconds. During this period, the validity of data from the failed sensors is continuously monitored for recovery, and the recovered sensor data is added to subsequent positioning and navigation calculations. If no other sensor recovers within this short period, navigation stops in place and a maintenance alert is issued. If only one sensor fails, the remaining two sensors are used for positioning and navigation. If the GPS sensor fails, AR fusion with visual-inertial SLAM is used for positioning and navigation; if the camera fails, IM is used. U-data verifies the consistency of GPS results and filters and corrects inconsistent absolute positioning data. If the IMU fails, visual-only real-time localization and mapping (VSLAM) is performed. After processing each frame, the VSLAM result and the current GPS positioning result are simultaneously fed into a Kalman filter. The validity of the data from the failed sensor is continuously monitored, and the recovered sensor data is added to the subsequent positioning and navigation calculations. If, after completing the lawn mowing and returning to the charging station, any sensor has not been recovered, an abnormality alert is issued. When all three sensors are working normally, the GPS positioning result is used to filter and correct the pose and environmental map generated by AR fusion visual inertial SLAM.

[0086] In practical applications, open-source AR software packages can be used to implement the Simultaneous Localization and Mapping (SLAM) process and call different application programming interfaces (APIs) to achieve rich functionality. For example, ARCore, a software platform launched by Google for building augmented reality applications, is based on SLAM by fusing image data and IMU data. Its three main functions integrate virtual content with the real world seen through a camera: 1. Motion tracking: Allows the machine to understand and track its position and posture relative to the real world; 2. Environment understanding: Allows the machine to detect various surfaces (such as the ground, tabletops, walls, and other horizontal or vertical surfaces) through feature point clustering, and know their boundaries, size, and location; 3. Lighting estimation: Allows the machine to estimate the current lighting conditions of the environment. In addition to Google's ARCore, Apple's ARKit and Huawei's AR Engine are also software packages that can provide similar functions.

[0087] In one embodiment, the functional application module 147 of the control program 145 of the intelligent lawnmower 110 can distinguish between grass and non-grass based on the feature points of the two-dimensional plane in the image frame and the texture features of the grass. If the working surface where the lawnmower is currently located is not grass, the blade 112 stops rotating; and along the boundary between grass and non-grass, it automatically generates the boundary of the mowing area using motion tracking functions of software packages such as ARCore. Furthermore, the intelligent lawnmower 110 can also be used with an interactive display interface to display the constructed map and the boundary of the mowing area, allowing the user to confirm and modify it. During the confirmation process, to facilitate more intuitive and careful identification of the boundary lines by the user, two identification modes can be set. One identification mode displays the boundary lines of the mowing area on a two-dimensional map on the interactive display interface, see [link to relevant documentation]. Figure 11A In the 2D map, lawn 222 is located between house 223 and road 224, and the boundary line 221 of the mowing area is represented by a thick dashed line. Users can manually adjust the boundary line 221 on the 2D map of the interactive display interface; for example, by dragging a segment of boundary line 221 up, down, left, or right, or by deleting or adding a segment of boundary line 221 (drawing with a finger). If desired, users can also choose to directly enter this recognition mode and draw all the boundary lines 221 on the 2D map of the interactive display interface with their fingers. Another recognition mode overlays a virtual fence 211 icon onto the real-time image captured by the camera component 120 displayed on the interactive display interface. See [link to relevant documentation]. Figure 11BIn this recognition mode, the boundary lines automatically generated by the smart lawnmower 110 are displayed as virtual fence icons 211. Users can manually adjust the position of the virtual fence icons 211 superimposed on the real image on the interactive display interface, such as zooming in or out, deleting, or adding new sections of virtual fence 211. Furthermore, with the motion tracking capabilities of software packages like ARCore, users can check the appropriateness of the virtual fence 211 from various angles as the camera component 120 moves and switches angles. Compared to the boundary lines 221 on a two-dimensional map, the virtual fence icons 211 superimposed on the real image are more intuitive and precise, allowing users to determine the precise position of the virtual fence 211 (i.e., the boundary line) based on specific ground conditions (e.g., terrain, vegetation type). During the confirmation process, users can combine the two modes: first, check the boundary lines on the two-dimensional map to see if they meet expectations, adjusting any that don't; then, for boundary areas requiring special attention, check the virtual fence icons 211 superimposed on the real image and refine any necessary adjustments. Once the user confirms the boundary of the mowing area, the smart lawnmower 110 stores the confirmed boundary line (including the virtual fence 211) as discrete anchor point coordinates. The position of this boundary line (discrete anchor points) does not change as the smart lawnmower 110 moves. When planning its path, the smart lawnmower 110 is confined to operating within the boundary of the mowing area. It is worth noting that the interactive display interface can be a component on the smart lawnmower 110, a standalone display device, or an interactive display interface of a mobile terminal such as a smartphone or tablet that can interact with the smart lawnmower 110.

[0088] In one embodiment, the functional application module of the control program 145 of the intelligent lawnmower 110 can identify the materials of different surfaces. Besides identifying lawns and non-lawns, the intelligent lawnmower 110 can also analyze the feature points of two-dimensional planes in the image frames captured by the camera component 120. Based on the different plane textures (i.e., the distribution patterns of feature points), and by comparing them with the texture features of common types of planes preset by the control program 145, it can identify different types of ground (including water surfaces). If the intelligent lawnmower 110 simultaneously travels across grounds of different materials, the different hardness and materials of the ground can cause varying support and friction forces on the wheels 114 of the intelligent lawnmower 110, easily leading to problems such as bumping, tilting, and veering off course. Therefore, when the intelligent lawnmower 110 is traveling on non-lawn ground, for example, from one lawn to another, and it identifies multiple surfaces with different feature point textures (i.e., different hardness) within the area 212 directly in front, it selects to travel on one of the harder surfaces. See also... Figure 12When the intelligent lawnmower 110 detects multiple road surfaces in the area 212 directly in front of it—cement and dirt—with the cement surface on the left and the dirt surface on the right, the road selection program in control program 145 plans a path, controlling the intelligent lawnmower 110 to adjust its direction and travel to the left until it detects that the area 128 directly in front of it is entirely cement. Then, it adjusts its direction back to the original planned direction. This road selection is beneficial for the intelligent lawnmower 110's movement control, machine maintenance, and safety. In the road selection program, the environmental understanding function of software packages such as ARCore can be used to classify surfaces of different materials, and the texture features of common planes can be compared to assist the intelligent lawnmower 110 in determining the plane type. After determining the plane type, the program selects the harder surface according to the ground type-hardness comparison table stored in memory and controls the intelligent lawnmower 110's direction of travel accordingly. In addition, by comparing the texture features with common planes and judging the positional relationship between planes, the smart lawnmower 110 can identify terrain such as water surfaces, steps, and cliffs that may pose a risk of falling and damage to the smart lawnmower 110, making the function of automatically generating the boundary of the mowing area more complete.

[0089] In one embodiment, the functional application module of the control program 145 of the intelligent lawnmower 110 may further include an AI object recognition program, which calculates the category information of obstacles from the image data acquired by the camera component 120, thereby enabling the intelligent lawnmower 110 to actively and intelligently avoid obstacles. Different obstacle avoidance strategies and appropriate avoidance distances are adopted for different categories of obstacles to balance mowing coverage and obstacle avoidance success rate. Figures 13A-13B For a selected object, the object recognition program outputs a category and its corresponding confidence probability (C:P), where the confidence probability P ranges from 0 to 1. The control program 145 may also include a confidence threshold P1, for example, P1 = 0.7. It will accept judgments greater than the confidence threshold and proceed to select an obstacle avoidance strategy, such as... Figure 13A (bird: 0.99); however, judgments based on values ​​less than or equal to the confidence threshold are not accepted, such as... Figure 13B In the cases of (bird: 0.55) and (bird: 0.45), if the distance D between the obstacle and the smart lawnmower 110 is greater than the recognition threshold distance D3, then the robot continues to drive normally and uses the next frame or the next n frames of images for object recognition. The control program 145 makes a higher confidence probability object recognition judgment as the smart lawnmower 110 approaches the obstacle. If the distance D between the obstacle and the smart lawnmower 110 is less than or equal to the recognition threshold distance D3, then a long-distance avoidance strategy is adopted, for example, driving around the obstacle at a distance of 0.5m.

[0090] like Figure 14As shown, different obstacle avoidance strategies are adopted according to the type of obstacle. If the detected obstacle is a material that can be cut by the blade 112 and can decompose naturally, such as fallen leaves, branches, pine cones, or even animal excrement, the smart lawnmower 110 can ignore these obstacles and continue along the original path. Although animal excrement may soil the blade 112 and chassis of the smart lawnmower 110, similar to soil, this dirt will be cleaned up to some extent during frequent cutting, so there is no need to avoid it. If the detected obstacle is an animal, such as a person, bird, squirrel, or dog, a first threshold distance D1 and a second threshold distance D2 can be preset. When the distance D between the smart lawnmower 110 and the detected animal obstacle is greater than the first threshold distance D1, it will travel normally along the original path. When the distance D between the smart lawnmower 110 and the detected animal obstacle is less than or equal to the first threshold distance D1 but greater than the second threshold distance D2, it will slow down and emit a warning sound to alert animals such as people, birds, squirrels, and dogs to notice the smart lawnmower 110 and actively avoid it. When the distance D between the smart lawnmower 110 and the detected animal obstacle is less than or equal to the second threshold distance D2, a long-distance avoidance strategy will be adopted to avoid accidentally causing harm to people and animals. If the detected obstacle is a small, movable (temporary) object such as a plastic toy, shovel, or rope, the smart lawnmower 110 can maintain a certain distance to avoid accidental damage to these small objects, or in other words, adopt a long-distance avoidance strategy and issue a cleanup prompt to the user, reminding them to remove the small objects from the lawn. Furthermore, for animal obstacles and movable (temporary) obstacles, the smart lawnmower 110 can store the coordinates of the obstacle and the avoidance area while taking avoidance action. Before mowing ends, if the image data collected by the camera component 120 shows that the obstacle at the coordinates of the obstacle has been removed, it plans a return path and re-moves the previously avoided area. If the detected obstacle is a large, immovable (permanent) object such as a tree or garden furniture (e.g., a bench or swing), the intelligent lawnmower 110 can adopt a close-range avoidance strategy. This involves slowing down and getting as close to the obstacle as possible to maximize mowing coverage. For example, it can avoid the obstacle at a distance of 0.1 meters. Alternatively, when the intelligent lawnmower 110 is equipped with a collision sensor, minor collisions at low speeds will not cause significant damage to these large objects, thus allowing for close-range avoidance. Simultaneously, the intelligent lawnmower 110 can store the actual avoidance path and optimize it when the processor 142 is idle, ensuring that the next time it avoids the same obstacle, it maintains mowing coverage while improving the efficiency of the avoidance path.

[0091] In addition to identifying real obstacles from the images acquired by the camera component 120, users can also manually overlay virtual obstacles 215 onto the real-time images captured by the camera component 120 displayed on the interactive display interface, and adjust the orientation, size, and dimensions of the virtual obstacles 215, such as... Figure 15 As shown. Using motion tracking capabilities from software packages like ARCore, users can check the appropriateness of the virtual obstacle 215 from various angles as the camera component 120 moves and changes angle. The position and size information of the virtual obstacle 215 are recorded as anchor points, and this virtual obstacle 215 does not change with the movement of the smart lawnmower 110. Thus, when the smart lawnmower 110 moves within the actual work area, it can compare its current position with the position information of the virtual obstacle 215 in real time and avoid collisions with it. The virtual obstacle 215 allows users to customize specific mowing areas according to specific situations. For example, if there is an unfenced flowerbed on the lawn, which may appear as a regular lawn in some seasons, to prevent the smart lawnmower from accidentally stepping into this flowerbed while mowing, users can add a virtual obstacle 215 with the same base area as the actual flowerbed to the real-time image of the flowerbed captured by the camera component 120 displayed on the interactive display interface. For example, if there's a doghouse on the lawn, a large doghouse will be automatically identified as an immovable, large object by the control program 145 as described above, and a close-range obstacle avoidance strategy will be adopted to improve lawn coverage. However, considering that the dog might be inside the doghouse, to avoid disturbing or frightening the dog from the operation of the smart lawnmower 110, the user can overlay virtual obstacles 215 or virtual fences 211 around the doghouse image captured in real time by the camera component 120 displayed on the interactive display interface to create a larger non-working area. Furthermore, since ARCore tracks trackable objects such as planes and feature points over time, virtual obstacles can also be anchored to specific trackable objects to ensure that the relationship between virtual obstacles and trackable objects remains stable. For example, if virtual obstacle 215 is anchored to the doghouse, then when the doghouse is moved later, virtual obstacle 215 will track the movement of the doghouse without requiring the user to reset the virtual obstacle.

[0092] In one embodiment, the function application module of the control program 145 of the intelligent lawn mower 110 can detect the light state of the surrounding environment. With the help of the light estimation function of software packages such as ARCore, the intelligent lawn mower 110 can know the light intensity L of the surrounding environment and adjust the lighting lamp 119 of the intelligent lawn mower 110 accordingly. The control program 145 can preset a first light intensity threshold L1. When the light intensity L of the surrounding environment is less than the first light intensity threshold L1, the intelligent lawn mower 110 turns on the lighting lamp 119 for supplementary lighting. In addition, different working modes can be set. According to the light intensity and direction, the mowing time can be reasonably arranged and different working modes can be selected. For example, when it is detected that the light in the surrounding environment is very weak, for example, when the light intensity L of the surrounding environment is less than a second light intensity threshold L2 (L2 < L1), if the user does not command to mow immediately, the lawn mower returns to the charging station and enters the charging mode or standby mode, because the lawn is most vulnerable to fungal and pest damage without light; if the user commands to mow immediately, the lighting lamp 119 is turned on and the lawn is mowed in a silent mode to reduce the disturbance of the mower noise to the quiet night. When it is detected that the light in the surrounding environment is very strong, for example, when the light intensity L of the surrounding environment is greater than a third light intensity threshold L3 (L3 > L1), if the user does not command to mow at this time, the lawn mower returns to the charging station and enters the charging mode or standby mode, because strong sunlight can easily dry out the cut grass; if the user commands to mow immediately, the lawn is mowed in a fast mode to reduce the time the mower is exposed to the hot sun to reduce aging caused by UV irradiation. When it is detected that the light in the surrounding environment is suitable, for example, when the light intensity L of the surrounding environment is greater than or equal to the first light intensity threshold L1 and less than or equal to the third light intensity threshold L3, the lawn can be mowed in a conventional mode.

[0093] In addition to the light state of the environment, the image data collected by the camera component 120, combined with AI object recognition operations, can also be used as a basis for judging the mowing time and mode selection. For example, when it is detected that there is dew on the vegetation, if the user does not command to mow immediately, the lawn mower returns to the charging station and enters the charging mode or standby mode, because the dew will reduce the cutting efficiency and even cause jamming. In addition, the wet lawn is likely to leave wheel tracks, affecting the appearance. When it is detected that there is frost or ice and snow on the vegetation, if the user does not command to mow immediately, the lawn mower returns to the charging station and enters the charging mode or standby mode, because the cold weather is also not conducive to the recovery of the cut grass incision.

[0094] It's worth noting that AR software packages like ARCore typically lack robust object recognition capabilities. For instance, ARCore's environment understanding function detects, distinguishes, and delineates 2D surfaces through feature point clustering on a plane, rather than determining the surface type of an object through object recognition. Even though the smart lawnmower 110's control program 145 incorporates texture features of common plane types to aid in plane type identification, this still falls short of true object recognition. Therefore, in practical applications, obstacle recognition, environment recognition, and other functions still require other AI software packages with object recognition capabilities, such as Google's TensorFlow. TensorFlow Lite is a suite of tools that helps developers run TensorFlow models on mobile devices, embedded devices, and IoT devices. It supports on-device machine learning inference (without sending data back and forth between the device and the server), has low latency, and small binary files. Alternatively, the smart lawnmower 110 could include a wireless network-connected device 150, delegating object recognition to a cloud server 200. Since the cloud server 200 possesses powerful cloud storage and computing capabilities, it can continuously refine the training set and model using the TensorFlow framework, resulting in more accurate judgments.

[0095] In fact, when the smart lawnmower 110 includes a wireless network connection device 150, the control program 145 can send the fusion calculation of visual data and IMU data, and even the calculation tasks of the entire positioning and mapping module 146 and the functional application module 147, to the cloud server 200. The cloud server 200 performs fusion, positioning, mapping, judgment, and generates navigation and mowing instructions based on the uploaded data according to a preset program. At this time, the control program 145 of the smart lawnmower 110 only needs to be responsible for acquiring data from the camera 120 and the inertial measurement unit 122, preprocessing and uploading the acquired data, and downloading and outputting instructions from the cloud server 200 locally, without having to perform computationally complex AR and / or AI calculations, thus reducing the requirements on the processor 142 of the smart lawnmower 110 and saving chip costs. Similarly, when the smart lawnmower 110 includes a wireless network connection device 150, the control program 145 can also send the fusion calculation of visual data and IMU data, and even the calculation tasks of the entire positioning and mapping module 146 and the functional application module 147, to other devices that can wirelessly transmit data with the smart lawnmower 110, such as a mobile terminal application. In this case, the control program 145 of the smart lawnmower 110 can be understood as providing an application programming interface (API) to implement the communication function between the smart lawnmower 110 and the mobile terminal, and to define the data communication protocol and format between the smart lawnmower 110 and the mobile terminal application. Through this API, the mobile terminal application can obtain image and pose data from the smart lawnmower 110, and according to a preset program, after a series of computationally complex AR and / or AI calculations, generate navigation and mowing command data, and then send the command data back to the smart lawnmower 110 through this API, thereby realizing the control of the smart lawnmower 110 by the mobile terminal. The mobile application can also provide users with selectable and modifiable parameters, such as preferred mowing time and mowing height, allowing users to customize intelligent control of the smart lawnmower 110 according to their needs. Therefore, reserving an application programming interface on the smart lawnmower 110 not only reduces the requirements for the processor 142 of the smart lawnmower 110, saving chip costs, but also facilitates users controlling the smart lawnmower 110 through other devices.

[0096] In another embodiment, the camera used to acquire image information can also be installed in the environmental scene. For example, see... Figure 16 The smart lawnmower 210 itself does not have a camera; instead, one or more cameras 190 are mounted on the roof and / or on top of the charging station 180. Because no mounting bracket or storage cavity is required, the smart lawnmower 210 has a more flexible casing design, for example... Figure 16The smart lawnmower 210 shown features a modern and attractive power head design. One or more cameras 190 positioned in the scene have a wireless connectivity device 191 for wirelessly connecting to the smart lawnmower 210 or to a wireless network, such as a user's home Wi-Fi network, to upload captured image data to the cloud server 200. The one or more cameras 190 can be commercially available rotatable cameras, providing a wider field of view and more precise positioning. The main components of the smart lawnmower 210 are similar to those of the smart lawnmower 110; identical components will not be repeated here. The main difference is that the smart lawnmower 210 does not have cameras directly mounted on the main body or connected to it via a bracket or other connecting mechanism to move synchronously with it. Furthermore, the smart lawnmower 210 has a wireless connectivity device 250 that can receive image data from one or more cameras 190 or connect to the internet to interact with the cloud server 200. It is worth noting that, for the smart lawnmower 110 in the previous embodiment, since the sensors (camera assembly 120, inertial measurement unit 122, etc.) are integrated into the lawnmower body 113, and the sensors and control module are connected by a wired connection, the wireless connection device 150 is not necessary. However, for the sake of improving computing power, facilitating upgrades, utilizing big data, and reducing chip costs, the smart lawnmower 110 may also have a wireless connection device 150, such as a wireless network card or a mobile network receiver. For the smart lawnmower 210 in this embodiment, since the camera 190 is separate from the smart lawnmower 210 body, the data transmission between them depends on a wireless connection. Therefore, one or more cameras 190 and the smart lawnmower 210 both rely on a wireless connection device (camera 190 includes a wireless connection device 191, and the smart lawnmower 210 includes a wireless connection device 250) to achieve wireless transmission. For example, one or more cameras 190 send the collected image data to the smart lawnmower 210 for processing.

[0097] The high-level architecture of the control module of the intelligent lawnmower 210 can refer to that of the intelligent lawnmower 110 in the previous embodiment. However, since the image information collected by one or more cameras 190 in the scene has a different perspective than the image information collected by the camera assembly 120 on the intelligent lawnmower 110, the control program 245 of the intelligent lawnmower 210 is also different from the control program 145 of the intelligent lawnmower 110. The control program 245 of the intelligent lawnmower 210 mainly uses a visual target tracking algorithm to estimate the position of the intelligent lawnmower 210 in the visible area of ​​the camera, and generates navigation and mowing instructions accordingly. One or more cameras 190 can send raw image data or data after certain processing to the intelligent lawnmower 210. When there is only one camera 190, the control program 245 of the intelligent lawnmower 210 uses a single-view target tracking algorithm to estimate its own position; when there are multiple cameras 190, the control program 245 of the intelligent lawnmower 210 uses a multi-view target tracking algorithm to estimate its own position. Multi-view target tracking algorithms include centralized and distributed methods. In centralized technology, the data transmission between multiple cameras (190°) and the smart lawnmower is as follows: Figure 17A Under distributed technology, the data transmission method between multiple cameras (190°) and the smart lawnmower is as follows: Figure 17B . Figure 17A The intelligent lawnmower 210 in the system actually serves as the fusion center in the centralized multi-view target tracking algorithm. Each camera 190 sends the collected image data to the intelligent lawnmower 210 for processing. Figure 17B In this system, each camera 190 collects and processes video data locally and interacts and merges information with cameras 190 from other perspectives via the network. For example, each camera 190 merges its own acquired image to calculate a position estimate with the position estimates obtained from neighboring cameras 190 to obtain a new position estimate, and sends the new position estimate to the next neighboring camera 190 until the desired accuracy is achieved. Then, the camera 190 that has achieved the desired accuracy sends the position estimate to the intelligent lawnmower 210. The control program 245 of the intelligent lawnmower 210 generates navigation and mowing commands based on the obtained position estimate and information from its other sensors (if any). Compared with centralized technology, distributed technology has advantages such as low bandwidth requirements, low system power consumption, high real-time performance, and high reliability. The distributed multi-view target tracking algorithm reduces the requirements for the processor chip of the intelligent lawnmower 210, but increases the requirements for the data processing capabilities of the cameras 190. It is suitable for situations where the lawn is large and the scene is complex, requiring a large number of cameras 190. In contrast, the centralized multi-view target tracking algorithm is suitable for situations where the lawn is small and the scene is simple, requiring a small number of cameras 190.

[0098] Alternatively, one or more cameras 190 and the smart lawnmower 210 may each be equipped with a wireless connectivity device 191 capable of accessing the internet, such as a wireless network card or a mobile network receiver, and integrate and compute data from multiple devices through a cloud server 200. One or more cameras 190, the smart lawnmower 210, and the cloud server 200 can be connected via... Figure 17C The architecture facilitates data interaction. One or more cameras 190 each upload the raw image data or pre-processed data they have collected to the cloud server 200. Based on the data from the one or more cameras 190, the cloud server 200 selects a single-view target tracking algorithm or a multi-view target tracking algorithm. After calculating the real-time position estimate of the smart lawnmower 210, it sends the corresponding positioning estimate and map information to the smart lawnmower 210. The control program 245 of the smart lawnmower 210 then integrates data from other sensors (if any) to generate navigation and mowing commands. Alternatively, the smart lawnmower 210 can also upload data collected by its other sensors to the cloud server 200 via a wireless network. After calculating the real-time position estimate of the smart lawnmower 210, the cloud server 200, based on the preset program stored in the cloud server 200 and the other sensor data uploaded by the smart lawnmower 210, directly issues navigation and mowing commands corresponding to the current situation and sends them to the smart lawnmower 210.

[0099] This application also proposes a lower-cost solution, namely an intelligent lawnmower system 100, comprising an intelligent lawnmower 310 and a mobile terminal 130. The mobile terminal 130 can be a mobile phone, tablet, or wristband, or any device equipped with a camera, inertial measurement unit (IMU), and computing unit. Since the mobile terminal 130 provides the camera and IMU, the intelligent lawnmower 310 itself does not need to include a camera or IMU, reducing production costs. Data transmission between the intelligent lawnmower 310 and the mobile terminal 130 can be achieved through wired or wireless communication. Figure 18 As shown, the intelligent lawn mowing system 100 can adopt an intelligent lawnmower 310, including: a cutting blade 312 for cutting grass; a main body 313 for mounting the cutting blade 312; wheels 314 for rotating and supporting the main body 313; a fixing device 316 disposed on the main body 313 for fixing the mobile terminal 130 to the intelligent lawnmower 310; an interface 311 disposed on the main body 313 for cooperating with the interface 131 of the mobile terminal 130 to form a wired connection for data transmission; and a controller (not shown) electrically connected to the interface 311, which controls the behavior of the intelligent lawnmower 310 according to the instruction data received by the interface 311 when the interface 311 is connected to the mobile terminal 130.

[0100] In one embodiment, the structure of the fixing device 316 is shown below. Figures 19A-19C , Figure 19A The fixing device 316 includes a first baffle 381, a second baffle 382, ​​a support plate 383, a support rod 384, and a base 385. The first baffle 381 and the second baffle 382 are parallel, located at opposite ends of the support plate 383, and protrude outwards from the same side of the support plate 383, forming opposing barbs. This facilitates fixing mobile terminals such as mobile phones and tablets 130 between the first baffle 381 and the second baffle 382. Specifically, the surfaces of the support plate 383, the first baffle 381, and the second baffle 382 that contact the mobile terminals 130 are also lined with silicone to increase the friction between the support plate 383, the first baffle 381, the second baffle 382, ​​and the mobile terminals 130, preventing the mobile terminals 130 from shaking off due to uneven ground during the movement of the intelligent lawnmower 310. Meanwhile, the silicone liner also has a certain degree of elasticity, which can cushion the collision between mobile terminals such as mobile phones and tablets 130 and support plates 383, first baffles 381, and second baffles 382 during bumpy conditions, reducing wear and tear on these components and extending their service life. This article does not restrict the liner material of support plates 383 and first baffles 381 and second baffles 382; various silicone, rubber, and other materials are acceptable as long as they provide anti-slip and cushioning functions.

[0101] like Figure 19B-19CAs shown, when the mobile terminal 130 is not installed, the distance between the first baffle 381 and the second baffle 382 is L1. For example, to accommodate the size of commonly available mobile phones, tablets, and other mobile terminals 130 (currently, most mobile phones, tablets, and other mobile terminals are between 4 inches and 12 inches), L1 can be 10 cm. Furthermore, the distance between the first baffle 381 and the second baffle 382 can be changed. In other words, the second baffle 382 can be translated relative to the first baffle 381, or the first baffle 381 can be translated relative to the second baffle 382, ​​causing the distance between the two baffles to change, thereby firmly clamping mobile terminals 130 of different sizes, such as mobile phones and tablets. For example, by providing a tension spring 386 and an extension rod 387 on the back of the support plate 383, the first baffle 381 can be translated away from or towards the second baffle 382. For ease of description, the movement of the first baffle 381 translating away from the second baffle 382 is called outward stretching, and the movement of the first baffle 381 translating towards the second baffle 382 is called inward contraction. Specifically, the second baffle 382 is fixedly connected to the support plate 383, while the first baffle 381 is fixedly connected to the top end of the extension rod 387 on the back of the support plate 383, away from the second baffle 382. One end of the tension spring 386 is connected to the second baffle 382, ​​and the other end is connected to the end of the extension rod 387 near the second baffle 382. Therefore, the tension of the tension spring 386 always pulls the extension rod 387 towards the second baffle 382, ​​even when the extension rod 387 contracts inward. The whole assembly consisting of the support plate 383, the telescopic mechanism, and the first and second baffles 382 can also be called a clamp.

[0102] When the mobile terminal 130 is not installed, the tension spring 386 pulls the extension rod 387 toward the second baffle 382 until the first baffle 381 abuts against the end of the support plate 383. At this time, the first baffle 381 is fixed in the first position abutting against the end of the support plate 383 under the tension of the tension spring 386 and the reaction force of the contact surface at the end of the support plate 383. When installing a mobile terminal 130 such as a mobile phone or tablet, the user first grasps the first baffle 381 and pulls the extension rod 387 outward. Then, the mobile terminal 130 is placed flat on the support plate 383, between the first baffle 381 and the second baffle 382. The user then releases the first baffle 381, causing it and the extension rod 387 to retract inward under the tension of the tension spring 386 until the first baffle 381 touches the edge of the mobile terminal 130. At this point, the first baffle 381 is fixed in a second position against the edge of the mobile terminal 130 under the tension of the tension spring 386 and the reaction force of the contact surface at the edge of the mobile terminal 130. It is understood that when clamping mobile terminals 130 of different sizes, there will be multiple second positions with slightly different specific locations. Here, we collectively refer to these positions fixed against the edge of the mobile terminal 130 as the second positions of the first baffle 381. The maximum distance between the first baffle 381 and the second baffle 382 is L2, and the difference between L2 and L1 is ΔL. ΔL represents the extension and retraction of the clamp of the fixing device 316. For example, L2 can be 19cm, then ΔL is 9cm. This fixing device 316 of the mobile terminal 130 can fix mobile terminals 130 such as mobile phones and tablets with a width or length between 10cm and 19cm. In actual use, if the mobile terminal 130 is small, such as a mobile phone, the mobile phone can be clamped vertically between the first baffle 381 and the second baffle 382, ​​that is, the first baffle 381 and the second baffle 382 clamp the longer side of the mobile phone; if the mobile terminal 130 is large, such as a tablet computer, the tablet computer can be clamped horizontally between the first baffle 381 and the second baffle 382, ​​that is, the first baffle 381 and the second baffle 382 clamp the shorter side of the tablet computer. There are many clamps on the market, and although their structures differ, many of them can firmly clamp mobile terminals 130 of different sizes. Due to their wide use and low price, this application does not limit the specific structure of the clamp, as long as it can firmly clamp mobile terminals 130 of different sizes.

[0103] The base 385 of the fixing device 316 can be directly fixed to the surface of the main body 313 of the intelligent lawnmower 310 by fastening mechanisms such as screws and nuts, for example. Figure 18 As shown, this design requires minimal structural modifications to existing smart lawnmowers and is low-cost; however, it falls short in terms of aesthetics and tidiness. Alternatively, as... Figure 20The intelligent lawnmower 310 body 313 has an inwardly recessed receiving cavity 315. The top opening of the receiving cavity 315 is located on the upper surface of the intelligent lawnmower 310 body 313. The base 385 of the fixing device 316 is fixed in the receiving cavity 315 by fastening mechanisms such as screws and nuts. The top of the receiving cavity 315 has a cover plate 318, which can be opened and closed. For example, the cover plate 318 is hinged to one side of the top opening of the receiving cavity 315, including a first position when open and a second position when closed. Alternatively, the cover plate 318 consists of a sliding cover and a sliding cover guide rail that can slide back and forth, including a first position covering the top opening of the receiving cavity 315 and a second position exposing the opening of the receiving cavity 315. The advantages of the receiving cavity 315 and the cover plate 318 are that when the smart lawnmower 310 is not in use, the fixing device 316 can be hidden and stored inside the main body 313 of the smart lawnmower 310. This is not only neater and more aesthetically pleasing, but also waterproof, dustproof, and lightproof, reducing the need for cleaning the fixing device 316 and delaying its aging. Figure 20 As shown, the interface 311 can also be located on the inner wall of the receiving cavity 315, thereby reducing the intrusion of dust, water, and other substances. The specific forms of the receiving cavity 315 and the cover plate 318 are not limited in this application; furthermore, the specific location of the receiving cavity 315 can be determined based on the location of the motor, PCB board, and other devices of the intelligent lawnmower 310, to facilitate the acquisition of image information around the intelligent lawnmower 310, and to minimize the impact on the arrangement of internal components of the main body 313 of the intelligent lawnmower 310. No restrictions are set in this application. Figure 20 This is just an example.

[0104] During non-working hours, the fixing device 316 of the mobile terminal 130 is concealed within the main body 313 of the smart lawnmower 310. Therefore, before the smart lawnmower 310 with the mobile terminal 130 in operation, the clamp of the fixing device 316 needs to extend beyond the main body 313 of the smart lawnmower 310 so that the camera 132 of the mobile terminal 130 can collect image information around the smart lawnmower 310. To achieve this, the support rod 384 of the fixing device 316 can be designed as a telescopic structure, for example, referring to the inner and outer double tube structure of the bracket 123 in the first embodiment. Before the smart lawnmower 310 with the mobile terminal 130 in operation, the inner tube of the support rod 384 is pulled outward, increasing the length of the entire support rod 384, thereby allowing the clamp to extend beyond the main body 313 of the smart lawnmower 310. When the smart lawnmower 310 is not in operation or is not equipped with the mobile terminal 130, the inner tube of the support rod 384 is pushed back inward, shortening the overall length of the support rod 384 so that it is completely housed in the receiving cavity 315 of the smart lawnmower 310. This application does not limit the specific telescopic structure of the support rod 384 of the fixing device 316, as long as it can achieve the effect of extension and retraction. Other structures that achieve similar effects, such as flexible or foldable support rods 384, also fall within the protection scope of this application.

[0105] from Figure 19A It can also be seen that a rotatable connection is formed between the support rod 384 and the clamp via a damping pivot structure or a ball bearing structure 388. The advantage of this is that when the smart lawnmower 310 is equipped with the mobile terminal 130, the user can freely adjust the angle of the clamp according to the actual working conditions and the specific position of the camera 132 of the mobile terminal 130. This adjustment is also the angle at which the mobile terminal 130 is fixed, i.e., the angle at which the camera 132 of the mobile terminal 130 collects image information of the environment surrounding the smart lawnmower 310. This application does not limit the specific structure of the rotatable connection; it only needs to achieve the rotatable effect. In some examples, the support rod 384 consists of multiple short rods connected sequentially, which can be folded to save space and the angle of the clamp can be adjusted using the hinge points between the short rods. With the aid of the fixing device 316, when the mobile terminal 130 is fixed on the main body 313 of the smart lawnmower 310, and the position of the mobile terminal 130 is stationary relative to the smart lawnmower 310, it can be considered that the image information of the surrounding environment collected by the camera 132 of the mobile terminal 130 is the image information of the surrounding environment of the smart lawnmower 310, and the pose information collected by the inertial measurement unit 133 of the mobile terminal 130 is the pose information of the smart lawnmower 310.

[0106] See Figures 21A-21CThe mobile terminal 130 includes: a camera 132 for acquiring image data of the environment surrounding the intelligent lawnmower 310; an inertial measurement unit 133 for detecting the position and attitude data of the intelligent lawnmower 310; an interface 131 for at least data transmission and also for charging; a memory (not shown) for storing an application program 135 that controls the operation of the intelligent lawnmower 310; and a processor (not shown) electrically connected to the camera 132 and the inertial measurement unit 133 for calling the application program 135 and calculating and processing the information acquired by the camera 132 and the inertial measurement unit 133. The processor can call the application program 135 to fuse the data acquired by the camera 132 and the inertial measurement unit 133 to achieve real-time localization and mapping (SLAM) of the intelligent lawnmower 310, and generate corresponding navigation and mowing commands according to preset logic and real-time data to control the behavior of the intelligent lawnmower 310. Common mobile terminals 130 on the market, such as mobile phones and tablets, may include a monocular camera 132 or a dual (multi)-lens camera 132. In terms of ranging principle, the monocular camera 132 is completely different from the binocular (multi-lens) camera 132. The binocular (multi-lens) camera 132 is similar to human eyes, mainly determining distance by calculating the parallax of two images. It can perform depth estimation when stationary, resulting in better data accuracy. However, parallax calculation is quite resource-intensive, resulting in high computational load and high energy consumption. Although the image frames captured by the monocular camera 132 lose environmental depth information, this drawback can be mitigated to some extent by fusing the pose data collected by the inertial measurement unit 133. For example, based on the offset of feature points between consecutive frames captured by the monocular camera 132, the camera's own movement and rotation can be calculated by fusing the pose data collected by the inertial measurement unit 133. Therefore, this application does not impose a strict limit on the number of cameras 132 in the mobile terminal 130.

[0107] The inertial measurement unit 133 includes at least an accelerometer and a gyroscope, and may further include a magnetometer. Taking an Android phone as an example, its IMU data includes 9 data items: accelerometer (3-axis), gyroscope (3-axis), and magnetometer (3-axis). Normally, the IMU is placed at the center of gravity of an object. However, the inertial measurement unit 133 of the mobile terminal 130, fixed to the mounting device 316, is generally several tens of centimeters (e.g., 30 centimeters) away from the center of gravity G of the smart lawnmower 310. To alleviate this problem, sensor position offset compensation parameters can be set when the application 135 processes the IMU data. These parameters can include 3-axis data (X, Y, Z). Where X represents the front-to-back distance between the inertial measurement unit 133 of the mobile terminal 130 and the center of gravity G of the smart lawnmower 310, a positive value indicates that the center of gravity G of the smart lawnmower 310 is in front of the inertial measurement unit 133 of the mobile terminal 130, and a negative value indicates that the center of gravity G of the smart lawnmower 310 is behind the inertial measurement unit 133 of the mobile terminal 130. Y represents the left-to-right distance between the inertial measurement unit 133 of the mobile terminal 130 and the center of gravity G of the smart lawnmower 310, a positive value indicates that the center of gravity G of the smart lawnmower 310 is to the right of the inertial measurement unit 133 of the mobile terminal 130, and a negative value indicates that the center of gravity G of the smart lawnmower 310 is to the left of the inertial measurement unit 133 of the mobile terminal 130. Z represents the vertical distance between the inertial measurement unit 133 of the mobile terminal 130 and the center of gravity G of the smart lawnmower 310. A positive value indicates that the center of gravity G of the smart lawnmower 310 is below the inertial measurement unit 133 of the mobile terminal 130, while a negative value indicates that the center of gravity G of the smart lawnmower 310 is above the inertial measurement unit 133 of the mobile terminal 130.

[0108] In addition to the camera 132 and the inertial measurement unit 133, the mobile terminal 130 may also include other sensors such as a GPS sensor, and the corresponding sensor fusion logic code is preset in the application 135. The process of visual-inertial fusion SLAM in the application 135, as well as the process involving more sensor fusion, including applications involving specific functions such as lawn mowing area boundary generation, road surface selection, intelligent obstacle avoidance, virtual fence and virtual obstacle setting, intelligent lighting, and lawn mowing timing selection, are similar to the control program 145 of the intelligent lawnmower 110, and will not be described in detail here.

[0109] There are various ways to achieve communication between the smart lawnmower 310 and the mobile terminal 130, see [link / reference] Figures 22A-22EIn this application, the specific communication method between the smart lawnmower 310 and the mobile terminal 130 is not limited. For example, a male Type-C interface can be provided on the second baffle 382 of the fixing device 316. When the mobile terminal 130 is fixed to the fixing device 316, the female Type-C interface of the mobile terminal is plugged into the male Type-C interface of the fixing device 316 to realize data transmission between the mobile terminal 130 and the smart lawnmower 310. However, this connection method limits the type of interface. If the interface type of the user's mobile terminal 130 is different from the preset interface type of the smart lawnmower 310, an adapter is required. Using a separate data cable to connect the two interfaces can solve the problem of interface incompatibility, such as... Figure 22A The smart lawnmower 310 has a USB data transmission interface 311. If the mobile terminal 130 has a Type-C data transmission interface 131, data transmission between the mobile terminal 130 and the smart lawnmower 310 can be achieved by connecting one end of a USB-to-Type-C data cable to the USB data transmission interface 311 of the smart lawnmower 310 and the other end to the Type-C data transmission interface 131 of the mobile terminal 130. However, if the user's mobile terminal 130 has an Android data transmission interface 131, a USB-to-Android data cable is required, connecting one end of the smart lawnmower 310 to the USB data transmission interface 311 and the other end to the Android data transmission interface 131 of the mobile terminal 130 to achieve data transmission between the mobile terminal 130 and the smart lawnmower 310. The advantage of using a separate data cable is that it can accommodate the extension or rotation of the fixing device 316. Furthermore, the charging heads of mobile terminals such as mobile phones and tablets 130 generally use USB transmission interfaces. In other words, the end of the charging cable of mobile terminals such as mobile phones and tablets 130 that connects to the charging head is basically a USB transmission interface. This not only improves the universality of the USB data transmission interface 311 of the smart lawnmower 310, but also allows users to provide their own data cable, which is the charging cable of mobile terminals such as mobile phones and tablets 130, further reducing the cost of the smart lawnmower 310.

[0110] When a wired connection is used, the application 135 of the mobile terminal 130 calls upon image data collected by the camera 132 and pose data collected by the inertial measurement unit 133, and fuses the two types of data for real-time localization and mapping (SLAM). This process can call open-source AR resource packages; for example, the application 135 developed for Apple mobile terminals 130 can call the ARKit development toolkit, and the application 135 developed for Android mobile terminals 130 can call the ARCore development toolkit. Based on the results of real-time localization and mapping (SLAM), the application 135 of the mobile terminal 130 generates specific navigation and mowing commands according to a preset program and returns them to the smart lawnmower 310, such as... Figure 22A As shown by the solid arrow in the image. The preset program can specifically include multiple application functions, such as automatically generating mowing boundaries, setting virtual fences, road surface recognition, intelligent obstacle avoidance, and setting virtual obstacles; the preset program can also call resource packages with object recognition capabilities, such as TensorFlow Lite, to implement object recognition. Alternatively, considering that the smart lawnmower 310 may also include other sensors such as collision sensors and drop sensors, the smart lawnmower 310 can send the data collected by these sensors to the mobile terminal 130, such as... Figure 22A As shown by the dotted arrow in the diagram. After being coordinated by the application 135 of the mobile terminal 130, specific navigation and mowing instructions are generated according to a preset program, and then transmitted to the intelligent lawnmower 310 via wired transmission, as shown in the diagram. Figure 22A As shown by the solid arrow in the image.

[0111] Furthermore, based on the communication between the aforementioned intelligent lawnmower 310 and the mobile terminal 130, such as Figure 22BAs shown, the mobile terminal 130 also includes a wireless network connection device 134, which can transmit data with the cloud server 200. This allows the application 135 of the mobile terminal 130 to perform some or all of its calculations locally on the mobile terminal 130, rather than entirely on the cloud server 200. For example, during Simultaneous Localization and Mapping (SLAM), all image data collected by the camera 132 and angular velocity and acceleration data collected by the inertial measurement unit 133 are uploaded to the cloud server 200 for fusion. Alternatively, data preprocessing can be performed locally on the mobile terminal 130, such as feature point extraction from image frames, before the preprocessed data is sent to the cloud server 200 for fusion, reducing reliance on wireless communication speeds. Besides Simultaneous Localization and Mapping (SLAM), the cloud server 200 can also run other program logic. Leveraging its cloud computing and cloud storage capabilities, the cloud server 200 can excel in applications such as obstacle recognition, boundary recognition, road surface recognition, and path planning. The mobile terminal 130 can also upload user settings and preferences to the cloud server 200, such as preferred mowing height and lawn printing anchor points. The cloud server 200 can also autonomously obtain relevant information such as weather and season from the Internet to generate navigation and mowing commands to control the behavior of the smart lawnmower 310. After the application 135 of the mobile terminal 130 obtains the commands from the cloud server 200, it transmits the commands to the smart lawnmower 310 via wired transmission.

[0112] Alternatively, wireless data transmission can also be used between the smart lawnmower 310 and the mobile terminal 130. For example... Figure 22C Because the smart lawnmower 310, equipped with the mobile terminal 130, maintains a close proximity when operating, short-range wireless communication (such as Bluetooth, ZigBee, or NFC) is possible between them. This requires both the smart lawnmower 310 and the mobile terminal 130 to have compatible short-range wireless communication devices, such as Bluetooth. Figures 22A-22B Compared to the wired communication shown, the short-range wireless communication solution is essentially just changing the wired interface between the smart lawnmower 310 and the mobile terminal 130 to a wireless interface; there is no difference in other aspects (transmission content, system architecture, etc.).

[0113] Alternatively, the mobile terminal 130 may have a wireless network connection device 134, such as a wireless network card or WLAN module, and the smart lawnmower 310 may have a wireless network connection device 350, such as a wireless network card or WLAN module. Figure 22DWhen a user's lawn is fully covered by a wireless network, both the mobile terminal 130 and the smart lawnmower 310 can connect to the cloud server 200 via the wireless network. The application 135 on the mobile terminal 130 can upload all image data collected by the camera 132 and angular velocity and acceleration data collected by the inertial measurement unit 133 to the cloud server 200 for AR fusion; alternatively, it can perform data preprocessing locally on the mobile terminal 130, such as feature point extraction, and then send the preprocessed data to the cloud server 200 for AR fusion, thus reducing reliance on communication speed. Simultaneously, the smart lawnmower 310 can also transmit information collected by other sensors, such as collision sensors and drop sensors (if applicable), Figure 22D (Indicated by dashed arrows) This information is uploaded to cloud server 200, and can also be used as parameters in the cloud server 200's calculation and decision-making process. After the cloud server 200 makes navigation and mowing instructions based on the uploaded data and built-in programs, it directly returns the results to the intelligent lawnmower 310. Compared to... Figure 22B In this process, the cloud server 200 returns the calculation results to the mobile terminal 130, and then the mobile terminal 130 returns them to the smart lawnmower 310. The cloud server 200 directly returns the results to the smart lawnmower 310, which has the advantage of reducing latency.

[0114] When a user's lawn is too large to achieve full wireless network coverage, the above solution has a supplementary implementation method, see [link to relevant documentation]. Figure 22E Since mobile terminals such as mobile phones 130 generally have mobile network receiving 137 and Wi-Fi hotspot 138 functions, they can convert the mobile network signal received by the mobile terminal 130 into a Wi-Fi signal and transmit it. The smart lawnmower 310 has a wireless network connection device 350 such as a wireless network card or WLAN module, and can wirelessly communicate with the cloud server 200 through the Wi-Fi network emitted by the Wi-Fi hotspot 138 of the mobile terminal 130. When the smart lawnmower 310 and the mobile terminal 130 are not on the same Wi-Fi network—for example, the smart lawnmower 310 accesses the internet through the hotspot network of the mobile terminal 130, while the mobile terminal 130 accesses the internet through the mobile network—the cloud server 200 may not be able to automatically recognize the pairing of the smart lawnmower 310 and the mobile terminal 130. In this case, when the application 135 and the smart lawnmower 310 upload data, the smart lawnmower 310's ID can be added as an identification code. When the smart lawnmower 310 obtains instructions, it can use the smart lawnmower 310's ID as authentication.

[0115] Compared to the first embodiment, the intelligent lawnmower system 100, which integrates the intelligent lawnmower 310 with the mobile terminal 130, reduces the hardware requirements for the intelligent lawnmower 310. This not only saves on the cost of the camera 132 and the inertial measurement unit 133, but also reduces the requirements for the processing chip of the intelligent lawnmower 310 by transferring the computationally demanding AR calculations to the application on the mobile terminal 130, thus saving chip costs. Furthermore, people use the mobile terminal 130 more frequently in daily life; the application 135 on the mobile terminal 130, with the help of various application market platforms, is easier to upgrade, maintain, and expand. For example, application 135 V1.0.0 can be purely local computation, while application 135 V1.2.0 can mainly rely on local computation but upload images requiring object recognition calculations to the cloud server 200, using big data to more accurately determine obstacle types. Of course, from another perspective, fixing the mobile terminal 130 and the smart lawnmower 310 in place while the smart lawnmower 310 is working can also cause some inconvenience for users, as many people are now accustomed to having their phones with them at all times, only letting them out of their hands briefly when charging. To alleviate the anxiety caused by separating the phone from the user, and to prevent the mobile terminal 130 from having insufficient remaining battery power to complete a full lawnmower task, the smart lawnmower 310 can be configured to use its own battery pack to charge the mobile terminal 130's battery when connected. At the same time, to avoid the smart lawnmower 310 continuing to charge the mobile terminal 130 even when its own battery is low, which could lead to problems such as a sudden reduction in working time or excessive battery discharge, a charging threshold can be set, for example, 70%. That is, if the smart lawnmower 310's battery pack has a remaining charge of 70% or more, it will charge the connected mobile terminal 130; if the smart lawnmower 310's battery pack has a remaining charge of less than 70%, it will not charge the connected mobile terminal 130. It should be noted that 70% here is just an example and does not limit the scope of protection of this case. Any scheme that sets a threshold for the remaining power of a smart lawnmower 310 to determine whether the smart lawnmower 310 charges the connected mobile terminal 130 falls within the scope of protection of this application.

[0116] The foregoing has shown and described the basic principles, main features, and advantages of this application. Those skilled in the art should understand that the above embodiments do not limit this application in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of this application.

Claims

1. A smart mowing system, comprising a smart mower and a mobile terminal: wherein The mobile terminal comprises: an interface for connecting with the smart mower to realize data transmission; a plurality of sensors for collecting environmental data around the smart mower; a processor configured to perform sensor fusion on the environmental data around the smart mower collected by the plurality of sensors, generate instructions for controlling navigation and mowing, and send the instructions to the smart mower through the interface; the plurality of sensors at least comprise a camera and an inertial measurement unit, and the processor fuses visual data obtained by the camera and pose data obtained by the inertial measurement unit to perform instant positioning and map construction of the smart mower, and generate navigation and mowing action instructions; the smart mower comprises: a main body; a fixing device arranged on the main body for fixedly mounting the mobile terminal to the smart mower; an interface for connecting with the mobile terminal to realize data transmission; a controller electrically connected with the interface, which controls the behavior of the smart mower according to the instructions of the mobile terminal when the interface of the smart mower is connected with the interface of the mobile terminal; the smart mower further comprises a receiving cavity arranged on the main body, and the fixing device comprises a support structure which is retractable, foldable or rotatable, when the support structure is in a first state, the fixing device is entirely located within the receiving cavity, and when the support structure is in a second state, the fixing device is partially located outside the receiving cavity.

2. The intelligent mowing system of claim 1, wherein: The fixing device comprises an elastic clamping mechanism capable of clamping a mobile terminal with a size of 4 inches to 12 inches.

3. The intelligent mowing system of claim 1, wherein: The fixing device is arranged on the upper surface of the main body.

4. The intelligent mowing system of claim 3, wherein: The interface of the smart mower is arranged on the fixing device and directly cooperates with the interface of the mobile terminal.

5. The intelligent mowing system of claim 3, wherein: The interface of the smart mower is arranged on the upper surface of the main body and connected with the interface of the mobile terminal through a data line.

6. The intelligent mowing system of claim 1, wherein: The receiving cavity has a cover plate at the top for waterproof and dustproof, and the cover plate has a closed state and an open state;when the cover plate is in the closed state, the support structure is in the first state;when the support structure is in the second state, the cover plate is in the open state.

7. The intelligent mowing system of claim 1, wherein: The interface of the smart mower is arranged in the receiving cavity and connected with the interface of the mobile terminal through a data line.

8. The intelligent mowing system of claim 1, wherein: The interface of the smart mower is a USB interface and connected with the interface of the mobile terminal through a data line.

9. The intelligent mowing system of claim 1, wherein: When the interface of the smart mower is connected with the interface of the mobile terminal, the mobile terminal can be charged.

10. The intelligent mowing system of claim 9, wherein: The smart mower further comprises a first power threshold, and only when the remaining power of the smart mower is greater than the first power threshold, the remaining power of the smart mower is used to charge the mobile terminal.

11. The intelligent mowing system of claim 1, wherein: The interface of the smart mower is a short-distance wireless communication interface, and the interface of the mobile terminal is also a short-distance wireless communication interface.

12. The intelligent mowing system of claim 1, wherein: The interface of the smart mower is Bluetooth, and the interface of the mobile terminal is also Bluetooth.

13. The intelligent mowing system of claim 1, wherein: The interface of the smart mower is ZigBee, and the interface of the mobile terminal is also ZigBee.

14. The intelligent mowing system of claim 1, wherein: The intelligent mowing system further comprises a cloud server, the mobile terminal further comprises a wireless communication device, the mobile terminal uploads the environmental data collected by the sensors to the cloud server for operation through the wireless communication device, and then generates and sends the control navigation and mowing instructions to the intelligent mowing machine according to the operation result returned by the cloud server.

15. An intelligent mowing system comprising an intelligent mowing machine, a mobile terminal, and a cloud server. wherein, The mobile terminal comprises: a wireless connection device for connecting with the cloud server to realize data transmission; a plurality of sensors for collecting environmental data around the intelligent mowing machine; a processor configured to upload the environmental data collected by the plurality of sensors to the cloud server through the wireless connection device; the cloud server is configured to pair the mobile terminal and the intelligent mowing machine, perform sensor fusion operation on the environmental data collected by the plurality of sensors uploaded by the mobile terminal, generate control navigation and mowing instructions, and send the instructions to the intelligent mowing machine through wireless connection; the intelligent mowing machine comprises: a main body; a fixing device arranged on the main body for fixedly mounting the mobile terminal to the intelligent mowing machine; a wireless connection device for connecting with the cloud server to realize data transmission; a controller configured to obtain instructions from the cloud server through the wireless connection device to control the behavior of the intelligent mowing machine; the intelligent mowing machine further comprises a receiving cavity arranged on the main body, and the fixing device comprises a support structure that is retractable, foldable, or rotatable, when the support structure is in a first state, the fixing device is entirely located within the receiving cavity, and when the support structure is in a second state, the fixing device is partially located outside the receiving cavity.

16. The intelligent mowing system of claim 15, wherein: The plurality of sensors of the mobile terminal at least comprise a camera and an inertial measurement unit, the cloud server fuses visual data obtained by the camera and pose data obtained by the inertial measurement unit, performs instant positioning and map construction of the intelligent mowing machine, and generates control navigation and mowing instructions.

17. The intelligent mowing system of claim 15, wherein: The mobile terminal is movable up and down relative to the main body.

Images