Robot lawn mower

Abstract

Disclosed in the present application is a robot lawn mower. The robot lawn mower comprises: a drive component, comprising a blade and a drive motor for driving the blade; a walking component, comprising walking wheels and a walking motor for driving the walking wheels; a camera component, configured to collect two-dimensional images of the surroundings of the robot lawn mower; and a processor, which is in communication connection or electrical connection with the camera component and configured to at least control, according to the two-dimensional images, actions of the robot lawn mower. The processor is further configured to: identify an adverse road condition in the two-dimensional images by means of semantic segmentation, the adverse road condition belonging to one of multiple preset adverse road condition types; verify the adverse road condition according to a three-dimensional point cloud, so as to obtain a verification result; and, according to the verification result, plan a travel route for the robot lawn mower.

Description

lawnmower

[0001] This application claims priority to Chinese Patent Application No. 202310908086.7, filed with the Chinese Patent Office on July 24, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of garden tool technology, specifically to a lawn mowing robot. Background Technology

[0003] Lawn mowers, as a common gardening tool, are used for lawn mowing; specifically, they can perform mowing operations within a work area enclosed by boundary lines. Thanks to advancements in related technologies, and to meet the growing needs of users and provide more intelligent and diverse services, lawn mowers have gradually evolved into lawn mowing robots.

[0004] Robotic lawnmowers have multiple functions such as path planning, object recognition, and obstacle avoidance, which can avoid problems such as collisions and repetitive work. Compared with traditional lawnmowers, they can significantly improve work efficiency and reduce power consumption. They are more user-friendly in terms of both cost and user experience.

[0005] This section provides background information related to this application, which is not necessarily prior art.

[0006] Summary of the Invention

[0007] One object of this application is to solve or at least alleviate some or all of the aforementioned problems. Therefore, one object of this application is to provide a lawnmower robot.

[0008] To achieve the above objectives, this application adopts the following technical solution:

[0009] A lawnmower robot includes: a drive assembly including blades and a drive motor for driving the blades; a locomotion assembly including locomotion wheels and a locomotion motor for driving the locomotion wheels; a sensor assembly configured to collect environmental information surrounding the lawnmower robot; and a processor communicatively or electrically connected to the sensor assembly, configured to control the actions of the lawnmower robot based at least on the environmental information collected by the sensor assembly; wherein the processor is further configured to: identify obstacles based on the environmental information collected by the sensor assembly, and determine whether the obstacles are static or dynamic; when the obstacles are determined to be dynamic, navigate using a first map; and when the obstacles are determined to be static, navigate using a second map.

[0010] In some embodiments, the second map is a local map, which is a map within a certain distance of the lawnmower robot's real-time location.

[0011] In some embodiments, the second map is updated in real time as the lawnmower moves.

[0012] In some embodiments, the first map is a global map, which is an overall map of the lawnmower robot's work area.

[0013] In some embodiments, the second map has a higher resolution than the first map.

[0014] In some embodiments, the processor is also configured to control the walking motor to drive the walking wheels at a first speed when navigating using a first map, and to control the walking motor to drive the walking wheels at a second speed when navigating using a second map.

[0015] In some embodiments, the sensor assembly includes a camera assembly configured to acquire images of the environment surrounding the lawnmower robot.

[0016] In some embodiments, the processor uses a semantic segmentation algorithm to identify obstacles and their types in an environmental image. When an obstacle is identified as a person or an animal, it is determined to be dynamic.

[0017] In some embodiments, the processor uses a streaming or object tracking method to detect whether objects in an environmental image are dynamic.

[0018] In some embodiments, the sensor components include one or more of a binocular camera, a lidar (LiDAR) sensor, and an infrared sensor.

[0019] A lawnmower robot includes: a drive assembly including blades and a drive motor for driving the blades; a locomotion assembly including locomotion wheels and a locomotion motor for driving the locomotion wheels; a sensor assembly configured to collect environmental information around the lawnmower robot; a memory configured to store a global map of the lawnmower robot's working area; and a processor communicatively or electrically connected to the sensor assembly and the memory, configured to control the lawnmower robot's actions based at least on the environmental information collected by the sensors; wherein the processor is further configured to: construct a local map of the lawnmower robot's real-time position based on the environmental information collected by the sensor assembly, the local map having a higher resolution than the global map.

[0020] In some implementations, local maps contain more types of map attributes than global maps.

[0021] In some embodiments, the resolution of the local map is greater than or equal to 1 cm and less than or equal to 20 cm.

[0022] In some embodiments, the processor maps map information from the global map to the local map based on the real-time location of the lawnmower robot.

[0023] In some embodiments, the shape of the local map is fixed, and the distance between the boundary of the local map and the real-time position of the lawnmower robot is less than or equal to 5m.

[0024] In some embodiments, the processor is also configured to plan the path of the lawnmower robot based on a global map and a local map, and to control the walking motors based on the path.

[0025] In some embodiments, the local map consists of multiple map grids with different resolutions.

[0026] In some embodiments, the map grid with the smaller distance to the real-time position of the lawnmower robot has a higher resolution.

[0027] In some embodiments, the sensor assembly includes at least one of a vision sensor, a collision sensor, an inertial sensor, a lidar, and an odometer.

[0028] In some embodiments, the processor updates the local map in real time.

[0029] A lawnmower robot includes: a drive assembly including blades and a drive motor for driving the blades; a walking assembly including wheels and a drive motor for driving the wheels; a sensor assembly including at least one of a vision sensor, a collision sensor, an inertial sensor, an odometer, and a lidar; a memory configured to store a map of the lawnmower robot's working area, the map being divided into multiple map grids; and a processor communicatively or electrically connected to the sensor assembly and the memory, configured to control the lawnmower robot's actions based at least on environmental information collected by the sensor assembly; wherein the processor is further configured to: determine road conditions based on the environmental information collected by the sensor assembly, the road conditions belonging to at least one of a preset multiple road condition types; record the road conditions on the corresponding map grids in the map; the processor also includes a road condition update module configured to update the road conditions on the corresponding map grids in the map based on newly collected environmental information from the sensor assembly.

[0030] In some embodiments, the lawnmower robot also includes a human-machine interface component configured to allow a user to update road conditions on one or more map grids in a map using human-machine interaction methods.

[0031] In some embodiments, the preset multiple road condition types include at least two of the following: bumpy area, pothole area, slope area, non-grass area, obstacle-prone area, slippery area, and drowsy area.

[0032] In some embodiments, the processor is further configured to record the road condition probability corresponding to the road condition on the corresponding map grid in the map; the road condition update module is further configured to update the road condition probability on the corresponding map grid in the map based on the environmental information newly collected by the sensor components.

[0033] In some embodiments, the processor is further configured to filter the environmental information collected by the sensor components based on the temporal and spatial attributes of the environmental information collected by the sensor components; the traffic update module updates the traffic conditions and traffic probability based on the filtered environmental information collected by the sensor components.

[0034] In some embodiments, the processor is also configured to determine the cost of a map grid based on the road conditions and road condition probabilities on each map grid, and to plan the walking path of the lawnmower robot based on the cost of each map grid.

[0035] In some embodiments, the memory is also configured to store at least one second map attribute on each grid cell in the map; the processor is also configured to adjust the walking path based on the second map attribute.

[0036] In some embodiments, the second map attribute includes at least one of mowing, lift, collision, boundary, obstacle, and charging station.

[0037] In some embodiments, the processor controls the walking motor to drive the walking wheels at different speeds based on the cost value and / or second map attributes.

[0038] A lawnmower robot includes: a drive assembly including blades and drive motors for driving the blades; a locomotion assembly including locomotion wheels and drive motors for driving the locomotion wheels; a sensor assembly configured to collect environmental information around the lawnmower robot; a memory configured to store a global map of the lawnmower robot's working area; and a processor communicatively or electrically connected to the sensor assembly and the memory, configured to control the lawnmower robot's actions based at least on the environmental information collected by the sensors; wherein the processor is further configured to: map information in the global map to an initial local map around the real-time position of the lawnmower robot; update the initial local map using the environmental information collected by the sensor assembly to generate a local map around the real-time position; and control the drive motors based on the global map and the local map.

[0039] A lawnmower robot includes: a drive assembly comprising blades and a drive motor for driving the blades; a walking assembly comprising wheels and a drive motor for driving the wheels; a camera assembly configured to acquire two-dimensional images of the area surrounding the lawnmower robot; and a processor communicatively or electrically connected to the camera assembly, configured to control the movement of the lawnmower robot based at least on the two-dimensional images; wherein the processor is further configured to: identify adverse road conditions in the two-dimensional images through semantic segmentation, wherein the adverse road conditions belong to one of a preset plurality of adverse road condition types; verify the adverse road conditions based on a three-dimensional point cloud to obtain a verification result; and plan the movement route of the lawnmower robot based on the verification result.

[0040] In some embodiments, the processor is also configured to avoid or slow down verified adverse road conditions.

[0041] In some embodiments, the preset types of adverse road conditions include at least two of potholes, water, or haystacks.

[0042] In some embodiments, the camera assembly includes at least a first camera and a second camera, and the processor is further configured to generate a three-dimensional point cloud based on at least two two-dimensional images.

[0043] In some embodiments, the lawnmower robot also includes a sensor assembly that is communicatively or electrically connected to a processor and configured to acquire environmental information around the lawnmower robot to generate a three-dimensional point cloud and transmit the three-dimensional point cloud to the processor.

[0044] In some embodiments, the processor determines the depth information of the bad road conditions and the depth information of the walking plane of the lawnmower person based on the 3D point cloud, and uses the depth information of the bad road conditions and the depth information of the walking plane to verify the bad road conditions.

[0045] In some embodiments, the processor determines the depth information of the poor road conditions based on the coordinate information of the poor road conditions in the two-dimensional image and the registration relationship between the two-dimensional image and the three-dimensional point cloud.

[0046] In some embodiments, the processor obtains the bounding box of the poor road condition through semantic segmentation, and the coordinate information of the poor road condition is the coordinate information of the region within the bounding box in the two-dimensional image.

[0047] In some embodiments, the processor determines the depth information of the walking plane based on the coordinate information of the walking plane within a certain distance around the location of the poor road condition in the two-dimensional image and the registration relationship between the two-dimensional image and the three-dimensional point cloud.

[0048] In some embodiments, the processor obtains the bounding box of the poor road conditions through semantic segmentation, and the coordinate information of the walking plane is the coordinate information of the region outside the bounding box in the two-dimensional image.

[0049] In some embodiments, the processor determines that the poor road condition verification fails when the depth information of the walking plane does not conform to the preset plane depth information.

[0050] In some embodiments, the processor determines that the poor road condition verification is successful when the depth relationship between the depth information of the poor road condition and the depth information of the walking plane conforms to a preset depth relationship corresponding to the type of poor road condition.

[0051] A lawnmower robot includes: a drive assembly including blades and a drive motor for driving the blades; a walking assembly including walking wheels and a walking motor for driving the walking wheels; a camera assembly including a first camera and a second camera, the first camera being configured to acquire a first environmental image around the lawnmower robot, and the second camera being configured to acquire a second environmental image around the lawnmower robot, the first environmental image and the second environmental image being two-dimensional images; and a processor communicatively or electrically connected to the camera assembly, configured to control the movement of the lawnmower robot based at least on the first environmental image and the second environmental image; wherein the processor is further configured to: identify various objects contained in the first environmental image and the second environmental image respectively through semantic segmentation; associate repeated objects in the first environmental image and the second environmental image; perform stereo matching on the repeated objects in the first environmental image and the second environmental image to obtain depth information of the repeated objects; and control the walking motors based on the identification result of the repeated objects and the depth information.

[0052] In some embodiments, the first camera is positioned to the left of the second camera.

[0053] In some embodiments, the first camera and the second camera form a binocular camera.

[0054] In some embodiments, the first or second camera is a fisheye camera.

[0055] In some embodiments, the camera assembly also includes a structured light camera and / or a ToF (Time of Flight) camera.

[0056] A lawnmower robot includes: a drive assembly including blades and drive motors for driving the blades; a walking assembly including walking wheels and drive motors for driving the walking wheels; a camera assembly configured to acquire environmental images surrounding the lawnmower robot, the environmental images being two-dimensional or three-dimensional images; and a processor communicatively or electrically connected to the camera assembly, configured to control the movement of the lawnmower robot at least based on the environmental images; wherein the processor is further configured to: cluster pixels or voxels in the environmental images to obtain pixel sets or voxel sets corresponding to various objects contained in the environmental images; discard pixel sets or voxel sets when the size of the pixel sets or voxel sets is less than or equal to a first threshold; and identify the type of object corresponding to the pixel sets or voxel sets when the size of the pixel sets or voxel sets is greater than or equal to a second threshold, and control the walking motors according to the object type.

[0057] In some embodiments, the processor uses K-Means or region growing algorithms to cluster pixels or voxels in the environment image.

[0058] In some embodiments, the camera assembly includes at least two cameras, and the processor performs stereo matching on two two-dimensional environmental images captured by the two cameras at the same time to generate a corresponding three-dimensional environmental image.

[0059] In some embodiments, the processor converts the set of pixels into a corresponding set of voxels based on the camera parameters of the camera component.

[0060] In some embodiments, the size is the length of the planar diagonal of the bounding box of the pixel set, or the length of the three-dimensional diagonal of the bounding box of the voxel set.

[0061] In some embodiments, the size is one of the length, width, and height of the bounding box of the pixel set or voxel set.

[0062] In some embodiments, the size is the maximum value of the distance between two pixels or between two voxels in a set of pixels or voxels.

[0063] In some embodiments, the first threshold is less than or equal to 5 cm.

[0064] A lawnmower robot includes: a drive assembly including blades and drive motors for driving the blades; a walking assembly including wheels and drive motors for driving the wheels; a camera assembly configured to acquire environmental images surrounding the lawnmower robot, the environmental images being two-dimensional or three-dimensional images; and a processor communicatively or electrically connected to the camera assembly, configured to control the movement of the lawnmower robot at least based on the environmental images; wherein the processor is further configured to: cluster pixels or voxels in the environmental images to obtain pixel sets or voxel sets corresponding to various objects contained in the environmental images; discard pixel sets or voxel sets when the set parameter of the pixel set or voxel set is less than or equal to a parameter threshold; and identify the type of object corresponding to the pixel set or voxel set when the set parameter of the pixel set or voxel set is greater than or equal to the parameter threshold, and control the drive motors according to the object type.

[0065] In some embodiments, the set parameter is the number of pixels or the number of voxels.

[0066] In some embodiments, the parameter threshold corresponding to the number of voxel points is 15.

[0067] A lawnmower robot includes: a drive assembly including blades and a drive motor for driving the blades; a walking assembly including walking wheels and a walking motor for driving the walking wheels; a sensor assembly configured to collect environmental information around the lawnmower robot; a first processor configured to identify obstacles based on the environmental information and determine a set of coordinates of the obstacles; and a second processor configured to control the movement of the lawnmower robot; wherein the first processor is further configured to: determine representative coordinates of the obstacles based on the set of coordinates of the obstacles and transmit the representative coordinates to the second processor; and the second processor is further configured to: control the walking motors based on the representative coordinates of the obstacles.

[0068] In some embodiments, the sensor assembly includes a camera assembly configured to acquire images of the environment surrounding the lawnmower robot.

[0069] In some embodiments, the first processor is a GPU and the second processor is a CPU.

[0070] In some embodiments, the first processor is also configured to identify the type of obstacle in the environmental image and transmit the type of obstacle and the representative coordinates of the obstacle to the second processor.

[0071] In some embodiments, the representative coordinates are the coordinates of the vertices of the three-dimensional diagonal of the bounding box of the obstacle.

[0072] In some embodiments, the representative coordinates are the coordinates of the vertices of the plane diagonal of the bounding box of the obstacle.

[0073] In some embodiments, the sensor components include a binocular camera and / or a lidar (LiDAR) system.

[0074] In some embodiments, the amount of data representing coordinates is less than or equal to 100 bytes.

[0075] A control method for a lawnmower robot includes: a sensor assembly collecting environmental information around the lawnmower robot; a processor identifying obstacles based on the environmental information and determining whether the obstacles are static or dynamic; when the obstacles are determined to be dynamic, the processor using a first map for navigation; and when the obstacles are determined to be static, the processor using a second map for navigation.

[0076] A control method for a lawnmower robot includes: a sensor assembly collecting environmental information around the lawnmower robot; a processor constructing a local map of the real-time position of the lawnmower robot based on the environmental information, wherein the local map has a higher resolution than a global map, and the global map is stored in a memory.

[0077] A control method for a lawnmower robot includes: a sensor component collecting environmental information around the lawnmower robot; a processor determining road conditions based on the environmental information and recording the road conditions on a corresponding map grid in a map, wherein the road conditions belong to at least one of a plurality of preset road condition types, the map is stored in a memory and divided into a plurality of map grids; the processor also updates the road conditions on the corresponding map grid in the map based on newly collected environmental information from the sensor component.

[0078] A control method for a lawnmower robot includes: a sensor assembly collecting environmental information around the lawnmower robot; a processor mapping map information from a global map to an initial local map around the real-time position of the lawnmower robot, the global map being stored in a memory; the processor further updating the initial local map using the environmental information collected by the sensor assembly to generate a local map around the real-time position, and controlling the walking motor based on the global map and the local map.

[0079] A control method for a lawnmower robot includes: a camera assembly acquiring two-dimensional images of the area surrounding the lawnmower robot; a processor identifying adverse road conditions in the two-dimensional images through semantic segmentation, wherein the adverse road conditions belong to one of a set of preset adverse road condition types; the processor further verifying the adverse road conditions based on a three-dimensional point cloud to obtain a verification result, and planning the travel route of the lawnmower robot based on the verification result.

[0080] A control method for a lawnmower robot includes: a first camera in a camera assembly acquiring a first environmental image of the lawnmower robot's surroundings; a second camera in the camera assembly acquiring a second environmental image of the lawnmower robot's surroundings; the first environmental image and the second environmental image being two-dimensional images; a processor identifying objects contained in the first environmental image and the second environmental image through semantic segmentation, and associating repeating objects in the first environmental image and the second environmental image; the processor also performing stereo matching on the repeating objects in the first environmental image and the second environmental image to obtain depth information of the repeating objects, and controlling the walking motor based on the identification result of the repeating objects and the depth information.

[0081] A control method for a lawnmower robot includes: a camera assembly acquiring environmental images around the lawnmower robot, the environmental images being two-dimensional or three-dimensional images; a processor clustering pixels or voxels in the environmental images to obtain pixel sets or voxel sets corresponding to various objects contained in the environmental images; when the set parameter of the pixel set or voxel set is less than or equal to a parameter threshold, the processor discards the pixel set or voxel set; when the set parameter of the pixel set or voxel set is greater than or equal to the parameter threshold, the processor identifies the type of object corresponding to the pixel set or voxel set and controls the walking motor according to the type of object.

[0082] A control method for a lawnmower robot includes: a sensor assembly collecting environmental information around the lawnmower robot; a first processor identifying obstacles based on the environmental information and determining a set of coordinates for the obstacles; the first processor further determining representative coordinates for the obstacles based on the set of coordinates and transmitting the representative coordinates to a second processor; and the second processor controlling the walking motor based on the representative coordinates of the obstacles. Attached Figure Description

[0083] Figure 1 is a perspective view of a lawnmower robot as an example.

[0084] Figure 2 is a three-dimensional view of part of the internal structure of the lawnmower robot shown in Figure 1.

[0085] Figure 3 is a schematic diagram of the electronic control principle of the sensor components and processor in the lawnmower robot shown in Figure 1.

[0086] Figure 4 is a schematic diagram of the working area of ​​a lawnmower robot as an example.

[0087] Figure 5 is a schematic diagram of the first map of a lawnmower robot as an example.

[0088] Figure 6 is a schematic diagram of a second map of a lawnmower robot as an example.

[0089] Figure 7 is a schematic diagram of a partial map of a lawnmower robot as an example.

[0090] Figure 8 is a schematic diagram of the electrical control principle of the sensor components, memory, and processor in the lawnmower robot shown in Figure 1.

[0091] Figure 9 is a schematic diagram of the electrical control of the sensor components, the first processor, and the second processor in the lawnmower robot shown in Figure 1.

[0092] Captions: 100, lawnmower robot; 10, drive assembly; 20, walking assembly; 30, sensor assembly / camera assembly; 40, processor; 50, memory; 11, blade; 12, drive motor; 21, walking wheel; 22, walking motor; 41, first processor; 42, second processor. Detailed Implementation

[0093] Before explaining any implementation of this application in detail, it should be understood that this application is not limited to its application to the structural details and component arrangements set forth in the following description or shown in the above drawings.

[0094] In this application, the terms "comprising," "including," "having," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0095] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects have an "and / or" relationship.

[0096] In this application, the terms "connection," "combination," "coupling," and "installation" can refer to direct connection, combination, coupling, or installation, or indirect connection, combination, coupling, or installation. For example, a direct connection refers to two parts or components being connected together without the need for an intermediary, while an indirect connection refers to two parts or components each being connected to at least one intermediary, with the connection achieved through the intermediary. Furthermore, "connection" and "coupling" are not limited to physical or mechanical connections or couplings, but can also include electrical connections or couplings.

[0097] In this application, those skilled in the art will understand that relative terms (e.g., “about,” “approximately,” “basically,” etc.) used in conjunction with quantities or conditions are to include the values ​​and have the meaning indicated by the context. For example, such relative terms include at least the degree of error associated with the measurement of a particular value, tolerances associated with the particular value due to manufacturing, assembly, use, etc. Such terms should also be considered as disclosing a range defined by the absolute values ​​of the two endpoints. Relative terms may refer to a certain percentage (e.g., 1%, 5%, 10% or more) of the indicated value. Numerical values ​​not using relative terms should also be disclosed as specific values ​​with tolerances. Furthermore, “basically” when expressing relative angular relationships (e.g., substantially parallel, substantially perpendicular) may refer to a certain degree (e.g., 1 degree, 5 degrees, 10 degrees or more) added to or subtracted from the indicated angle.

[0098] In this application, those skilled in the art will understand that the function performed by a component can be performed by one component, multiple components, one part, or multiple parts. Similarly, the function performed by a part can also be performed by one part, one component, or a combination of multiple parts.

[0099] In this application, the directional terms "upper," "lower," "left," "right," "front," and "rear" are used to describe the orientation and positional relationships shown in the accompanying drawings and should not be construed as limiting the embodiments of this application. Furthermore, in the context, it should be understood that when an element is mentioned as being connected "upper" or "lower" to another element, it can be directly connected to the other element "upper" or "lower," or indirectly connected through an intermediate element. It should also be understood that directional terms such as upper side, lower side, left side, right side, front side, and rear side not only represent positive orientation but can also be understood as lateral orientation. For example, "below" can include directly below, lower left, lower right, lower front, and lower rear.

[0100] In this application, the terms "controller," "processor," "central processing unit," "CPU," and "MCU" are used interchangeably. When using the unit "controller," "processor," "central processing unit," "CPU," or "MCU" to perform a specific function, unless otherwise stated, these functions may be performed by a single or multiple of the aforementioned units.

[0101] In this application, the terms "device," "module," or "unit" are used to describe devices that can be implemented in hardware or software to perform a specific function.

[0102] In this application, the terms “calculation,” “judgment,” “control,” “determine,” “identify,” etc., refer to the operation and process of a computer system or similar electronic computing device (e.g., controller, processor, etc.).

[0103] As a trendy outdoor walking power tool, lawn mowing robots are increasingly favored by users due to their intelligent and efficient advantages. In areas where lawn mowing is to be done, traditional lawn mowers require users to push them manually, while lawn mowing robots can autonomously plan their paths and carry out mowing operations along those paths. Furthermore, lawn mowing robots can also perform functions such as object recognition and obstacle avoidance during their movement.

[0104] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0105] Figure 1 illustrates a lawnmower robot 100 as one embodiment of this application. It is understood that the technical solutions described in this application can also be adapted to different scenario requirements and selectively applied to other outdoor powered vehicles such as snowplows.

[0106] Referring to Figures 1 and 2, the lawnmower robot 100 includes a drive assembly 10 and a walking assembly 20. The drive assembly 10 includes blades 11 and a drive motor 12 that drives the blades. The motor shaft of the drive motor, upon rotation, transmits power directly or indirectly through a first transmission assembly to a first output shaft to drive the blades for mowing. The walking assembly 20 includes wheels 21 and a walking motor 22 that drives the wheels. The motor shaft of the walking motor, upon rotation, transmits power directly or indirectly through a second transmission assembly to a second output shaft to drive the wheels along a planned path. In some examples, the walking motor can be a hub motor, which integrates power, transmission, and braking-related devices within the wheel hub.

[0107] Referring to Figure 3, the lawnmower robot 100 also includes a sensor assembly 30 and a processor 40. The sensor assembly 30 may include one or more of the following: a vision sensor, a collision sensor, an inertial sensor, an electromagnetic sensor, a lidar sensor, and an odometry device. The specific number of each type of sensor included in the sensor assembly 30 is not limited. It is understood that the sensor assembly 30 may also include other types of sensors not listed above.

[0108] The sensor component 30 is configured to collect environmental information around the lawnmower robot 100. In some examples, the sensor component 30 can collect environmental information within a preset distance range centered on the real-time location of the lawnmower robot 100. The environmental information can be presented in the form of one-dimensional signals, two-dimensional images, or three-dimensional point clouds.

[0109] In some examples, sensor component 30 may include a camera component, an infrared sensor, a linear light sensor, or other visual sensors, which can acquire environmental images of the lawnmower robot 100 in its current pose and real-time position within the sensor's line of sight. In other examples, sensor component 30 may include a LiDAR (Light Detection and Ranging) system, which can acquire laser point clouds around the lawnmower robot 100 within the sensor's radiation range. It should be noted that in some cases, sensor component 30, such as a binocular camera, may also transmit the acquired two-dimensional images to processor 40, and then processor 40 generates three-dimensional images from the two-dimensional images. In still other examples, sensor component 30 may include an inertial sensor, which can detect data such as the three-axis acceleration of the lawnmower robot 100.

[0110] The processor 40 can be a CPU (Central Processing Unit) or an MCU (Microcontroller Unit), which reads and runs various executable instructions to control the actions of the lawnmower robot 100.

[0111] Referring to Figure 4, which illustrates the working area of ​​the lawnmower robot 100, the working area is typically defined by boundary lines to restrict the robot's movement to the designated mowing area. In some examples, these boundary lines can be physical boundaries; in others, they can be virtual boundaries implemented using software. Whether physical or virtual, these boundaries define the working area of ​​the lawnmower robot 100, and the specific shape of the working area is not limited. For example, the working area can be rectangular or other irregularly shaped areas.

[0112] Since the lawn mowing robot 100 generally aims to cover the working area for mowing operations, in order to achieve effective path planning, the lawn mowing robot 100 also needs to build a map corresponding to its working area. In some simple examples, the lawn mowing robot 100 can always use the same pre-stored map for navigation, but these maps provide limited information and have poor timeliness, which is not conducive to the path planning and obstacle avoidance of the lawn mowing robot 100.

[0113] In one embodiment, the lawnmower robot 100 includes at least a drive component 10, a walking component 20, a sensor component 30, and a processor 40. The sensor component 30 collects environmental information near the real-time position of the lawnmower robot 100, while the processor 40 is electrically or communicatively connected to the sensor component 30 and can acquire the environmental information collected by the sensor component 30. In some examples, the processor 40 can periodically receive environmental information from the sensor component 30 corresponding to the real-time position of the lawnmower robot 100. For example, the processor 40 can periodically receive environmental information collected by the sensor component 30 with a period of 100ms. Therefore, within 1 second, the processor 40 can interact with the sensor component 30 about environmental information 10 times. The 10 interactions can all be about the environment when the lawnmower robot 100 is in the same position, or they can be about the environment when the lawnmower robot 100 is in different positions.

[0114] After obtaining environmental information corresponding to the real-time position of the lawnmower 100, the processor 40 can identify obstacles near the real-time position of the lawnmower 100 based on the environmental information and determine whether the obstacles are dynamic or static. Dynamic obstacles can be people, animals, or other electronic devices moving near the lawnmower 100, while static obstacles can be non-moving obstacles such as stone steps, trees, benches, etc.

[0115] If the processor 40 determines that the obstacles appearing around the lawnmower robot 100 are dynamic obstacles, it can call the first map to navigate the lawnmower robot 100. Specifically, the processor 40 can call the first map, plan a first travel path on the first map, and control the walking motor 22 based on the first travel path to make the lawnmower robot 100 walk on the first travel path. In some cases, the processor 40 also controls the drive motor 12 based on the first map to make the lawnmower robot 100 adjust the blade cutting state on the first travel path, etc.

[0116] If the processor 40 determines that the obstacles appearing around the lawnmower robot 100 are static obstacles, it can call the second map to navigate the lawnmower robot 100. Specifically, the processor 40 can call the second map, plan a second travel path on the second map, and control the walking motor 22 based on the second travel path to make the lawnmower robot 100 walk on the second travel path. In some cases, the processor 40 also controls the drive motor 12 based on the second map to make the lawnmower robot 100 adjust the blade cutting state on the second path, etc.

[0117] In some embodiments, the processor 40 may also include a map pointer, which can be used to call the corresponding map when a map needs to be switched.

[0118] As shown in Figure 4, the working area of ​​the lawnmower robot 100 can contain multiple objects belonging to different types. These objects can be dynamic obstacles, static obstacles, backgrounds such as air and ground, or specific road conditions such as potholes and bumps. The processor 40 can adapt to the needs of the actual scene and use different algorithms to identify the type of each object in the working area, or it can identify objects belonging to one or more specific types in the working area.

[0119] In some embodiments, sensor assembly 30 includes a camera assembly that can acquire environmental images around the lawnmower robot 100. Processor 40 can use semantic segmentation algorithms to identify obstacles appearing in the environmental images and determine the type of obstacles. When the type of obstacle is a person, animal, or other moving electronic device, it can be determined to be dynamic. In some examples, geometric analysis methods can also be combined to identify dynamic and static obstacles in the three-dimensional environmental images. In other embodiments, processor 40 can also use light streaming or multi-frame target tracking methods to detect whether the environmental images contain dynamic objects.

[0120] Referring to Figure 5, in some embodiments, the first map is a global map, which is a map of the entire working area where the lawnmower robot 100 is to perform lawnmowing operations. When dynamic obstacles such as people or animals appear around the lawnmower robot 100, the area that may be affected is wider because the dynamic obstacles have a large range of movement. In cases such as when the lawnmower robot 100 performs lawnmowing operations in a bow-shaped path, the processor 40 can call the global map to replan the global travel path under the global map.

[0121] Referring to Figure 6, in some embodiments, the second map is a local map, which is a map of a portion of the working area within a certain distance of the real-time position of the lawnmower robot 100. In some examples, the local map may be a map of a portion of the working area with the real-time position of the lawnmower robot 100 as the origin and the direction of travel of the lawnmower robot 100 as the coordinate axis. In other examples, the local map may also be a map of a portion of the working area centered on the real-time position of the lawnmower robot 100 and within a preset distance threshold from the center. When static obstacles such as trees or stone steps appear around the lawnmower robot 100, since the positions of static obstacles are relatively fixed, the processor 40 can call the local map to replan the local travel path under the local map.

[0122] In addition, navigation using a global or local map can incorporate other attributes of dynamic or static obstacles during path planning. For example, when using a local map for navigation around obstacles such as trees or flower beds, the processor 40 can control the walking motor 22 to make the lawnmower robot 100 perform a circular mowing task around the trees or flower beds. Similarly, when using a local map for navigation around obstacles such as stone steps or boundary lines, the processor 40 can control the walking motor 22 to make the lawnmower robot 100 perform an edge mowing task along the stone steps or boundary lines.

[0123] In some embodiments, the second map has a higher resolution than the first map. In some examples, the resolution of the local map can be greater than or equal to 1 cm and less than or equal to 20 cm, while the resolution of the global map can be greater than or equal to 15 cm and less than or equal to 50 cm. In other examples, the resolution of the local map can be greater than or equal to 1 cm and less than or equal to 15 cm, while the resolution of the global map can be greater than or equal to 15 cm and less than or equal to 30 cm. It is understood that the resolution of the first and second maps should be adapted to the size of the current working area of ​​the lawnmower robot 100.

[0124] In some embodiments, the second map contains more map attributes than the first map. In some examples, the environmental information surrounding the real-time position of the lawnmower robot 100 collected by the sensor component 30 can be entirely recorded in the local map and partially recorded in the global map. For example, triaxial acceleration data detected by the inertial sensor and odometer data collected by the odometer can be recorded in the local map but not in the global map.

[0125] In some embodiments, the second map is updated in real time as the lawnmower 100 moves. For example, assuming the lawnmower 100 moves at a speed of 0.5 m / s, the second map can be updated periodically with a period of 100 ms, that is, the local map is updated once every 5 cm that the lawnmower 100 moves.

[0126] In some embodiments, the processor 40 controls the walking motor 22 to drive the walking wheels at a first speed when using first map navigation, and controls the walking motor 22 to drive the walking wheels at a second speed when using second map navigation. In some examples, the first speed may be lower than the second speed. For example, the walking speed of the lawnmower robot 100 when using global map navigation may be lower than the walking speed of the lawnmower robot 100 when using local map navigation, in order to ensure the safety of the lawnmower robot 100 when facing dynamic obstacles. In other examples, the walking speed of the lawnmower robot 100 can be further adjusted by combining other map attributes.

[0127] In some embodiments, sensor assembly 30 includes one or more of a binocular camera, a lidar (LiDAR) sensor, and an infrared sensor.

[0128] In the lawnmower robot described in the above embodiments, different maps are selected for navigation based on whether the obstacles appearing near the real-time location of the lawnmower robot are dynamic or static. The maps used can adapt to the different characteristics of dynamic and static obstacles and the requirements for path planning and obstacle avoidance in scenarios where dynamic and static obstacles appear, thereby improving the working efficiency and results of the lawnmower robot.

[0129] In one embodiment, the lawnmower robot 100 includes at least a drive assembly 10, a walking assembly 20, a sensor assembly 30, a memory 50, and a processor 40. The memory 50 stores a global map of the lawnmower robot 100's working area. In some examples, the memory 50 can be ROM (Read-Only Memory), RAM (Random Access Memory), or Flash memory; in other examples, it can be a remote memory. The global map is a map of the entire working area where the lawnmower robot 100 will perform mowing operations, and it can record relatively stable information within the overall working area in a relatively concise manner.

[0130] Referring to Figure 8, the processor 40 is electrically or communicatively connected to the sensor assembly 30 and the memory 50, and is able to acquire the global map stored in the memory 50 and various environmental information collected by the sensor assembly 30. In some examples, the processor 40 can periodically receive various environmental information collected by the sensor assembly 30, wherein the periods for different sensors to send environmental information can be the same or different.

[0131] After obtaining various environmental information, the processor 40 can construct a local map of the real-time position of the lawnmower robot 100 based on this information. The resolution of the local map is higher than that of the global map, allowing it to record more volatile information within the working area corresponding to the real-time position of the lawnmower robot 100 in a more detailed manner. Specifically, the processor 40 can identify obstacles based on environmental images from a visual sensor and construct a local map containing the location and type information of the obstacles; it can also construct a local map containing depth information of each object based on laser point clouds from a LiDAR sensor; and it can construct a local map containing the speed and direction of the lawnmower robot 100 based on three-axis acceleration data from an inertial sensor.

[0132] In some embodiments, the shape of the local map can be fixed, that is, the shape of the local map is a preset shape. In some examples, the shape of the local map can be a regular quadrilateral, and in other examples, the shape of the local map can also be a circle.

[0133] In some embodiments, the local map can be constructed using the real-time position of the lawnmower 100 as the origin and the direction of travel of the lawnmower 100 as the coordinate axis. In some examples, the local map can be a 4m*4m regular quadrilateral map with the real-time position of the lawnmower 100 as the origin, or it can be a 3m*3m regular quadrilateral map, a 5m*5m regular quadrilateral map, or a 6m*6m regular quadrilateral map. The side length of the local map can be adaptively adjusted according to the size of the lawnmower's working area and the lawnmower's traveling speed. For example, the larger the working area and the faster the lawnmower travels, the larger the side length of the local map, and vice versa. In other embodiments, the local map can be a map of the working area centered on the real-time position of the lawnmower 100 and within a preset distance threshold. For example, the boundary of the local map can be a circle with the real-time position of the lawnmower 100 as the center and a preset distance threshold as the radius. In some examples, the preset distance threshold can be 4m, or it can be 3m, 5m, or 6m. It is understandable that, given a defined work area boundary, the coverage area of ​​a local map does not need to exceed the boundary; that is, it is not necessary to construct a local map for areas that exceed the boundary.

[0134] As shown in Figures 5 and 6, a global or local map consists of multiple map grids, and map resolution characterizes the size of the map grids. In some cases, map grids can be regular quadrilaterals, and map resolution can be expressed by the grid side lengths. The map grid resolution described below refers to the grid side lengths. In this embodiment, the local map has a higher resolution than the global map; in some examples, the resolution of the local map can be 5 cm, and the resolution of the global map can be 15 cm.

[0135] In some embodiments, the resolution of multiple map grids in a local map may be different.

[0136] Referring to Figure 7, in some embodiments, the resolution of multiple map grids in the local map can vary depending on the distance of the map grid from the real-time position of the lawnmower 100. The closer the map grid is to the real-time position of the lawnmower 100, the higher the resolution of the map grid. That is, the map grid closer to the real-time position of the lawnmower 100 in the local map has a smaller grid side length, and the map grid farther away from the real-time position of the lawnmower 100 has a larger grid side length.

[0137] In other embodiments, in addition to varying the distance between the map grid and the real-time position of the lawnmower 100, a resolution threshold may also be provided. Once the resolution of the map grid reaches the resolution threshold, that is, once the grid side length reaches the maximum value of the grid side length, the map resolution will no longer be reduced even if the distance between the map grid and the real-time position of the lawnmower 100 is greater.

[0138] The resolution of the local map can be adjusted to suit the size of the local map area, the size of the working area of ​​the lawnmower 100, and the required working accuracy. In some embodiments, the maximum resolution of the map grid in the local map can be 1 cm, and the minimum resolution can be 15 cm.

[0139] In some embodiments, the local map contains more types of map attributes than the global map. In some examples, the global map may not record information about dynamic obstacles, while the local map will. In other examples, the global map may not record mileage data and travel speed, while the local map will record mileage data and travel speed.

[0140] In some embodiments, as the lawnmower 100 moves, the processor 40 updates the local map corresponding to the current real-time position in real time as its real-time position changes. For example, assuming the lawnmower 100's travel speed is 0.5 m / s, the processor 40 can periodically update the local map with a period of 100 ms. Compared to the local map generated in the previous period, the real-time position of the lawnmower 100 in the current local map has moved by 5 cm.

[0141] In some embodiments, the processor 40 can use a locally generated map corresponding to the real-time position of the lawnmower 100 and / or a stored global map to plan the path of the lawnmower 100 and control the walking motor 22 based on the planned path. In some examples, the processor 40 can also control the drive motor 12 based on the local map and / or the global map.

[0142] In some embodiments, sensor assembly 30 includes at least one of a vision sensor, a collision sensor, an inertial sensor, a lidar (LiDAR) system, and an odometer.

[0143] In some embodiments, when generating a local map corresponding to the real-time position of the lawnmower 100, the processor 40 can first read the global map stored in the memory 50, map the map information in the global map to the local map to be generated to obtain an initial local map, and then update the initial local map based on the environmental information collected by the sensors to obtain the local map used for subsequent navigation. Specifically, the processor 40 can determine the map area corresponding to the local map to be generated based on the real-time position of the lawnmower 100, read the global map and map the map information in that map area in the global map to the local map to be generated to generate the initial local map. That is, the initial local map is a crop of that map area in the global map. The processor 40 obtains the environmental information near the real-time position of the lawnmower 100 collected by the sensor component 30, and analyzes the environmental information to determine map information not contained in the global map, such as dynamic and static obstacles, bumps, potholes, etc., and uses them to update the initial local map, and then uses the updated local map to realize the navigation of the lawnmower 100.

[0144] In the lawnmower robot described in the above embodiments, the processor can generate a local map corresponding to the real-time position of the lawnmower robot. The local map has a higher resolution than the global map and can also contain more map attributes that can reflect the real-time movement status, which can provide more accurate guidance for the navigation and other movement control of the lawnmower robot.

[0145] In one embodiment, the lawnmower robot 100 includes at least a drive assembly 10, a walking assembly 20, a sensor assembly 30, a memory 50, and a processor 40. The memory 50 stores a map of the working area of ​​the lawnmower robot 100, which may be a complete map or a partial map of the working area.

[0146] The processor 40 is electrically or communicatively connected to the sensor assembly 30, enabling it to acquire various environmental information collected by the sensor assembly 30. After obtaining this environmental information, the processor 40 can determine road conditions based on it. Specifically, the processor 40 can acquire environmental information such as images, mileage data, and acceleration data near different locations within the working area of ​​the lawnmower 100, collected by the sensor assembly 30. Analyzing and processing this environmental information can determine the obstacle conditions, bump conditions, and collision conditions near different locations within the working area, thereby determining the road conditions near different locations within the working area. The processor 40 can be pre-set with multiple road condition types to determine the road conditions near each location, i.e., to determine the road condition type to which the road conditions near that location belong. In some examples, the processor 40 can determine the road conditions near the real-time location of the lawnmower 100 based on environmental information collected by the sensor assembly 30 near the real-time location of the lawnmower 100. In other examples, the processor 40 can also determine the road conditions near multiple different locations based on environmental information collected by the sensor assembly 30 near multiple different locations of the lawnmower 100.

[0147] In some embodiments, the preset road condition types include, but are not limited to, at least two of the following: bumpy areas, pothole areas, slope areas, non-grass areas, obstacle-prone areas, slippery areas, and areas prone to getting stuck.

[0148] In some embodiments, the processor 40 can determine whether the area near the current position is a bumpy region based on triaxial acceleration data collected by an inertial sensor. In other embodiments, the processor 40 can determine whether the area near the current position is a tunnel or a ramp based on images collected by a visual sensor and triaxial acceleration data collected by an inertial sensor. In still other embodiments, the processor 40 can determine whether the area near the current position is an area with many obstacles based on environmental images collected by a visual sensor and overload current of the encoder. In yet another embodiment, the processor 40 can locate and detect slippage areas and areas prone to getting stuck based on odometer positioning.

[0149] In some embodiments, the processor 40 can control the walking motor 22 based on road conditions to control the travel path and travel speed, etc. In some examples, the processor 40 can also control the drive motor 12 based on road conditions to adjust the blade cutting state, etc.

[0150] The processor 40 is also electrically or communicatively connected to the memory 50 and is able to interact with the memory 50 to exchange map information.

[0151] As shown in Figures 5 to 7, the map is composed of multiple map grids. Taking the global map as an example, each map grid corresponds to a different part of the working area. Each map grid records map information such as road conditions in the part of the working area corresponding to that map grid.

[0152] After determining the road conditions near a location in the working area, the processor 40 can interact with the memory 50 to record the determined road conditions on the map grid corresponding to that location. When the processor 40 reaches the area corresponding to that map grid again at other times, the road condition update module in the processor 40 can redetermine the road conditions based on the environmental information newly collected by the sensor component 30, and interact with the memory 50 to update the map information on the map grid with the redetermined road conditions.

[0153] In some embodiments, the lawnmower robot 100 also includes a human-machine interface component that allows a user to update road conditions on one or more map grids using human-machine interaction methods. Specifically, map information such as road conditions on one or more map grids in the map stored in the memory 50 can be modified through the data interface of the lawnmower robot 100.

[0154] In some embodiments, in addition to determining and recording the road conditions on the map grid, the processor 40 also determines and records the probability of different road conditions appearing on the map grid. Specifically, after the processor 40 analyzes various environmental information collected by sensors to obtain the road conditions on a map grid, it can determine the probability of the road condition appearing on the map grid based on the cumulative number of times the road condition appears on the map grid or the frequency of the road condition appearing on the map grid within a preset time period, and record it in the map. For example, the probability of frequently occurring road conditions is high, and the probability of occasional road conditions is low. Similarly, when the processor 40 reaches the corresponding area of ​​the map grid again at other times, the road condition update module in the processor 40 can redetermine the road condition probability based on the newly collected environmental information from the sensor component 30, and interact with the memory 50 to update the map information on the map grid with the redetermined road condition probability. In other embodiments, the processor 40 can also combine the temporal and spatial attributes of the environmental information collected by the sensor components 30 to determine the road condition probability of different road conditions on each map grid. For example, it can determine that the road condition probability on a map grid is high during one time period and low during other time periods. Alternatively, when the lawnmower robot 100 determines the same road condition on a map grid multiple times at different times, from different travel directions, or along different travel paths, it can increase the road condition probability of that road condition on that map grid.

[0155] In some embodiments, before recording or updating the road conditions and road condition probabilities on each map grid, the processor 40 may filter the environmental information collected by the sensor component 30 and use the filtered environmental information to record or update the road conditions and road condition probabilities on each map grid. Specifically, the processor 40 may filter the environmental information based on the temporal and spatial attributes of the environmental information collected by the sensor component 30. For example, environmental information collected at similar locations and times has high data similarity, and abnormal data can be filtered out.

[0156] In some embodiments, after the road conditions and probabilities of each map grid are recorded and updated, the processor 40 can also calculate the cost value of each map grid based on the road conditions and probabilities, and plan the walking path of the lawnmower robot 100 based on the cost values ​​of each map grid. Specifically, the processor 40 can calculate the cost value of each map grid based on the preset cost weights corresponding to each road condition type, as well as the road conditions and probabilities of each map grid, and then plan the walking path of the lawnmower robot 100 with the goal of lowering the cost value of each map grid. For example, the higher the road condition danger level and the higher the road condition probability, the higher the cost value of the map grid, and map grids with high cost values ​​can be avoided as much as possible when planning the walking path.

[0157] In some embodiments, in addition to determining and recording the road conditions and road condition probabilities on the map grid, the processor 40 can also determine at least one of the second map attributes such as lawn mowing, lifting, collision, boundary, obstacle, and charging pile near different locations in the working area based on various environmental information collected by the sensor, and record the determined second map attribute on the map grid corresponding to that location.

[0158] In some embodiments, in addition to planning the walking path of the lawnmower robot 100 based on the cost value of map grids, the processor 40 can also adjust the planned walking path based on at least one of the second map attributes such as mowing, lifting, collision, boundary, obstacle, and charging post. For example, after planning the walking path based on the cost value of each grid, the processor 40 can optimize and adjust the planned walking path based on the boundary attribute to further ensure that the planned walking path does not exceed the working area. In other embodiments, the second map attribute can also be combined when calculating the cost value of each map grid.

[0159] In the lawnmower robot described in the above embodiments, the processor can generate a map that is continuously and dynamically updated, with each map grid labeled with road conditions, road condition probabilities, cost values, and other map information. This provides more accurate guidance for the lawnmower robot's navigation and other action control.

[0160] In one embodiment, the lawnmower robot 100 includes at least a drive component 10, a walking component 20, a camera component 30, and a processor 40. The camera component 30 can capture two-dimensional images of the environment surrounding the lawnmower robot 100, specifically RGB images or grayscale images, without limitation.

[0161] The processor 40 is electrically or communicatively connected to the camera assembly 30, enabling it to acquire two-dimensional images captured by the camera assembly 30. After obtaining the two-dimensional images, the processor 40 can identify adverse road conditions in the two-dimensional images through semantic segmentation, and verify the identified adverse road conditions based on a three-dimensional point cloud to obtain verification results. Then, based on the verification results of the adverse road conditions, the processor 40 plans the travel route of the lawnmower robot 100. Specifically, the processor 40 uses a semantic segmentation algorithm to obtain segmented images corresponding to the two-dimensional images and the type of each object within them. The processor 40 can pre-set multiple adverse road condition types. When an object in the two-dimensional image belongs to one of the pre-set adverse road condition types, the location and type of an adverse road condition in the two-dimensional image are identified, thus obtaining the location and type of each adverse road condition in the two-dimensional image. Then, the processor 40 can use the three-dimensional point cloud to verify each adverse road condition, and plan the travel path of the lawnmower robot 100 based on the verification results of each adverse road condition.

[0162] In some embodiments, the preset types of adverse road conditions include, but are not limited to, at least two of the following: potholes, water, and haystacks.

[0163] In some embodiments, the processor 40 can determine the depth information of each adverse road condition and the depth information of the walking plane where the lawnmower robot 100 is located based on the 3D point cloud, and then verify the adverse road conditions based on the depth information of the adverse road conditions and the depth information of the walking plane. Specifically, the processor 40 can register the 2D image and the 3D point cloud to obtain the registration relationship between the 2D image and the 3D point cloud. The registration between the two is the calibration between the 2D image coordinate system and the 3D point cloud coordinate system. Based on the registration relationship between the two, the depth information of the adverse road condition can be calculated from the coordinate information of the identified adverse road condition in the 2D image. Furthermore, based on the registration relationship between the two, the depth information of the walking plane near the adverse road condition can also be calculated from the coordinate information of the walking plane in the 2D image.

[0164] In some embodiments, the coordinate information of each adverse road condition can be the coordinate information of the region within the bounding box obtained by the processor 40 when identifying the adverse road condition using a semantic segmentation algorithm. In some examples, the processor 40 can use a semantic segmentation algorithm to segment a two-dimensional image into multiple image regions and identify the type of objects within each image region. The coordinate information of the image region where the identified object belongs to a preset adverse road condition type is located is the coordinate information of the adverse road condition. In other examples, the processor 40 can also use a semantic segmentation algorithm to outline the adverse road conditions in the identified two-dimensional image in the form of a bounding box. The coordinate information of the image region within the bounding box is the coordinate information of the adverse road condition.

[0165] In some embodiments, the depth information of the walking plane can be the coordinate information of the region outside the bounding box obtained by the processor 40 when identifying the poor road condition using a semantic segmentation algorithm. Specifically, after determining the image region where the poor road condition is located, the processor 40 can take the complement of the image region in the two-dimensional image as the image region where the walking plane is located, and the coordinate information of the image region corresponding to the complement is the coordinate information of the walking plane.

[0166] In some embodiments, the processor 40 can detect whether the depth information of the walking plane on which the lawnmower 100 is located conforms to preset plane depth information, and if not, determine that the verification of poor road conditions has failed. Specifically, the processor 40 can fit a plane based on the depth information and verify the centroid and / or normal vector of the fitted plane. If its centroid or normal vector does not conform to the centroid or normal vector of a preset standard ground, for example, if the distance error between the centroid and the standard centroid exceeds a threshold or the angle error between the normal vector and the standard normal vector exceeds a threshold, the verification of poor road conditions is determined to have failed.

[0167] In some embodiments, the processor 40 can also verify the poor road condition based on the depth relationship between the depth information of the poor road condition and the depth information of the walking plane. If the depth relationship between the two matches the depth relationship corresponding to the type of poor road condition, the poor road condition is determined to be successfully verified. For example, for a poor road condition of type pothole, the depth relationship between the poor road condition and the walking plane should be that the poor road condition is below the walking plane. If the relationship between the depth information of the two satisfies the above relationship, the pothole can be determined to be successfully verified.

[0168] The processor 40 can generate the 3D point cloud for verifying poor road conditions, or it can collect the data from the sensor assembly 30. In some embodiments, the camera assembly 30 includes at least a first camera and a second camera, and the processor 40 can generate the 3D point cloud based on at least two 2D images collected by the camera assembly 30. For example, the processor 40 can generate the corresponding 3D point cloud from two 2D images collected by the first camera and the second camera at the same time using a binocular camera model. In other embodiments, the lawnmower 100 also includes sensors such as LiDAR, which can collect environmental information around the lawnmower 100, generate the corresponding 3D point cloud, and transmit it to the processor 40.

[0169] In the lawnmower robot described in the above embodiments, the processor can use three-dimensional point clouds to verify the adverse road conditions identified in the two-dimensional image, and combine the verification results of the adverse road conditions to plan the movement path of the lawnmower robot, thereby reducing the impact of misjudgment of adverse road conditions on path planning and improving the accuracy of the lawnmower robot's navigation and other action control.

[0170] In one embodiment, the lawnmower robot 100 includes at least a drive component 10, a walking component 20, a camera component 30, and a processor 40. The camera component 30 includes at least a first camera and a second camera. The first camera captures a first environmental image of the lawnmower robot 100, and the second camera captures a second environmental image of the lawnmower robot 100. Both the first and second environmental images are two-dimensional images, specifically RGB images or grayscale images, without limitation.

[0171] In some embodiments, the first camera is positioned to the left of the second camera. It is understood that this describes the relative positional relationship between the first and second cameras; the first camera may also be positioned to the right, above, or below the second camera. In other embodiments, the first and second cameras together form a binocular camera, which can be a passive or active binocular camera. In still other embodiments, the first and / or second camera is a fisheye camera. In yet another embodiment, the camera assembly 30 further includes other types of depth cameras such as a structured light camera or a ToF (Time of Flight) camera.

[0172] The processor 40 is electrically or communicatively connected to the camera assembly 30, and is capable of acquiring a first environmental image and a second environmental image captured by the camera assembly 30. After obtaining the first and second environmental images, the processor 40 can identify the location and type of each object in the first and second environmental images through semantic segmentation. Specifically, the processor 40 receives a first environmental image and a second environmental image at the same time. Through semantic segmentation, these two environmental images can be segmented and identified to obtain a first segmented image corresponding to the first environmental image and the type of each object therein, and a second segmented image corresponding to the second environmental image and the type of each object therein. In some examples, the segmented images obtained through semantic segmentation can divide the original environmental image into multiple image regions, and the type of objects within each image region can be identified. It is understood that in other embodiments, the processor 40 may also combine instance segmentation and panoramic segmentation on the basis of semantic segmentation.

[0173] After identifying each object in the first environmental image and the second environmental image, the processor 40 can associate duplicate objects that appear in both the first and second environmental images. Specifically, the processor 40 can match objects based on their position and type in the first and second environmental images and associate objects that are in the same or similar positions and belong to the same type.

[0174] The processor 40 can perform stereo matching on each associated repeating object in the first and second environmental images to obtain the depth information of each repeating object. Then, based on the recognition results and depth information of each repeating object, the processor 40 controls the walking motor 22. In some cases, the processor 40 also controls the drive motor 12 based on the recognition results and depth information of each repeating object. Specifically, for a repeating object, due to the difference in viewing angle between the first and second cameras, the processor 40 can obtain a disparity map corresponding to the repeating object from the difference in coordinate information between the first and second environmental images, and calculate its depth information based on the disparity map. In some examples, the processor 40 can use a corresponding binocular camera model to calculate the depth information.

[0175] In some embodiments, after associating duplicate objects, the processor 40 may perform stereo matching only on duplicate objects belonging to a preset type, so as to further reduce the amount of computation of the processor 40 and the amount of data transmission between devices.

[0176] In the lawnmower robot described in the above embodiments, the processor does not directly perform stereo matching on the first and second environmental images. Instead, it first performs object recognition and association of repeated objects on the first and second environmental images, and then performs stereo matching only on the repeated objects in the first and second environmental images. Subsequently, the depth information obtained from the stereo matching of repeated objects is used to control the lawnmower robot, which can achieve the beneficial effect of reducing the amount of data computation by the processor and the amount of data transmission between devices.

[0177] In one embodiment, the lawnmower robot 100 includes at least a drive component 10, a walking component 20, a camera component 30, and a processor 40. The camera component 30 acquires environmental images of the lawnmower robot 100. These environmental images can be two-dimensional or three-dimensional, and are specifically either RGB or grayscale images.

[0178] The processor 40 is electrically or communicatively connected to the camera assembly 30, enabling it to acquire environmental images captured by the camera assembly 30. After obtaining the environmental image, the processor 40 can cluster the pixels or voxels in the environmental image to obtain pixel sets or voxel sets corresponding to each object in the environmental image. Specifically, a two-dimensional environmental image consists of multiple pixels, while a three-dimensional environmental image consists of multiple voxels. Clustering algorithms are used to cluster pixels based on RGB values ​​or grayscale values, or based on voxel coordinates, to obtain multiple pixel sets from a two-dimensional environmental image or multiple voxel sets from a three-dimensional environmental image. Pixels or voxels within the same set are considered as one class; for example, pixels or voxels within the same set are pixels or voxels that constitute the same object.

[0179] The processor 40 determines whether to remove a pixel set or voxel set based on its size. If the size of the pixel set or voxel set is less than or equal to a first threshold, the pixel set or voxel set is removed. If the size of the pixel set or voxel set is greater than or equal to a second threshold, the processor 40 identifies the type of the object corresponding to the pixel set or voxel set, where the first threshold is less than or equal to the second threshold. After removal and identification, the processor 40 subsequently controls the walking motor 22 based on the type of the object corresponding to the undisturbed pixel sets or voxel sets. In some examples, the processor 40 can also control the drive motor 12 based on the type of the object corresponding to the undisturbed pixel sets or voxel sets.

[0180] In some embodiments, the processor 40 uses K-Means or region growing algorithms to cluster pixels in a two-dimensional environment image or voxels in a three-dimensional environment image. It is understood that other clustering algorithms (not shown) can also be used to cluster pixels or voxels to suit specific scenario requirements.

[0181] In a two-dimensional environment image, the set of pixels can represent an image region within that image, while in a three-dimensional environment image, the set of voxels can represent an image space within that image. In some embodiments, the size of the pixel set can be the length or width of the corresponding image region, and the size of the voxel set can be the length, width, or height of the corresponding image space. In other embodiments, the size of the pixel set can be the length of the diagonal of the corresponding image region, and the size of the voxel set can be the length of the three-dimensional diagonal of the corresponding image space. In still other embodiments, the size of the pixel set can be the maximum distance between two pixels within the corresponding image region, and the size of the voxel set can be the maximum distance between two voxels within the corresponding image space.

[0182] In some embodiments, the first threshold used for comparison with the size of the pixel set or voxel set can be 5 cm. In other embodiments, the first threshold used for comparison with the size of the pixel set or voxel set can also be 3 cm.

[0183] The information that a set of pixels in a two-dimensional environmental image can represent is limited. In some embodiments, the processor 40 can also convert each set of pixels into a corresponding set of voxels based on the camera parameters of the camera component 30. Specifically, the camera parameters of the camera component 30 can include camera intrinsic and extrinsic parameters, which can realize the mapping and transformation between the two-dimensional pixel coordinate system and the three-dimensional voxel coordinate system. In some examples, the first threshold used for comparison with the converted voxel set can be 3 cm.

[0184] In some embodiments, the camera assembly 30 includes at least a first camera and a second camera. The processor 40 can perform stereo matching on the two-dimensional environmental images captured by the two cameras at the same time to obtain the corresponding three-dimensional environmental image. Subsequently, the three-dimensional environmental image can be subjected to voxel clustering and voxel set removal and recognition.

[0185] In addition to using size as the criterion for eliminating pixel sets or voxel sets, other set parameters can also be used as the criteria for eliminating pixel sets or voxel sets. In some embodiments, after the processor 40 obtains pixel sets or voxel sets through clustering, it can also determine whether to eliminate the pixel set or voxel set based on the number of pixels in each pixel set or the number of voxels in each voxel set. If the number of pixels in a pixel set is less than or equal to a third threshold or the number of voxels in a voxel set is less than or equal to a fifth threshold, the pixel set or voxel set is eliminated. If the number of pixels in a pixel set is greater than or equal to a fourth threshold or the number of voxels in a voxel set is greater than or equal to a sixth threshold, the type of the object corresponding to the pixel set or voxel set is identified, where the third threshold is less than or equal to the fourth threshold and the fifth threshold is less than or equal to the sixth threshold. After elimination and identification are completed, the processor 40 subsequently controls the walking motor 22 based on the type of the object corresponding to each pixel set or voxel set that has not been eliminated. In some examples, the drive motor 12 can also be controlled based on the type of the object corresponding to the pixel set or voxel set that has not been eliminated.

[0186] In some embodiments, the fifth threshold used for comparison with the number of voxel points in the voxel set can be 15.

[0187] In the lawnmower robot 100 described in the above embodiments, the processor 40 does not directly perform object recognition on the environmental image. Instead, it first clusters the pixels or voxels in the environmental image and removes the pixel sets or voxel sets with a size smaller than a threshold. It only performs type recognition on the objects corresponding to the pixel sets or voxel sets with a size larger than the threshold. This can achieve the beneficial effect of reducing the amount of data computation of the processor 40 and the amount of data transmission between devices.

[0188] In one embodiment, the lawnmower robot 100 includes at least a drive component 10, a walking component 20, a sensor component 30, a first processor 41, and a second processor 42. The first processor 41 and the second processor 42 are configured to perform different processing tasks. Referring to FIG9, the first processor 41 is electrically or communicatively connected to the sensor component 30, and is able to receive environmental information about the lawnmower robot 100 collected by the sensor component 30, and can interact with the second processor 42. The second processor 42 can interact with the first processor 41 and control the movement of the lawnmower robot 100. In some examples, the first processor 41 and the second processor 42 can be physically different processors, for example, MCU1 and MCU2 respectively, or they can be virtually different processors, for example, VCPU1 and VCPU2 respectively.

[0189] After obtaining environmental information, the first processor 41 can identify and obtain the coordinate set of obstacles around the lawnmower robot 100 based on the environmental information. Then, it determines the representative coordinates of the obstacle based on the coordinate set and transmits the representative coordinates to the second processor 42. Specifically, the first processor 41 can identify obstacles around the lawnmower robot 100 based on environmental information such as environmental images collected by a visual sensor and laser point clouds collected by a lidar, and obtain the image region of the obstacle in a two-dimensional image, the image space in a three-dimensional image, or the point cloud space in a three-dimensional point cloud. Based on these image regions, image spaces, or point cloud spaces, the coordinate set of each obstacle can be determined, and representative coordinates to represent the position of the obstacle can be further selected from the coordinate set. Then, the representative coordinates of each obstacle are transmitted to the second processor 42.

[0190] After receiving the representative coordinates of each obstacle from the first processor 41, the second processor 42 can control the walking motor 22 based on the representative coordinates of each obstacle. In some examples, the drive motor 12 can also be controlled based on the representative coordinates.

[0191] In some embodiments, the sensor assembly 30 includes a camera assembly that can capture environmental images of the lawnmower robot 100 and transmit the environmental images to the first processor 41.

[0192] In some embodiments, sensor assembly 30 includes a binocular camera and / or lidar (LiDAR).

[0193] In some embodiments, when the first processor 41 identifies obstacles around the lawnmower robot 100 based on a two-dimensional environmental image, it obtains the bounding boxes of each obstacle in the two-dimensional environmental image. The coordinates of the two vertices on the diagonal of the bounding box can be used as the representative coordinates of the obstacle. In some examples, the first processor 41 can also convert the vertex coordinates of the diagonal to the vertex coordinates of the three-dimensional diagonal based on the camera intrinsic and extrinsic parameters of the camera assembly 30 and transmit them to the second processor 42.

[0194] In other embodiments, when the first processor 41 identifies obstacles around the lawnmower robot 100 based on a three-dimensional environmental image or a three-dimensional point cloud, it obtains the bounding box of each obstacle in the three-dimensional environmental image or a three-dimensional point cloud, and the coordinates of the two vertices on the three-dimensional diagonal of the bounding box can be used as the representative coordinates of the obstacle.

[0195] In some embodiments, when the first processor 41 identifies obstacles based on environmental images, it can obtain the type of each obstacle and transmit the type of each obstacle along with the representative coordinates of each obstacle to the second processor 42.

[0196] In some embodiments, the first processor 41 can be a GPU (Graphics Processing Unit), and the second processor 42 can be a CPU. It is understood that in some cases, the lawnmower robot actually contains two system-on-chips (SoCs), one of which has both a CPU and a GPU, and the other has only a CPU. The GPU-containing chip performs image processing and other computational tasks, and the processed data is exchanged between the two chips.

[0197] In some embodiments, the first processor 41 may be an NPU (Neural-Network Processing Unit), and the second processor 42 may be a CPU.

[0198] In some embodiments, the amount of data representing the coordinates of the obstacle that the first processor 41 and the second processor 42 interact with is less than or equal to 100 bytes.

[0199] In the lawnmower robot described in the above embodiments, when the first processor interacts with the second processor to exchange obstacle position information, it does not directly use the complete set of obstacle coordinates, but uses its representative coordinates. This can effectively reduce the amount of interactive data between the first processor and the second processor, and alleviate the processor's data transmission and processing burden.

[0200] It should be noted that the above embodiments are not isolated from each other. New embodiments obtained by combining all or part of the contents of multiple embodiments according to actual scenario requirements should also be within the protection scope of this application.

[0201] For example, the processor 40 can identify the type of each object in the two-dimensional environmental image captured by the camera component 30. After identifying the object belonging to the bad road condition type, it uses three-dimensional point cloud to verify the bad road condition. If the verification is successful, it calls the local map to perform path planning and obstacle avoidance, so as to improve the accuracy of object recognition, path planning and obstacle avoidance of the lawnmower robot.

[0202] For example, the processor 40 can update the real-time obstacle, travel speed, mileage data and other map information in the local map corresponding to the real-time position of the lawnmower 100 based on the various environmental information collected by the sensor component 30, and update the road conditions, road condition probabilities and cost values ​​on each map grid in the global map corresponding to the real-time position of the lawnmower 100, so as to improve the map generation capability of the lawnmower and the performance of the generated map.

[0203] For example, the first processor 41 can cluster the pixels in the environmental images captured by the first and second cameras in the camera assembly 30 to obtain pixel sets corresponding to each object in the environmental images, and remove and identify pixels based on the size of the pixel sets; then, the first processor 41 can associate duplicate objects in the first and second environmental images after removing pixel sets with sizes smaller than a threshold, and perform stereo matching on the associated duplicate objects to obtain the depth information of the duplicate objects; finally, the first processor 41 can determine the representative coordinates of the obstacles based on the depth information of the duplicate objects belonging to the obstacles, and transmit the representative coordinates of each obstacle to the second processor 42; the second processor 42 controls the walking motor 22 based on the representative coordinates of the obstacles to reduce the amount of data computation of the processor and the amount of data transmission between devices.

[0204] It is understood that the embodiments described above can also be applied independently or in combination to outdoor work vehicles. The outdoor work vehicle in this application can be a device capable of autonomously planning and deciding its route, such as the lawnmower robot described above; for example, it can also be a snowplow robot. The outdoor work vehicle can also be other devices with autonomous walking as a reference or alternative function, such as ride-on lawnmowers, stand-up lawnmowers, and other manned lawnmowers, as well as washing machines, multi-functional all-terrain vehicles (UTVs / farm vehicles), all-terrain vehicles (ATVs / ATVs), golf carts, etc.

[0205] The foregoing has shown and described the basic principles, main features, and advantages of this application. Those skilled in the art should understand that the above embodiments do not limit this application in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of this application.

Claims

1. A lawnmower robot, comprising: A drive assembly, including a blade and a drive motor that drives the blade; The walking assembly includes walking wheels and a walking motor that drives the walking wheels; The camera assembly is configured to capture two-dimensional images of the area surrounding the lawnmower robot; A processor, communicatively or electrically connected to the camera assembly, is configured to control the actions of the lawnmower robot based at least on the two-dimensional images; The processor is further configured to: identify adverse road conditions in the two-dimensional image through semantic segmentation, wherein the adverse road conditions belong to one of a preset plurality of adverse road condition types; verify the adverse road conditions based on the three-dimensional point cloud to obtain a verification result; and plan the travel route of the lawnmower robot based on the verification result.

2. The lawnmower robot according to claim 1, wherein, The processor is also configured to avoid or slow down the verified adverse road conditions.

3. The lawnmower robot according to claim 1, wherein, The preset multiple adverse road condition types include at least two of the following: potholes, water, or haystacks.

4. The lawnmower robot according to claim 1, wherein, The camera assembly includes at least a first camera and a second camera, and the processor is further configured to generate the three-dimensional point cloud based on at least two of the two-dimensional images.

5. The lawnmower robot according to claim 1, wherein, The lawnmower also includes a sensor assembly, which is communicatively or electrically connected to the processor and configured to collect environmental information around the lawnmower to generate the three-dimensional point cloud and transmit the three-dimensional point cloud to the processor.

6. The lawnmower robot according to claim 1, wherein, The processor determines the depth information of the poor road conditions and the depth information of the walking plane of the lawnmower operator based on the three-dimensional point cloud, and uses the depth information of the poor road conditions and the depth information of the walking plane to verify the poor road conditions.

7. The lawnmower robot according to claim 6, wherein, The processor determines the depth information of the poor road conditions based on the coordinate information of the poor road conditions in the two-dimensional image and the registration relationship between the two-dimensional image and the three-dimensional point cloud.

8. The lawnmower robot according to claim 7, wherein, The processor obtains the bounding box of the poor road condition through semantic segmentation, and the coordinate information of the poor road condition is the coordinate information of the region within the bounding box in the two-dimensional image.

9. The lawnmower robot according to claim 6, wherein, The processor determines the depth information of the walking plane based on the coordinate information of the walking plane within a certain distance of the location of the poor road condition in the two-dimensional image and the registration relationship between the two-dimensional image and the three-dimensional point cloud.

10. The lawnmower robot according to claim 9, wherein, The processor obtains the bounding box of the poor road conditions through semantic segmentation, and the coordinate information of the walking plane is the coordinate information of the region outside the bounding box in the two-dimensional image.

11. The lawnmower robot according to claim 6, wherein, When the depth information of the walking plane does not conform to the preset plane depth information, the processor determines that the poor road condition verification has failed.

12. The lawnmower robot according to claim 6, wherein, When the processor determines that the poor road condition verification is successful, the depth relationship between the depth information of the poor road condition and the depth information of the walking plane conforms to a preset depth relationship corresponding to the type of the poor road condition.

13. A lawnmower robot, comprising: A drive assembly, including a blade and a drive motor that drives the blade; The walking assembly includes walking wheels and a walking motor that drives the walking wheels; Sensor components are configured to collect environmental information around the lawnmower robot; The memory is configured to store a global map of the working area of ​​the lawnmower robot; The processor, which is communicatively or electrically connected to the sensor assembly and the memory, is configured to control the actions of the lawnmower robot based at least on the environmental information acquired by the sensors. The processor is further configured to: construct a local map of the real-time location of the lawnmower robot based on the environmental information collected by the sensor components, wherein the local map has a higher resolution than the global map.

14. The lawnmower robot according to claim 13, wherein, The local map contains more types of map attributes than the global map.

15. The lawnmower robot according to claim 13, wherein, The resolution of the local map is greater than or equal to 1 cm and less than or equal to 20 cm.

16. The lawnmower robot according to claim 13, wherein, The processor maps map information from the global map to the local map based on the real-time location of the lawnmower robot.

17. The lawnmower robot according to claim 13, wherein, The shape of the local map is fixed, and the distance between the boundary of the local map and the real-time position of the lawnmower robot is less than or equal to 5m.

18. The lawnmower robot according to claim 13, wherein, The processor is also configured to: plan the travel route of the lawnmower robot based on the global map and the local map, and control the walking motor based on the travel route.

19. The lawnmower robot according to claim 13, wherein, The local map is composed of multiple map grids, each with a different resolution.

20. The lawnmower robot according to claim 19, wherein, The map grid with the smaller distance to the real-time position of the lawnmower robot among the plurality of map grids has a higher resolution.

21. The lawnmower robot according to claim 13, wherein, The sensor assembly includes at least one of a vision sensor, a collision sensor, an inertial sensor, a lidar, and an odometer.

22. The lawnmower robot according to claim 13, wherein, The processor updates the local map in real time.

23. A lawnmower robot, comprising: A drive assembly, including a blade and a drive motor that drives the blade; The walking assembly includes walking wheels and a walking motor that drives the walking wheels; Sensor components are configured to collect environmental information around the lawnmower robot; The memory is configured to store a global map of the working area of ​​the lawnmower robot; The processor, which is communicatively or electrically connected to the sensor assembly and the memory, is configured to control the actions of the lawnmower robot based at least on the environmental information acquired by the sensors. The processor is further configured to: map the map information in the global map to an initial local map around the real-time location based on the real-time location of the lawnmower robot; update the initial local map using environmental information collected by the sensor components to generate a local map around the real-time location; and control the walking motor based on the global map and the local map.

24. A lawnmower robot, comprising: A drive assembly, including a blade and a drive motor that drives the blade; The walking assembly includes walking wheels and a walking motor that drives the walking wheels; Sensor components, including at least one of a vision sensor, a collision sensor, an inertial sensor, an odometer, and a lidar; The memory is configured to store a map of the working area of ​​the lawnmower robot, the map being divided into multiple map grids; The processor, which is communicatively or electrically connected to the sensor assembly and the memory, is configured to control the actions of the lawnmower robot based at least on environmental information collected by the sensor assembly. The processor is further configured to: determine road conditions based on environmental information collected by the sensor components, wherein the road conditions belong to at least one of a preset plurality of road condition types; record the road conditions on the corresponding map grid in the map; the processor also includes a road condition update module, wherein the road conditions... The update module is configured to update the road conditions on the corresponding map grid in the map based on the newly acquired environmental information from the sensor components.

25. The lawnmower robot according to claim 24, wherein, The lawnmower also includes a human-computer interaction component, which is configured to allow users to update road conditions on one or more map grids in the map using human-computer interaction methods.

26. The lawnmower robot according to claim 24, wherein, The preset multiple road condition types include at least two of the following: bumpy area, pothole area, slope area, non-grass area, obstacle-prone area, slippery area, and drowsy area.

27. The lawnmower robot according to claim 24, wherein, The processor is further configured to record the road condition probability corresponding to the road condition on the corresponding map grid in the map; the road condition update module is further configured to update the road condition probability on the corresponding map grid in the map based on the environmental information newly collected by the sensor components.

28. The lawnmower robot according to claim 27, wherein, The processor is further configured to filter the environmental information collected by the sensor component based on the temporal and spatial attributes of the environmental information collected by the sensor component; the road condition update module updates the road condition and the road condition probability based on the filtered environmental information collected by the sensor component.

29. The lawnmower robot according to claim 27, wherein, The processor is also configured to determine the cost value of each map grid based on the road conditions and road condition probabilities on each map grid, and to plan the walking path of the lawnmower robot based on the cost value of each map grid.

30. The lawnmower robot according to claim 29, wherein, The memory is also configured to store at least one second map attribute on each grid cell of the map; the processor is also configured to adjust the walking path according to the second map attribute.

31. The lawnmower robot according to claim 30, wherein, The second map attribute includes at least one of mowing, lift, collision, boundary, obstacle, and charging station.

32. The lawnmower robot according to claim 29 or 30, wherein, The processor controls the walking motor to drive the walking wheels at different speeds based on the cost value and / or the second map attribute.

33. A lawnmower robot, comprising: A drive assembly, including a blade and a drive motor that drives the blade; The walking assembly includes walking wheels and a walking motor that drives the walking wheels; The camera assembly includes a first camera and a second camera. The first camera is configured to capture a first environmental image around the lawnmower robot, and the second camera is configured to capture a second environmental image around the lawnmower robot. Both the first and second environmental images are two-dimensional images. picture; The processor, communicatively or electrically connected to the camera assembly, is configured to control the actions of the lawnmower robot based at least on the first environmental image and the second environmental image; The processor is further configured to: identify each object contained in the first environment image and the second environment image through semantic segmentation; associate duplicate objects in the first environment image and the second environment image; and perform stereo matching on the duplicate objects in the first environment image and the second environment image to obtain the depth information of the duplicate objects; The walking motor is controlled based on the identification result of the repeating object and the depth information.

34. The lawnmower robot according to claim 33, wherein, The first camera is positioned to the left of the second camera.

35. The lawnmower robot according to claim 33, wherein, The first camera and the second camera together form a binocular camera.

36. The lawnmower robot according to claim 33, wherein, The first camera or the second camera is a fisheye camera.

37. The lawnmower robot according to claim 33, wherein, The camera assembly also includes a structured light camera and / or a ToF (Time of Flight) camera.

38. A lawnmower robot, comprising: A drive assembly, including a blade and a drive motor that drives the blade; The walking assembly includes walking wheels and a walking motor that drives the walking wheels; Sensor components are configured to collect environmental information around the lawnmower robot; A processor, communicatively or electrically connected to the sensor assembly, is configured to control the actions of the lawnmower robot based at least on the environmental information acquired by the sensor assembly. The processor is further configured to: identify obstacles based on the environmental information collected by the sensor components, and determine whether the obstacles are static or dynamic; when the obstacles are determined to be dynamic, use a first map for navigation; when the obstacles are determined to be static, use a second map for navigation.

39. The lawnmower robot according to claim 38, wherein, The second map is a local map, which is a map within a certain distance of the real-time position of the lawnmower robot.

40. The lawnmower robot according to claim 38, wherein, The second map is updated in real time as the lawnmower moves.

41. The lawnmower robot according to claim 38, wherein, The first map is a global map, which is the overall map of the working area of ​​the lawnmower robot.

42. The lawnmower robot according to claim 38, wherein, The second map has a higher resolution than the first map.

43. The lawnmower robot according to claim 38, wherein, The processor is also configured to control the walking motor to drive the walking wheels at a first speed when navigating using the first map; When using the second map for navigation, the walking motor is controlled to drive the walking wheels at a second speed.

44. The lawnmower robot according to claim 38, wherein, The sensor assembly includes a camera assembly configured to acquire images of the environment surrounding the lawnmower robot.

45. The lawnmower robot according to claim 44, wherein, The processor uses a semantic segmentation algorithm to identify the obstacles and their types appearing in the environmental image. When the obstacle is identified as a person or an animal, it is determined to be dynamic.

46. ​​The lawnmower robot according to claim 44, wherein, The processor uses optical flow or object tracking methods to detect whether objects in the environmental image are dynamic.

47. The lawnmower robot according to claim 38, wherein, The sensor assembly includes one or more of a binocular camera, a lidar (LiDAR) sensor, and an infrared sensor.

48. A lawnmower robot, comprising: A drive assembly, including a blade and a drive motor that drives the blade; The walking assembly includes walking wheels and a walking motor that drives the walking wheels; A camera assembly is configured to acquire environmental images of the area surrounding the lawnmower robot, the environmental images being two-dimensional or three-dimensional images; A processor, communicatively or electrically connected to the camera assembly, is configured to control the actions of the lawnmower robot based at least on the environmental image. The processor is further configured to: cluster the pixels or voxels in the environmental image to obtain a set of pixels or a set of voxels corresponding to each object contained in the environmental image; when the size of the set of pixels or the set of voxels is less than or equal to a first threshold, discard the set of pixels or the set of voxels; when the size of the set of pixels or the set of voxels is greater than or equal to a second threshold, identify the type of the object corresponding to the set of pixels or the set of voxels, and control the walking motor according to the type of the object.

49. The lawnmower robot according to claim 48, wherein, The processor uses K-Means or region growing algorithms to cluster pixels or voxels in the environmental image.

50. The lawnmower robot according to claim 48, wherein, The camera assembly includes at least two cameras, and the processor performs stereo matching on two two-dimensional environmental images captured by the two cameras at the same time to generate a corresponding three-dimensional environmental image.

51. The lawnmower robot according to claim 48, wherein, The processor converts the pixel set into the corresponding voxel set according to the camera parameters of the camera component.

52. The lawnmower robot according to claim 48, wherein, The dimension is the length of the planar diagonal of the bounding box of the pixel set, or the length of the three-dimensional diagonal of the bounding box of the voxel set.

53. The lawnmower robot according to claim 48, wherein, The dimension is one of the length, width, and height of the bounding box of the pixel set or the voxel set.

54. The lawnmower robot according to claim 48, wherein, The size is the maximum value of the distance between two pixels or between two voxels in the pixel set or voxel set.

55. The lawnmower robot according to claim 48, wherein, The first threshold is less than or equal to 5cm.

56. A lawnmower robot, comprising: A drive assembly, including a blade and a drive motor that drives the blade; The walking assembly includes walking wheels and a walking motor that drives the walking wheels; A camera assembly is configured to acquire environmental images of the area surrounding the lawnmower robot, the environmental images being either two-dimensional or three-dimensional images; A processor, communicatively or electrically connected to the camera assembly, is configured to control the actions of the lawnmower robot based at least on the environmental image. The processor is further configured to: cluster the pixels or voxels in the environmental image to obtain a pixel set or voxel set corresponding to each object contained in the environmental image; when the set parameter of the pixel set or voxel set is less than or equal to a parameter threshold, discard the pixel set or voxel set; when the set parameter of the pixel set or voxel set is greater than or equal to the parameter threshold, identify the type of the object corresponding to the pixel set or voxel set, and control the walking motor according to the type of the object.

57. The lawnmower robot according to claim 56, wherein, The set parameter is the number of pixels or the number of voxels.

58. The lawnmower robot according to claim 57, wherein, The parameter threshold corresponding to the number of voxel points is 15.

59. A lawnmower robot, comprising: A drive assembly, including a blade and a drive motor that drives the blade; The walking assembly includes walking wheels and a walking motor that drives the walking wheels; Sensor components are configured to collect environmental information around the lawnmower robot; A first processor is configured to identify obstacles based on the environmental information and determine a set of coordinates for the obstacles; A second processor is configured to control the actions of the lawnmower robot; The first processor is further configured to: determine representative coordinates of the obstacle based on the set of coordinates of the obstacle, and transmit the representative coordinates to the second processor; the second processor is further configured to: control the walking motor based on the representative coordinates of the obstacle.

60. The lawnmower robot according to claim 59, wherein, The sensor assembly includes a camera assembly configured to acquire images of the environment surrounding the lawnmower robot.

61. The lawnmower robot according to claim 60, wherein, The first processor is a GPU, and the second processor is a CPU.

62. The lawnmower robot according to claim 61, wherein, The first processor is also configured to identify the type of obstacle in the environmental image and transmit the type of obstacle and the representative coordinates of the obstacle to the second processor.

63. The lawnmower robot according to claim 59, wherein, The representative coordinates are the coordinates of the vertices of the three-dimensional diagonal of the bounding box of the obstacle.

64. The lawnmower robot according to claim 59, wherein, The representative coordinates are the coordinates of the vertices of the plane diagonal of the bounding box of the obstacle.

65. The lawnmower robot according to claim 59, wherein, The sensor components include a binocular camera and / or a lidar (LiDAR) system.

66. The lawnmower robot according to claim 59, wherein, The data representing the coordinates is less than or equal to 100 bytes.

Images