Robot lawn mower

Abstract

Disclosed in the present application is a robot lawn mower. The robot lawn mower comprises: a drive component, comprising a blade and a drive motor for driving the blade; a walking component, comprising walking wheels and a walking motor for driving the walking wheels; a camera component, configured to collect two-dimensional images of the surroundings of the robot lawn mower; and a processor, which is in communication connection or electrical connection with the camera component and configured to at least control, according to the two-dimensional images, actions of the robot lawn mower. The processor is further configured to: identify an adverse road condition in the two-dimensional images by means of semantic segmentation, the adverse road condition belonging to one of multiple preset adverse road condition types; verify the adverse road condition according to a three-dimensional point cloud, so as to obtain a verification result; and, according to the verification result, plan a travel route for the robot lawn mower.

Description

lawn mowing robot

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on July 24, 2023, with application number 202310908086.7. The entire contents of the above application are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of garden tools, and in particular to a lawn mowing robot. Background Art

[0003] A lawn mower is a common gardening tool used for mowing lawns, specifically within a work area defined by a boundary line. Thanks to technological advancements, lawn mowers have evolved into robotic lawn mowers to meet growing user needs and provide more intelligent and diverse services.

[0004] The lawn mower robot has multiple functions such as path planning, object recognition, and obstacle avoidance, which can avoid problems such as collisions and repetitive work. Compared with traditional lawn mowers, it can significantly improve work efficiency and reduce energy consumption. Whether in terms of cost consumption or usage experience, it is more user-friendly.

[0005] This section provides background information related to the present application which is not necessarily prior art.

[0006] Summary of the Invention

[0007] One object of the present application is to solve or at least alleviate a part or all of the above problems. To this end, one object of the present application is to provide a lawn mowing robot.

[0008] In order to achieve the above objectives, this application adopts the following technical solutions:

[0009] A lawn mowing robot comprises: a drive assembly including a blade and a drive motor for driving the blade; a travel assembly including travel wheels and a travel motor for driving the travel wheels; a sensor assembly configured to collect environmental information surrounding the lawn mowing robot; a processor communicatively or electrically connected to the sensor assembly and configured to control the actions of the lawn mowing robot based at least on the environmental information collected by the sensor assembly; wherein the processor is further configured to: identify obstacles based on the environmental information collected by the sensor assembly and determine whether the obstacles are static or dynamic; when the obstacle is determined to be dynamic, use a first map for navigation; and when the obstacle is determined to be static, use a second map for navigation.

[0010] In some embodiments, the second map is a local map, and the local map is a map within a certain distance around the real-time position of the robotic mower.

[0011] In some embodiments, the second map is updated in real time during the movement of the robotic lawn mower.

[0012] In some embodiments, the first map is a global map, which is an overall map of the working area of ​​the robotic lawn mower.

[0013] In some embodiments, the second map has a higher resolution than the first map.

[0014] In some embodiments, the processor is further configured to control the travel motor to drive the travel wheels at a first speed when navigating using the first map; and to control the travel motor to drive the travel wheels at a second speed when navigating using the second map.

[0015] In some embodiments, the sensor assembly includes a camera assembly configured to capture images of the environment surrounding the robotic lawn mower.

[0016] In some embodiments, the processor uses a semantic segmentation algorithm to identify obstacles and their types appearing in the environment image, and when the obstacle is identified as a person or an animal, it is determined to be dynamic.

[0017] In some embodiments, the processor uses a light flow method or a target tracking method to detect whether an object in the environment image is dynamic.

[0018] In some embodiments, the sensor component includes one or more of a binocular camera, a laser radar (LiDAR), and an infrared sensor.

[0019] A lawn mowing robot comprises: a drive assembly including a blade and a drive motor for driving the blade; a travel assembly including travel wheels and a travel motor for driving the travel wheels; a sensor assembly configured to collect environmental information surrounding the lawn mowing robot; a memory configured to store a global map of the lawn mowing robot's working area; a processor communicatively connected or electrically connected to the sensor assembly and the memory, and configured to control the actions of the lawn mowing robot based at least on the environmental information collected by the sensor; wherein the processor is further configured to construct a local map around the real-time position of the lawn mowing robot based on the environmental information collected by the sensor assembly, the local map having a higher resolution than the global map.

[0020] In some embodiments, the local map contains more types of map attributes than the global map.

[0021] In some embodiments, the resolution of the local map is greater than or equal to 1 cm and less than or equal to 20 cm.

[0022] In some embodiments, the processor maps map information in the global map to the local map according to the real-time location of the lawn mowing robot.

[0023] In some embodiments, the shape of the local map is fixed, and the distance between the boundary of the local map and the real-time position of the mowing robot is less than or equal to 5 m.

[0024] In some embodiments, the processor is further configured to: plan a travel route of the lawn mowing robot according to the global map and the local map, and control the travel motor according to the travel route.

[0025] In some embodiments, the local map is composed of multiple map grids, and the resolutions of the multiple map grids are different.

[0026] In some embodiments, a map grid with a smaller distance from the real-time position of the robotic lawn mower among the plurality of map grids has a higher resolution.

[0027] In some embodiments, the sensor assembly includes at least one of a visual sensor, a collision sensor, an inertial sensor, a lidar, and an odometer.

[0028] In some embodiments, the processor updates the local map in real time.

[0029] A lawn mowing robot comprises: a drive assembly comprising a blade and a drive motor for driving the blade; a walking assembly comprising walking wheels and a walking motor for driving the walking wheels; a sensor assembly comprising at least one of a visual sensor, a collision sensor, an inertial sensor, an odometer, and a lidar; a memory configured to store a map of a working area of ​​the lawn mowing robot, wherein the map is divided into a plurality of map grids; a processor communicatively connected or electrically connected to the sensor assembly and the memory, and configured to control the actions of the lawn mowing robot based at least on environmental information collected by the sensor assembly; wherein the processor is further configured to: determine a road condition based on the environmental information collected by the sensor assembly, wherein the road condition belongs to at least one of a plurality of preset road condition types; and record the road condition on a corresponding map grid in a map; the processor further comprises a road condition update module, wherein the road condition update module is configured to update the road condition on a corresponding map grid in the map based on the newly collected environmental information by the sensor assembly.

[0030] In some embodiments, the lawn mowing robot further includes a human-computer interaction component, which is configured to allow a user to update road conditions on one or more map grids in the map using human-computer interaction means.

[0031] In some embodiments, the preset multiple road condition types include at least two of a bumpy area, a pit area, a slope area, a non-grass area, an obstacle-prone area, a slippery area, and an easily trapped area.

[0032] In some embodiments, the processor is further configured to record the traffic condition probability corresponding to the traffic condition on the corresponding map grid in the map; the traffic condition update module is further configured to update the traffic condition probability on the corresponding map grid in the map based on the newly collected environmental information by the sensor component.

[0033] In some embodiments, the processor is further configured to filter the environmental information collected by the sensor component according to the time attributes and spatial attributes of the environmental information collected by the sensor component; the road condition update module updates the road condition and road condition probability according to the filtered environmental information collected by the sensor component.

[0034] In some embodiments, the processor is further configured to determine a cost value of each map grid according to the road condition and road condition probability on each map grid, and plan a walking path of the lawn mowing robot according to the cost value of each map grid.

[0035] In some embodiments, the memory is further configured to store at least one second map attribute on each grid in the map; and the processor is further configured to adjust the walking path according to the second map attribute.

[0036] In some embodiments, the second map attribute includes at least one of mowing, elevation, collision, boundary, obstacle, and charging station.

[0037] In some embodiments, the processor controls the travel motor to drive the travel wheels at different speeds according to the cost value and / or the second map attribute.

[0038] A lawn mowing robot comprises: a drive assembly including a blade and a drive motor for driving the blade; a walking assembly including walking wheels and a walking motor for driving the walking wheels; a sensor assembly configured to collect environmental information surrounding the lawn mowing robot; a memory configured to store a global map of the working area of ​​the lawn mowing robot; a processor communicatively connected or electrically connected to the sensor assembly and the memory, and configured to control the actions of the lawn mowing robot based at least on the environmental information collected by the sensor; wherein the processor is further configured to: map map information in the global map to an initial local map surrounding the real-time position based on the real-time position of the lawn mowing robot; update the initial local map using the environmental information collected by the sensor assembly to generate a local map surrounding the real-time position; and control the walking motor based on the global map and the local map.

[0039] A lawn mowing robot comprises: a drive assembly including a blade and a drive motor for driving the blade; a travel assembly including travel wheels and a travel motor for driving the travel wheels; a camera assembly configured to capture two-dimensional images of the lawn mowing robot's surroundings; a processor communicatively or electrically connected to the camera assembly and configured to control the lawn mowing robot's actions based on at least the two-dimensional images; wherein the processor is further configured to: identify adverse road conditions in the two-dimensional images through semantic segmentation, the adverse road conditions belonging to one of a plurality of preset adverse road condition types; verify the adverse road conditions based on a three-dimensional point cloud to obtain a verification result; and plan a travel route of the lawn mowing robot based on the verification result.

[0040] In some embodiments, the processor is further configured to avoid or slow down the vehicle in response to verified adverse road conditions.

[0041] In some embodiments, the plurality of preset adverse road condition types include at least two of potholes, water areas, or haystacks.

[0042] In some embodiments, the camera assembly includes at least a first camera and a second camera, and the processor is further configured to generate a three-dimensional point cloud based on at least two two-dimensional images.

[0043] In some embodiments, the lawn mowing robot further includes a sensor component, which is communicatively or electrically connected to the processor and configured to collect environmental information around the lawn mowing robot to generate a three-dimensional point cloud and transmit the three-dimensional point cloud to the processor.

[0044] In some embodiments, the processor determines depth information of the bad road condition and depth information of the walking plane of the lawn mower according to the three-dimensional point cloud, and uses the depth information of the bad road condition and the depth information of the walking plane to verify the bad road condition.

[0045] In some embodiments, the processor determines depth information of the bad road condition based on coordinate information of the bad road condition in the two-dimensional image and a registration relationship between the two-dimensional image and the three-dimensional point cloud.

[0046] In some embodiments, the processor obtains a bounding box of the bad road condition through semantic segmentation, and the coordinate information of the bad road condition is the coordinate information of the area within the bounding box in the two-dimensional image.

[0047] In some embodiments, the processor determines the depth information of the walking plane based on the coordinate information of the walking plane within a certain distance around the location of the bad road condition in the two-dimensional image and the registration relationship between the two-dimensional image and the three-dimensional point cloud.

[0048] In some embodiments, the processor obtains a bounding box of the bad road condition through semantic segmentation, and the coordinate information of the walking plane is the coordinate information of the area outside the bounding box in the two-dimensional image.

[0049] In some embodiments, the processor determines that the bad road condition verification has failed when the depth information of the walking plane does not meet the preset plane depth information.

[0050] In some embodiments, the processor determines that the bad road condition verification is successful when the depth relationship between the depth information of the bad road condition and the depth information of the walking plane meets a preset depth relationship corresponding to the type of the bad road condition.

[0051] A lawn mowing robot comprises: a driving assembly including a blade and a driving motor for driving the blade; a walking assembly including walking wheels and a walking motor for driving the walking wheels; a camera assembly including a first camera and a second camera, the first camera being configured to capture a first environmental image surrounding the lawn mowing robot, and the second camera being configured to capture a second environmental image surrounding a person mowing the lawn, the first environmental image and the second environmental image being two-dimensional images; a processor being communicatively connected or electrically connected to the camera assembly and configured to control the actions of the lawn mowing robot based at least on the first environmental image and the second environmental image; wherein the processor is further configured to: respectively identify each object contained in the first environmental image and the second environmental image through semantic segmentation; associate repeated objects in the first environmental image and the second environmental image; perform stereo matching on the repeated objects in the first environmental image and the second environmental image to obtain depth information of the repeated objects; and control the walking motor based on the recognition results and depth information of the repeated objects.

[0052] In some embodiments, the first camera is disposed on the left side of the second camera.

[0053] In some embodiments, the first camera and the second camera form a binocular camera.

[0054] In some embodiments, the first camera or the second camera is a fisheye camera.

[0055] In some embodiments, the camera assembly further includes a structured light camera and / or a ToF (Time of Flight) camera.

[0056] A lawn mowing robot comprises: a drive assembly comprising a blade and a drive motor for driving the blade; a travel assembly comprising travel wheels and a travel motor for driving the travel wheels; a camera assembly configured to capture an image of the environment surrounding the lawn mowing robot, wherein the environment image is a two-dimensional image or a three-dimensional image; a processor communicatively connected or electrically connected to the camera assembly and configured to control the actions of the lawn mowing robot at least based on the environment image; wherein the processor is further configured to: cluster pixel points or voxel points in the environment image to obtain pixel sets or voxel sets corresponding to each object contained in the environment image; when the size of the pixel set or voxel set is less than or equal to a first threshold, eliminate the pixel set or voxel set; when the size of the pixel set or voxel set is greater than or equal to a second threshold, identify the type of object corresponding to the pixel set or voxel set, and control the travel motor according to the type of the object.

[0057] In some embodiments, the processor clusters pixels or voxels in the environment image using a K-Means or region growing algorithm.

[0058] In some embodiments, the camera assembly includes at least two cameras, and the processor performs stereo matching on two two-dimensional environment images captured by the two cameras at the same time to generate a corresponding three-dimensional environment image.

[0059] In some embodiments, the processor converts the pixel set into a corresponding voxel set based on camera parameters of the camera assembly.

[0060] In some embodiments, the size is the length of a two-dimensional diagonal of a bounding box of a set of pixels, or the length of a three-dimensional diagonal of a bounding box of a set of voxels.

[0061] In some embodiments, the size is one of the length, width, and height of a bounding box of a set of pixels or a set of voxels.

[0062] In some embodiments, the size is the maximum value of the distance between two pixels or two voxels in a pixel set or a voxel set.

[0063] In some embodiments, the first threshold is less than or equal to 5 cm.

[0064] A lawn mowing robot comprises: a drive assembly comprising a blade and a drive motor for driving the blade; a travel assembly comprising travel wheels and a travel motor for driving the travel wheels; a camera assembly configured to capture an image of the environment surrounding the lawn mowing robot, wherein the environment image is a two-dimensional image or a three-dimensional image; a processor communicatively connected or electrically connected to the camera assembly and configured to control the actions of the lawn mowing robot at least based on the environment image; wherein the processor is further configured to: cluster pixel points or voxel points in the environment image to obtain pixel sets or voxel sets corresponding to various objects contained in the environment image; when a set parameter of a pixel set or voxel set is less than or equal to a parameter threshold, eliminate the pixel set or voxel set; when a set parameter of a pixel set or voxel set is greater than or equal to the parameter threshold, identify the type of object corresponding to the pixel set or voxel set, and control the travel motor according to the type of the object.

[0065] In some embodiments, the aggregation parameter is the number of pixels or the number of voxels.

[0066] In some embodiments, the parameter threshold corresponding to the number of voxel points is 15.

[0067] A lawn mowing robot comprises: a drive assembly including a blade and a drive motor for driving the blade; a travel assembly including travel wheels and a travel motor for driving the travel wheels; a sensor assembly configured to collect environmental information surrounding the lawn mowing robot; a first processor configured to identify obstacles based on the environmental information and determine a coordinate set of the obstacles; a second processor configured to control the actions of the lawn mowing robot; wherein the first processor is further configured to determine representative coordinates of the obstacle based on the coordinate set of the obstacle and transmit the representative coordinates to the second processor; and the second processor is further configured to control the travel motor based on the representative coordinates of the obstacle.

[0068] In some embodiments, the sensor assembly includes a camera assembly configured to capture images of the environment surrounding the robotic lawn mower.

[0069] In some embodiments, the first processor is a GPU and the second processor is a CPU.

[0070] In some embodiments, the first processor is further configured to identify the type of obstacles in the environment image, and transmit the type of obstacles and representative coordinates of the obstacles to the second processor.

[0071] In some embodiments, the representative coordinates are the coordinates of vertices of a three-dimensional diagonal line of a bounding box of the obstacle.

[0072] In some embodiments, the representative coordinates are the coordinates of vertices of a plane diagonal line of the obstacle's bounding box.

[0073] In some embodiments, the sensor assembly includes a binocular camera and / or a laser radar (LiDAR).

[0074] In some embodiments, the amount of data representing the coordinates is less than or equal to 100 bytes.

[0075] A control method for a lawn mower robot includes: a sensor component collecting environmental information surrounding the lawn mower robot; a processor identifying obstacles based on the environmental information and determining whether the obstacles are static or dynamic; when the obstacles are determined to be dynamic, the processor using a first map for navigation; and when the obstacles are determined to be static, the processor using a second map for navigation.

[0076] A control method for a lawn mower robot comprises: a sensor component collecting environmental information around the lawn mower robot; a processor constructing a local map around the real-time position of the lawn mower robot based on the environmental information, wherein the local map has a higher resolution than a global map, and the global map is stored in a memory.

[0077] A control method for a lawn mower robot comprises: a sensor component collecting environmental information surrounding the lawn mower robot; a processor determining a road condition based on the environmental information and recording the road condition on a corresponding map grid in a map, wherein the road condition belongs to at least one of a plurality of preset road condition types, the map being stored in a memory and divided into a plurality of map grids; and the processor further updating the road condition on the corresponding map grid in the map based on the newly collected environmental information by the sensor component.

[0078] A control method for a lawn mower robot, comprising: a sensor component collecting environmental information surrounding the lawn mower robot; a processor mapping map information in a global map to an initial local map surrounding the real-time location based on the real-time location of the lawn mower robot, wherein the global map is stored in a memory; the processor further uses the environmental information collected by the sensor component to update the initial local map to generate a local map surrounding the real-time location, and controls the travel motor based on the global map and the local map.

[0079] A control method for a lawn mower robot comprises: a camera component collecting a two-dimensional image of the lawn mower robot's surroundings; a processor identifying an adverse road condition in the two-dimensional image through semantic segmentation, wherein the adverse road condition belongs to one of a plurality of preset adverse road condition types; the processor further verifying the adverse road condition based on a three-dimensional point cloud to obtain a verification result, and planning a travel route for the lawn mower robot based on the verification result.

[0080] A control method for a lawn mowing robot includes: a first camera in a camera assembly captures a first environment image around the lawn mowing robot, and a second camera in the camera assembly captures a second environment image around the lawn mowing robot, wherein the first environment image and the second environment image are two-dimensional images; a processor identifies objects contained in the first environment image and the second environment image respectively through semantic segmentation, and associates repeated objects in the first environment image and the second environment image; the processor also performs stereo matching on the repeated objects in the first environment image and the second environment image to obtain depth information of the repeated objects, and controls the travel motor according to the recognition results of the repeated objects and the depth information.

[0081] A control method for a lawn mowing robot comprises: a camera component collecting an environmental image surrounding the lawn mowing robot, wherein the environmental image is a two-dimensional image or a three-dimensional image; a processor clustering pixels or voxel points in the environmental image to obtain pixel sets or voxel sets corresponding to respective objects contained in the environmental image; when a set parameter of the pixel set or the voxel set is less than or equal to a parameter threshold, the processor eliminating the pixel set or the voxel set; when the set parameter of the pixel set or the voxel set is greater than or equal to the parameter threshold, the processor identifying the type of object corresponding to the pixel set or the voxel set, and controlling the travel motor according to the type of the object.

[0082] A control method for a lawn mower robot comprises: a sensor component collecting environmental information surrounding the lawn mower robot; a first processor identifying obstacles based on the environmental information and determining a coordinate set of the obstacles; the first processor also determining representative coordinates of the obstacles based on the coordinate set of the obstacles and transmitting the representative coordinates to a second processor; and the second processor controlling the travel motor based on the representative coordinates of the obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] FIG. 1 is a perspective view of a lawn mowing robot as one embodiment.

[0084] FIG. 2 is a perspective view of a portion of the internal structure of the lawn mowing robot shown in FIG. 1 .

[0085] FIG3 is a schematic diagram of the electrical control principle of the sensor assembly and processor in the lawn mowing robot shown in FIG1 .

[0086] FIG. 4 is a schematic diagram of a working area of ​​a lawn mowing robot as an embodiment.

[0087] FIG. 5 is a schematic diagram of a first map of a lawn mowing robot as an embodiment.

[0088] FIG. 6 is a schematic diagram of a second map of the lawn mowing robot as an embodiment.

[0089] FIG. 7 is a schematic diagram of a local map of a lawn mowing robot according to an embodiment.

[0090] FIG8 is a schematic diagram of the electrical control principle of the sensor assembly, memory, and processor in the lawn mowing robot shown in FIG1 .

[0091] FIG9 is a schematic diagram of the electrical control principle of the sensor assembly, the first processor, and the second processor in the lawn mowing robot shown in FIG1 .

[0092] Caption: 100, lawn mowing robot; 10, drive component; 20, travel component; 30, sensor component / camera component; 40, processor; 50, memory; 11, blade; 12, drive motor; 21, travel wheel; 22, travel motor; 41, first processor; 42, second processor. DETAILED DESCRIPTION

[0093] Before any embodiments of the present application are explained in detail, it is to be understood that the application is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the foregoing drawings.

[0094] In this application, the terms "comprises," "includes," "has," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0095] In this application, the term "and / or" describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " in this application generally indicates that the related objects are in an "and / or" relationship.

[0096] In this application, the terms "connect," "combine," "couple," and "install" may refer to direct connection, combination, coupling, or installation, or indirect connection, combination, coupling, or installation. For example, a direct connection refers to two parts or components being connected together without an intermediary, and an indirect connection refers to two parts or components being connected to at least one intermediary, with the two parts or components being connected via the intermediary. Furthermore, "connect" and "couple" are not limited to physical or mechanical connections or couplings and may include electrical connections or couplings.

[0097] In this application, it will be understood by those skilled in the art that relative terms (e.g., "about," "approximately," "substantially," etc.) used in conjunction with quantities or conditions include the values ​​and have the meaning indicated by the context. For example, the relative terms include at least the degree of error associated with the measurement of a specific value, the tolerance caused by manufacturing, assembly, use, etc. associated with a specific value. Such terms should also be considered to disclose a range defined by the absolute values ​​of the two endpoints. Relative terms may refer to plus or minus a certain percentage (e.g., 1%, 5%, 10% or more) of the indicated value. Numerical values ​​that do not use relative terms should also be disclosed as specific values ​​with tolerances. In addition, "substantially" may refer to plus or minus a certain degree (e.g., 1 degree, 5 degrees, 10 degrees or more) on the basis of the indicated angle when expressing a relative angular position relationship (e.g., substantially parallel, substantially perpendicular).

[0098] In this application, it will be understood by those skilled in the art that the function performed by an assembly can be performed by one assembly, multiple assemblies, one part, or multiple parts. Similarly, the function performed by a part can also be performed by one part, one assembly, or a combination of multiple parts.

[0099] In the present application, the terms "upper", "lower", "left", "right", "front", "back" and other directional words are described based on the orientation and positional relationship shown in the accompanying drawings, and should not be understood as limiting the embodiments of the present application. In addition, in the context, it is also necessary to understand that when it is mentioned that an element is connected to another element "upper" or "lower", it can not only be directly connected to the other element "upper" or "lower", but also be indirectly connected to the other element "upper" or "lower" through an intermediate element. It should also be understood that directional words such as upper side, lower side, left side, right side, front side, back side, etc. not only represent the positive orientation, but can also be understood as the lateral orientation. For example, below can include directly below, lower left, lower right, lower front and lower back, etc.

[0100] In this application, the terms "controller," "processor," "central processing unit," "CPU," and "MCU" are used interchangeably. Where a unit "controller," "processor," "central processing unit," "CPU," or "MCU" is used to perform a particular function, unless otherwise specified, the function may be performed by a single unit or multiple units.

[0101] In this application, the terms "device", "module" or "unit" can be implemented in the form of hardware or software to achieve specific functions.

[0102] In this application, the terms "calculate", "judge", "control", "determine", "identify", etc. refer to the operations and processes of a computer system or similar electronic computing device (e.g., controller, processor, etc.).

[0103] As a trendy outdoor powered walking tool, robotic lawn mowers are gaining popularity due to their intelligent and efficient features. While traditional lawn mowers require manual pushing to navigate the desired area, robotic lawn mowers can autonomously plan and mow along the planned path. They can also perform object recognition and obstacle avoidance while moving.

[0104] The technical solution of this application is described in detail below with reference to the accompanying drawings and specific implementation methods.

[0105] Figure 1 shows a lawn mowing robot 100 as an embodiment of the present application. It is understood that the technical solution described in the present application can also be selectively applied to other outdoor walking power tools such as snow sweeping robots to meet different scene requirements.

[0106] Referring to Figures 1 and 2, the lawn mower robot 100 includes a drive assembly 10 and a travel assembly 20. The drive assembly 10 includes a blade 11 and a drive motor 12 that drives the blade. The motor shaft of the drive motor rotates, transmitting power directly or indirectly through a first transmission assembly to a first output shaft to drive the blade to perform mowing operations. The travel assembly 20 includes travel wheels 21 and a travel motor 22 that drives the travel wheels. The motor shaft of the travel motor rotates, transmitting power directly or indirectly through a second transmission assembly to a second output shaft to drive the travel wheels along a planned path. In some examples, the travel motor may be a hub motor, which integrates power, transmission, and braking-related devices within the wheel hub.

[0107] Referring to Figure 3 , the lawn mower robot 100 further includes a sensor assembly 30 and a processor 40. The sensor assembly 30 may include one or more of a visual sensor, a collision sensor, an inertial sensor, an electromagnetic sensor, a lidar, and an odometer. The specific number of each type of sensor included in the sensor assembly 30 is not limited. It is understood that the sensor assembly 30 may also include other types of sensors not listed above.

[0108] The sensor component 30 is configured to collect environmental information around the lawn mower robot 100. In some examples, the sensor component 30 can collect environmental information within a preset distance range centered on the current real-time location of the lawn mower robot 100. The environmental information can be presented in the form of a one-dimensional signal, a two-dimensional image, or a three-dimensional point cloud.

[0109] In some examples, the sensor assembly 30 may include a visual sensor such as a camera assembly, an infrared sensor, a linear light sensor, etc., which can collect an environmental image of the lawn mower robot 100 in its current real-time position in its current posture within the sensor's line of sight. In other examples, the sensor assembly 30 may include a laser radar (LiDAR), which can collect a laser point cloud around the lawn mower robot 100 within the sensor's radiation range. It should be noted that in some cases, the sensor assembly 30 such as a binocular camera can also transmit the collected two-dimensional image to the processor 40 first, and then the processor 40 generates a three-dimensional image from the two-dimensional image. In some other examples, the sensor assembly 30 may include an inertial sensor, which can detect data such as the three-axis acceleration of the lawn mower robot 100.

[0110] The processor 40 may be a CPU (Central Processing Unit) or an MCU (Microcontroller Unit), which controls the actions of the robotic lawn mower 100 by reading and running various executable instructions.

[0111] Referring to Figure 4 , it illustrates the working area of ​​the robotic mower 100. To restrict the robotic mower 100 to the working area where mowing operations are to be performed, its working area is typically defined by a boundary. In some examples, the boundary may be a physical boundary; in other examples, the boundary may be a virtual boundary implemented using software. Whether physical or virtual, the working area of ​​the robotic mower 100 is defined. The specific shape of the working area is not limited; for example, the working area may be rectangular or have other irregular shapes.

[0112] Since the lawn mower robot 100 generally performs mowing operations with the goal of covering the working area, in order to achieve effective path planning, the lawn mower robot 100 also needs to build a map corresponding to its working area. In some simple examples, the lawn mower robot 100 can always use the same pre-stored map for navigation, but the information provided by these maps is limited and not timely, which is not conducive to the path planning and obstacle avoidance of the lawn mower robot 100.

[0113] In one embodiment, the lawn mower robot 100 includes at least a drive component 10, a walking component 20, a sensor component 30, and a processor 40. The sensor component 30 collects environmental information near the real-time location of the lawn mower robot 100, and the processor 40 is electrically connected or communicatively connected to the sensor component 30 and is capable of obtaining the environmental information collected by the sensor component 30. In some examples, the processor 40 can periodically receive environmental information corresponding to the real-time location of the lawn mower robot 100 from the sensor component 30. For example, the processor 40 can periodically receive environmental information collected by the sensor component 30 with a period of 100 ms. In this case, the processor 40 can exchange environmental information with the sensor component 30 10 times within 1 second. The environmental information of the 10 interactions can all be environmental information when the lawn mower robot 100 is at the same location, or can be environmental information when the lawn mower robot 100 is at different locations.

[0114] After obtaining environmental information corresponding to the real-time location of the robotic mower 100, the processor 40 can identify obstacles near the real-time location of the robotic mower 100 based on the environmental information and determine whether the obstacles are dynamic or static. Dynamic obstacles can include people, animals, or other electronic devices traveling near the robotic mower 100, while static obstacles can include immovable objects such as stone steps, trees, and benches.

[0115] If the processor 40 determines that the obstacle around the robotic mower 100 is a dynamic obstacle, it may invoke the first map to guide the robotic mower 100. Specifically, the processor 40 may invoke the first map, plan a first path based on the first map, and control the travel motor 22 based on the first path to guide the robotic mower 100 along the first path. In some cases, the processor 40 may also control the drive motor 12 based on the first map to adjust the blade cutting state of the robotic mower 100 along the first path.

[0116] If the processor 40 determines that the obstacle around the robotic mower 100 is a static obstacle, it may invoke the second map to guide the robotic mower 100. Specifically, the processor 40 may invoke the second map, plan a second path based on the second map, and control the travel motor 22 based on the second path to guide the robotic mower 100 along the second path. In some cases, the processor 40 may also control the drive motor 12 based on the second map to adjust the cutting state of the robotic mower 100 blades along the second path.

[0117] In some embodiments, the processor 40 may also be provided with a map pointer, and the processor 40 may use the map pointer to call a corresponding map when it is necessary to switch maps.

[0118] As shown in Figure 4 , the working area of ​​the robotic lawn mower 100 may contain multiple objects of different types. These objects may be dynamic or static obstacles, background objects such as air or ground, or specific road conditions such as potholes or bumps. The processor 40 may use different algorithms to identify the type of objects within the working area, depending on the actual scenario, or may identify objects within the working area that belong to one or more specific types.

[0119] In some embodiments, the sensor assembly 30 includes a camera assembly that can capture images of the environment surrounding the robotic lawn mower 100. The processor 40 can use a semantic segmentation algorithm to identify obstacles in the environment image and determine the type of obstacle. When the obstacle is a person, animal, or other moving electronic device, it can be determined to be dynamic. In some examples, geometric analysis methods can also be combined to identify dynamic and static obstacles in the three-dimensional environment image. In other embodiments, the processor 40 can also use a light stream method or a multi-frame target tracking method to detect whether the environment image contains dynamic objects.

[0120] Referring to Figure 5, in some embodiments, the first map is a global map, representing the entire work area of ​​the robot mower 100. When dynamic obstacles such as people or animals appear around the robot mower 100, the area potentially affected by these obstacles is wider due to their large range of motion. In situations such as when the robot mower 100 is mowing in a U-shaped path, the processor 40 can utilize the global map to replan the global path within the global map.

[0121] 6 , in some embodiments, the second map is a local map, which is a map of a portion of the working area within a certain distance from the real-time location of the mowing robot 100. In some examples, the local map may be a map of the portion of the working area with the real-time location of the mowing robot 100 as the coordinate origin and the direction of travel of the mowing robot 100 as the coordinate axis. In other examples, the local map may also be a map of the portion of the working area centered on the real-time location of the mowing robot 100 and within a preset distance threshold from the center. When static obstacles such as trees and stone steps appear around the mowing robot 100, since the positions of the static obstacles are relatively fixed, the processor 40 may call upon the local map to replan the local travel path under the local map.

[0122] Furthermore, when using a global map or a local map for navigation, other attributes of dynamic or static obstacles may also be considered when planning a route. For example, when using a local map for navigation, the processor 40 may control the travel motor 22 to cause the robot mower 100 to perform a circular mowing task around an obstacle such as a tree or flower bed. Alternatively, when using a local map for navigation, the processor 40 may control the travel motor 22 to cause the robot mower 100 to perform a side mowing task along the stone steps or boundary line.

[0123] In some embodiments, the second map has a higher resolution than the first map. In some examples, the resolution of the local map can be greater than or equal to 1 cm and less than or equal to 20 cm, while the resolution of the global map can be greater than or equal to 15 cm and less than or equal to 50 cm. In other examples, the resolution of the local map can be greater than or equal to 1 cm and less than or equal to 15 cm, while the resolution of the global map can be greater than or equal to 15 cm and less than or equal to 30 cm. It will be understood that the resolution of the first map and the second map should be adapted to the size of the current working area of ​​the lawn mower robot 100.

[0124] In some embodiments, the second map contains more map attributes than the first map. In some examples, the environmental information surrounding the real-time location of the robotic mower 100, as collected by the sensor assembly 30, can be recorded entirely in the local map and partially in the global map. For example, triaxial acceleration data detected by an inertial sensor and mileage data collected by an odometer can be recorded in the local map but not in the global map.

[0125] In some embodiments, the second map is updated in real time during the movement of the robotic mower 100. For example, assuming the robotic mower 100 is moving at a speed of 0.5 m / s, the second map may be updated periodically with a period of 100 ms, i.e., the local map is updated every 5 cm the robotic mower 100 moves.

[0126] In some embodiments, the processor 40 controls the travel motor 22 to drive the travel wheels at a first speed when navigating using the first map, and controls the travel motor 22 to drive the travel wheels at a second speed when navigating using the second map. In some examples, the first speed can be lower than the second speed. For example, the travel speed of the robot mower 100 when navigating using the global map can be lower than the travel speed of the robot mower 100 when navigating using the local map to ensure the safety of the robot mower 100 when encountering dynamic obstacles. In other examples, the travel speed of the robot mower 100 can be further adjusted based on other map attributes.

[0127] In some embodiments, the sensor assembly 30 includes one or more of a binocular camera, a laser radar (LiDAR), and an infrared sensor.

[0128] In the lawn mowing robot described in the above embodiment, different maps are switched for navigation based on whether the obstacles appearing near the real-time position of the lawn mowing robot are dynamic or static. The maps used can adapt to the different characteristics of dynamic and static obstacles and the requirements of path planning and obstacle avoidance in scenarios where dynamic and static obstacles appear, thereby improving the work efficiency and results of the lawn mowing robot.

[0129] In one embodiment, the mowing robot 100 includes at least a driving component 10, a walking component 20, a sensor component 30, a memory 50, and a processor 40. The memory 50 stores a global map of the working area of ​​the mowing robot 100. In some examples, the memory 50 can be a ROM (Read-Only Memory), a RAM (Random Access Memory), or a Flash memory. In other examples, the memory 50 can also be a remote memory. The global map is a map of the entire working area where the mowing robot 100 is to perform mowing operations. It can record relatively stable information in the entire working area in a relatively concise manner.

[0130] 8 , processor 40 is electrically or communicatively connected to sensor assembly 30 and memory 50, and is capable of accessing the global map stored in memory 50 and various types of environmental information collected by sensor assembly 30. In some examples, processor 40 may periodically receive various types of environmental information collected by sensor assembly 30, wherein different sensors may transmit environmental information at the same or different periods.

[0131] After obtaining various environmental information, the processor 40 can construct a local map around the real-time location of the robotic mower 100 based on this information. The resolution of the local map is higher than that of the global map, allowing it to record more detailed information about the more volatile portion of the working area corresponding to the real-time location of the robotic mower 100. Specifically, the processor 40 can identify obstacles based on environmental images from the visual sensor and construct a local map containing information about their location and type. It can also construct a local map containing depth information about each object based on laser point clouds from a laser radar (LiDAR). It can also construct a local map containing the speed and direction of the robotic mower 100 based on three-axis acceleration data from an inertial sensor.

[0132] In some embodiments, the shape of the local map may be fixed, that is, the shape of the local map is a preset shape. In some examples, the shape of the local map may be a regular quadrilateral, and in other examples, the shape of the local map may also be a circle.

[0133] In some embodiments, the local map can be constructed with the real-time location of the mowing robot 100 as the coordinate origin and the direction of travel of the mowing robot 100 as the coordinate axis. In some examples, the local map can be a 4m*4m regular quadrilateral map with the real-time location of the mowing robot 100 as the origin, or a 3m*3m regular quadrilateral map, a 5m*5m regular quadrilateral map, or a 6m*6m regular quadrilateral map. The side length of the local map can be adaptively adjusted based on the size of the mowing robot's working area and the mowing robot's travel speed. For example, the larger the working area and the faster the mowing robot's travel speed, the larger the side length of the local map, and vice versa. In other embodiments, the local map can be a map of the working area centered at the real-time location of the mowing robot 100 and within a preset distance threshold from the center. For example, the boundary of the local map can be a circle with the real-time location of the mowing robot 100 as the center and the preset distance threshold as the radius. In some examples, the preset distance threshold can be 4m, 3m, 5m, or 6m. It is understandable that, in the case where a boundary line of the working area is set, the coverage area of ​​the local map may not exceed the boundary line, that is, the local map corresponding to the area beyond the boundary line may not be constructed.

[0134] As shown in Figures 5 and 6, the global map or local map is composed of multiple map grids, and the map resolution represents the size of the map grid. In some cases, the map grid can be a regular quadrilateral, and the map resolution can be expressed by the grid side length of the map grid. The resolution of the map grid described below is the grid side length of the map grid. In this embodiment, the local map has a higher resolution than the global map. In some examples, the resolution of the local map can be 5 cm, and the resolution of the global map can be 15 cm.

[0135] In some embodiments, the resolutions of multiple map grids in the local map may be different.

[0136] 7 , in some embodiments, the resolution of multiple map grids in a local map may vary with the distance between the map grids and the real-time location of the mowing robot 100. The closer the map grids are to the real-time location of the mowing robot 100, the higher the resolution of the map grids. That is, the closer the map grids are to the real-time location of the mowing robot 100 in the local map, the smaller the grid side lengths are, and the farther the map grids are from the real-time location of the mowing robot 100, the larger the grid side lengths are.

[0137] In some other embodiments, in addition to changing with the distance between the map grid and the real-time location of the robotic lawn mower 100, a resolution threshold may also be set. After the resolution of the map grid reaches the resolution threshold, that is, after the grid side length reaches the maximum grid side length, the map resolution will not be reduced even if the distance between the map grid and the real-time location of the robotic lawn mower 100 becomes farther.

[0138] The resolution of the local map can be adjusted according to the size of the local map, the size of the working area of ​​the mowing robot 100, and the working accuracy requirements. In some embodiments, the maximum resolution of the map grid in the local map can be 1 cm, and the minimum resolution can be 15 cm.

[0139] In some embodiments, a local map contains more types of map attributes than a global map. In some examples, the global map may not record dynamic obstacle information, while the local map does. In other examples, the global map may not record mileage data and travel speed, while the local map does.

[0140] In some embodiments, as the robot mower 100 moves, the processor 40 will also update the local map corresponding to its current real-time location in real time as its real-time location changes. For example, assuming the robot mower 100 is traveling at a speed of 0.5 m / s, the processor 40 may periodically update the local map every 100 ms. Compared to the local map generated in the previous cycle, the real-time location of the robot mower 100 in the current local map has moved 5 cm.

[0141] In some embodiments, the processor 40 can use a local map generated in real time corresponding to the real-time position of the lawn mower robot 100 and / or a stored global map to plan the travel path of the lawn mower robot 100, and control the travel motor 22 based on the planned travel path. In some examples, the processor 40 can also control the drive motor 12 based on the local map and / or the global map.

[0142] In some embodiments, the sensor assembly 30 includes at least one of a visual sensor, a collision sensor, an inertial sensor, a laser radar (LiDAR), and an odometer.

[0143] In addition, in some embodiments, when generating a local map corresponding to the real-time location of the mowing robot 100, the processor 40 may first read the global map stored in the memory 50, map the map information in the global map to the local map to be generated to obtain an initial local map, and then update the initial local map based on the environmental information collected by the sensors to obtain the local map used for subsequent navigation. Specifically, the processor 40 may determine the map area corresponding to the local map to be generated based on the real-time location of the mowing robot 100, read the global map, and map the map information within the map area in the global map to the local map to be generated to generate the initial local map. That is, the initial local map is a cutout of the map area in the global map. The processor 40 obtains environmental information near the real-time location of the mowing robot 100 collected by the sensor assembly 30, and based on the environmental information, determines map information not included in the global map, such as dynamic and static obstacles, bumps, potholes, etc., uses this information to update the initial local map, and subsequently uses the updated local map to achieve navigation of the mowing robot 100.

[0144] In the lawn mower robot described in the above embodiment, the processor can generate a local map corresponding to the real-time position of the lawn mower robot. The local map has a higher resolution than the global map and may also contain more map attributes that can reflect the real-time moving conditions, which can provide more accurate guidance for the navigation and other action controls of the lawn mower robot.

[0145] In one embodiment, the lawn mower robot 100 includes at least a drive assembly 10, a travel assembly 20, a sensor assembly 30, a memory 50, and a processor 40. The memory 50 stores a map of the working area of ​​the lawn mower robot 100, which may be a full map or a partial map of the working area of ​​the lawn mower robot 100.

[0146] The processor 40 is electrically or communicatively connected to the sensor assembly 30 and is capable of acquiring various environmental information collected by the sensor assembly 30. After acquiring various environmental information, the processor 40 can determine road conditions based on the environmental information. Specifically, the processor 40 can acquire environmental information such as environmental images, mileage data, and acceleration data collected by the sensor assembly 30 near different locations within the working area of ​​the robot mower 100. By analyzing and processing this environmental information, the processor 40 can determine obstacle conditions, bump conditions, collision conditions, and other conditions near different locations within the working area, thereby determining road conditions near different locations within the working area. The processor 40 can be pre-configured with multiple road condition types to determine the road conditions near each location, that is, to determine the road condition type to which the road conditions near that location belong. In some examples, the processor 40 can determine the road conditions near the real-time location of the robot mower 100 based on the environmental information collected by the sensor assembly 30 near that location. In other examples, the processor 40 can determine road conditions near multiple different locations based on environmental information collected by the sensor assembly 30 near the robot mower 100 at multiple different locations.

[0147] In some embodiments, the preset multiple road condition types include but are not limited to: at least two of: a bumpy area, a pit area, a slope area, a non-grass area, an obstacle-prone area, a slippery area, and an easily trapped area.

[0148] In some embodiments, the processor 40 can determine whether the current location is in a bumpy area based on the three-axis acceleration data collected by the inertial sensor. In other embodiments, the processor 40 can determine whether the current location is in a pit area or a ramp area based on the image collected by the visual sensor and the three-axis acceleration collected by the inertial sensor. In still other embodiments, the processor 40 can determine whether the current location is in an obstacle-prone area based on the environmental image collected by the visual sensor and the encoder overload current. In still other embodiments, the processor 40 can detect slipping areas and easily trapped areas based on odometer positioning.

[0149] In some embodiments, the processor 40 can control the travel motor 22 to control the travel path and travel speed based on road conditions. In some examples, the processor 40 can also control the drive motor 12 to adjust the blade cutting state based on road conditions.

[0150] The processor 40 is also electrically connected or communicatively connected to the memory 50 and can exchange map information with the memory 50 .

[0151] As shown in Figures 5 to 7, the map is composed of multiple map grids. Taking the global map as an example, each map grid can correspond to a different part of the work area. Each map grid records map information such as road conditions on the part of the work area corresponding to the map grid.

[0152] After determining the road conditions near a location in the working area, the processor 40 can interact with the memory 50 to record the determined road conditions on the map grid corresponding to the location. When the processor arrives at the area corresponding to the map grid again at other times, the road condition update module in the processor 40 can redetermine the road conditions based on the newly collected environmental information of the sensor component 30, and interact with the memory 50 to update the map information on the map grid in the map with the redetermined road conditions.

[0153] In some embodiments, the robotic lawn mower 100 further includes a human-machine interaction component that allows a user to update road conditions on one or more map grids using human-machine interaction. Specifically, map information, such as road conditions, on one or more map grids in the map stored in the memory 50 can be modified via the robotic lawn mower 100's data interface.

[0154] In some embodiments, in addition to determining and recording the road conditions on the map grid, the processor 40 also determines and records the probabilities of different road conditions appearing on the map grid. Specifically, after the processor 40 obtains the road conditions on a map grid based on the analysis of various environmental information collected by the sensors, it can determine the road condition probability of the road condition appearing on the map grid based on the cumulative number of times the road condition appears on the map grid or the frequency of the road condition appearing on the map grid within a preset time period, and record the probability in the map. For example, the road condition probability of a frequently occurring road condition is high, while the road condition probability of an infrequent road condition is low. Similarly, when the vehicle arrives at the corresponding area of ​​the map grid again at other subsequent times, the road condition update module in the processor 40 can redetermine the road condition probability based on the newly collected environmental information by the sensor assembly 30, and interact with the memory 50 to update the map information on the map grid in the map with the redetermined road condition probability. In other embodiments, the processor 40 may also determine the road condition probabilities of different road conditions on each map grid by combining the temporal and spatial attributes of the environmental information collected by the sensor assembly 30. For example, the processor 40 may determine that the road condition probability on a map grid is high during a certain time period and low during other time periods. For another example, if the robotic mower 100 determines the same road condition on a map grid multiple times at different times, from different travel directions, or along different travel paths, the probability of that road condition on that map grid may be increased.

[0155] In some embodiments, the processor 40 may filter the environmental information collected by the sensor assembly 30 before recording or updating the road conditions and road condition probabilities for each map grid, and use the filtered environmental information to record or update the road conditions and road condition probabilities for each map grid. Specifically, the processor 40 may filter the environmental information based on the temporal and spatial attributes of the environmental information collected by the sensor assembly 30. For example, environmental information collected at similar locations and times has high data similarity, and abnormal data may be filtered out.

[0156] In some embodiments, after the road conditions and road condition probabilities for each map grid in the map are recorded and updated, the processor 40 may further calculate a cost value for each map grid based on the road conditions and road condition probabilities, and plan the path of the robotic mower 100 based on the cost values ​​for each map grid in the map. Specifically, the processor 40 may calculate a cost value for each map grid based on the preset cost weights corresponding to each road condition type, as well as the road conditions and road condition probabilities for each map grid. Then, based on the cost values ​​for each map grid, the robotic mower 100 may plan the path with the lowest cost value as the goal. For example, the more dangerous the road condition and the higher the road condition probability, the higher the cost value for that map grid. When planning the path, the processor 40 may try to avoid map grids with high cost values.

[0157] In some embodiments, in addition to determining and recording the road conditions and road condition probabilities on the map grid, the processor 40 may also determine at least one of the second map attributes, such as mowing, lifting, collision, boundary, obstacle, charging pile, etc., near different locations in the work area based on various environmental information collected by the sensors, and record the determined second map attribute on the map grid corresponding to the location in the map.

[0158] In some embodiments, in addition to planning the robot mower 100's path based on the cost values ​​of the map grids, the processor 40 may also adjust the planned path based on at least one of a second map attribute, such as mowing, elevation, collision, boundary, obstacle, or charging post. For example, after planning the path based on the cost values ​​of each grid, the processor 40 may optimize the planned path based on the second map attribute, such as boundary, to further ensure that the planned path does not exceed the work area. In other embodiments, the second map attribute may also be incorporated into the calculation of the cost values ​​of each map grid.

[0159] In the lawn mower robot described in the above embodiment, the processor can generate a map that is marked with map information such as road conditions, road condition probabilities, and cost values ​​of each map grid and is continuously and dynamically updated, which can provide more accurate guidance for the lawn mower robot's navigation and other action controls.

[0160] In one embodiment, the lawn mower robot 100 includes at least a drive assembly 10, a travel assembly 20, a camera assembly 30, and a processor 40. The camera assembly 30 can capture a two-dimensional image of the environment surrounding the lawn mower robot 100. The two-dimensional image, i.e., an image of the environment surrounding the lawn mower robot 100, can be an RGB image or a grayscale image.

[0161] The processor 40 is electrically or communicatively connected to the camera assembly 30 and is capable of acquiring the two-dimensional image captured by the camera assembly 30. After obtaining the two-dimensional image, the processor 40 can identify adverse road conditions in the two-dimensional image through semantic segmentation, and verify the identified adverse road conditions based on the three-dimensional point cloud to obtain a verification result. The processor 40 then plans the path of the mowing robot 100 based on the verification result of the adverse road condition. Specifically, the processor 40 uses a semantic segmentation algorithm to obtain a segmented image corresponding to the two-dimensional image and the type of each object therein. The processor 40 can be pre-set with multiple adverse road condition types. When an object in the two-dimensional image belongs to one of the multiple preset adverse road condition types, the processor 40 identifies the location and type of the adverse road condition in the two-dimensional image, thereby obtaining the location and type of each adverse road condition in the two-dimensional image. The processor 40 then uses the three-dimensional point cloud to verify each adverse road condition and plans the path of the mowing robot 100 based on the verification result of each adverse road condition.

[0162] In some embodiments, the preset multiple adverse road condition types include, but are not limited to, at least two of potholes, water areas, and haystacks.

[0163] In some embodiments, the processor 40 can determine the depth information of each adverse road condition and the depth information of the walking plane on which the mowing robot 100 is located based on the three-dimensional point cloud, and then verify the adverse road condition based on the depth information of the adverse road condition and the depth information of the walking plane. Specifically, the processor 40 can align the two-dimensional image and the three-dimensional point cloud to obtain a registration relationship between the two-dimensional image and the three-dimensional point cloud. The registration between the two is the calibration between the two-dimensional image coordinate system and the three-dimensional point cloud coordinate system. Based on the registration relationship between the two, the coordinate information of the identified adverse road condition in the two-dimensional image can be converted to obtain the depth information of the adverse road condition. Based on the registration relationship between the two, the coordinate information of the walking plane near the adverse road condition in the two-dimensional image can also be converted to obtain the depth information of the walking plane.

[0164] In some embodiments, the coordinate information of each adverse road condition may be the coordinate information of the area within the bounding box obtained when the processor 40 uses a semantic segmentation algorithm to identify the adverse road condition. In some examples, the processor 40 may use a semantic segmentation algorithm to segment a two-dimensional image into multiple image regions and identify the type of objects within each image region, wherein the coordinate information of the image region where objects belonging to a preset adverse road condition type are located is the coordinate information of the adverse road condition. In other examples, the processor 40 may use a semantic segmentation algorithm to frame the identified adverse road condition in the two-dimensional image in the form of a bounding box, and the coordinate information of the image region within the bounding box is the coordinate information of the adverse road condition.

[0165] In some embodiments, the depth information of the walking plane may be the coordinate information of the area outside the bounding box obtained when the processor 40 uses a semantic segmentation algorithm to identify the adverse road condition. Specifically, after determining the image region where the adverse road condition is located, the processor 40 may use the complement of this image region in the two-dimensional image as the image region where the walking plane is located. The coordinate information of the image region corresponding to this complement is the coordinate information of the walking plane.

[0166] In some embodiments, the processor 40 may detect whether the depth information of the walking plane on which the mowing robot 100 is located conforms to preset plane depth information, and if not, determine that the bad road condition verification has failed. Specifically, the processor 40 may fit a plane based on the depth information and verify the center of mass and / or normal vector of the fitted plane. If the center of mass or normal vector does not conform to the center of mass or normal vector of a preset standard ground surface, for example, if the distance error between the center of mass and the standard center of mass exceeds a threshold, or if the angle error between the normal vector and the standard normal vector exceeds a threshold, the bad road condition verification is determined to have failed.

[0167] In some embodiments, the processor 40 may further verify the undesirable road condition based on the depth relationship between the depth information of the undesirable road condition and the depth information of the walking surface. If the depth relationship between the two matches the depth relationship corresponding to the type of undesirable road condition, the undesirable road condition verification is determined to be successful. For example, if the depth relationship between a pothole-type undesirable road condition and the walking surface is such that the undesirable road condition is located below the walking surface, the pothole verification is determined to be successful if the depth relationship between the two meets the above relationship.

[0168] The three-dimensional point cloud used by the processor 40 to verify the adverse road conditions can be generated by the processor 40 or collected by the sensor assembly 30. In some embodiments, the camera assembly 30 includes at least a first camera and a second camera, and the processor 40 can generate a three-dimensional point cloud based on at least two two-dimensional images collected by the camera assembly 30. For example, the processor 40 can generate a corresponding three-dimensional point cloud from two two-dimensional images collected by the first camera and the second camera at the same time using a binocular camera model. In other embodiments, the lawn mower robot 100 also includes sensors such as a laser radar (LiDAR), which can collect environmental information around the lawn mower robot 100, generate a corresponding three-dimensional point cloud, and transmit it to the processor 40.

[0169] In the lawn mower robot described in the above embodiment, the processor can use the three-dimensional point cloud to verify the adverse road conditions identified in the two-dimensional image, and plan the travel path of the lawn mower robot based on the verification results of the adverse road conditions, thereby reducing the impact of misjudgment of adverse road conditions on path planning, and improving the accuracy of the lawn mower robot's navigation and other action controls.

[0170] In one embodiment, the lawn mower robot 100 includes at least a drive assembly 10, a travel assembly 20, a camera assembly 30, and a processor 40. The camera assembly 30 includes at least a first camera and a second camera. The first camera captures a first image of the environment surrounding the lawn mower robot 100, and the second camera captures a second image of the environment surrounding the lawn mower robot 100. The first and second images are two-dimensional images, specifically RGB images or grayscale images, without limitation.

[0171] In some embodiments, the first camera is arranged on the left side of the second camera. It will be understood that what is described here is the relative positional relationship between the first camera and the second camera, and the first camera may also be arranged on the right side, upper side or lower side of the second camera. In other embodiments, the first camera and the second camera together constitute a binocular camera, and the binocular camera may be a passive binocular camera or an active binocular camera. In some other embodiments, the first camera and / or the second camera is a fisheye camera. In some other embodiments, the camera assembly 30 also includes other types of depth cameras such as a structured light camera or a ToF (Time of Flight) camera.

[0172] The processor 40 is electrically connected or communicatively connected to the camera assembly 30, and can obtain the first environment image and the second environment image captured by the camera assembly 30. After obtaining the first environment image and the second environment image, the processor 40 can obtain the position and type of each object in the first environment image and the second environment image through semantic segmentation. Specifically, the processor 40 receives a frame of the first environment image and a frame of the second environment image at the same time. Through semantic segmentation, these two frames of environment images can be segmented and identified to obtain a first segmented image corresponding to the first environment image and the type of each object therein, and a second segmented image corresponding to the second environment image and the type of each object therein. In some examples, the segmented image obtained by semantic segmentation can divide the original environment image into multiple image areas, and can identify the type of objects in each image area. It is understandable that in other embodiments, the processor 40 can also combine instance segmentation and panoramic segmentation on the basis of semantic segmentation.

[0173] After identifying each object in the first environmental image and the second environmental image, the processor 40 can associate repeated objects that appear in both the first environmental image and the second environmental image. Specifically, the processor 40 can match objects based on their positions and types in the first environmental image and the second environmental image, and associate objects that are in the same or similar positions and belong to the same type.

[0174] The processor 40 can use the coordinate information of each associated repetitive object in the first environment image and the coordinate information in the second environment image to perform stereo matching on them to obtain the depth information of each repetitive object, and then control the walking motor 22 based on the recognition results and depth information of each repetitive object. In some cases, the processor 40 also controls the driving motor 12 based on the recognition results and depth information of each repetitive object. Specifically, for a repetitive object, due to the difference in viewing angle between the first camera and the second camera, the processor 40 can obtain a disparity map corresponding to the repetitive object based on the difference between the coordinate information of the repetitive object in the first environment image and the second environment image, and calculate its depth information based on the disparity map of the repetitive object. In some examples, the processor 40 can use the corresponding binocular camera model to calculate the depth information.

[0175] In some embodiments, after associating the repeated objects, the processor 40 may perform stereo matching only on repeated objects of a preset type, so as to further reduce the computational load of the processor 40 and the amount of data transmission between devices.

[0176] In the lawn mowing robot described in the above embodiment, the processor does not directly perform stereo matching on the first and second environmental images, but first performs object recognition and repeated object association on the first and second environmental images, and then only performs stereo matching on the repeated objects in the first and second environmental images. The depth information obtained from the stereo matching of repeated objects is subsequently used to control the lawn mowing robot, which can achieve the beneficial effect of reducing the processor data calculation amount and the data transmission amount between devices.

[0177] In one embodiment, the lawn mower robot 100 includes at least a drive assembly 10, a travel assembly 20, a camera assembly 30, and a processor 40. The camera assembly 30 captures an image of the environment surrounding the lawn mower robot 100. The image can be a two-dimensional image or a three-dimensional image, specifically an RGB image or a grayscale image.

[0178] The processor 40 is electrically connected or communicatively connected to the camera assembly 30 and is capable of acquiring the environmental image captured by the camera assembly 30. After obtaining the environmental image, the processor 40 can cluster the pixels or voxel points in the environmental image to obtain a pixel set or voxel set corresponding to each object in the environmental image. Specifically, a two-dimensional environmental image is composed of multiple pixels, and a three-dimensional environmental image is composed of multiple voxel points. A clustering algorithm is used to cluster the pixels based on RGB values ​​or grayscale values, or to cluster the voxel points based on voxel coordinates. Multiple pixel sets can be obtained from the two-dimensional environmental image, or multiple voxel sets can be obtained from the three-dimensional environmental image. The pixels or voxel points in the same set are regarded as one class. For example, the pixels or voxel points in the same set are the pixels or voxel points that constitute the same object.

[0179] The processor 40 determines whether to exclude each pixel set or voxel set based on the size of the pixel set or voxel set. If the size of the pixel set or voxel set is less than or equal to a first threshold, the pixel set or voxel set is excluded. If the size of the pixel set or voxel set is greater than or equal to a second threshold, the processor 40 identifies the type of object corresponding to the pixel set or voxel set, where the first threshold is less than or equal to the second threshold. After the exclusion and identification are completed, the processor 40 subsequently controls the travel motor 22 based on the type of object corresponding to each pixel set or voxel set that was not eliminated. In some examples, the processor 40 may also control the drive motor 12 based on the type of object corresponding to the pixel set or voxel set that was not eliminated.

[0180] In some embodiments, the processor 40 uses a K-Means or region growing algorithm to cluster pixels in a two-dimensional environment image or voxel points in a three-dimensional environment image. It is understood that other clustering algorithms (not shown) may also be used to cluster pixels or voxel points to meet actual scene requirements.

[0181] A pixel set in a two-dimensional environmental image may be an image region in the two-dimensional environmental image, and a voxel set in a three-dimensional environmental image may be an image space in the three-dimensional environmental image. In some embodiments, the size of a pixel set may be the length or width of the corresponding image region, and the size of a voxel set may be the length, width, or height of the corresponding image space. In other embodiments, the size of a pixel set may be the length of a plane diagonal of the corresponding image region, and the size of a voxel set may be the length of a three-dimensional diagonal of the corresponding image space. In still other embodiments, the size of a pixel set may be the maximum distance between two pixels in the corresponding image region, and the size of a voxel set may be the maximum distance between two voxel points in the corresponding image space.

[0182] In some embodiments, the first threshold used for comparison with the size of the pixel set or voxel set may be 5 cm. In other embodiments, the first threshold used for comparison with the size of the pixel set or voxel set may be 3 cm.

[0183] A pixel set in a two-dimensional environmental image can only convey limited information. In some embodiments, the processor 40 can also convert each pixel set into a corresponding voxel set based on the camera parameters of the camera assembly 30. Specifically, the camera parameters of the camera assembly 30 may include intrinsic and extrinsic parameters, which enable mapping and conversion between a two-dimensional pixel coordinate system and a three-dimensional voxel coordinate system. In some examples, the first threshold used for comparison with the converted voxel set can be 3 cm.

[0184] In some embodiments, the camera assembly 30 includes at least a first camera and a second camera, and the processor 40 can perform stereo matching on the two-dimensional environmental images captured by the two cameras at the same time to obtain a corresponding three-dimensional environmental image. Subsequently, the three-dimensional environmental image can be subjected to voxel point clustering and voxel set elimination and identification.

[0185] In addition, in addition to using size as a criterion for eliminating pixel sets or voxel sets, other set parameters can also be used as criteria for eliminating pixel sets or voxel sets. In some embodiments, after clustering to obtain pixel sets or voxel sets, the processor 40 can also determine whether to eliminate the pixel set or voxel set based on the number of pixels in each pixel set or the number of voxels in the voxel set. If the number of pixels in the pixel set is less than or equal to a third threshold or the number of voxels in the voxel set is less than or equal to a fifth threshold, the pixel set or voxel set is eliminated; if the number of pixels in the pixel set is greater than or equal to a fourth threshold or the number of voxels in the voxel set is greater than or equal to a sixth threshold, the type of object corresponding to the pixel set or voxel set is identified, wherein the third threshold is less than or equal to the fourth threshold and the fifth threshold is less than or equal to the sixth threshold. After elimination and identification are completed, the processor 40 subsequently controls the travel motor 22 based on the type of object corresponding to each pixel set or voxel set that was not eliminated. In some examples, the drive motor 12 can also be controlled based on the type of object corresponding to the pixel set or voxel set that was not eliminated.

[0186] In some embodiments, the fifth threshold value used for comparison with the number of voxel points in the voxel set may be 15.

[0187] In the lawn mowing robot 100 described in the above embodiment, the processor 40 does not directly perform object recognition on the environmental image, but first clusters the pixel points or voxel points in the environmental image, and eliminates the pixel sets or voxel sets with a size smaller than a threshold, and only performs type recognition on the objects corresponding to the pixel sets or voxel sets with a size larger than the threshold, which can achieve the beneficial effect of reducing the data calculation amount of the processor 40 and the data transmission amount between devices.

[0188] In one embodiment, the lawn mower robot 100 includes at least a drive assembly 10, a travel assembly 20, a sensor assembly 30, a first processor 41, and a second processor 42. The first processor 41 and the second processor 42 are configured to perform different processing tasks. Referring to FIG9 , the first processor 41 is electrically or communicatively connected to the sensor assembly 30, capable of receiving environmental information about the lawn mower robot 100 collected by the sensor assembly 30, and capable of exchanging data with the second processor 42. The second processor 42 is capable of exchanging data with the first processor 41 and controlling the actions of the lawn mower robot 100. In some examples, the first processor 41 and the second processor 42 can be different processors in a physical sense, for example, MCU1 and MCU2, respectively, or different processors in a virtual sense, for example, VCPU1 and VCPU2, respectively.

[0189] After obtaining the environmental information, the first processor 41 can identify and obtain a coordinate set of obstacles surrounding the robot mower 100 based on the environmental information, then determine representative coordinates of the obstacles based on the coordinate set, and transmit the representative coordinates to the second processor 42. Specifically, the first processor 41 can identify obstacles surrounding the robot mower 100 based on environmental information such as environmental images captured by a visual sensor or a laser point cloud captured by a lidar, and obtain the image area of ​​the obstacle in a two-dimensional image, the image space in a three-dimensional image, or the point cloud space in a three-dimensional point cloud. Based on these image areas, image spaces, or point cloud spaces, the first processor 41 can determine a coordinate set for each obstacle, further select representative coordinates from the coordinate set to represent the position of the obstacle, and then transmit the representative coordinates of each obstacle to the second processor 42.

[0190] After receiving the representative coordinates of each obstacle from the first processor 41 , the second processor 42 may control the travel motor 22 based on the representative coordinates of each obstacle. In some examples, the second processor 42 may also control the drive motor 12 based on the representative coordinates.

[0191] In some embodiments, the sensor assembly 30 includes a camera assembly, which can capture images of the environment around the robotic mower 100 and transmit the images to the first processor 41 .

[0192] In some embodiments, the sensor assembly 30 includes a binocular camera and / or a laser radar (LiDAR).

[0193] In some embodiments, when identifying obstacles around the robotic lawn mower 100 based on a two-dimensional environmental image, the first processor 41 obtains a bounding box for each obstacle in the two-dimensional environmental image. The coordinates of two vertices on a plane diagonal of the bounding box may be used as representative coordinates of the obstacle. In some examples, the first processor 41 may also convert the vertex coordinates of the plane diagonal into three-dimensional diagonal vertex coordinates based on the intrinsic and extrinsic parameters of the camera assembly 30 and transmit the converted coordinates to the second processor 42.

[0194] In other embodiments, when the first processor 41 identifies obstacles around the lawn mower robot 100 based on a three-dimensional environmental image or a three-dimensional point cloud, it obtains a bounding box of each obstacle in the three-dimensional environmental image or the three-dimensional point cloud, and the coordinates of the two vertices on the three-dimensional diagonal of the bounding box can be used as representative coordinates of the obstacle.

[0195] In some embodiments, the first processor 41 obtains the type of each obstacle when identifying obstacles based on the environment image, and may transmit the type of each obstacle when transmitting the representative coordinates of each obstacle to the second processor 42 .

[0196] In some embodiments, the first processor 41 may be a GPU (Graphics Processing Unit), and the second processor 42 may be a CPU. It is understood that in some cases, the lawn mower robot actually has two system-on-chips (SoCs), one of which has a CPU and GPU, and the other has a CPU. The chip containing the GPU performs image processing and other computations, and the processed data is exchanged between the two chips.

[0197] In some embodiments, the first processor 41 may be a Neural-Network Processing Unit (NPU), and the second processor 42 may be a CPU.

[0198] In some embodiments, the amount of data of the representative coordinates of the obstacle exchanged between the first processor 41 and the second processor 42 is less than or equal to 100 bytes.

[0199] In the lawn mowing robot described in the above embodiment, the first processor does not directly use the complete obstacle coordinate set when exchanging obstacle position information with the second processor, but uses its representative coordinates, which can effectively reduce the amount of interactive data between the first processor and the second processor and alleviate the data transmission and processing burden of the processor.

[0200] It should be noted that the above-mentioned embodiments are not separated and independent of each other. New embodiments obtained by combining all or part of the contents of multiple embodiments according to actual scenario requirements should also be within the scope of protection of this application.

[0201] For example, the processor 40 can identify the type of each object in the two-dimensional environmental image captured by the camera component 30. After identifying the object belonging to the bad road condition type, it uses the three-dimensional point cloud to verify the bad road condition. If the verification is successful, the local map is called for path planning and obstacle avoidance to improve the accuracy of object recognition, path planning and obstacle avoidance of the lawn mowing robot.

[0202] For example, the processor 40 can update the map information such as real-time obstacles, travel speed, mileage data in the local map corresponding to the real-time position of the lawn mower robot 100 based on various environmental information collected by the sensor component 30, and update the map information such as road conditions, road condition probabilities and cost values ​​on each map grid in the global map corresponding to the real-time position of the lawn mower robot 100, so as to improve the map generation capability of the lawn mower robot and the performance of the generated map.

[0203] For another example, the first processor 41 may cluster the pixel points in the environmental image captured by the first and second cameras in the camera assembly 30 to obtain pixel sets corresponding to each object in the environmental image, and eliminate and identify the pixel sets based on the size of the pixel sets; then, the first processor 41 may associate the repeated objects in the first and second environmental images after eliminating the pixel sets with a size smaller than a threshold, and perform stereo matching on the associated repeated objects to obtain depth information of the repeated objects; finally, the first processor 41 may determine the representative coordinates of the obstacle based on the depth information of the repeated objects belonging to the obstacle, and transmit the representative coordinates of each obstacle to the second processor 42; the second processor 42 controls the walking motor 22 based on the representative coordinates of the obstacle to reduce the data calculation amount of the processor and the data transmission amount between devices.

[0204] It is understandable that the various embodiments described above can also be applied independently or in combination to outdoor work vehicles. The outdoor work vehicle in this application can be a device that can independently plan and decide its itinerary, such as the lawn mower robot described above, for example, a snow sweeper robot, etc.; the outdoor work vehicle can also be other devices with autonomous walking as a reference or alternative function, such as riding lawn mowers, stand-on lawn mowers and other manned lawn mowers, as well as cleaning machines, multi-purpose all-terrain vehicles (UTVs) / farm vehicles, all-terrain vehicles (ATVs) / all-terrain vehicles (ATVs), beach cars, golf carts, etc.

[0205] The above shows and describes the basic principles, main features and advantages of this application. Those skilled in the art should understand that the above embodiments do not limit this application in any form, and any technical solutions obtained by equivalent replacement or equivalent transformation fall within the scope of protection of this application.

Claims

1. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A camera assembly configured to collect a two-dimensional image of the surroundings of the lawn mowing robot; a processor, communicatively connected or electrically connected to the camera assembly, and configured to control the action of the lawn mowing robot at least according to the two-dimensional image; Among them, the processor is further configured to: identify bad road conditions in the two-dimensional image through semantic segmentation, and the bad road conditions belong to one of multiple preset bad road condition types; verify the bad road conditions according to the three-dimensional point cloud to obtain a verification result; and plan the travel route of the lawn mowing robot according to the verification result.

2. The lawn mowing robot according to claim 1, wherein: The processor is further configured to avoid or slow down the verified adverse road condition.

3. The lawn mowing robot according to claim 1, wherein: The preset multiple adverse road condition types include at least two of potholes, water areas or haystacks.

4. The lawn mowing robot according to claim 1, wherein: The camera assembly includes at least a first camera and a second camera, and the processor is further configured to generate the three-dimensional point cloud based on at least two of the two-dimensional images.

5. The lawn mowing robot according to claim 1, wherein: The lawn mowing robot also includes a sensor component, which is communicatively connected or electrically connected to the processor and is configured to collect environmental information around the lawn mowing robot to generate the three-dimensional point cloud and transmit the three-dimensional point cloud to the processor.

6. The lawn mowing robot according to claim 1, wherein: The processor determines the depth information of the bad road condition and the depth information of the walking plane of the lawn mower person according to the three-dimensional point cloud, and verifies the bad road condition using the depth information of the bad road condition and the depth information of the walking plane.

7. The lawn mowing robot according to claim 6, wherein: The processor determines the depth information of the bad road condition according to the coordinate information of the bad road condition in the two-dimensional image and the registration relationship between the two-dimensional image and the three-dimensional point cloud.

8. The lawn mowing robot according to claim 7, wherein: The processor obtains a bounding box of the bad road condition through semantic segmentation, and the coordinate information of the bad road condition is the coordinate information of the area within the bounding box in the two-dimensional image.

9. The lawn mowing robot according to claim 6, wherein: The processor determines the depth information of the walking plane according to the coordinate information of the walking plane within a certain distance around the location of the bad road condition in the two-dimensional image and the registration relationship between the two-dimensional image and the three-dimensional point cloud.

10. The lawn mowing robot according to claim 9, wherein: The processor obtains a bounding box of the bad road condition through semantic segmentation, and the coordinate information of the walking plane is the coordinate information of the area outside the bounding box in the two-dimensional image.

11. The lawn mowing robot according to claim 6, wherein: When the depth information of the walking plane does not meet the preset plane depth information, the processor determines that the bad road condition verification fails.

12. The lawn mowing robot according to claim 6, wherein: The processor determines that the bad road condition verification is successful when the depth relationship between the depth information of the bad road condition and the depth information of the walking plane meets a preset depth relationship corresponding to the type to which the bad road condition belongs.

13. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A sensor component configured to collect environmental information around the lawn mowing robot; a memory configured to store a global map of a working area of ​​the lawn mowing robot; a processor, communicatively connected or electrically connected to the sensor assembly and the memory, and configured to control the action of the lawn mowing robot at least according to the environmental information collected by the sensor; The processor is further configured to construct a local map around the real-time position of the lawn mowing robot according to the environmental information collected by the sensor component, and the local map has a higher resolution than the global map.

14. The lawn mowing robot according to claim 13, wherein: The local map contains more types of map attributes than the global map.

15. The lawn mowing robot according to claim 13, wherein: The resolution of the local map is greater than or equal to 1 cm and less than or equal to 20 cm.

16. The lawn mowing robot according to claim 13, wherein: The processor maps the map information in the global map to the local map according to the real-time position of the lawn mowing robot.

17. The lawn mowing robot according to claim 13, wherein: The shape of the local map is fixed, and the distance between the boundary of the local map and the real-time position of the lawn mowing robot is less than or equal to 5m.

18. The lawn mowing robot according to claim 13, wherein: The processor is further configured to plan a travel route of the lawn mowing robot according to the global map and the local map, and control the travel motor according to the travel route.

19. The lawn mowing robot according to claim 13, wherein: The local map is composed of a plurality of map grids, and the resolutions of the plurality of map grids are different.

20. The lawn mowing robot according to claim 19, wherein: Among the plurality of map grids, a map grid with a smaller distance from the real-time position of the lawn mowing robot has a higher resolution.

21. The lawn mowing robot according to claim 13, wherein: The sensor component includes at least one of a visual sensor, a collision sensor, an inertial sensor, a laser radar, and an odometer.

22. The lawn mowing robot according to claim 13, wherein: The processor updates the local map in real time.

23. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A sensor component configured to collect environmental information around the lawn mowing robot; a memory configured to store a global map of a working area of ​​the lawn mowing robot; a processor, communicatively connected or electrically connected to the sensor assembly and the memory, and configured to control the action of the lawn mowing robot at least according to the environmental information collected by the sensor; The processor is further configured to: map the map information in the global map to an initial local map around the real-time position of the lawn mowing robot according to the real-time position; update the initial local map using the environmental information collected by the sensor component to generate a local map around the real-time position; and control the walking motor according to the global map and the local map.

24. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A sensor component, including at least one of a visual sensor, a collision sensor, an inertial sensor, an odometer, and a laser radar; A memory configured to store a map of a working area of ​​the lawn mowing robot, wherein the map is divided into a plurality of map grids; a processor, communicatively connected or electrically connected to the sensor assembly and the memory, and configured to control the action of the lawn mowing robot at least according to the environmental information collected by the sensor assembly; The processor is further configured to: determine the road condition according to the environmental information collected by the sensor component, the road condition belonging to at least one of the preset multiple road condition types; record the road condition on the corresponding map grid in the map; the processor also includes a road condition update module, the road condition The updating module is configured to update the road condition on the corresponding map grid in the map according to the environmental information newly collected by the sensor component.

25. The lawn mowing robot according to claim 24, wherein: The lawn mowing robot further comprises a human-computer interaction component, and the human-computer interaction component is configured to allow a user to update the road conditions on one or more map grids in the map by means of human-computer interaction.

26. The lawn mowing robot according to claim 24, wherein: The preset multiple road condition types include at least two of a bumpy area, a pit area, a ramp area, a non-grass area, an obstacle-prone area, a slippery area, and an easily trapped area.

27. The lawn mowing robot according to claim 24, wherein: The processor is further configured to record the road condition probability corresponding to the road condition on a corresponding map grid in the map; the road condition update module is further configured to update the road condition probability on the corresponding map grid in the map according to the newly collected environmental information by the sensor component.

28. The lawn mowing robot according to claim 27, wherein: The processor is also configured to filter the environmental information collected by the sensor component according to the time attribute and the space attribute of the environmental information collected by the sensor component; the road condition update module updates the road condition and the road condition probability according to the filtered environmental information collected by the sensor component.

29. The lawn mowing robot according to claim 27, wherein: The processor is further configured to determine a cost value of each map grid according to the road condition and the road condition probability on each map grid, and plan a walking path of the lawn mowing robot according to the cost value of each map grid.

30. The lawn mowing robot according to claim 29, wherein: The memory is further configured to store at least one second map attribute on each grid in the map; and the processor is further configured to adjust the walking path according to the second map attribute.

31. The lawn mowing robot according to claim 30, wherein: The second map attribute includes at least one of mowing, lifting, collision, boundary, obstacle, and charging pile.

32. The lawn mowing robot according to claim 29 or 30, wherein: The processor controls the travel motor to drive the travel wheel at different speeds according to the cost value and / or the second map attribute.

33. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; The camera assembly includes a first camera and a second camera, wherein the first camera is configured to collect a first environment image around the lawn mower robot, and the second camera is configured to collect a second environment image around the lawn mower robot, wherein the first environment image and the second environment image are two-dimensional images. picture; a processor, communicatively connected or electrically connected to the camera assembly, and configured to control the action of the lawn mowing robot at least according to the first environment image and the second environment image; The processor is further configured to: identify each object contained in the first environment image and the second environment image respectively through semantic segmentation; associate repeated objects in the first environment image and the second environment image; perform stereo matching on the repeated objects in the first environment image and the second environment image to obtain depth information of the repeated objects; The travel motor is controlled according to the recognition result of the repetitive object and the depth information.

34. The lawn mowing robot according to claim 33, wherein: The first camera is arranged on the left side of the second camera.

35. The lawn mowing robot according to claim 33, wherein: The first camera and the second camera form a binocular camera.

36. The lawn mowing robot according to claim 33, wherein: The first camera or the second camera is a fisheye camera.

37. The lawn mowing robot according to claim 33, wherein: The camera assembly also includes a structured light camera and / or a ToF (Time of Flight) camera.

38. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A sensor component configured to collect environmental information around the lawn mowing robot; a processor, communicatively connected or electrically connected to the sensor assembly, and configured to control the action of the lawn mowing robot at least according to the environmental information collected by the sensor assembly; The processor is further configured to: identify obstacles based on the environmental information collected by the sensor component, and determine whether the obstacle is static or dynamic; when the obstacle is determined to be dynamic, use the first map for navigation; when the obstacle is determined to be static, use the second map for navigation.

39. The lawn mowing robot according to claim 38, wherein: The second map is a local map, and the local map is a map within a certain distance around the real-time position of the lawn mowing robot.

40. The lawn mowing robot according to claim 38, wherein: The second map is updated in real time during the movement of the lawn mowing robot.

41. The lawn mowing robot according to claim 38, wherein: The first map is a global map, and the global map is an overall map of the working area of ​​the lawn mowing robot.

42. The lawn mowing robot according to claim 38, wherein: The second map has a higher resolution than the first map.

43. The lawn mowing robot according to claim 38, wherein: The processor is further configured to control the travel motor to drive the travel wheel at a first speed when navigating using the first map; When navigating using the second map, the travel motor is controlled to drive the travel wheel at a second speed.

44. The lawn mowing robot according to claim 38, wherein: The sensor assembly includes a camera assembly, and the camera assembly is configured to collect environmental images around the lawn mowing robot.

45. The lawn mowing robot according to claim 44, wherein: The processor uses a semantic segmentation algorithm to identify the obstacles and their types appearing in the environment image, and when the obstacle is identified as a person or an animal, it is determined to be dynamic.

46. ​​The lawn mowing robot according to claim 44, wherein: The processor uses an optical flow method or a target tracking method to detect whether the object in the environment image is dynamic.

47. The lawn mowing robot according to claim 38, wherein: The sensor component includes one or more of a binocular camera, a LiDAR, and an infrared sensor.

48. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A camera assembly is configured to collect an environmental image around the lawn mowing robot, wherein the environmental image is a two-dimensional image or a three-dimensional image; a processor, communicatively connected or electrically connected to the camera assembly, and configured to control the actions of the lawn mowing robot at least according to the environmental image; In which, the processor is further configured to: cluster the pixel points or voxel points in the environmental image to obtain a pixel set or voxel set corresponding to each object contained in the environmental image; when the size of the pixel set or the voxel set is less than or equal to a first threshold, eliminate the pixel set or the voxel set; when the size of the pixel set or the voxel set is greater than or equal to a second threshold, identify the type of object corresponding to the pixel set or the voxel set, and control the walking motor according to the type of the object.

49. The lawn mowing robot according to claim 48, wherein: The processor clusters the pixel points or voxel points in the environment image using a K-Means or region growing algorithm.

50. The lawn mowing robot according to claim 48, wherein: The camera assembly includes at least two cameras, and the processor performs stereo matching on two two-dimensional environment images captured by the two cameras at the same time to generate a corresponding three-dimensional environment image.

51. The lawn mowing robot according to claim 48, wherein: The processor converts the pixel set into the corresponding voxel set according to the camera parameters of the camera assembly.

52. The lawn mowing robot according to claim 48, wherein: The size is the length of a two-dimensional diagonal of a bounding box of the set of pixels, or the length of a three-dimensional diagonal of a bounding box of the set of voxels.

53. The lawn mowing robot according to claim 48, wherein: The size is one of the length, width, and height of a bounding box of the set of pixels or the set of voxels.

54. The lawn mowing robot according to claim 48, wherein: The size is the maximum value of the distance between two pixels or two voxels in the pixel set or the voxel set.

55. The lawn mowing robot according to claim 48, wherein: The first threshold is less than or equal to 5 cm.

56. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A camera assembly is configured to collect an environmental image around the lawn mowing robot, wherein the environmental image is a two-dimensional image or a three-dimensional image; a processor, communicatively connected or electrically connected to the camera assembly, and configured to control the actions of the lawn mowing robot at least according to the environmental image; Wherein, the processor is further configured to: cluster the pixel points or voxel points in the environmental image to obtain a pixel set or voxel set corresponding to each object contained in the environmental image; when the set parameter of the pixel set or the voxel set is less than or equal to a parameter threshold, eliminate the pixel set or the voxel set; when the set parameter of the pixel set or the voxel set is greater than or equal to the parameter threshold, identify the type of object corresponding to the pixel set or the voxel set, and control the walking motor according to the type of the object.

57. The lawn mowing robot according to claim 56, wherein: The set parameter is the number of pixels or the number of voxels.

58. The lawn mowing robot according to claim 57, wherein: The parameter threshold corresponding to the number of voxel points is 15.

59. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A sensor component configured to collect environmental information around the lawn mowing robot; A first processor is configured to identify obstacles according to the environmental information and determine a coordinate set of the obstacles; A second processor is configured to control the actions of the lawn mowing robot; Wherein, the first processor is further configured to: determine the representative coordinates of the obstacle according to the coordinate set of the obstacle, and transmit the representative coordinates to the second processor; the second processor is further configured to: control the walking motor according to the representative coordinates of the obstacle.

60. The lawn mowing robot according to claim 59, wherein: The sensor assembly includes a camera assembly, and the camera assembly is configured to collect environmental images around the lawn mowing robot.

61. The lawn mowing robot according to claim 60, wherein: The first processor is a GPU, and the second processor is a CPU.

62. The lawn mowing robot according to claim 61, wherein: The first processor is further configured to identify the type of the obstacle in the environment image, and transmit the type of the obstacle and the representative coordinates of the obstacle to the second processor.

63. The lawn mowing robot according to claim 59, wherein: The representative coordinates are coordinates of vertices of a three-dimensional diagonal line of a bounding box of the obstacle.

64. The lawn mowing robot according to claim 59, wherein: The representative coordinates are the coordinates of the vertices of the plane diagonal lines of the bounding box of the obstacle.

65. The lawn mowing robot according to claim 59, wherein: The sensor assembly includes a binocular camera and / or a laser radar (LiDAR).

66. The lawn mowing robot according to claim 59, wherein: The amount of data representing the coordinates is less than or equal to 100 bytes.

Images