Mowing robot

Abstract

The invention discloses a mowing robot and a control method thereof. The mowing robot comprises: a driving assembly; a walking assembly; the camera assembly is configured to collect an environment image around the mowing robot, and the environment image is a two-dimensional image or a three-dimensional image; the processor is in communication connection or electric connection with the camera assembly and is configured to control the action of the mowing robot at least according to the environment image; wherein the processor is further configured to cluster pixel points or voxel points in the environment image to obtain a pixel set or a voxel set corresponding to each object contained in the environment image; when the size of the pixel set or the voxel set is smaller than or equal to a first threshold value, the pixel set or the voxel set is removed; when the size of the pixel set or the voxel set is larger than or equal to a second threshold value, the type of an object corresponding to the pixel set or the voxel set is recognized, and a walking motor is controlled according to the type of the object.

Description

Technical Field

[0001] The present application relates to the technical field of garden tools, and in particular to a lawn mowing robot. Background Art

[0002] As a common gardening tool, a lawn mower can be used for lawn mowing. Specifically, it can mow the lawn within a working area surrounded by a boundary line. Thanks to the development of related technologies, in order to meet the growing needs of users and provide more intelligent and diversified services, lawn mowers have gradually evolved into lawn mowing robots.

[0003] The lawn mowing robot has multiple functions such as path planning, object recognition and obstacle avoidance, which can avoid problems such as collisions and repetitive work. Compared with traditional lawn mowers, it can significantly improve work efficiency and reduce energy consumption. It is more user-friendly in terms of both cost consumption and usage experience.

[0004] This section provides background information related to the present application which is not necessarily prior art. Summary of the invention

[0005] One object of the present application is to solve or at least alleviate part or all of the above problems. To this end, one object of the present application is to provide a lawn mowing robot.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] A lawn mowing robot comprises: a driving component, comprising a blade and a driving motor for driving the blade; a walking component, comprising walking wheels and a walking motor for driving the walking wheels; a sensor component, configured to collect environmental information around the lawn mowing robot; a processor, communicatively connected or electrically connected to the sensor component, and configured to control the action of the lawn mowing robot at least according to the environmental information collected by the sensor component; wherein the processor is further configured to: identify obstacles according to the environmental information collected by the sensor component, and determine whether the obstacles are static or dynamic; when the obstacle is determined to be dynamic, use a first map for navigation; when the obstacle is determined to be static, use a second map for navigation.

[0008] In some embodiments, the second map is a local map, and the local map is a map within a certain distance around the real-time position of the lawn mowing robot.

[0009] In some embodiments, the second map is updated in real time during the movement of the lawn mowing robot.

[0010] In some embodiments, the first map is a global map, which is an overall map of the working area of ​​the lawn mowing robot.

[0011] In some embodiments, the second map has a higher resolution than the first map.

[0012] In some embodiments, the processor is further configured to control the travel motor to drive the travel wheels at a first speed when navigating using the first map; and to control the travel motor to drive the travel wheels at a second speed when navigating using the second map.

[0013] In some embodiments, the sensor assembly includes a camera assembly configured to collect images of the environment around the lawn mowing robot.

[0014] In some embodiments, the processor uses a semantic segmentation algorithm to identify obstacles and their types appearing in the environment image, and when the obstacle is identified as a person or an animal, it is determined to be dynamic.

[0015] In some embodiments, the processor uses a light streaming method or a target tracking method to detect whether an object in the environment image is dynamic.

[0016] In some embodiments, the sensor component includes one or more of a binocular camera, a laser radar (LiDAR), and an infrared sensor.

[0017] A lawn mowing robot comprises: a driving component, comprising a blade and a driving motor for driving the blade; a walking component, comprising walking wheels and a walking motor for driving the walking wheels; a sensor component, configured to collect environmental information around the lawn mowing robot; a memory, configured to store a global map of the working area of ​​the lawn mowing robot; a processor, communicatively connected or electrically connected to the sensor component and the memory, and configured to control the action of the lawn mowing robot at least according to the environmental information collected by the sensor; wherein the processor is further configured to: construct a local map around the real-time position of the lawn mowing robot according to the environmental information collected by the sensor component, and the local map has a higher resolution than the global map.

[0018] In some embodiments, the local map contains more types of map attributes than the global map.

[0019] In some embodiments, the resolution of the local map is greater than or equal to 1 cm and less than or equal to 20 cm.

[0020] In some embodiments, the processor maps map information in the global map to the local map according to the real-time position of the lawn mowing robot.

[0021] In some embodiments, the shape of the local map is fixed, and the distance between the boundary of the local map and the real-time position of the lawn mowing robot is less than or equal to 5 m.

[0022] In some embodiments, the processor is further configured to: plan a travel route of the lawn mowing robot according to the global map and the local map, and control the travel motor according to the travel route.

[0023] In some embodiments, the local map is composed of multiple map grids, and the resolutions of the multiple map grids are different.

[0024] In some embodiments, a map grid with a smaller distance from the real-time position of the lawn mowing robot among the multiple map grids has a higher resolution.

[0025] In some embodiments, the sensor assembly includes at least one of a visual sensor, a collision sensor, an inertial sensor, a lidar, and an odometer.

[0026] In some embodiments, the processor updates the local map in real time.

[0027] A lawn mowing robot comprises: a driving component, comprising a blade and a driving motor for driving the blade; a walking component, comprising a walking wheel and a walking motor for driving the walking wheel; a sensor component, comprising at least one of a visual sensor, a collision sensor, an inertial sensor, an odometer, and a laser radar; a memory, configured to store a map of a working area of ​​the lawn mowing robot, wherein the map is divided into a plurality of map grids; a processor, communicatively connected or electrically connected to the sensor component and the memory, and configured to control the action of the lawn mowing robot at least according to environmental information collected by the sensor component; wherein the processor is further configured to: determine a road condition according to the environmental information collected by the sensor component, wherein the road condition belongs to at least one of a plurality of preset road condition types; and record the road condition on a corresponding map grid in a map; the processor also comprises a road condition update module, wherein the road condition update module is configured to update the road condition on a corresponding map grid in a map according to the environmental information newly collected by the sensor component.

[0028] In some embodiments, the lawn mowing robot further includes a human-computer interaction component, and the human-computer interaction component is configured to allow a user to update the road conditions on one or more map grids in the map by using human-computer interaction means.

[0029] In some embodiments, the preset multiple road condition types include at least two of a bumpy area, a pit area, a ramp area, a non-grass area, an obstacle-prone area, a slippery area, and an easily trapped area.

[0030] In some embodiments, the processor is further configured to record the traffic condition probability corresponding to the traffic condition on a corresponding map grid in the map; the traffic condition update module is further configured to update the traffic condition probability on the corresponding map grid in the map according to the newly collected environmental information by the sensor component.

[0031] In some embodiments, the processor is further configured to filter the environmental information collected by the sensor component according to the time attributes and spatial attributes of the environmental information collected by the sensor component; the road condition update module updates the road condition and road condition probability according to the filtered environmental information collected by the sensor component.

[0032] In some embodiments, the processor is further configured to determine a cost value of each map grid according to the road condition and the road condition probability on each map grid, and plan a walking path of the lawn mowing robot according to the cost value of each map grid.

[0033] In some embodiments, the memory is further configured to store at least one second map attribute on each grid in the map; and the processor is further configured to adjust the walking path according to the second map attribute.

[0034] In some embodiments, the second map attribute includes at least one of mowing, lifting, collision, boundary, obstacle, and charging station.

[0035] In some embodiments, the processor controls the travel motor to drive the travel wheel at different speeds according to the cost value and / or the second map attribute.

[0036] A lawn mowing robot comprises: a driving component, comprising a blade and a driving motor for driving the blade; a walking component, comprising walking wheels and a walking motor for driving the walking wheels; a sensor component, configured to collect environmental information around the lawn mowing robot; a memory, configured to store a global map of the working area of ​​the lawn mowing robot; a processor, communicatively connected or electrically connected to the sensor component and the memory, and configured to control the action of the lawn mowing robot at least according to the environmental information collected by the sensor; wherein the processor is further configured to: map the map information in the global map to an initial local map around the real-time position of the lawn mowing robot according to the real-time position; update the initial local map using the environmental information collected by the sensor component to generate a local map around the real-time position; and control the walking motor according to the global map and the local map.

[0037] A lawn mowing robot comprises: a driving component, comprising a blade and a driving motor for driving the blade; a walking component, comprising walking wheels and a walking motor for driving the walking wheels; a camera component, configured to collect two-dimensional images around the lawn mowing robot; a processor, communicatively connected or electrically connected to the camera component, and configured to control the actions of the lawn mowing robot at least according to the two-dimensional image; wherein the processor is further configured to: identify bad road conditions in the two-dimensional image through semantic segmentation, the bad road conditions belonging to one of a plurality of preset bad road condition types; verify the bad road conditions according to a three-dimensional point cloud to obtain a verification result; and plan a travel route of the lawn mowing robot according to the verification result.

[0038] In some embodiments, the processor is further configured to avoid or slow down the vehicle in response to verified adverse road conditions.

[0039] In some embodiments, the preset multiple adverse road condition types include at least two of potholes, water areas, or haystacks.

[0040] In some embodiments, the camera assembly includes at least a first camera and a second camera, and the processor is further configured to generate a three-dimensional point cloud based on at least two two-dimensional images.

[0041] In some embodiments, the lawn mowing robot further includes a sensor component, which is communicatively or electrically connected to the processor and configured to collect environmental information around the lawn mowing robot to generate a three-dimensional point cloud and transmit the three-dimensional point cloud to the processor.

[0042] In some embodiments, the processor determines depth information of the bad road condition and depth information of the walking plane of the lawn mower person based on the three-dimensional point cloud, and uses the depth information of the bad road condition and the depth information of the walking plane to verify the bad road condition.

[0043] In some embodiments, the processor determines the depth information of the bad road condition based on the coordinate information of the bad road condition in the two-dimensional image and the registration relationship between the two-dimensional image and the three-dimensional point cloud.

[0044] In some embodiments, the processor obtains a bounding box of the bad road condition through semantic segmentation, and the coordinate information of the bad road condition is the coordinate information of the area within the bounding box in the two-dimensional image.

[0045] In some embodiments, the processor determines the depth information of the walking plane based on the coordinate information of the walking plane within a certain distance around the location of the bad road condition in the two-dimensional image and the registration relationship between the two-dimensional image and the three-dimensional point cloud.

[0046] In some embodiments, the processor obtains a bounding box of the bad road condition through semantic segmentation, and the coordinate information of the walking plane is the coordinate information of the area outside the bounding box in the two-dimensional image.

[0047] In some embodiments, the processor determines that the bad road condition verification has failed when the depth information of the walking plane does not meet the preset plane depth information.

[0048] In some embodiments, the processor determines that the bad road condition verification is successful when the depth relationship between the depth information of the bad road condition and the depth information of the walking plane meets the preset depth relationship corresponding to the type of the bad road condition.

[0049] A lawn mowing robot comprises: a driving component, comprising a blade and a driving motor for driving the blade; a walking component, comprising a walking wheel and a walking motor for driving the walking wheel; a camera component, comprising a first camera and a second camera, the first camera being configured to collect a first environment image around the lawn mowing robot, and the second camera being configured to collect a second environment image around a lawn mower, the first environment image and the second environment image being two-dimensional images; a processor, being communicatively connected or electrically connected to the camera component, and being configured to control the action of the lawn mowing robot at least according to the first environment image and the second environment image; wherein the processor is further configured to: respectively identify each object contained in the first environment image and the second environment image by semantic segmentation; associate repeated objects in the first environment image and the second environment image; perform stereo matching on repeated objects in the first environment image and the second environment image to obtain depth information of the repeated objects; and control the walking motor according to the recognition result and depth information of the repeated objects.

[0050] In some embodiments, the first camera is disposed on the left side of the second camera.

[0051] In some embodiments, the first camera and the second camera form a binocular camera.

[0052] In some embodiments, the first camera or the second camera is a fisheye camera.

[0053] In some embodiments, the camera assembly also includes a structured light camera and / or a ToF (Time of Flight) camera.

[0054] A lawn mowing robot comprises: a driving component, comprising a blade and a driving motor for driving the blade; a walking component, comprising a walking wheel and a walking motor for driving the walking wheel; a camera component, configured to collect an environmental image around the lawn mowing robot, the environmental image being a two-dimensional image or a three-dimensional image; a processor, communicatively connected or electrically connected to the camera component, and configured to control the action of the lawn mowing robot at least according to the environmental image; wherein the processor is further configured to: cluster pixel points or voxel points in the environmental image to obtain a pixel set or voxel set corresponding to each object contained in the environmental image; when the size of the pixel set or voxel set is less than or equal to a first threshold, eliminate the pixel set or voxel set; when the size of the pixel set or voxel set is greater than or equal to a second threshold, identify the type of object corresponding to the pixel set or voxel set, and control the walking motor according to the type of the object.

[0055] In some embodiments, the processor uses K-Means or region growing algorithms to cluster pixels or voxels in the environment image.

[0056] In some embodiments, the camera assembly includes at least two cameras, and the processor performs stereo matching on two two-dimensional environment images captured by the two cameras at the same time to generate a corresponding three-dimensional environment image.

[0057] In some embodiments, the processor converts the set of pixels into a corresponding set of voxels based on camera parameters of the camera assembly.

[0058] In some embodiments, the dimension is the length of a two-dimensional diagonal of a bounding box of a set of pixels, or the length of a three-dimensional diagonal of a bounding box of a set of voxels.

[0059] In some embodiments, the size is one of the length, width, and height of a bounding box of a set of pixels or a set of voxels.

[0060] In some embodiments, the size is the maximum value of the distance between two pixels or two voxels in a pixel set or a voxel set.

[0061] In some embodiments, the first threshold is less than or equal to 5 cm.

[0062] A lawn mowing robot comprises: a driving component, comprising a blade and a driving motor for driving the blade; a walking component, comprising a walking wheel and a walking motor for driving the walking wheel; a camera component, configured to collect an environmental image around the lawn mowing robot, the environmental image being a two-dimensional image or a three-dimensional image; a processor, communicatively connected or electrically connected to the camera component, and configured to control the action of the lawn mowing robot at least according to the environmental image; wherein the processor is further configured to: cluster pixel points or voxel points in the environmental image to obtain a pixel set or voxel set corresponding to each object contained in the environmental image; when a set parameter of the pixel set or voxel set is less than or equal to a parameter threshold, remove the pixel set or voxel set; when a set parameter of the pixel set or voxel set is greater than or equal to the parameter threshold, identify the type of object corresponding to the pixel set or voxel set, and control the walking motor according to the type of the object.

[0063] In some embodiments, the set parameter is the number of pixels or the number of voxels.

[0064] In some embodiments, the parameter threshold corresponding to the number of voxel points is 15.

[0065] A lawn mowing robot comprises: a driving component, comprising a blade and a driving motor for driving the blade; a walking component, comprising walking wheels and a walking motor for driving the walking wheels; a sensor component, configured to collect environmental information around the lawn mowing robot; a first processor, configured to identify obstacles based on the environmental information and determine a coordinate set of the obstacles; a second processor, configured to control the action of the lawn mowing robot; wherein the first processor is further configured to determine representative coordinates of the obstacle based on the coordinate set of the obstacle and transmit the representative coordinates to the second processor; and the second processor is further configured to control the walking motor based on the representative coordinates of the obstacle.

[0066] In some embodiments, the sensor assembly includes a camera assembly configured to collect images of the environment around the lawn mowing robot.

[0067] In some embodiments, the first processor is a GPU and the second processor is a CPU.

[0068] In some embodiments, the first processor is further configured to identify the type of obstacles in the environment image, and transmit the type of obstacles and representative coordinates of the obstacles to the second processor.

[0069] In some embodiments, the representative coordinates are the coordinates of the vertices of the three-dimensional diagonal of the bounding box of the obstacle.

[0070] In some embodiments, the representative coordinates are the coordinates of the vertices of the plane diagonal of the obstacle's bounding box.

[0071] In some embodiments, the sensor assembly includes a binocular camera and / or a laser radar (LiDAR).

[0072] In some embodiments, the amount of data representing the coordinates is less than or equal to 100 bytes.

[0073] A control method for a lawn mowing robot comprises: a sensor component collects environmental information around the lawn mowing robot; a processor identifies obstacles according to the environmental information and determines whether the obstacles are static or dynamic; when the obstacles are determined to be dynamic, the processor uses a first map for navigation; when the obstacles are determined to be static, the processor uses a second map for navigation.

[0074] A control method for a lawn mowing robot comprises: a sensor component collects environmental information around the lawn mowing robot; a processor constructs a local map around the real-time position of the lawn mowing robot according to the environmental information, wherein the local map has a higher resolution than a global map, and the global map is stored in a memory.

[0075] A control method for a lawn mowing robot comprises: a sensor component collects environmental information around the lawn mowing robot; a processor determines a road condition based on the environmental information, and records the road condition on a corresponding map grid in a map, wherein the road condition belongs to at least one of a plurality of preset road condition types, and the map is stored in a memory and divided into a plurality of map grids; and the processor also updates the road condition on the corresponding map grid in the map based on the environmental information newly collected by the sensor component.

[0076] A control method for a lawn mowing robot, comprising: a sensor component collects environmental information around the lawn mowing robot; a processor maps map information in a global map to an initial local map around the real-time position according to the real-time position of the lawn mowing robot, and the global map is stored in a memory; the processor also uses the environmental information collected by the sensor component to update the initial local map to generate a local map around the real-time position, and controls the walking motor according to the global map and the local map.

[0077] A control method for a lawn mower robot comprises: a camera component collects a two-dimensional image around the lawn mower robot; a processor identifies a bad road condition in the two-dimensional image through semantic segmentation, wherein the bad road condition belongs to one of a plurality of preset bad road condition types; the processor also verifies the bad road condition according to a three-dimensional point cloud to obtain a verification result, and plans a travel route of the lawn mower robot according to the verification result.

[0078] A control method for a lawn mower robot comprises: a first camera in a camera assembly collects a first environment image around the lawn mower robot, and a second camera in the camera assembly collects a second environment image around the lawn mower robot, wherein the first environment image and the second environment image are two-dimensional images; a processor recognizes objects contained in the first environment image and the second environment image respectively through semantic segmentation, and associates repeated objects in the first environment image and the second environment image; the processor also performs stereo matching on the repeated objects in the first environment image and the second environment image to obtain depth information of the repeated objects, and controls the travel motor according to the recognition result of the repeated objects and the depth information.

[0079] A control method for a lawn mowing robot comprises: a camera component collects an environmental image around the lawn mowing robot, wherein the environmental image is a two-dimensional image or a three-dimensional image; a processor clusters pixel points or voxel points in the environmental image to obtain a pixel set or a voxel set corresponding to each object contained in the environmental image; when a set parameter of the pixel set or the voxel set is less than or equal to a parameter threshold, the processor removes the pixel set or the voxel set; when a set parameter of the pixel set or the voxel set is greater than or equal to the parameter threshold, the processor identifies the type of object corresponding to the pixel set or the voxel set, and controls the travel motor according to the type of the object.

[0080] A control method for a lawn mowing robot comprises: a sensor component collects environmental information around the lawn mowing robot; a first processor identifies obstacles based on the environmental information and determines a coordinate set of the obstacles; the first processor also determines representative coordinates of the obstacles based on the coordinate set of the obstacles and transmits the representative coordinates to a second processor; the second processor controls the travel motor based on the representative coordinates of the obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 is a perspective view of a lawn mowing robot as an embodiment;

[0082] Figure 2 yes Figure 1 A three-dimensional diagram of part of the internal structure of the lawn mowing robot;

[0083] Figure 3 yes Figure 1 The electrical control schematic diagram of the sensor assembly and processor in the lawn mowing robot shown;

[0084] Figure 4 is a schematic diagram of a working area of ​​a lawn mowing robot as an embodiment;

[0085] Figure 5 is a schematic diagram of a first map of a lawn mowing robot as an embodiment;

[0086] Figure 6 is a schematic diagram of a second map of a lawn mowing robot as an embodiment;

[0087] Figure 7 is a schematic diagram of a local map of a lawn mowing robot as an embodiment;

[0088] Figure 8 yes Figure 1 The electrical control schematic diagram of the sensor components, memory and processor in the lawn mowing robot shown;

[0089] Fig. 9 yes Figure 1 The schematic diagram of the electrical control of the sensor assembly, the first processor and the second processor in the lawn mowing robot is shown. DETAILED DESCRIPTION

[0090] Before any embodiments of the application are explained in detail, it is to be understood that the application is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the above drawings.

[0091] In this application, the terms "comprises", "includes", "has" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0092] In this application, the term "and / or" is a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the related objects before and after are in an "and / or" relationship.

[0093] In the present application, the terms "connect", "combine", "couple", and "install" may refer to direct connection, combination, coupling, or installation, or indirect connection, combination, coupling, or installation. For example, direct connection refers to two parts or components being connected together without the need for an intermediate piece, and indirect connection refers to two parts or components being connected to at least one intermediate piece respectively, and the two parts or components being connected via the intermediate piece. In addition, "connect" and "couple" are not limited to physical or mechanical connections or couplings, and may include electrical connections or couplings.

[0094] In the present application, it will be understood by those of ordinary skill in the art that relative terms (e.g., "about," "approximately," "substantially," etc.) used in conjunction with quantities or conditions include the values ​​and have the meaning indicated by the context. For example, the relative terms include at least the degree of error associated with the measurement of a specific value, the tolerances caused by manufacturing, assembly, and use associated with a specific value, and the like. Such terms should also be considered to disclose a range defined by the absolute values ​​of the two endpoints. Relative terms may refer to plus or minus a certain percentage (e.g., 1%, 5%, 10% or more) of the indicated value. Numerical values ​​that do not use relative terms should also be disclosed as specific values ​​with tolerances. In addition, "substantially" may refer to plus or minus a certain degree (e.g., 1 degree, 5 degrees, 10 degrees or more) on the basis of the indicated angle when expressing a relative angular position relationship (e.g., substantially parallel, substantially perpendicular).

[0095] In this application, it will be understood by those skilled in the art that the function performed by a component can be performed by one component, multiple components, one part, or multiple parts. Similarly, the function performed by a part can also be performed by one part, one component, or a combination of multiple parts.

[0096] In the present application, the terms "upper", "lower", "left", "right", "front", "back" and other directional words are described based on the orientation and positional relationship shown in the accompanying drawings, and should not be understood as limiting the embodiments of the present application. In addition, in the context, it is also necessary to understand that when it is mentioned that an element is connected to another element "upper" or "lower", it can not only be directly connected to another element "upper" or "lower", but also indirectly connected to another element "upper" or "lower" through an intermediate element. It should also be understood that directional words such as upper side, lower side, left side, right side, front side, back side, etc. not only represent the positive orientation, but can also be understood as the lateral orientation. For example, the bottom can include directly below, lower left, lower right, lower front, and lower back, etc.

[0097] In this application, the terms "controller", "processor", "central processing unit", "CPU", and "MCU" are interchangeable. When a unit "controller", "processor", "central processing unit", "CPU", or "MCU" is used to perform a specific function, unless otherwise specified, these functions can be performed by a single unit or multiple units.

[0098] In the present application, the terms “device”, “module” or “unit” may be implemented in the form of hardware or software to achieve specific functions.

[0099] In this application, the terms "compute", "determine", "control", "determine", "identify", etc. refer to the operations and processes of a computer system or similar electronic computing device (e.g., controller, processor, etc.).

[0100] As a new outdoor walking power tool, the lawn mower robot is increasingly favored by users due to its intelligent and efficient advantages. In the target area to be mowed, the traditional lawn mower needs the user to push it by hand, while the lawn mower robot can autonomously plan the path and mow on the planned path. In addition, the lawn mower robot can also realize functions such as object recognition and obstacle avoidance during the moving process.

[0101] The technical solution of the present application is described in detail below with reference to the accompanying drawings and specific implementation methods.

[0102] Figure 1 The lawn mowing robot 100 is shown as an embodiment of the present application. It is understandable that the technical solution described in the present application can also be selectively applied to other outdoor walking power tools such as snow sweeping robots to meet the needs of different scenarios.

[0103] refer to Figure 1 to Figure 2 The lawn mowing robot 100 includes a driving component 10 and a walking component 20. The driving component 10 includes a blade 11 and a driving motor 12 for driving the blade. After the motor shaft of the driving motor rotates, it directly or indirectly transmits power to the first output shaft through the first transmission component to drive the blade to perform a mowing operation. The walking component 20 includes a walking wheel 21 and a walking motor 22 for driving the walking wheel. After the motor shaft of the walking motor rotates, it directly or indirectly transmits power to the second output shaft through the second transmission component to drive the walking wheel to walk on the planned travel path. In some examples, the walking motor can be a hub motor, and the hub motor can integrate power, transmission and braking related devices into the hub.

[0104] refer to Figure 3 The lawn mowing robot 100 further includes a sensor assembly 30 and a processor 40. The sensor assembly 30 may include one or more of a visual sensor, a collision sensor, an inertial sensor, an electromagnetic sensor, a laser radar, and an odometer, and the specific number of each type of sensor included in the sensor assembly 30 is not limited. It is understandable that the sensor assembly 30 may also include other types of sensors not listed above.

[0105] The sensor component 30 is configured to collect environmental information around the lawn mowing robot 100. In some examples, the sensor component 30 can collect environmental information within a preset distance range centered on the current real-time position of the lawn mowing robot 100. The environmental information can be presented in the form of a one-dimensional signal, a two-dimensional image, or a three-dimensional point cloud.

[0106] In some examples, the sensor assembly 30 may include a visual sensor such as a camera assembly, an infrared sensor, a linear light sensor, etc., which can collect an environmental image of the lawn mower robot 100 in the current real-time position in the current posture within the sensor's line of sight. In other examples, the sensor assembly 30 may include a laser radar (LiDAR), which can collect a laser point cloud around the lawn mower robot 100 within the sensor's radiation range. It should be noted that in some cases, the sensor assembly 30 such as a binocular camera can also transmit the collected two-dimensional image to the processor 40 first, and then the processor 40 generates a three-dimensional image from the two-dimensional image. In some other examples, the sensor assembly 30 may include an inertial sensor, which can detect data such as the three-axis acceleration of the lawn mower robot 100.

[0107] The processor 40 may be a CPU (Central Processing Unit) or an MCU (Microcontroller Unit), which controls the actions of the lawn mower robot 100 by reading and running various executable instructions.

[0108] refer to Figure 4 , Figure 4 The working area of ​​the mowing robot 100 is shown. In order to limit the mowing robot 100 to move only in the working area where the mowing operation is to be performed, its working area is generally provided with a boundary line. In some examples, the boundary line may be a physical boundary line, and in other examples, the boundary line may also be a virtual boundary line implemented in combination with software. Whether it is a physical boundary line or a virtual boundary line, they will enclose the working area of ​​the mowing robot 100, and the specific shape of the working area is not limited. For example, the working area may be a rectangular area or an area of ​​other irregular shapes.

[0109] Since the lawn mower robot 100 generally performs mowing operations with the goal of covering the working area, in order to achieve effective path planning, the lawn mower robot 100 also needs to build a map corresponding to its working area. In some simple examples, the lawn mower robot 100 can always use the same pre-stored map for navigation, but the information provided by these maps is limited and has poor timeliness, which is not conducive to path planning and obstacle avoidance of the lawn mower robot 100.

[0110] In one embodiment, the lawn mower robot 100 includes at least a driving component 10, a walking component 20, a sensor component 30, and a processor 40. The sensor component 30 collects environmental information near the real-time position of the lawn mower robot 100, and the processor 40 is electrically connected or communicatively connected to the sensor component 30, and can obtain the environmental information collected by the sensor component 30. In some examples, the processor 40 can periodically receive environmental information corresponding to the real-time position of the lawn mower robot 100 from the sensor component 30. For example, the processor 40 can periodically receive environmental information collected by the sensor component 30 with a period of 100ms, then the processor 40 can interact with the sensor component 30 for 10 times within 1s, and the environmental information of the 10 interactions can all be environmental information when the lawn mower robot 100 is in the same position, or can be environmental information when the lawn mower robot 100 is in different positions.

[0111] After obtaining the environmental information corresponding to the real-time position of the lawn mower robot 100, the processor 40 can identify obstacles near the real-time position of the lawn mower robot 100 based on the environmental information, and determine whether the obstacles are dynamic or static. Dynamic obstacles can be people, animals, or other electronic devices traveling near the lawn mower robot 100, while static obstacles can be obstacles that do not move, such as stone steps, trees, benches, etc.

[0112] If the processor 40 determines that the obstacle around the lawn mower robot 100 is a dynamic obstacle, the processor 40 may call the first map to navigate the lawn mower robot 100. Specifically, the processor 40 may call the first map, plan a first travel path under the first map, and control the walking motor 22 based on the first travel path so that the lawn mower robot 100 walks on the first travel path. In some cases, the processor 40 also controls the driving motor 12 based on the first map so that the lawn mower robot 100 adjusts the blade cutting state on the first travel path.

[0113] If the processor 40 determines that the obstacle around the mowing robot 100 is a static obstacle, the second map may be called to navigate the mowing robot 100. Specifically, the processor 40 may call the second map, plan a second travel path under the second map, and control the travel motor 22 based on the second travel path so that the mowing robot 100 walks on the second travel path. In some cases, the processor 40 also controls the drive motor 12 based on the second map so that the mowing robot 100 adjusts the blade cutting state on the second path.

[0114] In some embodiments, the processor 40 may also be provided with a map pointer, and the processor 40 may use the map pointer to call a corresponding map when it is necessary to switch maps.

[0115] like Figure 4As shown, the working area of ​​the lawn mower robot 100 may include multiple objects of different types, which may be dynamic obstacles, static obstacles, backgrounds such as air and ground, or specific road conditions such as potholes and bumps. The processor 40 may use different algorithms to identify the types of objects in the working area according to actual scene requirements, or may identify objects of one or more specific types in the working area.

[0116] In some embodiments, the sensor assembly 30 includes a camera assembly, which can collect environmental images around the lawn mowing robot 100. The processor 40 can use a semantic segmentation algorithm to identify obstacles appearing in the environmental image and determine the type of obstacle. When the type of obstacle is a person, an animal, or other moving electronic device, it can be determined to be dynamic. In some examples, the dynamic and static obstacles in the three-dimensional environmental image can also be identified in combination with a geometric analysis method. In other embodiments, the processor 40 can also use a streamer method or a target tracking method for multiple frames to detect whether the environmental image contains dynamic objects.

[0117] refer to Figure 5 In some embodiments, the first map is a global map, which is a map of the entire working area where the mowing robot 100 is to perform mowing operations. When dynamic obstacles such as people and animals appear around the mowing robot 100, the area that may be affected by the dynamic obstacles is wider because the dynamic obstacles have a large range of movement. In the case where the mowing robot 100 performs mowing operations in a bow-shaped path, the processor 40 can call the global map to re-plan the global travel path under the global map.

[0118] refer to Figure 6 In some embodiments, the second map is a local map, which is a map of a portion of the working area within a certain distance around the real-time position of the mowing robot 100. In some examples, the local map may be a map of a portion of the working area with the real-time position of the mowing robot 100 as the coordinate origin and the direction of travel of the mowing robot 100 as the coordinate axis. In other examples, the local map may also be a map of a portion of the working area with the real-time position of the mowing robot 100 as the center and the distance from the center within a preset distance threshold. When static obstacles such as trees and stone steps appear around the mowing robot 100, since the positions of the static obstacles are relatively fixed, the processor 40 may call the local map to re-plan the local travel path under the local map.

[0119] In addition, calling the global map or calling the local map for navigation can also combine other attributes of dynamic obstacles or static obstacles when planning the path. For example, for obstacles such as trees and flower beds, when calling the local map navigation, the processor 40 can control the walking motor 22 to make the mowing robot 100 perform a circular mowing task around the tree or flower bed. For obstacles such as stone steps and boundary lines, when calling the local map navigation, the processor 40 can also control the walking motor 22 to make the mowing robot 100 perform a side mowing task along the stone steps or boundary lines.

[0120] In some embodiments, the second map has a higher resolution than the first map. In some examples, the resolution of the local map can be greater than or equal to 1 cm and less than or equal to 20 cm, while the resolution of the global map can be greater than or equal to 15 cm and less than or equal to 50 cm. In other examples, the resolution of the local map can be greater than or equal to 1 cm and less than or equal to 15 cm, while the resolution of the global map can be greater than or equal to 15 cm and less than or equal to 30 cm. It is understood that the resolution of the first map and the second map should be adapted to the size of the current working area of ​​the mowing robot 100.

[0121] In some embodiments, the second map contains more map attributes than the first map. In some examples, the environmental information around the real-time position of the mowing robot 100 collected by the sensor assembly 30 can be recorded in the local map in its entirety and in part in the global map. For example, the three-axis acceleration data detected by the inertial sensor and the mileage data collected by the odometer can be recorded in the local map, but not in the global map.

[0122] In some embodiments, the second map is updated in real time during the movement of the mowing robot 100. For example, assuming that the mowing robot 100 moves at a speed of 0.5 m / s, the second map can be periodically updated with a period of 100 ms, that is, the local map of the mowing robot 100 is updated once every 5 cm.

[0123] In some embodiments, the processor 40 controls the travel motor 22 to drive the travel wheels at a first speed when navigating using the first map, and controls the travel motor 22 to drive the travel wheels at a second speed when navigating using the second map. In some examples, the first speed may be less than the second speed. For example, the travel speed of the mowing robot 100 when navigating using the global map may be lower than the travel speed of the mowing robot 100 when navigating using the local map, so as to ensure the safety of the mowing robot 100 when facing dynamic obstacles. In other examples, the travel speed of the mowing robot 100 may be further adjusted in combination with other map attributes.

[0124] In some embodiments, the sensor assembly 30 includes one or more of a binocular camera, a laser radar (LiDAR), and an infrared sensor.

[0125] In the lawn mowing robot described in the above embodiment, different maps are selected for navigation based on whether the obstacles appearing near the real-time position of the lawn mowing robot are dynamic or static. The map used can adapt to the different characteristics of dynamic and static obstacles and the requirements of path planning and obstacle avoidance in scenarios where dynamic and static obstacles appear, thereby improving the work efficiency and results of the lawn mowing robot.

[0126] In one embodiment, the lawn mower robot 100 includes at least a driving component 10, a walking component 20, a sensor component 30, a memory 50, and a processor 40. The memory 50 stores a global map of the working area of ​​the lawn mower robot 100. In some examples, the memory 50 may be a ROM (Read-Only Memory), a RAM (Random Access Memory) or a Flash memory. In other examples, the memory 50 may also be a remote memory. The global map is a map of the entire working area where the lawn mower robot 100 is to perform mowing operations. It can record relatively stable information in the entire working area in a relatively concise manner.

[0127] refer to Figure 8 The processor 40 is electrically connected or communicatively connected to the sensor assembly 30 and the memory 50, and can obtain the global map stored in the memory 50, and obtain various environmental information collected by the sensor assembly 30. In some examples, the processor 40 can periodically receive various environmental information collected by the sensor assembly 30, wherein the periods for different sensors to send environmental information can be the same or different.

[0128] After obtaining various environmental information, the processor 40 can construct a local map around the real-time position of the mowing robot 100 based on the various environmental information. The resolution of the local map is higher than that of the global map, and it can record more volatile information in the partial working area corresponding to the real-time position of the mowing robot 100 in a more detailed manner. Specifically, the processor 40 can identify obstacles based on the environmental image from the visual sensor and construct a local map containing the location and type information of the obstacles, or construct a local map containing the depth information of each object based on the laser point cloud from the laser radar (LiDAR), and can also construct a local map containing the speed and direction of the mowing robot 100 based on the three-axis acceleration data from the inertial sensor.

[0129] In some embodiments, the shape of the local map may be fixed, that is, the shape of the local map is a preset shape. In some examples, the shape of the local map may be a regular quadrilateral, and in other examples, the shape of the local map may also be a circle.

[0130] In some embodiments, the local map can be constructed with the real-time position of the mowing robot 100 as the coordinate origin and the direction of travel of the mowing robot 100 as the coordinate axis. In some examples, the local map can be a 4m*4m regular quadrilateral map with the real-time position of the mowing robot 100 as the origin, or a 3m*3m regular quadrilateral map, a 5m*5m regular quadrilateral map, or a 6m*6m regular quadrilateral map. The side length of the local map can be adaptively adjusted according to the size of the working area of ​​the mowing robot and the travel speed of the mowing robot. For example, the larger the working area and the faster the travel speed of the mowing robot, the larger the side length of the local map, and vice versa. In other embodiments, the local map can be a map of the working area with the real-time position of the mowing robot 100 as the center and the distance from the center within a preset distance threshold. For example, the boundary of the local map can be a circle with the real-time position of the mowing robot 100 as the center and the preset distance threshold as the radius. In some examples, the preset distance threshold can be 4m, or 3m, 5m or 6m. It is understandable that, in the case where a boundary line of the working area is set, the coverage area of ​​the local map may not exceed the boundary line, that is, the local map corresponding to the area beyond the boundary line may not be constructed.

[0131] like Figure 5 , Figure 6 As shown, the global map or the local map is composed of multiple map grids, and the map resolution represents the size of the map grid. In some cases, the map grid may be a regular quadrilateral, and the map resolution may be expressed by the grid side length of the map grid. The resolution of the map grid described later is the grid side length of the map grid. In this embodiment, the local map has a higher resolution than the global map. In some examples, the resolution of the local map may be 5 cm, and the resolution of the global map may be 15 cm.

[0132] In some embodiments, the resolutions of multiple map grids in a local map may be different.

[0133] refer to Figure 7 In some embodiments, the resolution of multiple map grids in the local map can vary with the distance between the map grids and the real-time position of the lawn mower robot 100. The closer the map grids are to the real-time position of the lawn mower robot 100, the higher the resolution of the map grids. That is, the closer the map grids are to the real-time position of the lawn mower robot 100 in the local map, the smaller the grid side lengths are, and the farther the map grids are from the real-time position of the lawn mower robot 100, the larger the grid side lengths are.

[0134] In some other embodiments, in addition to changing with the distance between the map grid and the real-time position of the lawn mower robot 100, a resolution threshold may also be set. After the resolution of the map grid reaches the resolution threshold, that is, after the grid side length reaches the maximum grid side length, the map resolution will no longer be reduced even if the distance between the map grid and the real-time position of the lawn mower robot 100 is farther.

[0135] The resolution of the local map can be adjusted according to the size of the local map, the size of the working area of ​​the mowing robot 100, the working accuracy requirements, etc. In some embodiments, the maximum resolution of the map grid in the local map can be 1 cm, and the minimum resolution can be 15 cm.

[0136] In some embodiments, the local map contains more types of map attributes than the global map. In some examples, the global map may not record information about dynamic obstacles, while the local map will record information about dynamic obstacles. In other examples, the global map may not record mileage data and travel speed, while the local map will record mileage data and travel speed.

[0137] In some embodiments, during the movement of the mowing robot 100, as its real-time position changes, the processor 40 will also update the local map corresponding to the current real-time position in real time. For example, assuming that the moving speed of the mowing robot 100 is 0.5 m / s, the processor 40 can periodically update the local map with a period of 100 ms. Compared with the local map generated in the previous period, the real-time position of the mowing robot 100 in the current local map has moved 5 cm.

[0138] In some embodiments, the processor 40 can use a local map generated in real time corresponding to the real-time position of the lawn mower robot 100 and / or a stored global map to plan the travel path of the lawn mower robot 100, and control the travel motor 22 based on the planned travel path. In some examples, the processor 40 can also control the drive motor 12 based on the local map and / or the global map.

[0139] In some embodiments, the sensor assembly 30 includes at least one of a visual sensor, a collision sensor, an inertial sensor, a laser radar (LiDAR), and an odometer.

[0140] In addition, in some embodiments, when the processor 40 generates a local map corresponding to the real-time position of the mowing robot 100, it can first read the global map stored in the memory 50, map the map information in the global map to the local map to be generated to obtain an initial local map, and then update the initial local map based on the environmental information collected by the sensor to obtain the local map used for subsequent navigation. Specifically, the processor 40 can determine the map area corresponding to the local map to be generated based on the real-time position of the mowing robot 100, read the global map and map the map information in the map area in the global map to the local map to be generated to generate the initial local map, that is, the initial local map is a cutout of the map area in the global map, the processor 40 obtains the environmental information near the real-time position of the mowing robot 100 collected by the sensor assembly 30, and determines the map information not contained in the global map such as dynamic and static obstacles, bumps, potholes, etc. based on the environmental information analysis, uses them to update the initial local map, and subsequently uses the updated local map to realize the navigation of the mowing robot 100.

[0141] In the lawn mowing robot described in the above embodiment, the processor can generate a local map corresponding to the real-time position of the lawn mowing robot. The local map has a higher resolution than the global map and may also contain more map attributes that can reflect the real-time moving conditions, thereby providing more precise guidance for the navigation and other action controls of the lawn mowing robot.

[0142] In one embodiment, the lawn mower robot 100 includes at least a driving component 10, a walking component 20, a sensor component 30, a memory 50, and a processor 40. The memory 50 stores a map of the working area of ​​the lawn mower robot 100, which may be an overall map or a partial map of the working area of ​​the lawn mower robot 100.

[0143] The processor 40 is electrically connected or communicatively connected to the sensor assembly 30, and can obtain various environmental information collected by the sensor assembly 30. After obtaining various environmental information, the processor 40 can determine the road conditions based on the various environmental information. Specifically, the processor 40 can obtain environmental information such as environmental images, mileage data, acceleration data, etc. near different positions in the working area of ​​the mowing robot 100 collected by the sensor assembly 30. By analyzing and processing these environmental information, the obstacle conditions, bump conditions, collision conditions, etc. near different positions in the working area can be determined, so that the road conditions near different positions in the working area can be determined. The processor 40 can be pre-set with multiple road condition types to determine the road conditions near each position, that is, to determine the road condition type to which the road conditions near the position belong. In some examples, the processor 40 can determine the road conditions near the real-time position of the mowing robot 100 based on the environmental information near the real-time position collected by the sensor assembly 30. In other examples, the processor 40 can also determine the road conditions near multiple different positions based on the environmental information near the mowing robot 100 collected by the sensor assembly 30.

[0144] In some embodiments, the preset multiple road condition types include, but are not limited to: at least two of: a bumpy area, a pit area, a ramp area, a non-grass area, an obstacle-prone area, a slippery area, and an easily trapped area.

[0145] In some embodiments, the processor 40 can determine whether the current position is near a bumpy area based on the three-axis acceleration data collected by the inertial sensor. In other embodiments, the processor 40 can determine whether the current position is near a tunnel area or a ramp area based on the image collected by the visual sensor and the three-axis acceleration collected by the inertial sensor. In some other embodiments, the processor 40 can determine whether the current position is near an obstacle-prone area based on the environmental image collected by the visual sensor and the overload current of the code disk. In some other embodiments, the processor 40 can detect slipping areas and easy-to-get-stuck areas based on odometer positioning.

[0146] In some embodiments, the processor 40 can control the travel motor 22 to control the travel path and travel speed based on the road conditions. In some examples, the processor 40 can also control the drive motor 12 to adjust the blade cutting state based on the road conditions.

[0147] The processor 40 is also electrically connected or communicatively connected to the memory 50 , and can exchange map information with the memory 50 .

[0148] like Figures 5 to 7 As shown, the map is composed of multiple map grids. Taking the global map as an example, each map grid may correspond to a different part of the working area, and each map grid records map information such as the road conditions on the part of the working area corresponding to the map grid.

[0149] After determining the road condition near a location in the working area, the processor 40 can interact with the memory 50 to record the determined road condition on a map grid corresponding to the location in the map. When the processor reaches the area corresponding to the map grid again at other subsequent times, the road condition update module in the processor 40 can redetermine the road condition based on the newly collected environmental information of the sensor component 30, and interact with the memory 50 to update the map information on the map grid in the map with the redetermined road condition.

[0150] In some embodiments, the mowing robot 100 further includes a human-computer interaction component, which allows a user to update the road conditions on one or more map grids using human-computer interaction means. Specifically, map information such as road conditions on one or more map grids in the map in the memory 50 can be modified through the data interface of the mowing robot 100.

[0151] In some embodiments, in addition to determining and recording the road conditions on the map grid, the processor 40 also determines and records the probability of different road conditions appearing on the map grid. Specifically, after the processor 40 obtains the road conditions on a map grid based on the analysis of various environmental information collected by the sensor, it can determine the road condition probability of the road condition appearing on the map grid based on the cumulative number of times the road condition appears on the map grid or the frequency of the road condition appearing on the map grid within a preset time period, and record it in the map. For example, the road condition probability of a frequently occurring road condition is high, and the road condition probability of an occasional road condition is low. Similarly, when the corresponding area of ​​the map grid is reached again at other subsequent times, the road condition update module in the processor 40 can redetermine the road condition probability based on the newly collected environmental information of the sensor assembly 30, and interact with the memory 50 to update the map information on the map grid in the map with the redetermined road condition probability. In some other embodiments, the processor 40 may also determine the road condition probabilities of different road conditions on each map grid in combination with the time attribute and the spatial attribute of the environmental information collected by the sensor assembly 30. For example, it may be determined that the road condition probability of the road condition on the map grid is high in one period and low in other periods. For another example, when the lawn mower robot 100 determines the same road condition on the map grid multiple times at different times, from different travel directions, or on different travel paths, the road condition probability of the road condition on the map grid may be increased.

[0152] In some embodiments, the processor 40 may filter the environmental information collected by the sensor component 30 before recording or updating the road conditions and road condition probabilities on each map grid, and use the filtered environmental information to record or update the road conditions and road condition probabilities on each map grid. Specifically, the processor 40 may filter the environmental information based on the time attributes and spatial attributes of the environmental information collected by the sensor component 30. For example, the data similarity of environmental information collected at similar locations and similar times is high, and abnormal data may be filtered.

[0153] In some embodiments, after the road conditions and road condition probabilities on each map grid in the map are recorded and updated, the processor 40 can also calculate the cost value of each map grid based on the road conditions and road condition probabilities, and plan the walking path of the mowing robot 100 based on the cost values ​​of each map grid in the map. Specifically, the processor 40 can calculate the cost value of each map grid based on the preset cost weights corresponding to each road condition type, and the road conditions and road condition probabilities on each map grid, and then plan the walking path of the mowing robot 100 based on the cost values ​​of each map grid with the goal of lowering the cost value. For example, the higher the degree of road danger and the higher the road condition probability, the higher the cost value of the map grid, and the map grid with a high cost value can be avoided as much as possible when planning the walking path.

[0154] In some embodiments, in addition to determining and recording the road conditions and road condition probabilities on the map grid, the processor 40 may also determine at least one of the second map attributes such as mowing, lifting, collision, boundary, obstacle, charging pile, etc. near different locations in the working area based on various environmental information collected by the sensor, and record the determined second map attribute on the map grid corresponding to the location in the map.

[0155] In some embodiments, in addition to planning the walking path of the mowing robot 100 based on the cost value of the map grid, the processor 40 can also adjust the planned walking path based on at least one of the second map attributes such as mowing, lifting, collision, boundary, obstacle, charging column, etc. For example, after planning the walking path based on the cost value of each grid, the processor 40 can optimize and adjust the planned walking path based on the second map attribute of the boundary to further ensure that the planned walking path does not exceed the working area. In other embodiments, the second map attribute can also be combined when calculating the cost value of each map grid.

[0156] In the lawn mower robot described in the above embodiment, the processor can generate a map that is annotated with map information such as road conditions, road condition probabilities, and cost values ​​of each map grid and is continuously and dynamically updated, which can provide more accurate guidance for the navigation and other action controls of the lawn mower robot.

[0157] In one embodiment, the lawn mower robot 100 includes at least a driving component 10, a walking component 20, a camera component and a processor 40. The camera component can collect a two-dimensional image around the lawn mower robot 100, that is, an environmental image around the lawn mower robot 100, specifically an RGB image or a grayscale image without limitation.

[0158] The processor 40 is electrically connected or communicatively connected to the camera assembly, and can obtain the two-dimensional image collected by the camera assembly. After obtaining the two-dimensional image, the processor 40 can identify the bad road conditions in the two-dimensional image through semantic segmentation, and verify the identified bad road conditions based on the three-dimensional point cloud to obtain a verification result, and then plan the route of the mowing robot 100 based on the verification result of the bad road condition. Specifically, the processor 40 can obtain the segmented image corresponding to the two-dimensional image and the type of each object therein using the semantic segmentation algorithm. The processor 40 can be pre-set with multiple bad road condition types. When the object in the two-dimensional image belongs to one of the preset multiple bad road condition types, the position and type of a bad road condition in the two-dimensional image are identified, thereby obtaining the position and type of each bad road condition in the two-dimensional image. Then, the processor 40 can use the three-dimensional point cloud to verify each bad road condition, and plan the travel path of the mowing robot 100 based on the verification results of each bad road condition.

[0159] In some embodiments, the preset multiple adverse road condition types include, but are not limited to: at least two of potholes, water areas, and haystacks.

[0160] In some embodiments, the processor 40 can determine the depth information of each bad road condition and the depth information of the walking plane where the mowing robot 100 is located based on the three-dimensional point cloud, and then verify the bad road condition based on the depth information of the bad road condition and the depth information of the walking plane. Specifically, the processor 40 can register the two-dimensional image and the three-dimensional point cloud to obtain the registration relationship between the two-dimensional image and the three-dimensional point cloud. The registration between the two is the calibration between the two-dimensional image coordinate system and the three-dimensional point cloud coordinate system. Based on the registration relationship between the two, the coordinate information of the bad road condition obtained by the identification in the two-dimensional image can be converted to obtain the depth information of the bad road condition, and based on the registration relationship between the two, the coordinate information of the walking plane near the bad road condition in the two-dimensional image can also be converted to obtain the depth information of the walking plane.

[0161] In some embodiments, the coordinate information of each bad road condition may be the coordinate information of the area within the bounding box obtained when the processor 40 uses a semantic segmentation algorithm to identify the bad road condition. In some examples, the processor 40 may use a semantic segmentation algorithm to segment a two-dimensional image into multiple image areas, and identify the type of objects within each image area, wherein the coordinate information of the image area where the objects belonging to the preset bad road condition type are located is the coordinate information of the bad road condition. In other examples, the processor 40 may also use a semantic segmentation algorithm to frame the bad road condition in the identified two-dimensional image in the form of a bounding box, and the coordinate information of the image area within the bounding box is the coordinate information of the bad road condition.

[0162] In some embodiments, the depth information of the walking plane may be the coordinate information of the area outside the boundary box obtained when the processor 40 uses a semantic segmentation algorithm to identify the bad road condition. Specifically, after determining the image area where the bad road condition is located, the processor 40 may take the complement of the image area in the two-dimensional image as the image area where the walking plane is located, and the coordinate information of the image area corresponding to the complement is the coordinate information of the walking plane.

[0163] In some embodiments, the processor 40 can detect whether the depth information of the walking plane where the lawn mower robot 100 is located meets the preset plane depth information, and if it does not meet the preset plane depth information, it is determined that the bad road condition verification has failed. Specifically, the processor 40 can fit the plane based on the depth information, and verify the centroid and / or normal vector of the fitted plane. If the centroid or normal vector does not match the centroid or normal vector of the preset standard ground, for example, if the distance error between the centroid and the standard centroid exceeds a threshold or the angle error between the normal vector and the standard normal vector exceeds a threshold, it is determined that the bad road condition verification has failed.

[0164] In some embodiments, the processor 40 may also verify the bad road condition based on the depth relationship between the depth information of the bad road condition and the depth information of the walking plane, and determine that the bad road condition verification is successful when the depth relationship between the two is consistent with the depth relationship corresponding to the type of the bad road condition. For example, for a bad road condition of pothole type, the depth relationship between it and the walking plane should be that the bad road condition is below the walking plane. If the relationship between the depth information of the two satisfies the above relationship, it can be determined that the pothole verification is successful.

[0165] The three-dimensional point cloud of the processor 40 verifying the bad road condition can be generated by the processor 40 or collected by the sensor assembly 30. In some embodiments, the camera assembly includes at least a first camera and a second camera, and the processor 40 can generate a three-dimensional point cloud based on at least two two-dimensional images collected by the camera assembly. For example, the processor 40 can generate a corresponding three-dimensional point cloud from two two-dimensional images respectively collected by the first camera and the second camera at the same time through a binocular camera model. In other embodiments, the lawn mower robot 100 also includes sensors such as a laser radar (LiDAR), which can collect environmental information around the lawn mower robot 100, generate a corresponding three-dimensional point cloud and transmit it to the processor 40.

[0166] In the lawn mower robot described in the above embodiment, the processor can use the three-dimensional point cloud to verify the adverse road conditions identified in the two-dimensional image, and plan the travel path of the lawn mower robot based on the verification results of the adverse road conditions, thereby reducing the impact of misjudgment of adverse road conditions on path planning, and improving the accuracy of navigation and other action control of the lawn mower robot.

[0167] In one embodiment, the lawn mower robot 100 includes at least a driving component 10, a walking component 20, a camera component and a processor 40. The camera component includes at least a first camera and a second camera, the first camera collects a first environment image around the lawn mower robot 100, and the second camera collects a second environment image around the lawn mower robot 100. The first environment image and the second environment image are two-dimensional images, specifically RGB images or grayscale images without limitation.

[0168] In some embodiments, the first camera is arranged on the left side of the second camera. It is understandable that what is described here is the relative position relationship between the first camera and the second camera, and the first camera may also be arranged on the right side, upper side or lower side of the second camera. In other embodiments, the first camera and the second camera together constitute a binocular camera, and the binocular camera may be a passive binocular camera or an active binocular camera. In some other embodiments, the first camera and / or the second camera is a fisheye camera. In some other embodiments, the camera assembly also includes other types of depth cameras such as a structured light camera or a ToF (Time of Flight) camera.

[0169] The processor 40 is electrically connected or communicatively connected to the camera assembly, and can obtain the first environment image and the second environment image collected by the camera assembly. After obtaining the first environment image and the second environment image, the processor 40 can obtain the position and type of each object in the first environment image and the second environment image through semantic segmentation. Specifically, the processor 40 receives a frame of the first environment image and a frame of the second environment image at the same time. The two frames of environment images can be segmented and identified through semantic segmentation to obtain a first segmented image corresponding to the first environment image and the type of each object therein, and a second segmented image corresponding to the second environment image and the type of each object therein. In some examples, the segmented image obtained by semantic segmentation can divide the original environment image into multiple image areas, and the type of objects in each image area can be identified. It can be understood that in other embodiments, the processor 40 can also combine instance segmentation and panoramic segmentation on the basis of semantic segmentation.

[0170] After identifying each object in the first environmental image and the second environmental image, the processor 40 can associate repeated objects that appear in both the first environmental image and the second environmental image. Specifically, the processor 40 can match objects based on the position and type of each object in the first environmental image and the second environmental image, and associate objects that are in the same or similar positions and belong to the same type.

[0171] The processor 40 can use the coordinate information of each associated repetitive object in the first environment image and the coordinate information in the second environment image to perform stereo matching on the repetitive object to obtain the depth information of each repetitive object, and then control the walking motor 22 based on the recognition result and depth information of each repetitive object. In some cases, the processor 40 also controls the driving motor 12 based on the recognition result and depth information of each repetitive object. Specifically, for a repetitive object, due to the difference in viewing angle between the first camera and the second camera, the processor 40 can obtain the disparity map corresponding to the repetitive object based on the difference between the coordinate information of the repetitive object in the first environment image and the second environment image, and calculate its depth information based on the disparity map of the repetitive object. In some examples, the processor 40 can use the corresponding binocular camera model to calculate the depth information.

[0172] In some embodiments, after associating the repeated objects, the processor 40 may perform stereo matching only on repeated objects of a preset type, so as to further reduce the amount of computation of the processor 40 and the amount of data transmission between devices.

[0173] In the lawn mowing robot described in the above embodiment, the processor does not directly perform stereo matching on the first and second environmental images, but first performs object recognition and repeated object association on the first and second environmental images, and then only performs stereo matching on the repeated objects in the first and second environmental images. The depth information obtained from the stereo matching of repeated objects is subsequently used to control the lawn mowing robot, which can achieve the beneficial effect of reducing the data calculation amount of the processor and the data transmission amount between devices.

[0174] In one embodiment, the lawn mower robot 100 includes at least a driving component 10, a walking component 20, a camera component and a processor 40. The camera component collects an environmental image around the lawn mower robot 100, and the environmental image can be a two-dimensional image or a three-dimensional image, specifically an RGB image or a grayscale image without limitation.

[0175] The processor 40 is electrically connected or communicatively connected to the camera assembly, and can obtain the environmental image collected by the camera assembly. After obtaining the environmental image, the processor 40 can cluster the pixels or voxels in the environmental image to obtain a pixel set or voxel set corresponding to each object in the environmental image. Specifically, a two-dimensional environmental image is composed of multiple pixels, and a three-dimensional environmental image is composed of multiple voxels. A clustering algorithm is used to cluster pixels based on RGB values ​​or grayscale values, or voxels are clustered based on voxel coordinates. Multiple pixel sets can be obtained from the two-dimensional environmental image, or multiple voxel sets can be obtained from the three-dimensional environmental image. Pixels or voxels in the same set are regarded as one class. For example, pixels or voxels in the same set are pixels or voxels that constitute the same object.

[0176] The processor 40 determines whether to eliminate the pixel set or voxel set based on the size of each pixel set or voxel set. If the size of the pixel set or voxel set is less than or equal to the first threshold, the pixel set or voxel set is eliminated; if the size of the pixel set or voxel set is greater than or equal to the second threshold, the type of object corresponding to the pixel set or voxel set is identified, wherein the first threshold is less than or equal to the second threshold. After the elimination and identification are completed, the processor 40 subsequently controls the walking motor 22 based on the type of object corresponding to each pixel set or voxel set that has not been eliminated. In some examples, the driving motor 12 can also be controlled based on the type of object corresponding to the pixel set or voxel set that has not been eliminated.

[0177] In some embodiments, the processor 40 clusters pixels in a two-dimensional environment image or voxel points in a three-dimensional environment image using a K-Means or region growing algorithm. It is understandable that other clustering algorithms not shown may also be used to cluster pixels or voxel points to meet actual scene requirements.

[0178] A pixel set in a two-dimensional environment image may be an image region in the two-dimensional environment image, and a voxel set in a three-dimensional environment image may be an image space in the three-dimensional environment image. In some embodiments, the size of the pixel set may be the length or width of the corresponding image region, and the size of the voxel set may be the length, width or height of the corresponding image space. In other embodiments, the size of the pixel set may be the length of the plane diagonal of the corresponding image region, and the size of the voxel set may be the length of the three-dimensional diagonal of the corresponding image space. In still other embodiments, the size of the pixel set may be the maximum value of the distance between two pixel points in the corresponding image region, and the size of the voxel set may be the maximum value of the distance between two voxel points in the corresponding image space.

[0179] In some embodiments, the first threshold used for comparison with the size of the pixel set or voxel set may be 5 cm. In other embodiments, the first threshold used for comparison with the size of the pixel set or voxel set may also be 3 cm.

[0180] The pixel set in the two-dimensional environment image can reflect limited information. In some embodiments, the processor 40 can also convert each pixel set into a corresponding voxel set based on the camera parameters of the camera assembly. Specifically, the camera parameters of the camera assembly may include camera internal and external parameters, which can realize the mapping conversion between the two-dimensional pixel coordinate system and the three-dimensional voxel coordinate system. In some examples, the first threshold used for comparison with the converted voxel set can be 3 cm.

[0181] In some embodiments, the camera assembly includes at least a first camera and a second camera, and the processor 40 can perform stereo matching on the two-dimensional environmental images respectively captured by the two cameras at the same time to obtain a corresponding three-dimensional environmental image, and subsequently perform voxel point clustering, elimination and identification of voxel sets on the three-dimensional environmental image.

[0182] In addition, in addition to using size as a criterion for eliminating pixel sets or voxel sets, other set parameters can also be used as a criterion for eliminating pixel sets or voxel sets. In some embodiments, after clustering to obtain pixel sets or voxel sets, the processor 40 can also determine whether to eliminate the pixel set or voxel set based on the number of pixels in each pixel set or the number of voxel points in the voxel set. When the number of pixels in the pixel set is less than or equal to the third threshold or the number of voxel points in the voxel set is less than or equal to the fifth threshold, the pixel set or voxel set is eliminated; when the number of pixels in the pixel set is greater than or equal to the fourth threshold or the number of voxel points in the voxel set is greater than or equal to the sixth threshold, the type of object corresponding to the pixel set or voxel set is identified, wherein the third threshold is less than or equal to the fourth threshold, and the fifth threshold is less than or equal to the sixth threshold. After the elimination and identification are completed, the processor 40 subsequently controls the walking motor 22 based on the type of object corresponding to each pixel set or voxel set that has not been eliminated. In some examples, the driving motor 12 can also be controlled based on the type of object corresponding to the pixel set or voxel set that has not been eliminated.

[0183] In some embodiments, the fifth threshold value used for comparison with the number of voxel points in the voxel set may be 15.

[0184] In the lawn mowing robot 100 described in the above embodiment, the processor 40 does not directly perform object recognition on the environmental image, but first clusters the pixel points or voxel points in the environmental image, and eliminates the pixel sets or voxel sets with a size smaller than the threshold, and only performs type recognition on the objects corresponding to the pixel sets or voxel sets with a size larger than the threshold, which can achieve the beneficial effect of reducing the data calculation amount of the processor 40 and the data transmission amount between devices.

[0185] In one embodiment, the lawn mowing robot 100 includes at least a driving component 10, a walking component 20, a sensor component 30, a first processor 41 and a second processor 42. The first processor 41 and the second processor 42 are configured to undertake different processing tasks. Fig. 9 The first processor 41 is electrically connected or communicatively connected to the sensor assembly 30, can receive the environmental information around the lawn mower robot 100 collected by the sensor assembly 30, and can exchange data with the second processor 42. The second processor 42 can exchange data with the first processor 41 and can control the actions of the lawn mower robot 100. In some examples, the first processor 41 and the second processor 42 can be different processors in a physical sense, for example, they can be MCU1 and MCU2 respectively, or different processors in a virtual sense, for example, they can be VCPU1 and VCPU2 respectively.

[0186] After obtaining the environmental information, the first processor 41 can identify and obtain the coordinate set of obstacles around the mowing robot 100 based on the environmental information, and then determine the representative coordinates of the obstacle based on the coordinate set, and transmit the representative coordinates to the second processor 42. Specifically, the first processor 41 can identify the obstacles around the mowing robot 100 based on environmental information such as the environmental image collected by the visual sensor and the laser point cloud collected by the laser radar, and obtain the image area of ​​the obstacle in the two-dimensional image, the image space in the three-dimensional image, or the point cloud space in the three-dimensional point cloud, etc., and the coordinate set of each obstacle can be determined based on these image areas, image spaces, or point cloud spaces, and the representative coordinates used to characterize the position of the obstacle can be further selected from the coordinate set, and then the representative coordinates of each obstacle are transmitted to the second processor 42.

[0187] After receiving the representative coordinates of each obstacle from the first processor 41 , the second processor 42 may control the travel motor 22 based on the representative coordinates of each obstacle. In some examples, the second processor 42 may also control the drive motor 12 based on the representative coordinates.

[0188] In some embodiments, the sensor assembly 30 includes a camera assembly, which can capture environmental images around the lawn mowing robot 100 and transmit the environmental images to the first processor 41 .

[0189] In some embodiments, the sensor assembly 30 includes a binocular camera and / or a laser radar (LiDAR).

[0190] In some embodiments, when the first processor 41 identifies obstacles around the mowing robot 100 based on the two-dimensional environment image, it obtains the bounding box of each obstacle in the two-dimensional environment image, and the coordinates of the two vertices on the plane diagonal of the bounding box can be used as the representative coordinates of the obstacle. In some examples, the first processor 41 can also convert the vertex coordinates of the plane diagonal into the vertex coordinates of the three-dimensional diagonal based on the camera internal and external parameters of the camera assembly and transmit them to the second processor 42.

[0191] In other embodiments, when the first processor 41 identifies obstacles around the lawn mowing robot 100 based on the three-dimensional environmental image or the three-dimensional point cloud, it obtains the bounding box of each obstacle in the three-dimensional environmental image or the three-dimensional point cloud, and the coordinates of the two vertices on the three-dimensional diagonal of the bounding box can be used as the representative coordinates of the obstacle.

[0192] In some embodiments, the first processor 41 obtains the type of each obstacle when identifying obstacles based on the environment image, and may transmit the type of each obstacle when transmitting the representative coordinates of each obstacle to the second processor 42 .

[0193] In some embodiments, the first processor 41 may be a GPU (Graphics Processing Unit), and the second processor 42 may be a CPU. It is understandable that, in some cases, there are actually two system-on-chips (SoC) in the lawn mower robot, one of which has a CPU and a GPU, and the other has a CPU. The chip containing the GPU performs image processing and other computing tasks, and the processed data is exchanged between the two chips.

[0194] In some embodiments, the first processor 41 may be a NPU (Neural-Network Processing Unit), and the second processor 42 may be a CPU.

[0195] In some embodiments, the amount of data representing the coordinates of the obstacle exchanged between the first processor 41 and the second processor 42 is less than or equal to 100 bytes.

[0196] In the lawn mowing robot described in the above embodiment, when the first processor exchanges obstacle position information with the second processor, it does not directly use the complete obstacle coordinate set, but uses its representative coordinates, which can effectively reduce the amount of interactive data between the first processor and the second processor and alleviate the data transmission and processing burden of the processor.

[0197] It should be noted that the above-mentioned embodiments are not separated and independent of each other. New embodiments obtained by combining all or part of the contents of multiple embodiments according to actual scenario requirements should also be within the scope of protection of this application.

[0198] For example, the processor 40 can identify the type of each object in the two-dimensional environmental image captured by the camera component. After identifying the object belonging to the bad road condition type, the three-dimensional point cloud is used to verify the bad road condition. If the verification is successful, the local map is called for path planning and obstacle avoidance to improve the accuracy of object recognition, path planning and obstacle avoidance of the lawn mowing robot.

[0199] For example, the processor 40 can update the map information such as real-time obstacles, travel speed, mileage data, etc. in the local map corresponding to the real-time position of the mowing robot 100 based on various environmental information collected by the sensor assembly 30, and update the map information such as road conditions, road condition probabilities and cost values ​​on each map grid in the global map corresponding to the real-time position of the mowing robot 100, so as to improve the map generation capability of the mowing robot and the performance of the generated map.

[0200] For another example, the first processor 41 may cluster the pixel points in the environmental image captured by the first and second cameras in the camera assembly to obtain a pixel set corresponding to each object in the environmental image, and eliminate and identify the pixel set based on the size of the pixel set; then, the first processor 41 may associate the repeated objects in the first and second environmental images after eliminating the pixel sets with a size smaller than a threshold, and perform stereo matching on the associated repeated objects to obtain depth information of the repeated objects; finally, the first processor 41 may determine the representative coordinates of the obstacle based on the depth information of the repeated objects belonging to the obstacle, and transmit the representative coordinates of each obstacle to the second processor 42; the second processor 42 controls the travel motor 22 based on the representative coordinates of the obstacle to reduce the data calculation amount of the processor and the data transmission amount between devices.

[0201] It is understandable that the embodiments described above can also be applied to outdoor work vehicles independently or in combination. The outdoor work vehicle in this application can be a device that can plan and decide its own itinerary, such as the lawn mower robot described above, for example, it can also be a snow sweeper robot, etc. The outdoor work vehicle can also be other devices with autonomous walking as a reference or alternative function, such as riding lawn mowers, standing lawn mowers and other manned lawn mowers and cleaning machines, multi-purpose all-terrain vehicles UTV (Utility Task Vehicle) / farmer vehicles, all-terrain vehicles ATV (All Terrain Vehicle) / beach cars, golf carts, etc.

[0202] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the above embodiments do not limit the present application in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present application.

Claims

1. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A camera assembly is configured to collect an environmental image around the lawn mowing robot, wherein the environmental image is a two-dimensional image or a three-dimensional image; a processor, communicatively connected or electrically connected to the camera assembly, and configured to control the actions of the lawn mowing robot at least according to the environmental image; In which, the processor is further configured to: cluster the pixel points or voxel points in the environmental image to obtain a pixel set or voxel set corresponding to each object contained in the environmental image; when the size of the pixel set or the voxel set is less than or equal to a first threshold, eliminate the pixel set or the voxel set; when the size of the pixel set or the voxel set is greater than or equal to a second threshold, identify the type of object corresponding to the pixel set or the voxel set, and control the walking motor according to the type of the object.

2. The lawn mowing robot according to claim 1, wherein: The processor clusters the pixel points or voxel points in the environment image using a K-Means or region growing algorithm.

3. The lawn mowing robot according to claim 1, wherein: The camera assembly includes at least two cameras, and the processor performs stereo matching on two two-dimensional environment images captured by the two cameras at the same time to generate a corresponding three-dimensional environment image.

4. The lawn mowing robot according to claim 1, wherein: The processor converts the pixel set into the corresponding voxel set according to the camera parameters of the camera assembly.

5. The lawn mowing robot according to claim 1, wherein: The size is the length of a plane diagonal of a bounding box of the pixel set, or the length of a three-dimensional diagonal of a bounding box of the voxel set; or, the size is one of the length, width, and height of the bounding box of the pixel set or the voxel set.

6. The lawn mowing robot according to claim 1, wherein: The size is the maximum value of the distance between two pixels or two voxel points in the pixel set or the voxel set.

7. The lawn mowing robot according to claim 1, wherein: The first threshold is less than or equal to 5 cm.

8. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A sensor component configured to collect environmental information around the lawn mowing robot; A first processor is configured to identify obstacles according to the environmental information and determine a coordinate set of the obstacles; A second processor is configured to control the actions of the lawn mowing robot; Wherein, the first processor is further configured to: determine the representative coordinates of the obstacle according to the coordinate set of the obstacle, and transmit the representative coordinates to the second processor; the second processor is further configured to: control the walking motor according to the representative coordinates of the obstacle.

9. The lawn mowing robot according to claim 8, wherein: The first processor is a GPU, and the second processor is a CPU.

10. The lawn mowing robot according to claim 8, wherein: The representative coordinates are coordinates of vertices of a three-dimensional diagonal line of a bounding box of the obstacle, or the representative coordinates are coordinates of vertices of a two-dimensional diagonal line of a bounding box of the obstacle.

11. A lawn mowing robot, comprising: A drive assembly, comprising a blade and a drive motor for driving the blade; A travel assembly, comprising a travel wheel and a travel motor driving the travel wheel; A camera assembly is configured to collect an environmental image around the lawn mowing robot, wherein the environmental image is a two-dimensional image or a three-dimensional image; a processor, communicatively connected or electrically connected to the camera assembly, and configured to control the actions of the lawn mowing robot at least according to the environmental image; Wherein, the processor is further configured to: cluster the pixel points or voxel points in the environmental image to obtain a pixel set or voxel set corresponding to each object contained in the environmental image; when the set parameter of the pixel set or the voxel set is less than or equal to a parameter threshold, eliminate the pixel set or the voxel set; when the set parameter of the pixel set or the voxel set is greater than or equal to the parameter threshold, identify the type of object corresponding to the pixel set or the voxel set, and control the walking motor according to the type of the object.

12. The lawn mowing robot according to claim 11, wherein: The set parameter is the number of pixels or the number of voxels.

Images