Self-moving mower and control method

Abstract

The invention discloses a self-moving mower and a control method. The self-moving mower comprises a mower body; the walking wheel assembly is configured to support the machine body; the visual sensor is configured to acquire a lawn image of a working area where the self-moving mower is located; the controller is electrically connected with the visual sensor, and the controller is configured to obtain and receive a lawn image, obtained by the visual sensor, of the working area according to an inspection instruction; processing the lawn image to obtain feature information of grass; wherein the feature information at least comprises the height and color of the grass; a mowing plan or an inspection plan is made according to the feature information, and the self-moving mower is controlled to work according to the mowing plan or the inspection plan; the mowing plan comprises mowing task execution time; the inspection plan comprises the time for generating the inspection instruction next time. A mowing plan or an inspection plan of the self-moving mower is made at least according to feature information such as height and color of grass, lawn maintenance is more intelligent, and lawn maintenance quality can be improved.

Description

Technical Field

[0001] This application relates to a lawn mower and a control method, and more particularly to a self-propelled lawn mower and a control method. Background Art

[0002] Traditional self-propelled lawn mowers can perform mowing tasks automatically within a fixed area. The start of the mowing task usually requires the user to manually start it, or set a fixed time to execute the mowing task, or mow at a fixed height. However, with the changes in factors such as light, precipitation, season, temperature, and grass species, performing mowing at fixed time intervals or fixed heights is not the best choice.

[0003] Therefore, there is an urgent need for a more intelligent control method for lawn mowers, which can maintain the lawn more intelligently even when the user has no control for a long time.

[0004] This section provides background information related to this application, which is not necessarily prior art. Summary of the Invention

[0005] An object of this application is to solve or at least alleviate some or all of the above problems. To this end, an object of this application is to provide a self-propelled lawn mower that can maintain the lawn more intelligently even when the user has no control for a long time.

[0006] To achieve the above object, this application adopts the following technical solutions:

[0007] A self-propelled lawn mower, comprising:

[0008] A body;

[0009] A walking wheel assembly configured to support the body;

[0010] A vision sensor configured to acquire a lawn image of the working area where the self-propelled lawn mower is located;

[0011] A controller electrically connected to the vision sensor, the controller being configured to receive the lawn image of the working area acquired by the vision sensor according to an inspection instruction;

[0012] Process the lawn image to obtain grass feature information; wherein, the feature information at least includes the height and color of the grass;

[0013] Formulate a mowing plan or an inspection plan according to the feature information, and control the operation of the self-propelled lawn mower according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time to execute the mowing task; the inspection plan includes the time to generate the next inspection instruction.

[0014] A control method for a self - moving lawn mower, which is applied to the self - moving lawn mower. The method includes:

[0015] Obtain an inspection instruction, and obtain a lawn image of the working area of the self - moving lawn mower according to the inspection instruction;

[0016] Process the lawn image to obtain feature information of the grass; wherein, the feature information at least includes the height and color of the grass;

[0017] Formulate a mowing plan or an inspection plan according to the feature information, and control the self - moving lawn mower to work according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time for performing the mowing task; the inspection plan includes the time for generating the next inspection instruction.

[0018] In some embodiments, the formulating a mowing plan or an inspection plan according to the feature information includes:

[0019] Determine a mowing height threshold according to the color;

[0020] If the height is greater than the mowing height threshold, control the self - moving lawn mower to perform the mowing task;

[0021] If the height is less than or equal to the mowing height threshold, formulate a mowing plan or an inspection plan according to the difference between the mowing height threshold and the height.

[0022] In some embodiments, the feature information further includes size, shape and / or density; then the determining a mowing height threshold according to the color includes:

[0023] Obtain season information, and determine the type of grass according to the season information, the size, the shape and the color;

[0024] Determine the mowing height threshold according to the type, the density and the season information.

[0025] In some embodiments, the formulating a mowing plan or an inspection plan according to the difference between the mowing height threshold and the height includes:

[0026] If the difference is less than or equal to a preset threshold, formulate a mowing plan according to the difference;

[0027] If the difference is greater than the preset threshold, formulate an inspection plan according to the difference.

[0028] In some embodiments, the height is determined according to the point cloud data of the grass; the point cloud data is determined according to the lawn image.

[0029] A control device for a self - moving lawn mower, which is applied to a self - moving lawn mower. The device includes:

[0030] An acquisition unit, configured to acquire an inspection instruction and obtain a lawn image of the working area of the self - moving lawn mower according to the inspection instruction;

[0031] A processing unit, configured to process the lawn image to obtain feature information of the grass; wherein, the feature information at least includes the height and color of the grass;

[0032] The processing unit is further configured to formulate a mowing plan or an inspection plan according to the feature information;

[0033] A control unit, configured to control the self - moving lawn mower to work according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time for performing the mowing task; the inspection plan includes the time for generating the next inspection instruction.

[0034] In some embodiments, the processing unit is specifically configured to:

[0035] Determine a mowing height threshold according to the color;

[0036] If the height is greater than the mowing height threshold, control the self - moving lawn mower to perform a mowing task;

[0037] If the height is less than or equal to the mowing height threshold, formulate a mowing plan or an inspection plan according to the difference between the mowing height threshold and the height.

[0038] An electronic device, the electronic device includes:

[0039] At least one processor; and

[0040] A memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the control method of the self - moving lawn mower according to any embodiment of the present application.

[0042] A computer - readable storage medium, the computer - readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the control method of the self - moving lawn mower according to any embodiment of the present application when executed.

[0043] The advantages of the present application are as follows: Since the colors of different types of grass usually vary, and the colors of the same type of grass also usually have certain differences in different seasons, the suitable mowing heights for different types of grass usually have certain differences, and the suitable mowing heights for the same type of grass in different seasons usually also have certain differences. Therefore, in this solution, the growth state of the grass is identified based on at least characteristic information such as the height and color of the grass. For example, the type of grass and the season it is in can be determined based on the color, and a mowing plan or inspection plan for the self-propelled lawn mower is formulated according to the type, height, and season of the grass, making the maintenance of the lawn more intelligent and improving the quality of lawn maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 FIG. is a schematic diagram of the working scenario of a self-propelled lawn mower provided by an embodiment of the present application;

[0045] Figure 2 FIG. is a schematic diagram of the structure of a self-propelled lawn mower provided by an embodiment of the present application;

[0046] Figure 3 FIG. is a schematic diagram of the structure of a self-propelled lawn mower provided by an embodiment of the present application;

[0047] Figure 4 FIG. is a schematic diagram of the slope detection scenario of a self-propelled device provided by an embodiment of the present application;

[0048] Figure 5 FIG. is a flowchart of a control method for a self-propelled device provided by an embodiment of the present application;

[0049] Figure 6 FIG. is a flowchart of a slope detection algorithm based on point cloud provided by an embodiment of the present application;

[0050] Figure 7 FIG. is a schematic diagram of a lawn image provided by an embodiment of the present application;

[0051] Figure 8 FIG. is a flowchart of a control method for a self-propelled lawn mower provided by an embodiment of the present application;

[0052] Figure 9 FIG. is a flowchart of an intelligent mowing decision algorithm provided by an embodiment of the present application;

[0053] Figure 10 FIG. is a schematic diagram of the positioning scenario of a self-propelled device traveling on a lawn provided by an embodiment of the present application;

[0054] Figure 11 FIG. is a flowchart of a positioning method for a self-propelled device provided by an embodiment of the present application;

[0055] Figure 12It is a flowchart of a positioning parameter adjustment algorithm based on inertial measurement unit data statistics provided by an embodiment of the present application;

[0056] Figure 13 It is a flowchart of a model training method applied to lawn image processing provided by an embodiment of the present application;

[0057] Figures 14A to 14C They are schematic diagrams of a Gaussian blur image, a mean blur image, and a median blur image obtained by blurring a lawn image to be trained provided by an embodiment of the present application, respectively;

[0058] Figure 15 It is a flowchart of a lawn image processing method provided by an embodiment of the present application;

[0059] Figure 16 It is a flowchart of a model training method based on data augmentation provided by an embodiment of the present application;

[0060] Figure 17 It is a schematic diagram of a reference object provided by an embodiment of the present application;

[0061] Figure 18 It is a flowchart of a control method for a self-propelled lawn mower provided by an embodiment of the present application;

[0062] Figure 19 It is a flowchart of a method for establishing a feature transformation reference based on lawn markers provided by an embodiment of the present application;

[0063] Figure 20 It is a schematic structural diagram of an electronic device for implementing the method of an embodiment of the present application. Detailed implementation manners

[0064] Before explaining any embodiment of the present application in detail, it should be understood that the present application is not limited to the structural details and component arrangements described in the following description or shown in the above drawings.

[0065] In the present application, the terms "including", "comprising", "having" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device.

[0066] In the present application, the term "and / or" is an associative relationship describing associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the present application, the character " / " generally represents an "and / or" relationship between the associated objects before and after.

[0067] In this application, the terms "connected", "combined", "coupled", and "mounted" can be direct connections, combinations, couplings, or mountings, or can be indirect connections, combinations, couplings, or mountings. Among them, by way of example, a direct connection means that two parts or components are connected together without the need for an intermediate member, and an indirect connection means that two parts or components are respectively connected to at least one intermediate member, and these two parts or components are connected through the intermediate member. In addition, "connected" and "coupled" are not limited to physical or mechanical connections or couplings, and can include electrical connections or couplings.

[0068] In this application, those of ordinary skill in the art will understand that relative terms used in connection with a quantity or condition (e.g., "about", "approximately", "substantially", etc.) are intended to include the value and have the meaning indicated by the context. For example, such relative terms include at least the degree of error associated with the measurement of a particular value, tolerances resulting from manufacturing, assembly, use in relation to a particular value, etc. Such terms should also be considered to disclose a range defined by the absolute values of the two endpoints. The relative term may refer to a plus or minus a certain percentage (e.g., 1%, 5%, 10% or more) of the indicated value. A numerical value without the use of a relative term should also be disclosed as a particular value with a tolerance. In addition, "substantially" when expressing a relative angular positional relationship (e.g., substantially parallel, substantially perpendicular) may refer to a plus or minus a certain number of degrees (e.g., 1 degree, 5 degrees, 10 degrees or more) from the indicated angle.

[0069] In this application, those of ordinary skill in the art will understand that the functions performed by a component can be performed by one component, multiple components, one part, or multiple parts. Similarly, the functions performed by a part can also be performed by one part, one component, or a combination of multiple parts.

[0070] In this application, the orientation terms such as "upper", "lower", "left", "right", "front", "rear", etc. are described based on the orientation and positional relationship shown in the drawings, and should not be construed as a limitation on the embodiments of this application. In addition, in the context, it should also be understood that when it is mentioned that one element is connected "above" or "below" another element, it can not only be directly connected "above" or "below" another element, but also be indirectly connected "above" or "below" another element through an intermediate element. It should also be understood that orientation terms such as upper side, lower side, left side, right side, front side, rear side, etc. not only represent the positive orientation, but can also be understood as the side orientation. For example, the lower side can include directly below, lower left, lower right, lower front, and lower rear, etc.

[0071] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0072] In the present application, the terms "controller", "processor", "central processing unit", "CPU", "MCU" can be interchanged. When using the units "controller", "processor", "central processing unit", "CPU", or "MCU" to perform specific functions, unless otherwise specified, these functions can be performed by a single one of the above units or multiple of the above units.

[0073] In the present application, the terms "device", "module" or "unit" can be implemented in the form of hardware or software in order to achieve a specific function.

[0074] In the present application, the terms "calculate", "judge", "control", "determine", "identify", etc. refer to the operations and processes of a computer system or a similar electronic computing device (such as a controller, a processor, etc.).

[0075] The self-moving device provided by the present application can be, for example, a self-moving lawn mower.

[0076] Figure 1 It is a schematic diagram of the working scenario of a self-moving lawn mower provided by an embodiment of the present application. As Figure 1As shown, the self - propelled lawn mower can move and work within the working area. The working area is generally outdoors and includes a lawn. The outdoor working environment is complex and changeable, which poses challenges to the self - propelled lawn mower working therein. For example, the terrain of the lawn is diverse and may have slopes. If the slope is too high, it may cause the self - propelled lawn mower to tip over. There can be various types of grass on the lawn, and the growth of grass is generally affected by factors such as light, precipitation, season, and temperature, making it challenging for the self - propelled lawn mower to perform intelligent maintenance of the lawn. With the changes in weather and seasons, the lawn ground is prone to obvious changes in surface undulation, which in turn causes the self - propelled lawn mower working on the lawn ground to jolt. Moreover, due to the complex and changeable outdoor working environment, including smoke, dust, weather, and light changes, etc., it may lead to an overall shift in the distribution of the obtained lawn images. Intelligent recognition can be carried out based on the lawn images, but the diversity and shift of the lawn images will affect the recognition accuracy to a certain extent.

[0077] Figure 2 is a schematic structural diagram of a self - propelled lawn mower 100 provided by an embodiment of the present application. Refer to Figure 2 , an embodiment of the present application provides a self - propelled device, including: a body 110; a walking wheel assembly 120 configured to support the body 110; a vision sensor 130 configured to acquire an image in the traveling direction of the self - propelled device; combined with Figure 2 and Figure 3 , an attitude sensor 140 configured to acquire the attitude of the self - propelled device; a controller 150 electrically connected to the vision sensor 130 and the attitude sensor 140. The controller 150 is configured to calculate a first inclination angle of the ground in the traveling direction of the self - propelled device based on the image in the traveling direction of the self - propelled device acquired by the vision sensor 130, calculate a second inclination angle of the ground by combining the first inclination angle and the attitude of the self - propelled device, and control the walking wheel assembly 120 according to the second inclination angle.

[0078] Combined with Figure 2 and Figure 3 , the vision sensor 130 can be installed on the body of the self - propelled device above the axle, and the vision sensor 130 can be flexibly installed in various areas of the body. For example, it can be directly installed on the housing of the body 110, and the installation orientation of the vision sensor 130 can include all angles. By flexibly setting the installation position and installation orientation of the vision sensor 130 on the self - propelled device, the acquisition angle of the image can be flexibly adjusted. Refer to Figure 2 and Figure 3 , the attitude sensor 140 and the controller 150 are also respectively arranged on the body of the self - propelled device, and the controller 150 is electrically connected to the vision sensor 130 and the attitude sensor 140 respectively.

[0079] Refer toFigure 3 , the controller 150 may include an acquisition unit 151, a processing unit 152, and a control unit 153. Specifically, images of the self-mobile device in the traveling direction may be collected by the vision sensor 130, and the attitude of the self-mobile device may be obtained by the attitude sensor 140. The acquisition unit 151 in the controller 150 obtains images from the vision sensor 130 and obtains the attitude of the self-mobile device from the attitude sensor 140; the processing unit 152 processes each image in a preset manner to obtain a first inclination angle of the ground in the traveling direction of the self-mobile device; and the second inclination angle of the ground may be obtained according to the attitude of the self-mobile device and the first inclination angle. The control unit 153 may control the walking wheel assembly 120 to walk by using the second inclination angle.

[0080] In this solution, by combining the attitude of the self-mobile device and according to the first inclination angle, the second inclination angle is obtained, which improves the accuracy of slope angle recognition. Taking the self-mobile device as the self-propelled lawn mower 100 as an example, in the same slope scenario, even if the self-propelled lawn mower works on the slope in different attitudes, the second inclination angle of the slope can be accurately obtained by combining the attitude and the first inclination angle, thereby preventing the self-propelled lawn mower 100 from tipping over.

[0081] In an implementable manner, the first inclination angle is the relative inclination angle of the ground with respect to the self-mobile device; the second inclination angle is the absolute inclination angle of the ground with respect to the direction of gravity.

[0082] Figure 4 It is a schematic diagram of a slope detection scenario of a self-mobile device provided by an embodiment of the present application. Taking the self-mobile device as the self-propelled lawn mower 100 as an example, as Figure 4 shown, the dashed line CD represents the horizon; the dashed line BH is parallel to the dashed line CD; the solid line BC represents the ground on which the self-propelled lawn mower 100 is traveling; the solid line AB represents the ground in front of the traveling direction of the self-propelled lawn mower 100; the dashed line BG is the extension line of the solid line BC; the dashed line EF represents the direction of gravity; wherein, the dashed line CD and the dashed line EF are perpendicular to each other.

[0083] The self-propelled mower 100 can collect the front image in the traveling direction and process the image into a point cloud. Then, the ground point cloud corresponding to the solid line AB is extracted from the point cloud. The inclination angle of the front ground is extracted from the ground point cloud, and this inclination angle is the relative inclination angle of the solid line AB with respect to the self-propelled mower 100, and the relative inclination angle can be represented by ∠ABG. The pitch angle in the attitude of the self-propelled mower 100 is represented by ∠BCD. Since the dotted line BH is parallel to the dotted line CD and the dotted line BG is the extension of the solid line BC, then ∠BCD = ∠GBH. Adding the pitch angle to the relative inclination angle gives the first absolute inclination angle of the front ground with respect to the gravity direction, and the first absolute inclination angle can be represented by ∠ABH.

[0084] Similarly, the roll angle in the attitude of the self-propelled mower 100 can also be used to determine the second absolute inclination angle of the front ground with respect to the gravity direction.

[0085] The first absolute inclination angle and the second absolute inclination angle are used as the slope detection results. Further, the first absolute inclination angle can be compared with the first angle threshold, and the second absolute inclination angle can be compared with the second angle threshold. If the first absolute inclination angle is less than the first angle threshold and the second absolute inclination angle is less than the second angle threshold, it is determined that the front ground is a safe area. If the first absolute inclination angle is greater than or equal to the first angle threshold, or the second absolute inclination angle is greater than or equal to the second angle threshold, it is determined that the front ground is a non-safe area. Among them, both the first angle threshold and the second angle threshold can be preset according to the actual situation. The first angle threshold and the second angle threshold may not be equal. First, the first absolute inclination angle and the second absolute inclination angle are obtained respectively according to the pitch angle and the roll angle in the attitude, and then whether the front ground is a safe area is comprehensively determined according to the first absolute inclination angle and the second absolute inclination angle, which can improve the accuracy of the determination and further reduce the probability of the self-propelled device tipping over.

[0086] Among them, the first inclination angle refers to the relative inclination angle between the ground and the self-propelled device, while the second inclination angle refers to the absolute inclination angle of the ground, which is not affected by the position of the self-propelled device. Specifically, in combination with the attitude of the self-propelled device, the influence of the attitude is removed from the first inclination angle to obtain the second inclination angle.

[0087] Specifically, according to the attitude of the self-propelled device, the inclination angle of the ground where the self-propelled device is located with respect to the gravity direction can be determined. Then, based on this inclination angle and the relative inclination angle of the ground in the traveling direction of the self-propelled device with respect to the self-propelled device, the absolute inclination angle of the ground in the traveling direction of the self-propelled device with respect to the gravity direction can be obtained.

[0088] In an actual application scenario, what affects whether the self - moving device topples is the absolute tilt angle of the ground where the self - moving device is located relative to the direction of gravity. Therefore, taking the absolute tilt angle as the slope angle recognition result improves the accuracy of slope angle recognition.

[0089] In one implementable way, the vision sensor 130 may include at least one of the following: an active laser depth camera, a passive binocular stereo camera, and a passive binocular structured light camera.

[0090] In one implementable way, the vision sensor 130 includes at least one of a Time of Flight Camera (TOF) depth camera, an Infrared Ray (IR) structured light camera, and a binocular camera.

[0091] In one implementable way, the controller 150 creates a depth image from the image, extracts the ground point cloud from the depth image, and obtains the first tilt angle through the ground point cloud.

[0092] The processing unit 152 in the controller 150 can create a depth image based on the image in the traveling direction of the self - moving device. Specifically, the depth information of each pixel point can be estimated by analyzing the geometric relationship between the pixels and objects in the image. The depth image represents the distance of each point in the scene relative to the camera. There are various methods for creating a depth image from an image, including depth estimation based on a single image, stereo matching based on multiple images, etc. Among them, the stereo matching method refers to calculating the disparity by matching the corresponding pixel points of the same object in different images, thereby estimating the depth information.

[0093] The processing unit 152 can extract the ground point cloud from the depth image. Specifically, the depth image can be first processed into a point cloud, and then the ground point cloud can be extracted from the point cloud. Among them, various algorithms can be used to extract the ground point cloud from the point cloud. The various algorithms include detection based on three - dimensional features of the point cloud and detection based on planar features of the image. The detection algorithms are not limited to detection based on classical hand - crafted features and detection based on learned features. Among them, detection based on three - dimensional features of the point cloud refers to identifying and classifying target objects by extracting three - dimensional features in the point cloud data, such as shape, edge, texture, etc. Among them, detection based on planar features of the image refers to identifying and classifying target objects by extracting planar features in the image, such as edges, corner points, etc. Among them, detection based on classical hand - crafted features refers to identifying and classifying target objects by extracting hand - crafted features in the image, such as edges, corner points, texture, etc. Among them, detection based on learned features refers to learning the feature representation in the image and combining it with a classifier to identify and classify target objects.

[0094] In one possible implementation, the point cloud can also be combined with a color image, that is, the point cloud is colored to obtain a colored point cloud. Then, the ground point cloud is extracted from the colored point cloud, and the first tilt angle is obtained through the ground point cloud. Specifically, the colored point cloud contains the color information of the point cloud. Therefore, the colored point cloud contains more semantic information than the point cloud, and thus the first tilt angle obtained using the colored point cloud is more accurate to a certain extent.

[0095] The processing unit 152 can obtain the first tilt angle through the ground point cloud. Specifically, a geometric method can be used to obtain the first tilt angle through the ground point cloud. The geometric method includes at least one of the following methods: line-plane fitting method, voxel sampling method. Among them, the line-plane fitting method means that according to the lines and planes in the ground point cloud, through geometric reasoning and calculation, the lines and planes in the real three-dimensional scene corresponding to the ground point cloud are restored, and then the first tilt angle is calculated from the fitted lines and planes. Among them, the voxel sampling method means that the ground point cloud data is divided into a series of small cubes, called voxels, and then a representative point is selected as the sampling point in each voxel. Then, using the coordinate information of the sampling point, the tilt angle of each sampling point relative to the self-mobile device is calculated. By synthesizing the tilt angles of each sampling point relative to the self-mobile device, the first tilt angle is obtained.

[0096] Using the above method, the first tilt angle can be obtained quickly and accurately according to the image.

[0097] In one possible implementation, when the second tilt angle is greater than or equal to the first threshold, the controller 150 determines that the ground in the traveling direction of the self-mobile device is a non-safe area and changes the traveling direction.

[0098] Among them, the first threshold is a tilt angle threshold set in advance according to the actual situation.

[0099] Specifically, when it is determined that the second tilt angle is greater than the first threshold, the control unit 153 in the controller 150 determines that the ground in the traveling direction of the self-mobile device is a non-safe area and changes the traveling direction of the self-mobile device to prevent the self-mobile device from tipping over due to the ground tilt. Among them, the non-safe area refers to the ground area where the tilt angle is too high, which is likely to cause the self-mobile device to tip over.

[0100] In one possible implementation, when the second tilt angle is less than the first threshold, the controller 150 determines that the ground in the traveling direction of the self-mobile device is a safe area and maintains the traveling direction.

[0101] Specifically, when it is determined that the second tilt angle is less than the first threshold, the control unit 153 in the controller 150 may determine that the ground in the traveling direction of the self-mobile device is a safe area and continue to travel in that traveling direction. Herein, the safe area refers to a ground area where the tilt angle is not too high, and thus it is not easy for the self-mobile device to tip over.

[0102] In one implementable manner, the traveling direction of the self-mobile device is the front of the self-mobile device.

[0103] Among them, the traveling direction of the self-mobile device can be determined according to path planning.

[0104] In one implementable manner, the first threshold is determined according to the friction of the ground.

[0105] The first threshold can be determined according to the friction of the ground, and the value of the first threshold can be proportional to the friction of the ground. The greater the friction of the ground, the greater the value of the first threshold can be. Determining the first threshold according to the friction of the ground can make the value of the first threshold more accurate.

[0106] In one implementable manner, the attitude sensor 140 is an accelerometer.

[0107] Figure 5 is a flowchart of a control method for a self-mobile device provided by an embodiment of the present application. This embodiment is applicable to a traveling route planning scenario when the self-mobile device travels on a ground with a certain inclination. The self-mobile device can be, for example, a self-propelled lawn mower 100, and this method can be executed by the self-mobile device. As Figure 5 shown, the method includes:

[0108] Step 501, obtain the attitude of the self-mobile device; and obtain an image in the traveling direction of the self-mobile device.

[0109] Step 502, calculate the first tilt angle of the ground in the traveling direction of the self-mobile device according to the image.

[0110] Step 503, combine the first tilt angle and the attitude to calculate the second tilt angle of the ground; and determine whether to travel in the traveling direction according to the second tilt angle.

[0111] This solution combines the attitude of the self-mobile device to eliminate the influence of the attitude from the first tilt angle to obtain the second tilt angle. The second tilt angle is used as the slope detection result of the ground in the traveling direction of the self-mobile device. And it can be determined whether to travel in the traveling direction according to the second tilt angle, control the traveling of the self-mobile device, and prevent the self-mobile device from tipping over due to too large a slope. The accuracy of slope detection of the ground in the traveling direction of the self-mobile device is improved, and thus the tipping probability of the self-mobile device is reduced.

[0112] The embodiment of the present application also specifically provides a slope detection algorithm based on point cloud for a lawn mower equipped with a depth camera. Figure 6 is a flowchart of a slope detection algorithm based on point cloud provided by the embodiment of the present application. Refer to Figure 6 and the specific process of this algorithm is as follows:

[0113] Step 601: Power on the lawn mower and start the slope detection program in the lawn mower controller.

[0114] Step 602: Use the depth camera to collect the image in front of the lawn mower. The depth camera is not limited to an active laser depth camera, a passive binocular stereo camera, a passive binocular structured light camera, etc.

[0115] Step 603: Process the depth image collected by the lawn mower into point cloud through the algorithm in the slope detection program.

[0116] Step 604: Extract the ground point cloud from the point cloud in front of the lawn mower through algorithm processing. The algorithm includes detection based on three-dimensional features of the point cloud and detection based on image plane features. The detection algorithm is not limited to detection based on classical handcrafted features and detection based on learned features.

[0117] Step 605: Extract the relative inclination angle of the ground on the traveling route through geometric methods based on the ground point cloud information. The extraction method is not limited to methods such as line-plane fitting method and voxel sampling method, etc.

[0118] Step 606: Combine the attitude information provided by its own inertial navigation with the ground inclination information to comprehensively judge the absolute inclination angle of the ground relative to the gravity direction.

[0119] Step 607: Upload the inclination angle of the ground relative to the gravity direction to the planning and decision-making layer to judge whether it is a safe area where it can travel.

[0120] Step 608: Plan the route according to the judgment result of Step 607. Specifically, if it is determined to be an area where it can travel, control the lawn mower to continue moving forward in the traveling direction; if it is determined to be an area where it cannot travel, change the traveling direction and plan a safe traveling route.

[0121] Figure 7 is a schematic diagram of a lawn image provided by the embodiment of the present application. The growth state of grass is affected by factors such as light, precipitation, season, temperature, etc. and is not constant. As Figure 7 shown in the lawn image, it contains the characteristic information of grass, such as the height, color, type, etc. of the grass, which characterizes the growth state of the grass. It is a challenge for a self-propelled lawn mower to perform intelligent maintenance on the lawn for different growth states of grass.

[0122] In one embodiment, in combination with Figure 2and Figure 3 , the controller 150 is electrically connected to the vision sensor 130. The controller 150 is configured to obtain and, according to the inspection instruction, receive the lawn image of the working area acquired by the vision sensor 130; process the lawn image to obtain the feature information of the grass; wherein, the feature information at least includes the height and color of the grass; formulate a mowing plan or an inspection plan according to the feature information, and control the self-propelled mower 100 to work according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time for performing the mowing task; the inspection plan includes the time for generating the next inspection instruction. Refer to Figure 3 , the controller 150 may include an acquisition unit 151, a processing unit 152 and a control unit 153. Specifically, the lawn image of the working area where the self-propelled mower 100 is located may be collected by the vision sensor 130. Refer to Figure 3 , the acquisition unit 151 may obtain the inspection instruction and obtain the lawn image from the vision sensor 130 according to the inspection instruction; wherein, the inspection instruction may be generated in response to the user's operation or generated according to the inspection plan. The processing unit 152 may process the lawn image by using a preset method to obtain the feature information of the grass, wherein, the feature information at least includes the height and color of the grass. Specifically, the lawn image may be preprocessed first, including steps such as denoising, image enhancement, color correction, etc., to improve the image quality. Then, the deep learning semantic segmentation algorithm may be used to separate the grass plants in the preprocessed lawn image from the background. According to the segmentation result, parameter calculation may be performed on the segmented grass plants, at least including the height and color of the grass plants. Finally, statistical analysis may be performed on the calculated grass plant parameters to obtain the overall features of the lawn. For example, statistical data such as the average grass plant height and the average grass plant color may be calculated. The average grass plant height may be used as the height of the grass, and the average grass plant color may be used as the color of the grass. The processing unit 152 may formulate a mowing plan or an inspection plan according to the feature information of the grass. For example, if it is determined according to the feature information of the grass that the lawn needs to be trimmed in the near future, a mowing plan is generated. If it is determined according to the feature information of the grass that the lawn does not need to be trimmed in the near future, an inspection plan may be generated to monitor the growth of the grass and perform mowing when the grass needs to be trimmed. The control unit 153 may control the self-propelled mower 100 to work according to the mowing plan or the inspection plan. Wherein, the mowing plan may include the mowing task and the time for performing the mowing task. The inspection plan may include the time for the self-propelled mower 100 to generate the next inspection instruction.

[0123] Since the colors of different types of grass usually vary, and the colors of the same type of grass usually also have certain differences in different seasons, the suitable mowing heights for different types of grass usually have certain differences, and the suitable mowing heights for the same type of grass in different seasons usually also have certain differences. Therefore, in this solution, the growth state of the grass is identified at least based on characteristic information such as the height and color of the grass. For example, the type of grass and the season it is in can be determined based on the color, and a mowing plan or inspection plan for the self-propelled lawn mower 100 is formulated according to the type, height, and season of the grass, making the maintenance of the lawn more intelligent, and thus improving the quality of lawn maintenance.

[0124] In one implementable manner, a mowing height threshold is determined according to the color.

[0125] Specifically, a color-height mapping table can be preset according to experience, and the mapping relationship between the color and the height is included in the table. The processing unit 152 can query the color-height mapping table using the color of the grass, obtain the height corresponding to the color of the grass, and determine this height as the mowing height threshold.

[0126] If the height is greater than the mowing height threshold, control the self-propelled lawn mower to perform the mowing task.

[0127] The processing unit 152 can compare the height of the grass with the mowing height threshold. If it is determined that the height of the grass is greater than the mowing height threshold, a mowing task to be executed is generated. The control unit 153 controls the self-propelled lawn mower 100 to work according to the mowing task to be executed.

[0128] If the height is less than or equal to the mowing height threshold, a mowing plan or inspection plan is formulated according to the difference between the mowing height threshold and the height.

[0129] If the processing unit 152 determines that the height of the grass is less than or equal to the mowing height threshold, it calculates the difference between the mowing height threshold and the height of the grass, and formulates a mowing plan or inspection plan according to the difference. The control unit 153 controls the self-propelled lawn mower 100 to work according to the mowing plan or inspection plan. For example, if the difference is small, it can be determined that the grass needs to be trimmed in the near future, and at this time, a mowing plan can be formulated. If the difference is large, it can be determined that the grass does not need to be trimmed in the near future, and at this time, an inspection plan can be formulated to monitor the growth of the grass and mow the grass when it needs to be trimmed.

[0130] Since the colors of different types of grass usually vary, and the colors of the same type of grass also usually vary to some extent in different seasons, the suitable mowing heights for different types of grass usually vary to some extent, and the suitable mowing heights for the same type of grass also usually vary in different seasons. Therefore, in this solution, the growth state of the grass is identified by the color of the grass. For example, the type of grass and the season it is in can be determined based on the color, and then the mowing height threshold can be determined according to the type of grass and the season. Thus, a mowing plan or an inspection plan can be formulated based on the mowing height threshold and the height of the grass, which can improve the maintenance quality of the lawn.

[0131] In an implementable manner, the feature information further includes size, shape, and / or density; the obtaining unit 151 obtains the season information, and the processing unit 152 determines the type of grass according to the season information, size, shape, and color.

[0132] In this application, a feature information mapping table can be preset according to experience. The table includes the mapping relationships between season, size, shape, color, and the type of grass. The obtaining unit 151 can also obtain the season information. Specifically, the obtaining unit 151 can calculate the season information based on the time information. Or, the season information can be determined at least according to the color of the grass. The processing unit 152 can process the lawn image to obtain the size, shape, and density of the grass. Specifically, the grass plants in the lawn image can be separated from the background, and then parameter calculations are performed on the segmented grass plants, including the size, shape, and density of the grass plants. Among them, the size includes at least height and width; the shape includes at least roundness and complexity; the density includes at least the number of grass plants per unit area. Finally, statistical analysis is performed on the calculated grass plant parameters to obtain the overall characteristics of the lawn. For example, statistical data such as the average grass plant height, average grass plant width, average grass plant roundness, average grass plant complexity, and average density can be calculated. Information such as the average grass plant height and average grass plant width can be used as the size of the grass, information such as the average grass plant roundness and average grass plant complexity can be used as the shape of the grass, and information such as the average density can be used as the density of the grass. Then, using the season information, the size, shape, and color of the grass, the feature information mapping table is queried to obtain the type of grass.

[0133] The processing unit 152 determines the mowing height threshold according to the type of grass, density, and season information.

[0134] Specifically, a feature mapping table can be preset according to experience. The table includes the mapping relationships between type, density, season, and height. The processing unit 152 can query the feature mapping table according to the season information and the type and density of the grass to obtain the height corresponding to the season information and the type and density of the grass, and determine this height as the mowing height threshold.

[0135] Since different types of grass usually have differences in characteristics such as size, shape, and color in different seasons, the suitable mowing heights for different types of grass usually have certain differences. The suitable mowing heights for the same type of grass usually have certain differences under different seasonal conditions, and the suitable mowing heights for grass with different densities also usually have certain differences. Therefore, this solution can first accurately determine the type of grass according to the seasonal information and the size, shape, and color of the grass, and then more accurately determine the mowing height threshold according to the type of grass, density, and seasonal information. Controlling the operation of the automatic lawn mower 100 according to the mowing plan or inspection plan formulated based on this mowing height threshold can improve the quality of lawn maintenance.

[0136] In an implementable manner, if the processing unit 152 determines that the difference between the mowing height threshold and the height of the grass is less than or equal to a preset threshold, it formulates a mowing plan according to the difference; if the difference is greater than the preset threshold, it formulates an inspection plan according to the difference.

[0137] Among them, the preset threshold is a value set in advance according to the actual situation. If the difference between the mowing height threshold and the height of the grass is less than or equal to the preset threshold, it can indicate that the difference between the height of the grass and the mowing height threshold is not large. In this case, generally, the growth rate of the grass can be more accurately estimated, and then a more accurate mowing plan can be formulated according to the estimated growth rate of the grass.

[0138] If the difference between the mowing height threshold and the height of the grass is greater than the preset threshold, it can indicate that the difference between the height of the grass and the mowing height threshold is relatively large. In this case, it is generally difficult to accurately estimate the growth rate of the grass, and it is very difficult to accurately formulate a mowing plan. Therefore, at this time, an inspection plan can be formulated according to the difference to monitor the growth of the grass. For example, when the difference is greater than the preset threshold and if the difference is relatively large, a relatively long time interval can be set to inspect the lawn to determine whether mowing is required; when the difference is greater than the preset threshold and if the difference is relatively small, a relatively short time interval can be set to inspect the lawn to determine whether mowing is required. This can not only improve the quality of lawn maintenance but also save resources.

[0139] This solution can first determine the difference between the mowing height threshold and the height of the grass, and then formulate a mowing plan or inspection plan based on the difference and the comparison result between the difference and the preset threshold, which can more precisely control the operation of the self-propelled lawn mower 100, and thus achieve a better lawn maintenance effect.

[0140] In an implementable manner, the height of the grass is determined according to the point cloud data of the grass; the point cloud data is determined according to the lawn image.

[0141] The processing unit 152 can first establish a depth image from the lawn image, and then process the depth image into a point cloud, which is the point cloud data of the grass. Then, the height of the grass can be identified using the point cloud data of the grass. In this way, the height of the grass can be obtained quickly and accurately. Specifically, each point on the depth image can be transformed into the world coordinate system through the relationship of similar triangles or coordinate system transformation to obtain the point cloud data of the grass. First, the vision sensor 130 needs to be aligned with the lawn surface, and the three-dimensional point cloud data of the lawn is obtained. The vision sensor 130 can capture the three-dimensional coordinates of each point on the lawn surface, including the X, Y, and Z coordinates. The obtained point cloud data needs to be processed to remove noise, smooth the data, and perform data registration, etc. These processes can include operations such as filtering, smoothing, and resampling to improve the quality and accuracy of the data. Features can be extracted from the point cloud data, including the shape and height of the grass. Then, a classifier can be designed based on the extracted features to distinguish the grass from other objects. After the classifier identifies the grass, in the processed point cloud data, a certain area of the point cloud data can be selected, and the average height of the grass in this area can be obtained by calculating the average height of these points. Specifically, some representative point cloud data can be selected and their average height can be calculated to obtain the grass height in this area. It should be noted that the accuracy and stability of the vision sensor 130 may be affected by factors such as environmental light and camera settings, so fine calibration and calibration are required in advance. Of course, the point cloud data can also be obtained by lidar. Therefore, in this solution, lidar can also be used as an alternative to the vision sensor 130.

[0142] Figure 8 FIG. 4 is a flowchart of a control method for a self-propelled lawn mower provided by an embodiment of the present application. This embodiment is applicable to the scenario where the self-propelled lawn mower 100 performs intelligent maintenance on the lawn, especially applicable to the situation of being out of user control for a long time. This method can be executed by the self-propelled lawn mower 100. As Figure 8 shown, this method includes:

[0143] Step 801, obtain an inspection instruction, and obtain a lawn image of the working area of the self-propelled lawn mower according to the inspection instruction.

[0144] Step 802, process the lawn image to obtain the feature information of the grass; wherein, the feature information at least includes the height and color of the grass.

[0145] Step 803, formulate a mowing plan or an inspection plan according to the feature information, and control the self-propelled lawn mower to work according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time to execute the mowing task; the inspection plan includes the time to generate the next inspection instruction.

[0146] Since the colors of different types of grass usually vary, and the colors of the same type of grass also usually vary to some extent in different seasons, the suitable mowing heights for different types of grass usually vary to some extent, and the suitable mowing heights for the same type of grass also usually vary to some extent in different seasons. Therefore, this solution can formulate a more accurate mowing plan or inspection plan at least based on the height and color of the grass, so as to maintain the lawn more intelligently and achieve a better lawn maintenance effect.

[0147] The embodiment of the present application also specifically provides an intelligent mowing decision algorithm based on semantic recognition for the lawn mower. Figure 9 is a flowchart of an intelligent mowing decision algorithm provided by the embodiment of the present application. Refer to Figure 9 and the specific process of this algorithm is as follows:

[0148] Step 901, the lawn mower is powered on.

[0149] Step 902, start to execute the inspection mode in the lawn mower controller and determine whether to start the inspection.

[0150] Step 903, the lawn mower inspects the lawn along a certain route, which can be either a user-specified route, a route independently planned by the lawn mower according to the lawn conditions, or a completely random route.

[0151] Step 904, obtain an image through a vision sensor and use a certain algorithm to extract the semantic information in the image, and determine whether the lawn mower is traveling in the lawn area through the semantic information. Among them, this algorithm can be a deep learning algorithm. If the semantic information indicates that there is grass in the image, it is determined that the lawn mower is traveling in the lawn area; if the semantic information indicates that there is no grass in the image, it is determined that the lawn mower is not traveling in the lawn area. It is also possible to first process the image into a colored point cloud and identify the semantic information through the colored point cloud. It is also possible to combine the semantic information recognition results of the image and the semantic information recognition results of the colored point cloud to obtain the combined semantic information.

[0152] Step 905, if not in the lawn area, continue to move forward and continue to execute step 903; if in the lawn area, collect the lawn-related semantic information and feature information extracted by the algorithm. Among them, the semantic information includes identification labels such as whether it is grass, the type of grass, density, and the season to which it belongs. The semantic information can include semantic labels at the pixel or sub-pixel level of the lawn within the field of view. The feature information of the grass includes the color, height, size, shape, etc. of the grass. Among them, the season can also be judged by time.

[0153] Step 906: Determine whether the collected feature information is sufficient according to a preset statistical algorithm. If it is not sufficient, continue to collect; if it is sufficient, stop collecting and further extract the information required for decision-making based on the set of currently collected feature information. Among them, the decision-making information is not limited to the mean and variance of the grass feature information in the distribution of the entire lawn area, as well as the mean and variance of the grass feature information in different subdivided areas, and the semantic information of the grass, etc. Among them, determining whether the collected feature information is sufficient according to the preset statistical algorithm includes: if it is determined that the area of the lawn for statistics is large enough, it is determined that the collected feature information is sufficient.

[0154] Step 907: Overall judge whether to execute the mowing task according to the decision-making information. Among them, the mowing task is not limited to mowing the entire area, mowing part of the area, and multiple mowing tasks at different times. For example, the working area of the lawn mower can include various scenarios, such as uneven grass growth in the working area, uneven mowing in the previous mowing, or there are multiple types of grass in the working area and the mowing height requirements for each type of grass are different, etc. For different scenarios, the mowing task can be adaptively set.

[0155] Step 908: Execute the mowing task planned in step 907. If the mowing task cannot be executed due to other reasons, such as manual intervention, weather reasons, battery power, etc., record the current condition of the lawn for comprehensive decision-making in the next plan.

[0156] Figure 10 It is a schematic diagram of the positioning scenario of a self-moving device traveling on a lawn provided by an embodiment of the present application. As Figure 10 shown, the lawn is generally outdoors. Due to the changes in outdoor weather and seasons, obvious changes in the surface conditions of the lawn ground will occur. And the different undulation degrees of the lawn ground will affect the acquisition of inertial data by the inertial measurement unit on the self-moving device, which will easily cause inaccurate positioning of the self-moving device. This is a challenge for the self-moving device.

[0157] In one embodiment, in combination with Figure 2 and Figure 3 , taking the attitude sensor 140 as an example of the inertial measurement unit (Inertial Measurement Unit, IMU), it is disposed in the fuselage 110. The inertial measurement unit includes at least one of an accelerometer and a gyroscope; the controller 150 is electrically connected to the inertial measurement unit. The controller 150 is configured to control the self-moving device to walk; receive the inertial data of the inertial measurement unit; perform statistics on the inertial data received within the first time window, and adjust the positioning parameters of the inertial measurement unit according to the statistical results.

[0158] In combination with Figure 2 and Figure 3, the attitude sensor 140 and the controller 150 are respectively arranged on the body of the self - moving device, and the controller 150 is electrically connected to the attitude sensor 140.

[0159] Reference Figure 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, the self - moving device can be controlled to move by the controller 150, and the inertial data of the self - moving device can be collected by the inertial measurement unit. The acquisition unit 151 in the controller 150 can obtain the inertial data from the inertial measurement unit; the processing unit 152 can statistically process the inertial data received within the first time window by a preset method to obtain a statistical result, where the duration of the first time window is preset according to the actual situation; and adjust the positioning parameters of the inertial measurement unit according to the statistical result; and then determine the position of the self - moving device according to the inertial data and the positioning parameters.

[0160] The self - moving device can be positioned using inertial data. The surface bumpiness of the driving area of the self - moving device will affect the accuracy of the inertial measurement unit in collecting inertial data, and thus affect the positioning accuracy of the self - moving device. The statistical result of the inertial data can reflect the surface bumpiness. Therefore, in this solution, the positioning parameters of the inertial data are adjusted according to the statistical result of the inertial data of the self - moving device to minimize the influence of surface bumpiness on positioning as much as possible, and then the self - moving device is positioned through the inertial data and the adjusted positioning parameters, which can improve the positioning accuracy.

[0161] In one implementable manner, the statistical result reflects the flatness of the road section passed by the self - moving device within the first time window.

[0162] The higher the flatness of the road section, the flatter the road section and the smaller the bumpiness.

[0163] In one implementable manner, the inertial data collected by the inertial measurement unit at least includes the acceleration and angular velocity of the self - moving device. Then, statistically processing the inertial data received within the first time window by a preset method to obtain a statistical result includes: respectively integrating the acceleration and angular velocity in the inertial data received within the first time window to obtain the velocity and angle respectively; then respectively performing smoothing processing on the acceleration, angular velocity, velocity and angle to obtain the acceleration processing result, angular velocity processing result, velocity processing result and angle processing result; determining the first variance between each acceleration and the acceleration processing result, determining the second variance between the angular velocity and the angular velocity processing result, determining the third variance between the velocity and the velocity processing result, and determining the fourth variance between the angle and the angle processing result; and determining the first variance, second variance, third variance and fourth variance as the statistical result.

[0164] Among them, smoothing the acceleration to obtain the acceleration processing result includes: processing the acceleration by using a sliding window smoothing method to obtain the acceleration processing result. For example, averaging the accelerations collected within the first time window and using the average value as the acceleration processing result.

[0165] Similarly, smoothing the angular velocity to obtain the angular velocity processing result; smoothing the velocity to obtain the velocity processing result; smoothing the angle to obtain the angle processing result.

[0166] Among them, determining the first variance between each acceleration and the acceleration processing result includes: determining the square of the difference between each acceleration and the acceleration processing result, and taking the weighted sum of each square as the first variance. Among them, different weight values can be adopted for the accelerations in different sections, and the weight values corresponding to different sections can be set by methods such as manual parameter adjustment or machine learning.

[0167] Similarly, the second variance between the angular velocity and the angular velocity processing result can be determined; the third variance between the velocity and the velocity processing result can be determined; the fourth variance between the angle and the angle processing result can be determined.

[0168] Smoothing processing can obtain the approximate mean value in dynamic motion, so as to count the bumps in the motion process. The variance can better show the distribution difference of the data near the mean value. The smaller the distribution difference, the smaller the variance, and the larger the distribution difference, the larger the variance.

[0169] In one implementable manner, it is possible to first determine whether the positioning parameters of the inertial measurement unit need to be adjusted according to the statistical results. If it is determined that adjustment is needed, the positioning parameters are adjusted; if it is determined that adjustment is not needed, the positioning parameters are not adjusted.

[0170] This solution can determine whether the positioning parameters need to be adjusted based on the statistical results, and only adjust the positioning parameters when the conditions are met, thereby improving the positioning accuracy to a certain extent.

[0171] Specifically, a preset range can be set according to experience, and the statistical result is compared with the preset range. If it is determined that the statistical result is within the preset range, it is determined that the positioning parameters need to be adjusted. This method can quickly and accurately determine whether the positioning parameters need to be adjusted.

[0172] Alternatively, a binary classification model can be trained using the statistical results for training. The classification result of the binary classification model is used to characterize whether the positioning parameters of the inertial measurement unit need to be adjusted. Then, the trained binary classification model can be used to process the statistical results to obtain the classification result. If the classification result indicates that the positioning parameters need to be adjusted, the positioning parameters are adjusted; if the classification result indicates that the positioning parameters do not need to be adjusted, the positioning parameters are not adjusted. This method can quickly and accurately determine whether the positioning parameters need to be adjusted.

[0173] Specifically, a mapping table can be preset according to experience. The mapping table includes the mapping relationship between the flatness and the adjustment parameters. Furthermore, the mapping table can be queried using the statistical results to obtain the adjustment parameters corresponding to the statistical results, and the positioning parameters of the inertial measurement unit are updated using the adjustment parameters. This method can quickly and accurately adjust the positioning parameters.

[0174] Alternatively, the statistical results are processed using a preset regression model to obtain a regression result. Then, the positioning parameters are updated using the regression result. This method can quickly and accurately adjust the positioning parameters.

[0175] In one implementable manner, the first time window is determined according to the walking speed of the self-mobile device.

[0176] Specifically, the faster the walking speed of the self-mobile device, the shorter the duration of the first time window.

[0177] In one implementable manner, the value range of the first time window is greater than or equal to the first duration threshold and less than the second duration threshold; wherein, the first duration threshold is less than the second duration threshold.

[0178] Among them, both the first duration threshold and the second duration threshold are preset according to experience. The first time window can take any value between the first duration threshold and the second duration threshold.

[0179] The statistical results obtained by statistically analyzing a sufficient amount of inertial data can more accurately characterize the flatness of the road section. The method of setting the first time window in this solution is a relatively convenient and accurate way to determine whether the acquired inertial data is sufficient.

[0180] In one implementable manner, the positioning parameter is the noise of the inertial data.

[0181] Specifically, the surface roughness of the driving area of the self - moving device will affect the accuracy of the inertial measurement unit in collecting inertial data. The positioning parameter can be used as the noise of the inertial data to characterize the accuracy of the inertial data. The statistical result of the inertial data can reflect the surface roughness. Therefore, in this solution, the positioning parameter of the inertial data is adjusted according to the statistical result of the inertial data to minimize the impact of surface roughness on positioning as much as possible, which can improve the positioning accuracy.

[0182] In an implementable manner, the positioning parameter is the weight of the inertial data when the inertial data is fused with other modality data for positioning.

[0183] In this application, a lidar and a camera can also be set on the self - moving device. The self - moving device can be positioned by multi - modality fusion using inertial data and other modality data. Specifically, different types of data, such as inertial data, point cloud data, and image data, can be obtained through sensors such as the inertial measurement unit, lidar, and camera. These data are processed by a multi - modality fusion algorithm to achieve more accurate positioning of the self - moving device. Among them, the multi - modality fusion algorithm includes the weighted sum of multi - modality data, and the positioning parameter can be the weight of the inertial data when it is fused with other modality data for positioning. The rougher the surface of the driving area of the self - moving device is, the less accurate the inertial data obtained by the inertial measurement unit is. Therefore, when the surface is relatively rough, by reducing the weight of the inertial data in the fusion positioning, the accuracy of the fusion positioning can be improved to a certain extent. And the statistical result of the inertial data can reflect the surface roughness. Therefore, in this solution, the positioning parameter of the inertial data of the self - moving device is adjusted according to the statistical result of the inertial data, and the positioning of the self - moving device calculated using the adjusted positioning parameter is more accurate.

[0184] In one implementable manner, the inertial measurement unit can be shock-absorbed to reduce the impact on the inertial measurement unit caused by the unevenness of the driving section of the self-mobile device. Specifically, the shock-absorption measures for the inertial measurement unit can be implemented from two aspects: design and structure. In terms of design, in order to reduce the impact of vibration and shock on the inertial measurement unit, the adaptability of the inertial measurement unit can be improved through material selection and reasonable structural design. For example, select high-precision next-generation inertial measurement units, such as MPU6050, BMI055, etc. These chips have high adaptability and stability, and can reduce the impact of body vibration on the measurement accuracy of the inertial measurement unit. In terms of structure, a series of shock-absorption designs can be adopted, such as using shock-absorbing materials, such as silica gel, rubber, silicone rubber, etc., and shock-absorbing structures to isolate vibration. Specifically, by reasonably arranging the structure of the inertial measurement unit circuit board, the center of gravity of the chip and the center of gravity of the inertial measurement unit can be at the same point to achieve the best shock-absorption effect. In addition, shock absorbers and air dampers and other shock-absorbing devices can also be considered. These devices can effectively reduce the impact of body vibration on the inertial measurement unit.

[0185] Figure 11 FIG. 4 is a flowchart of a positioning method for a self-mobile device provided by an embodiment of the present application. This embodiment is applicable to the positioning scenario of the self-mobile device during driving, and this embodiment is particularly applicable to the positioning scenario where the self-mobile device is driving on a bumpy section. This method can be executed by the self-mobile device. As Figure 11 shown, the method includes:

[0186] Step 1101, obtain a positioning request, and obtain inertial data of the self-mobile device within a first time window according to the positioning request.

[0187] Step 1102, perform statistics on the inertial data, and adjust the positioning parameters corresponding to the inertial data according to the statistical results.

[0188] Step 1103, determine the position of the self-mobile device according to the inertial data and the positioning parameters.

[0189] The bumpy condition of the ground surface in the driving area of the self-mobile device will affect the accuracy of the inertial data collected by the inertial measurement unit, and further affect the positioning accuracy of the self-mobile device. The statistical result of the inertial data can reflect the bumpy condition of the ground surface. Therefore, in this solution, the positioning parameters of the inertial data are adjusted according to the statistical result of the inertial data of the self-mobile device to minimize the impact of the ground surface bump on the positioning as much as possible, and then the self-mobile device is positioned through the inertial data and the adjusted positioning parameters, which can improve the positioning accuracy.

[0190] The embodiment of the present application also specifically provides a positioning parameter adjustment algorithm based on inertial measurement unit data statistics for a lawn mower. Figure 12It is a flowchart of a positioning parameter adjustment algorithm based on inertial measurement unit data statistics provided by an embodiment of the present application. Refer to Figure 12 , the specific process of this algorithm is as follows:

[0191] Step 1201, collect inertial data through the inertial measurement unit in the lawn mower. The inertial data includes but is not limited to acceleration, angular velocity, etc.

[0192] Step 1202, determine whether the inertial data is collected sufficiently. The judgment basis includes but is not limited to status information such as the time of collecting data, the distance traveled by the lawn mower, and whether the lawn mower is mowing grass. Since the vibration generated by the lawn mower during mowing work will also affect the inertial measurement unit, and this influence is easily confused with the influence of road surface bumps on the inertial measurement unit, it is necessary to collect inertial data when the lawn mower is not mowing grass. If the inertial data is insufficient, the adjustment of the positioning parameters based on the inertial data will not be accurate. Therefore, it is necessary to collect sufficient inertial data. If the collection time length of the inertial data reaches the preset time length threshold, or the distance traveled by the lawn mower during the collection of inertial data reaches the preset distance threshold, it can be determined that the inertial data is collected sufficiently. When the inertial data is sufficient, proceed to step 1203; if it is insufficient, continue to collect.

[0193] Step 1203, use an algorithm to process the collected inertial data to obtain statistical indicators of the inertial data. The algorithm includes but is not limited to machine learning methods or statistical learning methods, etc. Specifically, the inertial data includes at least acceleration and angular velocity. Integrate the inertial data to obtain velocity data and angle data. Use the sliding window smoothing method to smooth the inertial data, velocity data, and angle data, and respectively calculate the variances of the inertial data, velocity data, and angle data based on the smoothed data. Different weighting weights are adopted for data in different numerical ranges, and the weight values can be set by methods such as manual parameter adjustment or machine learning methods.

[0194] Step 1204, based on the series of statistical indicators obtained in step 1203, use a qualitative method to determine whether parameter adjustment is required. The qualitative method includes but is not limited to the empirical judgment method or the machine learning binary classification method.

[0195] Step 1205, for the scenarios that require parameter adjustment in step 1204, according to the obtained series of statistical indicators, use the look-up table interpolation method or other machine learning regression methods to adjust the positioning parameters of the inertial measurement unit.

[0196] Figure 13 It is a flowchart of a model training method applied to lawn image processing provided by an embodiment of the present application. This embodiment is applicable to the model training scenario of identifying grass or non-grass objects in a lawn image, and this method can be executed by an electronic device. As Figure 13 shown, this method includes:

[0197] Step 1301: Obtain the dataset to be trained; among them, the dataset to be trained includes multiple lawn images to be trained; the lawn images to be trained have annotation types.

[0198] Specifically, a vision sensor can be used to obtain the lawn images to be trained, and the lawn images to be trained can be processed such as cropped, sampled, and annotated. Among them, the lawn images to be trained can include information such as the ground, objects, and sky within the field of view of the vision sensor.

[0199] It is also possible to obtain the lawn images to be trained with annotation types publicly available on the network. Among them, the annotation types can include the object types within the field of view of the vision sensor, such as grass, pedestrians, trees, houses, sky, and ground, etc.

[0200] Step 1302: Add noise and / or interference to at least some of the lawn images to be trained in the dataset to be trained to obtain the processed lawn images; and add the processed lawn images as new lawn images to be trained to the dataset to be trained to update the dataset to be trained.

[0201] Various methods can be used to add noise and / or interference to the lawn images to be trained to obtain the processed lawn images, so as to enrich the training samples and simulate as many possible situations in the actual scenario as possible. Specifically, refer to Figure 7 and Figures 14A to 14C , the lawn images to be trained can be blurred, including Gaussian blur, mean blur, and median blur. Among them, Figure 7 can represent the lawn images to be trained, Figure 14A is the Gaussian blur map obtained by performing Gaussian blur processing on the lawn images to be trained, Figure 14B is the mean blur map obtained by performing mean blur processing on the lawn images to be trained, Figure 14C is the median blur map obtained by performing median blur processing on the lawn images to be trained. Among them, Gaussian blur means blurring the image by convolving the image with a normal distribution. Mean blur mainly operates on each pixel point in the image and replaces its value with the average value of adjacent pixel points. Median blur mainly operates on each pixel point in the image and replaces its value with the median value of adjacent pixel points.

[0202] Step 1303: Process the lawn images to be trained using a preset model to obtain the predicted types of the lawn images to be trained; optimize the parameters of the preset model according to the annotation types and predicted types of the lawn images to be trained to obtain an identification model.

[0203] Among them, the preset model is a pre-set network model. This network model can be set based on a convolutional neural network. Specifically, the annotation type of the lawn image to be trained can be used as the label of the prediction type to train the preset model and optimize the parameters in the preset model, so that the prediction type is getting closer and closer to the annotation type. When the preset condition is reached, for example, the similarity between the prediction type and the annotation type reaches 90%, the training is stopped, and an identification model is obtained. The identification model can be used to process the lawn image to be identified and identify the target type of the lawn image to be identified.

[0204] In this solution, by adding noise and / or interference to the lawn image to be trained, the training samples are enriched, and as many possible situations in the actual scene as possible can be simulated. Using the richer training samples to train the identification model can improve the identification accuracy of the identification model to a certain extent.

[0205] In an implementable manner, at least some of the lawn images to be trained in the training dataset are subjected to one or more of the following combined processes to obtain the processed lawn images: blur processing, distortion processing, overexposure processing, adding pixel blocks, adding shadows of target objects, and adding preset objects.

[0206] Specifically, an image processing software or a blur filter in a programming language can be used to implement the blur processing of the lawn image to be identified. The blur processing methods include at least one of the following: motion blur, depth of field blur, rotation blur, zoom blur, Gaussian blur, mean blur, and median blur. Among them, motion blur is caused by the movement of objects or the camera in the image. To simulate this effect, a method of randomly offsetting or rotating pixels can be used to produce a blur effect. Depth of field blur is a blur effect caused by the change of the focal length of the camera lens. This effect can be simulated by blurring the background and foreground pixels in the image to different degrees. Rotation blur can be simulated by rotating the image and applying a filter. Zoom blur is a blur effect caused by the enlargement or reduction of the image. This effect can be simulated by applying a filter after scaling the image.

[0207] Specifically, image processing software or transformation functions in programming languages can be used to perform distortion processing on the lawn image to be recognized. The distortion processing methods include at least one of the following: distortion transformation, shear transformation, rotation transformation, skew transformation, and affine transformation. Among them, the distortion transformation is a transformation method that maps the pixels in the image and can distort or deform a part of the image. This transformation can usually be achieved by defining a transformation matrix. The shear transformation is a transformation method that displaces the pixels in the image along the vertical or horizontal direction. This transformation can be achieved by translating the pixels in the horizontal or vertical direction. The rotation transformation is a transformation method that rotates the image and can be achieved by defining the rotation center and rotation angle. The skew transformation is a transformation method that displaces the pixels in the image along a straight line. This transformation can be achieved by defining two displacement points. The affine transformation is a transformation method that maps the pixels in the image and can distort or deform a part of the image. This transformation can be achieved by defining an affine matrix.

[0208] Specifically, taking the visual sensor 130 as a camera as an example, for the camera, overexposure processing of the image can be achieved by increasing the exposure time or increasing the ISO sensitivity. The effect of overexposure can also be simulated by adjusting parameters such as the brightness, contrast, and color of the lawn image to be recognized. The overexposure processing methods include at least one of the following: increasing brightness, increasing contrast, color adjustment, highlight adjustment, and sharpening processing. Among them, increasing brightness means simulating the effect of overexposure by increasing the overall brightness of the image. It can be achieved using the brightness adjustment function in image processing software or programming languages. Increasing contrast means simulating the effect of overexposure by increasing the contrast of the image. It can be achieved using the contrast adjustment function in image processing software or programming languages. Color adjustment means simulating the effect of overexposure by adjusting the color of the image. It can be achieved using the color adjustment function in image processing software or programming languages. Highlight adjustment means simulating the effect of overexposure by increasing the highlight part of the image. It can be achieved using the highlight adjustment function in image processing software or programming languages. Sharpening processing means simulating the effect of overexposure by sharpening the edges and details of the image. It can be achieved using the sharpening function in image processing software or programming languages.

[0209] Specifically, solid color pixel blocks can be added to the lawn image to be trained.

[0210] Specifically, the shadows of objects that may appear on the lawn or the objects that may appear on the lawn can be added to the lawn image to be trained to enrich the training samples, and then a recognition model with higher recognition accuracy can be trained based on the more abundant training samples.

[0211] In one implementable way, the shadow of any one of the following target objects is added to the lawn image to be trained: trees, fences, and houses.

[0212] In one implementable manner, add any one of the following preset objects to the lawn image to be trained: water pipe, faucet, water gun, toy, animal feces, fallen leaves, and branches.

[0213] In one implementable manner, the lawn image to be trained includes a color image and / or a grayscale image.

[0214] In one embodiment, in combination Figure 2 and Figure 3 , the controller 150 is electrically connected to the vision sensor 130. The controller 150 is configured to obtain the lawn image to be recognized from the vision sensor 130, and use a preset recognition model to process the lawn image to be recognized to obtain the target type of the lawn image to be recognized. Among them, the preset recognition model is obtained by training a preset model using the lawn image to be trained and the processed lawn image; the lawn image to be trained has a labeled type; the processed lawn image is obtained by adding noise and / or interference to at least part of the lawn images to be trained.

[0215] Refer to Figure 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, the lawn image to be recognized can be collected by the vision sensor 130, and the acquisition unit 151 in the controller 150 obtains the lawn image to be recognized from the vision sensor 130; the processing unit 152 uses a preset recognition model to process the lawn image to be recognized to obtain the target type of the lawn image to be recognized.

[0216] In this solution, by adding noise and / or interference to the lawn image to be trained, the training samples are enriched, and the recognition accuracy of the recognition model trained with more abundant training samples is higher. Using a recognition model with higher recognition accuracy can improve the maintenance effect of the self-propelled lawn mower on the lawn to a certain extent.

[0217] Figure 15 is a flowchart of a lawn image processing method provided by an embodiment of the present application. This embodiment is applicable to the scenario of recognizing grass and non-grass objects in a lawn image when the self-propelled lawn mower 100 maintains the lawn. This method can be executed by the self-propelled lawn mower 100. As Figure 15 shown, this method includes:

[0218] Step 1501, obtain the lawn image to be recognized.

[0219] Step 1502: Process the lawn image to be recognized using a preset recognition model to obtain the target type of the lawn image to be recognized. Among them, the preset recognition model is obtained by training a preset model using the lawn images to be trained and the processed lawn images. The lawn images to be trained have labeled types. The processed lawn images are obtained by adding noise and / or interference to at least some of the lawn images to be trained.

[0220] In this solution, by adding noise and / or interference to the lawn images to be trained, the training samples are enriched, and the recognition accuracy of the recognition model trained with more abundant training samples is higher. Using a recognition model with higher recognition accuracy to identify grass and non-grass objects in the lawn image can improve the maintenance effect of the self-propelled lawn mower to a certain extent.

[0221] The embodiment of the present application also specifically provides a model training method based on data augmentation for the lawn mower. Figure 16 is a flowchart of a model training method based on data augmentation provided by the embodiment of the present application. Refer to Figure 16 and the specific process of this method is as follows:

[0222] Step 1601: Use the camera installed on the lawn mower to collect visual information data, such as color images, grayscale images, etc.

[0223] Step 1602: Screen and label the data. The labeling methods include but are not limited to manual labeling methods and automatic labeling methods.

[0224] Step 1603: Perform data augmentation on the labeled data. The main augmentation methods include noise augmentation and interference augmentation. The noise augmentation methods include but are not limited to blurring, distortion, overexposure, etc. The interference augmentation methods include but are not limited to adding light and shadow, solid color pixel blocks, objects that may appear on the lawn, etc.

[0225] Step 1604: Use the augmented data set after data augmentation to train the machine learning model.

[0226] Step 1605: After training, obtain an algorithm model that can distinguish grass and non-grass areas.

[0227] Figure 2 is a schematic structural diagram of a self-propelled lawn mower 100 provided by the embodiment of the present application. Figure 17 is a schematic diagram of a reference object 200 provided by the embodiment of the present application. Refer to Figure 2 and Figure 17, an embodiment of the present application provides a self - moving lawn mower system, including: a self - moving lawn mower 100 and a reference object 200; the self - moving lawn mower 100 includes: a body 110; a walking wheel assembly 120 configured to support the body; a vision sensor 130 mounted on the body and configured to acquire images around the self - moving lawn mower; combined with Figure 2 and Figure 3 , a controller 150 is electrically connected to the vision sensor 130 and identifies semantic information around the self - moving lawn mower 100 through image recognition; the reference object 200 is used to provide a reference for the controller, and the reference object 200 simulates at least one of a lawn, land, water surface, and sky.

[0228] Combined with Figure 2 and Figure 17 , the reference object can be installed in an area that can be captured by the vision sensor 130, such as on a wall, a tree trunk, a charging pile, etc.

[0229] Referring to Figure 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, images around the self - moving lawn mower 100 can be collected by the vision sensor 130, and the acquisition unit 151 in the controller 150 obtains the images from the vision sensor 130; the processing unit 152 obtains the reference information of the reference object 200 and identifies the semantic information around the self - moving lawn mower 100 through the reference information and the images. Specifically, images of the reference object 200 can be collected by the vision sensor 130, the acquisition unit 151 obtains the images of the reference object 200 from the vision sensor 130, and extracts the reference information of the reference object 200 from the images; the acquisition unit 151 can also obtain the real information of the reference object 200; the processing unit 152 performs transformation processing on the images around the self - moving lawn mower 100 according to the real information and the reference information of the reference object 200 to obtain transformed images; the transformed images are processed by a semantic recognition program to obtain the semantic information around the self - moving lawn mower 100. Among them, the semantic recognition program can be a machine - learning program, such as an intelligent recognition model based on machine learning; the semantic information includes at least grass, grass species, and non - grass objects, etc.

[0230] In the process of identifying the semantic information around the self - moving lawn mower 100 through image recognition in this solution, the reference information of the reference object is combined to reduce the deviation between the visual information and the real information on the image, and thus the accuracy of semantic information recognition can be improved.

[0231] In an implementable manner, referring to Figure 17 , the self - moving lawn mower system further includes a charging pile 300, and the reference object 200 is arranged on the charging pile 300.

[0232] Setting the reference object 200 at the charging pile 300 reduces the installation steps when the reference object 200 and the charging pile 300 are installed separately.

[0233] In one implementable manner, the reference object 200 has geometric features, and the geometric features are used to assist the controller 150 in identifying the reference object 200.

[0234] The processing unit 152 extracts geometric features from the images around the mobile mower 100. If the geometric features are target geometric features, the target images corresponding to the geometric features are determined as the reference object 200, and the reference information of the reference object 200 is extracted. The reference object 200 can be identified more conveniently and quickly through the geometric features. Among them, the target geometric features can be planar geometric shape features or three-dimensional geometric body features. The target geometric features include but are not limited to object features such as cubes and spheres with a certain special visual recognition structure. Specifically, at least one of the following methods can be used to extract geometric features from the images: edge extraction and feature point extraction. Since the edges are the regions with the most obvious local changes in the images, the geometric features of the images can be extracted by detecting these regions. Edge extraction usually uses the extreme value regions of the first derivative to detect edges, that is, by calculating the derivative of the image function in a certain direction and finding the regions where the derivative has extreme value changes, and these regions are the edges of the images. Feature point extraction refers to finding the feature points in the images, that is, those points with obvious changes or special shapes in the images. For example, corner points are the intersections of two edges in the images, and they usually have a larger curvature value than the surrounding regions, so the geometric features can be extracted by detecting these points. Similarly, key points and blobs are also regions with significant features in the images, and the geometric features can be extracted through these regions. Once the geometric features in the images are detected, descriptors can be used to describe these features. The descriptors can be gray-level co-occurrence matrices, histograms of oriented gradients, scale-invariant feature transforms, speeded-up robust features, etc. These descriptors can be used to compare and match the feature points in the images for tasks such as image recognition, classification, and retrieval.

[0235] In one implementable manner, the controller 150 is configured to identify the reference object 200 through geometric features, extract the color features of the reference object 200, and perform transformation processing on the images according to the color features to obtain transformed images.

[0236] The reference information provided by the reference object 200 to the controller 150 may be a color feature. The color feature of the reference object 200 includes, but is not limited to, colors similar to natural references, such as lawn color and sky color. After the processing unit 152 in the controller 150 identifies the reference object 200 through geometric features, it can extract the color feature of the reference object 200; the acquisition unit 151 in the controller 150 can also acquire the real color information of the reference object 200; the processing unit 152 can acquire the difference between the extracted color feature of the reference object 200 and the real color information; and then subtract the difference from the color information of the image to obtain a transformed image.

[0237] Color features are important features of images. Changes in the outdoor working environment, such as changes in light, cause a deviation between the visual information on the image and the real information, which mainly affects the color features. Therefore, the difference between the extracted color features of the reference object 200 and the real color features is used as a transformation benchmark to transform the color features of the image in order to reduce the deviation between the visual information on the image and the real information. Using the transformed image for intelligent recognition can improve recognition accuracy.

[0238] Figure 18 1 is a flow chart of a control method for a self-propelled lawn mower provided in an embodiment of the present application. This embodiment can be applied to a scene in which an image around the self-propelled lawn mower 100 is intelligently recognized when the self-propelled lawn mower 100 is maintaining a lawn. The method can be executed by the self-propelled lawn mower 100. The self-propelled lawn mower 100 includes a visual sensor 130, such as Figure 18 As shown, the method includes:

[0239] Step 1801 , obtain an image of the reference object 200 through the visual sensor 130 and extract reference information.

[0240] Step 1802 : Acquire an image of the surroundings of the self-propelled lawn mower 100 through the visual sensor 130 .

[0241] Step 1803 , transform the image around the self-propelled lawn mower 100 according to the reference information to obtain a transformed image.

[0242] Step 1804 , the transformed image is processed by a semantic recognition program to obtain semantic information of the surroundings of the self-propelled lawn mower 100 .

[0243] This scheme uses the reference information of the reference object to transform the image around the self-propelled lawn mower to obtain a transformed image, which minimizes the deviation between the visual information on the image and the real information, and then uses the transformed image for intelligent recognition to improve the recognition accuracy.

[0244] In one possible implementation, the reference object is a known object in a natural environment.

[0245] The known object can be recognized by a semantic recognition program. The known object includes at least lawn, land, water surface, sky, etc.

[0246] In an implementable manner, the reference object is an artificially set simulated object.

[0247] The embodiment of the present application also provides a method for determining a feature transformation reference based on lawn markers for a lawn mower equipped with a camera. Figure 19 It is a flowchart of a method for establishing a feature transformation reference based on lawn markers provided by the embodiment of the present application. Refer to Figure 19 , and the specific process of this method is as follows:

[0248] Step 1901, the lawn mower is powered on and starts mowing the lawn.

[0249] Step 1902, during the mowing process, visual information on the lawn is collected through the camera.

[0250] Step 1903, according to the preset geometric information of the marker, search for it in the image.

[0251] Step 1904, extract relevant visual information from the marker in the searched image.

[0252] Step 1905, obtain a transformation method for visual information based on the preset visual information of the marker and the extracted visual information.

[0253] Step 1906, use the obtained transformation method for visual information to correct the entire image.

[0254] Step 1907, the corrected image is used as the input of the visual processing algorithm for visual algorithm processing.

[0255] Figure 20The structural schematic diagram of an electronic device 10 that can be used to implement the embodiments of the present application is shown. The electronic device 10 can be a controller in a self-propelled lawn mower 100, which can be installed on the body of the self-propelled lawn mower 100 and communicate locally with a vision sensor 130 and an attitude sensor 140 in the self-propelled lawn mower 100; the electronic device 10 can also be a remote server that interacts remotely with the vision sensor 130 and the attitude sensor 140 in the self-propelled lawn mower 100. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.). The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0256] As Figure 20 shown, the electronic device 10 includes at least one processor 11 and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0257] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0258] The processor 11 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above.

[0259] In some embodiments, any of the above methods may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of any of the methods described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute any of the above methods by any other suitable means (e.g., by means of firmware).

[0260] It should be understood that various forms of the processes shown above may be used, with steps reordered, added, or deleted. For example, the steps recited in this application may be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved, and no limitation is made herein.

[0261] The basic principles, main features, and advantages of this application have been shown and described above. Those skilled in the art of this industry should understand that the above embodiments do not limit this application in any form. Any technical solution obtained by using an equivalent replacement or equivalent transformation falls within the protection scope of this application.

Claims

1. A self - propelled lawn mower, characterized in that, it includes: a fuselage; a walking wheel assembly configured to support the fuselage; a vision sensor configured to acquire a lawn image of the working area where the self - propelled lawn mower is located; a controller electrically connected to the vision sensor, the controller being configured to receive the lawn image of the working area acquired by the vision sensor according to an inspection instruction; process the lawn image to obtain feature information of the grass; wherein, the feature information at least includes the height and color of the grass; formulate a mowing plan or an inspection plan according to the feature information, and control the operation of the self - propelled lawn mower according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time for performing the mowing task; the inspection plan includes the time for generating the next inspection instruction.

2. A control method for a self - propelled lawn mower, characterized in that, applied to a self - propelled lawn mower, the method includes: acquire an inspection instruction, and acquire a lawn image of the working area of the self - propelled lawn mower according to the inspection instruction; process the lawn image to obtain feature information of the grass; wherein, the feature information at least includes the height and color of the grass; formulate a mowing plan or an inspection plan according to the feature information, and control the operation of the self - propelled lawn mower according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time for performing the mowing task; the inspection plan includes the time for generating the next inspection instruction.

3. The method according to claim 2, characterized in that, the formulating a mowing plan or an inspection plan according to the feature information includes: determine a mowing height threshold according to the color; if the height is greater than the mowing height threshold, control the self - propelled lawn mower to perform a mowing task; if the height is less than or equal to the mowing height threshold, formulate a mowing plan or an inspection plan according to the difference between the mowing height threshold and the height.

4. The method according to claim 3, characterized in that, the feature information further includes size, shape and / or density; then the determining a mowing height threshold according to the color includes: acquire season information, and determine the type of grass according to the season information, the size, the shape and the color; determine the mowing height threshold according to the type, the density and the season information.

5. The method according to claim 3, characterized in that, the formulating a mowing plan or an inspection plan according to the difference between the mowing height threshold and the height includes: if the difference is less than or equal to a preset threshold, formulate a mowing plan according to the difference; if the difference is greater than the preset threshold, formulate an inspection plan according to the difference.

6. The method according to claim 2, characterized in that, the height is determined according to the point cloud data of the grass; the point cloud data is determined according to the lawn image.

7. A control device for a self - propelled lawn mower, characterized in that, applied to a self - propelled lawn mower, the device includes: An acquisition unit, configured to acquire an inspection instruction and acquire a lawn image of the working area of the self-propelled lawn mower according to the inspection instruction; A processing unit, configured to process the lawn image to obtain grass feature information; wherein, the feature information at least includes the height and color of the grass; The processing unit is further configured to formulate a mowing plan or an inspection plan according to the feature information; A control unit, configured to control the operation of the self-propelled lawn mower according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time for performing the mowing task; the inspection plan includes the time for generating the next inspection instruction.

8. The device according to claim 7, wherein, The processing unit is specifically configured to: Determine a mowing height threshold according to the color; If the height is greater than the mowing height threshold, control the self-propelled lawn mower to perform a mowing task; If the height is less than or equal to the mowing height threshold, formulate a mowing plan or an inspection plan according to the difference between the mowing height threshold and the height.

9. An electronic device, wherein, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the control method of the self-propelled lawn mower according to any one of claims 2-6.

10. A computer-readable storage medium, wherein, The computer-readable storage medium stores computer instructions for causing a processor to implement the control method of the self-propelled lawn mower according to any one of claims 2-6 when executed.

Images