Self-moving device and control method

Abstract

A self-moving device and a control method. The self-moving device comprises: a body (110); traveling wheel assemblies (120), configured to support the body (110); a visual sensor (130), configured to acquire an image in a traveling direction of the self-moving device; an attitude sensor (140), configured to acquire an attitude of the self-moving device; and a controller (150), electrically connected to the visual sensor (130) and the attitude sensor (140), the controller (150) being configured to calculate a first inclination angle of the ground in the traveling direction of the self-moving device on the basis of the image, calculate a second inclination angle of the ground on the basis of the first inclination angle and the attitude, and control the traveling wheel assemblies (120) on the basis of the second inclination angle.

Description

Self-contained equipment and control method

[0001] This application claims priority to the Chinese patent application with application number 202311574467.2 filed with the China Patent Office on November 22, 2023, priority to the Chinese patent application with application number 202311571279.4 filed with the China Patent Office on November 22, 2023, priority to the Chinese patent application with application number 202311572907.0 filed with the China Patent Office on November 22, 2023, priority to the Chinese patent application with application number 202311574454.5 filed with the China Patent Office on November 22, 2023, and priority to the Chinese patent application with application number 202311572961.5 filed with the China Patent Office on November 22, 2023. The entire contents of the above applications are incorporated by reference into this application. Technical Field

[0002] The present application relates to a device and a control method, for example, to a self-propelled device and a control method. Background Art

[0003] If a smart lawn mower tilts too far while mowing, it can easily tip over, causing downtime or exposing the cutter disc, creating an unsafe situation. Therefore, smart lawn mowers need to be able to identify slopes.

[0004] In the related art, the slope angle can be identified by using images in the direction of travel collected by the smart lawn mower.

[0005] However, the accuracy of slope angle recognition using this approach needs to be further improved when dealing with user lawns with diverse terrain. For example, traversing the same slope from different routes may cause the smart mower to tilt at different angles. If the tilt angle exceeds a certain range, the mower may easily tip over.

[0006] This section provides background information related to the present application which is not necessarily prior art.

[0007] Summary of the Invention

[0008] An object of the present application is to solve or at least alleviate some or all of the above problems.

[0009] To this end, one object of the present application is to provide an intelligent lawn mower that has a higher slope angle recognition accuracy when dealing with user lawns with diverse terrain.

[0010] In order to achieve the above objectives, this application adopts the following technical solutions:

[0011] A self-propelled device comprising:

[0012] body;

[0013] a running wheel assembly configured to support the fuselage;

[0014] a visual sensor configured to acquire an image in a direction of travel of the self-moving device;

[0015] A posture sensor configured to obtain the posture of the self-moving device;

[0016] A controller is electrically connected to the visual sensor and the posture sensor, and the controller is configured to calculate a first inclination angle of the ground in the direction of travel of the self-moving device based on the image, calculate a second inclination angle of the ground based on the first inclination angle and the posture, and control the walking wheel assembly based on the second inclination angle.

[0017] In some embodiments, the first tilt angle is a relative tilt angle of the ground relative to the self-moving device; and the second tilt angle is an absolute tilt angle of the ground relative to the direction of gravity.

[0018] In some embodiments, the visual sensor includes at least one of a TOF depth camera, an IR structured light camera, and a binocular camera.

[0019] In some embodiments, the controller creates a depth image from the image, extracts a ground point cloud from the depth image, and obtains the first tilt angle through the ground point cloud.

[0020] In some embodiments, when the second tilt angle is greater than or equal to a first threshold, the controller determines that the ground in the direction of travel of the self-moving device is a non-safe area and changes the direction of travel.

[0021] In some embodiments, when the second tilt angle is less than a first threshold, the controller determines that the ground in the direction of travel of the self-moving device is a safe area and maintains the direction of travel.

[0022] In some embodiments, the first threshold is determined according to the friction of the ground.

[0023] In some embodiments, the moving direction of the self-moving device is the front of the self-moving device.

[0024] In some embodiments, the gesture sensor is an accelerometer.

[0025] A method for controlling a self-propelled device, comprising:

[0026] Acquire the posture of the self-moving device; and acquire an image in the moving direction of the self-moving device;

[0027] Calculating a first inclination angle of the ground in the direction of travel of the self-moving device according to the image;

[0028] Combining the first tilt angle and the posture, a second tilt angle of the ground is calculated; and determining whether to move along the moving direction is determined according to the second tilt angle.

[0029] A control device for a mobile device, comprising:

[0030] An acquisition unit configured to acquire the posture of the self-moving device; and acquire an image in the moving direction of the self-moving device;

[0031] a processing unit configured to calculate a first tilt angle of the ground in the direction of travel of the self-moving device based on the image;

[0032] The processing unit is further configured to calculate a second tilt angle of the ground by combining the first tilt angle and the posture;

[0033] The control unit is configured to determine whether to move along the moving direction according to the second inclination angle.

[0034] An electronic device, comprising:

[0035] at least one processor; and

[0036] a memory communicatively connected to the at least one processor; wherein,

[0037] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the control method of the mobile device described in any embodiment of the present application.

[0038] A computer-readable storage medium stores computer instructions, which are used to enable a processor to implement the control method of a mobile device described in any embodiment of the present application when executed.

[0039] The benefit of the present application is that: the second tilt angle is obtained according to the first tilt angle in combination with the posture of the self-moving device, thereby improving the accuracy of slope angle recognition.

[0040] To this end, one object of the present application is to provide a self-propelled lawn mower that can perform more intelligent maintenance on the lawn even when it is out of user control for a long time.

[0041] In order to achieve the above objectives, this application adopts the following technical solutions:

[0042] A self-propelled lawn mower comprising:

[0043] body;

[0044] a running wheel assembly configured to support the fuselage;

[0045] a visual sensor configured to acquire an image of a lawn in a working area where the self-propelled lawn mower is located;

[0046] a controller electrically connected to the visual sensor, the controller being configured to obtain and receive, according to an inspection instruction, an image of the lawn of the working area obtained by the visual sensor;

[0047] Processing the lawn image to obtain characteristic information of the grass; wherein the characteristic information includes at least the height and color of the grass;

[0048] A mowing plan or an inspection plan is formulated according to the characteristic information, and the self-propelled lawn mower is controlled to operate according to the mowing plan or the inspection plan; wherein the mowing plan includes the time for executing the mowing task; and the inspection plan includes the time for generating the next inspection instruction.

[0049] A control method for a self-propelled lawn mower is applied to the self-propelled lawn mower, the method comprising:

[0050] Obtaining an inspection instruction, and obtaining a lawn image of a working area of ​​the self-propelled lawn mower according to the inspection instruction;

[0051] Processing the lawn image to obtain characteristic information of the grass; wherein the characteristic information includes at least the height and color of the grass;

[0052] A mowing plan or an inspection plan is formulated according to the characteristic information, and the self-propelled lawn mower is controlled to operate according to the mowing plan or the inspection plan; wherein the mowing plan includes the time for executing the mowing task; and the inspection plan includes the time for generating the next inspection instruction.

[0053] In some embodiments, formulating a mowing plan or an inspection plan based on the feature information includes:

[0054] determining a mowing height threshold based on the color;

[0055] If the height is greater than the mowing height threshold, controlling the self-propelled lawn mower to perform a mowing task;

[0056] If the height is less than or equal to the mowing height threshold, a mowing plan or an inspection plan is formulated according to the difference between the mowing height threshold and the height.

[0057] In some embodiments, the characteristic information further includes size, shape, and / or density; determining the mowing height threshold based on the color includes:

[0058] acquiring season information, and determining the type of grass according to the season information, the size, the shape, and the color;

[0059] A mowing height threshold is determined according to the type, the density, and the season information.

[0060] In some embodiments, formulating a mowing plan or an inspection plan based on the difference between the mowing height threshold and the height includes:

[0061] If the difference is less than or equal to a preset threshold, formulating a mowing plan according to the difference;

[0062] If the difference is greater than a preset threshold, an inspection plan is formulated based on the difference.

[0063] In some embodiments, the height is determined based on point cloud data of grass; and the point cloud data is determined based on the lawn image.

[0064] A control device for a self-propelled lawn mower, applied to the self-propelled lawn mower, comprising:

[0065] an acquisition unit configured to acquire an inspection instruction and acquire a lawn image of a working area of ​​the self-propelled lawn mower according to the inspection instruction;

[0066] a processing unit configured to process the lawn image to obtain characteristic information of the grass; wherein the characteristic information includes at least the height and color of the grass;

[0067] The processing unit is further configured to formulate a mowing plan or an inspection plan based on the feature information;

[0068] A control unit is configured to control the self-propelled lawn mower to operate according to the mowing plan or the inspection plan; wherein the mowing plan includes the time for performing the mowing task; and the inspection plan includes the time for generating the next inspection instruction.

[0069] In some embodiments, the processing unit is specifically configured to:

[0070] determining a mowing height threshold based on the color;

[0071] If the height is greater than the mowing height threshold, controlling the self-propelled lawn mower to perform a mowing task;

[0072] If the height is less than or equal to the mowing height threshold, a mowing plan or an inspection plan is formulated according to the difference between the mowing height threshold and the height.

[0073] An electronic device, comprising:

[0074] at least one processor; and

[0075] a memory communicatively connected to the at least one processor; wherein,

[0076] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the control method of the self-moving lawn mower described in any embodiment of the present application.

[0077] A computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable a processor to implement the control method of the self-propelled lawn mower described in any embodiment of the present application when executed.

[0078] The benefit of the present application lies in that: since different types of grass usually have different colors, and the colors of the same type of grass usually also have certain differences in different seasons, the suitable mowing heights for different types of grass usually have certain differences, and the suitable mowing heights for the same type of grass in different seasons usually also have certain differences. Therefore, the present solution identifies the growth status of the grass based on at least characteristic information such as the height and color of the grass. For example, the type of grass and the season can be determined based on the color, and a mowing plan or inspection plan for the self-propelled lawn mower can be formulated according to the type, height and season of the grass, which makes lawn maintenance more intelligent and can improve the quality of lawn maintenance.

[0079] To this end, one object of the present application is to provide a self-propelled device with higher positioning accuracy.

[0080] In order to achieve the above objectives, this application adopts the following technical solutions:

[0081] A self-propelled device comprising:

[0082] body,

[0083] a running wheel assembly configured to support the fuselage;

[0084] an inertial measurement unit disposed in the fuselage, the inertial measurement unit comprising at least one of an accelerometer and a gyroscope;

[0085] A controller is electrically connected to the inertial measurement unit, and the controller is configured to:

[0086] Controlling the self-moving device to move;

[0087] receiving inertial data from the inertial measurement unit;

[0088] Statistics are performed on the inertial data received within the first time window, and positioning parameters of the inertial measurement unit are adjusted according to the statistical results.

[0089] In some embodiments, the statistical result reflects the smoothness of the road segment passed by the mobile device within the first time window.

[0090] In some embodiments, the positioning parameter is noise in the inertial data.

[0091] In some embodiments, the positioning parameter is a weight of the inertial data when the inertial data is fused with data of other modalities for positioning.

[0092] In some embodiments, the value range of the first time window is greater than or equal to a first duration threshold and less than or equal to a second duration threshold;

[0093] The first duration threshold is smaller than the second duration threshold.

[0094] In some embodiments, the first time window is determined according to the walking speed of the self-moving device.

[0095] A positioning method for a mobile device, comprising:

[0096] Obtaining a positioning request, and obtaining inertial data of the mobile device within a first time window according to the positioning request;

[0097] Performing statistics on the inertial data, and adjusting positioning parameters corresponding to the inertial data according to the statistical results;

[0098] The position of the self-moving device is determined according to the inertial data and the positioning parameters.

[0099] A positioning device for a mobile device, comprising:

[0100] an acquiring unit, configured to acquire a positioning request, and acquire inertial data of the mobile device within a first time window according to the positioning request;

[0101] a processing unit configured to perform statistics on the inertial data and adjust positioning parameters corresponding to the inertial data according to the statistical results;

[0102] The processing unit is further configured to determine the position of the self-moving device based on the inertial data and the positioning parameters.

[0103] An electronic device, comprising:

[0104] at least one processor; and

[0105] a memory communicatively connected to the at least one processor; wherein,

[0106] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the positioning method for a mobile device described in any embodiment of the present application.

[0107] A computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable a processor to implement the positioning method for a mobile device described in any embodiment of the present application when executed.

[0108] The benefit of this application is that it can use inertial data to locate the self-moving device. The surface bumps in the area where the self-moving device is traveling will affect the accuracy of the inertial data collected by the inertial measurement unit, and thus affect the positioning accuracy of the self-moving device. The statistical results of the inertial data can reflect the surface bumps. Therefore, this solution adjusts the positioning parameters of the inertial data based on the statistical results of the inertial data of the self-moving device to minimize the impact of surface bumps on positioning. Then, the self-moving device can be positioned using the inertial data and the adjusted positioning parameters, which can improve positioning accuracy.

[0109] To this end, one purpose of this application is to provide a recognition model with higher recognition accuracy.

[0110] In order to achieve the above objectives, this application adopts the following technical solutions:

[0111] A model training method for lawn image processing, comprising:

[0112] Acquire a dataset to be trained; wherein the dataset to be trained includes a plurality of lawn images to be trained; the lawn images to be trained have a label type;

[0113] adding noise and / or interference to at least a portion of the lawn images to be trained in the training data set to obtain processed lawn images; and adding the processed lawn images as new lawn images to be trained into the training data set to update the training data set;

[0114] The lawn image to be trained is processed using a preset model to obtain a predicted type of the lawn image to be trained; and the parameters of the preset model are optimized according to the labeling type and the predicted type of the lawn image to be trained to obtain a recognition model.

[0115] In some embodiments, at least a portion of the lawn images to be trained in the training data set is subjected to one or more of the following combined processing to obtain processed lawn images:

[0116] Blur, distort, overexpose, add pixel blocks, add shadows to target objects, and add preset objects.

[0117] In some embodiments, the preset object includes one or more of the following combinations:

[0118] Pipes, taps, water guns, toys, animal droppings, fallen leaves, and branches.

[0119] In some embodiments, the target object includes one or more of the following combinations:

[0120] Trees, fences, and houses.

[0121] In some embodiments, the lawn image to be trained includes a color image and / or a grayscale image.

[0122] A lawn image processing method, applied to a self-propelled lawn mower, comprising:

[0123] Obtaining a lawn image to be identified;

[0124] Processing the lawn image to be identified using a preset recognition model to obtain a target type of the lawn image to be identified;

[0125] Among them, the preset recognition model is obtained by training the preset model using the lawn image to be trained and the processed lawn image; the lawn image to be trained has a labeled type; the processed lawn image is obtained by adding noise and / or interference to at least part of the lawn image to be trained.

[0126] In some embodiments, at least a portion of the lawn image to be trained is subjected to one or more of the following combined processing to obtain a processed lawn image:

[0127] Blur, distort, overexpose, add pixel blocks, add shadows to target objects, and add preset objects.

[0128] In some embodiments, the preset object includes one or more of the following combinations:

[0129] Pipes, taps, water guns, toys, animal droppings, fallen leaves, and branches.

[0130] In some embodiments, the target object includes one or more of the following combinations:

[0131] Trees, fences, and houses.

[0132] In some embodiments, the lawn image to be trained includes a color image and / or a grayscale image.

[0133] A model training device for lawn image processing, comprising:

[0134] An acquisition unit is configured to acquire a dataset to be trained; wherein the dataset to be trained includes a plurality of lawn images to be trained; and the lawn images to be trained have a label type;

[0135] a processing unit configured to add noise and / or interference to at least a portion of the lawn images to be trained in the training data set to obtain processed lawn images; and add the processed lawn images as new lawn images to be trained into the training data set to update the training data set;

[0136] The processing unit is further configured to process the lawn image to be trained using a preset model to obtain a predicted type of the lawn image to be trained; and optimize the parameters of the preset model according to the labeling type and predicted type of the lawn image to be trained to obtain a recognition model.

[0137] A lawn image processing device, applied to a self-propelled lawn mower, comprising:

[0138] An acquisition unit, configured to acquire an image of a lawn to be identified;

[0139] a processing unit configured to process the lawn image to be identified using a preset recognition model to obtain a target type of the lawn image to be identified;

[0140] Among them, the preset recognition model is obtained by training the preset model using the lawn image to be trained and the processed lawn image; the lawn image to be trained has a labeled type; the processed lawn image is obtained by adding noise and / or interference to at least part of the lawn image to be trained.

[0141] An electronic device, comprising:

[0142] at least one processor; and

[0143] a memory communicatively connected to the at least one processor; wherein,

[0144] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the method described in any embodiment of the present application.

[0145] A computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable a processor to implement the method described in any embodiment of the present application when executed.

[0146] The benefit of this application is that: this solution adds noise and / or interference to the lawn image to be trained, which can enrich the training samples and simulate as many possible situations in actual scenes as possible. The recognition model trained with richer training samples can improve the recognition accuracy of the recognition model.

[0147] To this end, one object of the present application is to provide a self-propelled lawn mower system with higher intelligent recognition accuracy.

[0148] In order to achieve the above objectives, this application adopts the following technical solutions:

[0149] A self-moving lawn mower system comprises: a self-moving lawn mower and a reference object;

[0150] The self-propelled lawn mower comprises:

[0151] body;

[0152] a running wheel assembly configured to support the fuselage;

[0153] a visual sensor mounted to the body and configured to acquire an image of the periphery of the self-propelled lawn mower;

[0154] a controller electrically connected to the visual sensor, and configured to recognize semantic information surrounding the self-propelled lawn mower through the image;

[0155] The reference object is configured to provide a comparison reference for the controller, and the reference object simulates at least one of lawn, land, water surface, and sky.

[0156] In some embodiments, the self-propelled lawn mower system further includes a charging station, and the reference object is disposed on the charging station.

[0157] In some embodiments, the reference object has geometric features, and the geometric features are used to assist the controller in identifying the reference object.

[0158] In some embodiments, the geometric feature is a planar geometric shape feature or a solid geometric body feature.

[0159] In some embodiments, the controller is configured to identify the reference object through the geometric features, extract color features of the reference object, and transform the image according to the color features to obtain a transformed image.

[0160] In some embodiments, the controller is further configured to process the transformed image through a semantic recognition program to obtain the semantic information.

[0161] A control method for a self-propelled lawn mower, wherein the self-propelled lawn mower includes a visual sensor, and the control method includes:

[0162] Acquire an image of a reference object through the visual sensor and extract reference information;

[0163] Acquiring an image of the surroundings of the self-propelled lawn mower by using the visual sensor;

[0164] performing transformation processing on the image of the periphery of the self-propelled lawn mower according to the reference information to obtain a transformed image;

[0165] The transformed image is processed by a semantic recognition program to obtain semantic information about the surroundings of the self-propelled lawn mower.

[0166] In some embodiments, the reference object is a known object in the natural environment.

[0167] In some embodiments, the reference object is an artificial simulated object.

[0168] In some embodiments, the reference object is set on the charging pile.

[0169] In some embodiments, the reference information includes color characteristics.

[0170] In some embodiments, the semantic recognition program is a machine learning program.

[0171] A control device for a self-propelled lawn mower, the self-propelled lawn mower including a visual sensor, the control device comprising:

[0172] an acquisition unit, configured to acquire an image of a reference object through the visual sensor;

[0173] a processing unit configured to extract reference information of the reference object;

[0174] The acquisition unit is further configured to acquire an image of the surrounding area of ​​the self-propelled lawn mower through the visual sensor;

[0175] The processing unit is further configured to perform transformation processing on the image of the periphery of the self-propelled lawn mower according to the reference information to obtain a transformed image;

[0176] The processing unit is further configured to process the transformed image through a semantic recognition program to obtain semantic information about the surroundings of the self-propelled lawn mower.

[0177] An electronic device, comprising:

[0178] at least one processor; and

[0179] a memory communicatively connected to the at least one processor; wherein,

[0180] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the control method of the self-moving lawn mower described in any embodiment of the present application.

[0181] A computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable a processor to implement the control method of the self-propelled lawn mower according to any embodiment of the present application when executed.

[0182] The benefit of this application is that: the reference information of the reference object is used to transform the image to obtain a transformed image, thereby reducing the deviation between the visual information on the image and the real information, and then using the transformed image for intelligent recognition can improve recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0183] FIG1 is a schematic diagram of a working scene of a self-propelled lawn mower provided by an embodiment of the present application;

[0184] FIG2 is a schematic structural diagram of a self-propelled lawn mower provided in an embodiment of the present application;

[0185] FIG3 is a structural diagram of a self-propelled lawn mower provided in an embodiment of the present application;

[0186] FIG4 is a schematic diagram of a slope detection scenario of a self-moving device provided in an embodiment of the present application;

[0187] FIG5 is a flow chart of a method for controlling a self-moving device provided by an embodiment of the present application;

[0188] FIG6 is a flow chart of a slope detection algorithm based on a point cloud provided in an embodiment of the present application;

[0189] FIG7 is a schematic diagram of a lawn image provided in an embodiment of the present application;

[0190] FIG8 is a flow chart of a control method for a self-propelled lawn mower provided by an embodiment of the present application;

[0191] FIG9 is a flow chart of an intelligent mowing decision algorithm provided by an embodiment of the present application;

[0192] FIG10 is a schematic diagram of a positioning scenario of a mobile device driving on a lawn provided in an embodiment of the present application;

[0193] FIG11 is a flowchart of a positioning method for a mobile device provided in an embodiment of the present application;

[0194] FIG12 is a flowchart of a positioning parameter adjustment algorithm based on inertial measurement unit data statistics provided in an embodiment of the present application;

[0195] FIG13 is a flow chart of a model training method for lawn image processing provided by an embodiment of the present application;

[0196] FIG14A is a schematic diagram of a lawn image to be trained provided by an embodiment of the present application;

[0197] FIG14B is a schematic diagram of performing Gaussian blur processing on a lawn image to be trained, provided by an embodiment of the present application;

[0198] FIG14C is a schematic diagram of performing mean blurring processing on a lawn image to be trained, provided by an embodiment of the present application;

[0199] FIG14D is a schematic diagram of performing median blurring processing on a lawn image to be trained, provided by an embodiment of the present application;

[0200] FIG15 is a flowchart of a lawn image processing method provided by an embodiment of the present application;

[0201] FIG16 is a flow chart of a model training method based on data enhancement provided in an embodiment of the present application;

[0202] FIG17 is a schematic diagram of a reference object provided in an embodiment of the present application;

[0203] FIG18 is a flow chart of a control method for a self-propelled lawn mower provided by an embodiment of the present application;

[0204] FIG19 is a flow chart of a method for establishing a feature conversion benchmark based on lawn markers provided in an embodiment of the present application;

[0205] FIG20 is a schematic diagram of the structure of an electronic device for implementing the method of an embodiment of the present application. DETAILED DESCRIPTION

[0206] Before any embodiments of the present application are explained in detail, it is to be understood that the application is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the foregoing drawings.

[0207] In this application, the terms "comprises," "includes," "has," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0208] In this application, the term "and / or" describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " in this application generally indicates that the related objects are in an "and / or" relationship.

[0209] In this application, the terms "connect," "combine," "couple," and "install" may refer to direct connection, combination, coupling, or installation, or indirect connection, combination, coupling, or installation. For example, a direct connection refers to two parts or components being connected together without an intermediary, and an indirect connection refers to two parts or components being connected to at least one intermediary, with the two parts or components being connected via the intermediary. Furthermore, "connect" and "couple" are not limited to physical or mechanical connections or couplings and may include electrical connections or couplings.

[0210] In this application, it will be understood by those skilled in the art that relative terms (e.g., "about," "approximately," "substantially," etc.) used in conjunction with quantities or conditions include values ​​and have the meaning indicated by the context. For example, the relative terms include at least the degree of error associated with the measurement of a specific value, the tolerance caused by manufacturing, assembly, use, etc. associated with a specific value. Such terms should also be considered to disclose a range defined by the absolute values ​​of the two endpoints. Relative terms may refer to plus or minus a certain percentage (e.g., 1%, 5%, 10% or more) of the indicated value. Numerical values ​​that do not use relative terms should also be disclosed as specific values ​​with tolerances. In addition, "substantially" may refer to plus or minus a certain degree (e.g., 1 degree, 5 degrees, 10 degrees or more) on the basis of the indicated angle when expressing a relative angular position relationship (e.g., substantially parallel, substantially perpendicular).

[0211] In this application, it will be understood by those skilled in the art that the function performed by an assembly can be performed by one assembly, multiple assemblies, one part, or multiple parts. Similarly, the function performed by a part can also be performed by one part, one assembly, or a combination of multiple parts.

[0212] In the present application, the terms "upper", "lower", "left", "right", "front", "back" and other directional words are described based on the orientation and positional relationship shown in the accompanying drawings, and should not be understood as limiting the embodiments of the present application. In addition, in the context, it is also necessary to understand that when it is mentioned that an element is connected to another element "upper" or "lower", it can not only be directly connected to the other element "upper" or "lower", but also be indirectly connected to the other element "upper" or "lower" through an intermediate element. It should also be understood that directional words such as upper side, lower side, left side, right side, front side, back side, etc. not only represent the positive orientation, but can also be understood as the lateral orientation. For example, below can include directly below, lower left, lower right, lower front and lower back, etc.

[0213] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0214] In this application, the terms "controller," "processor," "central processing unit," "CPU," and "MCU" are used interchangeably. Where a unit "controller," "processor," "central processing unit," "CPU," or "MCU" is used to perform a particular function, unless otherwise specified, the function may be performed by a single unit or multiple units.

[0215] In this application, the terms "device", "module" or "unit" can be implemented in the form of hardware or software to achieve specific functions.

[0216] In this application, the terms "calculate", "judge", "control", "determine", "identify", etc. refer to the operations and processes of a computer system or similar electronic computing device (e.g., controller, processor, etc.).

[0217] The self-moving device provided in the present application may be, for example, a self-moving lawn mower.

[0218] Figure 1 is a schematic diagram of a working scenario for a self-propelled lawn mower provided in an embodiment of the present application. As shown in Figure 1, the self-propelled lawn mower can move and operate within a working area. The working area is typically outdoors, including a lawn. The outdoor working environment is complex and changeable, presenting challenges for the self-propelled lawn mower operating therein. For example, lawn terrain varies greatly, including slopes. Excessively high slopes can cause the self-propelled lawn mower to tip over. Lawns can be of various types, and their growth is generally affected by factors such as light, precipitation, season, and temperature, making intelligent lawn maintenance by self-propelled lawn mowers challenging. Lawn surfaces can experience significant surface undulations as the weather and seasons change, which can cause the self-propelled lawn mower to jolt while operating on the lawn. Furthermore, the complex and changeable outdoor working environment, including smoke, dust, weather, and lighting variations, can cause overall shifts in the distribution of acquired lawn images. Intelligent recognition can be performed based on lawn images, but the diversity and shifts in lawn images can affect recognition accuracy to a certain extent.

[0219] Figure 2 is a structural schematic diagram of a self-moving lawn mower 100 provided in an embodiment of the present application. Referring to Figure 2, an embodiment of the present application provides a self-moving device, including: a body 110; a walking wheel assembly 120, configured to support the body 110; a visual sensor 130, configured to obtain an image in the traveling direction of the self-moving device; in combination with Figures 2 and 3, a posture sensor 140, configured to obtain the posture of the self-moving device; a controller 150, electrically connected to the visual sensor 130 and the posture sensor 140, the controller 150 being configured to calculate a first inclination angle of the ground in the traveling direction of the self-moving device based on the image in the traveling direction of the self-moving device obtained by the visual sensor 130, calculate a second inclination angle of the ground in combination with the first inclination angle and the posture of the self-moving device, and control the walking wheel assembly 120 according to the second inclination angle.

[0220] In conjunction with Figures 2 and 3 , the visual sensor 130 can be mounted on the body of the self-moving device above the axle. The visual sensor 130 can be flexibly mounted in various areas of the body, for example, directly on the housing of the fuselage 110. The visual sensor 130 can be mounted in any orientation. By flexibly setting the mounting position and orientation of the visual sensor 130 on the self-moving device, the image acquisition angle can be flexibly adjusted. Referring to Figures 2 and 3 , the posture sensor 140 and the controller 150 are also respectively mounted on the body of the self-moving device, and the controller 150 is electrically connected to the visual sensor 130 and the posture sensor 140, respectively.

[0221] Referring to Figure 3 , the controller 150 may include an acquisition unit 151, a processing unit 152, and a control unit 153. Specifically, the visual sensor 130 may capture images of the mobile device in its direction of travel, and the posture sensor 140 may capture the posture of the mobile device. The acquisition unit 151 in the controller 150 acquires images from the visual sensor 130 and the posture of the mobile device from the posture sensor 140. The processing unit 152 processes each image using a preset method to obtain a first inclination angle of the ground in the direction of travel of the mobile device. Furthermore, the processing unit 152 may obtain a second inclination angle of the ground based on the posture of the mobile device and the first inclination angle. The control unit 153 may use the second inclination angle to control the movement of the walking wheel assembly 120.

[0222] This solution improves slope angle recognition accuracy by combining the posture of the self-moving device with the first tilt angle to determine the second tilt angle. For example, using a self-moving lawn mower 100 as an example, even if the self-moving lawn mower is operating on the same slope in different postures, the second tilt angle can be accurately determined by combining the posture and the first tilt angle, thereby preventing the self-moving lawn mower 100 from tipping over.

[0223] In one implementation, the first tilt angle is a relative tilt angle of the ground relative to the self-moving device; and the second tilt angle is an absolute tilt angle of the ground relative to the direction of gravity.

[0224] Figure 4 is a schematic diagram of a slope detection scenario for a self-moving device provided by an embodiment of the present application. Taking a self-moving lawn mower 100 as an example, as shown in Figure 4 , the dashed line CD represents the horizon; the dashed line BH is parallel to the dashed line CD; the solid line BC represents the ground on which the self-moving lawn mower 100 is traveling; the solid line AB represents the ground in front of the self-moving lawn mower 100; the dashed line BG is an extension of the solid line BC; and the dashed line EF represents the direction of gravity. The dashed lines CD and EF are perpendicular to each other.

[0225] The self-propelled lawn mower 100 can capture an image of the front in the direction of travel and process the image into a point cloud. A ground point cloud is then extracted from the point cloud, and the ground point cloud is the ground point cloud corresponding to the solid line AB. The inclination angle of the ground in front is extracted through the ground point cloud. The inclination angle is the relative inclination angle of the solid line AB relative to the self-propelled lawn mower 100, and the relative inclination angle can be represented by ∠ABG. The pitch angle in the posture of the self-propelled lawn mower 100 is represented by ∠BCD. Since the dotted line BH is parallel to the dotted line CD, and the dotted line BG is the extension line of the solid line BC, ∠BCD = ∠GBH. The pitch angle plus the relative inclination angle is obtained to obtain the first absolute inclination angle of the ground in front relative to the direction of gravity, and the first absolute inclination angle can be represented by ∠ABH.

[0226] Similarly, the roll angle in the posture of the self-propelled lawn mower 100 may also be used to determine the second absolute tilt angle of the front ground relative to the gravity direction.

[0227] The first absolute tilt angle and the second absolute tilt angle are used as slope detection results. Furthermore, the first absolute tilt angle can be compared with the first angle threshold, and the second absolute tilt angle can be compared with the second angle threshold. If the first absolute tilt angle is less than the first angle threshold, and the second absolute tilt angle is less than the second angle threshold, then the ground in front is determined to be a safe area. If the first absolute tilt angle is greater than or equal to the first angle threshold, or the second absolute tilt angle is greater than or equal to the second angle threshold, then the ground in front is determined to be a non-safe area. Among them, the first angle threshold and the second angle threshold can be pre-set according to actual conditions. The first angle threshold and the second angle threshold can be unequal. First, the first absolute tilt angle and the second absolute tilt angle are obtained based on the pitch angle and roll angle in the posture respectively, and then the first absolute tilt angle and the second absolute tilt angle are used to comprehensively determine whether the ground in front is a safe area. This can improve the accuracy of the determination and thereby reduce the probability of the self-moving device tipping over.

[0228] The first tilt angle is the relative tilt between the ground and the self-moving device, while the second tilt angle is the absolute tilt of the ground and is not affected by the position of the self-moving device. Specifically, the second tilt angle is calculated by removing the influence of the self-moving device's posture from the first tilt angle.

[0229] Specifically, the tilt angle of the ground on which the mobile device is located relative to the direction of gravity can be determined based on the posture of the mobile device. The absolute tilt angle of the ground relative to the direction of gravity in the direction of travel of the mobile device can then be obtained based on this tilt angle and the relative tilt angle of the ground relative to the mobile device in the direction of travel of the mobile device.

[0230] In actual application scenarios, what affects whether a self-moving device will tip over is the absolute tilt angle of the ground where the self-moving device is located relative to the direction of gravity. Therefore, the absolute tilt angle is used as the slope angle recognition result to improve the accuracy of slope angle recognition.

[0231] In one implementation, the visual sensor 130 may include at least one of the following: an active laser depth camera, a passive binocular stereo camera, and a passive binocular structured light camera.

[0232] In one implementation, the visual sensor 130 includes at least one of a time of flight (TOF) depth camera, an infrared (IR) structured light camera, and a binocular camera.

[0233] In one achievable manner, the controller 150 creates a depth image from the image, extracts a ground point cloud from the depth image, and acquires the first tilt angle through the ground point cloud.

[0234] The processing unit 152 in the controller 150 can create a depth image based on the image in the direction of travel of the self-mobile device. Specifically, the depth information of each pixel can be estimated by analyzing the geometric relationship between the pixels and the objects in the image. The depth image represents the distance of each point in the scene relative to the camera. There are many methods for creating a depth image from an image, including depth estimation based on a single image, stereo matching based on multiple images, etc. Among them, the stereo matching method refers to calculating the disparity by matching the corresponding pixels of the same object in different images, thereby estimating the depth information.

[0235] The processing unit 152 can extract a ground point cloud from a depth image. Specifically, the depth image can be processed into a point cloud first, and then the ground point cloud can be extracted from the point cloud. Among them, a variety of algorithms can be used to extract the ground point cloud from the point cloud. The various algorithms include detection based on point cloud three-dimensional features and detection based on image plane features. The detection algorithm is not limited to detection based on classic manual features and detection based on learning features. Among them, detection based on point cloud three-dimensional features refers to identifying and classifying target objects by extracting three-dimensional features in point cloud data, such as shape, edge, texture, etc. Among them, detection based on image plane features refers to identifying and classifying target objects by extracting plane features in the image, such as edges, corners, etc. Among them, detection based on classic manual features refers to identifying and classifying target objects by extracting manual features in the image, such as edges, corners, textures, etc. Among them, detection based on learning features refers to identifying and classifying target objects by learning feature expressions in the image and combining them with classifiers.

[0236] In one possible implementation, the point cloud can be combined with a color image, that is, the point cloud is colorized to obtain a color point cloud. A ground point cloud is then extracted from the color point cloud, and the first tilt angle is obtained from the ground point cloud. Specifically, the color point cloud contains the color information of the point cloud, and therefore contains more semantic information than the point cloud. Therefore, the first tilt angle obtained using the color point cloud is more accurate to a certain extent.

[0237] The processing unit 152 can obtain the first tilt angle through the ground point cloud. Specifically, the first tilt angle can be obtained through the ground point cloud using a geometric method. The geometric method includes at least one of the following methods: a line-surface fitting method and a voxel sampling method. Among them, the line-surface fitting method refers to restoring the straight lines and planes in the real three-dimensional scene corresponding to the ground point cloud through geometric reasoning and calculation based on the straight lines and planes in the ground point cloud, and then calculating the first tilt angle from the fitted straight lines and planes. Among them, the voxel sampling method refers to dividing the ground point cloud data into a series of small cubes, called voxels, and then selecting a representative point in each voxel as a sampling point. The coordinate information of the sampling points is then used to calculate the tilt angle of each sampling point relative to the self-moving device. The first tilt angle is obtained by combining the tilt angles of each sampling point relative to the self-moving device.

[0238] The above method can be used to quickly and accurately obtain the first tilt angle based on the image.

[0239] In one implementation, when the second tilt angle is greater than or equal to the first threshold, the controller 150 determines that the ground in the direction of travel of the mobile device is a non-safe area and changes the direction of travel.

[0240] The first threshold is a tilt angle threshold preset according to actual conditions.

[0241] Specifically, when the second tilt angle is determined to be greater than the first threshold, the control unit 153 in the controller 150 determines that the ground in the direction of travel of the self-moving device is an unsafe area and changes the direction of travel of the self-moving device to prevent the self-moving device from tipping over due to the tilt of the ground. The unsafe area refers to an area on the ground where the self-moving device is prone to tipping over due to an excessively high tilt angle.

[0242] In one implementation, when the second tilt angle is less than the first threshold, the controller 150 determines that the ground in the direction of travel of the mobile device is a safe area and maintains the direction of travel.

[0243] Specifically, when it is determined that the second tilt angle is less than the first threshold, the control unit 153 in the controller 150 may determine that the ground in the direction of travel of the self-moving device is a safe area, and save the travel direction and continue to move. The safe area refers to an area of ​​the ground where the tilt angle is not too high and is unlikely to cause the self-moving device to fall.

[0244] In one possible implementation, the moving direction of the self-moving device is the front of the self-moving device.

[0245] The direction of travel of the mobile device can be determined based on path planning.

[0246] In one implementation, the first threshold is determined according to the friction of the ground.

[0247] The first threshold value can be determined based on the friction of the ground, and the value of the first threshold value can be proportional to the friction of the ground. The greater the friction of the ground, the greater the value of the first threshold value can be. Determining the first threshold value based on the friction of the ground can make the value of the first threshold value more accurate.

[0248] In one implementation, the gesture sensor 140 is an accelerometer.

[0249] FIG5 is a flow chart of a control method for a self-propelled device provided in an embodiment of the present application. This embodiment is applicable to a scenario where a self-propelled device is planning a route when traveling on a ground with a certain slope. The self-propelled device may be, for example, a self-propelled lawn mower 100. The method may be executed by the self-propelled device. As shown in FIG5 , the method includes:

[0250] Step 501: Acquire the posture of the mobile device; and acquire an image in the moving direction of the mobile device.

[0251] Step 502: Calculate a first tilt angle of the ground in the direction of travel of the mobile device based on the image.

[0252] Step 503: Calculate a second tilt angle of the ground based on the first tilt angle and the posture; and decide whether to move along the moving direction based on the second tilt angle.

[0253] This solution combines the posture of the self-propelled device and eliminates the influence of the posture from the first tilt angle to obtain a second tilt angle. This second tilt angle is used as the slope detection result of the ground in the direction of the self-propelled device's travel. Based on this second tilt angle, the self-propelled device can be controlled to determine whether to proceed in the direction of travel, thereby preventing the self-propelled device from tipping over due to excessive slopes. This improves the accuracy of the ground slope detection in the direction of the self-propelled device's travel, thereby reducing the probability of the self-propelled device toppling over.

[0254] The present application also provides a point cloud-based slope detection algorithm for a lawn mower equipped with a depth camera. FIG6 is a flow chart of a point cloud-based slope detection algorithm provided by the present application. Referring to FIG6 , the specific process of the algorithm is as follows:

[0255] Step 601: The lawn mower is powered on and a slope detection program in the lawn mower controller is started.

[0256] Step 602: Use a depth camera to capture an image in front of the lawn mower. The depth camera is not limited to an active laser depth camera, a passive binocular stereo camera, or a passive binocular structured light camera.

[0257] Step 603 : Process the depth image collected by the lawn mower into a point cloud using an algorithm in the slope detection program.

[0258] In step 604, a ground point cloud is extracted from the point cloud in front of the lawn mower through algorithm processing. The algorithm includes detection based on three-dimensional features of the point cloud and detection based on image plane features. The detection algorithm is not limited to detection based on classic manual features and detection based on learned features.

[0259] Step 605 , extracting the relative inclination angle of the ground on the route through the ground point cloud information by a geometric method, and the extraction method is not limited to the line-surface fitting method and the voxel sampling method.

[0260] Step 606: Combine the attitude information provided by the inertial navigation system and the ground tilt information to comprehensively determine the absolute tilt angle of the ground relative to the gravity direction.

[0261] Step 607: Upload the inclination angle of the ground relative to the gravity direction to the planning decision layer to determine whether it is a safe area for driving.

[0262] Step 608: Plan a route based on the result of step 607. Specifically, if the area is determined to be a drivable area, the lawn mower is controlled to continue to move in the direction of travel; if the area is determined to be a non-drivable area, the direction of travel is changed and a safe driving route is planned.

[0263] Figure 7 is a schematic diagram of a lawn image provided by an embodiment of the present application. Grass growth is affected by factors such as light, precipitation, season, and temperature, and is not static. The lawn image shown in Figure 7 contains characteristic information about the grass, such as its height, color, and type, which characterizes its growth state. Intelligently maintaining a lawn based on its varying growth states presents a challenge for autonomous lawn mowers.

[0264] In one embodiment, referring to Figures 2 and 3 , the controller 150 is electrically connected to the visual sensor 130. The controller 150 is configured to acquire and, based on a patrol instruction, receive lawn images of the working area captured by the visual sensor 130; process the lawn images to obtain grass feature information, where the feature information includes at least grass height and color; formulate a mowing plan or patrol plan based on the feature information, and control the operation of the self-propelled lawn mower 100 according to the mowing plan or patrol plan. The mowing plan includes a time for performing mowing tasks, and the patrol plan includes a time for generating the next patrol instruction. Referring to Figure 3 , the controller 150 may include an acquisition unit 151, a processing unit 152, and a control unit 153. Specifically, the visual sensor 130 may acquire lawn images of the working area of ​​the self-propelled lawn mower 100. Referring to Figure 3 , the acquisition unit 151 may acquire patrol instructions and, based on the patrol instructions, acquire lawn images from the visual sensor 130. The patrol instructions may be generated in response to user operations or based on a patrol plan. The processing unit 152 can process the lawn image using a preset method to obtain grass characteristic information, where the characteristic information includes at least grass height and color. Specifically, the lawn image can first be preprocessed, including steps such as denoising, image enhancement, and color correction, to improve image quality. A deep learning semantic segmentation algorithm can then be used to separate the grass plants from the background in the preprocessed lawn image. Based on the segmentation results, parameters can be calculated for the segmented grass plants, including at least their height and color. Finally, statistical analysis can be performed on the calculated grass parameters to obtain overall lawn characteristics. For example, statistical data such as average grass plant height and average grass plant color can be calculated. The average grass plant height can be used as the grass height, and the average grass plant color can be used as the grass color. The processing unit 152 can then formulate a mowing plan or inspection plan based on the grass characteristic information. For example, if the grass characteristic information determines that the lawn needs to be mowed in the near future, a mowing plan can be generated. If the grass characteristic information determines that the lawn does not need to be mowed in the near future, an inspection plan can be generated to monitor grass growth and re-mow when necessary. The control unit 153 can control the self-propelled lawn mower 100 to operate according to a mowing plan or an inspection plan. The mowing plan may include a mowing task and a time to execute the mowing task. The inspection plan may include the time when the self-propelled lawn mower 100 generates the next inspection instruction.

[0265] Since different types of grass usually have different colors, and the colors of the same type of grass usually vary in different seasons, the suitable mowing heights for different types of grass usually vary, and the suitable mowing heights for the same type of grass in different seasons usually vary. Therefore, this solution identifies the growth status of the grass based on at least characteristic information such as the height and color of the grass. For example, the type of grass and the season can be determined based on the color, and a mowing plan or inspection plan for the self-propelled lawn mower 100 can be formulated based on the type, height and season of the grass, making lawn maintenance more intelligent and thus improving the quality of lawn maintenance.

[0266] In one implementation, the mowing height threshold is determined based on color.

[0267] Specifically, a color-height mapping table can be pre-set based on experience, which includes the mapping relationship between color and height. The processing unit 152 can use the color of the grass to query the color-height mapping table, obtain the height corresponding to the color of the grass, and determine the height as the mowing height threshold.

[0268] If the height is greater than the mowing height threshold, the self-propelled mower is controlled to perform the mowing task.

[0269] The processing unit 152 can compare the grass height with a mowing height threshold, and if it is determined that the grass height is greater than the mowing height threshold, a mowing task to be performed is generated. The control unit 153 controls the self-propelled mower 100 to operate according to the mowing task to be performed.

[0270] If the height is less than or equal to the mowing height threshold, a mowing plan or inspection plan is formulated based on the difference between the mowing height threshold and the height.

[0271] If the processing unit 152 determines that the grass height is less than or equal to the mowing height threshold, it calculates the difference between the mowing height threshold and the grass height and formulates a mowing plan or inspection plan based on the difference. The control unit 153 controls the self-propelled lawn mower 100 according to the mowing plan or inspection plan. For example, if the difference is small, it can be determined that the grass needs to be mowed in the near future, and a mowing plan can be formulated. If the difference is large, it can be determined that the grass does not need to be mowed in the near future, and an inspection plan can be formulated to monitor grass growth and mow the grass when it needs to be mowed.

[0272] Since different types of grass usually have different colors, and the colors of the same type of grass usually vary in different seasons, the suitable mowing heights for different types of grass usually vary, and the suitable mowing heights for the same type of grass in different seasons usually vary. Therefore, this solution identifies the growth status of grass by its color. For example, the type of grass and the season can be determined based on the color, and then the mowing height threshold can be determined based on the type of grass and season. Therefore, a mowing plan or inspection plan can be formulated based on the mowing height threshold and the grass height, which can improve the maintenance quality of the lawn.

[0273] In one implementation, the characteristic information further includes size, shape and / or density; the acquisition unit 151 acquires season information, and the processing unit 152 determines the type of grass according to the season information, size, shape and color.

[0274] In the present application, a feature information mapping table can be pre-set based on experience, and the table includes mapping relationships between seasons, sizes, shapes, colors, and grass types. The acquisition unit 151 can also obtain season information. Specifically, the acquisition unit 151 can infer season information based on time information. Alternatively, the season information can be determined at least based on the color of the grass. The processing unit 152 can process the lawn image to obtain the size, shape, and density of the grass. Specifically, the grass plants in the lawn image can be separated from the background, and then the parameters of the segmented grass plants can be calculated, including the size, shape, and density of the grass plants. Among them, the size includes at least height and width; the shape includes at least roundness and complexity; and the density includes at least the number of grass plants per unit area. Finally, the calculated grass plant parameters are statistically analyzed to obtain the overall characteristics of the lawn. For example, statistical data such as average grass plant height, average grass plant width, average grass plant roundness, average grass plant complexity, and average density can be calculated. We can use information like average plant height and width as grass size, average plant roundness and complexity as grass shape, and average density as grass density. Then, using seasonal information, grass size, shape, and color, we query the feature information mapping table to determine the grass type.

[0275] The processing unit 152 determines a mowing height threshold according to the grass type, density, and season information.

[0276] Specifically, a feature mapping table can be pre-set based on experience, including mapping relationships between species, density, season, and height. The processing unit 152 can query the feature mapping table based on the seasonal information and the grass species and density to obtain the height corresponding to the seasonal information, grass species, and density, and determine the height as the mowing height threshold.

[0277] Because different grass types often have different characteristics such as size, shape, and color in different seasons, the appropriate mowing heights for different grass types often vary. The appropriate mowing heights for the same grass type often vary in different seasons, and the appropriate mowing heights for grass of different densities also often vary. Therefore, this solution can first accurately determine the grass type based on seasonal information and the grass's size, shape, and color. It can then more accurately determine the mowing height threshold based on the grass type, density, and seasonal information. Controlling the operation of the automatic lawn mower 100 using a mowing plan or inspection schedule based on this mowing height threshold can improve lawn maintenance quality.

[0278] In one implementation, if the processing unit 152 determines that the difference between the mowing height threshold and the grass height is less than or equal to a preset threshold, a mowing plan is formulated based on the difference; if the difference is greater than the preset threshold, an inspection plan is formulated based on the difference.

[0279] The preset threshold is a value pre-set based on actual conditions. If the difference between the mowing height threshold and the grass height is less than or equal to the preset threshold, it can indicate that the grass height is not much different from the mowing height threshold. In this case, the grass growth rate can generally be estimated more accurately, and a more accurate mowing plan can be formulated based on the estimated grass growth rate.

[0280] If the difference between the grass height threshold and the mowing height is greater than a preset threshold, this indicates a significant discrepancy between the grass height and the threshold. In this case, it's generally difficult to accurately estimate the grass's growth rate, and therefore, to accurately plan a mowing schedule. Therefore, a patrol plan based on the difference can be developed to monitor grass growth. For example, if the difference is greater than the preset threshold and relatively large, a longer interval can be set to patrol the lawn to determine if mowing is necessary. If the difference is greater than the preset threshold and relatively small, a shorter interval can be set to patrol the lawn to determine if mowing is necessary. This improves lawn maintenance quality while conserving resources.

[0281] This solution can first determine the difference between the mowing height threshold and the grass height, and then formulate a mowing plan or inspection plan based on the difference and the comparison result between the difference and the preset threshold. It can more accurately control the operation of the self-propelled lawn mower 100, thereby achieving better lawn maintenance effects.

[0282] In one implementable manner, the height of the grass is determined based on point cloud data of the grass; and the point cloud data is determined based on an image of a lawn.

[0283] The processing unit 152 can first create a depth image from the lawn image, and then process the depth image into a point cloud. The point cloud is the point cloud data of the grass. The height of the grass can then be identified using the point cloud data of the grass. In this way, the height of the grass can be quickly and accurately obtained. Specifically, each point on the depth image can be converted into a world coordinate system through the relationship between similar triangles or coordinate system transformation to obtain the point cloud data of the grass. First, the visual sensor 130 needs to be aligned with the lawn surface and the three-dimensional point cloud data of the lawn needs to be acquired. The visual sensor 130 can capture the three-dimensional coordinates of each point on the lawn surface, including X, Y, and Z coordinates. The acquired point cloud data needs to be processed to remove noise, smooth the data, and perform data registration. These processes can include filtering, smoothing, resampling, and other operations to improve the quality and accuracy of the data. Features can be extracted from the point cloud data, including the shape and height of the grass, and then a classifier can be designed based on the extracted features to distinguish the grass from other objects; after the classifier identifies the grass, the point cloud data of a certain area can be selected from the processed point cloud data, and the grass height in the area can be obtained by calculating the average height of these points. Specifically, some representative point cloud data can be selected and their average height can be calculated to obtain the grass height in the area. It should be noted that the accuracy and stability of the visual sensor 130 may be affected by factors such as ambient lighting and camera settings, so careful calibration and calibration are required in advance. Of course, point cloud data can also be obtained through lidar, so in this solution, lidar can also be used as a substitute for the visual sensor 130.

[0284] FIG8 is a flow chart of a control method for a self-propelled lawn mower according to an embodiment of the present application. This embodiment is applicable to scenarios where the self-propelled lawn mower 100 performs intelligent lawn maintenance, especially when the lawn mower is out of user control for a long period of time. The method can be executed by the self-propelled lawn mower 100. As shown in FIG8 , the method includes:

[0285] Step 801: Obtain a patrol instruction, and obtain a lawn image of a working area of ​​a self-propelled lawn mower according to the patrol instruction.

[0286] Step 802: Process the lawn image to obtain characteristic information of the grass; wherein the characteristic information at least includes the height and color of the grass.

[0287] Step 803: formulate a mowing plan or an inspection plan based on the characteristic information, and control the self-propelled lawn mower to work according to the mowing plan or the inspection plan; wherein the mowing plan includes the time for executing the mowing task; the inspection plan includes the time for generating the next inspection instruction.

[0288] Since different types of grass usually have different colors, and the colors of the same type of grass usually vary in different seasons, the suitable mowing heights for different types of grass usually vary, and the suitable mowing heights for the same type of grass in different seasons usually vary. Therefore, this solution can more accurately formulate mowing plans or inspection plans based on at least the height and color of the grass, thereby being able to maintain the lawn more intelligently and achieve better lawn maintenance results.

[0289] The present application also provides a specific intelligent mowing decision algorithm based on semantic recognition for lawn mowers. FIG9 is a flow chart of an intelligent mowing decision algorithm provided by the present application embodiment. Referring to FIG9 , the specific process of the algorithm is as follows:

[0290] Step 901: Power on the lawn mower.

[0291] Step 902: Start executing the inspection mode in the lawn mower controller and determine whether to start the inspection.

[0292] In step 903, the lawn mower inspects the lawn along a certain route. This route can be a user-specified route, a route planned by the lawn mower based on the lawn conditions, or a completely random route.

[0293] In step 904, an image is acquired using a visual sensor and an algorithm is used to extract semantic information from the image. This semantic information is then used to determine whether the lawn mower is operating in a lawn area. This algorithm may be a deep learning algorithm. If the semantic information indicates the presence of grass in the image, the lawn mower is determined to be operating in a lawn area. If the semantic information indicates the absence of grass in the image, the lawn mower is determined to be operating in a lawn area. Alternatively, the image may be processed into a color point cloud, and semantic information may be identified from the color point cloud. Alternatively, the semantic information recognition results of the image and the color point cloud may be combined to obtain the combined speech information.

[0294] In step 905, if the vehicle is not in a lawn area, the vehicle continues to proceed and proceeds to step 903. If the vehicle is in a lawn area, the algorithm collects the semantic and feature information related to the lawn. This semantic information includes identification labels such as whether the lawn is grass, grass type, density, and season. This semantic information can include pixel or sub-pixel semantic labels for each lawn within the field of view. Grass feature information includes grass color, height, size, shape, and more. Time of day can also be used to determine the season.

[0295] Step 906: Determine whether the collected feature information is sufficient based on a preset statistical algorithm. If insufficient, continue collecting; if sufficient, cease collecting and extract further information necessary for decision-making based on the currently collected feature information. This decision-making information is not limited to the mean and variance of the grass feature information distributed across the entire lawn area, the mean and variance of the grass feature information for each subdivided area, and the semantic information of the grass. Determine whether the collected feature information is sufficient based on the preset statistical algorithm, including determining that the collected feature information is sufficient if the lawn area being counted is sufficiently large.

[0296] Step 907 comprehensively determines whether to execute the mowing task based on the decision information. Mowing tasks are not limited to mowing the entire area, mowing a partial area, or multiple mowing tasks in different time periods. For example, the mower's operating area can encompass a variety of scenarios, such as uneven grass growth in the operating area, uneven mowing from the previous mowing, or the presence of multiple types of grass in the operating area with different mowing height requirements for each type. Mowing tasks can be adaptively configured for different scenarios.

[0297] Step 908, execute the mowing task planned in step 907. If the mowing task cannot be executed due to other reasons, such as human intervention, weather reasons, battery power, etc., the current condition of the lawn is recorded for the next comprehensive decision-making plan.

[0298] Figure 10 is a schematic diagram of a positioning scenario for a self-propelled device traveling on a lawn, as provided in an embodiment of the present application. As shown in Figure 10 , lawns are typically located outdoors, and changes in outdoor weather and seasons can cause significant surface variations in the lawn's surface conditions. The varying undulations of the lawn's surface can affect the inertial measurement unit's collection of inertial data, potentially leading to inaccurate positioning of the self-propelled device. This presents a significant challenge for self-propelled devices.

[0299] In one embodiment, in conjunction with Figures 2 and 3, an inertial measurement unit (IMU), taking the attitude sensor 140 as an example, is disposed in the body 110, and the IMU includes at least one of an accelerometer and a gyroscope; a controller 150 is electrically connected to the IMU, and the controller 150 is configured to control the self-moving device to walk; receive inertial data from the IMU; perform statistics on the inertial data received within a first time window, and adjust the positioning parameters of the IMU based on the statistical results.

[0300] 2 and 3 , the posture sensor 140 and the controller 150 are respectively disposed on the body of the self-moving device, and the controller 150 is electrically connected to the posture sensor 140 .

[0301] 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, the controller 150 may control the self-mobile device to move, and may collect inertial data from the self-mobile device via an inertial measurement unit. The acquisition unit 151 in the controller 150 may acquire the inertial data from the inertial measurement unit; the processing unit 152 may use a preset method to perform statistics on the inertial data received within a first time window to obtain statistical results, where the duration of the first time window is pre-set based on actual conditions; and adjust the positioning parameters of the inertial measurement unit based on the statistical results. Furthermore, the position of the self-mobile device may be determined based on the inertial data and the positioning parameters.

[0302] Inertial data can be used to locate a self-propelled device. The bumpy surface conditions in the area where the self-propelled device is traveling can affect the accuracy of the inertial data collected by the inertial measurement unit, and thus the positioning accuracy of the self-propelled device. The statistical results of the inertial data can reflect the surface bumps. Therefore, this solution adjusts the positioning parameters of the inertial data based on the statistical results of the self-propelled device's inertial data to minimize the impact of surface bumps on positioning. The self-propelled device can then be positioned using the inertial data and the adjusted positioning parameters, thereby improving positioning accuracy.

[0303] In one achievable manner, the statistical result reflects the smoothness of the road segment passed by the mobile device within the first time window.

[0304] A higher road smoothness indicates a smoother road with fewer bumps.

[0305] In one implementable manner, the inertial data collected by the inertial measurement unit includes at least acceleration and angular velocity from the mobile device, and the inertial data received within a first time window is statistically analyzed using a preset method to obtain statistical results, including: integrating the acceleration and angular velocity in the inertial data received within the first time window to obtain velocity and angle, respectively; then smoothing the acceleration, angular velocity, velocity, and angle to obtain acceleration processing results, angular velocity processing results, velocity processing results, and angle processing results; determining a first variance between each acceleration and the acceleration processing result, determining a second variance between the angular velocity and the angular velocity processing result, determining a third variance between the velocity and the velocity processing result, and determining a fourth variance between the angle and the angle processing result; and determining the first variance, the second variance, the third variance, and the fourth variance as the statistical results.

[0306] Smoothing the acceleration to obtain the acceleration processing result includes: processing the acceleration using a sliding window smoothing method to obtain the acceleration processing result. For example, averaging the accelerations collected within the first time window and using the average value as the acceleration processing result.

[0307] Similarly, the angular velocity is smoothed to obtain an angular velocity processing result; the velocity is smoothed to obtain a velocity processing result; and the angle is smoothed to obtain an angle processing result.

[0308] Determining the first variance between each acceleration and the acceleration processing result includes determining the square of the difference between each acceleration and the acceleration processing result, and taking a weighted sum of the squares as the first variance. Accelerations in different segments may have different weights, and the weights corresponding to different segments may be set through manual parameter adjustment or machine learning.

[0309] Similarly, a second variance between the angular velocity and the angular velocity processing result can be determined; a third variance between the speed and the speed processing result can be determined; and a fourth variance between the angle and the angle processing result can be determined.

[0310] Smoothing can approximate the mean of dynamic motion, thereby statistically analyzing the bumps during motion. Variance can better reflect the distribution of data around the mean. The smaller the distribution difference, the smaller the variance, and the larger the distribution difference, the larger the variance.

[0311] In one implementable manner, it is first determined based on the statistical results whether the positioning parameters of the inertial measurement unit need to be adjusted. If it is determined that adjustment is required, the positioning parameters are adjusted; if it is determined that adjustment is not required, the positioning parameters are not adjusted.

[0312] This solution can determine whether the positioning parameters need to be adjusted through statistical results, and adjust the positioning parameters only when the conditions are met, thereby improving the positioning accuracy to a certain extent.

[0313] Specifically, a preset range can be set based on experience, and the statistical results can be compared with the preset range. If the statistical results are determined to be within the preset range, it is determined that the positioning parameters need to be adjusted. This method can quickly and accurately determine whether the positioning parameters need to be adjusted.

[0314] Alternatively, a binary classification model can be trained using the training statistical results. The classification results of the binary classification model are used to determine whether the IMU's positioning parameters need adjustment. The trained binary classification model can then be used to process the statistical results to obtain a classification result. If the classification result indicates that the positioning parameters need adjustment, the positioning parameters are adjusted; if the classification result indicates that the positioning parameters do not need adjustment, the positioning parameters are not adjusted. This method can quickly and accurately determine whether the positioning parameters need adjustment.

[0315] Specifically, a mapping table can be pre-set based on experience, containing the mapping relationship between flatness and adjustment parameters. The statistical results can then be used to query the mapping table, obtain the corresponding adjustment parameters, and use these adjustment parameters to update the positioning parameters of the inertial measurement unit. This approach allows for quick and accurate adjustment of positioning parameters.

[0316] Alternatively, the statistical results can be processed using a pre-set regression model to obtain a regression result. This regression result is then used to update the positioning parameters. This method allows for quick and accurate adjustment of the positioning parameters.

[0317] In one implementable manner, the first time window is determined according to the walking speed of the mobile device.

[0318] Specifically, the faster the walking speed of the mobile device is, the shorter the duration of the first time window is.

[0319] In one possible implementation, the value range of the first time window is greater than or equal to a first duration threshold and less than a second duration threshold; wherein the first duration threshold is less than the second duration threshold.

[0320] The first duration threshold and the second duration threshold are both preset based on experience. The first time window can take any value between the first duration threshold and the second duration threshold.

[0321] The statistical results obtained by statistically analyzing enough inertial data can more accurately represent the smoothness of the road section. The method of setting the first time window in this solution is a more convenient and accurate way to determine whether the acquired inertial data is sufficient.

[0322] In one implementation, the positioning parameter is the noise of the inertial data.

[0323] Specifically, the bumpy surface conditions in the area where a mobile device is traveling can affect the accuracy of inertial data collected by the inertial measurement unit. Positioning parameters can act as noise in the inertial data, characterizing its accuracy. The statistical results of the inertial data can also reflect the level of surface bumps. Therefore, this solution adjusts the positioning parameters of the inertial data based on these statistical results to minimize the impact of surface bumps on positioning, thereby improving positioning accuracy.

[0324] In one implementation, the positioning parameter is the weight of the inertial data when the inertial data is fused with data of other modalities for positioning.

[0325] In this application, the self-moving device may also be equipped with a lidar and a camera. Inertial data and other modal data can be used to perform multimodal fusion positioning of the self-moving device. Specifically, different types of data, such as inertial data, point cloud data, and image data, can be acquired through sensors such as an inertial measurement unit, lidar, and a camera. This data is processed using a multimodal fusion algorithm to achieve more accurate positioning of the self-moving device. The multimodal fusion algorithm includes a weighted summation of the multimodal data, and the positioning parameters can be the weights used in the fusion positioning of the inertial data and the other modal data. The bumpier the surface in the self-moving device's driving area, the less accurate the inertial data acquired by the inertial measurement unit. Therefore, when the surface is relatively bumpy, reducing the weight of the inertial data used in fusion positioning can improve the accuracy of fusion positioning to a certain extent. The statistical results of the inertial data can reflect the bumpiness of the surface. Therefore, this solution adjusts the positioning parameters of the inertial data based on the statistical results of the self-moving device's inertial data. The positioning of the self-moving device calculated using the adjusted positioning parameters is more accurate.

[0326] In one implementation, the inertial measurement unit (IMU) can be designed to reduce the impact of uneven road surfaces on the autonomous vehicle. Specifically, IMU vibration reduction measures can be implemented from both design and structural perspectives. Design-wise, to reduce the impact of vibration and shock on the IMU, the adaptability of the IMU can be improved through material selection and appropriate structural design. For example, high-precision, next-generation IMUs, such as the MPU6050 and BMI055, can be selected. These chips offer high adaptability and stability, reducing the impact of body vibration on the IMU's measurement accuracy. Structurally, a range of vibration reduction designs can be implemented, such as using damping materials (such as silicone, rubber, and silicone rubber) and damping structures to isolate vibration. Specifically, the IMU circuit board can be structured so that the chip's center of gravity is aligned with the IMU's center of gravity, achieving optimal vibration reduction. Furthermore, the use of vibration reduction devices such as shock absorbers and air dampers can be considered; these devices can effectively reduce the impact of body vibration on the IMU.

[0327] FIG11 is a flow chart of a method for positioning a self-moving device provided in an embodiment of the present application. This embodiment is applicable to positioning scenarios in which a self-moving device is in motion, and is particularly applicable to positioning scenarios in which a self-moving device is in motion on a relatively bumpy road. The method can be performed by the self-moving device. As shown in FIG11 , the method includes:

[0328] Step 1101: Obtain a positioning request, and obtain inertial data from a mobile device within a first time window according to the positioning request.

[0329] Step 1102 : Count the inertial data and adjust the positioning parameters corresponding to the inertial data according to the statistical results.

[0330] Step 1103: Determine the position of the mobile device based on the inertial data and positioning parameters.

[0331] The bumpy surface of the autonomous vehicle's driving area can affect the accuracy of the inertial measurement unit's inertial data collection, and thus the autonomous vehicle's positioning accuracy. The statistical results of the inertial data can reflect the surface bumpiness. Therefore, this solution adjusts the positioning parameters of the inertial data based on the statistical results of the autonomous vehicle's inertial data to minimize the impact of surface bumps on positioning. This solution then uses the inertial data and the adjusted positioning parameters to locate the autonomous vehicle, thereby improving positioning accuracy.

[0332] The present application also provides a positioning parameter adjustment algorithm based on inertial measurement unit data statistics for lawn mowers. FIG12 is a flow chart of a positioning parameter adjustment algorithm based on inertial measurement unit data statistics provided by the present application. Referring to FIG12 , the specific process of the algorithm is as follows:

[0333] Step 1201: Collect inertial data through an inertial measurement unit in the lawn mower. The inertial data includes but is not limited to acceleration, angular velocity, etc.

[0334] Step 1202 determines whether sufficient inertial data has been collected. The basis for this determination includes, but is not limited to, status information such as the time the data was collected, the distance the lawn mower has traveled, and whether the lawn mower is mowing. Since the vibrations generated by the lawn mower during mowing will also affect the inertial measurement unit, this effect can easily be confused with the effect of road bumps on the inertial measurement unit. Therefore, it is necessary to collect inertial data when the lawn mower is not mowing. If the inertial data is insufficient, the adjustment of the positioning parameters based on the inertial data will not be accurate, so sufficient inertial data needs to be collected. If the length of the inertial data collection time reaches a preset time length threshold, or if the distance the lawn mower has traveled during the inertial data collection process reaches a preset distance threshold, it can be determined that the inertial data has been collected sufficiently. When the inertial data is sufficient, proceed to step 1203; if it is insufficient, continue collecting.

[0335] Step 1203: Process the collected inertial data using an algorithm to obtain statistical indicators of the inertial data. The algorithm includes, but is not limited to, machine learning methods or statistical learning methods. Specifically, the inertial data includes at least acceleration and angular velocity. The inertial data is integrated to obtain velocity data and angle data. The inertial data, velocity data, and angle data are smoothed using a sliding window smoothing method. Based on the smoothed data, the variance of the inertial data, velocity data, and angle data is calculated. Different weightings are applied to data in different numerical ranges. The weighting values ​​can be set through manual parameter adjustment or machine learning methods.

[0336] In step 1204, based on the series of statistical indicators obtained in step 1203, a qualitative method is used to determine whether parameter adjustment is required. Qualitative methods include but are not limited to empirical judgment methods or machine learning binary classification methods.

[0337] Step 1205: For the scenario where parameter adjustment is required in step 1204, the positioning parameters of the inertial measurement unit are adjusted based on the obtained series of statistical indicators using a table lookup interpolation method or other machine learning regression method.

[0338] Figure 13 is a flow chart of a model training method for lawn image processing provided by an embodiment of the present application. This embodiment is applicable to model training scenarios for identifying grass or non-grass objects in lawn images. The method can be executed by an electronic device. As shown in Figure 13, the method includes:

[0339] Step 1301: Obtain a dataset to be trained. The dataset to be trained includes a plurality of lawn images to be trained. The lawn images to be trained have a label type.

[0340] Specifically, a visual sensor may be used to obtain an image of the lawn to be trained, and the image may be cropped, sampled, labeled, etc. The image of the lawn to be trained may include information such as the ground, objects, and sky within the visual sensor's field of view.

[0341] Alternatively, publicly available lawn images to be trained with labeled types can be obtained from the Internet. The labeled types may include object types in the visual sensor's field of view, such as grass, pedestrians, trees, houses, sky, and ground.

[0342] Step 1302: adding noise and / or interference to at least a portion of the lawn images to be trained in the training data set to obtain processed lawn images; and adding the processed lawn images as new lawn images to be trained into the training data set to update the training data set.

[0343] A variety of methods can be used to add noise and / or interference to the training lawn image to obtain a processed lawn image, so as to enrich the training samples and simulate as many possible situations in the actual scene as possible. Specifically, referring to Figures 14A, 14B, 14C and 14D, the training lawn image can be blurred, including Gaussian blur, mean blur and median blur. Among them, Gaussian blur refers to blurring the image by convolving the image with a normal distribution. Mean blur mainly operates on each pixel in the image and replaces its value with the average value of the adjacent pixels. Median blur mainly operates on each pixel in the image and replaces its value with the median value of the adjacent pixels.

[0344] Step 1303 : Process the lawn image to be trained using a preset model to obtain the predicted type of the lawn image to be trained; optimize the parameters of the preset model according to the labeling type and the predicted type of the lawn image to be trained to obtain a recognition model.

[0345] The preset model is a pre-set network model. This network model can be based on a convolutional neural network. Specifically, the labeled type of the lawn image to be trained can be used as the label for the predicted type. The preset model is trained and its parameters are optimized to make the predicted type increasingly close to the labeled type. When a preset condition is met, such as when the predicted type and the labeled type are 90% identical, training is terminated, resulting in a recognition model. The recognition model can be used to process the lawn image to be identified and identify the target type of the lawn image to be identified.

[0346] This solution enriches the training samples by adding noise and / or interference to the lawn images to be trained, which can simulate as many possible situations in actual scenes as possible. Using richer training samples to train the recognition model can improve the recognition accuracy of the recognition model to a certain extent.

[0347] In one possible implementation, at least a portion of the lawn images to be trained in the training data set is subjected to one or more of the following combined processing to obtain a processed lawn image: blurring processing, distortion processing, overexposure processing, adding pixel blocks, adding shadows of target objects, and adding preset objects.

[0348] Specifically, blur processing of the lawn image to be identified can be achieved using blur filters in image processing software or programming languages. Blur processing methods include at least one of the following: motion blur, depth of field blur, rotation blur, scale blur, Gaussian blur, mean blur, and median blur. Motion blur is blurring caused by the movement of objects or cameras in the image. To simulate this effect, a method of randomly offsetting or rotating pixels can be used to produce a blurring effect. Depth of field blur is a blurring effect caused by changes in the focal length of the camera lens. This effect can be simulated by blurring the background and foreground pixels in the image to varying degrees. Rotation blur can be simulated by rotating the image and applying a filter. Scale blur is a blurring effect caused by zooming in or out of the image. This effect can be simulated by zooming the image and then applying a filter.

[0349] Specifically, the image of the lawn to be identified can be distorted using transformation functions in image processing software or programming languages. Distortion methods include at least one of the following: distortion transformation, shearing transformation, rotation transformation, skew transformation, and affine transformation. Distortion transformation maps pixels in an image, which can distort or deform a portion of the image. This transformation can typically be implemented by defining a transformation matrix. Shearing transformation displaces pixels in an image vertically or horizontally. This transformation can be implemented by translating pixels horizontally or vertically. Rotation transformation rotates an image, which can be implemented by defining a rotation center and rotation angle. Skew transformation displaces pixels in an image along a straight line, which can be implemented by defining two dislocation points. Affine transformation maps pixels in an image, which can distort or deform a portion of the image. This transformation can be implemented by defining an affine matrix.

[0350] Specifically, taking the visual sensor 130 as a camera, for example, overexposure of the image can be achieved by increasing the exposure time or increasing the ISO sensitivity. Alternatively, the overexposure effect can be simulated by adjusting parameters such as the brightness, contrast, and color of the lawn image to be identified. Overexposure processing methods include at least one of the following: increasing brightness, increasing contrast, color adjustment, highlight adjustment, and sharpening. Increasing brightness simulates overexposure by increasing the overall brightness of the image. This can be achieved using a brightness adjustment function in image processing software or programming languages. Increasing contrast simulates overexposure by increasing the contrast of the image. This can be achieved using a contrast adjustment function in image processing software or programming languages. Color adjustment simulates overexposure by adjusting the color of the image. This can be achieved using a color adjustment function in image processing software or programming languages. Highlight adjustment simulates overexposure by increasing the highlights of the image. This can be achieved using a highlight adjustment function in image processing software or programming languages. Sharpening simulates overexposure by sharpening the edges and details of the image. This can be achieved using a sharpening function in image processing software or programming languages.

[0351] Specifically, pure color pixel blocks may be added to the lawn image to be trained.

[0352] Specifically, shadows of objects that may appear on the lawn can be added to the lawn image to be trained, or objects that may appear on the lawn can be added to enrich the training samples, and then a recognition model with higher recognition accuracy can be obtained based on the richer training samples.

[0353] In one implementation, shadows of any of the following target objects are added to the lawn image to be trained: trees, fences, and houses.

[0354] In one implementation, any of the following preset objects is added to the lawn image to be trained: a water pipe, a faucet, a water gun, a toy, animal feces, fallen leaves, and branches.

[0355] In one implementation, the lawn image to be trained includes a color image and / or a grayscale image.

[0356] In one embodiment, referring to Figures 2 and 3 , a controller 150 is electrically connected to a visual sensor 130. The controller 150 is configured to obtain a lawn image to be identified from the visual sensor 130 and process the lawn image using a preset recognition model to determine the target type of the lawn image to be identified. The preset recognition model is obtained by training the preset model using a training lawn image and a processed lawn image; the training lawn image has a labeled type; and the processed lawn image is obtained by adding noise and / or interference to at least a portion of the training lawn image.

[0357] 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, the visual sensor 130 may be used to capture an image of the lawn to be identified. The acquisition unit 151 in the controller 150 acquires the image of the lawn to be identified from the visual sensor 130. The processing unit 152 processes the image of the lawn to be identified using a preset recognition model to determine the target type of the lawn to be identified.

[0358] This solution enriches the training samples by adding noise and / or interference to the lawn images to be trained. This enriched training sample yields a higher recognition accuracy for the recognition model. This more accurate recognition model can improve the lawn maintenance effectiveness of autonomous lawn mowers to a certain extent.

[0359] FIG15 is a flowchart of a lawn image processing method provided in an embodiment of the present application. This embodiment is applicable to scenarios where a self-propelled lawn mower 100 is performing lawn maintenance and identifying grass and non-grass objects in a lawn image. The method can be executed by the self-propelled lawn mower 100. As shown in FIG15 , the method includes:

[0360] Step 1501: Acquire a lawn image to be identified.

[0361] Step 1502: Process the lawn image to be identified using a preset recognition model to obtain a target type of the lawn image to be identified; wherein the preset recognition model is obtained by training the preset model using the lawn image to be trained and the processed lawn image; the lawn image to be trained has a labeled type; and the processed lawn image is obtained by adding noise and / or interference to at least a portion of the lawn image to be trained.

[0362] This solution enriches the training samples by adding noise and / or interference to the lawn images to be trained. This enriched training sample results in a higher recognition accuracy for the recognition model. Using this more accurate recognition model to identify grass and non-grass objects in lawn images can improve the effectiveness of self-propelled lawn mowers in lawn maintenance.

[0363] The embodiment of the present application also specifically provides a data-enhanced model training method for lawn mowers. FIG16 is a flow chart of a data-enhanced model training method provided in the embodiment of the present application. Referring to FIG16 , the specific process of the method is as follows:

[0364] Step 1601: Use a camera installed on the lawn mower to collect visual information data, such as color images, grayscale images, etc.

[0365] Step 1602: Screen and label the data. Labeling methods include but are not limited to manual labeling and automatic labeling.

[0366] Step 1603: perform data enhancement and expansion on the labeled data. The main enhancement methods include noise enhancement and interference enhancement. Noise enhancement methods include but are not limited to blurring, distortion, overexposure, etc. Interference enhancement methods include but are not limited to adding light and shadow, pure color pixel blocks, objects that may appear on the lawn, etc.

[0367] Step 1604: Use the data-enhanced expanded dataset to train the machine learning model.

[0368] Step 1605: After the training is completed, an algorithm model that can distinguish between grass and non-grass areas is obtained.

[0369] Figure 2 is a structural schematic diagram of a self-moving lawn mower 100 provided in an embodiment of the present application, and Figure 17 is a schematic diagram of a reference object 200 provided in an embodiment of the present application. Referring to Figures 2 and 17, an embodiment of the present application provides a self-moving lawn mower system, including: a self-moving lawn mower 100 and a reference object 200; the self-moving lawn mower 100 includes: a body 110; a walking wheel assembly 120, configured to support the body; a visual sensor 130, installed to the body, configured to obtain images of the surroundings of the self-moving lawn mower; in combination with Figures 2 and 3, the controller 150 is electrically connected to the visual sensor 130, and recognizes semantic information of the surroundings of the self-moving lawn mower 100 through images; the reference object 200 is configured to provide a comparison reference to the controller, and the reference object 200 simulates at least one of lawn, land, water surface, and sky.

[0370] 2 and 17 , the reference object may be installed in an area that can be captured by the visual sensor 130 , such as a wall, a tree trunk, a charging pile, and the like.

[0371] 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, the visual sensor 130 may be used to capture images of the surroundings of the self-propelled lawn mower 100. The acquisition unit 151 in the controller 150 acquires the images from the visual sensor 130. The processing unit 152 acquires reference information of the reference object 200 and recognizes semantic information about the surroundings of the self-propelled lawn mower 100 using the reference information and the images. Specifically, the visual sensor 130 may be used to capture images of the reference object 200. The acquisition unit 151 acquires the images of the reference object 200 from the visual sensor 130 and extracts the reference information of the reference object 200 from the images. The acquisition unit 151 may also acquire real information about the reference object 200. The processing unit 152 transforms the images of the surroundings of the self-propelled lawn mower 100 based on the real information and reference information of the reference object 200 to obtain a transformed image. The transformed image is processed using a semantic recognition program to obtain semantic information about the surroundings of the self-propelled lawn mower 100. The semantic recognition program may be a machine learning program, such as an intelligent recognition model based on machine learning; the semantic information includes at least grass, grass types, and non-grass objects.

[0372] This solution reduces the deviation between the visual information on the image and the real information by combining the reference information of the reference object in the process of image recognition of the semantic information around the self-propelled lawn mower 100, thereby improving the accuracy of semantic information recognition.

[0373] In one implementation, referring to FIG. 17 , the self-propelled lawn mower system further includes a charging station 300 , and the reference object 200 is disposed on the charging station 300 .

[0374] The reference object 200 is arranged at the charging pile 300, which reduces the installation steps when the reference object 200 and the charging pile 300 are installed separately.

[0375] In one implementation, the reference object 200 has geometric features, and the geometric features are used to assist the controller 150 in identifying the reference object 200 .

[0376] Processing unit 152 extracts geometric features from an image surrounding the mobile lawn mower 100. If the geometric features are target geometric features, the target image corresponding to the geometric features is identified as reference object 200, and reference information of reference object 200 is extracted. Geometric features allow for quick and convenient identification of reference object 200. Target geometric features can be planar geometric shape features or solid geometric body features. Target geometric features include, but are not limited to, features of objects such as cubes and spheres that have a specific visual recognition structure. Specifically, geometric features from the image can be extracted using at least one of the following methods: edge extraction and feature point extraction. Because edges are areas of the image where local changes are most pronounced, geometric features can be extracted by detecting these areas. Edge extraction typically uses extreme value regions of first-order derivatives to detect edges. Specifically, by calculating the derivative of an image function in a certain direction, regions where the derivative undergoes extreme value changes are identified; these regions are considered image edges. Feature point extraction involves finding feature points in an image, i.e., points that exhibit significant changes or unique shapes within the image. For example, corners are the intersection points of two edges in an image. They typically have a larger curvature value than the surrounding area, so geometric features can be extracted by detecting these points. Similarly, key points and spots are regions with significant features in an image, and geometric features can be extracted from these regions. Once the geometric features in an image are detected, descriptors can be used to describe these features. Descriptors can be gray-level co-occurrence matrices, histograms of oriented gradients, scale-invariant feature transforms, accelerated robust features, etc. These descriptors can be used to compare and match feature points in an image for tasks such as image recognition, classification, and retrieval.

[0377] In one implementation, the controller 150 is configured to identify the reference object 200 through geometric features, extract color features of the reference object 200 , and perform transformation processing on the image according to the color features to obtain a transformed image.

[0378] The reference information provided by reference object 200 to controller 150 may be color features. The color features of reference object 200 include, but are not limited to, colors similar to natural references, such as lawn color and sky color. After identifying reference object 200 using geometric features, processing unit 152 in controller 150 can extract the color features of reference object 200. Acquisition unit 151 in controller 150 can also acquire the actual color information of reference object 200. Processing unit 152 can then obtain the difference between the extracted color features of reference object 200 and the actual color information. This difference is then subtracted from the color information of the image to generate a transformed image.

[0379] Color features are important features of images. Changes in the outdoor working environment, such as changes in light, cause a deviation between the visual information on the image and the real information, which mainly affects the color features. Therefore, the difference between the extracted color features of the reference object 200 and the real color features is used as a transformation benchmark to transform the color features of the image to reduce the deviation between the visual information on the image and the real information. Using the transformed image for intelligent recognition can improve recognition accuracy.

[0380] FIG18 is a flow chart of a control method for a self-propelled lawn mower according to an embodiment of the present application. This embodiment is applicable to a scenario in which the self-propelled lawn mower 100 performs intelligent recognition of images around the self-propelled lawn mower 100 while maintaining a lawn. The method can be executed by the self-propelled lawn mower 100, which includes a visual sensor 130. As shown in FIG18 , the method includes:

[0381] Step 1801 : Acquire an image of the reference object 200 through the visual sensor 130 and extract reference information.

[0382] In step 1802 , the visual sensor 130 is used to obtain an image of the surroundings of the self-propelled lawn mower 100 .

[0383] Step 1803 : transform the image around the self-propelled lawn mower 100 according to the reference information to obtain a transformed image.

[0384] In step 1804 , the transformed image is processed by a semantic recognition program to obtain semantic information about the surroundings of the self-propelled lawn mower 100 .

[0385] This scheme uses the reference information of the reference object to transform the image around the self-propelled lawn mower to obtain a transformed image, which minimizes the deviation between the visual information on the image and the real information. Then, the transformed image is used for intelligent recognition to improve the recognition accuracy.

[0386] In one implementation, the reference object is a known object in the natural environment.

[0387] The known objects can be identified by a semantic recognition program, and the known objects include at least lawn, land, water, sky, etc.

[0388] In one implementation, the reference object is an artificially placed simulated object.

[0389] The present invention also provides a method for determining a feature conversion benchmark based on lawn markers for a lawn mower equipped with a camera. FIG19 is a flow chart of a method for determining a feature conversion benchmark based on lawn markers provided by the present invention. Referring to FIG19 , the specific process of the method is as follows:

[0390] Step 1901: Power on the lawn mower and start mowing.

[0391] Step 1902: During the mowing process, visual information on the lawn is collected through a camera.

[0392] Step 1903: Search for the marker in the image based on the preset geometric information of the marker.

[0393] Step 1904: extract relevant visual information based on the landmarks in the searched image.

[0394] Step 1905: Obtain a method for transforming the visual information based on the preset visual information of the landmark and the extracted visual information.

[0395] Step 1906: Use the obtained visual information transformation method to correct the entire image.

[0396] Step 1907: The corrected image is used as input to the visual processing algorithm for visual algorithm processing.

[0397] FIG20 shows a schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present application. The electronic device 10 can be a controller in the self-propelled lawn mower 100, which can be mounted on the body of the self-propelled lawn mower 100 and communicate locally with the visual sensor 130 and the posture sensor 140 in the self-propelled lawn mower 100. The electronic device 10 can also be a remote server that remotely interacts with the visual sensor 130 and the posture sensor 140 in the self-propelled lawn mower 100. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, and wearable devices (such as helmets, glasses, watches, etc.). The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0398] As shown in FIG20 , the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 and a random access memory (RAM) 13, that is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0399] Multiple components in electronic device 10 are connected to I / O interface 15, including an input unit 16, such as a keyboard and mouse; an output unit 17, such as various types of displays and speakers; a storage unit 18, such as a magnetic disk and optical disk; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0400] The processor 11 may be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above.

[0401] In some embodiments, any of the above methods may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of any of the above methods may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform any of the above methods in any other appropriate manner (e.g., by means of firmware).

[0402] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This is not limited herein.

[0403] The above shows and describes the basic principles, main features and advantages of this application. Those skilled in the art should understand that the above embodiments do not limit this application in any form, and any technical solutions obtained by equivalent replacement or equivalent transformation fall within the scope of protection of this application.

Claims

1. A self-propelled device, comprising: body; A running wheel assembly configured to support the fuselage; A visual sensor configured to acquire an image in a direction of travel of the self-moving device; A posture sensor configured to obtain the posture of the self-moving device; A controller is electrically connected to the visual sensor and the posture sensor, and the controller is configured to calculate a first inclination angle of the ground in the direction of travel of the self-moving device based on the image, calculate a second inclination angle of the ground based on the first inclination angle and the posture, and control the walking wheel assembly based on the second inclination angle.

2. The self-moving device according to claim 1, wherein: The first inclination angle is a relative inclination angle of the ground relative to the self-moving device; the second inclination angle is an absolute inclination angle of the ground relative to the direction of gravity.

3. The self-moving device according to claim 1, wherein: The visual sensor includes at least one of a TOF depth camera, an IR structured light camera, and a binocular camera.

4. The self-moving device according to claim 3, wherein: The controller creates a depth image from the image, extracts a ground point cloud from the depth image, and acquires the first tilt angle through the ground point cloud.

5. The self-moving device according to claim 1, wherein: When the second tilt angle is greater than or equal to a first threshold, the controller determines that the ground in the direction of travel of the self-moving device is a non-safe area and changes the direction of travel.

6. The self-moving device according to claim 5, wherein: When the second tilt angle is less than a first threshold, the controller determines that the ground in the moving direction of the self-moving device is a safe area and maintains the moving direction.

7. The self-moving device according to claim 6, wherein: The first threshold is determined according to the friction of the ground.

8. The self-moving device according to claim 1, wherein: The moving direction of the self-moving device is the front of the self-moving device.

9. The self-moving device according to claim 1, wherein: The attitude sensor is an accelerometer.

10. A method for controlling a self-moving device, comprising: Acquire the posture of the self-mobile device; and acquiring an image in the traveling direction of the mobile device; Calculating a first inclination angle of the ground in the direction of travel of the self-moving device according to the image; The second inclination angle of the ground is calculated by combining the first inclination angle and the posture; and whether to move along the moving direction is determined according to the second inclination angle.

11. A control device for a self-moving device, comprising: An acquisition unit, configured to acquire the posture of the self-moving device; and acquiring an image in the traveling direction of the mobile device; A processing unit, configured to calculate a first tilt angle of the ground in the direction of travel of the self-moving device according to the image; The processing unit is further configured to calculate a second tilt angle of the ground in combination with the first tilt angle and the posture; The control unit is configured to determine whether to move along the moving direction according to the second inclination angle.

12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the control method of the self-moving device according to claim 10 .

13. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the control method of the self-moving device according to claim 10 when executed.

14. A self-propelled lawn mower comprising: body; A running wheel assembly configured to support the fuselage; a visual sensor configured to acquire an image of a lawn in a working area where the self-propelled lawn mower is located; A controller is electrically connected to the visual sensor, and the controller is configured to obtain and receive the lawn image of the working area obtained by the visual sensor according to the inspection instruction; Processing the lawn image to obtain characteristic information of the grass; wherein the characteristic information at least includes the height and color of the grass; A mowing plan or an inspection plan is formulated according to the characteristic information, and the self-propelled lawn mower is controlled to work according to the mowing plan or the inspection plan; wherein the mowing plan includes the time for executing the mowing task; and the inspection plan includes the time for generating the next inspection instruction.

15. A control method for a self-propelled lawn mower, applied to the self-propelled lawn mower, the method comprising: Obtaining an inspection instruction, and obtaining a lawn image of a working area of ​​the self-propelled lawn mower according to the inspection instruction; Processing the lawn image to obtain characteristic information of the grass; wherein the characteristic information at least includes the height and color of the grass; A mowing plan or an inspection plan is formulated according to the characteristic information, and the self-propelled lawn mower is controlled to work according to the mowing plan or the inspection plan; wherein the mowing plan includes the time for executing the mowing task; and the inspection plan includes the time for generating the next inspection instruction.

16. The method according to claim 15, wherein: The formulating of a mowing plan or an inspection plan according to the feature information includes: determining a mowing height threshold based on the color; If the height is greater than the mowing height threshold, controlling the self-propelled mower to perform a mowing task; If the height is less than or equal to the mowing height threshold, a mowing plan or an inspection plan is formulated according to the difference between the mowing height threshold and the height.

17. The method according to claim 16, wherein: The characteristic information also includes size, shape and / or density; then determining the mowing height threshold according to the color includes: Acquire season information, and determine the type of grass according to the season information, the size, the shape, and the color; A mowing height threshold is determined according to the type, the density and the season information.

18. The method according to claim 16, wherein: The formulating a mowing plan or an inspection plan according to the difference between the mowing height threshold and the height includes: If the difference is less than or equal to a preset threshold, formulating a mowing plan according to the difference; If the difference is greater than a preset threshold, an inspection plan is formulated according to the difference.

19. The method according to claim 15, wherein: The height is determined based on point cloud data of grass; and the point cloud data is determined based on the lawn image.

20. A control device for a self-propelled lawn mower, applied to the self-propelled lawn mower, the device comprising: an acquisition unit, configured to acquire an inspection instruction, and acquire a lawn image of a working area of ​​the self-propelled lawn mower according to the inspection instruction; A processing unit, configured to process the lawn image to obtain characteristic information of the grass; wherein the characteristic information at least includes the height and color of the grass; The processing unit is further configured to formulate a mowing plan or an inspection plan according to the feature information; A control unit is configured to control the self-propelled lawn mower to work according to the mowing plan or the inspection plan; wherein the mowing plan includes the time for executing the mowing task; and the inspection plan includes the time for generating the next inspection instruction.

21. The device according to claim 20, wherein: The processing unit is specifically configured as follows: determining a mowing height threshold based on the color; If the height is greater than the mowing height threshold, controlling the self-propelled lawn mower to perform a mowing task; If the height is less than or equal to the mowing height threshold, a mowing plan or an inspection plan is formulated according to the difference between the mowing height threshold and the height.

22. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, wherein the computer program is executed by the at least one processor so as to enable the at least one processor to perform the control method of the self-moving lawn mower according to any one of claims 15 to 19.

23. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the control method of the self-propelled lawn mower according to any one of claims 15 to 19 when executed.

24. A self-propelled device, comprising: body, A running wheel assembly configured to support the fuselage; an inertial measurement unit, disposed in the fuselage, the inertial measurement unit comprising at least one of an accelerometer and a gyroscope; A controller is electrically connected to the inertial measurement unit, and the controller is configured to: Controlling the self-moving device to walk; receiving inertial data from the inertial measurement unit; The inertial data received within the first time window is statistically analyzed, and the positioning parameters of the inertial measurement unit are adjusted according to the statistical results.

25. The self-moving device according to claim 24, wherein: The statistical result reflects the flatness of the road section passed by the mobile device within the first time window.

26. The self-moving device according to claim 24, wherein: The positioning parameter is the noise of the inertial data.

27. The self-moving device according to claim 24, wherein: The positioning parameter is the weight of the inertial data when the inertial data is fused with data of other modalities for positioning.

28. The self-moving device according to claim 24, wherein: The value range of the first time window is greater than or equal to the first duration threshold and less than or equal to the second duration threshold; The first duration threshold is smaller than the second duration threshold.

29. The self-moving device according to claim 24, wherein: The first time window is determined according to the walking speed of the self-moving device.

30. A positioning method for a mobile device, comprising: Obtaining a positioning request, and obtaining inertial data of the mobile device within a first time window according to the positioning request; Performing statistics on the inertial data, and adjusting positioning parameters corresponding to the inertial data according to the statistical results; The position of the self-moving device is determined according to the inertial data and the positioning parameters.

31. A positioning device for a mobile device, wherein: include: an acquisition unit, configured to acquire a positioning request, and acquire inertial data of the mobile device within a first time window according to the positioning request; a processing unit, configured to perform statistics on the inertial data and adjust positioning parameters corresponding to the inertial data according to the statistical results; The processing unit is further configured to determine the position of the self-moving device according to the inertial data and the positioning parameters.

32. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the positioning method for a mobile device according to claim 30.

33. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the positioning method for a mobile device as claimed in claim 30 when executed.

34. A model training method for lawn image processing, comprising: Acquire a data set to be trained; wherein the data set to be trained includes a plurality of lawn images to be trained; the lawn images to be trained have a label type; Adding noise and / or interference to at least part of the lawn images to be trained in the training data set to obtain processed lawn images; and adding the processed lawn images as new lawn images to be trained into the training data set to update the training data set; The lawn image to be trained is processed by using a preset model to obtain the predicted type of the lawn image to be trained; and the parameters of the preset model are optimized according to the annotation type and the predicted type of the lawn image to be trained to obtain a recognition model.

35. The method of claim 34, wherein: The adding noise and / or interference to at least a portion of the lawn images to be trained in the training data set to obtain processed lawn images includes: Perform one or more of the following combined processing on at least part of the lawn images to be trained in the training data set to obtain processed lawn images: Blur, distort, overexpose, add pixel blocks, add shadows to target objects, and add preset objects.

36. The method of claim 35, wherein: The preset objects include one or more combinations of the following: Hoses, faucets, water guns, toys, animal droppings, fallen leaves and branches.

37. The method of claim 35, wherein: The target object includes one or more of the following combinations: Trees, fences, and houses.

38. The method of claim 34, wherein: The lawn image to be trained includes a color image and / or a grayscale image.

39. A lawn image processing method, applied to a self-propelled lawn mower, the method comprising: Acquire a lawn image to be identified; Processing the lawn image to be identified by using a preset recognition model to obtain a target type of the lawn image to be identified; The preset recognition model is obtained by training the preset model using the lawn image to be trained and the processed lawn image; the lawn image to be trained has a label type; The processed lawn image is obtained by adding noise and / or interference to at least a portion of the lawn image to be trained.

40. The method of claim 39, wherein: The processed lawn image is obtained by adding noise and / or interference to at least a portion of the lawn image to be trained, and includes: Perform one or more of the following combined processing on at least part of the lawn image to be trained to obtain Image of lawn after grooming: Blur, distort, overexpose, add pixel blocks, add shadows to target objects, and add preset objects.

41. The method of claim 40, wherein: The preset objects include one or more combinations of the following: Hoses, faucets, water guns, toys, animal droppings, fallen leaves and branches.

42. The method of claim 40, wherein: The target object includes one or more of the following combinations: Trees, fences, and houses.

43. The method of claim 39, wherein: The lawn image to be trained includes a color image and / or a grayscale image.

44. A model training device for lawn image processing, comprising: An acquisition unit is configured to acquire a data set to be trained; wherein the data set to be trained includes a plurality of lawn images to be trained; and the lawn images to be trained have a label type; A processing unit is configured to add noise and / or interference to at least a portion of the lawn images to be trained in the dataset to be trained to obtain a processed lawn image; and add the processed lawn image as a new lawn image to be trained into the dataset to be trained to update the dataset to be trained; The processing unit is further configured to process the lawn image to be trained using a preset model to obtain the predicted type of the lawn image to be trained; and optimize the parameters of the preset model according to the annotation type and predicted type of the lawn image to be trained to obtain the recognition model.

45. A lawn image processing device, applied to a self-propelled lawn mower, comprising: An acquisition unit, configured to acquire a lawn image to be identified; A processing unit, configured to process the lawn image to be identified using a preset recognition model to obtain a target type of the lawn image to be identified; The preset recognition model is obtained by training the preset model using the lawn image to be trained and the processed lawn image; the lawn image to be trained has a label type; The processed lawn image is obtained by adding noise and / or interference to at least a portion of the lawn image to be trained.

46. ​​An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The machine program is executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 34 to 43.

47. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the method of any one of claims 34-43 when executed.

48. A self-propelled lawn mower system comprising: Self-moving mowers and reference objects; The self-propelled lawn mower comprises: body; A running wheel assembly configured to support the fuselage; a visual sensor mounted to the body and configured to acquire an image of the periphery of the self-propelled lawn mower; a controller, electrically connected to the visual sensor, for recognizing semantic information of the periphery of the self-propelled lawn mower through the image; The reference object is configured to provide a control reference for the controller, and the reference object simulates at least one of lawn, land, water surface, and sky. The system according to claim 48, further comprising a charging pile, wherein the reference object is arranged at the charging pile.

50. The system of claim 48, wherein: The reference object has a geometric feature, and the geometric feature is used to assist the controller in identifying the reference object.

51. The system of claim 50, wherein: The geometric feature is a plane geometric shape feature or a solid geometric body feature.

52. The system of claim 51, wherein: The controller is configured to identify the reference object through the geometric features, extract the color features of the reference object, and transform the image according to the color features to obtain a transformed image.

53. The system of claim 52, wherein: The controller is further configured to process the transformed image through a semantic recognition program to obtain the semantic information.

54. A control method for a self-propelled lawn mower, the self-propelled lawn mower comprising a visual sensor, the control method comprising: Acquire an image of a reference object through the visual sensor to extract reference information; Acquire an image of the periphery of the self-propelled lawn mower by means of the visual sensor; Performing transformation processing on the image of the periphery of the self-propelled lawn mower according to the reference information to obtain a transformed image; The transformed image is processed by a semantic recognition program to obtain semantic information of the surroundings of the self-propelled lawn mower.

55. The method of claim 54, wherein: The reference objects are known objects in the natural environment.

56. The method of claim 54, wherein: The reference object is an artificially placed simulated object.

57. The method of claim 54, wherein: The reference object is arranged on the charging pile.

58. The method of claim 54, wherein: The reference information includes color characteristics.

59. The method of claim 54, wherein: The semantic recognition program is a machine learning program.

60. A control device for a self-propelled lawn mower, the self-propelled lawn mower comprising a visual sensor, the control device comprising: An acquisition unit, configured to acquire an image of a reference object through the visual sensor; A processing unit, configured to extract reference information of the reference object; The acquisition unit is further configured to acquire an image of the periphery of the self-propelled lawn mower through the visual sensor; The processing unit is further configured to perform transformation processing on the image of the periphery of the self-propelled lawn mower according to the reference information to obtain a transformed image; The processing unit is further configured to process the transformed image through a semantic recognition program to obtain semantic information of the surroundings of the self-propelled lawn mower.

61. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, wherein the computer program is executed by the at least one processor so as to enable the at least one processor to perform the control method of the self-moving lawn mower according to any one of claims 54 to 59.

62. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the control method of the self-propelled lawn mower according to any one of claims 54 to 59 when executed.

Images