Model training method and lawn image processing method

Abstract

The invention discloses a model training method and a lawn image processing method. The method comprises the steps of obtaining a to-be-trained data set; the to-be-trained data set comprises a plurality of to-be-trained lawn images; the to-be-trained lawn image has an annotation type; adding noise and / or interference into at least part of the to-be-trained lawn image in the to-be-trained data set to obtain a processed lawn image; the processed lawn image is used as a new to-be-trained lawn image to be added into the to-be-trained data set so as to update the to-be-trained data set; processing the to-be-trained lawn image by using a preset model to obtain a prediction type of the to-be-trained lawn image; and according to the marking type and the prediction type of the to-be-trained lawn image, optimizing parameters of a preset model to obtain an identification model. The noise and / or interference are / is added into the to-be-trained lawn image, so that the training samples can be enriched, possible conditions in an actual scene can be simulated as much as possible, and the recognition accuracy of the recognition model obtained by training the richer training samples is higher.

Description

Technical Field

[0001] This application relates to a model training and image processing method, specifically to a model training method and a lawn image processing method. Background Art

[0002] When a self-propelled lawn mower mows the lawn, it needs to intelligently identify the grass and non-grass areas on the lawn. The obtained recognition results are very important information for tasks such as obstacle avoidance or work boundary confirmation.

[0003] In the related art, the recognition ability of a self-propelled lawn mower for grass and non-grass areas on the lawn mainly comes from machine learning algorithms. Specifically, through supervised learning with a large number of labeled lawn image samples, a recognition model is obtained.

[0004] However, due to the limited training samples in the above method, it usually cannot cover all possible situations in the actual scenario, resulting in the recognition accuracy of the recognition model needing to be further improved.

[0005] This section provides background information related to this application, and these background information are not necessarily prior art. Summary of the Invention

[0006] One objective of this application is to solve or at least mitigate part or all of the above problems. For this reason, one objective of this application is to provide a recognition model with higher recognition accuracy.

[0007] To achieve the above objective, this application adopts the following technical solutions:

[0008] A model training method applied to lawn image processing, including:

[0009] Obtain a dataset to be trained; wherein, the dataset to be trained includes multiple lawn images to be trained; the lawn images to be trained have labeled types;

[0010] Add noise and / or interference to at least some of the lawn images to be trained in the dataset to be trained to obtain processed lawn images; and add the processed lawn images as new lawn images to be trained to the dataset to be trained to update the dataset to be trained;

[0011] Process the lawn images to be trained using a preset model to obtain the predicted types of the lawn images to be trained; optimize the parameters of the preset model according to the labeled types and predicted types of the lawn images to be trained to obtain a recognition model.

[0012] In some embodiments, perform one or more of the following combined processing on at least some of the lawn images to be trained in the dataset to be trained to obtain processed lawn images:

[0013] Blurring, warping, overexposure processing, adding pixel blocks, adding shadows of target objects, and adding preset objects.

[0014] In some embodiments, the preset objects include one or more of the following combinations:

[0015] Water pipes, faucets, water guns, toys, animal feces, fallen leaves, and tree branches.

[0016] In some embodiments, the target objects include one or more of the following combinations:

[0017] Trees, fences, and houses.

[0018] In some embodiments, the lawn images to be trained include color images and / or grayscale images.

[0019] A method for processing lawn images, applied to a self-propelled lawn mower, the method comprising:

[0020] Obtaining a lawn image to be recognized;

[0021] Processing the lawn image to be recognized by using a preset recognition model to obtain the target type of the lawn image to be recognized;

[0022] Wherein, the preset recognition model is obtained by training a preset model by using lawn images to be trained and processed lawn images; the lawn images to be trained have labeled types; the processed lawn images are obtained by adding noise and / or interference to at least part of the lawn images to be trained.

[0023] In some embodiments, perform one or more of the following combined processing on at least part of the lawn images to be trained to obtain processed lawn images:

[0024] Blurring, warping, overexposure processing, adding pixel blocks, adding shadows of target objects, and adding preset objects.

[0025] In some embodiments, the preset objects include one or more of the following combinations:

[0026] Water pipes, faucets, water guns, toys, animal feces, fallen leaves, and tree branches.

[0027] In some embodiments, the target objects include one or more of the following combinations:

[0028] Trees, fences, and houses.

[0029] In some embodiments, the lawn images to be trained include color images and / or grayscale images.

[0030] A model training device for lawn image processing, comprising:

[0031] An acquisition unit, configured to acquire a dataset to be trained; wherein, the dataset to be trained includes multiple lawn images to be trained; the lawn images to be trained have annotation types;

[0032] A processing unit, configured to add noise and / or interference to at least part of the lawn images to be trained in the dataset to be trained, to obtain processed lawn images; and add the processed lawn images as new lawn images to be trained to the dataset to be trained, so as to update the dataset to be trained;

[0033] The processing unit is further configured to process the lawn images to be trained by using a preset model, to obtain prediction types of the lawn images to be trained; and optimize parameters of the preset model according to the annotation types and prediction types of the lawn images to be trained, to obtain an identification model.

[0034] A lawn image processing device, applied to a self-propelled lawn mower, the device comprising:

[0035] An acquisition unit, configured to acquire a lawn image to be recognized;

[0036] A processing unit, configured to process the lawn image to be recognized by using a preset identification model, to obtain a target type of the lawn image to be recognized;

[0037] Wherein, the preset identification model is obtained by training a preset model by using the lawn images to be trained and the processed lawn images; the lawn images to be trained have annotation types; the processed lawn images are obtained by adding noise and / or interference to at least part of the lawn images to be trained.

[0038] An electronic device, the electronic device comprising:

[0039] At least one processor; and

[0040] A memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor can execute the method according to any embodiment of the present application.

[0042] A computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any embodiment of the present application when executed.

[0043] The advantages of the present application are as follows: Adding noise and / or interference to the lawn image to be trained can enrich the training samples and simulate as many possible situations in the actual scenario as possible. The recognition model trained with more abundant training samples can improve the recognition accuracy of the recognition model. Description of the Drawings

[0044] Figure 1 is a schematic diagram of the working scenario of a self-propelled lawn mower provided by an embodiment of the present application;

[0045] Figure 2 is a schematic diagram of the structure of a self-propelled lawn mower provided by an embodiment of the present application;

[0046] Figure 3 is a schematic diagram of the structure of a self-propelled lawn mower provided by an embodiment of the present application;

[0047] Figure 4 is a schematic diagram of the slope detection scenario of a self-propelled device provided by an embodiment of the present application;

[0048] Figure 5 is a flowchart of a control method for a self-propelled device provided by an embodiment of the present application;

[0049] Figure 6 is a flowchart of a slope detection algorithm based on point cloud provided by an embodiment of the present application;

[0050] Figure 7 is a schematic diagram of a lawn image provided by an embodiment of the present application;

[0051] Figure 8 is a flowchart of a control method for a self-propelled lawn mower provided by an embodiment of the present application;

[0052] Figure 9 is a flowchart of an intelligent mowing decision algorithm provided by an embodiment of the present application;

[0053] Figure 10 is a schematic diagram of the positioning scenario of a self-propelled device driving on a lawn provided by an embodiment of the present application;

[0054] Figure 11 is a flowchart of a positioning method for a self-propelled device provided by an embodiment of the present application;

[0055] Figure 12 is a flowchart of a positioning parameter adjustment algorithm based on inertial measurement unit data statistics provided by an embodiment of the present application;

[0056] Figure 13 is a flowchart of a model training method applied to lawn image processing provided by an embodiment of the present application;

[0057] Figures 14A to 14C They are schematic diagrams of a Gaussian blur map, a mean blur map, and a median blur map for blurring a lawn image to be trained provided by an embodiment of the present application;

[0058] Figure 15 It is a flowchart of a lawn image processing method provided by an embodiment of the present application;

[0059] Figure 16 It is a flowchart of a model training method based on data augmentation provided by an embodiment of the present application;

[0060] Figure 17 It is a schematic diagram of a reference object provided by an embodiment of the present application;

[0061] Figure 18 It is a flowchart of a control method for a self-propelled lawn mower provided by an embodiment of the present application;

[0062] Figure 19 It is a flowchart of a method for establishing a feature transformation reference based on lawn markers provided by an embodiment of the present application;

[0063] Figure 20 It is a schematic structural diagram of an electronic device for implementing the method of an embodiment of the present application. Detailed Embodiments

[0064] Before explaining any embodiment of the present application in detail, it should be understood that the present application is not limited to the structural details and component arrangements described in the following description or shown in the above drawings.

[0065] In the present application, the terms "comprising", "including", "having" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device.

[0066] In the present application, the term "and / or" is an associative relationship describing associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, in the present application, the character " / " generally indicates that the associated objects before and after are in an "and / or" relationship.

[0067] In this application, the terms "connected", "combined", "coupled", and "mounted" can be direct connections, combinations, couplings, or mountings, or indirect connections, combinations, couplings, or mountings. For example, a direct connection means that two parts or components are connected together without an intermediate member, and an indirect connection means that two parts or components are respectively connected to at least one intermediate member, and these two parts or components are connected through the intermediate member. In addition, "connected" and "coupled" are not limited to physical or mechanical connections or couplings, and may include electrical connections or couplings.

[0068] In this application, those of ordinary skill in the art will understand that relative terms used in connection with a quantity or condition (e.g., "about", "approximately", "substantially", etc.) are intended to include the value and have the meaning indicated by the context. For example, such relative terms include at least the degree of error associated with the measurement of a particular value, tolerances resulting from manufacture, assembly, use, etc. associated with a particular value. Such terms should also be considered to disclose a range defined by the absolute values of two endpoints. A relative term may refer to a plus or minus a certain percentage (e.g., 1%, 5%, 10% or more) of the indicated value. A numerical value that does not employ a relative term should also be disclosed as a particular value with a tolerance. In addition, "substantially" when expressing a relative angular positional relationship (e.g., substantially parallel, substantially perpendicular) may refer to a plus or minus a certain number of degrees (e.g., 1 degree, 5 degrees, 10 degrees or more) from the indicated angle.

[0069] In this application, those of ordinary skill in the art will understand that the functions performed by a component can be performed by one component, multiple components, one part, or multiple parts. Similarly, the functions performed by a part can also be performed by one part, one component, or a combination of multiple parts.

[0070] In this application, the orientation terms such as "upper", "lower", "left", "right", "front", "rear", etc. are described based on the orientation and positional relationship shown in the drawings, and should not be construed as a limitation on the embodiments of this application. In addition, in the context, it should also be understood that when it is mentioned that one element is connected "above" or "below" another element, it can not only be directly connected "above" or "below" another element, but also be indirectly connected "above" or "below" another element through an intermediate element. It should also be understood that orientation terms such as upper side, lower side, left side, right side, front side, rear side, etc. not only represent the positive orientation, but can also be understood as the side orientation. For example, the lower side can include directly below, lower left, lower right, lower front, and lower rear, etc.

[0071] It should be noted that in the description, claims and the above-mentioned drawings of this application, terms such as "first", "second", etc. are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0072] In this application, the terms "controller", "processor", "central processing unit", "CPU", "MCU" can be interchanged. When using the units "controller", "processor", "central processing unit", "CPU", or "MCU" to perform specific functions, unless otherwise specified, these functions can be performed by a single one of the above units or multiple of the above units.

[0073] In this application, the terms "device", "module" or "unit" can be implemented in the form of hardware or software in order to achieve specific functions.

[0074] In this application, terms such as "calculate", "judge", "control", "determine", "identify", etc. refer to the operations and processes of a computer system or similar electronic computing devices (such as a controller, a processor, etc.).

[0075] The self-moving device provided by this application can be, for example, a self-moving lawn mower.

[0076] Figure 1 It is a schematic diagram of the working scenario of a self-moving lawn mower provided by an embodiment of this application. As Figure 1As shown, the self - propelled lawn mower can move and work within the working area. The working area is generally outdoors and includes a lawn. The outdoor working environment is complex and changeable, which is challenging for the self - propelled lawn mower working in it. For example, the lawn terrain is diverse and may have slopes. If the slope is too high, it may cause the self - propelled lawn mower to tip over. There can be various types of grass on the lawn, and the growth of grass is generally affected by factors such as light, precipitation, season, and temperature, making it challenging for the self - propelled lawn mower to perform intelligent maintenance of the lawn. With the changes in weather and seasons, the lawn ground is prone to obvious changes in surface undulation, which will in turn cause the self - propelled lawn mower working on the lawn ground to jolt. Moreover, due to the complex and changeable outdoor working environment, including smoke, dust, weather, and light changes, etc., it may lead to an overall shift in the distribution of the acquired lawn images. Intelligent recognition can be carried out based on the lawn images, but the diversity and shift of the lawn images will affect the recognition accuracy to a certain extent.

[0077] Figure 2 is a schematic structural diagram of a self - propelled lawn mower 100 provided by an embodiment of the present application. Refer to Figure 2 , an embodiment of the present application provides a self - propelled device, including: a body 110; a traveling wheel assembly 120 configured to support the body 110; a vision sensor 130 configured to acquire an image in the traveling direction of the self - propelled device; in combination with Figure 2 and Figure 3 , an attitude sensor 140 configured to acquire the attitude of the self - propelled device; a controller 150 electrically connected to the vision sensor 130 and the attitude sensor 140. The controller 150 is configured to calculate a first inclination angle of the ground in the traveling direction of the self - propelled device based on the image in the traveling direction of the self - propelled device acquired by the vision sensor 130, calculate a second inclination angle of the ground in combination with the first inclination angle and the attitude of the self - propelled device, and control the traveling wheel assembly 120 according to the second inclination angle.

[0078] In combination with Figure 2 and Figure 3 , the vision sensor 130 can be installed on the body of the self - propelled device above the axle, and the vision sensor 130 can be flexibly installed in various areas of the body. For example, it can be directly installed on the housing of the body 110, and the installation orientation of the vision sensor 130 can include all angles. By flexibly setting the installation position and installation orientation of the vision sensor 130 on the self - propelled device, the acquisition angle of the image can be flexibly adjusted. Refer to Figure 2 and Figure 3 , the attitude sensor 140 and the controller 150 are also respectively arranged on the body of the self - propelled device, and the controller 150 is electrically connected to the vision sensor 130 and the attitude sensor 140 respectively.

[0079] Refer toFigure 3 The controller 150 may include an acquisition unit 151, a processing unit 152, and a control unit 153. Specifically, the image of the mobile device in the traveling direction may be collected by the vision sensor 130, and the attitude of the mobile device may be obtained by the attitude sensor 140. The acquisition unit 151 in the controller 150 obtains the image from the vision sensor 130 and obtains the attitude of the mobile device from the attitude sensor 140. The processing unit 152 processes each image in a preset manner to obtain a first inclination angle of the ground in the traveling direction of the mobile device. And the second inclination angle of the ground may be obtained according to the attitude of the mobile device and the first inclination angle. The control unit 153 may control the walking wheel assembly 120 to walk by using the second inclination angle.

[0080] In this solution, by combining the attitude of the mobile device and according to the first inclination angle, the second inclination angle is obtained, which improves the accuracy of slope angle recognition. Taking the mobile device as the self-propelled mower 100 as an example, in the same slope scenario, even if the self-propelled mower works on the slope in different attitudes, the second inclination angle of the slope can be accurately obtained by combining the attitude and the first inclination angle, thereby preventing the self-propelled mower 100 from tipping over.

[0081] In an implementable manner, the first inclination angle is the relative inclination angle of the ground with respect to the mobile device; the second inclination angle is the absolute inclination angle of the ground with respect to the gravity direction.

[0082] Figure 4 It is a schematic diagram of a slope detection scenario of a mobile device provided by an embodiment of the present application. Taking the mobile device as the self-propelled mower 100 as an example, as Figure 4 shown, the dashed line CD represents the horizon; the dashed line BH is parallel to the dashed line CD; the solid line BC represents the ground on which the self-propelled mower 100 is traveling; the solid line AB represents the ground in front of the traveling direction of the self-propelled mower 100; the dashed line BG is the extension line of the solid line BC; the dashed line EF represents the gravity direction; wherein, the dashed line CD and the dashed line EF are perpendicular to each other.

[0083] The self-propelled mower 100 can collect the front image in the traveling direction and process the image into a point cloud. Then, the ground point cloud is extracted from the point cloud, and the ground point cloud corresponds to the solid line AB. The inclination angle of the front ground is extracted through the ground point cloud, and the inclination angle is the relative inclination angle of the solid line AB with respect to the self-propelled mower 100, and the relative inclination angle can be represented by ∠ABG. Let ∠BCD represent the pitch angle in the attitude of the self-propelled mower 100. Since the dashed line BH is parallel to the dashed line CD and the dashed line BG is the extension of the solid line BC, then ∠BCD = ∠GBH. Adding the pitch angle to the relative inclination angle gives the first absolute inclination angle of the front ground with respect to the gravity direction, and the first absolute inclination angle can be represented by ∠ABH.

[0084] Similarly, the roll angle in the attitude of the self-propelled mower 100 can also be used to determine the second absolute inclination angle of the front ground with respect to the gravity direction.

[0085] The first absolute inclination angle and the second absolute inclination angle are used as the slope detection results. Further, the first absolute inclination angle can be compared with the first angle threshold, and the second absolute inclination angle can be compared with the second angle threshold. If the first absolute inclination angle is less than the first angle threshold and the second absolute inclination angle is less than the second angle threshold, it is determined that the front ground is a safe area. If the first absolute inclination angle is greater than or equal to the first angle threshold, or the second absolute inclination angle is greater than or equal to the second angle threshold, it is determined that the front ground is a non-safe area. Among them, both the first angle threshold and the second angle threshold can be preset according to the actual situation. The first angle threshold and the second angle threshold may not be equal. First, obtain the first absolute inclination angle and the second absolute inclination angle respectively according to the pitch angle and the roll angle in the attitude, and then comprehensively determine whether the front ground is a safe area according to the first absolute inclination angle and the second absolute inclination angle, which can improve the accuracy of the determination and further reduce the probability of the self-propelled device tipping over.

[0086] Among them, the first inclination angle refers to the relative inclination angle between the ground and the self-propelled device, while the second inclination angle refers to the absolute inclination angle of the ground and is not affected by the position of the self-propelled device. Specifically, combining the attitude of the self-propelled device, the influence of the attitude is removed from the first inclination angle to obtain the second inclination angle.

[0087] Specifically, according to the attitude of the self-propelled device, the inclination angle of the ground where the self-propelled device is located with respect to the gravity direction can be determined. Then, according to the inclination angle and the relative inclination angle of the ground in the traveling direction of the self-propelled device with respect to the self-propelled device, the absolute inclination angle of the ground in the traveling direction of the self-propelled device with respect to the gravity direction can be obtained.

[0088] In an actual application scenario, the absolute tilt angle of the ground where the self-moving device is located relative to the direction of gravity affects whether the self-moving device topples. Therefore, using the absolute tilt angle as the slope angle recognition result improves the accuracy of slope angle recognition.

[0089] In one implementable way, the vision sensor 130 may include at least one of the following: an active laser depth camera, a passive binocular stereo camera, and a passive binocular structured light camera.

[0090] In one implementable way, the vision sensor 130 includes at least one of a Time of Flight Camera (TOF) depth camera, an Infrared Ray (IR) structured light camera, and a binocular camera.

[0091] In one implementable way, the controller 150 creates a depth image from an image, extracts ground point cloud from the depth image, and obtains a first tilt angle through the ground point cloud.

[0092] The processing unit 152 in the controller 150 can create a depth image based on the image in the traveling direction of the self-moving device. Specifically, the depth information of each pixel point can be estimated by analyzing the geometric relationship between the pixels and objects in the image. The depth image represents the distance of each point in the scene relative to the camera. There are various methods to create a depth image from an image, including depth estimation based on a single image, stereo matching based on multiple images, etc. Among them, the stereo matching method refers to calculating the disparity by matching the corresponding pixel points of the same object in different images, so as to estimate the depth information.

[0093] The processing unit 152 can extract the ground point cloud from the depth image. Specifically, the depth image can be processed into a point cloud first, and then the ground point cloud can be extracted from the point cloud. Among them, various algorithms can be used to extract the ground point cloud from the point cloud. The various algorithms include detection based on three-dimensional features of the point cloud and detection based on image plane features. The detection algorithms are not limited to detection based on classical handcrafted features and detection based on learned features. Among them, the detection based on three-dimensional features of the point cloud refers to identifying and classifying target objects by extracting three-dimensional features in the point cloud data, such as shape, edge, texture, etc. Among them, the detection based on image plane features refers to identifying and classifying target objects by extracting plane features in the image, such as edges, corner points, etc. Among them, the detection based on classical handcrafted features refers to identifying and classifying target objects by extracting handcrafted features in the image, such as edges, corner points, texture, etc. Among them, the detection based on learned features refers to learning the feature representation in the image and combining a classifier to identify and classify target objects.

[0094] In one implementable manner, the point cloud can also be combined with a color image, that is, the point cloud is colored to obtain a colored point cloud. Then, the ground point cloud is extracted from the colored point cloud, and further, a first tilt angle is obtained through the ground point cloud. Specifically, the colored point cloud contains the color information of the point cloud. Therefore, the colored point cloud contains more semantic information than the point cloud, and thus the first tilt angle obtained using the colored point cloud is more accurate to a certain extent.

[0095] The processing unit 152 can obtain the first tilt angle through the ground point cloud. Specifically, a geometric method can be used to obtain the first tilt angle through the ground point cloud. The geometric method includes at least one of the following methods: line-plane fitting method, voxel sampling method. Among them, the line-plane fitting method means that according to the straight lines and planes in the ground point cloud, through geometric reasoning and calculation, the straight lines and planes in the real three-dimensional scene corresponding to the ground point cloud are restored, and then the first tilt angle is calculated from the fitted straight lines and planes. Among them, the voxel sampling method means that the ground point cloud data is divided into a series of small cubes, called voxels, and then a representative point is selected as a sampling point in each voxel. Then, using the coordinate information of the sampling point, the tilt angle of each sampling point relative to the self-mobile device is calculated. By synthesizing the tilt angles of each sampling point relative to the self-mobile device, the first tilt angle is obtained.

[0096] Using the above method, the first tilt angle can be obtained quickly and accurately according to the image.

[0097] In one implementable manner, when the second tilt angle is greater than or equal to the first threshold, the controller 150 determines that the ground in the traveling direction of the self-mobile device is a non-safe area and changes the traveling direction.

[0098] Among them, the first threshold is a tilt angle threshold set in advance according to the actual situation.

[0099] Specifically, when it is determined that the second tilt angle is greater than the first threshold, the control unit 153 in the controller 150 determines that the ground in the traveling direction of the self-mobile device is a non-safe area and changes the traveling direction of the self-mobile device to prevent the self-mobile device from tipping over due to the ground tilt. Among them, the non-safe area refers to the ground area where the self-mobile device is likely to tip over due to too high a tilt angle.

[0100] In one implementable manner, when the second tilt angle is less than the first threshold, the controller 150 determines that the ground in the traveling direction of the self-mobile device is a safe area and maintains the traveling direction.

[0101] Specifically, when it is determined that the second inclination angle is less than the first threshold, the control unit 153 in the controller 150 may determine that the ground in the traveling direction of the self - moving device is a safe area, and save the traveling direction and continue to travel. Herein, the safe area refers to a ground area where the inclination angle is not too high, and thus it is not easy for the self - moving device to tip over.

[0102] In one implementable manner, the traveling direction of the self - moving device is the front of the self - moving device.

[0103] Wherein, the traveling direction of the self - moving device can be determined according to path planning.

[0104] In one implementable manner, the first threshold is determined according to the friction of the ground.

[0105] The first threshold can be determined according to the friction of the ground, and the value of the first threshold can be proportional to the friction of the ground. The greater the friction of the ground, the greater the value of the first threshold can be. Determining the first threshold according to the friction of the ground can make the value of the first threshold more accurate.

[0106] In one implementable manner, the attitude sensor 140 is an accelerometer.

[0107] Figure 5 is a flowchart of a control method for a self - moving device provided by an embodiment of the present application. This embodiment is applicable to the scenario of route planning when the self - moving device travels on the ground with a certain inclination. The self - moving device can be, for example, the self - moving lawn mower 100, and this method can be executed by the self - moving device. As Figure 5 shown, the method includes:

[0108] Step 501, obtain the attitude of the self - moving device; and obtain the image in the traveling direction of the self - moving device.

[0109] Step 502, calculate the first inclination angle of the ground in the traveling direction of the self - moving device according to the image.

[0110] Step 503, combine the first inclination angle and the attitude to calculate the second inclination angle of the ground; and determine whether to travel along the traveling direction according to the second inclination angle.

[0111] This solution combines the attitude of the self - moving device, eliminates the influence of the attitude from the first inclination angle to obtain the second inclination angle. The second inclination angle is used as the slope detection result of the ground in the traveling direction of the self - moving device. And it can be determined whether to travel along the traveling direction according to the second inclination angle, control the traveling of the self - moving device, and prevent the self - moving device from tipping over due to too large a slope. The accuracy of slope detection of the ground in the traveling direction of the self - moving device is improved, and thus the tipping probability of the self - moving device is reduced.

[0112] The embodiment of the present application also specifically provides a slope detection algorithm based on point cloud for a lawn mower equipped with a depth camera. Figure 6 is a flowchart of a slope detection algorithm based on point cloud provided by an embodiment of the present application. Refer to Figure 6 and the specific process of this algorithm is as follows:

[0113] Step 601: The lawn mower is powered on, and the slope detection program in the lawn mower controller is started.

[0114] Step 602: Use the depth camera to collect the image in front of the lawn mower. The depth camera is not limited to an active laser depth camera, a passive binocular stereo camera, a passive binocular structured light camera, etc.

[0115] Step 603: Process the depth image collected by the lawn mower into point cloud through the algorithm in the slope detection program.

[0116] Step 604: Through algorithm processing, extract the ground point cloud from the point cloud in front of the lawn mower. The algorithm includes detection based on three-dimensional features of the point cloud and detection based on image plane features. The detection algorithm is not limited to detection based on classical handcrafted features and detection based on learned features.

[0117] Step 605: Through geometric methods, extract the relative inclination angle of the ground on the travel route through the ground point cloud information. The extraction method is not limited to the line-plane fitting method and the voxel sampling method, etc.

[0118] Step 606: Combine the attitude information provided by its own inertial navigation with the ground inclination information to comprehensively judge the absolute inclination angle of the ground relative to the gravity direction.

[0119] Step 607: Upload the inclination angle of the ground relative to the gravity direction to the planning and decision-making layer to judge whether it is a safe area where it can travel.

[0120] Step 608: Plan the route according to the judgment result of Step 607. Specifically, if it is determined to be an area where it can travel, control the lawn mower to continue moving forward in the travel direction; if it is determined to be an area where it cannot travel, change the travel direction and plan a safe travel route.

[0121] Figure 7 is a schematic diagram of a lawn image provided by an embodiment of the present application. The growth state of grass is affected by factors such as light, precipitation, season, temperature, etc. and is not constant. As Figure 7 shown in the lawn image, which contains the characteristic information of grass, such as the height, color, species, etc. of the grass, representing the growth state of the grass. It is a challenge for a self-propelled lawn mower to perform intelligent maintenance on the lawn for different growth states of grass.

[0122] In one embodiment, in combination with Figure 2and Figure 3 , the controller 150 is electrically connected to the vision sensor 130. The controller 150 is configured to obtain and, according to the inspection instruction, receive the lawn image of the working area acquired by the vision sensor 130; process the lawn image to obtain the feature information of the grass; wherein, the feature information at least includes the height and color of the grass; formulate a mowing plan or an inspection plan according to the feature information, and control the self-propelled mower 100 to work according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time for performing the mowing task; the inspection plan includes the time for generating the next inspection instruction. Refer to Figure 3 , the controller 150 may include an acquisition unit 151, a processing unit 152 and a control unit 153. Specifically, the lawn image of the working area where the self-propelled mower 100 is located can be collected by the vision sensor 130. Refer to Figure 3 , the acquisition unit 151 can obtain the inspection instruction and obtain the lawn image from the vision sensor 130 according to the inspection instruction; wherein, the inspection instruction can be generated in response to the user's operation or can be generated according to the inspection plan. The processing unit 152 can process the lawn image by using a preset method to obtain the feature information of the grass, wherein, the feature information at least includes the height and color of the grass. Specifically, the lawn image can be preprocessed first, including steps such as denoising, image enhancement, color correction, etc., to improve the image quality. Then, the deep learning semantic segmentation algorithm can be used to separate the grass plants in the preprocessed lawn image from the background. According to the segmentation result, parameter calculation can be performed on the segmented grass plants, at least including the height and color of the grass plants. Finally, statistical analysis is performed on the calculated grass plant parameters to obtain the overall features of the lawn. For example, statistical data such as the average grass plant height and the average grass plant color can be calculated. The average grass plant height can be used as the height of the grass, and the average grass plant color can be used as the color of the grass. The processing unit 152 can formulate a mowing plan or an inspection plan according to the feature information of the grass. For example, if it is determined according to the feature information of the grass that the lawn needs to be trimmed recently, a mowing plan is generated. If it is determined according to the feature information of the grass that the lawn does not need to be trimmed recently, an inspection plan can be generated to monitor the growth of the grass and perform mowing when the grass needs to be trimmed. The control unit 153 can control the self-propelled mower 100 to work according to the mowing plan or the inspection plan. Among them, the mowing plan can include the mowing task and the time for performing the mowing task. The inspection plan can include the time for the self-propelled mower 100 to generate the next inspection instruction.

[0123] Since the colors of different types of grass usually vary, and the colors of the same type of grass also usually vary to some extent in different seasons, and the suitable mowing heights for different types of grass usually vary to some extent, and the suitable mowing heights for the same type of grass also usually vary in different seasons, this solution identifies the growth state of the grass by at least relying on characteristic information such as the height and color of the grass. For example, the type of grass and the season it is in can be determined based on the color, and a mowing plan or inspection plan for the self-propelled mower 100 can be formulated according to the type, height, and season of the grass, making the maintenance of the lawn more intelligent, and thus improving the quality of lawn maintenance.

[0124] In one implementable way, a mowing height threshold is determined according to the color.

[0125] Specifically, a color-height mapping table can be preset according to experience, and the mapping relationship between the color and the height is included in the table. The processing unit 152 can query the color-height mapping table using the color of the grass to obtain the height corresponding to the color of the grass, and determine this height as the mowing height threshold.

[0126] If the height is greater than the mowing height threshold, control the self-propelled mower to perform the mowing task.

[0127] The processing unit 152 can compare the height of the grass with the mowing height threshold. If it is determined that the height of the grass is greater than the mowing height threshold, a mowing task to be executed is generated. The control unit 153 controls the self-propelled mower 100 to work according to the mowing task to be executed.

[0128] If the height is less than or equal to the mowing height threshold, a mowing plan or inspection plan is formulated according to the difference between the mowing height threshold and the height.

[0129] If the processing unit 152 determines that the height of the grass is less than or equal to the mowing height threshold, it calculates the difference between the mowing height threshold and the height of the grass, and formulates a mowing plan or inspection plan according to the difference. The control unit 153 controls the self-propelled mower 100 to work according to the mowing plan or inspection plan. For example, if the difference is small, it can be determined that the grass needs to be trimmed in the near future, and at this time, a mowing plan can be formulated. If the difference is large, it can be determined that the grass does not need to be trimmed in the near future, and at this time, an inspection plan can be formulated to monitor the growth of the grass and mow the grass when it needs to be trimmed.

[0130] Since the colors of different types of grass usually vary, and the colors of the same type of grass usually also have certain differences in different seasons, the suitable mowing heights for different types of grass usually have certain differences, and the suitable mowing heights for the same type of grass usually also have certain differences in different seasons. Therefore, in this solution, the growth state of the grass is identified by the color of the grass. For example, the type of grass and the season it is in can be determined based on the color, and then the mowing height threshold can be determined according to the type of grass and the season. Thus, the mowing plan or inspection plan can be formulated based on the mowing height threshold and the height of the grass, which can improve the maintenance quality of the lawn.

[0131] In an implementable manner, the feature information further includes size, shape, and / or density. The acquisition unit 151 acquires the season information, and the processing unit 152 determines the type of grass according to the season information, size, shape, and color.

[0132] In this application, a feature information mapping table can be preset according to experience. The mapping table includes the mapping relationships between season, size, shape, color, and the type of grass. The acquisition unit 151 can also acquire the season information. Specifically, the acquisition unit 151 can calculate the season information based on the time information. Or, the season information can be determined at least according to the color of the grass. The processing unit 152 can process the lawn image to obtain the size, shape, and density of the grass. Specifically, the grass plants in the lawn image can be separated from the background, and then parameter calculations can be performed on the segmented grass plants, including the size, shape, and density of the grass plants. Among them, the size includes at least height and width; the shape includes at least roundness and complexity; the density includes at least the number of grass plants per unit area. Finally, statistical analysis is performed on the calculated grass plant parameters to obtain the overall characteristics of the lawn. For example, statistical data such as the average grass plant height, average grass plant width, average grass plant roundness, average grass plant complexity, and average density can be calculated. Information such as the average grass plant height and average grass plant width can be used as the size of the grass, information such as the average grass plant roundness and average grass plant complexity can be used as the shape of the grass, and information such as the average density can be used as the density of the grass. Then, using the season information, the size, shape, and color of the grass, the feature information mapping table is queried to obtain the type of grass.

[0133] The processing unit 152 determines the mowing height threshold according to the type of grass, density, and season information.

[0134] Specifically, a feature mapping table can be preset according to experience. The mapping table includes the mapping relationships between type, density, season, and height. The processing unit 152 can query the feature mapping table according to the season information and the type and density of the grass to obtain the height corresponding to the season information and the type and density of the grass, and determine the height as the mowing height threshold.

[0135] Since different types of grass usually have differences in characteristics such as size, shape, and color in different seasons, the suitable mowing heights for different types of grass usually have certain differences. The suitable mowing heights for the same type of grass usually have certain differences under different seasonal conditions, and the suitable mowing heights for grass with different densities also usually have certain differences. Therefore, this solution can first accurately determine the type of grass based on the seasonal information and the size, shape, and color of the grass, and then more accurately determine the mowing height threshold according to the type of grass, density, and seasonal information. Controlling the operation of the automatic lawn mower 100 according to the mowing plan or inspection plan formulated based on this mowing height threshold can improve the quality of lawn maintenance.

[0136] In an implementable manner, if the processing unit 152 determines that the difference between the mowing height threshold and the height of the grass is less than or equal to a preset threshold, it formulates a mowing plan according to the difference; if the difference is greater than the preset threshold, it formulates an inspection plan according to the difference.

[0137] Among them, the preset threshold is a value set in advance according to the actual situation. If the difference between the mowing height threshold and the height of the grass is less than or equal to the preset threshold, it can indicate that the difference between the height of the grass and the mowing height threshold is not large. In this case, generally, the growth rate of the grass can be estimated more accurately, and then a more accurate mowing plan can be formulated according to the estimated growth rate of the grass.

[0138] If the difference between the mowing height threshold and the height of the grass is greater than the preset threshold, it can indicate that the difference between the height of the grass and the mowing height threshold is relatively large. In this case, it is generally difficult to accurately estimate the growth rate of the grass, and it is very difficult to accurately formulate a mowing plan. Therefore, at this time, an inspection plan can be formulated according to the difference to monitor the growth of the grass. For example, when the difference is greater than the preset threshold and if the difference is relatively large, a longer time interval can be set to inspect the lawn to determine whether mowing is required; when the difference is greater than the preset threshold and if the difference is relatively small, a relatively shorter time interval can be set to inspect the lawn to determine whether mowing is required. This can improve the quality of lawn maintenance and save resources.

[0139] This solution can first determine the difference between the mowing height threshold and the height of the grass, and then formulate a mowing plan or inspection plan based on the difference and the comparison result between the difference and the preset threshold, which can more precisely control the operation of the self-propelled lawn mower 100, and thus achieve a better lawn maintenance effect.

[0140] In an implementable manner, the height of the grass is determined according to the point cloud data of the grass; the point cloud data is determined according to the lawn image.

[0141] The processing unit 152 can first create a depth image from the lawn image and then process the depth image into a point cloud, which is the point cloud data of the grass. Then, the height of the grass can be identified using the point cloud data of the grass. In this way, the height of the grass can be obtained quickly and accurately. Specifically, each point on the depth image can be transformed into the world coordinate system through the relationship of similar triangles or coordinate system transformation to obtain the point cloud data of the grass. First, the vision sensor 130 needs to be aligned with the lawn surface and the three-dimensional point cloud data of the lawn is obtained. The vision sensor 130 can capture the three-dimensional coordinates of each point on the lawn surface, including the X, Y, and Z coordinates. The obtained point cloud data needs to be processed to remove noise, smooth the data, and perform data registration, etc. These processes can include operations such as filtering, smoothing, and resampling to improve the quality and accuracy of the data. Features can be extracted from the point cloud data, including the shape and height of the grass. Furthermore, a classifier can be designed based on the extracted features to distinguish the grass from other objects. After the classifier identifies the grass, in the processed point cloud data, a certain area of the point cloud data can be selected, and the average height of the grass in this area can be obtained by calculating the average height of these points. Specifically, some representative point cloud data can be selected and their average height can be calculated to obtain the grass height in this area. It should be noted that the accuracy and stability of the vision sensor 130 may be affected by factors such as environmental light and camera settings. Therefore, fine calibration and calibration need to be carried out in advance. Of course, the point cloud data can also be obtained by lidar. Therefore, in this solution, lidar can also be used as an alternative to the vision sensor 130.

[0142] Figure 8 FIG. 4 is a flowchart of a control method for a self-propelled lawn mower provided by an embodiment of the present application. This embodiment is applicable to the scenario where the self-propelled lawn mower 100 performs intelligent maintenance on the lawn, especially applicable to the situation of being out of the user's control for a long time. This method can be executed by the self-propelled lawn mower 100. As Figure 8 shown, this method includes:

[0143] Step 801, obtain an inspection instruction, and obtain a lawn image of the working area of the self-propelled lawn mower according to the inspection instruction.

[0144] Step 802, process the lawn image to obtain the feature information of the grass; wherein, the feature information at least includes the height and color of the grass.

[0145] Step 803, formulate a mowing plan or an inspection plan according to the feature information, and control the self-propelled lawn mower to work according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time to execute the mowing task; the inspection plan includes the time to generate the next inspection instruction.

[0146] Since the colors of different types of grass usually vary, and the colors of the same type of grass also usually vary to some extent in different seasons, the suitable mowing heights for different types of grass usually vary to some extent, and the suitable mowing heights for the same type of grass also usually vary to some extent in different seasons. Therefore, this solution can formulate a more accurate mowing plan or inspection plan at least based on the height and color of the grass, so as to be able to maintain the lawn more intelligently and achieve a better lawn maintenance effect.

[0147] The embodiment of the present application also specifically provides an intelligent mowing decision-making algorithm based on semantic recognition for the lawn mower. Figure 9 is a flowchart of an intelligent mowing decision-making algorithm provided by the embodiment of the present application. Refer to Figure 9 and the specific process of this algorithm is as follows:

[0148] Step 901, power on the lawn mower.

[0149] Step 902, start to execute the inspection mode in the lawn mower controller and determine whether to start the inspection.

[0150] Step 903, the lawn mower patrols on the lawn along a certain route, which can be either a user-specified route, a route autonomously planned by the lawn mower according to the lawn conditions, or a completely random route.

[0151] Step 904, obtain an image through a vision sensor and use a certain algorithm to extract the semantic information in the image, and determine whether the lawn mower is traveling in the lawn area through the semantic information. Among them, this algorithm can be a deep learning algorithm. If the semantic information indicates that there is grass in the image, it is determined that the lawn mower is traveling in the lawn area; if the semantic information indicates that there is no grass in the image, it is determined that the lawn mower is not traveling in the lawn area. It is also possible to first process the image into a color point cloud and identify the semantic information through the color point cloud. It is also possible to combine the semantic information recognition results of the image and the semantic information recognition results of the color point cloud to obtain the combined semantic information.

[0152] Step 905, if not in the lawn area, continue to move forward and continue to execute Step 903; if in the lawn area, collect the lawn-related semantic information and feature information extracted by the algorithm. Among them, the semantic information includes identification labels such as whether it is grass, the type of grass, density, and the season to which it belongs. The semantic information can include semantic labels at the pixel or sub-pixel level of the lawn within the field of view. The feature information of the grass includes the color, height, size, shape, etc. of the grass. Among them, the season can also be judged by time.

[0153] Step 906: Determine whether the collected feature information is sufficient according to a preset statistical algorithm. If it is not sufficient, continue to collect; if it is sufficient, stop collecting and further extract the information required for decision-making based on the set of currently collected feature information. Among them, the decision-making information is not limited to the mean and variance of the grass feature information in the distribution of the entire lawn area, as well as the mean and variance of the grass feature information in different subdivided areas, and the semantic information of the grass, etc. Among them, determining whether the collected feature information is sufficient according to the preset statistical algorithm includes: if it is determined that the area of the lawn being statistically analyzed is large enough, it is determined that the collected feature information is sufficient.

[0154] Step 907: Overall judge whether to execute the mowing task according to the decision-making information. Among them, the mowing task is not limited to full-area mowing, partial-area mowing, and multiple mowing tasks at different times. For example, the working area of the lawn mower can include various scenarios, such as uneven grass growth in the working area, uneven mowing in the previous mowing, or there are multiple types of grass in the working area and the mowing height requirements for each type of grass are different, etc. For different scenarios, the mowing task can be adaptively set.

[0155] Step 908: Execute the mowing task planned in Step 907. If the mowing task cannot be executed due to other reasons, such as manual intervention, weather conditions, battery power, etc., record the current condition of the lawn for comprehensive decision-making in the next plan.

[0156] Figure 10 It is a schematic diagram of the positioning scenario of a self-moving device traveling on a lawn provided by an embodiment of the present application. As Figure 10 shown, the lawn is generally outdoors. Due to the changes in outdoor weather and seasons, obvious changes in the surface conditions of the lawn ground will occur. And the different undulation degrees of the lawn ground will affect the acquisition of inertial data by the inertial measurement unit on the self-moving device, which will easily cause inaccurate positioning of the self-moving device. This is a challenge for the self-moving device.

[0157] In one embodiment, in combination with Figure 2 and Figure 3 , taking the attitude sensor 140 as an example, the inertial measurement unit (Inertial Measurement Unit, IMU) is disposed inside the fuselage 110. The inertial measurement unit includes at least one of an accelerometer and a gyroscope; the controller 150 is electrically connected to the inertial measurement unit. The controller 150 is configured to control the self-moving device to move; receive the inertial data of the inertial measurement unit; statistically analyze the inertial data received within the first time window, and adjust the positioning parameters of the inertial measurement unit according to the statistical results.

[0158] In combination with Figure 2 and Figure 3, the attitude sensor 140 and the controller 150 are respectively disposed on the body of the self - moving device, and the controller 150 is electrically connected to the attitude sensor 140.

[0159] Reference Figure 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, the self - moving device can be controlled to move by the controller 150, and the inertial data of the self - moving device can be collected by the inertial measurement unit. The acquisition unit 151 in the controller 150 can obtain the inertial data from the inertial measurement unit; the processing unit 152 can statistically process the inertial data received within the first time window in a preset manner to obtain a statistical result, where the duration of the first time window is preset according to the actual situation; and adjust the positioning parameters of the inertial measurement unit according to the statistical result; and then determine the position of the self - moving device according to the inertial data and the positioning parameters.

[0160] The self - moving device can be positioned using inertial data. The surface bumpiness of the driving area of the self - moving device will affect the accuracy of the inertial measurement unit in collecting inertial data, and thus affect the positioning accuracy of the self - moving device. The statistical result of the inertial data can reflect the surface bumpiness. Therefore, in this solution, the positioning parameters of the inertial data are adjusted according to the statistical result of the inertial data of the self - moving device to minimize the influence of surface bumpiness on positioning as much as possible, and then the self - moving device is positioned through the inertial data and the adjusted positioning parameters, which can improve the positioning accuracy.

[0161] In one implementable manner, the statistical result reflects the flatness of the road section passed by the self - moving device within the first time window.

[0162] The higher the flatness of the road section, the flatter the road section and the smaller the bumpiness.

[0163] In one implementable manner, the inertial data collected by the inertial measurement unit at least includes the acceleration and angular velocity of the self - moving device. Then, statistically processing the inertial data received within the first time window in a preset manner to obtain a statistical result includes: respectively integrating the acceleration and angular velocity in the inertial data received within the first time window to obtain the velocity and angle respectively; then respectively smoothing the acceleration, angular velocity, velocity and angle to obtain the acceleration processing result, angular velocity processing result, velocity processing result and angle processing result; determining the first variance between each acceleration and the acceleration processing result, determining the second variance between the angular velocity and the angular velocity processing result, determining the third variance between the velocity and the velocity processing result, determining the fourth variance between the angle and the angle processing result; and determining the first variance, second variance, third variance and fourth variance as the statistical result.

[0164] Among them, smoothing the acceleration to obtain the acceleration processing result includes: processing the acceleration by using a sliding window smoothing method to obtain the acceleration processing result. For example, averaging the accelerations collected within the first time window and using the average value as the acceleration processing result.

[0165] Similarly, smoothing the angular velocity to obtain the angular velocity processing result; smoothing the velocity to obtain the velocity processing result; smoothing the angle to obtain the angle processing result.

[0166] Among them, determining the first variance between each acceleration and the acceleration processing result includes: determining the square of the difference between each acceleration and the acceleration processing result, and taking the weighted sum of each square as the first variance. Among them, different weight values can be adopted for the accelerations in different sections, and the weight values corresponding to different sections can be set by methods such as manual parameter adjustment or machine learning.

[0167] Similarly, the second variance between the angular velocity and the angular velocity processing result can be determined; the third variance between the velocity and the velocity processing result can be determined; the fourth variance between the angle and the angle processing result can be determined.

[0168] Smoothing processing can obtain the approximate mean value in dynamic motion, so as to count the bumps in the motion process. The variance can better show the distribution difference of the data near the mean value. The smaller the distribution difference, the smaller the variance, and the larger the distribution difference, the larger the variance.

[0169] In one implementable manner, it is possible to first determine whether the positioning parameters of the inertial measurement unit need to be adjusted according to the statistical results. If it is determined that adjustment is needed, the positioning parameters are adjusted; if it is determined that adjustment is not needed, the positioning parameters are not adjusted.

[0170] This solution can determine whether the positioning parameters need to be adjusted based on the statistical results, and only adjust the positioning parameters when the conditions are met, thereby improving the positioning accuracy to a certain extent.

[0171] Specifically, a preset range can be set according to experience, and the statistical result is compared with the preset range. If it is determined that the statistical result is within the preset range, it is determined that the positioning parameters need to be adjusted. This method can quickly and accurately determine whether the positioning parameters need to be adjusted.

[0172] Alternatively, a binary classification model can be trained using the statistical results for training. The classification result of the binary classification model is used to represent whether the positioning parameters of the inertial measurement unit need to be adjusted. Then, the trained binary classification model can be used to process the statistical results to obtain the classification result. If the classification result indicates that the positioning parameters need to be adjusted, the positioning parameters are adjusted; if the classification result indicates that the positioning parameters do not need to be adjusted, the positioning parameters are not adjusted. This method can quickly and accurately determine whether the positioning parameters need to be adjusted.

[0173] Specifically, a mapping table can be preset according to experience. The mapping table includes the mapping relationship between the flatness and the adjustment parameters. Furthermore, the mapping table can be queried using the statistical results to obtain the adjustment parameters corresponding to the statistical results, and the positioning parameters of the inertial measurement unit are updated using the adjustment parameters. This method can quickly and accurately adjust the positioning parameters.

[0174] Alternatively, the statistical results are processed using a preset regression model to obtain a regression result. Then, the positioning parameters are updated using the regression result. This method can quickly and accurately adjust the positioning parameters.

[0175] In one implementable manner, the first time window is determined according to the walking speed of the self-mobile device.

[0176] Specifically, the faster the walking speed of the self-mobile device, the shorter the duration of the first time window.

[0177] In one implementable manner, the value range of the first time window is greater than or equal to the first duration threshold and less than the second duration threshold; wherein, the first duration threshold is less than the second duration threshold.

[0178] Among them, both the first duration threshold and the second duration threshold are preset according to experience. The first time window can take any value between the first duration threshold and the second duration threshold.

[0179] The statistical results obtained by statistically analyzing a sufficient amount of inertial data can more accurately represent the flatness of the road section. The method of setting the first time window in this solution is a relatively convenient and accurate way to determine whether the acquired inertial data is sufficient.

[0180] In one implementable manner, the positioning parameter is the noise of the inertial data.

[0181] Specifically, the surface roughness of the driving area of the self-mobile device will affect the accuracy of the inertial measurement unit in collecting inertial data. The positioning parameter can be used as the noise of the inertial data to characterize the accuracy of the inertial data. The statistical result of the inertial data can reflect the surface roughness. Therefore, in this solution, the positioning parameter of the inertial data is adjusted according to the statistical result of the inertial data to minimize the impact of surface roughness on positioning as much as possible, which can improve the positioning accuracy.

[0182] In an implementable manner, the positioning parameter is the weight of the inertial data when the inertial data is fused with other modality data for positioning.

[0183] In this application, a lidar and a camera can also be set on the self-mobile device. The inertial data and other modality data can be used for multi-modal fusion positioning of the self-mobile device. Specifically, different types of data, such as inertial data, point cloud data, and image data, can be obtained through sensors such as the inertial measurement unit, lidar, and camera. These data are processed by a multi-modal fusion algorithm to achieve more accurate positioning of the self-mobile device. Among them, the multi-modal fusion algorithm includes the weighted summation of multi-modal data, and the positioning parameter can be the weight of the inertial data when it is fused with other modality data for positioning. The rougher the surface of the driving area of the self-mobile device is, the less accurate the inertial data obtained by the inertial measurement unit will be. Therefore, when the surface is relatively rough, by reducing the weight of the inertial data in the fusion positioning, the accuracy of the fusion positioning can be improved to a certain extent. And the statistical result of the inertial data can reflect the surface roughness. Therefore, in this solution, the positioning parameter of the inertial data of the self-mobile device is adjusted according to the statistical result of the inertial data, and the positioning of the self-mobile device calculated using the adjusted positioning parameter is more accurate.

[0184] In one implementable manner, the inertial measurement unit can be shock-absorbed to reduce the impact on the inertial measurement unit caused by the unevenness of the driving section of the self-mobile device. Specifically, the shock-absorption measures for the inertial measurement unit can be implemented from two aspects: design and structure. In terms of design, in order to reduce the impact of vibration and shock on the inertial measurement unit, the adaptability of the inertial measurement unit can be improved through material selection and reasonable structural design. For example, high-precision next-generation inertial measurement units are selected, such as MPU6050, BMI055, etc. These chips have high adaptability and stability, and can reduce the impact of body vibration on the measurement accuracy of the inertial measurement unit. In terms of structure, a series of shock-absorption designs can be adopted, such as using shock-absorbing materials, such as silica gel, rubber, silicone rubber, etc., and shock-absorbing structures to isolate vibration. Specifically, by reasonably arranging the structure of the inertial measurement unit circuit board, the center of gravity of the chip and the center of gravity of the inertial measurement unit are at the same point, achieving the best shock-absorption effect. In addition, shock absorbers and air dampers and other shock-absorbing devices can also be considered. These devices can effectively reduce the impact of body vibration on the inertial measurement unit.

[0185] Figure 11 FIG. 4 is a flowchart of a positioning method for a self-mobile device provided by an embodiment of the present application. This embodiment is applicable to the positioning scenario during the driving process of the self-mobile device, and this embodiment is particularly applicable to the positioning scenario where the self-mobile device is driving on a relatively bumpy section. This method can be executed by the self-mobile device. As Figure 11 shown, the method includes:

[0186] Step 1101, obtain a positioning request, and obtain inertial data of the self-mobile device within a first time window according to the positioning request.

[0187] Step 1102, perform statistics on the inertial data, and adjust the positioning parameters corresponding to the inertial data according to the statistical results.

[0188] Step 1103, determine the position of the self-mobile device according to the inertial data and the positioning parameters.

[0189] The bumpy condition of the ground surface in the driving area of the self-mobile device will affect the accuracy of the inertial data collected by the inertial measurement unit, and further affect the positioning accuracy of the self-mobile device. The statistical results of the inertial data can reflect the bumpy condition of the ground surface. Therefore, in this solution, the positioning parameters of the inertial data are adjusted according to the statistical results of the inertial data of the self-mobile device to minimize the impact of the ground surface bump on the positioning as much as possible, and then the self-mobile device is positioned through the inertial data and the adjusted positioning parameters, which can improve the positioning accuracy.

[0190] The embodiment of the present application also specifically provides a positioning parameter adjustment algorithm based on inertial measurement unit data statistics for a lawn mower. Figure 12It is a flowchart of a positioning parameter adjustment algorithm based on inertial measurement unit data statistics provided by an embodiment of the present application. Refer to Figure 12 , the specific process of this algorithm is as follows:

[0191] Step 1201, collect inertial data through the inertial measurement unit in the lawn mower. The inertial data includes but is not limited to acceleration, angular velocity, etc.

[0192] Step 1202, determine whether the inertial data is collected sufficiently. The judgment basis includes but is not limited to status information such as the time of collecting data, the distance traveled by the lawn mower, and whether the lawn mower is mowing grass. Since the vibration generated by the lawn mower during mowing work will also affect the inertial measurement unit, and this influence is easily confused with the influence of road surface bumps on the inertial measurement unit, it is necessary to collect inertial data when the lawn mower is not mowing grass. If the inertial data is not sufficient, the adjustment of the positioning parameters based on the inertial data will not be accurate either. Therefore, it is necessary to collect sufficient inertial data. If the collection time length of the inertial data reaches the preset time length threshold, or the distance traveled by the lawn mower during the collection process of the inertial data reaches the preset distance threshold, it can be determined that the inertial data is collected sufficiently. When the inertial data is sufficient, proceed to step 1203; if it is not sufficient, continue to collect.

[0193] Step 1203, use an algorithm to process the collected inertial data to obtain statistical indicators of the inertial data. The algorithm includes but is not limited to machine learning methods or statistical learning methods, etc. Specifically, the inertial data includes at least acceleration and angular velocity. Integrate the inertial data to obtain velocity data and angle data. Use the sliding window smoothing method to smooth the inertial data, velocity data, and angle data, and respectively calculate the variances of the inertial data, velocity data, and angle data based on the smoothed data. Different weighting weights are adopted for data in different numerical ranges, and the weight values can be set by methods such as manual parameter adjustment or machine learning methods.

[0194] Step 1204, based on the series of statistical indicators obtained in step 1203, use a qualitative method to determine whether parameter adjustment is required. The qualitative method includes but is not limited to the empirical judgment method or the machine learning binary classification method.

[0195] Step 1205, for the scenarios that require parameter adjustment in step 1204, according to the obtained series of statistical indicators, use the look-up table interpolation method or other machine learning regression methods to adjust the positioning parameters of the inertial measurement unit.

[0196] Figure 13 It is a flowchart of a model training method applied to lawn image processing provided by an embodiment of the present application. This embodiment is applicable to the model training scenario of identifying grass or non-grass objects in lawn images, and this method can be executed by an electronic device. As Figure 13 shown, this method includes:

[0197] Step 1301: Obtain the dataset to be trained. Among them, the dataset to be trained includes multiple lawn images to be trained, and the lawn images to be trained have annotation types.

[0198] Specifically, visual sensors can be used to obtain the lawn images to be trained, and the lawn images to be trained can be processed such as cropping, sampling, and annotation. Among them, the lawn images to be trained can include information such as the ground, objects, and sky within the field of view of the visual sensor.

[0199] It is also possible to obtain publicly available lawn images to be trained with annotation types on the network. Among them, the annotation types can include object types within the field of view of the visual sensor, such as grass, pedestrians, trees, houses, sky, and ground, etc.

[0200] Step 1302: Add noise and / or interference to at least some of the lawn images to be trained in the dataset to be trained to obtain processed lawn images; and add the processed lawn images as new lawn images to be trained to the dataset to be trained to update the dataset to be trained.

[0201] Various methods can be used to add noise and / or interference to the lawn images to be trained to obtain processed lawn images, so as to enrich the training samples and simulate as many possible situations in the actual scenario as possible. Specifically, refer to Figure 7 and Figures 14A to 14C , the lawn images to be trained can be blurred, including Gaussian blur, mean blur, and median blur. Among them, Figure 7 can represent the lawn images to be trained, Figure 14A is the Gaussian blur image obtained by performing Gaussian blur processing on the lawn images to be trained, Figure 14B is the mean blur image obtained by performing mean blur processing on the lawn images to be trained, Figure 14C is the median blur image obtained by performing median blur processing on the lawn images to be trained. Among them, Gaussian blur means blurring the image by convolving the image with a normal distribution. Mean blur mainly operates on each pixel point in the image and replaces its value with the average value of adjacent pixel points. Median blur mainly operates on each pixel point in the image and replaces its value with the median value of adjacent pixel points.

[0202] Step 1303: Use a preset model to process the lawn images to be trained to obtain the predicted types of the lawn images to be trained; optimize the parameters of the preset model according to the annotation types and predicted types of the lawn images to be trained to obtain an identification model.

[0203] Among them, the preset model is a pre-set network model. This network model can be set based on a convolutional neural network. Specifically, the annotation type of the lawn image to be trained can be used as the label of the prediction type to train the preset model and optimize the parameters in the preset model, so that the prediction type is getting closer and closer to the annotation type. When the preset condition is reached, for example, the similarity between the prediction type and the annotation type reaches 90%, the training is stopped, and an identification model is obtained. The identification model can be used to process the lawn image to be identified and identify the target type of the lawn image to be identified.

[0204] In this solution, by adding noise and / or interference to the lawn image to be trained, the training samples are enriched, and as many possible situations in the actual scenario as possible can be simulated. Using the richer training samples to train the identification model can, to a certain extent, improve the identification accuracy of the identification model.

[0205] In one implementable way, at least some of the lawn images to be trained in the training dataset are subjected to one or more of the following combined processes to obtain the processed lawn images: blur processing, distortion processing, overexposure processing, adding pixel blocks, adding shadows of target objects, and adding preset objects.

[0206] Specifically, image processing software or a blur filter in a programming language can be used to implement the blur processing of the lawn image to be identified. The blur processing methods include at least one of the following: motion blur, depth-of-field blur, rotation blur, zoom blur, Gaussian blur, mean blur, and median blur. Among them, motion blur is the blur caused by the movement of objects or the camera in the image. To simulate this effect, methods such as random pixel offset or rotation can be used to produce the blur effect. Depth-of-field blur is the blur effect caused by the change of the focal length of the camera lens. This effect can be simulated by blurring the background and foreground pixels in the image to different degrees. Rotation blur can be simulated by rotating the image and applying a filter. Zoom blur is the blur effect caused by the image being enlarged or reduced. This effect can be simulated by applying a filter after scaling the image.

[0207] Specifically, an image processing software or a transformation function in a programming language can be used to implement the distortion processing of the lawn image to be recognized. The distortion processing methods include at least one of the following: distortion transformation, shear transformation, rotation transformation, skew transformation, and affine transformation. Among them, the distortion transformation is a transformation method that maps the pixels in the image and can distort or deform a part of the image. This transformation can usually be achieved by defining a transformation matrix. The shear transformation is a transformation method that displaces the pixels in the image along the vertical or horizontal direction. This transformation can be achieved by translating the pixels in the horizontal or vertical direction. The rotation transformation is a transformation method that rotates the image and can be achieved by defining the rotation center and rotation angle. The skew transformation is a transformation method that displaces the pixels in the image along a straight line. This transformation can be achieved by defining two displacement points. The affine transformation is a transformation method that maps the pixels in the image and can distort or deform a part of the image. This transformation can be achieved by defining an affine matrix.

[0208] Specifically, taking the visual sensor 130 as a camera as an example, for the camera, overexposure processing of the image can be achieved by increasing the exposure time or increasing the ISO sensitivity. The overexposure effect can also be simulated by adjusting parameters such as the brightness, contrast, and color of the lawn image to be recognized. The overexposure processing methods include at least one of the following: increasing brightness, increasing contrast, color adjustment, highlight adjustment, and sharpening processing. Among them, increasing brightness means simulating the overexposure effect by increasing the overall brightness of the image and can be achieved by using a brightness adjustment function in an image processing software or a programming language. Increasing contrast means simulating the overexposure effect by increasing the contrast of the image and can be achieved by using a contrast adjustment function in an image processing software or a programming language. Color adjustment means simulating the overexposure effect by adjusting the color of the image and can be achieved by using a color adjustment function in an image processing software or a programming language. Highlight adjustment means simulating the overexposure effect by increasing the highlight part of the image and can be achieved by using a highlight adjustment function in an image processing software or a programming language. Sharpening processing means simulating the overexposure effect by sharpening the edges and details of the image and can be achieved by using a sharpening function in an image processing software or a programming language.

[0209] Specifically, solid color pixel blocks can be added to the lawn image to be trained.

[0210] Specifically, shadows of objects that may appear on the lawn or objects that may appear on the lawn can be added to the lawn image to be trained to enrich the training samples, and then a recognition model with a higher recognition accuracy can be trained based on the more abundant training samples.

[0211] In an implementable manner, the shadow of any one of the following target objects is added to the lawn image to be trained: trees, fences, and houses.

[0212] In one implementable manner, any one of the following preset objects is added to the lawn image to be trained: water pipe, faucet, water gun, toy, animal feces, fallen leaves, and branches.

[0213] In one implementable manner, the lawn image to be trained includes a color image and / or a grayscale image.

[0214] In one embodiment, in combination with Figure 2 and Figure 3 , the controller 150 is electrically connected to the vision sensor 130. The controller 150 is configured to obtain the lawn image to be recognized from the vision sensor 130, and process the lawn image to be recognized by using a preset recognition model to obtain the target type of the lawn image to be recognized. Among them, the preset recognition model is obtained by training a preset model by using the lawn image to be trained and the processed lawn image; the lawn image to be trained has an annotated type; the processed lawn image is obtained by adding noise and / or interference to at least part of the lawn images to be trained.

[0215] Referring to Figure 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, the lawn image to be recognized can be collected by the vision sensor 130, and the acquisition unit 151 in the controller 150 obtains the lawn image to be recognized from the vision sensor 130; the processing unit 152 processes the lawn image to be recognized by using a preset recognition model to obtain the target type of the lawn image to be recognized.

[0216] In this solution, by adding noise and / or interference to the lawn image to be trained, the training samples are enriched, and the recognition accuracy of the recognition model trained with the richer training samples is higher. Using a recognition model with higher recognition accuracy can improve the maintenance effect of the self-propelled lawn mower on the lawn to a certain extent.

[0217] Figure 15 is a flowchart of a lawn image processing method provided by an embodiment of the present application. This embodiment is applicable to the scenario where the self-propelled lawn mower 100 maintains the lawn and recognizes grass and non-grass objects in the lawn image. This method can be executed by the self-propelled lawn mower 100. As Figure 15 shown, this method includes:

[0218] Step 1501, obtain the lawn image to be recognized.

[0219] Step 1502: Process the lawn image to be recognized using a preset recognition model to obtain the target type of the lawn image to be recognized. Among them, the preset recognition model is obtained by training a preset model using the lawn images to be trained and the processed lawn images. The lawn images to be trained have labeled types. The processed lawn images are obtained by adding noise and / or interference to at least some of the lawn images to be trained.

[0220] In this solution, by adding noise and / or interference to the lawn images to be trained, the training samples are enriched, and the recognition accuracy of the recognition model trained with more abundant training samples is higher. Using a recognition model with higher recognition accuracy to identify grass and non-grass objects in the lawn image can improve the maintenance effect of the self-propelled lawn mower to a certain extent.

[0221] The embodiment of the present application also specifically provides a model training method based on data augmentation for the lawn mower. Figure 16 is a flowchart of a model training method based on data augmentation provided by the embodiment of the present application. Refer to Figure 16 and the specific process of this method is as follows:

[0222] Step 1601: Use the camera installed on the lawn mower to collect visual information data, such as color images, grayscale images, etc.

[0223] Step 1602: Screen and label the data. The labeling methods include but are not limited to manual labeling methods and automatic labeling methods.

[0224] Step 1603: Augment the labeled data through data augmentation. The main augmentation methods include noise augmentation and interference augmentation. The noise augmentation methods include but are not limited to blurring, distortion, overexposure, etc. The interference augmentation methods include but are not limited to adding light and shadow, solid color pixel blocks, objects that may appear on the lawn, etc.

[0225] Step 1604: Use the augmented dataset after data augmentation to train a machine learning model.

[0226] Step 1605: After training, an algorithm model that can distinguish grass and non-grass areas is obtained.

[0227] Figure 2 is a schematic structural diagram of a self-propelled lawn mower 100 provided by the embodiment of the present application. Figure 17 is a schematic diagram of a reference object 200 provided by the embodiment of the present application. Refer to Figure 2 and Figure 17, an embodiment of the present application provides a self - moving lawn mower system, including: a self - moving lawn mower 100 and a reference object 200; the self - moving lawn mower 100 includes: a body 110; a walking wheel assembly 120 configured to support the body; a vision sensor 130 installed on the body and configured to acquire images around the self - moving lawn mower; combined with Figure 2 and Figure 3 , a controller 150 is electrically connected to the vision sensor 130 and identifies semantic information around the self - moving lawn mower 100 through image recognition; the reference object 200 is used to provide a reference for the controller, and the reference object 200 simulates at least one of a lawn, land, water surface, and sky.

[0228] Combined with Figure 2 and Figure 17 , the reference object can be installed in an area that can be captured by the vision sensor 130, such as on a wall, a tree trunk, a charging pile, etc.

[0229] Referring to Figure 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, images around the self - moving lawn mower 100 can be collected by the vision sensor 130, and the acquisition unit 151 in the controller 150 obtains the images from the vision sensor 130; the processing unit 152 obtains the reference information of the reference object 200 and identifies the semantic information around the self - moving lawn mower 100 through the reference information and the images. Specifically, images of the reference object 200 can be collected by the vision sensor 130, the acquisition unit 151 obtains the images of the reference object 200 from the vision sensor 130, and extracts the reference information of the reference object 200 from the images; the acquisition unit 151 can also obtain the real information of the reference object 200; the processing unit 152 performs transformation processing on the images around the self - moving lawn mower 100 according to the real information and the reference information of the reference object 200 to obtain transformed images; the transformed images are processed by a semantic recognition program to obtain the semantic information around the self - moving lawn mower 100. Among them, the semantic recognition program can be a machine - learning program, such as an intelligent recognition model based on machine learning; the semantic information at least includes grass, the type of grass, and non - grass objects, etc.

[0230] In the process of identifying the semantic information around the self - moving lawn mower 100 through image recognition in this solution, the reference information of the reference object is combined to reduce the deviation between the visual information and the real information on the image, and thus the accuracy of semantic information recognition can be improved.

[0231] In an implementable manner, referring to Figure 17 , the self - moving lawn mower system further includes a charging pile 300, and the reference object 200 is arranged on the charging pile 300.

[0232] Setting the reference object 200 at the charging pile 300 reduces the installation steps when the reference object 200 and the charging pile 300 are installed separately.

[0233] In one implementable manner, the reference object 200 has geometric features, and the geometric features are used to assist the controller 150 in identifying the reference object 200.

[0234] The processing unit 152 extracts geometric features from the images around the mobile mower 100. If the geometric features are target geometric features, the target images corresponding to the geometric features are determined as the reference object 200, and the reference information of the reference object 200 is extracted. The reference object 200 can be identified more conveniently and quickly through the geometric features. Among them, the target geometric features can be planar geometric shape features or three-dimensional geometric body features. The target geometric features include but are not limited to object features such as cubes and spheres with a certain special visual recognition structure. Specifically, at least one of the following methods can be used to extract the geometric features in the image: edge extraction and feature point extraction. Since the edges are the regions with the most obvious local changes in the image, the geometric features of the image can be extracted by detecting these regions. Edge extraction usually uses the extreme value regions of the first-order derivative to detect edges, that is, by calculating the derivative of the image function in a certain direction and finding the regions where the derivative has extreme value changes, and these regions are the edges of the image. Feature point extraction refers to finding the feature points in the image, that is, those points with obvious changes or special shapes in the image. For example, corner points are the intersections of two sides in the image, and they usually have a larger curvature value than the surrounding regions, so the geometric features can be extracted by detecting these points. Similarly, key points and blobs are also regions with significant features in the image, and the geometric features can be extracted through these regions. Once the geometric features in the image are detected, descriptors can be used to describe these features. The descriptors can be gray-level co-occurrence matrices, histograms of oriented gradients, scale-invariant feature transforms, speeded-up robust features, etc. These descriptors can be used to compare and match the feature points in the image for tasks such as image recognition, classification, and retrieval.

[0235] In one implementable manner, the controller 150 is configured to identify the reference object 200 through geometric features, extract the color features of the reference object 200, and perform transformation processing on the image according to the color features to obtain a transformed image.

[0236] The reference information provided by the reference object 200 to the controller 150 may be color features. The color features of the reference object 200 include but are not limited to colors similar to natural references such as lawn color, sky color, etc. After the processing unit 152 in the controller 150 identifies the reference object 200 through geometric features, it can extract the color features of the reference object 200; the acquisition unit 151 in the controller 150 can also acquire the true color information of the reference object 200; the processing unit 152 can obtain the difference between the extracted color features of the reference object 200 and the true color information; then subtract this difference from the color information of the image to obtain a transformed image.

[0237] Color features are important features of an image. Changes in the outdoor working environment, such as changes in light, cause an offset between the visual information and the true information on the image, mainly affecting the color features. Therefore, taking the difference between the extracted color features of the reference object 200 and the true color features as the transformation reference, and performing a transformation on the color features of the image to reduce the deviation between the visual information and the true information on the image. Using the transformed image for intelligent recognition can improve the recognition accuracy.

[0238] Figure 18 is a flowchart of a control method for a self-propelled lawn mower provided by an embodiment of the present application. This embodiment is applicable to the scenario where the self-propelled lawn mower 100 performs intelligent recognition on the images around the self-propelled lawn mower 100 when maintaining the lawn. This method can be executed by the self-propelled lawn mower 100, and the automatic lawn mower 100 includes a visual sensor 130, such as Figure 18 shown, the method includes:

[0239] Step 1801, obtain an image of the reference object 200 through the visual sensor 130, and extract reference information.

[0240] Step 1802, obtain an image around the self-propelled lawn mower 100 through the visual sensor 130.

[0241] Step 1803, perform transformation processing on the image around the self-propelled lawn mower 100 according to the reference information to obtain a transformed image.

[0242] Step 1804, process the transformed image through a semantic recognition program to obtain semantic information around the self-propelled lawn mower 100.

[0243] This solution uses the reference information of the reference object to perform transformation processing on the image around the self-propelled lawn mower, obtaining a transformed image, minimizing the deviation between the visual information and the true information on the image as much as possible. Furthermore, using the transformed image for intelligent recognition can improve the recognition accuracy.

[0244] In a realizable manner, the reference object is a known object in the natural environment.

[0245] The known object can be recognized by a semantic recognition program. The known object includes at least lawn, land, water surface, sky, etc.

[0246] In an implementable manner, the reference object is an artificially set simulated object.

[0247] The embodiment of the present application also provides a method for determining a feature transformation reference based on lawn markers for a lawn mower equipped with a camera. Figure 19 It is a flowchart of a method for establishing a feature transformation reference based on lawn markers provided by the embodiment of the present application. Refer to Figure 19 , and the specific process of this method is as follows:

[0248] Step 1901, the lawn mower is powered on and starts mowing the lawn.

[0249] Step 1902, during the mowing process, collect visual information on the lawn through the camera.

[0250] Step 1903, search for it in the image according to the preset geometric information of the marker.

[0251] Step 1904, extract relevant visual information from the marker in the searched image.

[0252] Step 1905, obtain a transformation method for visual information according to the preset visual information of the marker and the extracted visual information.

[0253] Step 1906, correct the entire image with the obtained transformation method for visual information.

[0254] Step 1907, use the corrected image as the input of the visual processing algorithm for visual algorithm processing.

[0255] Figure 20The structural schematic diagram of an electronic device 10 that can be used to implement the embodiments of the present application is shown. The electronic device 10 can be a controller in a self-propelled lawn mower 100, which can be installed on the body of the self-propelled lawn mower 100 and communicate locally with a vision sensor 130 and an attitude sensor 140 in the self-propelled lawn mower 100; the electronic device 10 can also be a remote server that interacts remotely with the vision sensor 130 and the attitude sensor 140 in the self-propelled lawn mower 100. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.). The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0256] As Figure 20 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0257] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0258] The processor 11 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above.

[0259] In some embodiments, any of the above methods may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of any of the methods described above may be executed. Alternatively, in other embodiments, the processor 11 may be configured to execute any of the above methods by any other suitable means (e.g., by means of firmware).

[0260] It should be understood that various forms of the flow shown above may be used, with steps reordered, added, or deleted. For example, the steps recited in this application may be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved, and no limitation is made herein.

[0261] The basic principles, main features, and advantages of this application have been shown and described above. Those skilled in the art of this industry should understand that the above embodiments do not limit this application in any form. Any technical solutions obtained by means of equivalent replacement or equivalent transformation fall within the protection scope of this application.

Claims

1. A model training method applied to lawn image processing, characterized in that, it includes: Obtain a dataset to be trained; wherein, the dataset to be trained includes multiple lawn images to be trained; the lawn images to be trained have annotation types; Add noise and / or interference to at least some of the lawn images to be trained in the dataset to be trained to obtain processed lawn images; and add the processed lawn images as new lawn images to be trained to the dataset to be trained to update the dataset to be trained; Use a preset model to process the lawn images to be trained to obtain the predicted types of the lawn images to be trained; according to the annotation types and predicted types of the lawn images to be trained, optimize the parameters of the preset model to obtain an identification model.

2. The method according to claim 1, characterized in that, The step of adding noise and / or interference to at least some of the lawn images to be trained in the dataset to be trained to obtain processed lawn images includes: Perform one or more of the following combined processes on at least some of the lawn images to be trained in the dataset to be trained to obtain processed lawn images: Blur processing, distortion processing, overexposure processing, adding pixel blocks, adding shadows of target objects, and adding preset objects.

3. The method according to claim 2, characterized in that, The preset objects include one or more of the following combinations: Water pipes, faucets, water guns, toys, animal feces, fallen leaves, and tree branches.

4. The method according to claim 2, characterized in that, The target objects include one or more of the following combinations: Trees, fences, and houses.

5. The method according to claim 1, characterized in that, The lawn images to be trained include color images and / or grayscale images.

6. A lawn image processing method, characterized in that, Applied to a self-propelled lawn mower, the method includes: Obtain a lawn image to be recognized; Use a preset recognition model to process the lawn image to be recognized to obtain the target type of the lawn image to be recognized; Wherein, the preset recognition model is obtained by training a preset model using lawn images to be trained and processed lawn images; the lawn images to be trained have annotation types; the processed lawn images are obtained by adding noise and / or interference to at least some of the lawn images to be trained.

7. The method according to claim 6, characterized in that, The processed lawn images are obtained by adding noise and / or interference to at least some of the lawn images to be trained, including: Perform one or more of the following combined processes on at least some of the lawn images to be trained to obtain processed lawn images: Blur processing, distortion processing, overexposure processing, adding pixel blocks, adding shadows of target objects, and adding preset objects.

8. The method according to claim 7, characterized in that, The preset objects include one or more of the following combinations: Water pipes, faucets, water guns, toys, animal feces, fallen leaves, and tree branches.

9. The method according to claim 7, characterized in that, The target objects include one or more of the following combinations: Trees, fences, and houses.

10. The method according to claim 6, wherein, the lawn image to be trained includes a color image and / or a grayscale image.

11. A model training device applied to lawn image processing, wherein, it includes: an acquisition unit, configured to acquire a dataset to be trained; wherein, the dataset to be trained includes multiple lawn images to be trained; the lawn images to be trained have labeled types; a processing unit, configured to add noise and / or interference to at least some of the lawn images to be trained in the dataset to be trained, to obtain a processed lawn image; and add the processed lawn image as a new lawn image to be trained to the dataset to be trained, so as to update the dataset to be trained; the processing unit is further configured to process the lawn image to be trained by using a preset model, to obtain a predicted type of the lawn image to be trained; and optimize parameters of the preset model according to the labeled type and the predicted type of the lawn image to be trained, to obtain a recognition model.

12. A lawn image processing device, wherein, applied to a self-propelled lawn mower, the device includes: an acquisition unit, configured to acquire a lawn image to be recognized; a processing unit, configured to process the lawn image to be recognized by using a preset recognition model, to obtain a target type of the lawn image to be recognized; wherein, the preset recognition model is obtained by training the preset model by using the lawn image to be trained and the processed lawn image; the lawn image to be trained has a labeled type; the processed lawn image is obtained by adding noise and / or interference to at least some of the lawn images to be trained.

13. An electronic device, wherein, the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the method according to any one of claims 1-10.

14. A computer-readable storage medium, wherein, the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1-10 is implemented.

Images