Self-moving device and positioning method

Abstract

The invention discloses a self-moving device and a positioning method, and the device comprises a machine body and a walking wheel assembly which is configured to support the machine body; the inertial measurement unit is arranged in the fuselage, and the inertial measurement unit comprises at least one of an accelerometer and a gyroscope; the controller is electrically connected with the inertial measurement unit, and the controller is configured to control the self-moving equipment to walk; receiving inertial data of an inertial measurement unit; and counting the inertial data received in the first time window, and adjusting the positioning parameter of the inertial measurement unit according to the counting result. The positioning parameter of the inertial data is adjusted according to the statistical result of the inertial data of the self-moving device, and the positioning of the self-moving device calculated by using the adjusted positioning parameter is more accurate.

Description

Technical Field

[0001] The present application relates to a device and a positioning method, and more particularly to a self - moving device and a positioning method. Background Art

[0002] Self - moving devices, such as self - moving lawn mowers, generally perform mowing work outdoors. Due to the changes in outdoor weather and seasons, obvious changes in the surface conditions of the lawn ground will occur.

[0003] In related technologies, the inertial measurement unit in the self - moving lawn mower can be used to collect inertial data, and then the inertial data and other modal data can be fused to position the self - moving lawn mower.

[0004] However, different undulation degrees of the lawn ground will affect the collected inertial data, and thus it is easy to cause inaccurate positioning of the self - moving lawn mower.

[0005] This part provides background information related to the present application, and these background information are not necessarily prior art. Summary of the Invention

[0006] An object of the present application is to solve or at least mitigate part or all of the above problems. For this purpose, an object of the present application is to provide a self - moving device with higher positioning accuracy.

[0007] To achieve the above object, the present application adopts the following technical solutions:

[0008] A self - moving device, comprising:

[0009] A fuselage,

[0010] A walking wheel assembly configured to support the fuselage;

[0011] An inertial measurement unit disposed in the fuselage, the inertial measurement unit including at least one of an accelerometer and a gyroscope;

[0012] A controller electrically connected to the inertial measurement unit, the controller being configured to,

[0013] Control the walking of the self - moving device;

[0014] Receive the inertial data of the inertial measurement unit;

[0015] Statistically analyze the inertial data received within a first time window, and adjust the positioning parameters of the inertial measurement unit according to the statistical results.

[0016] In some embodiments, the statistical results reflect the flatness of the section passed by the self - moving device within the first time window.

[0017] In some embodiments, the positioning parameter is the noise of the inertial data.

[0018] In some embodiments, the positioning parameter is the weight of the inertial data when the inertial data is fused with data of other modalities for positioning.

[0019] In some embodiments, the value range of the first time window is greater than or equal to a first duration threshold and less than or equal to a second duration threshold;

[0020] wherein, the first duration threshold is less than the second duration threshold.

[0021] In some embodiments, the first time window is determined according to the walking speed of the self - moving device.

[0022] A positioning method for a self - moving device, comprising:

[0023] Obtaining a positioning request, and obtaining inertial data of the self - moving device within a first time window according to the positioning request;

[0024] Performing statistics on the inertial data, and adjusting the positioning parameter corresponding to the inertial data according to the statistical result;

[0025] Determining the position of the self - moving device according to the inertial data and the positioning parameter.

[0026] A positioning device for a self - moving device, comprising:

[0027] An obtaining unit, configured to obtain a positioning request, and obtain inertial data of the self - moving device within a first time window according to the positioning request;

[0028] A processing unit, configured to perform statistics on the inertial data, and adjust the positioning parameter corresponding to the inertial data according to the statistical result;

[0029] The processing unit is further configured to determine the position of the self - moving device according to the inertial data and the positioning parameter.

[0030] An electronic device, the electronic device comprising:

[0031] At least one processor; and

[0032] A memory communicatively connected to the at least one processor; wherein,

[0033] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the positioning method for a self - moving device according to any embodiment of the present application.

[0034] A computer-readable storage medium stores computer instructions for implementing the positioning method of the self-mobile device according to any embodiment of the present application when executed by a processor.

[0035] The advantages of the present application are as follows: The self-mobile device can be positioned using inertial data. The bumpy condition of the ground surface in the driving area of the self-mobile device will affect the accuracy of the inertial measurement unit in collecting inertial data, and thus affect the positioning accuracy of the self-mobile device. The statistical result of the inertial data can reflect the bumpy condition of the ground surface. Therefore, in this solution, the positioning parameters of the inertial data are adjusted according to the statistical result of the inertial data of the self-mobile device to minimize the influence of the ground surface bump on positioning as much as possible, and then the self-mobile device is positioned through the inertial data and the adjusted positioning parameters, which can improve the positioning accuracy. Description of the Drawings

[0036] Figure 1 It is a schematic diagram of the working scenario of a self-mobile mower provided by an embodiment of the present application;

[0037] Figure 2 It is a schematic diagram of the structure of a self-mobile mower provided by an embodiment of the present application;

[0038] Figure 3 It is a schematic diagram of the structure of a self-mobile mower provided by an embodiment of the present application;

[0039] Figure 4 It is a schematic diagram of the slope detection scenario of a self-mobile device provided by an embodiment of the present application;

[0040] Figure 5 It is a flowchart of a control method for a self-mobile device provided by an embodiment of the present application;

[0041] Figure 6 It is a flowchart of a slope detection algorithm based on point cloud provided by an embodiment of the present application;

[0042] Figure 7 It is a schematic diagram of a lawn image provided by an embodiment of the present application;

[0043] Figure 8 It is a flowchart of a control method for a self-mobile mower provided by an embodiment of the present application;

[0044] Figure 9 It is a flowchart of an intelligent mowing decision algorithm provided by an embodiment of the present application;

[0045] Figure 10 It is a schematic diagram of the positioning scenario of a self-mobile device driving on a lawn provided by an embodiment of the present application;

[0046] Figure 11 is a flowchart of a positioning method for a self - moving device provided by an embodiment of the present application;

[0047] Figure 12 is a flowchart of a positioning parameter adjustment algorithm based on inertial measurement unit data statistics provided by an embodiment of the present application;

[0048] Figure 13 is a flowchart of a model training method applied to lawn image processing provided by an embodiment of the present application;

[0049] Figures 14A to 14C are schematic diagrams of a Gaussian blur map, a mean blur map, and a median blur map for performing blurring processing on a lawn image to be trained, respectively, provided by an embodiment of the present application;

[0050] Figure 15 is a flowchart of a lawn image processing method provided by an embodiment of the present application;

[0051] Figure 16 is a flowchart of a model training method based on data augmentation provided by an embodiment of the present application;

[0052] Figure 17 is a schematic diagram of a reference object provided by an embodiment of the present application;

[0053] Figure 18 is a flowchart of a control method for a self - moving lawn mower provided by an embodiment of the present application;

[0054] Figure 19 is a flowchart of a method for establishing a feature transformation reference based on lawn markers provided by an embodiment of the present application;

[0055] Figure 20 is a schematic structural diagram of an electronic device for implementing the method of an embodiment of the present application. Detailed implementation manners

[0056] Before explaining any embodiment of the present application in detail, it should be understood that the present application is not limited to the structural details and component arrangements set forth in the following description or shown in the above drawings.

[0057] In the present application, the terms "comprising", "including", "having" or any other variant thereof are intended to cover a non - exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0058] In this application, the term "and / or" describes the relationship between associated objects and represents three possible relationships. For example, A and / or B can represent three cases: A exists alone, both A and B exist simultaneously, and B exists alone. Additionally, in this application, the character " / " generally indicates that the associated objects before and after are in an "and / or" relationship.

[0059] In this application, the terms "connect", "combine", "couple", and "mount" can be direct connections, combinations, couplings, or mountings, or they can be indirect connections, combinations, couplings, or mountings. Among them, for example, a direct connection means that two parts or components are connected together without the need for an intermediate member, and an indirect connection means that two parts or components are respectively connected to at least one intermediate member, and these two parts or components are connected through the intermediate member. In addition, "connect" and "couple" are not limited to physical or mechanical connections or couplings and can include electrical connections or couplings.

[0060] In this application, those of ordinary skill in the art will understand that relative terms used in combination with quantities or conditions (such as "about", "approximately", "substantially", etc.) include the value and have the meaning indicated by the context. For example, such relative terms at least include the degree of error associated with the measurement of a specific value, tolerances caused by manufacturing, assembly, use, etc. related to a specific value. Such terms should also be considered to disclose a range defined by the absolute values of two endpoints. The relative term can refer to the addition or subtraction of a certain percentage (such as 1%, 5%, 10% or more) of the indicated value. Numerical values without the use of relative terms should also be disclosed as specific values with tolerances. In addition, when expressing a relative angular positional relationship (such as substantially parallel, substantially perpendicular), "substantially" can refer to the addition or subtraction of a certain number of degrees (such as 1 degree, 5 degrees, 10 degrees or more) based on the indicated angle.

[0061] In this application, those of ordinary skill in the art will understand that the functions performed by a component can be performed by one component, multiple components, one part, or multiple parts. Similarly, the functions performed by a part can also be performed by one part, one component, or a combination of multiple parts.

[0062] In this application, the directional terms such as "upper", "lower", "left", "right", "front", and "rear" are described based on the orientation and positional relationship shown in the drawings, and should not be construed as limiting the embodiments of this application. In addition, in the context, it should also be understood that when it is mentioned that one component is connected "above" or "below" another component, it can not only be directly connected "above" or "below" another component, but also be indirectly connected "above" or "below" another component through an intermediate component. It should also be understood that the directional terms such as the upper side, the lower side, the left side, the right side, the front side, and the rear side not only represent the positive direction, but can also be understood as the side direction. For example, the lower side can include directly below, lower left, lower right, lower front, and lower rear, etc.

[0063] It should be noted that the terms "first", "second", etc. in the description, claims, and the above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0064] In this application, the terms "controller", "processor", "central processing unit", "CPU", and "MCU" can be interchanged. When using the units "controller", "processor", "central processing unit", "CPU", or "MCU" to perform specific functions, unless otherwise specified, these functions can be performed by a single one of the above units or multiple ones of the above units.

[0065] In this application, the terms "device", "module", or "unit" can be implemented in the form of hardware or software in order to achieve specific functions.

[0066] In this application, the terms "calculate", "judge", "control", "determine", "identify", etc. refer to the operations and processes of a computer system or a similar electronic computing device (such as a controller, a processor, etc.).

[0067] The self-moving device provided by this application can be, for example, a self-moving lawn mower.

[0068] Figure 1 It is a schematic diagram of the working scenario of a self-moving lawn mower provided by an embodiment of this application. As Figure 1As shown, the self - propelled lawn mower can move and work within the working area. The working area is generally outdoors and includes a lawn. The outdoor working environment is complex and variable, which is challenging for the self - propelled lawn mower working in it. For example, the lawn terrain is diverse and may have slopes. If the slope is too high, it may cause the self - propelled lawn mower to tip over. There can be various types of grass on the lawn, and the growth of grass is generally affected by factors such as light, precipitation, season, and temperature, making it challenging for the self - propelled lawn mower to perform intelligent lawn maintenance. With the changes in weather and seasons, the lawn ground is prone to obvious changes in surface undulation, which will in turn cause the self - propelled lawn mower working on the lawn ground to jolt. And due to the complex and variable outdoor working environment, including smoke, dust, weather, and light changes, etc., it may cause an overall shift in the distribution of the acquired lawn images. Intelligent recognition can be carried out based on the lawn images, but the diversity and shift of the lawn images will affect the recognition accuracy to a certain extent.

[0069] Figure 2 is a schematic structural diagram of a self - propelled lawn mower 100 provided by an embodiment of the present application. Refer to Figure 2 , an embodiment of the present application provides a self - propelled device, including: a fuselage 110; a walking wheel assembly 120 configured to support the fuselage 110; a vision sensor 130 configured to acquire an image in the traveling direction of the self - propelled device; in combination with Figure 2 and Figure 3 , an attitude sensor 140 configured to acquire the attitude of the self - propelled device; a controller 150 electrically connected to the vision sensor 130 and the attitude sensor 140, and the controller 150 is configured to calculate a first inclination angle of the ground in the traveling direction of the self - propelled device according to the image in the traveling direction of the self - propelled device acquired by the vision sensor 130, calculate a second inclination angle of the ground in combination with the first inclination angle and the attitude of the self - propelled device, and control the walking wheel assembly 120 according to the second inclination angle.

[0070] In combination with Figure 2 and Figure 3 , the vision sensor 130 can be installed on the fuselage of the self - propelled device above the axle, and the vision sensor 130 can be flexibly installed in various areas of the fuselage. For example, it can be directly installed on the housing of the fuselage 110, and the installation orientation of the vision sensor 130 can include all angles. By flexibly setting the installation position and installation orientation of the vision sensor 130 on the self - propelled device, the acquisition angle of the image can be flexibly adjusted. Refer to Figure 2 and Figure 3 , the attitude sensor 140 and the controller 150 are also respectively arranged on the fuselage of the self - propelled device, and the controller 150 is electrically connected to the vision sensor 130 and the attitude sensor 140 respectively.

[0071] Refer toFigure 3 The controller 150 may include an acquisition unit 151, a processing unit 152, and a control unit 153. Specifically, the image of the mobile device in the traveling direction may be collected by the vision sensor 130, and the attitude of the mobile device may be obtained by the attitude sensor 140. The acquisition unit 151 in the controller 150 obtains the image from the vision sensor 130 and the attitude of the mobile device from the attitude sensor 140; the processing unit 152 processes each image by a preset method to obtain a first inclination angle of the ground in the traveling direction of the mobile device; and the second inclination angle of the ground may be obtained according to the attitude of the mobile device and the first inclination angle. The control unit 153 may control the traveling of the walking wheel assembly 120 by using the second inclination angle.

[0072] In this solution, by combining the attitude of the mobile device and according to the first inclination angle, the second inclination angle is obtained, which improves the accuracy of slope angle recognition. Taking the mobile device as the self-propelled lawn mower 100 as an example, in the same slope scenario, even if the self-propelled lawn mower works on the slope in different attitudes, the second inclination angle of the slope can be accurately obtained by combining the attitude and the first inclination angle, thereby preventing the self-propelled lawn mower 100 from tipping over.

[0073] In an implementable manner, the first inclination angle is the relative inclination angle of the ground with respect to the mobile device; the second inclination angle is the absolute inclination angle of the ground with respect to the gravity direction.

[0074] Figure 4 This is a schematic diagram of a slope detection scenario of a mobile device provided by an embodiment of the present application. Taking the mobile device as the self-propelled lawn mower 100 as an example, as Figure 4 shown, the dashed line CD represents the horizon; the dashed line BH is parallel to the dashed line CD; the solid line BC represents the ground on which the self-propelled lawn mower 100 is traveling; the solid line AB represents the ground in front of the traveling direction of the self-propelled lawn mower 100; the dashed line BG is the extension line of the solid line BC; the dashed line EF represents the gravity direction; wherein, the dashed line CD and the dashed line EF are perpendicular to each other.

[0075] The self-propelled mower 100 can collect the front image in the traveling direction and process the image into a point cloud. Then, the ground point cloud is extracted from the point cloud, and the ground point cloud corresponds to the solid line AB. The inclination angle of the front ground is extracted through the ground point cloud, and the inclination angle is the relative inclination angle of the solid line AB with respect to the self-propelled mower 100, and the relative inclination angle can be represented by ∠ABG. The pitch angle in the attitude of the self-propelled mower 100 is represented by ∠BCD. Since the dotted line BH is parallel to the dotted line CD, and the dotted line BG is the extension line of the solid line BC, then ∠BCD = ∠GBH. Adding the pitch angle and the relative inclination angle gives the first absolute inclination angle of the front ground with respect to the gravity direction, and the first absolute inclination angle can be represented by ∠ABH.

[0076] Similarly, the roll angle in the attitude of the self-propelled mower 100 can also be used to determine the second absolute inclination angle of the front ground with respect to the gravity direction.

[0077] The first absolute inclination angle and the second absolute inclination angle are used as the slope detection results. Further, the first absolute inclination angle can be compared with the first angle threshold, and the second absolute inclination angle can be compared with the second angle threshold. If the first absolute inclination angle is less than the first angle threshold and the second absolute inclination angle is less than the second angle threshold, it is determined that the front ground is a safe area. If the first absolute inclination angle is greater than or equal to the first angle threshold, or the second absolute inclination angle is greater than or equal to the second angle threshold, it is determined that the front ground is a non-safe area. Among them, both the first angle threshold and the second angle threshold can be preset according to the actual situation. The first angle threshold and the second angle threshold may not be equal. First, obtain the first absolute inclination angle and the second absolute inclination angle respectively according to the pitch angle and the roll angle in the attitude, and then comprehensively determine whether the front ground is a safe area according to the first absolute inclination angle and the second absolute inclination angle, which can improve the accuracy of the determination and further reduce the probability of the self-propelled device tipping over.

[0078] Among them, the first inclination angle refers to the relative inclination angle between the ground and the self-propelled device, while the second inclination angle refers to the absolute inclination angle of the ground and is not affected by the position of the self-propelled device. Specifically, in combination with the attitude of the self-propelled device, the influence of the attitude is removed from the first inclination angle to obtain the second inclination angle.

[0079] Specifically, according to the attitude of the self-propelled device, the inclination angle of the ground where the self-propelled device is located with respect to the gravity direction can be determined. Then, according to the inclination angle and the relative inclination angle of the ground in the traveling direction of the self-propelled device with respect to the self-propelled device, the absolute inclination angle of the ground in the traveling direction of the self-propelled device with respect to the gravity direction can be obtained.

[0080] In an actual application scenario, what affects whether the self - moving device topples is the absolute tilt angle of the ground where the self - moving device is located relative to the direction of gravity. Therefore, using the absolute tilt angle as the slope angle recognition result improves the accuracy of slope angle recognition.

[0081] In one implementable way, the vision sensor 130 may include at least one of the following: an active laser depth camera, a passive binocular stereo camera, and a passive binocular structured light camera.

[0082] In one implementable way, the vision sensor 130 includes at least one of a Time of Flight (TOF) depth camera, an Infrared Ray (IR) structured light camera, and a binocular camera.

[0083] In one implementable way, the controller 150 creates a depth image from the image, extracts the ground point cloud from the depth image, and obtains the first tilt angle through the ground point cloud.

[0084] The processing unit 152 in the controller 150 can create a depth image based on the image in the traveling direction of the self - moving device. Specifically, the depth information of each pixel point can be estimated by analyzing the geometric relationship between the pixels and objects in the image. The depth image represents the distance of each point in the scene relative to the camera. There are various methods for creating a depth image from an image, including single - image - based depth estimation, multi - image - based stereo matching, etc. Among them, the stereo matching method refers to calculating the disparity by matching the corresponding pixel points of the same object in different images, thereby estimating the depth information.

[0085] The processing unit 152 can extract the ground point cloud from the depth image. Specifically, the depth image can be first processed into a point cloud, and then the ground point cloud can be extracted from the point cloud. Among them, various algorithms can be used to extract the ground point cloud from the point cloud. The various algorithms include detection based on three - dimensional features of the point cloud and detection based on planar features of the image. The detection algorithms are not limited to detection based on classical hand - crafted features and detection based on learned features. Among them, detection based on three - dimensional features of the point cloud refers to identifying and classifying target objects by extracting three - dimensional features in the point cloud data, such as shape, edge, texture, etc. Among them, detection based on planar features of the image refers to identifying and classifying target objects by extracting planar features in the image, such as edges, corner points, etc. Among them, detection based on classical hand - crafted features refers to identifying and classifying target objects by extracting hand - crafted features in the image, such as edges, corner points, texture, etc. Among them, detection based on learned features refers to learning the feature representation in the image and combining it with a classifier to identify and classify target objects.

[0086] In one implementable manner, the point cloud can also be combined with a color image, that is, the point cloud is colored to obtain a colored point cloud. Then, the ground point cloud is extracted from the colored point cloud, and further, a first tilt angle is obtained through the ground point cloud. Specifically, the colored point cloud contains the color information of the point cloud. Therefore, the colored point cloud contains more semantic information than the point cloud, and thus the first tilt angle obtained using the colored point cloud is more accurate to a certain extent.

[0087] The processing unit 152 can obtain the first tilt angle through the ground point cloud. Specifically, a geometric method can be used to obtain the first tilt angle through the ground point cloud. The geometric method includes at least one of the following methods: line-plane fitting method, voxel sampling method. Among them, the line-plane fitting method means that according to the straight lines and planes in the ground point cloud, through geometric reasoning and calculation, the straight lines and planes in the real three-dimensional scene corresponding to the ground point cloud are restored, and then the first tilt angle is calculated from the fitted straight lines and planes. Among them, the voxel sampling method means that the ground point cloud data is divided into a series of small cubes, called voxels, and then a representative point is selected as the sampling point in each voxel. Then, using the coordinate information of the sampling point, the tilt angle of each sampling point relative to the self-mobile device is calculated. By synthesizing the tilt angles of each sampling point relative to the self-mobile device, the first tilt angle is obtained.

[0088] Using the above method, the first tilt angle can be obtained quickly and accurately according to the image.

[0089] In one implementable manner, when the second tilt angle is greater than or equal to the first threshold, the controller 150 determines that the ground in the traveling direction of the self-mobile device is a non-safe area and changes the traveling direction.

[0090] Among them, the first threshold is a tilt angle threshold set in advance according to the actual situation.

[0091] Specifically, when it is determined that the second tilt angle is greater than the first threshold, the control unit 153 in the controller 150 determines that the ground in the traveling direction of the self-mobile device is a non-safe area and changes the traveling direction of the self-mobile device to prevent the self-mobile device from tipping over due to the ground tilt. Among them, the non-safe area refers to the ground area where the self-mobile device is likely to tip over due to an excessive tilt angle.

[0092] In one implementable manner, when the second tilt angle is less than the first threshold, the controller 150 determines that the ground in the traveling direction of the self-mobile device is a safe area and maintains the traveling direction.

[0093] Specifically, when it is determined that the second tilt angle is less than the first threshold, the control unit 153 in the controller 150 may determine that the ground in the traveling direction of the self - moving device is a safe area, and save the traveling direction and continue to travel. Herein, the safe area refers to a ground area where the tilt angle is not too high, and thus it is not easy for the self - moving device to tip over.

[0094] In one implementable manner, the traveling direction of the self - moving device is the front of the self - moving device.

[0095] Among them, the traveling direction of the self - moving device can be determined according to path planning.

[0096] In one implementable manner, the first threshold is determined according to the friction of the ground.

[0097] The first threshold can be determined according to the friction of the ground, and the value of the first threshold can be proportional to the friction of the ground. The greater the friction of the ground, the greater the value of the first threshold can be. Determining the first threshold according to the friction of the ground can make the value of the first threshold more accurate.

[0098] In one implementable manner, the attitude sensor 140 is an accelerometer.

[0099] Figure 5 is a flowchart of a control method for a self - moving device provided by an embodiment of the present application. This embodiment is applicable to the scenario of route planning when the self - moving device travels on the ground with a certain inclination. The self - moving device can be, for example, the self - moving lawn mower 100, and this method can be executed by the self - moving device. As Figure 5 shown, the method includes:

[0100] Step 501, obtain the attitude of the self - moving device; and obtain an image in the traveling direction of the self - moving device.

[0101] Step 502, calculate the first tilt angle of the ground in the traveling direction of the self - moving device according to the image.

[0102] Step 503, combine the first tilt angle and the attitude to calculate the second tilt angle of the ground; and decide whether to travel along the traveling direction according to the second tilt angle.

[0103] This solution combines the attitude of the self - moving device, eliminates the influence of the attitude from the first tilt angle to obtain the second tilt angle. The second tilt angle is used as the slope detection result of the ground in the traveling direction of the self - moving device. And it can be decided whether to travel along the traveling direction according to the second tilt angle, control the traveling of the self - moving device, and prevent the self - moving device from tipping over due to too large a slope. The accuracy of slope detection of the ground in the traveling direction of the self - moving device is improved, and thus the tipping probability of the self - moving device is reduced.

[0104] The embodiment of the present application also specifically provides a slope detection algorithm based on point cloud for a lawn mower equipped with a depth camera. Figure 6 is a flowchart of a slope detection algorithm based on point cloud provided by an embodiment of the present application. Refer to Figure 6 , and the specific process of this algorithm is as follows:

[0105] Step 601, power on the lawn mower and start the slope detection program in the lawn mower controller.

[0106] Step 602, use the depth camera to collect the image in front of the lawn mower. The depth camera is not limited to an active laser depth camera, a passive binocular stereo camera, a passive binocular structured light camera, etc.

[0107] Step 603, process the depth image collected by the lawn mower into point cloud through the algorithm in the slope detection program.

[0108] Step 604, through algorithm processing, extract the ground point cloud from the point cloud in front of the lawn mower. The algorithm includes detection based on three-dimensional features of the point cloud and detection based on image plane features. The detection algorithm is not limited to detection based on classical handcrafted features and detection based on learned features.

[0109] Step 605, through geometric methods, extract the relative inclination angle of the ground on the travel route through the ground point cloud information. The extraction method is not limited to methods such as line-plane fitting method and voxel sampling method, etc.

[0110] Step 606, combine the attitude information provided by its own inertial navigation with the ground inclination information to comprehensively judge the absolute inclination angle of the ground relative to the gravity direction.

[0111] Step 607, upload the inclination angle of the ground relative to the gravity direction to the planning and decision-making layer to judge whether it is a safe area where it can travel.

[0112] Step 608, plan the route according to the judgment result of step 607. Specifically, if it is determined to be an area where it can travel, then control the lawn mower to continue moving forward in the travel direction; if it is determined to be an area where it cannot travel, then change the travel direction and plan a safe travel route.

[0113] Figure 7 is a schematic diagram of a lawn image provided by an embodiment of the present application. The growth state of grass is affected by factors such as light, precipitation, season, temperature, etc. and is not constant. As Figure 7 shown in the lawn image, which contains the characteristic information of grass, such as information about the height, color, variety, etc. of grass, characterizing the growth state of grass. It is a challenge for a self-propelled lawn mower to perform intelligent maintenance on the lawn for different growth states of grass.

[0114] In one embodiment, in combination with Figure 2And Figure 3 , the controller 150 is electrically connected to the vision sensor 130. The controller 150 is configured to obtain and receive, according to the inspection instruction, the lawn image of the working area acquired by the vision sensor 130; process the lawn image to obtain the feature information of the grass; wherein, the feature information at least includes the height and color of the grass; formulate a mowing plan or an inspection plan according to the feature information, and control the self-propelled mower 100 to work according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time for performing the mowing task; the inspection plan includes the time for generating the next inspection instruction. Refer to Figure 3 , the controller 150 may include an acquisition unit 151, a processing unit 152 and a control unit 153. Specifically, the lawn image of the working area where the self-propelled mower 100 is located can be collected by the vision sensor 130. Refer to Figure 3 , the acquisition unit 151 can obtain the inspection instruction and obtain the lawn image from the vision sensor 130 according to the inspection instruction; wherein, the inspection instruction can be generated in response to the user's operation or generated according to the inspection plan. The processing unit 152 can process the lawn image by using a preset method to obtain the feature information of the grass, wherein, the feature information at least includes the height and color of the grass. Specifically, the lawn image can be preprocessed first, including steps such as denoising, image enhancement, color correction, etc., to improve the image quality. Then, the deep learning semantic segmentation algorithm can be used to separate the grass plants in the preprocessed lawn image from the background. According to the segmentation result, parameter calculation can be performed on the segmented grass plants, at least including the height and color of the grass plants. Finally, statistical analysis is performed on the calculated grass plant parameters to obtain the overall features of the lawn. For example, statistical data such as the average grass plant height and the average grass plant color can be calculated. The average grass plant height can be used as the height of the grass, and the average grass plant color can be used as the color of the grass. The processing unit 152 can formulate a mowing plan or an inspection plan according to the feature information of the grass. For example, if it is determined according to the feature information of the grass that the lawn needs to be trimmed recently, a mowing plan is generated. If it is determined according to the feature information of the grass that the lawn does not need to be trimmed recently, an inspection plan can be generated to monitor the growth of the grass and perform mowing when the grass needs to be trimmed. The control unit 153 can control the self-propelled mower 100 to work according to the mowing plan or the inspection plan. Wherein, the mowing plan may include the mowing task and the time for performing the mowing task. The inspection plan may include the time for the self-propelled mower 100 to generate the next inspection instruction.

[0115] Since the colors of different types of grass usually vary, and the colors of the same type of grass usually also vary to a certain extent in different seasons, the suitable mowing heights for different types of grass usually vary to a certain extent, and the suitable mowing heights for the same type of grass usually also vary in different seasons. Therefore, in this solution, the growth state of the grass is identified based on at least characteristic information such as the height and color of the grass. For example, the type of grass and the season it is in can be determined based on the color, and a mowing plan or inspection plan for the self-propelled mower 100 can be formulated according to the type, height, and season of the grass, making the maintenance of the lawn more intelligent, and thus improving the quality of lawn maintenance.

[0116] In one implementable manner, a mowing height threshold is determined according to the color.

[0117] Specifically, a color-height mapping table can be preset according to experience, and the mapping relationship between the color and the height is included in the table. The processing unit 152 can query the color-height mapping table using the color of the grass to obtain the height corresponding to the color of the grass, and determine this height as the mowing height threshold.

[0118] If the height is greater than the mowing height threshold, control the self-propelled mower to perform the mowing task.

[0119] The processing unit 152 can compare the height of the grass with the mowing height threshold. If it is determined that the height of the grass is greater than the mowing height threshold, a mowing task to be executed is generated. The control unit 153 controls the self-propelled mower 100 to work according to the mowing task to be executed.

[0120] If the height is less than or equal to the mowing height threshold, a mowing plan or inspection plan is formulated according to the difference between the mowing height threshold and the height.

[0121] If the processing unit 152 determines that the height of the grass is less than or equal to the mowing height threshold, it calculates the difference between the mowing height threshold and the height of the grass, and formulates a mowing plan or inspection plan according to the difference. The control unit 153 controls the self-propelled mower 100 to work according to the mowing plan or inspection plan. For example, if the difference is small, it can be determined that the grass needs to be trimmed in the near future, and at this time, a mowing plan can be formulated. If the difference is large, it can be determined that the grass does not need to be trimmed in the near future, and at this time, an inspection plan can be formulated to monitor the growth of the grass and mow the grass when it needs to be trimmed.

[0122] Since the colors of different types of grass usually vary, and the colors of the same type of grass usually also vary to some extent in different seasons, the suitable mowing heights for different types of grass usually have certain differences, and the suitable mowing heights for the same type of grass usually also vary in different seasons. Therefore, in this solution, the growth state of the grass is identified by the color of the grass. For example, the type of grass and the season it is in can be determined based on the color, and then the mowing height threshold can be determined according to the type of grass and the season. Thus, a mowing plan or an inspection plan can be formulated based on the mowing height threshold and the height of the grass, which can improve the maintenance quality of the lawn.

[0123] In one implementable manner, the feature information further includes size, shape, and / or density; the obtaining unit 151 obtains the season information, and the processing unit 152 determines the type of grass according to the season information, size, shape, and color.

[0124] In this application, a feature information mapping table can be preset according to experience. The table includes the mapping relationships between season, size, shape, color, and the type of grass. The obtaining unit 151 can also obtain the season information. Specifically, the obtaining unit 151 can calculate the season information based on the time information. Or, the season information can be determined at least according to the color of the grass. The processing unit 152 can process the lawn image to obtain the size, shape, and density of the grass. Specifically, the grass plants in the lawn image can be separated from the background, and then parameter calculations can be performed on the segmented grass plants, including the size, shape, and density of the grass plants. Among them, the size includes at least height and width; the shape includes at least roundness and complexity; the density includes at least the number of grass plants per unit area. Finally, statistical analysis is performed on the calculated grass plant parameters to obtain the overall characteristics of the lawn. For example, statistical data such as the average grass plant height, average grass plant width, average grass plant roundness, average grass plant complexity, and average density can be calculated. Information such as the average grass plant height and average grass plant width can be used as the size of the grass, information such as the average grass plant roundness and average grass plant complexity can be used as the shape of the grass, and information such as the average density can be used as the density of the grass. Then, using the season information, the size, shape, and color of the grass, the feature information mapping table is queried to obtain the type of grass.

[0125] The processing unit 152 determines the mowing height threshold according to the type of grass, density, and season information.

[0126] Specifically, a feature mapping table can be preset according to experience. The table includes the mapping relationships between type, density, season, and height. The processing unit 152 can query the feature mapping table according to the season information and the type and density of the grass to obtain the height corresponding to the season information and the type and density of the grass, and determine this height as the mowing height threshold.

[0127] Since different types of grass usually have differences in characteristics such as size, shape, and color in different seasons, the suitable mowing heights for different types of grass usually have certain differences. The suitable mowing heights for the same type of grass usually have certain differences under different seasonal conditions, and the suitable mowing heights for grass with different densities also usually have certain differences. Therefore, this solution can first accurately determine the type of grass according to the seasonal information and the size, shape, and color of the grass, and then more accurately determine the mowing height threshold according to the type of grass, density, and seasonal information. Controlling the operation of the automatic lawn mower 100 according to the mowing plan or inspection plan formulated based on this mowing height threshold can improve the quality of lawn maintenance.

[0128] In an implementable manner, if the processing unit 152 determines that the difference between the mowing height threshold and the height of the grass is less than or equal to a preset threshold, it formulates a mowing plan according to the difference; if the difference is greater than the preset threshold, it formulates an inspection plan according to the difference.

[0129] Among them, the preset threshold is a value set in advance according to the actual situation. If the difference between the mowing height threshold and the height of the grass is less than or equal to the preset threshold, it can indicate that the difference between the height of the grass and the mowing height threshold is not large. In this case, generally, the growth rate of the grass can be more accurately estimated, and then a more accurate mowing plan can be formulated according to the estimated growth rate of the grass.

[0130] If the difference between the mowing height threshold and the height of the grass is greater than the preset threshold, it can indicate that the difference between the height of the grass and the mowing height threshold is relatively large. In this case, it is generally difficult to accurately estimate the growth rate of the grass, and it is very difficult to accurately formulate a mowing plan. Therefore, at this time, an inspection plan can be formulated according to the difference to monitor the growth of the grass. For example, when the difference is greater than the preset threshold and if the difference is relatively large, a longer time interval can be set to inspect the lawn to determine whether mowing is needed; when the difference is greater than the preset threshold and if the difference is relatively small, a relatively shorter time interval can be set to inspect the lawn to determine whether mowing is needed. This can improve the quality of lawn maintenance and save resources.

[0131] This solution can first determine the difference between the mowing height threshold and the height of the grass, and then formulate a mowing plan or inspection plan based on the difference and the comparison result between the difference and the preset threshold, which can more precisely control the operation of the self-propelled lawn mower 100, and thus achieve a better lawn maintenance effect.

[0132] In an implementable manner, the height of the grass is determined according to the point cloud data of the grass; the point cloud data is determined according to the lawn image.

[0133] The processing unit 152 can first create a depth image from the lawn image and then process the depth image into a point cloud, which is the point cloud data of the grass. Then, the height of the grass can be identified using the point cloud data of the grass. In this way, the height of the grass can be obtained quickly and accurately. Specifically, each point on the depth image can be transformed into the world coordinate system through the relationship of similar triangles or coordinate system transformation to obtain the point cloud data of the grass. First, the vision sensor 130 needs to be aligned with the lawn surface and the three-dimensional point cloud data of the lawn is obtained. The vision sensor 130 can capture the three-dimensional coordinates of each point on the lawn surface, including the X, Y, and Z coordinates. The obtained point cloud data needs to be processed to remove noise, smooth the data, and perform data registration, etc. These processes can include operations such as filtering, smoothing, and resampling to improve the quality and accuracy of the data. Features can be extracted from the point cloud data, including the shape and height of the grass. Furthermore, a classifier can be designed based on the extracted features to distinguish the grass from other objects. After the classifier identifies the grass, in the processed point cloud data, a certain area of the point cloud data can be selected, and the average height of the grass in this area can be obtained by calculating the average height of these points. Specifically, some representative point cloud data can be selected and their average height can be calculated to obtain the grass height in this area. It should be noted that the accuracy and stability of the vision sensor 130 may be affected by factors such as environmental lighting and camera settings, so fine calibration and calibration need to be carried out in advance. Of course, the point cloud data can also be obtained by lidar. Therefore, in this solution, lidar can also be used as an alternative to the vision sensor 130.

[0134] Figure 8 FIG. 4 is a flowchart of a control method for a self-propelled lawn mower provided by an embodiment of the present application. This embodiment is applicable to the scenario where the self-propelled lawn mower 100 performs intelligent maintenance on the lawn, especially applicable to the situation of being out of user control for a long time. This method can be executed by the self-propelled lawn mower 100. As Figure 8 shown, this method includes:

[0135] Step 801, obtain an inspection instruction and obtain a lawn image of the working area of the self-propelled lawn mower according to the inspection instruction.

[0136] Step 802, process the lawn image to obtain the characteristic information of the grass; wherein, the characteristic information at least includes the height and color of the grass.

[0137] Step 803, formulate a mowing plan or an inspection plan according to the characteristic information, and control the self-propelled lawn mower to work according to the mowing plan or the inspection plan; wherein, the mowing plan includes the time for performing the mowing task; the inspection plan includes the time for generating the next inspection instruction.

[0138] Since the colors of different types of grass usually vary, and the colors of the same type of grass usually also have certain differences in different seasons, the suitable mowing heights for different types of grass usually vary to some extent, and the suitable mowing heights for the same type of grass in different seasons usually also have certain differences. Therefore, this solution can formulate a more accurate mowing plan or inspection plan at least based on the height and color of the grass, so as to maintain the lawn more intelligently and achieve a better lawn maintenance effect.

[0139] The embodiment of the present application also specifically provides an intelligent mowing decision-making algorithm for a lawn mower based on semantic recognition. Figure 9 is a flowchart of an intelligent mowing decision-making algorithm provided by an embodiment of the present application. Refer to Figure 9 and the specific process of this algorithm is as follows:

[0140] Step 901, power on the lawn mower.

[0141] Step 902, start executing the inspection mode in the lawn mower controller and determine whether to start the inspection.

[0142] Step 903, the lawn mower patrols on the lawn along a certain route, which can be either a user-specified route, a route autonomously planned by the lawn mower according to the lawn conditions, or a completely random route.

[0143] Step 904, obtain an image through a vision sensor, and use a certain algorithm to extract the semantic information in the image, and determine whether the lawn mower is traveling in the lawn area through the semantic information. Among them, this algorithm can be a deep learning algorithm. If the semantic information indicates that there is grass in the image, it is determined that the lawn mower is traveling in the lawn area; if the semantic information indicates that there is no grass in the image, it is determined that the lawn mower is not traveling in the lawn area. It is also possible to first process the image into a colored point cloud and identify the semantic information through the colored point cloud. It is also possible to combine the semantic information recognition results of the image and the semantic information recognition results of the colored point cloud to obtain the combined semantic information.

[0144] Step 905, if not in the lawn area, continue to move forward and continue to execute step 903; if in the lawn area, collect the lawn-related semantic information and feature information extracted by the algorithm. Among them, the semantic information includes identification labels such as whether it is grass, the type of grass, density, and the season to which it belongs. The semantic information can include semantic labels at the pixel or sub-pixel level of the lawn within the field of view. The feature information of the grass includes the color, height, size, shape, etc. of the grass. Among them, the season can also be judged by time.

[0145] Step 906: Determine whether the collected feature information is sufficient according to a preset statistical algorithm. If it is not sufficient, continue to collect; if it is sufficient, stop collecting, and further extract the information required for decision-making based on the set of currently collected feature information. Among them, the decision-making information is not limited to the average value and variance of the grass feature information in the distribution of the entire lawn area, as well as the average value and variance of the grass feature information in different subdivided areas, and the semantic information of the grass, etc. Among them, determining whether the collected feature information is sufficient according to the preset statistical algorithm includes: if it is determined that the area of the lawn being statistically analyzed is large enough, it is determined that the collected feature information is sufficient.

[0146] Step 907: Comprehensively judge whether to perform a mowing task based on the decision-making information. Among them, the mowing task is not limited to mowing the entire area, mowing part of the area, and multiple mowing tasks at different times. For example, the working area of the lawn mower can include various scenarios, such as uneven grass growth in the working area, uneven mowing in the previous mowing, or there are multiple types of grass in the working area and the mowing height requirements for each type of grass are different, etc. For different scenarios, the mowing task can be adaptively set.

[0147] Step 908: Execute the mowing task planned in Step 907. If the mowing task cannot be executed due to other reasons, such as manual intervention, weather conditions, battery power, etc., record the current condition of the lawn for comprehensive decision-making in the next plan.

[0148] Figure 10 It is a schematic diagram of the positioning scenario when a self-mobile device travels on a lawn provided by an embodiment of the present application. As Figure 10 shown, the lawn is generally outdoors. Due to the changes in outdoor weather and seasons, obvious changes in the surface conditions of the lawn ground will occur. And the different undulation degrees of the lawn ground will affect the acquisition of inertial data by the inertial measurement unit on the self-mobile device, and thus easily cause inaccurate positioning of the self-mobile device, which is a challenge for the self-mobile device.

[0149] In one embodiment, in combination with Figure 2 and Figure 3 , taking the attitude sensor 140 as an example of the inertial measurement unit (Inertial Measurement Unit, IMU), it is disposed inside the fuselage 110. The inertial measurement unit includes at least one of an accelerometer and a gyroscope; the controller 150 is electrically connected to the inertial measurement unit. The controller 150 is configured to control the self-mobile device to move; receive the inertial data of the inertial measurement unit; perform statistics on the inertial data received within the first time window, and adjust the positioning parameters of the inertial measurement unit according to the statistical results.

[0150] In combination with Figure 2 and Figure 3, the attitude sensor 140 and the controller 150 are respectively disposed on the body of the self - moving device, and the controller 150 is electrically connected to the attitude sensor 140.

[0151] Reference Figure 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, the self - moving device can be controlled to move by the controller 150, and the inertial data of the self - moving device can be collected by the inertial measurement unit. The acquisition unit 151 in the controller 150 can obtain the inertial data from the inertial measurement unit; the processing unit 152 can statistically process the inertial data received within the first time window in a preset manner to obtain a statistical result, where the duration of the first time window is preset according to the actual situation; and adjust the positioning parameters of the inertial measurement unit according to the statistical result; and then determine the position of the self - moving device according to the inertial data and the positioning parameters.

[0152] The self - moving device can be positioned using inertial data. The surface bumpiness of the driving area of the self - moving device will affect the accuracy of the inertial measurement unit in collecting inertial data, and thus affect the positioning accuracy of the self - moving device. The statistical result of the inertial data can reflect the surface bumpiness. Therefore, in this solution, the positioning parameters of the inertial data are adjusted according to the statistical result of the inertial data of the self - moving device to minimize the influence of surface bumpiness on positioning as much as possible, and then the self - moving device is positioned through the inertial data and the adjusted positioning parameters, which can improve the positioning accuracy.

[0153] In one implementable manner, the statistical result reflects the flatness of the road section passed by the self - moving device within the first time window.

[0154] The higher the flatness of the road section, the flatter the road section and the smaller the bumpiness.

[0155] In one implementable manner, the inertial data collected by the inertial measurement unit at least includes the acceleration and angular velocity of the self - moving device. Then, statistically processing the inertial data received within the first time window in a preset manner to obtain a statistical result includes: respectively integrating the acceleration and angular velocity in the inertial data received within the first time window to obtain the velocity and angle respectively; then respectively performing smoothing processing on the acceleration, angular velocity, velocity and angle to obtain the acceleration processing result, angular velocity processing result, velocity processing result and angle processing result; determining the first variance between each acceleration and the acceleration processing result, determining the second variance between the angular velocity and the angular velocity processing result, determining the third variance between the velocity and the velocity processing result, and determining the fourth variance between the angle and the angle processing result; and determining the first variance, second variance, third variance and fourth variance as the statistical result.

[0156] Among them, smoothing the acceleration to obtain the processed acceleration result includes: processing the acceleration by using the sliding window smoothing method to obtain the processed acceleration result. For example, averaging the accelerations collected within the first time window and using the average value as the processed acceleration result.

[0157] Similarly, smoothing the angular velocity to obtain the processed angular velocity result; smoothing the velocity to obtain the processed velocity result; smoothing the angle to obtain the processed angle result.

[0158] Among them, determining the first variance between each acceleration and the processed acceleration result includes: determining the square of the difference between each acceleration and the processed acceleration result, and taking the weighted sum of each square as the first variance. Among them, different weight values can be adopted for the accelerations in different sections, and the weight values corresponding to different sections can be set by methods such as manual parameter adjustment or machine learning.

[0159] Similarly, the second variance between the angular velocity and the processed angular velocity result can be determined; the third variance between the velocity and the processed velocity result can be determined; the fourth variance between the angle and the processed angle result can be determined.

[0160] Smoothing processing can obtain the approximate mean value in dynamic motion, thereby statistically analyzing the bumpy situation during the motion process. The variance can better show the distribution difference of the data near the mean value. The smaller the distribution difference, the smaller the variance, and the larger the distribution difference, the larger the variance.

[0161] In an implementable manner, it is possible to first determine whether the positioning parameters of the inertial measurement unit need to be adjusted according to the statistical results. If it is determined that adjustment is needed, the positioning parameters are adjusted; if it is determined that adjustment is not needed, the positioning parameters are not adjusted.

[0162] This solution can determine whether the positioning parameters need to be adjusted based on the statistical results, and only adjust the positioning parameters when the conditions are met, thereby improving the positioning accuracy to a certain extent.

[0163] Specifically, a preset range can be set according to experience, and the statistical result is compared with the preset range. If it is determined that the statistical result is within the preset range, it is determined that the positioning parameters need to be adjusted. This method can quickly and accurately determine whether the positioning parameters need to be adjusted.

[0164] Alternatively, a binary classification model can be trained using the statistical results for training. The classification result of the binary classification model is used to represent whether the positioning parameters of the inertial measurement unit need to be adjusted. Then, the trained binary classification model can be used to process the statistical results to obtain the classification result. If the classification result indicates that the positioning parameters need to be adjusted, the positioning parameters are adjusted; if the classification result indicates that the positioning parameters do not need to be adjusted, the positioning parameters are not adjusted. This method can quickly and accurately determine whether the positioning parameters need to be adjusted.

[0165] Specifically, a mapping table can be preset according to experience. The mapping table includes the mapping relationship between the flatness and the adjustment parameters. Furthermore, the mapping table can be queried using the statistical results to obtain the adjustment parameters corresponding to the statistical results, and the positioning parameters of the inertial measurement unit are updated using the adjustment parameters. This method can quickly and accurately adjust the positioning parameters.

[0166] Alternatively, the statistical results are processed using a preset regression model to obtain a regression result. Then, the positioning parameters are updated using the regression result. This method can quickly and accurately adjust the positioning parameters.

[0167] In one implementable manner, the first time window is determined according to the walking speed of the self - moving device.

[0168] Specifically, the faster the walking speed of the self - moving device, the shorter the duration of the first time window.

[0169] In one implementable manner, the value range of the first time window is greater than or equal to a first duration threshold and less than a second duration threshold; wherein, the first duration threshold is less than the second duration threshold.

[0170] Among them, both the first duration threshold and the second duration threshold are preset according to experience. The first time window can take any value between the first duration threshold and the second duration threshold.

[0171] Only the statistical results obtained by statistically analyzing a sufficient amount of inertial data can more accurately represent the flatness of the road section. The method of setting the first time window in this solution is a relatively convenient and accurate way to determine whether the acquired inertial data is sufficient.

[0172] In one implementable manner, the positioning parameter is the noise of the inertial data.

[0173] Specifically, the surface roughness of the driving area of the self - moving device will affect the accuracy of the inertial measurement unit in collecting inertial data. The positioning parameter can be used as the noise of the inertial data to characterize the accuracy of the inertial data. The statistical result of the inertial data can reflect the surface roughness. Therefore, in this solution, the positioning parameter of the inertial data is adjusted according to the statistical result of the inertial data to minimize the impact of surface roughness on positioning as much as possible, which can improve the positioning accuracy.

[0174] In an implementable manner, the positioning parameter is the weight of the inertial data when the inertial data is fused with other - modality data for positioning.

[0175] In this application, a lidar and a camera can also be set on the self - moving device. The self - moving device can be positioned by multi - modality fusion using inertial data and other - modality data. Specifically, different types of data, such as inertial data, point cloud data, and image data, can be obtained through sensors such as the inertial measurement unit, lidar, and camera. These data are processed by a multi - modality fusion algorithm to achieve more accurate positioning of the self - moving device. Among them, the multi - modality fusion algorithm includes the weighted sum of multi - modality data, and the positioning parameter can be the weight of the inertial data when it is fused with other - modality data for positioning. The rougher the surface of the driving area of the self - moving device is, the less accurate the inertial data obtained by the inertial measurement unit is. Therefore, when the surface is relatively rough, by reducing the weight of the inertial data in the fusion positioning, the accuracy of the fusion positioning can be improved to a certain extent. And the statistical result of the inertial data can reflect the surface roughness. Therefore, in this solution, the positioning parameter of the inertial data of the self - moving device is adjusted according to the statistical result of the inertial data, and the positioning of the self - moving device calculated using the adjusted positioning parameter is more accurate.

[0176] In one implementable manner, the inertial measurement unit can be shock-absorbed to reduce the impact on the inertial measurement unit caused by the unevenness of the driving section of the self-mobile device. Specifically, the shock-absorption measures for the inertial measurement unit can be implemented from two aspects: design and structure. In terms of design, in order to reduce the impact of vibration and shock on the inertial measurement unit, the adaptability of the inertial measurement unit can be improved through material selection and reasonable structural design. For example, select high-precision next-generation inertial measurement units, such as MPU6050, BMI055, etc. These chips have high adaptability and stability, and can reduce the impact of body vibration on the measurement accuracy of the inertial measurement unit. In terms of structure, a series of shock-absorption designs can be adopted, such as using shock-absorbing materials, such as silica gel, rubber, silicone rubber, etc., and shock-absorbing structures to isolate vibration. Specifically, by reasonably arranging the structure of the inertial measurement unit circuit board, the center of gravity of the chip and the center of gravity of the inertial measurement unit can be at the same point to achieve the best shock-absorption effect. In addition, shock absorbers and air dampers and other shock-absorbing devices can also be considered. These devices can effectively reduce the impact of body vibration on the inertial measurement unit.

[0177] Figure 11 FIG. 4 is a flowchart of a positioning method for a self-mobile device provided by an embodiment of the present application. This embodiment is applicable to the positioning scenario of a self-mobile device during driving, and this embodiment is particularly applicable to the positioning scenario where the self-mobile device is driving on a bumpy section. This method can be executed by the self-mobile device. As Figure 11 shown, the method includes:

[0178] Step 1101, obtain a positioning request, and obtain inertial data of the self-mobile device within a first time window according to the positioning request.

[0179] Step 1102, statistically analyze the inertial data, and adjust the positioning parameters corresponding to the inertial data according to the statistical results.

[0180] Step 1103, determine the position of the self-mobile device according to the inertial data and the positioning parameters.

[0181] The bumpy condition of the ground surface in the driving area of the self-mobile device will affect the accuracy of the inertial data collected by the inertial measurement unit, and further affect the positioning accuracy of the self-mobile device. The statistical result of the inertial data can reflect the bumpy condition of the ground surface. Therefore, in this solution, the positioning parameters of the inertial data are adjusted according to the statistical result of the inertial data of the self-mobile device to minimize the impact of ground surface bumps on positioning as much as possible, and then the self-mobile device is positioned through the inertial data and the adjusted positioning parameters, which can improve the positioning accuracy.

[0182] The embodiment of the present application also specifically provides a positioning parameter adjustment algorithm based on the statistical data of the inertial measurement unit for a lawn mower. Figure 12It is a flowchart of a positioning parameter adjustment algorithm based on inertial measurement unit data statistics provided by an embodiment of the present application. Refer to Figure 12 , the specific process of this algorithm is as follows:

[0183] Step 1201, collect inertial data through the inertial measurement unit in the lawn mower. The inertial data includes but is not limited to acceleration, angular velocity, etc.

[0184] Step 1202, determine whether the inertial data is collected sufficiently. The judgment basis includes but is not limited to status information such as the time of collecting data, the distance traveled by the lawn mower, and whether the lawn mower is mowing grass. Since the vibration generated by the lawn mower during mowing work will also affect the inertial measurement unit, and this influence is easily confused with the influence of road surface bumps on the inertial measurement unit, it is necessary to collect inertial data when the lawn mower is not mowing grass. If the inertial data is not sufficient, then the adjustment of the positioning parameters based on the inertial data will not be accurate. Therefore, it is necessary to collect sufficient inertial data. If the collection time length of the inertial data reaches the preset time length threshold, or the distance traveled by the lawn mower during the collection process of the inertial data reaches the preset distance threshold, it can be determined that the inertial data is collected sufficiently. When the inertial data is sufficient, proceed to step 1203; if it is not sufficient, continue to collect.

[0185] Step 1203, use an algorithm to process the collected inertial data to obtain statistical indicators of the inertial data. The algorithm includes but is not limited to machine learning methods or statistical learning methods, etc. Specifically, the inertial data includes at least acceleration and angular velocity. Integrate the inertial data to obtain velocity data and angle data. Use the sliding window smoothing method to smooth the inertial data, velocity data, and angle data, and respectively calculate the variances of the inertial data, velocity data, and angle data based on the smoothed data. Different weighting weights are adopted for data in different numerical ranges, and the weight values can be set by methods such as manual parameter adjustment or machine learning methods.

[0186] Step 1204, based on the series of statistical indicators obtained in step 1203, use a qualitative method to determine whether parameter adjustment is required. The qualitative method includes but is not limited to the empirical judgment method or the machine learning binary classification method.

[0187] Step 1205, for the scenarios that require parameter adjustment in step 1204, according to the obtained series of statistical indicators, use the look-up table interpolation method or other machine learning regression methods to adjust the positioning parameters of the inertial measurement unit.

[0188] Figure 13 It is a flowchart of a model training method applied to lawn image processing provided by an embodiment of the present application. This embodiment is applicable to the model training scenario of identifying grass or non-grass objects in lawn images, and this method can be executed by an electronic device. As Figure 13 shown, this method includes:

[0189] Step 1301: Obtain the dataset to be trained. Among them, the dataset to be trained includes multiple lawn images to be trained, and the lawn images to be trained have annotation types.

[0190] Specifically, a vision sensor can be used to obtain the lawn images to be trained, and the lawn images to be trained can be processed such as cropped, sampled, and annotated. Among them, the lawn images to be trained can include information such as the ground, objects, and sky within the field of view of the vision sensor.

[0191] It is also possible to obtain the lawn images to be trained with annotation types publicly available on the network. Among them, the annotation types can include object types within the field of view of the vision sensor, such as grass, pedestrians, trees, houses, sky, and ground, etc.

[0192] Step 1302: Add noise and / or interference to at least some of the lawn images to be trained in the dataset to be trained to obtain the processed lawn images; and add the processed lawn images as new lawn images to be trained to the dataset to be trained to update the dataset to be trained.

[0193] Various methods can be used to add noise and / or interference to the lawn images to be trained to obtain the processed lawn images, so as to enrich the training samples and simulate as many possible situations in the actual scenario as possible. Specifically, refer to Figure 7 and Figures 14A to 14C , the lawn images to be trained can be blurred, including Gaussian blur, mean blur, and median blur. Among them, Figure 7 can represent the lawn images to be trained, Figure 14A is the Gaussian blur map obtained by performing Gaussian blur processing on the lawn images to be trained, Figure 14B is the mean blur map obtained by performing mean blur processing on the lawn images to be trained, Figure 14C is the median blur map obtained by performing median blur processing on the lawn images to be trained. Among them, Gaussian blur means blurring the image by convolving the image with a normal distribution. Mean blur mainly operates on each pixel point in the image and replaces its value with the average value of adjacent pixel points. Median blur mainly operates on each pixel point in the image and replaces its value with the median value of adjacent pixel points.

[0194] Step 1303: Process the lawn images to be trained using a preset model to obtain the predicted types of the lawn images to be trained; optimize the parameters of the preset model according to the annotation types and predicted types of the lawn images to be trained to obtain an identification model.

[0195] Among them, the preset model is a pre-set network model. This network model can be set based on a convolutional neural network. Specifically, the annotation type of the lawn image to be trained can be used as the label of the prediction type to train the preset model, optimize the parameters in the preset model, so that the prediction type is getting closer and closer to the annotation type. When reaching the preset condition, such as the similarity between the prediction type and the annotation type reaches 90%, the training is stopped to obtain the recognition model. The recognition model can be used to process the lawn image to be recognized and identify the target type of the lawn image to be recognized.

[0196] In this solution, by adding noise and / or interference to the lawn image to be trained, the training samples are enriched, and as many possible situations in the actual scenario as possible can be simulated. Using the richer training samples to train the recognition model can improve the recognition accuracy of the recognition model to a certain extent.

[0197] In one implementable way, at least some of the lawn images to be trained in the training dataset are subjected to one or more of the following combined processes to obtain the processed lawn images: blurring process, distortion process, overexposure process, adding pixel blocks, adding shadows of target objects, and adding preset objects.

[0198] Specifically, the blurring process of the lawn image to be recognized can be implemented using a blurring filter in an image processing software or programming language. The blurring process methods include at least one of the following: motion blur, depth-of-field blur, rotation blur, scaling blur, Gaussian blur, mean blur, and median blur. Among them, motion blur is caused by the movement of objects or the camera in the image. To simulate this effect, the method of randomly offsetting or rotating pixels can be used to generate the blurring effect. Depth-of-field blur is the blurring effect caused by the change of the focal length of the camera lens. This effect can be simulated by blurring the background and foreground pixels in the image to different degrees. Rotation blur can be simulated by rotating the image and applying a filter. Scaling blur is the blurring effect caused by the image being enlarged or reduced. This effect can be simulated by applying a filter after scaling the image.

[0199] Specifically, an image processing software or a transformation function in a programming language can be used to implement the distortion processing of the lawn image to be recognized. The distortion processing methods include at least one of the following: distortion transformation, shear transformation, rotation transformation, skew transformation, and affine transformation. Among them, the distortion transformation is a transformation method that maps the pixels in the image and can distort or deform a part of the image. This transformation can usually be achieved by defining a transformation matrix. The shear transformation is a transformation method that misplaces the pixels in the image along the vertical or horizontal direction. This transformation can be achieved by translating the pixels in the horizontal or vertical direction. The rotation transformation is a transformation method that rotates the image and can be achieved by defining the rotation center and rotation angle. The skew transformation is a transformation method that misplaces the pixels in the image along a straight line. This transformation can be achieved by defining two misplacement points. The affine transformation is a transformation method that maps the pixels in the image and can distort or deform a part of the image. This transformation can be achieved by defining an affine matrix.

[0200] Specifically, taking the visual sensor 130 as a camera as an example, for the camera, overexposure processing of the image can be achieved by increasing the exposure time or increasing the ISO sensitivity. The effect of overexposure can also be simulated by adjusting parameters such as the brightness, contrast, and color of the lawn image to be recognized. The overexposure processing methods include at least one of the following: increasing brightness, increasing contrast, color adjustment, highlight adjustment, and sharpening processing. Among them, increasing brightness means simulating the effect of overexposure by increasing the overall brightness of the image. It can be achieved by using a brightness adjustment function in an image processing software or a programming language. Increasing contrast means simulating the effect of overexposure by increasing the contrast of the image. It can be achieved by using a contrast adjustment function in an image processing software or a programming language. Color adjustment means simulating the effect of overexposure by adjusting the color of the image. It can be achieved by using a color adjustment function in an image processing software or a programming language. Highlight adjustment means simulating the effect of overexposure by increasing the highlight part of the image. It can be achieved by using a highlight adjustment function in an image processing software or a programming language. Sharpening processing means simulating the effect of overexposure by sharpening the edges and details of the image. It can be achieved by using a sharpening function in an image processing software or a programming language.

[0201] Specifically, solid color pixel blocks can be added to the lawn image to be trained.

[0202] Specifically, the shadows of objects that may appear on the lawn or the objects that may appear on the lawn can be added to the lawn image to be trained to enrich the training samples, and then a recognition model with higher recognition accuracy can be trained based on the more abundant training samples.

[0203] In one implementable manner, the shadow of any one of the following target objects is added to the lawn image to be trained: trees, fences, and houses.

[0204] In one implementable manner, add any one of the following preset objects to the lawn image to be trained: water pipe, faucet, water gun, toy, animal feces, fallen leaves, and branches.

[0205] In one implementable manner, the lawn image to be trained includes a color image and / or a grayscale image.

[0206] In one embodiment, in combination Figure 2 and Figure 3 , the controller 150 is electrically connected to the vision sensor 130. The controller 150 is configured to obtain the lawn image to be recognized from the vision sensor 130, and process the lawn image to be recognized by using a preset recognition model to obtain the target type of the lawn image to be recognized. Among them, the preset recognition model is obtained by training a preset model by using the lawn image to be trained and the processed lawn image; the lawn image to be trained has a labeled type; the processed lawn image is obtained by adding noise and / or interference to at least part of the lawn images to be trained.

[0207] Refer to Figure 3 , the controller 150 may include an obtaining unit 151 and a processing unit 152. Specifically, the lawn image to be recognized can be collected by the vision sensor 130, and the obtaining unit 151 in the controller 150 obtains the lawn image to be recognized from the vision sensor 130; the processing unit 152 processes the lawn image to be recognized by using a preset recognition model to obtain the target type of the lawn image to be recognized.

[0208] In this solution, by adding noise and / or interference to the lawn image to be trained, the training samples are enriched, and the recognition accuracy of the recognition model trained with the richer training samples is higher. Using the recognition model with higher recognition accuracy can improve the maintenance effect of the self-propelled lawn mower on the lawn to a certain extent.

[0209] Figure 15 is a flowchart of a lawn image processing method provided by an embodiment of the present application. This embodiment is applicable to the scenario of recognizing grass and non-grass objects in a lawn image when the self-propelled lawn mower 100 maintains the lawn. This method can be executed by the self-propelled lawn mower 100. As Figure 15 shown, this method includes:

[0210] Step 1501, obtain the lawn image to be recognized.

[0211] Step 1502: Process the lawn image to be recognized using a preset recognition model to obtain the target type of the lawn image to be recognized. Among them, the preset recognition model is obtained by training a preset model using the lawn image to be trained and the processed lawn image. The lawn image to be trained has an annotated type. The processed lawn image is obtained by adding noise and / or interference to at least part of the lawn image to be trained.

[0212] In this solution, by adding noise and / or interference to the lawn image to be trained, the training samples are enriched, and the recognition accuracy of the recognition model trained with more abundant training samples is higher. Using a recognition model with higher recognition accuracy to identify grass and non-grass objects in the lawn image can improve the maintenance effect of the self-propelled lawn mower to a certain extent.

[0213] The embodiment of the present application also specifically provides a model training method based on data augmentation for the lawn mower. Figure 16 is a flowchart of a model training method based on data augmentation provided by the embodiment of the present application. Refer to Figure 16 , and the specific process of this method is as follows:

[0214] Step 1601: Use the camera installed on the lawn mower to collect visual information data, such as color images, grayscale images, etc.

[0215] Step 1602: Screen and annotate the data. The annotation methods include but are not limited to manual annotation methods and automatic annotation methods.

[0216] Step 1603: Perform data augmentation on the annotated data. The main augmentation methods include noise augmentation and interference augmentation. The noise augmentation methods include but are not limited to blurring, distortion, overexposure, etc. The interference augmentation methods include but are not limited to adding light and shadow, solid color pixel blocks, objects that may appear on the lawn, etc.

[0217] Step 1604: Use the augmented data set after data augmentation to train a machine learning model.

[0218] Step 1605: After training, an algorithm model that can distinguish grass and non-grass areas is obtained.

[0219] Figure 2 is a schematic structural diagram of a self-propelled lawn mower 100 provided by the embodiment of the present application. Figure 17 is a schematic diagram of a reference object 200 provided by the embodiment of the present application. Refer to Figure 2 and Figure 17, an embodiment of the present application provides a self - moving lawn mower system, including: a self - moving lawn mower 100 and a reference object 200; the self - moving lawn mower 100 includes: a body 110; a walking wheel assembly 120 configured to support the body; a vision sensor 130 mounted on the body and configured to acquire images around the self - moving lawn mower; combined with Figure 2 and Figure 3 , a controller 150 is electrically connected to the vision sensor 130 and identifies semantic information around the self - moving lawn mower 100 through images; the reference object 200 is used to provide a reference for the controller, and the reference object 200 simulates at least one of a lawn, land, water surface, and sky.

[0220] Combined with Figure 2 and Figure 17 , the reference object can be installed in an area that can be captured by the vision sensor 130, such as on a wall, a tree trunk, a charging pile, etc.

[0221] Referring to Figure 3 , the controller 150 may include an acquisition unit 151 and a processing unit 152. Specifically, images around the self - moving lawn mower 100 can be collected by the vision sensor 130, and the acquisition unit 151 in the controller 150 acquires images from the vision sensor 130; the processing unit 152 acquires reference information of the reference object 200 and identifies semantic information around the self - moving lawn mower 100 through the reference information and the images. Specifically, images of the reference object 200 can be collected by the vision sensor 130, the acquisition unit 151 acquires images of the reference object 200 from the vision sensor 130, and extracts reference information of the reference object 200 from the images; the acquisition unit 151 can also acquire real information of the reference object 200; the processing unit 152 performs transformation processing on the images around the self - moving lawn mower 100 according to the real information and reference information of the reference object 200 to obtain transformed images; the transformed images are processed by a semantic recognition program to obtain semantic information around the self - moving lawn mower 100. Among them, the semantic recognition program can be a machine learning program, such as an intelligent recognition model based on machine learning; the semantic information includes at least grass, grass species, and non - grass objects, etc.

[0222] In the process of identifying semantic information around the self - moving lawn mower 100 through images in this solution, the reference information of the reference object is combined to reduce the deviation between visual information and real information on the images, thereby improving the accuracy of semantic information recognition.

[0223] In an implementable manner, referring to Figure 17 , the self - moving lawn mower system further includes a charging pile 300, and the reference object 200 is arranged on the charging pile 300.

[0224] Setting the reference object 200 at the charging pile 300 reduces the installation steps when the reference object 200 and the charging pile 300 are installed separately.

[0225] In one implementable manner, the reference object 200 has geometric features, and the geometric features are used to assist the controller 150 in identifying the reference object 200.

[0226] The processing unit 152 extracts geometric features from the images around the mobile mower 100. If the geometric features are target geometric features, the target image corresponding to the geometric features is determined as the reference object 200, and the reference information of the reference object 200 is extracted. The reference object 200 can be identified more conveniently and quickly through the geometric features. Among them, the target geometric features can be planar geometric shape features or three-dimensional geometric body features. The target geometric features include but are not limited to object features such as cubes and spheres with a certain special visual recognition structure. Specifically, at least one of the following methods can be used to extract the geometric features in the image: edge extraction and feature point extraction. Since the edge is the region with the most obvious local change in the image, the geometric features of the image can be extracted by detecting these regions. Edge extraction usually uses the extreme value region of the first derivative to detect the edge, that is, by calculating the derivative of the image function in a certain direction and finding the region where the derivative has an extreme value change, and these regions are the edges of the image. Feature point extraction refers to finding the feature points in the image, that is, those points with obvious changes or special shapes in the image. For example, corner points are the intersections of two sides in the image, and they usually have a larger curvature value than the surrounding area, so the geometric features can be extracted by detecting these points. Similarly, key points and blobs are also regions with significant features in the image, and the geometric features can be extracted through these regions. Once the geometric features in the image are detected, descriptors can be used to describe these features. The descriptors can be gray-level co-occurrence matrix, histogram of oriented gradients, scale-invariant feature transform, speeded up robust features, etc. These descriptors can be used to compare and match the feature points in the image for tasks such as image recognition, classification, and retrieval.

[0227] In one implementable manner, the controller 150 is configured to identify the reference object 200 through geometric features, extract the color features of the reference object 200, and perform transformation processing on the image according to the color features to obtain a transformed image.

[0228] The reference information provided by the reference object 200 for the controller 150 may be color features. The color features of the reference object 200 include but are not limited to colors similar to natural references such as lawn color, sky color, etc. After the processing unit 152 in the controller 150 identifies the reference object 200 through geometric features, it can extract the color features of the reference object 200; the acquisition unit 151 in the controller 150 can also acquire the true color information of the reference object 200; the processing unit 152 can obtain the difference between the extracted color features of the reference object 200 and the true color information; then subtract this difference from the color information of the image to obtain a transformed image.

[0229] Color features are important features of images. Changes in the outdoor working environment, such as changes in light, cause a deviation between the visual information and the true information on the image, mainly affecting the color features. Therefore, taking the difference between the extracted color features of the reference object 200 and the true color features as the transformation reference to transform the color features of the image can reduce the deviation between the visual information and the true information on the image, and using the transformed image for intelligent recognition can improve the recognition accuracy.

[0230] Figure 18 is a flowchart of a control method for a self-propelled lawn mower provided by an embodiment of the present application. This embodiment is applicable to the scenario where the self-propelled lawn mower 100 performs intelligent recognition on the images around the self-propelled lawn mower 100 when maintaining the lawn. This method can be executed by the self-propelled lawn mower 100, and the automatic lawn mower 100 includes a visual sensor 130, such as Figure 18 shown, and the method includes:

[0231] Step 1801, obtain an image of the reference object 200 through the visual sensor 130 and extract reference information.

[0232] Step 1802, obtain an image around the self-propelled lawn mower 100 through the visual sensor 130.

[0233] Step 1803, perform transformation processing on the image around the self-propelled lawn mower 100 according to the reference information to obtain a transformed image.

[0234] Step 1804, process the transformed image through a semantic recognition program to obtain semantic information around the self-propelled lawn mower 100.

[0235] This solution uses the reference information of the reference object to perform transformation processing on the image around the self-propelled lawn mower to obtain a transformed image, minimizing the deviation between the visual information and the true information on the image as much as possible. Furthermore, using the transformed image for intelligent recognition can improve the recognition accuracy.

[0236] In a realizable manner, the reference object is a known object in the natural environment.

[0237] The known object can be recognized by a semantic recognition program. The known object includes at least lawn, land, water surface, sky, etc.

[0238] In an implementable manner, the reference object is an artificially set simulated object.

[0239] An embodiment of the present application also provides a method for determining a feature transformation reference based on lawn markers for a lawn mower equipped with a camera. Figure 19 It is a flowchart of a method for establishing a feature transformation reference based on lawn markers provided by an embodiment of the present application. Refer to Figure 19 , and the specific process of this method is as follows:

[0240] Step 1901, the lawn mower is powered on and starts mowing the lawn.

[0241] Step 1902, during the mowing process, collect visual information on the lawn through the camera.

[0242] Step 1903, search for it in the image according to the preset geometric information of the marker.

[0243] Step 1904, extract relevant visual information from the marker in the searched image.

[0244] Step 1905, obtain a transformation method for visual information according to the preset visual information of the marker and the extracted visual information.

[0245] Step 1906, use the obtained transformation method for visual information to correct the entire image.

[0246] Step 1907, the corrected image is used as the input of the visual processing algorithm for visual algorithm processing.

[0247] Figure 20The structural schematic diagram of an electronic device 10 that can be used to implement the embodiments of the present application is shown. The electronic device 10 can be a controller in the self-propelled lawn mower 100, can be installed on the body of the self-propelled lawn mower 100, and locally communicate with the vision sensor 130 and the attitude sensor 140 in the self-propelled lawn mower 100; the electronic device 10 can also be a remote server, and remotely interact with the vision sensor 130 and the attitude sensor 140 in the self-propelled lawn mower 100. The electronic device 10 is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.). The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present application described herein and / or claimed.

[0248] As Figure 20 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0249] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0250] The processor 11 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above.

[0251] In some embodiments, any of the above methods may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of any of the methods described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute any of the above methods by any other suitable means (e.g., by means of firmware).

[0252] It should be understood that various forms of the flow shown above may be used, with steps reordered, added, or deleted. For example, the steps recited in this application may be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved, and no limitation is made herein.

[0253] The basic principles, main features, and advantages of this application have been shown and described above. Those skilled in the art of this industry should understand that the above embodiments do not limit this application in any form. Any technical solutions obtained by using equivalent replacement or equivalent transformation fall within the protection scope of this application.

Claims

1. A self - moving device, characterized in that, it includes: a fuselage, a walking wheel assembly configured to support the fuselage; an inertial measurement unit disposed within the fuselage, the inertial measurement unit including at least one of an accelerometer and a gyroscope; a controller electrically connected to the inertial measurement unit, the controller being configured to, control the self - moving device to walk; receive inertial data from the inertial measurement unit; statistically process the inertial data received within a first time window and adjust the positioning parameters of the inertial measurement unit according to the statistical result.

2. The self - moving device according to claim 1, characterized in that, the statistical result reflects the flatness of the section passed by the self - moving device within the first time window.

3. The self - moving device according to claim 1, characterized in that, the positioning parameter is the noise of the inertial data.

4. The self - moving device according to claim 1, characterized in that, the positioning parameter is the weight of the inertial data when the inertial data is fused with data of other modalities for positioning.

5. The self - moving device according to claim 1, characterized in that, the value range of the first time window is greater than or equal to a first duration threshold and less than or equal to a second duration threshold; wherein, the first duration threshold is less than the second duration threshold.

6. The self - moving device according to claim 1, characterized in that, the first time window is determined according to the walking speed of the self - moving device.

7. A positioning method for a self - moving device, characterized in that, it includes: obtaining a positioning request and obtaining the inertial data of the self - moving device within a first time window according to the positioning request; statistically processing the inertial data and adjusting the positioning parameters corresponding to the inertial data according to the statistical result; determining the position of the self - moving device according to the inertial data and the positioning parameters.

8. A positioning device for a self - moving device, characterized in that, it includes: an obtaining unit configured to obtain a positioning request and obtain the inertial data of the self - moving device within a first time window according to the positioning request; a processing unit configured to statistically process the inertial data and adjust the positioning parameters corresponding to the inertial data according to the statistical result; the processing unit is further configured to determine the position of the self - moving device according to the inertial data and the positioning parameters.

9. An electronic device, characterized in that, the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the positioning method of the self - moving device according to claim 7.

10. A computer - readable storage medium, characterized in that, the computer - readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the positioning method of the self - moving device according to claim 7 is implemented.

Images