Autonomous operation device, storage medium, program product, and control method

Abstract

Provided are an autonomous operation device (100), a storage medium, a program product, and a control method. The control method for the autonomous operation device comprises: performing an operation in a working area in a first working mode; during the operation, acquiring a recorded obstacle area and / or an area inputted by a user; determining whether the recorded obstacle area and / or the area inputted by the user meets a preset condition, and if yes, determining the recorded obstacle area and / or the area inputted by the user as a lawn maintenance area; and performing an operation in the lawn maintenance area in a second working mode. The autonomous operation device can identify a typical area requiring lawn maintenance, and use an operation strategy of a second working mode in the area to avoid damage to the area requiring lawn maintenance.

Description

An autonomous operating device, storage medium, program product and control method Technical Field

[0001] This invention relates to the field of mobile autonomous operation equipment such as intelligent lawnmowers and sweepers, and more specifically to an autonomous operation device, storage medium, program product, and control method. Background Technology

[0002] Mobile autonomous operating equipment, such as smart lawnmowers and sweepers, typically operates within a predetermined work area. This area includes the area boundaries, grassy areas, dry areas, and obstacles. During operation, the autonomous equipment avoids obstacles. For non-obstacle areas, such as hard surfaces or stone paths with a slight height difference from the lawn, the equipment maintains its operating strategy and proceeds normally to mow the lawn.

[0003] However, some areas with poor grass condition that require lawn maintenance may be identified as non-obstacle zones by autonomous mowing equipment and allowed to pass through and mow normally. This prevents the grass from recovering effectively, affecting the lawn's appearance and user experience. For example, if a statue previously stood in the lawn and was removed, the area where it once stood may become overgrown and require maintenance. However, such areas may be identified as non-obstacle zones, allowing autonomous mowing equipment to pass through and mow normally, which is detrimental to grass recovery and affects the garden's aesthetics and user experience. Summary of the Invention

[0004] To address the problem that existing technologies allow normal passage through non-obstacle areas with poor grass conditions during mowing, resulting in ineffective grass restoration in these areas and affecting lawn aesthetics and user experience, this solution proposes an autonomous operating device, storage medium, program product, and control method. This device can identify typical areas requiring lawn maintenance and employ an operating strategy that accelerates the passage of the cutter head in these areas, thus avoiding damage to the lawns requiring maintenance.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is: a control method for autonomous operating equipment, the method comprising:

[0006] Perform the work in the work area using the first working mode;

[0007] During the execution of the task, the recorded obstacle area and / or the area entered by the user are acquired;

[0008] Determine whether the recorded obstacle area and / or the user-input area meet the preset conditions. If they do, then determine that the recorded obstacle area and / or the user-input area is a grass-growing area.

[0009] The operation is carried out in the grass-growth area in the second working mode.

[0010] Furthermore, the recorded obstacle region specifically includes: calculating the three-dimensional spatial coordinates of the obstacle identified through the environmental image based on the pose information when acquiring the environmental image and the image coordinates of the obstacle, updating the map information in the pre-stored map, and recording the first area where the obstacle is located as the obstacle region.

[0011] Furthermore, determining whether the recorded obstacle area meets the preset conditions, and if so, determining the recorded obstacle area as a grass-growing area specifically includes: after completing an operation recorded as an obstacle area, during the execution of another operation, if it is detected that the recorded obstacle area meets the preset conditions, the autonomous operation equipment updates the map information and records the first area as a grass-growing area; wherein, the preset conditions are: there are no obstacles in the first area, and the first area is detected as a grass-free area.

[0012] Furthermore, the user-input area is: the second area delineated by the user on the map through the interaction module is recorded as the user-input area.

[0013] Furthermore, determining that the user-inputted area meets preset conditions and is a grass-growing area specifically includes: during the execution of the operation, if the second area is detected to be a grassless area, updating the map information and recording the second area as a grass-growing area.

[0014] Furthermore, the recorded obstacle area meets the preset conditions. Determining the recorded obstacle area as a grass-growing area specifically includes: after completing an operation recorded as an obstacle area, during the execution of another operation, if the recorded obstacle area meets the preset conditions, and when the duration of the first area being occupied by obstacles is less than a first time threshold, the first area is recorded as a first grass-growing area; when the duration of the first area being occupied by obstacles is greater than or equal to the first time threshold, the first area is recorded as a second grass-growing area.

[0015] Furthermore, determining the recorded obstacle area and / or user-input area as a grass-growing area, which meets preset conditions, specifically includes: after completing an operation in an area recorded as an obstacle area and / or user-input area, during the execution of another operation, if it is detected that the recorded obstacle area and / or user-input area meets preset conditions, and the grass condition in the first area and / or the second area is greater than or equal to a first height or density threshold, then the first area and / or the second area is determined to be a first grass-growing area; if the grass condition in the first area and / or the second area is less than the first height or density threshold, then the first area and / or the second area is determined to be a second grass-growing area.

[0016] Furthermore, the first working mode is a first travel speed and / or a first cutter head height; the second working mode is a second travel speed and / or a second cutter head height, or the second working mode is that the working path does not pass through the grass-growth area in the map; wherein, the second travel speed is greater than the first travel speed, and the second cutter head height is greater than the first cutter head height.

[0017] Furthermore, the second working mode of performing the operation in the grass-growing area specifically means: when performing the operation, when passing through the first grass-growing area on the map, passing through at a second travel speed and / or a second cutterhead height.

[0018] Furthermore, the second working mode of performing operations in the grass-growing area specifically means that when performing operations, the autonomous operating equipment does not pass through the second grass-growing area on the map.

[0019] Furthermore, determining that the recorded obstacle area meets preset conditions and is a grass-growing area specifically includes: after completing an operation recorded as an obstacle area, during the execution of another operation, if the recorded obstacle area meets preset conditions, and when the duration of the first area being occupied by obstacles is less than a first time threshold, the first area is recorded as a first grass-growing area; when the duration of the first area being occupied by obstacles is greater than or equal to the first time threshold and less than a second time threshold, the first area is recorded as a third grass-growing area; when the duration of the first area being occupied by obstacles is greater than or equal to the second time threshold, the first area is recorded as a fourth grass-growing area; wherein, the second time threshold is greater than the first time threshold.

[0020] Furthermore, determining the recorded obstacle area and / or user-input area as a grass-growing area, which meets preset conditions, specifically includes: after completing an operation recorded as an obstacle area and / or user-input area, during the subsequent operation, if the recorded obstacle area and / or user-input area meets preset conditions, and when the grass condition of the first area and / or the second area is detected to be greater than or equal to a first height or density threshold, the first area and / or the second area is determined to be a first grass-growing area; when the grass condition of the first area and / or the second area is detected to be less than the first height or density threshold, but greater than or equal to the second height or density threshold, the first area and / or the second area is recorded as a third grass-growing area; when the grass condition of the first area and / or the second area is detected to be less than the second height or density threshold, the first area and / or the second area is recorded as a fourth grass-growing area; wherein the first height or density threshold is greater than the second height or density threshold.

[0021] Furthermore, the second working mode of performing the operation in the grass-growing area specifically means: when performing the operation, for the third grass-growing area, not passing through the third grass-growing area at a predetermined time.

[0022] Furthermore, the second working mode of performing the operation in the grass-growing area specifically means: when performing the operation, for the fourth grass-growing area, not passing through the fourth grass-growing area within a predetermined time, and prompting the user to check whether replanting is needed.

[0023] Furthermore, the method also includes: detecting the grass condition of a first region and / or a second region, and adjusting the type of the first region and / or the second region based on a comparison of the grass condition with a first height or density threshold or a second height or density threshold.

[0024] Furthermore, the grass condition refers to the height of the grass or the density of the grass.

[0025] Furthermore, the process for determining the duration of the first area being occupied by an obstacle is as follows: after detecting an obstacle and recording the first area where the obstacle is located as an obstacle area, a first time point is recorded when the first area is recorded as an obstacle area; after completing an operation recorded as an obstacle area, during the process of performing another operation, when it is determined that the first area is recorded as a grass-growing area, a second time point is obtained when the first area is recorded as the grass-growing area; the duration of the first area being occupied by an obstacle is determined by the difference between the second time point and the first time point.

[0026] Furthermore, the method also includes: when the recorded obstacle area and / or the user-input area does not meet the preset conditions, determining that the second area is a grass area or an obstacle area.

[0027] Furthermore, the method also includes: receiving user input information and canceling the grass-growing area on the map based on the user input information.

[0028] Furthermore, the first working mode is a first travel speed and / or a first cutter head height; the second working mode is a second travel speed and / or a second cutter head height; wherein the second travel speed is greater than the first travel speed, and the second cutter head height is greater than the first cutter head height.

[0029] Furthermore, the method also includes: when performing the operation, if a grassless area is detected, the grassless area meets the preset area condition, and the grassless area is not identified as the grass-growth area, then the autonomous operating equipment avoids the grassless area.

[0030] Furthermore, obstacles in the obstacle region are identified using an image segmentation model or an object detection model; wherein, the image segmentation model classifies the input image pixel by pixel, marks objects at the pixel level, and outputs the category label of the object; the object detection model detects objects in the input image, marks the position of the object in the image, and the category to which the object at that position belongs.

[0031] A specific embodiment of the present invention also provides an autonomous operating device for performing the above-described control method for an autonomous operating device.

[0032] A specific embodiment of the present invention also provides a non-transitory computer-readable storage medium storing processor-executable instructions configured to cause the processor of an autonomous operating device to perform the above-described control method.

[0033] A specific embodiment of the present invention also provides a computer program product, including a computer program or instructions, wherein the computer program or instructions are executed by a processor using the control method described above. Attached Figure Description

[0034] Figure 1 is a schematic diagram of using a semantic segmentation model to segment grass in an image according to the present invention;

[0035] Figure 2 is a schematic diagram of a three-dimensional point cloud of the environment acquired by the autonomous operating device of the present invention through a depth camera;

[0036] Figure 3 is a schematic diagram of the point cloud with a certain height obtained after removing the ground plane point cloud from Figure 2;

[0037] Figure 4 is a flowchart illustrating the autonomous operation equipment control method of the present invention;

[0038] Figure 5 is a schematic diagram of the autonomous operating system of the present invention;

[0039] Figure 6 is a structural schematic diagram of the autonomous operating device of the present invention. Detailed Implementation

[0040] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings, so as to more clearly understand the purpose, features and advantages of this invention. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of this invention, but are only for illustrating the essential spirit of the technical solutions of this invention. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0041] Unless the context requires otherwise, throughout the specification and claims, the word “comprising” and its variations, such as “including” and “having”, shall be understood to have an open, inclusive meaning, that is, to be interpreted as “including, but not limited to”.

[0042] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0043] The singular forms “a” and “the” used in this specification and the appended claims include plural references unless otherwise expressly stated herein. It should be noted that the term “or” is generally used to mean “and / or” unless otherwise expressly stated herein.

[0044] In the following description, in order to clearly demonstrate the structure and working method of the present invention, a number of directional terms will be used. However, terms such as "front", "back", "left", "right", "outside", "inside", "outward", "inward", "up", and "down" should be understood as convenient terms and not as limiting terms.

[0045] The implementation details of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following content is only for the convenience of understanding the implementation details and is not necessary for implementing this solution.

[0046] It should be noted that obstacle detection can refer to the identification of target obstacles using models such as image segmentation and object detection; it can also refer to the detection of obstacles at a specific height above the ground plane using depth cameras or LiDAR. Grassless areas: Non-grass areas segmented using image segmentation models (such as semantic segmentation models). Grass-grown areas: Areas where obstacles were detected in historical tasks, but no obstacles are detected in the current task and they are identified as grassless areas; or areas set by the user on the map that are identified as grassless areas during task execution.

[0047] Image segmentation models classify input images pixel by pixel, labeling objects at the pixel level. In other words, they segment objects in an image and output their category labels. For example, as shown in Figure 1, to segment grass 103 in an image using an image segmentation model, first, grass 103 is labeled in the training image; then, the image segmentation model is used to segment the grass 103 and the background 101 (i.e., the grassless region). 102 in Figure 1 is the segmentation line.

[0048] The task of an object detection model is to detect objects in an input image, label the location of the objects in the image, and determine the category to which the object at that location belongs. For example, to detect obstacles in an image using an object detection model, obstacles are first labeled in the training image; then, the object detection model is used to detect obstacles in the image.

[0049] As shown in Figure 2, the device can acquire a 3D point cloud of the environment using a depth camera, including a height point cloud 201 and a ground plane point cloud 202. After removing the ground plane point cloud 202, a point cloud 301 with a certain height is obtained, as shown in Figure 3. The device can also use a height detection mechanism to detect obstacles in the environment. That is, by acquiring a 3D point cloud of the environment using a depth camera, if there are point clouds that are higher than a specific height above the ground plane, they are considered obstacle point clouds.

[0050] Therefore, image segmentation models can be used to segment grassy and non-grassy areas in an image; object detection models and height detection mechanisms can be used to identify obstacles in the environment. Example 1

[0051] To address the technical problem in existing technologies where autonomous mowing equipment continues to pass through and mow non-obstacle areas with poor grass conditions during operation, resulting in ineffective grass restoration in these areas and affecting lawn aesthetics and user experience, this embodiment provides an autonomous mowing equipment control method, as shown in Figure 4. The autonomous mowing equipment control method includes:

[0052] S100: Performing work in the work area in the first working mode;

[0053] S200: During the execution of the task, acquire the recorded obstacle area and / or the area entered by the user;

[0054] S300: Determine whether the recorded obstacle area and / or the user-input area meet the preset conditions. If they do, S400: Determine that the recorded obstacle area and / or the user-input area is a grass growing area; otherwise, S500: Determine that the recorded obstacle area and / or the user-input area is an obstacle area or a grassland area.

[0055] S600: Performs operations in the grass-growth area in the second working mode.

[0056] In one embodiment of this invention, the control method for the autonomous operating equipment specifically includes the following steps:

[0057] S101: If an obstacle is detected during the operation, the first area where the obstacle is located is recorded as the obstacle area.

[0058] Autonomous operating equipment is equipped with an image recognition module, a 3D perception module, or a positioning module. The image recognition module acquires environmental images and identifies obstacles using these images. The 3D perception module acquires a 3D environmental point cloud and identifies obstacles using this point cloud at a specific height above the ground plane. The positioning module acquires the autonomous operating equipment's pose information. Furthermore, the autonomous operating equipment can calculate the 3D spatial coordinates of obstacles identified in the images by combining its pose information from acquiring environmental images with the image coordinates of the obstacles.

[0059] After the autonomous operating equipment identifies an obstacle through the image recognition module, it obtains the three-dimensional spatial coordinates of the obstacle and updates the map information in the pre-stored map, recording the first area where the obstacle is located as the obstacle area for subsequent operation.

[0060] Before performing operations in the area containing the first area, the autonomous operating equipment plans the operation path according to the updated map in step S101, and the operation path avoids the first area recorded as an obstacle area.

[0061] S102: In subsequent operations, if it is detected that there are no obstacles in the first area and that the first area is a grassless area, the first area is recorded as a grass-growing area and the map information is updated.

[0062] During subsequent operations, after the initial recording of the obstacle area, when the autonomous operating equipment passes through a road segment adjacent to the first area, it identifies obstacles using an image recognition module or a 3D perception module. If no obstacles are detected in the first area, and the first area is detected as a grassless area, the autonomous operating equipment updates the map information and records the first area as a grass-growing area. In other words, the first area recorded as a grass-growing area is the area where obstacles were previously detected, but no obstacles are detected in this instance, and it is detected as a grassless area.

[0063] If obstacles are still detected in the first region, the region attributes of the first region will not be changed, that is, the first region will still be recorded as an obstacle region.

[0064] If no obstacles are detected in the first area, but the first area is detected as a grassland area, the autonomous operating equipment updates the map information, clears the obstacle area record information of the first area, and then the area attribute of the first area is the default grassland area.

[0065] S103: When the autonomous operating equipment is performing operations, it raises the cutter head and accelerates through the grass-growing area on the map.

[0066] It should be noted that step S103 has no sequential relationship with the aforementioned steps S101-S102.

[0067] When the autonomous working equipment performs its work, it raises the cutter head and accelerates as it passes through a grass-growing area on the map. Specifically, the work path is planned to cover the grass-growing area on the map. When the autonomous working equipment performs its work, it passes through the grass-growing area. The autonomous working equipment operates at a first travel speed and / or a first cutter head height within the work area. When passing through the grass-growing area, it operates at a second travel speed and / or a second cutter head height within the grass-growing area, where the second travel speed is greater than the first travel speed, and the second cutter head height is greater than the first cutter head height.

[0068] In one embodiment of this invention, the grass-growing area can also be determined based on user input. The specific steps of the autonomous operation equipment control method are as follows:

[0069] S201: Receive user input and record the grass growing area on the map based on the user input.

[0070] Specifically, unlike the method for determining the grass-growth area in steps S101-S102, the user can designate an area on the application software to specify the grass-growth area. The autonomous operating equipment records the grassless areas within the user-designated area as grass-growth areas, while the area types of the grass areas and obstacle areas within the user-designated area remain unchanged.

[0071] S202: Same as step S103.

[0072] Optionally, this implementation may also include S203: receiving user input and canceling the designated grass-growing area on the map based on the user input.

[0073] In one embodiment of this invention, to achieve more precise lawn maintenance, the autonomous operation equipment control method in this embodiment further divides the lawn maintenance area:

[0074] S301: Same as step S101.

[0075] S302: The method including step S102 further includes: determining the duration for which the first area is occupied by obstacles; and recording the first area as a first grass-growing area and a second grass-growing area based on the duration for which the first area is occupied by obstacles.

[0076] Specifically, after the autonomous operating equipment detects an obstacle and records the area containing the obstacle as an obstacle area, it also records the time point at which the first area was recorded as an obstacle area. In subsequent operations, when the autonomous operating equipment determines that the first area has been recorded as a grass-growth area, it obtains the time point at which the first area was recorded as a grass-growth area. The difference between these two time points determines the duration for which the first area was occupied by the obstacle.

[0077] When the duration of time that the first area is occupied by obstacles is less than the first time threshold, the first area is recorded as the first grass-growing area, that is, the first grass-growing area is the first area where obstacles have existed for a short period of time; when the duration of time that the first area is occupied by obstacles is greater than or equal to the first time threshold, the first area is recorded as the second grass-growing area, that is, the second grass-growing area is the first area where obstacles have existed for a long period of time.

[0078] In other embodiments, when the duration of time the first region is occupied by obstacles is less than a first time threshold, the first region is recorded as a first grass-growing region; when the duration of time the first region is occupied by obstacles is greater than or equal to the first time threshold and less than a second time threshold, the first region is recorded as a third grass-growing region, that is, the third grass-growing region is the first region where obstacles have existed for a long time but not for an extremely long time; when the duration of time the first region is occupied by obstacles is greater than or equal to the second time threshold, the first region is recorded as a fourth grass-growing region, that is, the fourth grass-growing region is the first region where obstacles have existed for a long time and for an extremely long time; wherein, the second time threshold is greater than the first time threshold.

[0079] S303: Unlike step S103, when planning the work path, the work path covers the first grass-growth area on the map. When the autonomous working equipment performs the work, it raises the cutter head and accelerates when passing through the first grass-growth area on the map. When planning the work path, it does not cover the second grass-growth area on the map, and the autonomous working equipment does not pass through the second grass-growth area on the map when performing the work. Furthermore, if a fourth grass-growth area exists, a prompt is issued to the user.

[0080] For secondary grassland restoration areas that have long been obstructed, autonomous operating equipment will not enter these areas during operations. For example, if the planned operation path covers the secondary grassland restoration area on the map, the autonomous operating equipment will not pass through the secondary grassland restoration area during operation.

[0081] Furthermore, for the third grass-growing area where obstacles have existed for a long time but not for an extremely long time, the autonomous operating equipment will not enter the area within the predetermined time. After the predetermined time, it will raise the cutter head to accelerate through the area. For the fourth grass-growing area where obstacles have existed for a long time and for an extremely long time, in addition to not entering the area within the predetermined time, the autonomous operating equipment will also prompt the user to check whether replanting is needed.

[0082] In one embodiment of this invention, the grass-growing area can also be divided based on the identified grass condition:

[0083] S401: Same as step S101.

[0084] S402: The method including step S102 further includes: detecting the grass condition in the first area, and recording the first area as the first grass-growing area and the second grass-growing area based on the grass condition in the first area.

[0085] If the autonomous operating equipment detects an obstacle in the first area during historical operations, and detects that there is no obstacle in the first area and the first area is a grassless area during the current operation, then the first area will be recorded as a grass-growing area.

[0086] The image recognition module of the autonomous operating equipment is also used to identify the sparseness of the grass, which refers to the grass density. Further, after the autonomous operating equipment detects the sparseness of the grass in the first region through the image recognition module, for example, by determining the sparseness of the grass based on the proportion of green areas in the image, the first region is recorded as the first grass-growing region and the second grass-growing region according to the sparseness of the grass.

[0087] Alternatively, the autonomous operating equipment is equipped with a 3D sensing module to acquire 3D spatial information of the environment, including grass height information, where grass condition refers to grass height information. Furthermore, after detecting the grass height information in the first region through the 3D sensing module, the autonomous operating equipment records the first region as a first grass-growing region and a second grass-growing region based on the grass height information.

[0088] In other embodiments, when the grass condition in the first region and / or the second region is detected to be greater than or equal to a first height or density threshold, the first region and / or the second region is determined to be a first grass-raising region; when the grass condition in the first region and / or the second region is detected to be less than the first height or density threshold, but greater than or equal to a second height or density threshold, the first region and / or the second region is recorded as a third grass-raising region; when the grass condition in the first region and / or the second region is detected to be less than the second height or density threshold, the first region and / or the second region is recorded as a fourth grass-raising region; wherein the first height or density threshold is greater than the second height or density threshold.

[0089] S403: Same as step S303.

[0090] Optionally, this implementation may further include S404: periodically or during each operation, detecting the grass condition of the first area, and adjusting the type of the first area based on a comparison of the grass condition with a height or density threshold. Adjusting the type of the first area includes adjusting between a first grass-growth area and a second grass-growth area, or between a first grass-growth area, a third grass-growth area, and a fourth grass-growth area, or adjusting the grass-growth area to a grassland area. For example, after a predetermined time, if the grass condition in the fourth grass-growth area is detected to be greater than a first height or density threshold, then the fourth grass-growth area is adjusted to a first grass-growth area. As another example, if, based on grass condition detection, it is determined that the grass-growth area has returned to normal grass condition, then the grass-growth area is adjusted to a grassland area.

[0091] In one embodiment of this invention, the autonomous operating equipment can also identify grassless areas that may pose a risk:

[0092] S501-S502: Same as steps S101-S102.

[0093] S503: The method including step S103 further includes: when the autonomous operating equipment is performing operations, if a grassless area is detected, and if the grassless area meets a preset area condition and the grassless area is not a grass-growing area, the autonomous operating equipment avoids the grassless area.

[0094] The working area of ​​an autonomous operating device may include grass, obstacles, and non-grass objects that the device cannot identify. Non-grass objects that the device cannot identify, such as swimming pools, will be identified as grassless areas by the device. Therefore, when the autonomous device detects a grassless area that meets a preset area condition, such as an area larger than a preset area threshold, that area may pose a safety risk. However, if the grassless area is a grass-grown area, it indicates the presence of known obstacles, and thus it can be considered a safe passage area rather than a swimming pool.

[0095] Therefore, in step S503, when the autonomous operating equipment detects a grassless area that meets a preset area condition (e.g., the area of ​​the grassless area is greater than a preset area) and is not a grass-growing area (meaning the area corresponding to the grassless area was not recorded as an obstacle area in the previous state), the autonomous operating equipment is not identified as a grass-growing area, and thus avoids the grassless area. Conversely, when the autonomous operating equipment detects a grassless area, and the area corresponding to the grassless area was recorded as an obstacle area in the previous state, the grassless area is recorded as a grass-growing area, and the autonomous operating equipment raises its cutter head to accelerate its passage. By identifying and avoiding grassless areas that may pose safety risks, operational safety is improved. By recognizing the historical obstacles in grass-growing areas, the safety of these areas can be confirmed, allowing the autonomous operating equipment to travel within them without requiring other safety mechanisms. Example 2

[0096] This embodiment relates to an autonomous operating device for executing the control method of the autonomous operating device in Embodiment 1. Since Embodiment 2 corresponds to Embodiment 1, Embodiment 2 can be implemented in conjunction with Embodiment 1. The relevant technical details mentioned in Embodiment 1 remain valid in this embodiment, and the technical effects achievable in Embodiment 1 can also be realized in this embodiment. To reduce repetition, these details will not be repeated here. Example 3

[0097] As shown in Figure 5, this embodiment relates to an autonomous operation system 1, which includes an autonomous operation device 100, a docking station 900, and a boundary 800.

[0098] The autonomous operating device 100 is, in particular, an autonomous operating device capable of moving autonomously within a preset area and performing specific tasks, typically such as a smart sweeper / vacuum cleaner performing cleaning tasks, or a smart lawnmower performing mowing tasks. The specific tasks specifically refer to tasks that treat the work surface and change its state. This invention uses a smart lawnmower as an example for detailed description. The autonomous operating device 100 can autonomously move on the surface of the work area, and in particular, as a smart lawnmower, it can autonomously perform mowing tasks on the ground. The autonomous operating device 100 includes at least a main body mechanism, a moving mechanism, a working mechanism, an energy module, a detection module, an interaction module, and a control module. The control module is used to execute the control methods described in embodiments 1 to 5.

[0099] As shown in Figure 6, the main structure typically includes a chassis 20 and a housing 10. The chassis 20 is used to install and accommodate functional mechanisms and modules such as the moving mechanism, working mechanism, energy module, detection module, interaction module, and control module. The housing 10 is typically constructed to at least partially cover the chassis 20, primarily serving to enhance the aesthetics and recognizability of the autonomous operating equipment 100. In this embodiment, the housing 10 is constructed to be able to translate and / or rotate relative to the chassis 20 under external force, and, in conjunction with an appropriate detection module, such as a Hall sensor, can further detect events such as collisions and lifting.

[0100] The mobile mechanism is configured to support the main body on the ground and drive the main body to move on the ground. It typically includes wheeled, tracked, or half-tracked mobile mechanisms and walking mobile mechanisms. In this embodiment, the mobile mechanism is a wheeled mobile mechanism, including at least one drive wheel 2001 and at least one prime mover. The prime mover is preferably an electric motor, but in other embodiments it can also be an internal combustion engine or a machine powered by other types of energy. In this embodiment, preferably, a left drive wheel, a left prime mover driving the left drive wheel, a right drive wheel, and a right prime mover driving the right drive wheel are provided. In this embodiment, the straight-line movement of the autonomous operating device 100 is achieved by the same-speed rotation of the left and right drive wheels in the same direction, and turning movement is achieved by differential rotation or opposite rotation of the left and right drive wheels in the same direction. In other embodiments, the mobile mechanism may also include a steering mechanism independent of the drive wheels and a steering prime mover independent of the prime mover. In this embodiment, the moving mechanism further includes at least one driven wheel 2002, which is typically constructed as a caster wheel. The drive wheel 2001 and the driven wheel 2002 are located at the front and rear ends of the autonomous operating device, respectively.

[0101] The working mechanism is configured to perform specific tasks and includes working parts and a prime mover that drives the working parts. For example, in a smart sweeper / vacuum cleaner, the working parts include a roller brush, a suction pipe, and a dust collection chamber; in a smart lawnmower, the working parts include cutting blades or a cutting disc, and further include other components such as a height adjustment mechanism for adjusting the mowing height to optimize or adjust the mowing effect. The prime mover is preferably an electric motor, but in other embodiments it can also be an internal combustion engine or a machine powered by other types of energy. In some other embodiments, the prime mover and the driving prime mover 110 are constructed as the same prime mover.

[0102] The energy module is configured to provide energy for the various operations of the autonomous operating device 100. In this embodiment, the energy module includes a battery and a charging connection structure, wherein the battery is preferably a rechargeable battery, and the charging connection structure is preferably a charging electrode that can be exposed to the outside of the autonomous operating device.

[0103] The detection module is constructed as at least one sensor that senses environmental parameters or its own operating parameters of the autonomous operating device 100. Typically, the detection module may include sensors related to the defined working area, such as magnetic induction, impact, ultrasonic, infrared, and radio sensors, with the sensor type corresponding to the location and number of the corresponding signal generating devices. The detection module may also include sensors related to positioning and navigation, such as GPS positioning devices, laser positioning devices, electronic compasses, accelerometers, odometers, angle sensors, and geomagnetic sensors. The detection module may also include sensors related to its own operational safety, such as obstacle sensors, lift sensors, and battery pack temperature sensors. The detection module may also include sensors related to the external environment, such as ambient temperature sensors, ambient humidity sensors, light sensors, and rain sensors.

[0104] The interaction module is configured to at least receive user-input control commands, send information that the user needs to perceive, and communicate with other systems or devices to send and receive information. In this embodiment, the interaction module includes a communication module mounted on the autonomous operating device 100 and a terminal device independent of the autonomous operating device 100, such as a mobile phone, computer, or network server. User control commands or other information can be input on the terminal device and reach the autonomous operating device 100 via wired or wireless communication modules. The terminal device can also receive information sent from the autonomous operating device 100. In other embodiments, the interaction module includes an input device mounted on the autonomous operating device 100 for receiving user-input control commands, typically such as a control panel or emergency stop button. The interaction module may also include a display screen, indicator lights, and / or a buzzer mounted on the autonomous operating device 100 to make the user perceive information through light or sound.

[0105] The control module typically includes at least one processor and at least one non-volatile memory. The memory stores pre-written computer programs or instruction sets, and the processor controls the autonomous operating device 100 to perform actions such as movement and operation according to the computer programs or instruction sets. Furthermore, the control module can also control and adjust the corresponding behavior of the autonomous operating device 100 and modify parameters in the memory based on signals from the detection module and / or user control commands.

[0106] The boundary 800 defines the working area of ​​the autonomous operating equipment system, within which the autonomous operating equipment 100 moves and operates. The boundary can be physical, typically such as a wall, fence, or railing; it can also be virtual, typically such as a virtual boundary signal emitted by a boundary signal generator, which is usually an electromagnetic or optical signal, or, for the autonomous operating equipment 100 equipped with a positioning device (such as GPS), a virtual boundary set in an electronic map, exemplarily formed by two-dimensional or three-dimensional coordinates. In this embodiment, the boundary 800 is constructed as a closed conductor electrically connected to the boundary signal generator, which is typically located within a docking station 900.

[0107] The docking station 900 is typically constructed on or within the boundary 800 to provide parking for the autonomous operating equipment 100, and in particular, to supply energy to the autonomous operating equipment 100 parked at the docking station.

[0108] Optionally, the processor connects various parts within the autonomous operating device using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by calling data stored in memory. Optionally, the processor can be implemented using at least one hardware form of Digital Signal Processing (DSP), Field Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor can integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and Neural-network Processing Unit (NPU). The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content displayed on the touchscreen; and the NPU is used to implement Artificial Intelligence (AI) functions.

[0109] The memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory may include non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the autonomous operating device, etc.

[0110] The present invention also provides a non-transitory computer-readable storage medium storing a computer program for execution by a processor to implement the control method for autonomous operating equipment as described in the above embodiments.

[0111] A specific embodiment of the present invention also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of the methods in any of the above embodiments.

[0112] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the invention by those skilled in the art without departing from the spirit and essence of the invention. Such modifications or substitutions should all fall within the scope of the invention, or any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the invention should be covered within the protection scope of the invention. Therefore, the protection scope of the invention should be determined by the scope of the claims.

Claims

1. A control method of an autonomous work apparatus, characterized by, The method includes: Perform the work in the work area using the first working mode; During the execution of the task, the recorded obstacle area and / or the area entered by the user are acquired; Determine whether the recorded obstacle area and / or the user-input area meet the preset conditions. If they do, then determine that the recorded obstacle area and / or the user-input area is a grass-growing area. The operation is carried out in the grass-growth area in the second working mode.

2. The method of claim 1, wherein, The recorded obstacle region specifically includes: calculating the three-dimensional spatial coordinates of the obstacle identified through the environmental image based on the pose information when acquiring the environmental image and the image coordinates of the obstacle, updating the map information in the pre-stored map, and recording the first area where the obstacle is located as the obstacle region.

3. The method of claim 2, wherein, The determination of whether the recorded obstacle area meets the preset conditions, and if so, the determination of the recorded obstacle area as a grass-growing area specifically includes: after completing an operation recorded as an obstacle area, during the execution of another operation, if the recorded obstacle area is detected to meet the preset conditions, the autonomous operation equipment updates the map information and records the first area as a grass-growing area; wherein, the preset conditions are: there are no obstacles in the first area, and the first area is detected as a grass-free area.

4. The method of claim 1, wherein, The area input by the user is: the second area marked by the user on the map through the interaction module is recorded as the area input by the user.

5. The method of claim 4, wherein, The user-inputted area meets the preset conditions. Determining the user-inputted area as a grass-growing area specifically includes: during the execution of the operation, if the second area is detected to be a grassless area, the map information is updated, and the second area is recorded as a grass-growing area.

6. The method of claim 3, wherein, The determination of a recorded obstacle area as a grass-growing area specifically includes: after completing an operation recorded as an obstacle area, during the subsequent operation, if the recorded obstacle area is detected to meet the preset conditions, and the duration of time the first area is occupied by obstacles is less than a first time threshold, the first area is recorded as a first grass-growing area; if the duration of time the first area is occupied by obstacles is greater than or equal to the first time threshold, the first area is recorded as a second grass-growing area.

7. The method according to claim 3 or 5, characterized in that, The determination of the recorded obstacle area and / or user-input area as a grass-growing area specifically includes: after completing an operation in an area recorded as an obstacle area and / or user-input area, during the execution of another operation, if it is detected that the recorded obstacle area and / or user-input area meets the preset conditions, and the grass condition in the first area and / or the second area is greater than or equal to a first height or density threshold, the first area and / or the second area is determined to be a first grass-growing area; if the grass condition in the first area and / or the second area is less than the first height or density threshold, the first area and / or the second area is determined to be a second grass-growing area.

8. The method of claim 7, wherein, The first working mode is a first travel speed and / or a first cutter head height; the second working mode is a second travel speed and / or a second cutter head height, or the second working mode is that the working path does not pass through the grass-growth area in the map; wherein, the second travel speed is greater than the first travel speed, and the second cutter head height is greater than the first cutter head height.

9. The method of claim 8, wherein, The operation in the grass-growing area in the second working mode specifically refers to the following: when performing the operation, when passing through the first grass-growing area on the map, the operation is carried out at a second travel speed and / or a second cutterhead height.

10. The method of claim 8, wherein, The second working mode of performing operations in the grass-growing area specifically means that when performing operations, the autonomous operating equipment does not pass through the second grass-growing area on the map.

11. The method of claim 3, wherein, The determination of a recorded obstacle area as a grass-growing area based on preset conditions specifically includes: after completing an operation recorded as an obstacle area, during the subsequent execution of an operation, if the recorded obstacle area meets the preset conditions, and when the duration of time the first area is occupied by obstacles is less than a first time threshold, the first area is recorded as a first grass-growing area; when the duration of time the first area is occupied by obstacles is greater than or equal to the first time threshold and less than a second time threshold, the first area is recorded as a third grass-growing area; when the duration of time the first area is occupied by obstacles is greater than or equal to the second time threshold, the first area is recorded as a fourth grass-growing area; wherein, the second time threshold is greater than the first time threshold.

12. The method of claim 3 or 5, wherein, The determination of a recorded obstacle area and / or user-input area as a grass-growing area specifically includes: after completing an operation that records the obstacle area and / or user-input area as an obstacle area and / or user-input area, during a subsequent operation, if the recorded obstacle area and / or user-input area are detected to meet the preset conditions, and when the grass condition of the first area and / or the second area is detected to be greater than or equal to a first height or density threshold, the first area and / or the second area are determined to be a first grass-growing area; when the grass condition of the first area and / or the second area is detected to be less than the first height or density threshold, but greater than or equal to the second height or density threshold, the first area and / or the second area are recorded as a third grass-growing area; when the grass condition of the first area and / or the second area is detected to be less than the second height or density threshold, the first area and / or the second area are recorded as a fourth grass-growing area; wherein the first height or density threshold is greater than the second height or density threshold.

13. The method of claim 12, wherein, The operation in the grass-growing area in the second working mode specifically means that, when performing the operation, for the third grass-growing area, the third grass-growing area is not passed through at a predetermined time.

14. The method of claim 12, wherein, The operation in the grass-growing area in the second working mode specifically involves: when performing the operation, for the fourth grass-growing area, not passing through the fourth grass-growing area within a predetermined time, and prompting the user to check whether replanting is needed.

15. The method of claim 12, wherein, The method further includes: detecting the grass condition of a first region and / or a second region, and adjusting the type of the first region and / or the second region based on a comparison of the grass condition with a first height or density threshold or a second height or density threshold.

16. The method of claim 15, wherein, The grass condition refers to the height or sparseness of the grass.

17. The method of claim 6, wherein, The process for determining the duration of the first area being occupied by an obstacle is as follows: after detecting an obstacle and recording the first area where the obstacle is located as an obstacle area, the first time point at which the first area is recorded as an obstacle area is recorded; after completing an operation recorded as an obstacle area, during the process of performing another operation, when it is determined that the first area is recorded as a grass-growing area, the second time point at which the first area is recorded as the grass-growing area is obtained; the duration of the first area being occupied by an obstacle is determined by the difference between the second time point and the first time point.

18. The method of claim 1, wherein, The method further includes: when the recorded obstacle area and / or the user-input area do not meet the preset conditions, determining that the second area is a grass area or an obstacle area.

19. The method of claim 1, wherein, The method further includes: receiving user input information and canceling the grass-growing area on the map based on the user input information.

20. The method of claim 1 or 3 or 5, wherein, The first working mode is a first travel speed and / or a first cutter head height; the second working mode is a second travel speed and / or a second cutter head height; wherein the second travel speed is greater than the first travel speed, and the second cutter head height is greater than the first cutter head height.

21. The method of claim 1, wherein, The method further includes: when performing the operation, if a grassless area is detected, the grassless area meets the preset area condition, and the grassless area is not identified as the grass-growth area, then the autonomous operating equipment avoids the grassless area.

22. The method of claim 1, wherein, Obstacles in the obstacle region are identified using an image segmentation model or an object detection model. The image segmentation model classifies the input image pixel by pixel, identifies objects at the pixel level, and outputs the category label of the object. The object detection model detects objects in the input image, labels the position of the object in the image, and identifies the category to which the object at that position belongs.

23. An autonomous work apparatus characterized by comprising: Used to perform the control method according to any one of claims 1 to 22.

24. A non-transitory computer-readable storage medium, characterized by, The non-transitory computer-readable storage medium stores processor-executable instructions configured to cause the processor of the autonomous operating device to perform the control method according to any one of claims 1 to 22.

25. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions are executed by the processor using the control method as described in any one of claims 1 to 22.

Images