Autonomous operation device and control method therefor, and computer-readable storage medium

Abstract

A control method for an autonomous operation device. The method comprises: when an autonomous operation device is in an operating state of traveling to a second operation area along a path, upon detecting an obstacle, controlling a passage behavior of the autonomous operation device in response to at least one of a first passage determination and a second passage determination, wherein the first passage determination refers to whether the obstacle is repeatedly detected, and the second passage determination refers to whether geometric features of the obstacle meet a preset condition. In the method, whether to activate an obstacle avoidance mechanism of the autonomous operation device is determined by means of making a determination on an obstacle on the path, thereby avoiding the technical problem of the autonomous operation device failing to pass through same due to the fact that the autonomous operation device identifies a movable door as an obstacle and thus triggers the obstacle avoidance mechanism. The present invention further relates to an autonomous operation device and a computer-readable storage medium.

Description

An autonomous operating device and its control method, and a computer-readable storage medium Technical Field

[0001] This invention relates to autonomous mobile operating equipment such as intelligent lawnmowers and sweepers, and more specifically to an autonomous operating device and its control method, as well as a computer-readable storage medium. Background Technology

[0002] Mobile autonomous operating equipment, such as lawnmowers and sweepers, sometimes needs to perform tasks in multiple work areas. When multiple work areas exist, after completing the work in one work area, the autonomous operating equipment needs to move along a pre-set path to the second work area to perform tasks.

[0003] In some scenarios, work areas may be separated by architectural elements, which may include movable doors, with pre-set passageways located in the movable door areas.

[0004] However, although the movable door can be opened as the autonomous operating equipment moves, due to the obstacle avoidance mechanism of the autonomous operating equipment, when it encounters the movable door while traveling along the preset channel, the autonomous operating equipment will identify the movable door as an obstacle and trigger the obstacle avoidance mechanism, thus preventing the autonomous operating equipment from passing through. Summary of the Invention

[0005] In view of the technical problem in the prior art that mobile autonomous operation equipment recognizes movable doors as obstacles and triggers obstacle avoidance mechanisms, thus preventing the autonomous operation equipment from passing through, the present invention provides an autonomous operation equipment and its control method, as well as a computer-readable storage medium.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A control method for an autonomous operating device, the method comprising: the autonomous operating device being in a working state of traveling along a channel to a second working area; upon detecting an obstacle, controlling the passage behavior of the autonomous operating device in response to at least one of a first passage judgment and a second passage judgment; wherein the first passage judgment is whether an obstacle is repeatedly detected, and the second passage judgment is whether the geometric features of the obstacle meet preset conditions.

[0008] On the other hand, this application also provides a control method for an autonomous operating device, the method comprising: when the autonomous operating device is traveling to the location of the passage or traveling along the passage from the first working area to the second working area, when an obstacle is detected, a third passage judgment is made based on the image features of the obstacle.

[0009] This application provides an autonomous operating device for performing the control method described above.

[0010] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the control method of the autonomous operating device described above. Attached Figure Description

[0011] Figure 1 is a flowchart of the process of acquiring historical passage obstacle images provided in an embodiment of the present invention;

[0012] Figure 2 is a flowchart of a judgment process for autonomous operating equipment to pass through an obstacle passage according to an embodiment of the present invention;

[0013] Figure 3 is a flowchart of another judgment process for autonomous operating equipment to pass through an obstacle passage provided in an embodiment of the present invention;

[0014] Figure 4 is a schematic diagram of the autonomous operating system provided in an embodiment of the present invention;

[0015] Figure 5 is a schematic diagram of the structure of the autonomous operating equipment provided in an embodiment of the present invention. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0017] It should be noted that "channel" refers to the pre-recorded channel between multiple work areas of the autonomous operating equipment; "work area boundary" can be the boundary of the work area range, obstacle boundary, partition boundary within the work area, or other boundaries used to define the work area of ​​the autonomous operating equipment.

[0018] It should also be noted that the obstacle point cloud described in each embodiment includes the location coordinate information of the obstacles. The autonomous operating device triggers the obstacle avoidance mechanism based on the obstacle point cloud location information, the autonomous operating device's own location information, and the autonomous operating device's travel path. When the obstacle avoidance mechanism is turned off, the autonomous operating device clears the already generated obstacle point cloud and stops generating new obstacle point clouds based on the environmental perception module or collision perception module. After clearing and stopping the generation of obstacle point clouds, the autonomous operating device will not trigger obstacle avoidance.

[0019] Furthermore, the application scenarios applicable to the technical solution of this application include passageway shielding devices such as spring doors, automatic sensor doors, pet doorways installed on walls or room doors, room doors, and door curtains. Specifically, passageway shielding devices can be categorized by whether they have a controller or not. Passageway shielding devices with controllers include: 1) those that can communicate with autonomous operating equipment, such as elevator doors that can be linked with autonomous operating equipment; 2) those that cannot communicate with autonomous operating equipment, such as automatic sensor doors. Passageway shielding devices without controllers include: 1) those driven by elastic elements, such as spring doors; 2) those without elastic elements, such as pet doorways, room doors, and door curtains.

[0020] It should be further noted that the application scenarios described in the embodiments of this application are: channel shields without controllers or with controllers but without communication with autonomous operating equipment. These application scenarios have common characteristics: 1) they are not controlled by the instructions of autonomous operating equipment to open or close; 2) they are detected as obstacles by autonomous operating equipment and can be opened as autonomous operating equipment moves. Example

[0021] Autonomous operating equipment, such as lawnmower robots working outdoors, requires setting up its working area before it can perform tasks. Current technologies often define this working area by establishing its boundaries. Once the working area is determined, a map is generated and saved to the autonomous operating equipment or the user control device.

[0022] For example, the autonomous operating equipment is equipped with one or more of a satellite positioning system, a visual positioning system, a laser positioning system, and an inertial positioning system to detect its position information in real time. The user controls the autonomous operating equipment to walk around the boundary of the work area via a control device. During this movement, the autonomous operating equipment records its own position information in real time, thus obtaining its trajectory along the work area boundary. After obtaining the trajectory, the trajectory is expanded according to the width of the machine body and optimized using methods such as smoothing to obtain the work area boundary. The area defined by the work area boundary is taken as the work area. After obtaining the work area boundary and / or the work area, a map is generated based on the position coordinates of the work area boundary and / or the work area, and the map is saved to the control device.

[0023] In some complex work scenarios, such as two grassy areas separated by a road in a courtyard, users may set up multiple work areas and define passageways between them. For example, the user controls an autonomous working device to move from a first work area to a second work area via a control device. The autonomous working device records its own movement trajectory based on a positioning system and uses this trajectory as the passageway between the first and second work areas.

[0024] In some other embodiments, the channel can also be obtained based on user input information on the control device. Specifically, after obtaining the first working area and the second working area, the first working area and the second working area are displayed on the control device, and the control device provides the user with a channel drawing mode, generating a channel between the first working area and the second working area based on the path drawn by the user through the control device.

[0025] During the autonomous operation phase, the equipment moves to the first work area to perform tasks based on real-time control commands from the user or control commands generated according to a preset work schedule. After completing the tasks in the first work area, it moves to the location of the passageway based on real-time control commands from the user or control commands generated according to a preset work schedule, and then moves along the passageway to the second work area to perform tasks.

[0026] In outdoor lawn mowing robots, particularly in ranch applications, the ranch is often divided into zones for easier management. These zones are separated by fences, and specific areas within the fences feature spring-loaded doors that allow passage for some animals and autonomous equipment. In these scenarios, users can define multiple work areas based on the ranch zones and establish pathways between these areas for the autonomous equipment based on the locations of the spring-loaded doors. It should be noted that the spring-loaded doors are maintained closed by spring mechanisms and can be opened by the movement or force exerted by the animals or autonomous equipment. Applications of this invention include, but are not limited to, spring-loaded doors; this is merely an example of a preferred solution.

[0027] When a user sets up a passageway between different work areas, they can control an autonomous work device to move along a spring-loaded door from a first work area to a second work area, recording the device's trajectory and generating the passageway based on that trajectory. During this process, the autonomous work device will not trigger obstacle avoidance mechanisms due to user control commands. In some other embodiments, the user can also draw a path through the area containing the spring-loaded door using a control device, generating the passageway between different work areas based on the drawn path.

[0028] During the aforementioned channel generation process, the autonomous operating equipment can successfully generate a channel to pass through the spring-loaded door. However, during the operation of the autonomous operating equipment, due to its obstacle avoidance mechanism, when encountering a spring-loaded door channel, the autonomous operating equipment will identify the spring-loaded door as an obstacle and trigger the obstacle avoidance mechanism, thus preventing the autonomous operating equipment from passing through the spring-loaded door.

[0029] To address the aforementioned issues, one possible solution is to disable the obstacle avoidance mechanism of the autonomous operating equipment when it reaches the location of the passage or while traveling along the passage to the second working area. This would prevent the autonomous operating equipment from triggering obstacle avoidance when it detects an obstacle, allowing it to continue traveling along the passage to pass through the spring door.

[0030] However, the presence of obstacles in the passageway does not necessarily mean a spring-loaded door. When dynamic obstacles such as people or animals are present, the autonomous operating equipment will also detect them through its environmental perception module. For example, the environmental perception module can be a visual sensor such as a binocular camera, structured light camera, or TOF camera, or a sensor such as LiDAR. The environmental perception module can be used to detect obstacle information in the environment and generate obstacle point clouds. If the autonomous operating equipment directly disables its obstacle avoidance mechanism, it may be unable to detect dynamic obstacles, posing a safety risk.

[0031] Therefore, in this solution, when the autonomous operating equipment travels to the location of the passage or travels along the passage to the second working area, if an obstacle is detected, the first passage judgment is made based on whether the obstacle is detected repeatedly.

[0032] In one embodiment, the first passage determination based on whether an obstacle is repeatedly detected specifically involves the autonomous operating equipment waiting for a predetermined time, determining whether the detected obstacle continues to exist during the waiting period, or determining whether the obstacle is still detected after the autonomous operating equipment has waited for the predetermined time.

[0033] In another embodiment, the first passage determination based on whether an obstacle is repeatedly detected specifically involves the autonomous operating device leaving the location of the passage and returning to the location of the passage upon reaching a first return condition, and then determining whether an obstacle is still detected. After leaving the passage, the autonomous operating device can perform other tasks in a preset work schedule to improve its working efficiency and avoid waiting in the passage. The first return condition can be a predetermined time, the completion of other tasks, or other conditions.

[0034] In the above embodiment, the autonomous operating equipment delays the detection of obstacles at the location of the passage for a period of time and performs a first passage judgment to determine whether an obstacle has been detected repeatedly. Using the first passage judgment, it can be determined whether the obstacle is a dynamic obstacle and whether the dynamic obstacle has left the passage area. Based on the first passage judgment, different passage strategies can be set for the autonomous operating equipment.

[0035] For example, if the first passage determination indicates that no obstacle has been detected after repeated detection, the autonomous operating equipment proceeds along the passage to the second working area. If the first passage determination indicates that an obstacle has still been detected after repeated detection: in some embodiments, the obstacle is identified as a spring door, and it is determined that the obstacle is pushed aside as the autonomous operating equipment moves. In this case, the autonomous operating equipment adjusts its obstacle avoidance mechanism and proceeds along the passage to the second working area. In some embodiments, adjusting the obstacle avoidance mechanism may include: for visual obstacle avoidance, i.e., obstacle avoidance based on the environmental perception module, the obstacle avoidance mechanism may be turned off; for collision obstacle avoidance, the obstacle avoidance mechanism may be turned off, or the threshold for triggering a collision may be adjusted, as long as it meets the requirement of being able to push open the door and pass through.

[0036] While traveling along the channel, multiple trajectory points are generated based on the channel path. Using PID control (Proportional-Integral-Derivative), the autonomous working equipment is controlled to sequentially reach the trajectory points not yet visited on the channel. When the machine reaches the end of the channel, completing channel passage, the autonomous working equipment's operating mode is switched according to the preset work schedule. If the autonomous working equipment's position remains unchanged for a specific time threshold, it abandons its journey along the channel to the second working area. Example

[0037] An alternative implementation of Embodiment 1 may further include, or, based on the implementation of Embodiment 1, the following: when the autonomous operating equipment travels to the location of the passage or travels along the passage to the second working area, if an obstacle is detected, a second passage judgment is made based on the geometric characteristics of the obstacle, and different passage strategies are adopted based on the second passage judgment.

[0038] Although the initial passage judgment based on whether an obstacle is repeatedly detected can be interpreted as a dynamic obstacle if no obstacle is detected at the passage location after a certain delay, allowing the autonomous equipment to safely pass through the passage once the obstacle is no longer present, it cannot be determined that the obstacle is a spring-loaded door if an obstacle is still detected after a certain delay. In this case, directly adjusting the obstacle avoidance mechanism to attempt passage may still pose a risk or cause the autonomous equipment to make ineffective passage attempts, reducing work efficiency.

[0039] Therefore, in this embodiment, when the autonomous operating equipment travels to the location of the passage or along the passage to the second working area, it makes a second passage judgment based on the geometric feature information of the obstacle if an obstacle is detected.

[0040] For example, the autonomous operating equipment is equipped with an environmental perception module, which can be a visual sensor such as a binocular camera, a structured light camera, or a TOF camera, or a sensor such as a lidar. The environmental perception module can be used to detect obstacle information in the environment and generate obstacle point clouds.

[0041] For example, the autonomous operating equipment is also equipped with a collision sensing module, which can also be used to generate obstacle point clouds based on collision information and the location information of the autonomous operating equipment.

[0042] In one embodiment, when making a second passage determination based on the geometric feature information of the obstacle, the autonomous operating device acquires the obstacle point cloud through an environmental perception module and determines whether the obstacle surface constitutes a plane using the Hough plane detection method. In other embodiments, a preset pattern can be set on the spring door or the spring door itself can be set to a preset shape. The autonomous operating device uses a vision sensor to detect the obstacle and then makes a second passage determination based on the collected geometric feature information of the obstacle.

[0043] In other embodiments, when making a second passage determination based on the geometric feature information of the obstacle, the autonomous operating device acquires an obstacle point cloud, projects the obstacle point cloud onto a two-dimensional plane, and uses the Hough line detection method to determine whether the two-dimensional projection of the obstacle point cloud constitutes a straight line. Specifically, when using the Hough line detection method to determine whether the two-dimensional projection of the obstacle point cloud constitutes a straight line, the obstacle points are converted into straight lines in the Hough parameter space, and the Hough parameter space is divided into discrete grids at equal intervals. If the intersection of the straight lines in the Hough parameter space in a certain discrete grid is greater than a certain proportional threshold, then the two-dimensional projection of the obstacle point cloud is considered to constitute a straight line, and thus the obstacle surface is considered to constitute a plane.

[0044] In the process of implementing this solution and in the inventor's creative research and development practice, the inventor discovered that when a passageway has a spring-loaded door, the spring-loaded door or the fence or wall near it often has relatively obvious planar features. This planarity is caused by the artificial architectural nature of the spring-loaded door, fence, or wall. However, other obstacles, such as people, animals, vehicles, and temporarily placed boxes, often do not have the obvious planarity of spring-loaded doors, fences, or walls, or they may not have only a single plane. Therefore, when performing planar detection based on obstacle point clouds, spring-loaded doors, fences, and walls will be detected as planar, while other irregular obstacles or multi-planar obstacles will not be detected as planar because the intersection points of the parametric space surfaces or lines of the obstacle point cloud are not concentrated in a discrete grid. Through the aforementioned second passage judgment, determining whether the obstacle surface constitutes a plane can confirm whether the obstacle on the passageway is an obstacle caused by a spring-loaded door, thereby controlling the autonomous operating equipment to adopt different passage strategies.

[0045] For example, if the second passage assessment determines that the obstacle surface forms a flat plane, the obstacle can be considered a spring-loaded door, and the autonomous operating equipment can adjust its obstacle avoidance mechanism to proceed along the passage to the second work area. If the second passage assessment determines that the obstacle surface does not form a flat plane, the obstacle can be considered not a spring-loaded door, and the autonomous operating equipment can abandon passage.

[0046] While traveling along the channel, multiple trajectory points are generated based on the channel path. Using PID control (Proportional-Integral-Derivative), the autonomous working equipment is controlled to sequentially reach the trajectory points not yet visited on the channel. When the machine reaches the end of the channel, completing channel passage, the autonomous working equipment's operating mode is switched according to the preset work schedule. If the autonomous working equipment's position remains unchanged for a specific time threshold, it abandons its journey along the channel to the second working area. Example

[0047] The alternative implementation of Embodiment 1 or Embodiment 2, or based on the implementation of Embodiment 1 and / or Embodiment 2, further includes: when the autonomous operating equipment travels to the location of the passage or travels along the passage to the second working area, if an obstacle is detected, a third passage judgment is made based on the image features of the obstacle.

[0048] In some embodiments, the autonomous operating equipment is equipped with an environmental perception module, which can be a visual sensor such as a binocular camera, a structured light camera, or a TOF camera, or a sensor such as a lidar. The environmental perception module is used to acquire environmental images.

[0049] When the autonomous working device reaches the location of the passage or travels along the passage to the second working area, if an obstacle is detected, the autonomous working device moves to a specific position near the passage, sets itself to a specific posture, and acquires an image of the obstacle in the passage. In some embodiments, the image of the obstacle in the passage refers to an image of the obstacle near the passage acquired by the autonomous working device through its own environmental perception module.

[0050] The system determines whether historical obstacle images of the passage are stored. If no historical obstacle images are stored, it determines whether to proceed along the passage to the second working area, including whether to adjust the obstacle avoidance mechanism to proceed along the passage, based at least on a first passage judgment and / or a second passage judgment. If historical obstacle images are stored, a third passage judgment is made based on the image characteristics of the obstacles. In some embodiments, historical obstacle images refer to passage obstacle images acquired when the obstacle is confirmed to be a spring door during historical passage. Specifically, the autonomous operating equipment acquires passage obstacle images through the environmental perception module, determines whether the autonomous operating equipment's passage strategy is to adjust obstacle avoidance and proceed along the passage. If it adjusts the obstacle avoidance mechanism and proceeds along the passage, and successfully passes through the passage, the current passage obstacle image is stored as a historical passage obstacle image for subsequent passage processes.

[0051] After the autonomous operating equipment acquires images of obstacles in the passage using the above method, if no historical images of obstacles in the passage are saved, it determines to adjust obstacle avoidance and proceed along the passage based on the first and / or second passage judgments. Upon successfully passing through the passage (indicating a spring-loaded door obstacle), the current obstacle image is saved as a historical image for subsequent passage processes. If it proceeds directly along the passage (indicating a dynamic obstacle that has been cleared), or fails to pass through the passage (indicating another impassable obstacle), or if a historical image of obstacles in the passage is already saved, the current obstacle image is discarded.

[0052] For example, as shown in Figure 1, the process of acquiring historical channel obstacle images is as follows:

[0053] Step 101: The autonomous operating equipment acquires images of obstacles in the passage and proceeds to the end of the passage to complete the passage according to the passage method of Embodiment 1 or Embodiment 2;

[0054] Step 102: Determine whether historical obstacle images of the passage are saved. If the result is that no historical obstacle images of the passage are saved, proceed to step 103; otherwise, proceed to step 106.

[0055] Step 103: Determine whether the autonomous operating equipment’s passage strategy is to adjust obstacle avoidance and travel along the passage. If it is to adjust the obstacle avoidance mechanism and travel along the passage, proceed to step 104; otherwise, proceed to step 106.

[0056] Step 104: Save the current obstacle image of the passage as the historical obstacle image of the passage process in the subsequent passage. According to the passage method of Embodiment 1 or Embodiment 2, if the obstacle avoidance mechanism is adjusted and passage is completed, it can be determined that the detected obstacle is a spring door.

[0057] Step 106: Discard the current obstacle image. According to the passage method of Embodiment 1 or Embodiment 2, if the obstacle avoidance mechanism is not adjusted and passage is completed, it cannot be determined that the detected obstacle is a spring door, such as a dynamic obstacle.

[0058] For example, as shown in Figure 1, the process of acquiring historical channel obstacle images further includes:

[0059] Step 105, passage failed;

[0060] Step 106: Discard the current obstacle image. According to the passage method of Embodiment 1 or Embodiment 2, if passage fails, it cannot be determined that the detected obstacle is a spring door, or for example, other impassable obstacles.

[0061] For example, third-party access determination based on obstacle image features includes extracting feature information from two frames of images based on the currently acquired obstacle image and historical obstacle images, and determining whether the two frames are similar using the feature information. In one embodiment, a color image is converted to a grayscale image through grayscale transformation, grayscale histograms of the two frames are obtained, the cross-entropy of the grayscale histograms of the two frames is calculated, and the obtained cross-entropy is compared with a specific threshold to determine whether the cross-entropy of the grayscale histograms of the two frames is less than the specific threshold, thereby determining whether the images are similar. In other embodiments, other feature information of the two frames can also be extracted and compared to determine image similarity. In another embodiment, image segmentation is performed on the two frames, and then image masking is performed to eliminate holes and gaps in the images. The number of sub-regions segmented from the two frames is determined to be the same to determine whether the images are similar. In other embodiments, other feature information of the image mask after segmentation of the two frames can also be extracted and compared to determine image similarity.

[0062] The third passage judgment determines whether the current obstacle information is similar to the historical obstacle information. The historical obstacle information is recorded when the autonomous equipment adjusted its obstacle avoidance methods and attempted passage, successfully reaching the second working area. This historical obstacle information can indicate that the obstacle is a spring-loaded door. Therefore, by judging whether the current obstacle is similar to the historical obstacle information, the third passage judgment can quickly confirm whether the currently detected obstacle is a spring-loaded door, and then set different passage strategies for the autonomous equipment.

[0063] For example, if the third passage judgment result indicates that the current obstacle image is similar to the historical obstacle image, the autonomous operating equipment adjusts its obstacle avoidance and proceeds along the passage to the second working area. If the third passage judgment result indicates that the current obstacle image is not similar to the historical obstacle image: in some embodiments, the autonomous operating equipment abandons passage; in other embodiments, the autonomous operating equipment performs a first passage judgment and / or a second passage judgment, and determines a passage strategy based on the results of the first passage judgment and / or the second passage judgment, including direct passage, passage with adjusted obstacle avoidance mechanism, or abandoning passage. Example

[0064] Based on the above three embodiments, this embodiment provides a method for determining the passage strategy of autonomous operating equipment based on a first passage judgment, a second passage judgment, a third passage judgment, or a combination thereof. However, it does not limit whether the first passage judgment, the second passage judgment, the third passage judgment, or a combination thereof constitutes a sufficient condition for determining that the autonomous operating equipment travels along the passage to the second working area.

[0065] For example, as shown in Figure 2, the process for determining whether an autonomous operating device can pass through an obstacle passage is as follows:

[0066] Step 201: The autonomous operating equipment moves to the area where the passage is located;

[0067] Step 202: Perform obstacle detection on the area where the passage is located. If an obstacle is detected, proceed to step 203; otherwise, proceed to step 206.

[0068] In step 203, when making the first passage judgment, the autonomous operating equipment delays for a period of time to detect obstacles at the location of the passage. If obstacles are still detected, proceed to step 204; otherwise, proceed to step 206.

[0069] Step 204: The autonomous operating equipment performs a second passage judgment to detect whether the point cloud on the obstacle surface forms a plane. If the point cloud on the obstacle surface forms a plane, proceed to step 205; otherwise, proceed to step 209.

[0070] Step 205: The autonomous operating equipment adjusts its obstacle avoidance mechanism;

[0071] Step 206: Proceed along the passage to the second work area;

[0072] Step 207: During the process of the autonomous operating equipment traveling along the channel to the second working area, after a set time, it is determined whether the position of the autonomous operating equipment has changed. If the position has changed, proceed to step 208; otherwise, proceed to step 209.

[0073] Step 208: The autonomous operating equipment continues to travel to the end of the passage to complete the passage;

[0074] Step 209, passage failed.

[0075] For example, as shown in Figure 3, another determination process for autonomous operating equipment to pass through an obstacle passage is as follows:

[0076] Step 301: When the autonomous operating equipment moves to the area where the passage is located;

[0077] Step 302: Perform obstacle detection on the area where the passage is located. If an obstacle is detected, proceed to step 303; otherwise, proceed to step 307.

[0078] Step 303: Obtain the image of the obstacle in the passage;

[0079] Step 304: Determine whether historical obstacle images of the passage are saved. If the determination result is that historical obstacle images of the passage exist, proceed to step 305; otherwise, proceed to step 310.

[0080] Step 305: Perform the third passage judgment. The autonomous operation equipment judges whether the current passage obstacle image is similar to the historical passage obstacle image. If the similarity judgment result indicates that the two frames are similar, proceed to step 306; otherwise, proceed to step 310.

[0081] Step 306: The autonomous operating equipment adjusts its obstacle avoidance mechanism;

[0082] Step 307: The autonomous operating equipment travels along the passage to the second working area;

[0083] Step 308: During the process of the autonomous operating equipment traveling along the channel to the second working area, after a set time, it is determined whether the position of the autonomous operating equipment has changed. If the position has changed, proceed to step 309; otherwise, proceed to step 312.

[0084] Step 309: The autonomous operating equipment continues to travel to the end of the passage to complete the passage;

[0085] Step 310: The autonomous operating equipment performs the first passage judgment: after a period of time, it judges whether there is still an obstacle. If the judgment result is that there is still an obstacle, it proceeds to step 311; otherwise, it proceeds to step 307.

[0086] Step 311: The autonomous operating equipment performs a second passage judgment: it judges whether the surface of the obstacle forms a plane. If the judgment result is that it forms a plane, proceed to step 306; otherwise, proceed to step 312.

[0087] Step 312, autonomous operation equipment failed to pass.

[0088] It should be noted that preset conditions or a set of preset conditions are set to determine whether the autonomous operating equipment should travel along the passage to the second working area. The first passage judgment, second passage judgment, and third passage judgment are used to determine whether the environment meets one or more of the preset conditions or the set of preset conditions. If the first passage judgment, second passage judgment, and third passage judgment determine that the environment meets one or more of the preset conditions or the set of preset conditions, those skilled in the art can further set other judgment conditions to determine whether the environment meets other conditions in the preset conditions or the set of preset conditions.

[0089] For example, during the first passage determination, the autonomous operating device delays the detection of obstacles at the location of the passage for a period of time. If no obstacles are detected: in some embodiments, the autonomous operating device can travel along the passage to the second working area; in other embodiments, the autonomous operating device can also perform a fourth passage determination, and if both the detection of no obstacles and a specific condition in the fourth passage determination are met, the autonomous operating device travels along the passage to the second working area. As another example, during the second passage determination, the autonomous operating device detects whether the obstacle surface point cloud forms a plane. If the obstacle surface point cloud forms a plane, in some embodiments, the autonomous operating device adjusts its obstacle avoidance mechanism and attempts to travel along the passage to the second working area; in other embodiments, the autonomous operating device can also perform a fifth passage determination, and if both the obstacle surface point cloud forms a plane and a specific condition in the fifth passage determination are met, the autonomous operating device travels along the passage to the second working area.

[0090] While traveling along the channel, multiple trajectory points are generated based on the channel path. Using PID control (Proportional-Integral-Derivative), the autonomous working equipment is controlled to sequentially reach the trajectory points not yet visited on the channel. When the machine reaches the end of the channel, completing channel passage, the autonomous working equipment's operating mode is switched according to the preset work schedule. If the autonomous working equipment's position remains unchanged for a specific time threshold, it abandons its journey along the channel to the second working area. Example

[0091] As shown in Figure 4, this embodiment relates to an autonomous operating system 1, including an autonomous operating device 100, a docking station 900, and a boundary 800. The autonomous operating device 100 is, in particular, an autonomous operating device capable of autonomously moving within a preset area and performing specific tasks, typically such as a smart sweeper / vacuum cleaner performing cleaning tasks, or a smart lawnmower performing mowing tasks. The specific tasks specifically refer to tasks that process the work surface, changing its state. This invention uses a smart lawnmower as an example for detailed description. The autonomous operating device 100 can autonomously move on the surface of the work area, and in particular, as a smart lawnmower, it can autonomously perform mowing tasks on the ground. The autonomous operating device 100 includes at least a main body mechanism, a moving mechanism, a working mechanism, an energy module, a detection module, an interaction module, and a control module. The control module is used to execute the control methods described in embodiments 1 to 4.

[0092] As shown in Figure 5, the main structure typically includes a chassis 20 and a housing 10. The chassis 20 is used to install and accommodate functional mechanisms and modules such as the moving mechanism, working mechanism, energy module, detection module, interaction module, and control module. The housing 10 is typically constructed to at least partially cover the chassis 20, primarily serving to enhance the aesthetics and recognizability of the autonomous operating equipment 100. In this embodiment, the housing 10 is constructed to be able to translate and / or rotate relative to the chassis 20 under external force, and, in conjunction with an appropriate detection module, such as a Hall sensor, can further detect events such as collisions and lifting.

[0093] The mobile mechanism is configured to support the main body on the ground and drive it to move on the ground. It typically includes wheeled, tracked, or half-tracked mobile mechanisms and walking mobile mechanisms. In this embodiment, the mobile mechanism is a wheeled mobile mechanism, including at least one drive wheel 2001 and at least one prime mover. The prime mover is preferably an electric motor, but in other embodiments it can also be an internal combustion engine or a machine powered by other types of energy. In this embodiment, preferably, a left drive wheel, a left prime mover driving the left drive wheel, a right drive wheel, and a right prime mover driving the right drive wheel are provided. In this embodiment, the straight-line movement of the autonomous operating device is achieved by the same-speed rotation of the left and right drive wheels in the same direction, and turning movement is achieved by differential speed rotation or opposite-speed rotation of the left and right drive wheels in the same direction. In other embodiments, the mobile mechanism may also include a steering mechanism independent of the drive wheels and a steering prime mover independent of the prime mover. In this embodiment, the moving mechanism further includes at least one driven wheel 2002, which is typically configured as a caster wheel. The drive wheel 2001 and the driven wheel 2002 are located at the front and rear ends of the autonomous operating device, respectively.

[0094] The working mechanism is configured to perform specific tasks and includes working parts and a prime mover that drives the working parts. For example, in a smart sweeper / vacuum cleaner, the working parts include a roller brush, a suction pipe, and a dust collection chamber; in a smart lawnmower, the working parts include cutting blades or a cutting disc, and further include other components such as a height adjustment mechanism for adjusting the mowing height to optimize or adjust the mowing effect. The prime mover is preferably an electric motor, but in other embodiments it can also be an internal combustion engine or a machine powered by other types of energy. In some other embodiments, the prime mover and the driving prime mover 110 are constructed as the same prime mover.

[0095] The energy module is configured to provide energy for the various operations of the autonomous operating device 100. In this embodiment, the energy module includes a battery and a charging connection structure, wherein the battery is preferably a rechargeable battery, and the charging connection structure is preferably a charging electrode that can be exposed to the outside of the autonomous operating device.

[0096] The detection module is constructed as at least one sensor that senses environmental parameters or its own operating parameters of the autonomous operating device 100. Typically, the detection module may include sensors related to the defined working area, such as magnetic induction, impact, ultrasonic, infrared, and radio sensors, with the sensor type corresponding to the location and number of the corresponding signal generating devices. The detection module may also include sensors related to positioning and navigation, such as GPS positioning devices, laser positioning devices, electronic compasses, accelerometers, odometers, angle sensors, and geomagnetic sensors. The detection module may also include sensors related to its own operational safety, such as obstacle sensors, lift sensors, and battery pack temperature sensors. The detection module may also include sensors related to the external environment, such as ambient temperature sensors, ambient humidity sensors, light sensors, and rain sensors.

[0097] The interaction module is configured to at least receive user-input control commands, issue information that the user needs to perceive, and communicate with other systems or devices to send and receive information. In this embodiment, the interaction module includes an input device installed on the autonomous operating device 100 for receiving user-input control commands, typically such as a control panel or emergency stop button. The interaction module also includes a display screen, indicator lights, and / or a buzzer installed on the autonomous operating device 100 to make the user perceive information through light or sound. In other embodiments, the interaction module includes a communication module installed on the autonomous operating device 100 and a terminal device independent of the autonomous operating device 100, such as a mobile phone, computer, or network server. User control commands or other information can be input on the terminal device and reach the autonomous operating device 100 via wired or wireless communication modules.

[0098] The control module typically includes at least one processor and at least one non-volatile memory. The memory stores pre-written computer programs or instruction sets, and the processor controls the autonomous operating device 100 to perform actions such as movement and operation according to the computer programs or instruction sets. Furthermore, the control module can also control and adjust the corresponding behavior of the autonomous operating device 100 and modify parameters in the memory based on signals from the detection module and / or user control commands.

[0099] The boundary 800 is used to define the working area of ​​the autonomous operating equipment system, and typically includes an outer boundary 8001 and an inner boundary 8002. The autonomous operating equipment 100 is confined to move and operate within the outer boundary 8001, outside the inner boundary 8002, or between the outer boundary 8001 and the inner boundary 8002. The boundary can be physical, typically such as a wall, fence, or railing; the boundary can also be virtual, typically such as a virtual boundary signal emitted by a boundary signal generator, which is usually an electromagnetic signal or an optical signal, or, for the autonomous operating equipment 100 equipped with a positioning device (such as GPS), a virtual boundary set in an electronic map formed by two-dimensional or three-dimensional coordinates. In this embodiment, the boundary 800 is constructed as a closed wire electrically connected to the boundary signal generator, which is typically located within the docking station 900.

[0100] The docking station 900 is typically constructed on or within the boundary 800 to provide parking for the autonomous operating equipment 100, and in particular, to supply energy to the autonomous operating equipment 100 parked at the docking station.

[0101] The autonomous operating device in this disclosure may include one or more of the following components: processor and memory.

[0102] Optionally, the processor connects various parts within the autonomous operating device using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by calling data stored in memory. Optionally, the processor can be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor can integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and Neural-network Processing Unit (NPU). The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content displayed on the touchscreen; and the NPU is used to implement Artificial Intelligence (AI) functions.

[0103] The memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory may include non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the autonomous operating device, etc.

[0104] This disclosure also provides a computer-readable storage medium storing a computer program for execution by a processor to implement the control method for an autonomous operating device as described in the above embodiments.

[0105] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the invention by those skilled in the art without departing from the spirit and essence of the invention. Such modifications or substitutions should all fall within the scope of the invention, or any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the invention should be covered within the protection scope of the invention. Therefore, the protection scope of the invention should be determined by the scope of the claims.

[0106]

Claims

1. A control method of an autonomous work apparatus, characterized by, include: The autonomous operating equipment is in the working state of traveling along the passage to the second working area. After detecting an obstacle, it controls the passage behavior of the autonomous operating equipment in response to at least one of the first passage judgment and the second passage judgment. The first passage judgment is whether an obstacle is detected repeatedly, and the second passage judgment is whether the geometric features of the obstacle meet the preset conditions.

2. The method according to claim 1, characterized in that, The first passage determination includes, The result of the first passage determination indicates that repeated detection has failed to detect any obstacles, and the autonomous operating equipment proceeds along the passage to the second working area; If the result of the first passage judgment indicates that an obstacle is still detected after repeated detection, the autonomous operating equipment adjusts the obstacle avoidance mechanism and travels along the passage to the second working area.

3. The method according to claim 1 or 2, characterized in that, The second passage determination includes, The result of the second passage determination indicates that the surface of the obstacle forms a plane, and the autonomous operating equipment adjusts the obstacle avoidance mechanism and travels along the passage to the second working area; Alternatively, if the result of the second passage determination indicates that the surface of the obstacle does not form a plane, the autonomous operating equipment will abandon passage.

4. The method according to claim 1, characterized in that, Performing the first passage determination and performing the second passage determination includes... The result of the first passage judgment indicates that the obstacle was still detected after repeated detection, and the result of the second passage judgment indicates that the surface of the obstacle forms a plane. The autonomous operating equipment adjusts the obstacle avoidance mechanism and travels along the channel to the second working area. If the result of the first passage determination indicates that the obstacle is still detected after repeated detection, and the result of the second passage determination indicates that the surface of the obstacle does not form a plane, the autonomous operating equipment abandons passage.

5. The method according to any one of claims 1 to 4, characterized in that, The first passage determination includes: the autonomous operating equipment waiting for a predetermined time, during which the autonomous operating equipment determines whether the detected obstacle continues to exist; or after waiting for the predetermined time, it determines whether the detected obstacle still exists.

6. The method according to any one of claims 1 to 4, characterized in that, The first passage determination includes: the autonomous operating equipment leaves the location of the passage, and when the first return condition is met, it returns to the location of the passage, and determines whether the detected obstacle still exists.

7. The method according to any one of claims 1 to 4, characterized in that, The second passage determination includes: the autonomous operating equipment is equipped with an environmental perception module, which is used to detect obstacle information in the environment and generate obstacle point clouds.

8. The method according to claim 7, characterized in that, After acquiring obstacle point clouds, the autonomous operating equipment uses the Hough plane detection method to determine whether the surface of the obstacle constitutes a plane.

9. The method according to claim 7, characterized in that, After acquiring obstacle point clouds, the autonomous operating equipment projects the obstacle point clouds onto a two-dimensional plane and uses the Hough line detection method to determine whether the two-dimensional projection of the obstacle point cloud forms a straight line.

10. The method according to claim 9, characterized in that, The step of determining whether the two-dimensional projection of the obstacle point cloud constitutes a straight line using the Hough line detection method includes: converting the obstacle point cloud into a straight line in Hough parameter space, dividing the straight line in Hough parameter space into discrete grids at equal intervals, and determining whether there are any intersections of the Hough parameter space straight lines in the discrete grids that are greater than a predetermined ratio threshold.

11. The method according to any one of claims 2 to 4, characterized in that, The autonomous operating equipment is equipped with an environmental perception module for detecting obstacle information in the environment and generating obstacle point clouds; the obstacle avoidance mechanism adjustment of the autonomous operating equipment includes clearing the already generated obstacle point clouds and stopping the generation of new obstacle point clouds based on the environmental perception module.

12. The method according to any one of claims 2 to 4, characterized in that, The autonomous operating equipment is equipped with a collision sensing module, which is used to generate obstacle point clouds based on collision information and the location information of the autonomous operating equipment; the obstacle avoidance mechanism adjustment of the autonomous operating equipment includes the autonomous operating equipment clearing the already generated obstacle point clouds and stopping the generation of new obstacle point clouds based on the collision sensing module.

13. The method according to any one of claims 1 to 12, characterized in that, When the autonomous operating equipment travels along the channel to the second working area, it generates multiple trajectory points based on the channel path. Through PID control, it controls the autonomous operating equipment to sequentially travel to the trajectory points on the channel that have not yet been reached.

14. The method according to any one of claims 1 to 13, characterized in that, When the autonomous operating equipment reaches the end of the passage and completes passage, it switches to the operating mode of the autonomous operating equipment according to the preset work schedule.

15. The method according to any one of claims 1 to 14, characterized in that, When the autonomous operating equipment travels along the channel to the second working area, if it is detected that the position of the autonomous operating equipment has not changed and this continues for a predetermined time threshold, it will abandon its journey along the channel to the second working area.

16. The method according to any one of claims 1, 2, and 4, characterized in that, The repeated detection of obstacles includes the detection of obstacles by the environmental perception module of the autonomous operating equipment within a predetermined time period or after the predetermined time period at the current time point.

17. The method according to claim 16, characterized in that, In response to the judgment result of at least one of the first passage judgment and the second passage judgment, the passage behavior of the autonomous working equipment is controlled, including: in response to the result of the second passage judgment indicating that the surface of the obstacle forms a plane, the autonomous working equipment adjusts the obstacle avoidance mechanism and travels along the passage towards the second working area.

18. The method according to any one of claims 2 to 4, 11, 12, and 17, characterized in that, The adjustment of the obstacle avoidance mechanism is to disable the obstacle avoidance mechanism.

19. A control method for autonomous operating equipment, characterized in that, include: When the autonomous operating equipment detects an obstacle while traveling to the location of the passage or along the passage from the first working area to the second working area, it makes a third passage judgment based on the image characteristics of the obstacle.

20. The method according to claim 19, characterized in that, The result of the third passage judgment indicates that the current passage obstacle image is similar to the historical passage obstacle image. The autonomous operation equipment adjusts the obstacle avoidance mechanism and travels along the passage to the second working area.

21. The method according to claim 19, characterized in that, The third passage judgment result indicates that the current passage obstacle image is not similar to the historical passage obstacle image, and the autonomous operation equipment abandons passage; or the third passage judgment result indicates that the current passage obstacle image is not similar to the historical passage obstacle image, and the autonomous operation equipment performs the first passage judgment and the second passage judgment, and determines the passage behavior based on the results of the first passage judgment and the second passage judgment.

22. The method according to claim 19, characterized in that, The autonomous operating equipment makes a third passage determination based on the image features of the obstacles, including: the autonomous operating equipment is equipped with an environmental perception module for acquiring images of obstacles in the passage.

23. The method according to claim 20, characterized in that, The autonomous operating equipment is equipped with an environmental perception module, which is a vision sensor. When the autonomous operating equipment moves to the location of the channel or moves along the channel from the first working area to the second working area, it uses the vision sensor to detect obstacles and acquire images of the obstacles in the current channel.

24. The method according to claim 22 or 23, characterized in that, When the autonomous operating device detects an obstacle while traveling to the location of the passage or moving along the passage from the first working area to the second working area, it moves to a predetermined position near the passage, sets itself to a certain posture, and acquires an image of the obstacle through the environmental perception module.

25. The method according to claim 23, characterized in that, When an obstacle is detected, the third passage determination based on the image features of the obstacle further includes: when an obstacle is detected, after determining that the autonomous operating equipment has stored historical channel obstacle images, performing a third passage determination based on the current channel obstacle image and the historical channel obstacle images; and controlling the passage behavior of the autonomous operating equipment in response to the determination result of the third passage determination.

26. The method according to any one of claims 19 to 25, characterized in that, Before making the third passage judgment, it is determined whether historical passage obstacle images are saved, including: if no historical passage obstacle images are saved, it is determined whether to adjust the obstacle avoidance mechanism to travel along the passage to the second working area based on the first passage judgment and / or the second passage judgment; or if historical passage obstacle images are saved, it is determined to make the third passage judgment based on the image characteristics of the obstacles.

27. The method according to claim 26, characterized in that, The process of acquiring historical channel obstacle images. This includes determining, based on the first passage judgment and / or the second passage judgment, to adjust the obstacle avoidance mechanism to proceed along the passage, and after successfully passing through the passage, saving the current passage obstacle image as a historical passage obstacle image for subsequent passage processes.

28. The method according to claim 21, 26 or 27, characterized in that, The first passage determination, which is based on whether an obstacle is repeatedly detected, includes: when the result of the first passage determination indicates that no obstacle is detected after repeated detection, the autonomous operating equipment travels along the channel to the second working area; when the result of the first passage determination indicates that an obstacle is still detected after repeated detection, the autonomous operating equipment adjusts the obstacle avoidance mechanism and travels along the channel to the second working area.

29. The method according to claim 21, 26 or 27, characterized in that, The second passage determination, which is based on the geometric feature information of the obstacle, includes: when the result of the second passage determination indicates that the surface of the obstacle forms a plane, the autonomous operating equipment adjusts the obstacle avoidance mechanism and travels along the passage to the second working area; when the result of the second passage determination indicates that the surface of the obstacle does not form a plane, the autonomous operating equipment abandons passage.

30. The method according to any one of claims 19 to 29, characterized in that, The third passage determination based on the image features of obstacles includes extracting feature information from two frames of images based on the currently acquired obstacle image and the historical obstacle image, and determining whether the two frames of images are similar based on the feature information of the two frames of images.

31. The method according to claim 30, characterized in that, The step of determining whether two frames of images are similar based on their feature information includes: obtaining the grayscale histograms of the two frames of images; calculating the cross-entropy of the grayscale histograms of the two frames of images; comparing the obtained cross-entropy with a predetermined threshold; and determining whether the cross-entropy of the grayscale histograms of the two frames of images is less than the predetermined threshold.

32. The method according to claim 30, characterized in that, The step of determining whether two frames of images are similar by using feature information of the two frames of images includes performing image segmentation on the two frames of images, performing a closing operation on the image mask to eliminate holes or gaps in the image, and determining whether the number of sub-regions segmented from the two frames of images is the same.

33. The method according to any one of claims 19 to 32, characterized in that, When traveling along the channel to the second working area, multiple trajectory points are generated according to the channel path. Through PID control, the autonomous operating equipment is controlled to sequentially go to the trajectory points on the channel that have not yet been reached.

34. The method according to any one of claims 19 to 32, characterized in that, When the autonomous operating equipment reaches the end of the passage and completes passage, it switches to the operating mode according to the preset work schedule.

35. The method according to any one of claims 19 to 32, characterized in that, When the autonomous operating equipment travels along the channel to the second working area, if it is detected that the position of the autonomous operating equipment has not changed and this continues for a predetermined time threshold, then it will abandon its journey along the channel to the second working area.

36. The method according to claim 20, characterized in that, The autonomous operating equipment is equipped with an environmental perception module for detecting obstacle information in the environment and generating obstacle point clouds; the obstacle avoidance mechanism adjustment of the autonomous operating equipment includes clearing the already generated obstacle point clouds and stopping the generation of new obstacle point clouds based on the environmental perception module.

37. The method according to any one of claims 20, 26 to 29, characterized in that, The autonomous operating equipment is equipped with a collision sensing module, which is used to generate obstacle point clouds based on collision information and the location information of the autonomous operating equipment; the obstacle avoidance mechanism adjustment of the autonomous operating equipment includes the autonomous operating equipment clearing the already generated obstacle point clouds and stopping the generation of new obstacle point clouds based on the collision sensing module.

38. The method according to any one of claims 20, 23, 25 to 29, 36, and 37, characterized in that, The adjustment of the obstacle avoidance mechanism is to disable the obstacle avoidance mechanism.

39. An autonomous operating device, characterized in that, A control method for performing the autonomous operating equipment as described in any one of claims 1 to 38.

40. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it can implement the control method of the autonomous operating equipment according to any one of claims 1 to 38.

Images