Autonomous operation equipment and control method thereof, and computer readable storage medium

Abstract

The present invention relates to an autonomous operation device and a control method thereof, and a computer-readable storage medium. The method comprises: when the autonomous operation device is moving to the location of a passage or moving from a first working area to a second working area along the passage, when an obstacle is detected, the passage behavior of the autonomous operation device is controlled in response to a passage judgment. The present invention determines whether to turn on the obstacle avoidance mechanism of the autonomous operation device by judging the passage obstacle, thereby avoiding the technical problem that the autonomous operation device cannot pass because the autonomous operation device identifies a movable door as an obstacle and triggers the obstacle avoidance mechanism.

Description

Technical Field

[0001] The present invention relates to mobile autonomous working equipment such as intelligent lawn mowers and sweepers, and in particular to an autonomous working equipment and a control method thereof, and a computer-readable storage medium. Background Art

[0002] Autonomous mobile working equipment, such as mowing robots and sweeping robots, sometimes need to perform work in multiple work areas. When there are multiple work areas, after the autonomous working equipment completes the work in one work area, it needs to go to the second work area along the preset passage to perform the work. In some scenarios, the work areas may be separated by building elements with movable doors, and the preset passage is set in the movable door area. However, although the movable door can be opened as the autonomous working equipment moves, due to the obstacle avoidance mechanism of the autonomous working equipment, when encountering a movable door while moving along the preset passage, the autonomous working equipment will identify the movable door as an obstacle and trigger the obstacle avoidance mechanism, which will prevent the autonomous working equipment from passing. Summary of the invention

[0003] In view of the technical problem in the prior art that mobile operating autonomous operating equipment identifies a movable door as an obstacle and triggers an obstacle avoidance mechanism, which causes the autonomous operating equipment to be unable to pass, the present invention provides an autonomous operating equipment and a control method thereof, and a computer-readable storage medium.

[0004] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0005] A control method for an autonomous operating device, wherein the autonomous operating device is provided with an environment perception module, wherein the environment perception module is used for obstacle detection, and the control method comprises:

[0006] When the autonomous operating equipment is in a working state of moving along the passage toward the second working area, obstacle detection is performed;

[0007] After the obstacle is detected, at least one of a first pass judgment and a second pass judgment is performed;

[0008] In response to at least one of the first passage judgment and the second passage judgment, controlling the passage behavior of the autonomous operating device;

[0009] The first pass judgment is whether an obstacle is detected repeatedly, and the second pass judgment is whether the geometric features of the obstacle meet the preset conditions;

[0010] The repeatedly detecting the obstacle includes performing obstacle detection by the environment perception module within a predetermined time at the current time point or after a predetermined time, and still detecting the obstacle;

[0011] Wherein, in response to the judgment result of at least one of the first passage judgment and the second passage judgment, the passage behavior of the autonomous operation equipment is controlled, including: in response to the result of the second passage judgment indicating that the surface of the obstacle constitutes a plane, the autonomous operation equipment turns off the obstacle avoidance mechanism and moves along the channel to the second working area.

[0012] Furthermore, the first pass determination includes:

[0013] The result of the first passage judgment indicates that no obstacle is detected after repeated detection, and the autonomous working equipment moves along the passage toward the second working area;

[0014] Or the result of the first passage judgment indicates that an obstacle is still detected after repeated detection, the autonomous operating equipment turns off the obstacle avoidance mechanism and moves along the channel to the second working area.

[0015] Furthermore, the second pass judgment further includes:

[0016] The result of the second passage judgment indicates that the surface of the obstacle does not form a plane, and the autonomous operating device gives up passing.

[0017] Further, executing the first pass judgment and the second pass judgment includes:

[0018] The result of the first pass judgment indicates that an obstacle is still detected after repeated detection, and the result of the second pass judgment indicates that the surface of the obstacle forms a plane, the autonomous working device turns off the obstacle avoidance mechanism and moves along the passage to the second working area;

[0019] Or the result of the first passage judgment indicates that an obstacle is still detected after repeated detection, and the result of the second passage judgment indicates that the surface of the obstacle does not constitute a plane, and the autonomous operating equipment gives up passing.

[0020] Furthermore, the first passage judgment includes: the autonomous operating equipment leaves the location of the passage, and when a first return condition is met, returns to the location of the passage, and judges whether the detected obstacle still exists.

[0021] Furthermore, the second passage judgment includes: generating an obstacle point cloud according to the obstacle information detected by the environment perception module.

[0022] Furthermore, after the autonomous operation equipment obtains the obstacle point cloud, it determines whether the surface of the obstacle constitutes a plane through a Hough plane detection method.

[0023] Furthermore, after the autonomous operation equipment obtains the obstacle point cloud, it projects the obstacle point cloud onto a two-dimensional plane, and determines whether the two-dimensional projection of the obstacle point cloud forms a straight line through a Hough line detection method.

[0024] Furthermore, the determining whether the two-dimensional projection of the obstacle point cloud forms a straight line by the Hough line detection method includes: converting the obstacle point cloud into a Hough parameter space straight line, dividing the Hough parameter space straight line into discrete grids at equal intervals, and determining whether there are discrete grids whose intersection points of the Hough parameter space straight lines are greater than a predetermined ratio threshold.

[0025] Furthermore, the environmental perception module generates an obstacle point cloud based on the detected obstacle information; the autonomous operation equipment shuts down the obstacle avoidance mechanism, including the autonomous operation equipment clearing the obstacle point cloud that has been generated and stopping generating new obstacle point clouds based on the environmental perception module.

[0026] Furthermore, the autonomous operating equipment is provided with a collision sensing module for generating an obstacle point cloud based on collision information and position information of the autonomous operating equipment; the autonomous operating equipment shuts down the obstacle avoidance mechanism, including the autonomous operating equipment clearing the obstacle point cloud that has been generated and stopping generating a new obstacle point cloud based on the collision sensing module.

[0027] Furthermore, when the autonomous operating equipment moves along the channel toward the second working area, a plurality of track points are generated according to the channel path, and the autonomous operating equipment is controlled to move to the track points that have not been reached on the channel in sequence through PID control.

[0028] Furthermore, when the autonomous operation equipment reaches the end of the channel and completes the passage through the channel, the working mode of the autonomous operation equipment is switched according to the preset operation schedule.

[0029] Furthermore, when the autonomous operating equipment moves along the passage to the second working area, it is detected that the position of the autonomous operating equipment has not changed and continues for a predetermined time threshold, and the autonomous operating equipment abandons moving along the passage to the second working area.

[0030] A control method for an autonomous operation device, wherein the autonomous operation device is provided with an environment perception module, wherein the environment perception module is a visual sensor, and the control method comprises:

[0031] When the autonomous operation equipment moves to the location of the passage or moves from the first working area to the second working area along the passage, the autonomous operation equipment detects obstacles through the environment perception module and obtains an image of obstacles in the current passage through the visual sensor;

[0032] When an obstacle is detected, after determining that the autonomous operation equipment has saved a historical channel obstacle image, a third passage judgment is performed based on the current channel obstacle image and the historical channel obstacle image;

[0033] In response to the judgment result of the third passing judgment, controlling the passing behavior of the autonomous operating device;

[0034] Wherein, the third passage judgment includes judging whether the current channel obstacle image is similar to the historical channel obstacle image;

[0035] Among them, in response to the judgment result of the third passage judgment, the passage behavior of the autonomous operation equipment is controlled, including: in response to the result of the third passage judgment indicating that the current channel obstacle image is similar to the historical channel obstacle image, the autonomous operation equipment turns off the obstacle avoidance mechanism and moves along the channel to the second working area.

[0036] Further, the result of the third passage judgment indicates that the current channel obstacle image is not similar to the historical channel obstacle image, and the autonomous operation equipment gives up passing; or the result of the third channel judgment indicates that the current channel obstacle image is not similar to the historical channel obstacle image, and the autonomous operation equipment performs a first passage judgment and a second passage judgment, and determines the passage behavior according to the results of the first passage judgment and the second passage judgment.

[0037] Furthermore, when the autonomous operating equipment moves to the location of the channel or moves along the channel from the first working area to the second working area, when an obstacle is detected, the autonomous operating equipment walks to a predetermined position near the channel, sets the autonomous operating equipment to a certain posture, and obtains an image containing the channel obstacle through the environmental perception module.

[0038] Furthermore, before making the third passage judgment, it is determined whether the historical channel obstacle image is saved, including: determining that the historical channel obstacle image is not saved, and determining whether to turn off the obstacle avoidance mechanism and move along the channel to the second working area based on the first passage judgment and / or the second passage judgment; or determining that the historical channel obstacle image is saved, and making the third passage judgment based on the image features of the obstacle.

[0039] Furthermore, the process of acquiring the historical channel obstacle image includes, according to the first passage judgment and / or the second passage judgment, determining to turn off the obstacle avoidance mechanism and proceed along the channel, and after successfully passing the channel, saving the current channel obstacle image as the historical channel obstacle image for the subsequent channel passing process.

[0040] Furthermore, the first passage judgment is performed based on whether an obstacle is repeatedly detected, including: when the result of the first passage judgment indicates that no obstacle is detected after repeated detection, the autonomous operation equipment moves along the channel to the second working area; when the result of the first passage judgment indicates that an obstacle is still detected after repeated detection, the autonomous operation equipment turns off the obstacle avoidance mechanism and moves along the channel to the second working area.

[0041] Furthermore, the second passage judgment is to perform a passage judgment based on the geometric feature information of the obstacle, including: when the result of the second passage judgment indicates that the surface of the obstacle constitutes a plane, the autonomous operation equipment turns off the obstacle avoidance mechanism and moves along the channel to the second working area; when the result of the second passage judgment indicates that the surface of the obstacle does not constitute a plane, the autonomous operation equipment gives up passing.

[0042] Furthermore, the third passage judgment is performed based on the current channel obstacle image and the historical channel obstacle image, including extracting feature information of the two frames of images based on the currently acquired channel obstacle image and the historical channel obstacle image, and judging whether the two frames of images are similar through the feature information of the two frames of images.

[0043] Furthermore, determining whether the two frames of images are similar is performed based on feature information of the two frames of images, including obtaining grayscale histograms of the two frames of images, calculating the cross entropy of the grayscale histograms of the two frames of images, comparing the obtained cross entropy with a predetermined threshold, and determining whether the cross entropy of the grayscale histograms of the two frames of images is less than the predetermined threshold.

[0044] Furthermore, whether the two frames of images are similar is determined based on their feature information, including segmenting the two frames of images, performing a closing operation on the image masks to eliminate holes or gaps in the images, and determining whether the number of sub-regions segmented from the two frames of images is consistent.

[0045] Furthermore, when traveling along the channel toward the second working area, a plurality of track points are generated according to the channel path, and the autonomous operating equipment is controlled to sequentially move to the track points that have not been reached on the channel by means of PID control.

[0046] Furthermore, when the autonomous operation equipment reaches the end of the channel and completes the passage through the channel, the working mode of the autonomous operation equipment is switched according to the preset operation schedule.

[0047] Furthermore, when the autonomous operating equipment moves along the passage to the second working area, if it is detected that the position of the autonomous operating equipment has not changed and continues for a predetermined time threshold, the autonomous operating equipment abandons moving along the passage to the second working area.

[0048] Furthermore, the environmental perception module generates an obstacle point cloud based on the detected obstacle information; the autonomous operation equipment shuts down the obstacle avoidance mechanism, including the autonomous operation equipment clearing the obstacle point cloud that has been generated and stopping generating new obstacle point clouds based on the environmental perception module.

[0049] Furthermore, the autonomous operating equipment is provided with a collision sensing module for generating an obstacle point cloud based on collision information and position information of the autonomous operating equipment; the autonomous operating equipment shuts down the obstacle avoidance mechanism, including the autonomous operating equipment clearing the obstacle point cloud that has been generated and stopping generating a new obstacle point cloud based on the collision sensing module.

[0050] On the other hand, a computer-readable storage medium stores a computer program, which can implement the above-mentioned control method of the autonomous working equipment when executed by a processor.

[0051] An autonomous operating device is used to execute the control method of the aforementioned autonomous operating device, or includes the aforementioned computer-readable storage medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of a process for acquiring a two-dimensional image of a historical channel obstacle provided by an embodiment of the present invention;

[0053] Figure 2 A flow chart of a process for determining whether an autonomous operating device passes through an obstacle passage provided by an embodiment of the present invention;

[0054] Figure 3 A flowchart of another process for determining an obstacle passage for an autonomous operating device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0056] It should be noted that the "channel" is a passage path between multiple work areas pre-recorded by the autonomous working equipment; the "work area boundary" can be a work area range boundary, an obstacle boundary, a partition boundary within the work area, or other boundaries used to limit the work area of ​​the autonomous working equipment.

[0057] It should also be noted that the obstacle point cloud described in each embodiment contains the position coordinate information of the obstacle, and the autonomous operation device triggers the obstacle avoidance mechanism based on the obstacle point cloud position information, the autonomous operation device's own position information, the autonomous operation device's travel path, etc. When the obstacle avoidance mechanism is turned off, the autonomous operation device clears the obstacle point cloud that has been generated and stops generating new obstacle point clouds based on the environment perception module or the collision perception module. After clearing and stopping the generation of the obstacle point cloud, the autonomous operation device will not trigger obstacle avoidance.

[0058] Example 1

[0059] For mobile autonomous working equipment, such as a lawn mowing robot working outdoors, it is necessary to set the working area of ​​the autonomous working equipment before the autonomous working equipment performs the work. In the prior art, the working area of ​​the autonomous working equipment is often determined by determining the boundary of the working area of ​​the autonomous working equipment. After the working area is determined, a map is generated according to the working area and saved to the autonomous working equipment or the user control device.

[0060] Exemplarily, the autonomous operation equipment is configured with one or more of a satellite positioning system, a visual positioning system, a laser positioning system, and an inertial positioning system for real-time detection of the position information of the autonomous operation equipment. The user controls the autonomous operation equipment to walk along the boundary of the work area through the control device. During the walking process, the autonomous operation equipment records its own position information in real time, thereby obtaining a travel trajectory along the boundary of the work area. After obtaining the travel trajectory of the autonomous operation equipment, the travel trajectory is expanded according to the width of the fuselage, and after optimization such as smoothing, the work area boundary is obtained, and the area defined by the work area boundary is used as the work area. After obtaining the work area boundary and / or the work area, a map is generated according to the position coordinates of the work area boundary and / or the work area, and the map is saved in the user control device.

[0061] In some complex work scenarios, such as two grassy areas separated by a road in a courtyard, the user may set multiple work areas and set the passage paths between the multiple work areas. For example, the user controls the autonomous work equipment to move from the first work area to the second work area through the control device, and the autonomous work equipment records its own moving track according to the positioning system, and uses the moving track as the passage between the first work area and the second work area.

[0062] In some other embodiments, the channel can also be obtained according to the input information of the user in the control device. Specifically, after the first working area and the second working area are obtained, the first working area and the second working area are displayed on the user control device, and the control device provides the user with a channel drawing mode, and generates a channel between the first working area and the second working area according to the path drawn by the user through the control device.

[0063] During the operation phase, the autonomous operation equipment goes to the first work area to perform the operation according to the user's real-time control instructions or the control instructions generated according to the preset operation schedule. After completing the operation in the first work area, it moves to the location of the passage according to the user's real-time control instructions or the control instructions generated according to the preset operation schedule, moves along the passage to the second work area, and performs the operation in the second work area.

[0064] In some pasture use scenarios, such as lawn mowing robots for outdoor operations, users divide pastures into zones for easier management. The zones are separated by fences, and movable gates are set in specific areas of the fences for some animals and robots to pass through. In these scenarios, users can determine multiple work areas based on multiple pasture zones, and set passages between different work areas for autonomous operation equipment based on the locations of movable gates.

[0065] When the user sets the passage between different working areas, the autonomous working device can be controlled to move from the first working area to the second working area along the movable door, and the moving track of the autonomous working device can be recorded, and the working area passage can be obtained according to the moving track of the autonomous working device. In this process, the autonomous working device will not trigger the obstacle avoidance mechanism due to the control of the user control instruction. In some other embodiments, the user can also draw a path through the area where the movable door is located through the control device, and generate the passage between different working areas according to the drawn path.

[0066] In the above-mentioned passage generation process, the autonomous operation equipment can successfully generate a working area passage through the movable door. However, during the operation of the autonomous operation equipment, due to the obstacle avoidance mechanism of the autonomous operation equipment, when encountering the movable door passage, the autonomous operation equipment will identify the movable door as an obstacle and trigger the obstacle avoidance mechanism, resulting in the autonomous operation equipment being unable to pass through the movable door.

[0067] To solve the above problem, a possible solution is to turn off the obstacle avoidance mechanism of the autonomous operating equipment when the autonomous operating equipment moves to the location of the passage or moves along the passage to the second working area, so that when the autonomous operating equipment detects an obstacle, it will not trigger the obstacle avoidance and continue to move along the passage to pass through the movable door.

[0068] However, obstacles on the passage path are not necessarily movable doors. When there are dynamic obstacles such as people and animals on the passage, the autonomous operation equipment will also detect the obstacles through sensors. If the autonomous operation equipment directly turns off the obstacle avoidance mechanism, it may cause the autonomous operation equipment to fail to detect dynamic obstacles, causing safety risks.

[0069] Therefore, in this solution, when the autonomous operating equipment moves to the location of the passage or moves along the passage to the second working area, if an obstacle is detected, a first passage judgment is performed based on whether the obstacle is detected repeatedly.

[0070] In one embodiment, the first passage judgment is performed based on the duration of the obstacle. Specifically, the autonomous operating device waits for a predetermined time, and determines whether the detected obstacle continues to exist during the waiting process of the autonomous operating device, or determines whether the obstacle is still detected after the autonomous operating device waits for the predetermined time.

[0071] In another embodiment, the first passage judgment is performed according to the duration of the obstacle. Specifically, the autonomous operation device leaves the location of the passage, and when the first return condition is met, it returns to the location of the passage, and determines whether the obstacle is still detected. After the autonomous operation device leaves the passage, it can execute other tasks in the preset operation schedule to improve the working efficiency of the autonomous operation device and avoid the autonomous operation device waiting in the passage. The first return condition can be a predetermined time, or it can be the completion of other tasks, or it can be other conditions.

[0072] In the above embodiment, the autonomous operation equipment delays for a period of time to detect obstacles at the location of the passage, and performs a first passage judgment to determine whether the obstacle is detected repeatedly. The first passage judgment can be used to determine whether the obstacle is a temporary obstacle and whether the temporary obstacle has left the area where the passage is located. According to the first passage judgment, the autonomous operation equipment can be set to adopt different passage strategies.

[0073] For example, if the first passage judgment determines that the repeated detection has not detected any obstacles, the autonomous operation equipment moves along the passage to the second working area. If the first passage judgment determines that the repeated detection still detects obstacles: In some embodiments, the autonomous operation equipment turns off obstacle avoidance and moves along the passage to the second working area.

[0074] When traveling along the channel, multiple track points are generated according to the channel path, and the autonomous operation equipment is controlled to go to the track points that have not been traveled to on the channel in sequence through PID control, where PID is the abbreviation of Proportional-Integral-Derivative. When the machine travels to the end of the channel and completes the channel passage, the autonomous operation equipment working mode is switched according to the preset operation schedule; when it is detected that the position of the autonomous operation equipment has not changed and continues for a specific time threshold, it abandons traveling along the channel to the second working area.

[0075] Example 2

[0076] An alternative implementation method to Example 1 is, or in addition to the implementation method of Example 1, further comprising: when the autonomous operating equipment moves to the location of the channel or moves along the channel to the second working area, if an obstacle is detected, a second passage judgment is performed based on the geometric features of the obstacle, and different passage strategies are adopted based on the second passage judgment.

[0077] Although in the first passage judgment based on the duration of the obstacle, if it is detected that there is no obstacle at the location of the passage after a delay, it can be judged that the obstacle is a temporary obstacle, and the autonomous operation equipment can safely pass through the passage when the obstacle no longer exists. However, if the obstacle is still detected after a delay, it cannot be determined that the obstacle is definitely a movable door. At this time, directly turning off the obstacle avoidance mechanism and trying to pass through the passage may still cause risks, or cause the autonomous operation equipment to make invalid passage attempts, reducing work efficiency.

[0078] Therefore, in this embodiment, when the autonomous operating equipment moves to the location of the channel or moves along the channel to the second working area, if an obstacle is detected, a second passage judgment is performed based on the geometric feature information of the obstacle.

[0079] Exemplarily, the autonomous operation equipment is provided with an environment perception module, which may be a sensor such as a binocular camera, a structured light camera, a TOF camera, a lidar, etc. The environment perception module may be used to detect obstacle information in the environment and generate an obstacle point cloud.

[0080] Exemplarily, the autonomous operating equipment is also provided with a collision sensing module, and the collision sensing module can also be used to generate an obstacle point cloud according to the collision information and the position information of the autonomous operating equipment.

[0081] In one embodiment, when performing the second pass judgment based on the geometric feature information of the obstacle, the autonomous operation equipment obtains the obstacle point cloud through the environment perception module, and determines whether the obstacle surface constitutes a plane through the Hough plane detection method.

[0082] In other embodiments, when the second pass judgment is performed based on the geometric feature information of the obstacle, after the autonomous operation equipment obtains the obstacle point cloud, the obstacle point cloud is projected onto a two-dimensional plane, and the Hough line detection method is used to determine whether the two-dimensional projection of the obstacle point cloud forms a straight line. Specifically, when the Hough line detection method is used to determine whether the two-dimensional projection of the obstacle point cloud forms a straight line, the obstacle points are converted into Hough parameter space lines, and the Hough parameter space is divided into discrete grids at equal intervals. If the intersection of the Hough parameter space lines in a discrete grid is greater than a specific ratio threshold, it is considered that the two-dimensional projection of the obstacle point cloud forms a straight line, and then it is considered that the obstacle surface forms a plane.

[0083] In the process of implementing this solution and the inventor's creative research and development time, the inventor found that when there is a movable door in the passage, the movable door or the fence or wall near the movable door often has a more obvious plane feature. Such planarity is caused by the artificial construction nature of the movable door, fence, and wall. In the case of other obstacles, such as people, animals, vehicles, temporarily placed boxes, etc., they often do not have obvious planarity like movable doors, fences, walls, etc., or they do not have only a single plane. Therefore, when performing plane detection based on the obstacle point cloud, movable doors, fences, walls, etc. will be detected as planes, while other irregular obstacles or multi-plane obstacles will not be detected as planes because the parameter space surfaces or straight line intersections of the obstacle point cloud are not concentrated in a discrete grid. Through the above-mentioned second passage judgment, it is determined whether the obstacle surface constitutes a plane, and it can be confirmed whether the obstacle on the passage is an obstacle caused by the movable door, and then the autonomous operation equipment is controlled to adopt different passage strategies.

[0084] For example, if the second passage judgment determines that the obstacle surface forms a plane, the obstacle can be considered to be a movable door obstacle, and the autonomous operation equipment is set to close the obstacle avoidance and go to the second working area along the passage. If the second passage judgment determines that the obstacle surface does not form a plane, the obstacle can be considered not to be a movable door obstacle, and the autonomous operation equipment is set to give up passage.

[0085] When traveling along the channel, multiple track points are generated according to the channel path, and the autonomous operation equipment is controlled to go to the track points that have not been traveled to on the channel in sequence through PID control, where PID is the abbreviation of Proportional-Integral-Derivative. When the machine travels to the end of the channel and completes the channel passage, the autonomous operation equipment working mode is switched according to the preset operation schedule; when it is detected that the position of the autonomous operation equipment has not changed and continues for a specific time threshold, it abandons traveling along the channel to the second working area.

[0086] Example 3

[0087] An alternative implementation method to Embodiment 1 or Embodiment 2, or an implementation method based on Embodiment 1 and / or Embodiment 2 also includes: when the autonomous operating equipment moves to the location of the channel or moves along the channel to the second working area, if an obstacle is detected, a third passage judgment is performed based on the two-dimensional image features of the obstacle.

[0088] In some embodiments, the autonomous operation equipment is provided with an environment perception module, which may be a sensor such as a binocular camera, a structured light camera, a TOF camera, a lidar, etc. The environment perception module is used to obtain a two-dimensional image of the environment.

[0089] When the autonomous operating equipment moves to the location of the channel or moves along the channel to the second working area, if an obstacle is detected, the autonomous operating equipment moves to a specific position near the channel, sets the autonomous operating equipment to a specific posture, and obtains a two-dimensional image containing the channel obstacle.

[0090] Determine whether a historical two-dimensional image of a channel obstacle is saved. If there is no historical two-dimensional image of a channel obstacle, determine whether to proceed along the channel to the second working area based on at least the first passage judgment and / or the second passage judgment, including whether to turn off the obstacle avoidance mechanism and proceed along the channel. If a historical channel obstacle image is saved, make a third passage judgment based on the two-dimensional image features of the obstacle.

[0091] After the autonomous operation equipment obtains the two-dimensional image of the passage obstacle in the above manner, if there is no historical passage obstacle image saved, it determines to close the obstacle avoidance and move along the passage according to the first passage judgment and / or the second passage judgment, and after successfully passing the passage (indicating that it is a movable door obstacle), it saves the current passage obstacle two-dimensional image as the historical passage obstacle two-dimensional image for the subsequent passage process. If it moves directly along the passage (indicating that it is a temporary obstacle and the temporary obstacle has left), or fails to pass the passage (indicating that it is other inaccessible obstacles), or if the historical passage obstacle two-dimensional image has been saved, the current passage obstacle two-dimensional image is discarded.

[0092] For example, Figure 1 As shown, the process of obtaining the two-dimensional image of the historical channel obstacles is as follows:

[0093] Step 101, the autonomous operation equipment obtains a two-dimensional image of the channel obstacle, and moves to the end of the channel to complete the passage according to the passage method of Example 1 or Example 2;

[0094] Step 102, determining whether a two-dimensional image of a historical passage obstacle is saved, if the determination result is that a two-dimensional image of a historical passage obstacle is not saved, proceeding to step 103, otherwise, proceeding to step 106;

[0095] Step 103, determining whether the autonomous operation equipment's passage strategy is to turn off the obstacle avoidance mechanism and move along the channel, if so, proceed to step 104, otherwise, go to step 106;

[0096] Step 104, saving the current channel obstacle two-dimensional image as a historical channel obstacle two-dimensional image for the subsequent passage process. According to the passage method of embodiment 1 or embodiment 2, if the obstacle avoidance mechanism is turned off and the passage is completed, it can be determined that the detected obstacle is a movable door obstacle;

[0097] Step 106, discarding the two-dimensional image of the current passage obstacle. According to the passage method of Example 1 or Example 2, unless the obstacle avoidance mechanism is turned off and the passage is completed, it cannot be determined that the detected obstacle is a movable door obstacle, such as a temporary obstacle.

[0098] For example, Figure 1 As shown, the process of obtaining the two-dimensional image of the historical channel obstacle also includes:

[0099] Step 105, the passage fails;

[0100] Step 106, discarding the current passage obstacle two-dimensional image. According to the passage method of embodiment 1 or embodiment 2, if passage fails, it cannot be determined that the detected obstacle is a movable door obstacle, for example, it is other inaccessible obstacles.

[0101] Exemplarily, the third passage judgment based on the two-dimensional image features of the obstacle includes extracting feature information of two frames of two-dimensional images based on the currently acquired two-dimensional image of the channel obstacle and the historical two-dimensional image of the channel obstacle, and judging whether the two frames of images are similar through the feature information of the two frames of images. In one embodiment, the color image is converted into a grayscale image through grayscale transformation, the grayscale histograms of the two frames of images are obtained, the cross entropy of the grayscale histograms of the two frames of images is calculated, the obtained cross entropy is compared with a specific threshold, and it is judged whether the cross entropy of the grayscale histograms of the two frames of images is less than the specific threshold to judge whether the images are similar. In other embodiments, other feature information of the two frames of images can also be extracted for comparison to judge the image similarity. In another embodiment, the two frames of images are segmented, and then the image mask is closed to eliminate the holes and gaps in the image, and it is judged whether the number of sub-regions segmented from the two frames of images is consistent to judge whether the images are similar. In other embodiments, other feature information of the image mask after the two frames of images are segmented can also be extracted for comparison to judge the image similarity.

[0102] Through the above-mentioned third passage judgment, it is judged whether the current channel obstacle information is similar to the historical channel obstacle information. Among them, the historical channel obstacle information is the channel obstacle information recorded when the autonomous operation equipment turns off obstacle avoidance and attempts to pass through the historical passage, and successfully walks along the passage to the second working area. The historical channel obstacle information can indicate that the channel obstacle is a movable door obstacle. Therefore, the third passage judgment judges whether the current obstacle is similar to the historical channel obstacle information, and can quickly confirm whether the currently detected obstacle is a movable door obstacle, and then set the autonomous operation equipment to adopt different passage strategies.

[0103] For example, when the third passage judgment result indicates that the current passage obstacle image is similar to the historical passage obstacle image, the autonomous operation device turns off the obstacle avoidance mechanism and moves along the passage to the second working area. When the third passage judgment result indicates that the current passage obstacle image is not similar to the historical passage obstacle image: in some embodiments, the autonomous operation device abandons passage; in other embodiments, the autonomous operation device performs the first passage judgment and / or the second passage judgment, and determines the passage strategy according to the results of the first passage judgment and / or the second passage judgment, including direct passage, passage with the obstacle avoidance mechanism turned off, and abandonment of passage.

[0104] Example 4

[0105] Based on the implementation methods of the above three embodiments, this embodiment provides a method for determining the passage strategy of the autonomous operating device according to the first passage judgment, the second passage judgment, the third passage judgment or a combination thereof. However, it does not limit whether the first passage judgment, the second passage judgment, the third passage judgment or a combination thereof constitutes a sufficient condition for determining that the autonomous operating device moves along the passage to the second working area.

[0106] For example, Figure 2 As shown in the figure, the judgment process of the autonomous operation equipment passing through the obstacle channel is as follows:

[0107] Step 201, the autonomous operation equipment moves to the area where the channel is located;

[0108] Step 202, performing obstacle detection on the area where the passage is located, and if an obstacle is detected, proceeding to step 203, otherwise, proceeding to step 206;

[0109] Step 203, when making the first passage judgment, the autonomous operation equipment delays for a period of time to detect obstacles at the location of the passage. If it is detected that there are still obstacles, it proceeds to step 204, otherwise it goes to step 206;

[0110] Step 204, the autonomous operation device performs a second pass judgment to detect whether the obstacle surface point cloud forms a plane. If it is detected that the obstacle surface point cloud forms a plane, the process proceeds to step 205, otherwise, the process proceeds to step 209;

[0111] Step 205, the autonomous operation device turns off the obstacle avoidance mechanism;

[0112] Step 206, moving along the passage to the second working area;

[0113] Step 207, when the autonomous operation equipment moves along the passage to the second working area, it is determined whether the position of the autonomous operation equipment changes after a set time. If the position changes, the process proceeds to step 208, otherwise, the process proceeds to step 209;

[0114] Step 208, the autonomous operation equipment continues to move to the end of the channel to complete the passage;

[0115] Step 209, passage failed.

[0116] For example, Figure 3 As shown, another judgment process of the autonomous operation equipment passing through the obstacle channel is as follows:

[0117] Step 301, when the autonomous operation equipment moves to the area where the passage is located;

[0118] Step 302, performing obstacle detection on the area where the passage is located, and if an obstacle is detected, proceed to step 303, otherwise go to step 307;

[0119] Step 303, obtaining a two-dimensional image of the channel obstacle;

[0120] Step 304, determining whether a historical channel obstacle two-dimensional image is saved, if the determination result is that a historical channel obstacle two-dimensional image exists, proceeding to step 305, otherwise, turning to step 310;

[0121] Step 305, performing a third pass judgment, the autonomous operation device judges whether the current channel obstacle two-dimensional image is similar to the historical channel obstacle two-dimensional image, and if the similarity judgment result indicates that the two frames of images are similar, proceed to step 306, otherwise go to step 310;

[0122] Step 306, the autonomous operation device turns off the obstacle avoidance mechanism;

[0123] Step 307, the autonomous operation equipment moves along the passage to the second working area;

[0124] Step 308, when the autonomous operation equipment moves along the passage to the second working area, it is determined whether the position of the autonomous operation equipment changes after a set time. If the position changes, the process proceeds to step 309, otherwise, the process proceeds to step 312;

[0125] Step 309, the autonomous operation equipment continues to move to the end of the channel to complete the passage;

[0126] Step 310, the autonomous operation equipment performs a first pass judgment: after a delay, it is judged whether there is still an obstacle. If the judgment result is that there is still an obstacle, it proceeds to step 311, otherwise it goes to step 307;

[0127] Step 311, the autonomous operation device performs a second pass judgment: judging whether the obstacle surface forms a plane, if the judgment result is that it forms a plane, proceed to step 306, otherwise go to step 312;

[0128] Step 312: The autonomous operation equipment fails to pass.

[0129] It should be noted that a preset condition or a set of preset conditions is set to determine whether the autonomous operating equipment moves along the passage to the second working area. The first pass judgment, the second pass judgment, and the third pass judgment are used to determine whether the environment meets one or more conditions in the preset condition or the set of preset conditions. In the case where the first pass judgment, the second pass judgment, and the third pass judgment determine that the environment meets one or more conditions in the preset condition or the set of preset conditions, those skilled in the art may further set other pass judgment conditions to determine whether the environment meets the preset condition or other conditions in the set of preset conditions.

[0130] For example, when making the first passage judgment, the autonomous operation equipment delays for a period of time to detect obstacles at the location of the passage. Under the condition that no obstacles are detected: in some embodiments, the autonomous operation equipment can move along the passage to the second working area; in other embodiments, the autonomous operation equipment can also make a fourth passage judgment. Under the condition that no obstacles are detected and the specific conditions in the fourth passage judgment are met at the same time, the autonomous operation equipment moves along the passage to the second working area. For another example, when making the second passage judgment, the autonomous operation equipment detects whether the point cloud of the obstacle surface forms a plane. Under the condition that the point cloud of the obstacle surface forms a plane: in some embodiments, the autonomous operation equipment turns off the obstacle avoidance mechanism and tries to move along the passage to the second working area; in other embodiments, the autonomous operation equipment can also make a fifth passage judgment. Under the condition that the point cloud of the obstacle surface forms a plane and the specific conditions in the fifth passage judgment are met at the same time, the autonomous operation equipment moves along the passage to the second working area.

[0131] When traveling along the channel, multiple track points are generated according to the channel path, and the autonomous operation equipment is controlled to go to the track points that have not been traveled to on the channel in sequence through PID control, where PID is the abbreviation of Proportional-Integral-Derivative. When the machine travels to the end of the channel and completes the channel passage, the autonomous operation equipment working mode is switched according to the preset operation schedule; when it is detected that the position of the autonomous operation equipment has not changed and continues for a specific time threshold, it abandons traveling along the channel to the second working area.

[0132] The autonomous working equipment in the present disclosure may include one or more of the following components: a processor and a memory.

[0133] Optionally, the processor uses various interfaces and lines to connect various parts within the entire autonomous operation equipment, and executes various functions of the autonomous operation equipment and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a neural network processing unit (NPU). Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content that needs to be displayed on the touch display; and the NPU is used to implement artificial intelligence (AI) functions.

[0134] The memory may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing each of the above method embodiments, etc.; the data storage area may store data created according to the use of the autonomous operation device, etc.

[0135] The present disclosure also provides a computer-readable storage medium storing a computer program, wherein the computer program is used to be executed by a processor to implement the control method of the autonomous working equipment as described in the above embodiment.

[0136] Although the present invention has been described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention or any person skilled in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A control method for an autonomous operating device, wherein the autonomous operating device is provided with an environment perception module, and the environment perception module is used for obstacle detection, characterized in that: The control method comprises: When the autonomous operating equipment is in a working state of moving along the passage toward the second working area, obstacle detection is performed; After the obstacle is detected, at least one of a first pass judgment and a second pass judgment is performed; In response to at least one of the first passage judgment and the second passage judgment, controlling the passage behavior of the autonomous operating device; The first pass judgment is whether an obstacle is detected repeatedly, and the second pass judgment is whether the geometric features of the obstacle meet the preset conditions; The repeatedly detecting the obstacle includes performing obstacle detection by the environment perception module within a predetermined time at the current time point or after a predetermined time, and still detecting the obstacle; Wherein, in response to at least one of the first passage judgment and the second passage judgment, controlling the passage behavior of the autonomous operation device includes: in response to the result of the second passage judgment indicating that the obstacle surface forms a plane, the autonomous operation device turns off the obstacle avoidance mechanism and moves along the passage to the second working area; Among them, executing the first passage judgment and the second passage judgment includes: the result of the first passage judgment indicates that an obstacle is still detected after repeated detection, and the result of the second passage judgment indicates that the surface of the obstacle constitutes a plane, and the autonomous operation equipment turns off the obstacle avoidance mechanism and moves along the channel to the second working area; or the result of the first passage judgment indicates that an obstacle is still detected after repeated detection, and the result of the second passage judgment indicates that the surface of the obstacle does not constitute a plane, and the autonomous operation equipment gives up passing.

2. The method according to claim 1, characterized in that The first pass judgment includes: The result of the first passage judgment indicates that no obstacle is detected after repeated detection, and the autonomous working equipment moves along the passage toward the second working area; Or the result of the first passage judgment indicates that an obstacle is still detected after repeated detection, the autonomous operating equipment turns off the obstacle avoidance mechanism and moves along the channel to the second working area.

3. The method according to claim 1, characterized in that The second pass judgment also includes: The result of the second passage judgment indicates that the surface of the obstacle does not form a plane, and the autonomous operating device gives up passing.

4. The method according to any one of claims 1 to 3, characterized in that: The first passage judgment includes: the autonomous operating equipment leaves the location of the passage, and when the first return condition is met, returns to the location of the passage, and judges whether the detected obstacle still exists.

5. The method according to any one of claims 1 to 3, characterized in that: The second passage judgment includes: generating an obstacle point cloud according to the obstacle information detected by the environment perception module.

6. The method according to claim 5, characterized in that After the autonomous operation equipment obtains the obstacle point cloud, it determines whether the surface of the obstacle forms a plane through the Hough plane detection method.

7. The method according to claim 5, characterized in that After acquiring the obstacle point cloud, the autonomous operation equipment projects the obstacle point cloud onto a two-dimensional plane, and determines whether the two-dimensional projection of the obstacle point cloud forms a straight line through the Hough line detection method.

8. The method according to claim 7, characterized in that The method of using the Hough line detection method to determine whether the two-dimensional projection of the obstacle point cloud forms a straight line includes: converting the obstacle point cloud into a Hough parameter space straight line, dividing the Hough parameter space straight line into discrete grids at equal intervals, and determining whether there are discrete grids whose intersection points of the Hough parameter space straight lines are greater than a predetermined ratio threshold.

9. The method according to any one of claims 1 to 3, characterized in that: The environmental perception module generates an obstacle point cloud according to the detected obstacle information; the autonomous operation equipment shuts down the obstacle avoidance mechanism, including that the autonomous operation equipment clears the obstacle point cloud that has been generated and stops generating new obstacle point clouds according to the environmental perception module.

10. The method according to any one of claims 1 to 3, characterized in that: The autonomous operating equipment is provided with a collision sensing module, which is used to generate an obstacle point cloud based on collision information and position information of the autonomous operating equipment; the autonomous operating equipment shuts down the obstacle avoidance mechanism, including the autonomous operating equipment clearing the obstacle point cloud that has been generated and stopping generating a new obstacle point cloud based on the collision sensing module.

11. The method according to claim 1, characterized in that: When the autonomous operation equipment moves along the channel to the second working area, a plurality of track points are generated according to the channel path, and the autonomous operation equipment is controlled to move to the track points that have not been reached on the channel in sequence through PID control.

12. The method according to claim 1, characterized in that When the autonomous operation equipment reaches the end of the channel and completes the passage through the channel, the working mode of the autonomous operation equipment is switched according to the preset operation schedule.

13. The method according to claim 1, characterized in that When the autonomous operating equipment moves along the passage to the second working area, it detects that the position of the autonomous operating equipment has not changed and continues for a predetermined time threshold, and then gives up moving along the passage to the second working area.

14. A control method for an autonomous operation device, wherein the autonomous operation device is provided with an environment perception module, wherein the environment perception module is a visual sensor, characterized in that: The control method comprises: When the autonomous operation equipment moves to the location of the passage or moves from the first working area to the second working area along the passage, the autonomous operation equipment detects obstacles through the environment perception module and obtains an image of obstacles in the current passage through the visual sensor; When an obstacle is detected, after determining that the autonomous operation equipment has saved a historical channel obstacle image, a third passage judgment is performed based on the current channel obstacle image and the historical channel obstacle image; In response to the judgment result of the third passing judgment, controlling the passing behavior of the autonomous operating device; Wherein, the third passage judgment includes judging whether the current channel obstacle image is similar to the historical channel obstacle image; Wherein, in response to the judgment result of the third passage judgment, controlling the passage behavior of the autonomous operation equipment includes: in response to the result of the third passage judgment indicating that the current channel obstacle image is similar to the historical channel obstacle image, the autonomous operation equipment turns off the obstacle avoidance mechanism and moves along the channel to the second working area; or the result of the third passage judgment indicates that the current channel obstacle image is not similar to the historical channel obstacle image, and the autonomous operation equipment gives up passage; or the result of the third passage judgment indicates that the current channel obstacle image is not similar to the historical channel obstacle image, the autonomous operation equipment performs the first passage judgment and the second passage judgment, and determines the passage behavior according to the results of the first passage judgment and the second passage judgment; Among them, the first passage judgment is to make a passage judgment based on whether an obstacle is repeatedly detected, including: when the result of the first passage judgment indicates that no obstacle is detected after repeated detection, the autonomous operation equipment moves along the channel to the second working area; when the result of the first passage judgment indicates that an obstacle is still detected after repeated detection, the autonomous operation equipment turns off the obstacle avoidance mechanism and moves along the channel to the second working area.

15. The method according to claim 14, characterized in that When the autonomous operating device moves to the location of the channel or moves along the channel from the first working area to the second working area, when an obstacle is detected, the autonomous operating device walks to a predetermined position near the channel, sets the autonomous operating device to a certain posture, and obtains an image containing the channel obstacle through the environmental perception module.

16. The method according to claim 14, characterized in that Before making the third passage judgment, determine whether the historical channel obstacle image is saved, including: determining that the historical channel obstacle image is not saved, and determining whether to turn off the obstacle avoidance mechanism and move along the channel to the second working area based on the first passage judgment and / or the second passage judgment; or determining that the historical channel obstacle image is saved, and making the third passage judgment based on the image features of the obstacle.

17. The method according to claim 16, characterized in that The process of acquiring the historical channel obstacle image includes, according to the first passage judgment and / or the second passage judgment, determining to turn off the obstacle avoidance mechanism and proceed along the channel, and after successfully passing the channel, saving the current channel obstacle image as the historical channel obstacle image for the subsequent passage process.

18. The method according to claim 14, 16 or 17, characterized in that The second passage judgment is to make a passage judgment based on the geometric feature information of the obstacle, including: when the result of the second passage judgment indicates that the surface of the obstacle constitutes a plane, the autonomous operation equipment turns off the obstacle avoidance mechanism and moves along the channel to the second working area; when the result of the second passage judgment indicates that the surface of the obstacle does not constitute a plane, the autonomous operation equipment gives up passing.

19. The method according to claim 14, characterized in that The third passage judgment is performed based on the current channel obstacle image and the historical channel obstacle image, including extracting feature information of the two frames of images based on the currently acquired channel obstacle image and the historical channel obstacle image, and judging whether the two frames of images are similar through the feature information of the two frames of images.

20. The method according to claim 19, characterized in that Determining whether the two frames of images are similar based on feature information of the two frames of images includes obtaining grayscale histograms of the two frames of images, calculating the cross entropy of the grayscale histograms of the two frames of images, comparing the obtained cross entropy with a predetermined threshold, and determining whether the cross entropy of the grayscale histograms of the two frames of images is less than the predetermined threshold.

21. The method according to claim 19, characterized in that Determining whether the two frames of images are similar is based on feature information of the two frames of images, including segmenting the two frames of images, performing a closing operation on the image masks to eliminate holes or gaps in the images, and determining whether the number of sub-regions segmented from the two frames of images is consistent.

22. The method according to claim 14, characterized in that When traveling along the channel to the second working area, a plurality of track points are generated according to the channel path, and the autonomous operating equipment is controlled to sequentially move to the track points that have not been reached on the channel through PID control.

23. The method according to claim 14, characterized in that When the autonomous operation equipment reaches the end of the channel and completes the passage through the channel, the working mode of the autonomous operation equipment is switched according to the preset operation schedule.

24. The method according to claim 14, characterized in that When the autonomous operating equipment moves along the passage to the second working area, if it is detected that the position of the autonomous operating equipment has not changed and continues for a predetermined time threshold, the autonomous operating equipment abandons moving along the passage to the second working area.

25. The method according to claim 14, characterized in that The environmental perception module generates an obstacle point cloud according to the detected obstacle information; the autonomous operation equipment shuts down the obstacle avoidance mechanism, including that the autonomous operation equipment clears the obstacle point cloud that has been generated and stops generating new obstacle point clouds according to the environmental perception module.

26. The method according to claim 14, characterized in that The autonomous operating equipment is provided with a collision sensing module, which is used to generate an obstacle point cloud based on collision information and position information of the autonomous operating equipment; the autonomous operating equipment shuts down the obstacle avoidance mechanism, including the autonomous operating equipment clearing the obstacle point cloud that has been generated and stopping generating a new obstacle point cloud based on the collision sensing module.

27. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it is possible to implement the control method of the autonomous working equipment according to any one of claims 1 to 26.

28. An autonomous operating device, characterized in that: Used to execute the control method of the autonomous working equipment as described in any one of claims 1 to 26, or including the computer-readable storage medium as described in claim 27.

Images