Obstacle detection method and device, robot and storage medium

Abstract

The invention discloses an obstacle detection method and device, a robot and a storage medium, and the method comprises the steps: obtaining the pre-stored obstacle information corresponding to a target position, and the pre-stored obstacle information at least comprises the pre-stored information of the position occupied by an obstacle and the type of the obstacle; acquiring multi-frame current image data of the target position according to pre-stored position information occupied by the obstacle; judging whether an obstacle exists in the multi-frame current image data or not; and determining whether an obstacle exists in the target position or not according to the number of times that the obstacle does not exist and a preset first threshold value corresponding to the type of the obstacle.

Description

[0001] This application is a divisional application of application number "202211126579.7", filed on "September 16, 2022", entitled "Method and apparatus for detecting obstacles, robot and storage medium". Technical Field

[0002] This invention belongs to the field of cleaning device technology, specifically relating to an obstacle detection method and device, a robot, and a storage medium. Background Technology

[0003] With the development of technology and the improvement of people's living standards, robots, such as vacuum cleaners and floor scrubbers, free people from tedious cleaning work. They can keep the environment of homes and offices clean and allow people to enjoy more free time, which is why they are favored by people.

[0004] In real-world applications, the positions of some obstacles frequently change, such as stools and small items scattered on the ground (like shoes and socks). Cleaning equipment needs to continuously identify these obstacles during its movement to update the cleaning map and route promptly when they change, ensuring the cleaning work better reflects the actual scenario and improves the user experience. However, due to complex external environments or the influence of recognition algorithms, the accuracy of these object identifications is inherently difficult to guarantee, easily leading to misidentification or inaccurate obstacle location information. This can result in the cleaning equipment being unable to effectively avoid obstacles or the cleaning path not conforming to the actual scenario, affecting the safe operation of the cleaning equipment and the cleaning effect. Summary of the Invention

[0005] This invention provides an obstacle detection method and device, a robot and a storage medium, which can further improve the accuracy of obstacle detection, thereby improving the safe operation of cleaning equipment and the cleaning effect.

[0006] To achieve the above objectives, the present invention provides a method for detecting obstacles, the method comprising:

[0007] Obtain pre-stored obstacle information corresponding to the target location, wherein the pre-stored obstacle information includes at least the pre-stored location information and obstacle type of the obstacle;

[0008] Based on the pre-stored location information of the obstacles, obtain multiple frames of current image data of the target location;

[0009] Determine whether there are obstacles in the current image data of the multiple frames;

[0010] The presence or absence of an obstacle at the target location is determined based on the number of times the absence of an obstacle is determined and a preset first threshold corresponding to the type of obstacle.

[0011] Preferably, in the obstacle detection method, the position information occupied by the obstacle is characterized at least using laser point cloud data;

[0012] The method further includes:

[0013] Obtain the current laser point cloud data corresponding to the target location;

[0014] Based on the laser point cloud data corresponding to the location information occupied by the obstacle, and the comparison result of the current laser point cloud data, a first probability is determined that there is no obstacle at the target location;

[0015] Accordingly, the step of determining whether an obstacle exists at the target location based on the number of times the absence of an obstacle is determined and a preset first threshold corresponding to the type of obstacle includes:

[0016] Based on the number of times it is determined that there is no obstacle, and a preset first threshold corresponding to the type of obstacle, a second probability that there is no obstacle at the target location is determined;

[0017] Based on the first probability and the second probability, it is determined whether there is an obstacle at the target location.

[0018] Preferably, in the obstacle detection method, the step of determining whether an obstacle exists at the target location based on a first probability and a second probability specifically includes:

[0019] Based on the first probability, the second probability, and the obstacle type, determine the fusion coefficients of the first probability and the second probability, respectively;

[0020] Based on the first probability, the second probability, and the corresponding fusion coefficient, it is determined whether there is an obstacle at the target location.

[0021] Preferably, in the obstacle detection method, the step of determining the first probability that there is no obstacle at the target location based on the comparison result of the laser point cloud data corresponding to the location information occupied by the obstacle and the current laser point cloud data specifically includes:

[0022] Based on the laser point cloud data corresponding to the location information occupied by the obstacle and the current laser point cloud data, determine the idle rate of the laser point cloud data corresponding to the location information occupied by the obstacle in the current laser point cloud data;

[0023] Based on the idle rate and a preset second threshold, a first probability is determined that there are no obstacles at the target location.

[0024] Preferably, in the obstacle detection method, the current image data is acquired with the center of the location occupied by the obstacle as the center of the image acquisition field of view.

[0025] Preferably, in the obstacle detection method, the pre-stored obstacle information further includes the first field of view range when the obstacle is constructed;

[0026] Accordingly, the step of obtaining multiple frames of current image data of the target location based on pre-stored location information of obstacles includes:

[0027] Extract the second field of view corresponding to the current image data acquisition from the first field of view, wherein the second field of view is smaller than the first field of view;

[0028] Collect multiple frames of current image data of the target position within the second field of view.

[0029] Preferably, in the obstacle detection method, after the step of determining whether an obstacle exists at the target location based on the number of times the absence of an obstacle is determined and a preset first threshold corresponding to the obstacle type, the method further includes:

[0030] If there are no obstacles at the target location, delete the pre-stored obstacle information corresponding to the target location.

[0031] Preferably, in the obstacle detection method, the pre-stored obstacle information further includes pre-stored image data of the obstacle;

[0032] Accordingly, the detection method further includes:

[0033] When there is an obstacle at the target location, it is determined whether the obstacle is the same obstacle based on the current image data and the pre-stored image data.

[0034] When the judgment result is different obstacles, the pre-stored obstacle information corresponding to the target location is updated.

[0035] To achieve the above objectives, the present invention also provides a method for detecting obstacles, characterized in that it includes:

[0036] Obtain pre-stored obstacle information corresponding to the target location, wherein the pre-stored obstacle information includes at least the pre-stored location information occupied by the obstacles;

[0037] Based on the pre-stored location information of obstacles, obtain the current laser point cloud data and multiple frames of current image data of the target location;

[0038] Based on the current laser point cloud data of the target location and multiple frames of current image data, determine whether there is an obstacle at the target location.

[0039] To achieve the above objectives, the present invention also provides an obstacle identification device, which includes:

[0040] An information acquisition unit is used to acquire pre-stored obstacle information corresponding to a target location. The pre-stored obstacle information includes at least the location information occupied by the pre-stored obstacle and the obstacle type.

[0041] The image acquisition unit is used to acquire multiple frames of current image data of the target location based on the pre-stored location information of the obstacles.

[0042] The data judgment unit is used to determine whether there are obstacles in the current image data of the multiple frames;

[0043] The result determination unit is used to determine whether there is an obstacle at the target location based on the number of times the obstacle is determined to be non-existent and a preset first threshold corresponding to the obstacle type.

[0044] To achieve the above objectives, the present invention also provides a robot comprising:

[0045] At least one processor; and,

[0046] A memory communicatively connected to the at least one processor; wherein,

[0047] The memory stores instructions that can be executed by the at least one processor, which, when executed, enable the at least one processor to perform the obstacle detection method described above.

[0048] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the above-described obstacle detection method.

[0049] The technical solution provided by this invention has the following advantages:

[0050] This invention acquires pre-stored obstacle information corresponding to a target location. This pre-stored obstacle information includes at least pre-stored location information and obstacle type. Based on the pre-stored obstacle location information, it acquires multiple frames of current image data of the target location. Then, it determines whether an obstacle exists in the multiple frames of current image data. Finally, based on the number of times the obstacle is not identified and a preset first threshold corresponding to the obstacle type, it determines whether an obstacle exists at the target location. This allows for the re-collection of obstacle information based on its original location information, and the use of the acquired multi-frame real-time images to determine if the obstacle at its original location has changed. If, based on the recognition failure threshold corresponding to the original obstacle type, the obstacle is not identified at its original location, it can be determined that the original obstacle's location has indeed changed significantly. In this case, the cleaning path or cleaning map can be updated, effectively reducing the impact of obstacle location changes and complex environments on real-time obstacle recognition, ensuring the safe operation of the cleaning equipment and the cleaning effect. Attached Figure Description

[0051] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the first embodiment of the obstacle detection method of the present invention;

[0053] Figure 2 This is a schematic diagram of the second embodiment of the obstacle detection method of the present invention;

[0054] Figure 3 This is a schematic diagram of the third embodiment of the obstacle detection method of the present invention;

[0055] Figure 4 This is a schematic diagram of the fourth embodiment of the obstacle detection method of the present invention;

[0056] Figure 5 This is a schematic diagram of the fifth embodiment of the obstacle detection method of the present invention;

[0057] Figure 6 This is a schematic diagram of the sixth embodiment of the obstacle detection method of the present invention;

[0058] Figure 7 This is a schematic diagram of the seventh embodiment of the obstacle detection method of the present invention;

[0059] Figure 8 This is a schematic diagram of the eighth embodiment of the obstacle detection method of the present invention;

[0060] Figure 9 This is a frame diagram of the first embodiment of the obstacle detection device of the present invention;

[0061] Figure 10 This is a schematic diagram of an embodiment of the robot of the present invention;

[0062] Figure 11 A diagram illustrating the field of view captured by the camera during shooting;

[0063] Figure 12 This is a schematic diagram of an obstacle map.

[0064] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0066] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0067] In this embodiment of the invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0068] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the embodiments of the present invention to facilitate a better understanding of the invention. However, the technical solutions claimed in the present invention can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the following embodiments is for ease of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0069] This embodiment relates to an obstacle detection method. In this embodiment, a robot is used as an example. The obstacle detection method is mainly applied to sweeping robots, floor scrubbers, etc., and will not be listed here. This embodiment does not limit the application of these methods.

[0070] The implementation details of the obstacle detection method according to the first embodiment of the present invention are described below. The following implementation details are provided for ease of understanding only and are not necessary for implementing this solution.

[0071] The specific process of this implementation method is as follows: Figure 1 As shown, it specifically includes:

[0072] Step S100: Obtain the pre-stored obstacle information corresponding to the target location. The pre-stored obstacle information includes at least the pre-stored location information and obstacle type of the obstacle.

[0073] It should be understood that the implementing entity in this embodiment may be, but is not limited to, a robot, such as a cleaning robot (sweeping robot, floor scrubber, etc.), and no specific limitation is made here. The following detailed description uses a robot as an example.

[0074] The target location can be any position the robot is in during its movement. The pre-stored obstacle information can be obstacle information for a preset location stored by the user, or it can be the obstacle information updated when the robot most recently confirmed whether the obstacles at the target location had moved. Obstacles can be any non-ground object or person, such as pedestrians, tables, chairs, televisions, fans, etc. Obstacles can be frequently moved or infrequently moved.

[0075] Pre-stored obstacle information may include, but is not limited to, the pre-stored location information of obstacles, obstacle types, and the field of view when acquiring obstacles (i.e., the field of view when the camera captures the image). The location information of obstacles can be absolute, relative, or their position on the accumulated map and the area they occupy. Obstacle types can be categorized by movement frequency or by purpose; no specific restrictions are placed here. When categorized by movement probability, for example, they can be divided into items with high movement probability (e.g., shoes, benches, fans, etc.) and items with low movement probability (e.g., cabinets, tables, etc.).

[0076] Step S200: Based on the pre-stored location information of the obstacles, obtain multiple frames of current image data of the target location.

[0077] The boundaries of the pre-stored obstacle location information can be used as the boundaries for current image data acquisition, capturing multiple frames of current image data at the target location. Alternatively, the center of the obstacle's location can be used as the center of the image acquisition field of view for current image data acquisition, to more accurately determine whether the obstacle has moved significantly. Of course, other methods can also be used to acquire multiple frames of current image data at the target location based on the pre-stored obstacle location information; this is not limited here.

[0078] Step S300: Determine whether there are obstacles in the current image data of the multiple frames;

[0079] Determining whether there are obstacles in multiple frames of current image data can be done using conventional obstacle identification methods, without any specific restrictions.

[0080] Step S400: Based on the number of times the obstacle is determined to be non-existent and a preset first threshold corresponding to the obstacle type, determine whether there is an obstacle at the target location.

[0081] Obstacles can be detected based on the current image data of each frame. If no obstacle is detected, the count is 1, and so on. Obstacle detection is performed on the current image data of each frame to obtain the number of times no obstacle exists.

[0082] The preset first threshold corresponding to the obstacle type can be understood as a first threshold determined according to the obstacle type. For example, for obstacles with high movement frequency (such as shoes, socks, headphones, etc.), a smaller first threshold can be set, such as setting the first threshold to 2 times. After determining that the obstacle does not exist 2 times, the obstacle is considered to have been removed. On the other hand, for obstacles with low movement frequency (such as stools, air purifiers, fans, etc.), a larger first threshold can be set, such as setting the first threshold to 5 times. Then, the obstacle is considered to have been removed only after determining that the obstacle does not exist 5 times or more. The specific setting of the first threshold corresponding to different obstacle types can be determined according to the specific working environment.

[0083] This invention acquires pre-stored obstacle information corresponding to a target location. This pre-stored obstacle information includes at least pre-stored location information and obstacle type. Based on the pre-stored obstacle location information, it acquires multiple frames of current image data of the target location. Then, it determines whether an obstacle exists in the multiple frames of current image data. Finally, based on the number of times the obstacle is not identified and a preset first threshold corresponding to the obstacle type, it determines whether an obstacle exists at the target location. This allows for the re-collection of obstacle information based on its original location information, and the use of the acquired multi-frame real-time images to determine if the obstacle at its original location has changed. If, based on the recognition failure threshold corresponding to the original obstacle type, the obstacle is not identified at its original location, it can be determined that the original obstacle's location has indeed changed significantly. In this case, the cleaning path or cleaning map can be updated, effectively reducing the impact of obstacle location changes and complex environments on real-time obstacle recognition, ensuring the safe operation of the cleaning equipment and the cleaning effect.

[0084] The second embodiment of the present invention relates to a method for detecting obstacles. The second embodiment mainly describes the specific implementation of obstacle detection based on the location information occupied by the obstacle, characterized at least using laser point cloud data.

[0085] like Figure 2 As shown, the second embodiment relates to a method for detecting obstacles, which further includes:

[0086] Step S510: Obtain the current laser point cloud data corresponding to the target location;

[0087] It should be understood that laser point cloud data refers to data collected by a lidar radar (i.e., a set of scan points). In this embodiment, obtaining the current laser point cloud data corresponding to the target location can be achieved by obtaining the laser point cloud data corresponding to the target location from the accumulated laser point cloud grid map. For example... Figure 12 As shown, the dashed lines represent obstacle points identified by the linear laser. These points are not categorized; it simply indicates that an obstacle exists there. The double-dotted lines represent obstacles identified by the AI. The AI ​​camera can identify the specific obstacle category. The grids in other locations are either not identified by the AI ​​or do not belong to the obstacles identified by the AI.

[0088] Step S520: Based on the laser point cloud data corresponding to the location information occupied by the obstacle and the comparison result of the current laser point cloud data, determine the first probability that there is no obstacle at the target location;

[0089] It should be understood that the laser point cloud data corresponding to the location information occupied by the obstacle includes the pre-stored grid information occupied by the obstacle in the laser point cloud grid map, and the current laser point cloud data includes the grid information occupied by the current obstacle in the laser point cloud grid map.

[0090] It should be noted that steps S510 and S520 can be before or after step S300, without any specific restrictions.

[0091] Further, step S520 specifically includes:

[0092] Step S521': Based on the current laser point cloud data, determine that the obstacle is updated to a ground grid in the laser point cloud grid map;

[0093] In practice, by analyzing the height information of the grid corresponding to the obstacle in the laser point cloud grid map, if the height is zero, the obstacle corresponding to the grid can be considered as the ground, otherwise it is considered as an obstacle. In this way, by analyzing each grid at the position occupied by the obstacle, the grid that the obstacle is updated as the ground in the laser point cloud grid map is determined.

[0094] Step S522': Determine whether the number of grids in the laser point cloud grid map that the obstacle has been updated to be the ground is greater than a second threshold, wherein the second threshold corresponds to the laser point cloud data corresponding to the location information occupied by the obstacle.

[0095] It should be noted that the second threshold can be determined based on the number of grids occupied by the pre-stored obstacle in the laser point cloud grid map. For example, if the pre-stored obstacle occupies 10 grids in the laser point cloud grid map, the second threshold can be 10, 9, ... The specific setting can be determined according to the working environment, and no specific restrictions are made here.

[0096] Step S523': Based on the judgment result, determine the first probability that there is no obstacle at the target location.

[0097] Specifically, if the number of grid cells in the laser point cloud grid map that are updated to represent the ground for an obstacle is greater than the second threshold, it indicates a higher probability that the obstacle has been removed. For example, when the number of grid cells in the laser point cloud grid map that are updated to represent the ground for an obstacle is greater than the second threshold, the first probability can be either 100% or 90%, without specific limitations. Since the type of obstacle cannot be determined from the data collected by the LiDAR, the second threshold is not different for different obstacles.

[0098] Accordingly, step S400 specifically includes:

[0099] Step S410: Based on the number of times it is determined that there is no obstacle and a preset first threshold corresponding to the type of obstacle, determine the second probability that there is no obstacle at the target location;

[0100] In practical implementation, the preset first threshold corresponding to the obstacle type can be understood as the first threshold corresponding to different obstacle types determined according to the obstacle type. For example, for obstacles with high movement frequency (such as shoes, socks, headphones, etc.), a smaller first threshold can be set, such as setting the first threshold to 2 times. After judging that there is no obstacle twice, it is considered that the obstacle has been moved with a high probability. For example, if it is judged that there is no obstacle twice and the first threshold is 2 times, the second probability is considered to be 80%; if it is judged that there is no obstacle five times and the first threshold is 2 times, the second probability is considered to be 100%...

[0101] For example, for obstacles with low movement frequency (such as stools, air purifiers, fans, etc.), a larger first threshold can be set. If the first threshold is set to 5 times, then the obstacle is considered to have been moved only after it has been determined to be absent 5 times or more. For instance, if the first threshold is 5 times and the obstacle has been determined to be absent 5 times, the probability is considered to be 80%; if the obstacle has been determined to be absent 10 times and the first threshold is 5 times, the probability is considered to be 100%... The specific first threshold setting for different obstacle types can be determined according to the specific working environment.

[0102] Step S420: Determine whether there is an obstacle at the target location based on the first probability and the second probability.

[0103] It should be understood that determining whether an obstacle exists at the target location based on the first probability and the second probability can be done in several ways. If one of the probabilities exceeds a preset value, then the target location is considered to be free of obstacles (i.e., the obstacle has been removed). Alternatively, it can be determined based on the type of obstacle, whether the first probability or the second probability is dominant. For example, for obstacles with a high probability of movement (such as shoes), if the first probability reaches a preset value, then the target location is considered to be free of obstacles. In other embodiments, other solutions may also be adopted, and no specific limitations are made here.

[0104] Because deleting grid points occupied by obstacles using line lasers typically requires scanning the entire area where the obstacle is located (i.e., the grid area corresponding to the pre-stored obstacle location), and all grid points corresponding to the pre-stored obstacle locations must be scanned and confirmed as ground before the obstacle can be identified as ground. However, in practice, it is difficult to guarantee that the grid points originally occupied by the obstacle will be scanned again by the line laser after the obstacle moves, which can lead to obstacle avoidance in empty areas. This invention determines the first probability that there is no obstacle at the target location by comparing the laser point cloud data corresponding to the location information occupied by the obstacle with the current laser point cloud data. Then, it determines the second probability that there is no obstacle at the target location by the number of times the obstacle is determined and a preset first threshold corresponding to the obstacle type. Finally, it determines whether there is an obstacle at the target location by combining the first probability and the second probability, thus improving the accuracy of obstacle removal identification.

[0105] like Figure 3 As shown, the obstacle detection method according to the third embodiment specifically includes step S420:

[0106] Step S421: Determine the fusion coefficients of the first probability and the second probability based on the first probability, the second probability, and the obstacle type, respectively;

[0107] Specifically, the accuracy of recognition can be improved by setting a fusion coefficient based on the type of obstacle. For example, if the obstacle is a special obstacle (such as wires, feces, etc.), the fusion coefficient can be used.

[0108] Step S422: Determine whether there is an obstacle at the target location based on the first probability, the second probability, and the corresponding fusion coefficient.

[0109] like Figure 4 As shown, in the obstacle detection method of the fourth embodiment, step S520 specifically includes:

[0110] Step S521: Based on the laser point cloud data corresponding to the position information occupied by the obstacle and the current laser point cloud data, determine the idle rate of the laser point cloud data corresponding to the position information occupied by the obstacle in the current laser point cloud data.

[0111] It should be understood that the idle rate can be interpreted as the proportion of grid cells in the current laser point cloud data that have been updated to the ground, and the proportion of grid cells in the laser point cloud data corresponding to the location information of obstacles. Assuming that the grid cell occupied by the obstacle in the laser point cloud grid map is A, and the grid cell currently updated to the ground at location A is B, the idle rate = the number of grid cells in B / the number of grid cells in A.

[0112] In specific implementation, step S521 includes:

[0113] Step S5211: Based on the current laser point cloud data, determine that the obstacle is updated to a ground grid in the laser point cloud grid map;

[0114] In practice, by analyzing the height information of the grid corresponding to the obstacle in the laser point cloud grid map, if the height is zero, the obstacle corresponding to the grid can be considered as the ground, otherwise it is considered as an obstacle. In this way, by analyzing each grid at the position occupied by the obstacle, the grid that the obstacle is updated as the ground in the laser point cloud grid map is determined.

[0115] Step S5212: Based on the grid of the obstacle that is updated to the ground in the laser point cloud grid map, and the proportion of the grid occupied by the laser point cloud data corresponding to the location information occupied by the obstacle.

[0116] It should be understood that the idle rate in the current laser point cloud data is calculated based on the proportion of laser point cloud data that occupies the grid corresponding to the location information of the obstacle in the laser point cloud grid map that has been updated as the ground.

[0117] Step S522: Based on the idle rate and the preset second threshold, determine the first probability that there is no obstacle at the target location.

[0118] It should be understood that the second threshold can be a threshold for the idle rate, or a threshold for the number of grid cells in the laser point cloud grid map that are updated to represent the ground relative to obstacles. In this embodiment, the second threshold is a threshold for the idle rate, such as 80%, 90%, ...

[0119] Specifically, if the idle rate is greater than the second threshold, it indicates that the obstacle is more likely to be removed. For example, when the idle rate is greater than the second threshold, the first probability can be 100% or 90%, without specific restrictions. Since the type of obstacle cannot be determined from the data collected by the LiDAR, the second threshold is usually not different for different obstacles.

[0120] like Figure 5 As shown, the obstacle detection method according to the fifth embodiment includes a first field of view during obstacle construction, where the pre-stored obstacle information further includes the first field of view during obstacle construction. Obstacle construction refers to the processing operation of collecting and identifying obstacle information stored in the robot's map. The first field of view can be the field of view corresponding to the camera collecting information about the obstacle during obstacle construction. The field of view can be determined based on the robot's pose information and the camera's field of view angle when collecting images corresponding to the obstacle. The pose information can include the robot's position information and orientation information. Figure 11As shown, assuming the robot's field of view is b, the robot's field of view range is the area divided by the two sides of the field of view angle b.

[0121] In some implementations, the first field of view can be characterized using the robot's pose relative to the obstacle during obstacle construction and the camera's field of view angle. Correspondingly, when the robot re-enters the position corresponding to the first field of view, its pose can be adjusted, and current image data can be acquired based on a pre-stored field of view angle or a smaller field of view angle, to further verify whether the obstacle's position has moved based on the acquired current image data. For ease of description, the field of view corresponding to the current image data acquisition can be described as the second field of view. For example... Figure 11 As shown, the current image data can be acquired using the field of view angle c, and based on the acquired current image data, it can be determined whether there are still obstacles at the locations previously occupied by obstacles. Correspondingly, Figure 11 The area divided by the two sides of the middle field of view angle c is the second field of view range. Figure 11 In this context, 'a' represents the camera's theoretical maximum field of view.

[0122] Alternatively, the center of the location occupied by the obstacle can be used as the center of the field of view to collect current image data, so as to more accurately determine whether the obstacle has moved significantly.

[0123] If the camera's field of view is not adjustable when acquiring images, image acquisition can be performed based on the robot's pose information relative to the obstacle during obstacle construction and the camera's field of view to obtain initial current image data. Then, the initial current image data is cropped to obtain the current image data, ensuring that the field of view corresponding to the current image data is smaller than the field of view during obstacle construction. Alternatively, the robot's position can be adjusted while keeping the camera's field of view constant. For example, the robot can be controlled to move towards the pre-stored location information of the obstacle, and its pose can be adjusted in real time to ensure that the center of the obstacle's location serves as the center of the second field of view. This method also ensures that the second field of view is within the first field of view but smaller than it.

[0124] The image acquisition range is a second field of view, located within the first field of view but smaller than it. Multiple frames of current image data showing the target position within this second field of view are acquired. If an obstacle has not moved or has moved only a small range, it can usually be identified even within a smaller second field of view than the first. If no obstacle information is found within this smaller field of view, it indicates that the obstacle's overall position has moved significantly, meaning the change in its position has greatly affected the robot's path. In this case, the original obstacle information can be deleted, and obstacle identification can be performed again. Conversely, if the obstacle is identified within this smaller field of view, it indicates that it has not moved or has moved only a small range, having little impact on the robot's original path. In this case, the pre-stored obstacle information does not need to be updated. Therefore, by selecting a second field of view smaller than the first, the amount of data processing can be reduced, and the movement of obstacles can be more accurately determined, effectively guiding the robot's map update method.

[0125] like Figure 11 As shown, during the robot's movement, when identifying obstacles, obstacle recognition is performed based on the first field of view b. When an obstacle is identified, its position information, obstacle type information, and the first field of view can be pre-stored. After the robot leaves that position for a period of time and then moves back to that position (e.g., for...),... Figure 12 When the robot moves back to the location where it created the obstacle (stool), it can identify the obstacle (stool) based on the pre-stored location information of the obstacle (stool). For example, it can scan the second field of view (c) by using the center position of the obstacle's location as the center position to determine whether the obstacle is identified in the second field of view (c). If no obstacle (stool) is identified, the failure count is incremented by 1. The number of identification failures is counted for several consecutive frames. If the total number of failures is greater than a certain threshold, it is considered that the obstacle has been moved (i.e., the position of the stool has changed significantly).

[0126] Accordingly, step S200 may include:

[0127] Step S210: Extract the second field of view corresponding to the current image data acquisition from the first field of view, wherein the second field of view is smaller than the first field of view.

[0128] Preferably, the second field of view is centered on the location occupied by the obstacle.

[0129] Step S220: Collect multiple frames of current image data of the target position within the second field of view.

[0130] like Figure 6 As shown, in the sixth embodiment of the obstacle detection method, after step S400, the detection method further includes:

[0131] Step S610: When there are no obstacles at the target location, delete the pre-stored obstacle information corresponding to the target location.

[0132] It should be understood that deleting the pre-stored obstacle information corresponding to the target location can be understood as updating the pre-stored obstacle information. Specifically, it can be updating the target location in the obstacle map to ground or other non-obstacle markers.

[0133] like Figure 7 As shown, in the obstacle detection method according to the seventh embodiment, the pre-stored obstacle information further includes pre-stored image data of the obstacle, and the detection method further includes:

[0134] Step S620: When there is an obstacle at the target location, determine whether the obstacle in the current image data and the pre-stored image data is the same obstacle.

[0135] It should be understood that when an obstacle is detected at the target location, there are two possibilities: either the obstacle is the original obstacle, or the obstacle is a new obstacle.

[0136] In specific implementation, based on the current image data and the pre-stored image data, it is determined whether the obstacles in the two are the same obstacle. This can be determined based on the type of obstacle. If the types are different, then they must be different obstacles. If the types are the same, conventional image recognition technology can be used to compare and identify whether they are the same obstacle. Other recognition methods can also be used, and no specific restrictions are made here.

[0137] Step S630: When the judgment result is different obstacles, update the pre-stored obstacle information corresponding to the target position.

[0138] It should be understood that when the obstacle is not the original obstacle, the pre-stored obstacle information corresponding to the target location is updated. Specifically, the target location in the obstacle map can be updated with the new obstacle information.

[0139] To achieve the above objectives, the eighth embodiment of the present invention also provides a method for detecting obstacles, such as... Figure 8 As shown, the obstacle detection method includes:

[0140] Step S710: Obtain the pre-stored obstacle information corresponding to the target location. The pre-stored obstacle information includes at least the pre-stored location information of the obstacles.

[0141] It should be understood that the executing entity in this embodiment may be, but is not limited to, a robot, such as a cleaning robot (sweeping robot, floor scrubbing robot, etc.), and no specific limitation is made here. The following is a detailed description using a robot as an example.

[0142] The target location can be any position the robot is in during its movement. The pre-stored obstacle information can be obstacle information pre-stored by the user at a preset location, obstacle information established when the robot first detects an obstacle at the target location during its movement, or obstacle information updated when the robot most recently confirmed whether an obstacle at the target location has moved. Obstacles can be any non-ground object or person, such as pedestrians, tables, chairs, televisions, fans, etc. Obstacles can be frequently moved or infrequently moved.

[0143] Pre-stored obstacle information may include, but is not limited to, the pre-stored location information of obstacles, obstacle types, and the field of view when acquiring obstacles (i.e., the field of view when the camera captures the image). The location information of obstacles can be absolute, relative, or their position on the accumulated map and the area they occupy. Obstacle types can be categorized by movement frequency or by purpose; no specific restrictions are placed here. When categorized by movement probability, for example, they can be divided into items with high movement probability (e.g., shoes, benches, fans, etc.) and items with low movement probability (e.g., cabinets, tables, etc.).

[0144] Step S720: Based on the pre-stored location information of the obstacle, obtain the current laser point cloud data and multiple frames of current image data of the target location;

[0145] The process of acquiring multiple frames of current image data of the target location based on pre-stored obstacle position information can be achieved by using the pre-stored field of view corresponding to the obstacle, or by capturing a smaller field of view than the pre-stored obstacle-corresponding field of view, with the center of the obstacle's position as the center of the image acquisition field of view. For example, if the pre-stored obstacle-corresponding field of view is A, and the field of view for acquiring multiple frames of current image data of the target location in step S200 is B, B can be the same as A, or B can be smaller than A, with the center of the obstacle's position as the center of the field of view. By using a smaller field of view than the pre-stored obstacle-corresponding field of view, the amount of data processing can be effectively reduced, while also allowing for a more accurate determination of the obstacle's movement. For example, if an obstacle is present within field of view B, it can definitely be identified; if it is not identified, the overall position of the obstacle may have moved significantly, and the obstacle's position information can be updated.

[0146] It should be understood that laser point cloud data refers to data collected by lidar (i.e., a set of scan points). In this embodiment, obtaining the current laser point cloud data corresponding to the target location can be achieved by obtaining the laser point cloud data of the target location from the accumulated laser point cloud grid map.

[0147] Step S730: Determine whether there is an obstacle at the target location based on the current laser point cloud data and multiple frames of current image data.

[0148] It should be understood that determining whether there is an obstacle at the target location based on the current laser point cloud data and multiple frames of current image data can be, but is not limited to, using the embodiments of the above-described embodiments two to four.

[0149] To achieve the above objectives, the present invention also provides an obstacle recognition device, such as... Figure 9 As shown, the obstacle recognition device includes:

[0150] The information acquisition unit 810 is used to acquire pre-stored obstacle information corresponding to the target location. The pre-stored obstacle information includes at least the location information occupied by the obstacle and the obstacle type that are pre-stored.

[0151] The target location can be any position the robot is in during its movement. The pre-stored obstacle information can be obstacle information pre-stored by the user at a preset location, obstacle information established when the robot first detects an obstacle at the target location during its movement, or obstacle information updated when the robot most recently confirmed whether an obstacle at the target location has moved. Obstacles can be any non-ground object or person, such as pedestrians, tables, chairs, televisions, fans, etc. Obstacles can be frequently moved or infrequently moved.

[0152] Pre-stored obstacle information may include, but is not limited to, the pre-stored location information of obstacles, obstacle types, and the field of view when acquiring obstacles (i.e., the field of view when the camera captures the image). The location information of obstacles can be absolute, relative, or their position on the accumulated map and the area they occupy. Obstacle types can be categorized by movement frequency or by purpose; no specific restrictions are placed here. When categorized by movement probability, for example, they can be divided into items with high movement probability (e.g., shoes, benches, fans, etc.) and items with low movement probability (e.g., cabinets, tables, etc.).

[0153] The image acquisition unit 820 is used to acquire multiple frames of current image data of the target location based on the pre-stored location information of the obstacles.

[0154] The process of acquiring multiple frames of current image data of the target location based on pre-stored obstacle position information can be achieved by using the pre-stored field of view corresponding to the obstacle, or by capturing a smaller field of view than the pre-stored obstacle-corresponding field of view, with the center of the obstacle's position as the center of the image acquisition field of view. For example, if the pre-stored obstacle-corresponding field of view is A, and the field of view for acquiring multiple frames of current image data of the target location in step S200 is B, B can be the same as A, or B can be smaller than A, with the center of the obstacle's position as the center of the field of view. By using a smaller field of view than the pre-stored obstacle-corresponding field of view, the amount of data processing can be effectively reduced, while also allowing for a more accurate determination of the obstacle's movement. For example, if an obstacle is present within field of view B, it can definitely be identified; if it is not identified, the overall position of the obstacle may have moved significantly, and the obstacle's position information can be updated.

[0155] Data judgment unit 830 is used to determine whether there are obstacles in the current image data of the multiple frames;

[0156] Determining whether there are obstacles in multiple frames of current image data can be done using conventional obstacle identification methods, without any specific restrictions.

[0157] The result determination unit 840 is used to determine whether there is an obstacle at the target location based on the number of times the obstacle is determined to be non-existent and a preset first threshold corresponding to the obstacle type.

[0158] Obstacles can be detected based on the current image data of each frame. If no obstacle is detected, the count is 1, and so on. Obstacle detection is performed on the current image data of each frame to obtain the number of times no obstacle exists.

[0159] The preset first threshold corresponding to the obstacle type can be understood as a first threshold determined according to the obstacle type. For example, for obstacles with high movement frequency (such as shoes, benches, fans, etc.), a smaller first threshold can be set, such as setting the first threshold to 2 times. After determining that the obstacle does not exist 2 times, the obstacle is considered to have been moved. On the other hand, for obstacles with low movement frequency (such as cabinets, tables), a larger first threshold can be set, such as setting the first threshold to 5 times. Then, the obstacle is considered to have been moved only after determining that the obstacle does not exist 5 times or more. The specific setting of the first threshold corresponding to different obstacle types can be determined according to the specific working environment.

[0160] To achieve the above objectives, the present invention also provides a robot, such as... Figure 10 As shown, the robot includes at least one processor 901; and a memory 902 communicatively connected to the at least one processor 901; wherein the memory 902 stores instructions executable by the at least one processor 901, the instructions being executed by the at least one processor 901 to enable the at least one processor 901 to perform the obstacle detection method of the first to eighth embodiments described above.

[0161] The memory 902 and processor 901 are connected via a bus, which can include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 901 and memory 902. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 901 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 901.

[0162] Processor 901 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 902 can be used to store data used by processor 901 during operation.

[0163] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the above-described obstacle detection method.

[0164] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0165] Obviously, the embodiments described above are merely some, not all, embodiments of the present invention. Based on the embodiments of the present invention, those skilled in the art can make other variations or modifications without creative effort, and all such variations or modifications should fall within the scope of protection of the present invention.

Claims

1. A method for detecting obstacles, characterized in that, include: Obtain pre-stored obstacle information corresponding to the target location. The pre-stored obstacle information includes at least the pre-stored location information occupied by the obstacle, the obstacle type, and the first field of view range when the obstacle is constructed. Based on the pre-stored location information of the obstacles, the second field of view corresponding to the current image data acquisition is extracted from the first field of view, and multiple frames of current image data of the target position within the second field of view are acquired; wherein, the second field of view is smaller than the first field of view; Determine whether there are obstacles in the current image data of the multiple frames; The presence or absence of an obstacle at the target location is determined based on the number of times the absence of an obstacle is determined and a preset first threshold corresponding to the type of obstacle.

2. The obstacle detection method as described in claim 1, characterized in that, The current image data is acquired with the center of the location occupied by the obstacle as the center of the image acquisition field of view; The second field of view is centered on the location occupied by the obstacle.

3. The obstacle detection method as described in claim 1, characterized in that, The step of extracting the second field of view corresponding to the current image data acquisition from the first field of view, wherein the second field of view is smaller than the first field of view, includes: If the field of view of the camera is not adjustable when acquiring images, the initial current image data is obtained by acquiring images based on the robot's pose information relative to the obstacle and the camera's field of view during obstacle construction. The initial current image data is cropped to obtain the current image data, so that the second field of view corresponding to the current image data is smaller than the first field of view when the obstacle is built. Alternatively, with the camera's field of view unchanged, the robot's position can be adjusted so that the center of the position occupied by the obstacle is used as the center of the second field of view, making the second field of view within the first field of view but smaller than the first field of view.

4. The obstacle detection method as described in claim 1, characterized in that, The first field of view is the field of view corresponding to the camera collecting information about the obstacle when the obstacle is being built. The field of view is determined based on the robot's pose information and the camera's field of view angle when the image corresponding to the obstacle is collected. The pose information includes the robot's position information and orientation information. The first field of view is characterized by the robot's pose relative to the obstacle during obstacle construction and the camera's field of view angle.

5. The obstacle detection method as described in claim 1, characterized in that, After determining whether an obstacle exists at the target location based on the number of times the absence of an obstacle is determined and a preset first threshold corresponding to the type of obstacle, the method further includes: If there are no obstacles at the target location, delete the pre-stored obstacle information corresponding to the target location.

6. The obstacle detection method as described in claim 5, characterized in that, Deleting the pre-stored obstacle information corresponding to the target location means updating the target location in the obstacle map to ground or other non-obstacle identifiers.

7. The obstacle detection method as described in claim 1, characterized in that, The pre-stored obstacle information also includes pre-stored image data of the obstacles; After determining whether an obstacle exists at the target location based on the number of times the absence of an obstacle is determined and a preset first threshold corresponding to the type of obstacle, the method further includes: When there is an obstacle at the target location, it is determined whether the obstacle is the same obstacle based on the current image data and the pre-stored image data. When the judgment result is different obstacles, the pre-stored obstacle information corresponding to the target location is updated.

8. The obstacle detection method as described in claim 7, characterized in that, The step of determining whether the obstacles present in the current image data and the pre-stored image data are the same obstacle includes: Based on the current image data and the pre-stored image data, determine whether the types of obstacles present in the two are the same; If the obstacles present in both are of different types, then they are different obstacles; If the obstacles in both objects are of the same type, image recognition technology can be used to compare and identify whether the obstacles in both objects are the same.

9. An obstacle identification device, characterized in that, include: The information acquisition unit is used to acquire pre-stored obstacle information corresponding to the target location. The pre-stored obstacle information includes at least the pre-stored location information occupied by the obstacle, the obstacle type, and the first field of view range when the obstacle is constructed. The image acquisition unit is used to extract the second field of view corresponding to the current image data acquisition from the first field of view based on the pre-stored position information occupied by the obstacle, and acquire multiple frames of current image data of the target position within the second field of view; wherein, the second field of view is smaller than the first field of view. The data judgment unit is used to determine whether there are obstacles in the current image data of the multiple frames; The result determination unit is used to determine whether there is an obstacle at the target location based on the number of times the obstacle is determined to be non-existent and a preset first threshold corresponding to the obstacle type.

10. A robot, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the obstacle detection method as described in any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the obstacle detection method according to any one of claims 1 to 8.

Images