Robot obstacle avoidance reminder method, device, equipment and computer readable medium

Abstract

The embodiment of the present invention relates to the field of artificial intelligence technology, and discloses a robot obstacle avoidance reminder method, device, equipment and computer storage medium, the method comprising: obtaining positioning sensor data and human body sensor data of a target area; determining a plurality of feature points to be matched according to the positioning sensor data; matching the feature points to be matched with a target environment feature map, and determining the feature matching degree corresponding to the positioning sensor data; when the feature matching degree is less than a matching degree threshold, and when it is determined according to the human body sensor data that a human body exists in the target area, determining the occlusion ratio of the human body to the feature points to be matched; when the occlusion ratio is greater than the ratio threshold, performing an obstacle avoidance reminder. In the above manner, the embodiment of the present invention improves the accuracy of the robot obstacle avoidance reminder.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and specifically to a robot obstacle avoidance reminder method, device, equipment and computer-readable medium. Background Art

[0002] With the development of artificial intelligence, more and more mobile robots have been widely used in various scenarios. In the control of robots, SLAM (Simultaneous Localization And Mapping) is the basis, and when there is a crowd blocking the robot, it will affect the accuracy of the robot's SLAM positioning, so it is necessary to remind the people who form obstacles to avoid the robot.

[0003] The prior art generally adopts a solution of directly judging whether there is a person blocking the front of the robot, and if so, giving a reminder. In the process of implementing the present invention, the inventor found that the obstacle avoidance accuracy of the prior art solution is low. Summary of the invention

[0004] In view of the above problems, the embodiments of the present invention provide a robot obstacle avoidance reminder method, device, equipment and computer storage medium, which are used to solve the problem of low obstacle avoidance accuracy in the prior art.

[0005] According to one aspect of an embodiment of the present invention, a robot obstacle avoidance reminder method is provided, the method comprising:

[0006] Acquire positioning sensor data and human body sensor data of the target area;

[0007] Determine a plurality of feature points to be matched according to the positioning sensor data;

[0008] Matching the feature points to be matched with the target environment feature map to determine the feature matching degree corresponding to the positioning sensor data;

[0009] When the feature matching degree is less than the matching degree threshold, and it is determined according to the human body sensing data that a human body exists in the target area, determining the occlusion ratio of the human body to the feature point to be matched;

[0010] When the occlusion ratio is greater than the ratio threshold, an obstacle avoidance reminder is performed.

[0011] In an optional manner, the target environment feature map includes a plurality of reference feature points corresponding to the target environment: the method further includes:

[0012] Calculating the distance between each of the to-be-matched feature points and each of the reference feature points respectively;

[0013] Determine whether each feature point to be matched exists in the target environment feature map according to the distance;

[0014] The ratio of the number of the to-be-matched feature points that have matches to the total number of the to-be-matched feature points is determined as the feature matching degree.

[0015] In an optional manner, the method further includes:

[0016] When the human body sensing data is not empty, determining that the human body exists in the target area;

[0017] Analyzing the human body sensing data to obtain a human body occlusion area;

[0018] Determine the to-be-matched feature points within the human body occlusion area as the human body occlusion feature points;

[0019] The occlusion ratio is determined according to the ratio of the number of the human body occlusion feature points to the number of the feature points to be matched.

[0020] In an optional manner, the method further includes:

[0021] Obtaining the robot's forward route information;

[0022] Determine the target indication area and target reminder information according to the front-line route information;

[0023] Generate multimedia content corresponding to the target reminder information; the multimedia content includes projection image content; the target indication area includes a ground area where the robot's forward route is located;

[0024] The multimedia content is played in the target indication area.

[0025] In an optional manner, the method further includes:

[0026] Determining whether there is projection sensor data corresponding to the projection image content generated by the last obstacle avoidance reminder in the positioning sensor data;

[0027] If so, filtering the positioning sensor data according to the projection sensor data;

[0028] Feature extraction is performed on the filtered positioning sensor data to obtain the feature points to be matched.

[0029] In an optional manner, the method further includes:

[0030] Analyzing the projection sensing data to obtain projection texture information;

[0031] Analyzing the positioning sensor data within the target indication area to obtain background texture information;

[0032] Comparing the projection texture information with the background texture information to determine a projection impact area;

[0033] Determine a plurality of original feature points according to the positioning sensor data;

[0034] The original feature points in the projection influence area are filtered to obtain the feature points to be matched.

[0035] In an optional manner, the multimedia content also includes audio content; and the method further includes:

[0036] Play the audio content in the target indication area;

[0037] Determining the positioning sensor data after the audio content is played for a preset time as the post-reminder positioning sensor data;

[0038] Matching the post-reminder positioning sensor data with the target environment feature map to determine the post-reminder feature matching degree;

[0039] When the post-reminder feature matching degree is less than the matching degree threshold, the projection image content is projected into the target indication area.

[0040] According to another aspect of an embodiment of the present invention, a robot obstacle avoidance reminder device is provided, comprising:

[0041] An acquisition module is used to acquire positioning sensor data and human body sensor data of a target area;

[0042] A first determination module, used to determine a plurality of feature points to be matched according to the positioning sensor data;

[0043] A matching module, used to match the feature points to be matched with the target environment feature map, and determine the feature matching degree corresponding to the positioning sensor data;

[0044] A second determination module is used to determine the occlusion ratio of the human body to the feature point to be matched when the feature matching degree is less than a matching degree threshold and it is determined according to the human body sensing data that there is a human body in the target area;

[0045] The reminder module is used to provide obstacle avoidance reminder when the occlusion ratio is greater than a ratio threshold.

[0046] According to another aspect of an embodiment of the present invention, there is provided a robot obstacle avoidance reminder device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0047] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the robot obstacle avoidance reminder method.

[0048] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores at least one executable instruction, and the executable instruction enables a robot obstacle avoidance reminder device to perform the operation of the robot obstacle avoidance reminder method:

[0049] The embodiment of the present invention obtains positioning sensor data and human body sensor data of a target area; determines a plurality of feature points to be matched according to the positioning sensor data; matches the feature points to be matched with a feature map of a target environment to determine a feature matching degree corresponding to the positioning sensor data; when the feature matching degree is less than a matching degree threshold, i.e., positioning fails, and when it is determined according to the human body sensor data that a human body exists in the target area, determines an occlusion ratio of the feature points to be matched by the human body; when the occlusion ratio is greater than the ratio threshold, it can be determined that the main reason for the decrease in the feature matching degree is the occlusion by the human body, and therefore an obstacle avoidance reminder is performed to remind people to avoid it, thereby eliminating the improvement of the feature matching degree of the positioning sensor data after the occlusion, so that the robot can complete the positioning according to the positioning sensor data after the obstacle avoidance reminder.

[0050] Different from the prior art in which the robot performs an obstacle avoidance reminder when a person is detected, which results in low obstacle avoidance accuracy and poor interactive experience, the embodiments of the present invention first determine the feature matching degree corresponding to the currently acquired positioning sensor data, and determine the positioning failure based on the feature matching degree, and only when it is determined that there is a human body in the target area based on the human body sensor data and the occlusion ratio of the human body meets the ratio threshold, an obstacle avoidance reminder is given to the cause of the positioning failure, that is, the human body. This improves the accuracy of the obstacle avoidance reminder on the one hand, and avoids the impact of blind reminders on the interactive experience when the occlusion of the human body is not the cause of the positioning failure on the other hand.

[0051] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0053] Figure 1 A schematic diagram of the process of a robot obstacle avoidance reminder method provided by an embodiment of the present invention is shown;

[0054] Figure 2 A schematic diagram of the structure of a robot obstacle avoidance reminder device provided by an embodiment of the present invention is shown;

[0055] Figure 3 A schematic structural diagram of a robot obstacle avoidance reminder device provided in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0056] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0057] Figure 1 The flowchart of the robot obstacle avoidance reminder method provided by an embodiment of the present invention is shown, and the method is executed by a computer processing device. The computer processing device may include a robot, etc. Figure 1 As shown, the method comprises the following steps:

[0058] Step 101: Acquire positioning sensor data and human body sensor data of a target area.

[0059] In one embodiment of the present invention, the positioning sensor data can be acquired in real time or periodically by a visual sensor provided on the robot when performing a positioning action. The visual sensor can include a laser radar, a depth vision camera, etc., wherein the depth vision camera can be a ToF (Time of flight Camera), an RGB binocular camera, and a structured light.

[0060] The positioning sensor data includes sensor data within the robot's visual range, which is used to characterize the sensor characteristics of the robot's position in the current environment. The robot's visual range can be a certain angle in front of the robot, such as 180° or 360°.

[0061] In one embodiment of the present invention, the target area is a fan-shaped area with a preset angle in front of the robot, wherein the preset angle is set according to the size and movement mode of the robot, so that the target area can cover the area of ​​the robot's moving target distance, and the target distance is the distance between the current position and the position to be positioned next time.

[0062] It is easy to understand that in order to complete positioning, the collection range of the robot's positioning sensor data will include the area it needs to pass through when moving the target distance. Therefore, the target area is included in the visual range corresponding to the aforementioned positioning sensor data.

[0063] In one embodiment of the present invention, human body sensing data refers to data obtained by detecting human body features. The human body sensing data may be obtained through infrared human body detection equipment or RGB image sensing equipment, etc. The human body features may correspond to human body infrared features and human body image features, wherein the human body image features may include image contour features and image depth features, etc.

[0064] Step 102: Determine a plurality of feature points to be matched based on the positioning sensor data.

[0065] In one embodiment of the present invention, the positioning sensor data may be preprocessed to obtain a laser lightning dense point cloud map or a visual camera sparse point cloud map. Then, feature extraction is performed on the preprocessed point cloud map to obtain a plurality of feature points to be matched. The process of extracting features from the point cloud map includes calculating the curvature of the point cloud, determining a feature point cloud according to the curvature of the point cloud, the feature point cloud includes corner feature points and surface feature points, and the corner feature points and surface feature points are determined as the plurality of feature points to be matched.

[0066] In another embodiment of the present invention, corner point detection can be performed on the positioning sensor data according to a preset feature detection algorithm, such as the FAST algorithm and the SIFT algorithm to obtain multiple corner points, and then the descriptor of each corner point can be determined according to a descriptor extraction algorithm such as the domain module matching algorithm or the ORB (Oriented FAST and Rotated BRIEF) algorithm, and a corner point and its corresponding descriptor can be determined as a feature point to be matched.

[0067] In yet another embodiment of the present invention, considering the influence of the projection content on the positioning sensor data, the process of determining the feature points to be matched in step 102 further includes:

[0068] Step 1021: Determine whether there is projection sensor data corresponding to the projection image content generated by the last obstacle avoidance reminder in the positioning sensor data.

[0069] In one embodiment of the present invention, determining whether there is a projection sensor corresponding to the last obstacle avoidance reminder can be done by detecting whether the projection device on the robot is turned on. If it is turned on, it is determined that there is projection sensor data corresponding to the projection sensor content generated by the last obstacle avoidance reminder, otherwise it is deemed that the projection sensor data does not exist.

[0070] Step 1022: If present, filter the positioning sensor data according to the projection sensor data.

[0071] In one embodiment of the present invention, in order to eliminate the noise effect on the positioning sensor data caused by the change of the projected image content to the ambient light, the projection influence area corresponding to the projection sensor data can be firstly projected, and the positioning sensor data in the projection influence area can be filtered. The projection influence area can be determined by comparing the image features such as texture and color corresponding to the projection image content with the reference features of the positioning sensor data, because the projection image content may be consistent with the background texture in some cases, so the generation of the projection image content does not cause the change of the ambient light.

[0072] Therefore, in yet another embodiment of the present invention, step 1022 further includes:

[0073] Step 221: Analyze the projection sensing data to obtain projection texture information.

[0074] In one embodiment of the present invention, a texture extraction algorithm such as Local Binary Patterns (LBP) may be used to analyze the projection sensing data to obtain projection texture information.

[0075] Step 222: Analyze the positioning sensor data within the target indication area to obtain background texture information.

[0076] In one embodiment of the present invention, a texture extraction algorithm such as Local Binary Patterns (LBP) may be used to analyze the positioning sensor data to obtain background texture information.

[0077] In order to ensure the accuracy of filtering, in another embodiment of the present invention, the algorithm for extracting the background texture information should be the same as the extraction algorithm corresponding to the projection texture information.

[0078] Step 223: Compare the projection texture information with the background texture information to determine the projection impact area.

[0079] In one embodiment of the present invention, the texture distance between the projection texture information and the background texture information in each unit area of ​​the target indication area is calculated, and the areas with texture distances greater than a preset threshold are combined to obtain the projection influence area.

[0080] Step 224: Determine a plurality of original feature points based on the positioning sensing data.

[0081] Feature extraction is performed on the positioning sensor data according to the method described in step 102 to obtain the multiple original feature points.

[0082] Step 225: Filter the original feature points in the projection influence area to obtain the feature points to be matched.

[0083] In one embodiment of the present invention, the original feature points in the projection influence area are deleted, and the original feature points obtained after the deletion are determined as feature points to be matched.

[0084] In another embodiment of the present invention, when the device used to collect positioning sensor data is a laser radar, the method of filtering the projection sensor data in the positioning sensor data may also include the following: determining the emission band of the laser radar and the projection band of the projection device, and filtering the projection band from the positioning sensor data accordingly.

[0085] Optionally, the light beam emitted by the laser radar can be encoded to obtain the coding information of the collected returned light beam, the collected coding information can be matched with the coding corresponding to the laser radar, and the positioning sensor data corresponding to the matched light beam can be determined as the filtered positioning sensor data.

[0086] Step 1023: extract features from the filtered positioning sensor data to obtain the feature points to be matched.

[0087] In one embodiment of the present invention, the manner of extracting features from the filtered positioning sensor data is similar to that in step 102 and will not be described in detail.

[0088] Step 103: Match the feature points to be matched with the target environment feature map to determine the feature matching degree corresponding to the positioning sensor data.

[0089] In one embodiment of the present invention, the target environment may be the environment in which the robot is currently located, such as a preset environment such as a hall, corridor, and doorway, and the target environment feature map includes a plurality of reference feature points corresponding to the target environment. The target environment feature map may be an objective sensor feature map of the target environment created according to a preset mapping algorithm in a SLAM algorithm.

[0090] It should be noted that in order to ensure the accuracy of positioning based on the target environment feature map, the equipment parameters of the sensor equipment for collecting reference feature points when establishing the target environment feature map should be consistent with the equipment parameters of the sensor equipment for collecting positioning sensor data in step 101, wherein the equipment parameters may include the light source wavelength, scanning frequency, accuracy and angular resolution of the lidar, and may also include the resolution, frame rate and accuracy of the depth vision camera.

[0091] In one embodiment of the present invention, the method of matching the feature points to be matched with the reference feature points may be brute force matching, K-dimensional Tree (Kd tree) matching or semantic tree matching.

[0092] In yet another embodiment of the present invention, step 103 further includes:

[0093] Step 1031: respectively calculating the distance between each of the to-be-matched feature points and each of the reference feature points.

[0094] Step 1031 adopts a brute force matching method, that is, each feature point to be matched is compared with the reference feature point one by one, and then the feature point with the smallest difference is taken as a matching pair. When calculating the distance between two feature points, the Euclidean distance or the Hamming distance can be used according to the data type of the descriptor.

[0095] In another embodiment of the present invention, when the number of reference feature points is large, a K-dimensional Tree (Kd tree) matching method can also be adopted, that is, an index is established in the description subset of the above-mentioned feature points to be matched, and the dimension with the largest variance between all points in the current subset is determined as the segmentation dimension, and then the median is selected as the segmentation position according to the sorting of the segmentation dimension, so as to establish a Kd tree, and find the nearest neighbors in the Kd tree, so as to obtain the reference feature points that match each of the feature points to be matched.

[0096] Step 1032: Determine whether each feature point to be matched has a match in the target environment feature map according to the distance.

[0097] In one embodiment of the present invention, when the distance between a feature point to be matched and a reference feature point is less than a distance threshold, the feature point to be matched is determined to be matched in the target environment feature map. The distance threshold can be determined according to the accuracy of the acquisition device of the positioning sensor data.

[0098] Step 1033: Determine the ratio of the number of the to-be-matched feature points that have matches to the total number of the to-be-matched feature points as the feature matching degree.

[0099] For example, the total number of extracted feature points to be matched is 100, and the number of feature points to be matched that are determined to be matched is 60, then the feature matching degree is 60 / 100=60%.

[0100] Step 104: When the feature matching degree is less than the matching degree threshold, and it is determined according to the human body sensing data that a human body exists in the target area, the occlusion ratio of the human body to the feature point to be matched is determined.

[0101] In one embodiment of the present invention, step 104 further includes:

[0102] Step 1041: When the human body sensing data is not empty, determine that the human body exists in the target area.

[0103] In one embodiment of the present invention, the human body sensor data is as described in step 101,

[0104] Step 1042: Analyze the human body sensing data to obtain a human body occlusion area.

[0105] In one embodiment of the present invention, when the human body sensing data is infrared detection data, the infrared detection data determines the infrared detection features corresponding to each sampling point in the target area, and the area where the sampling points whose infrared detection features meet the human body infrared features are located is determined as the human body occlusion area.

[0106] In another embodiment of the present invention, when the human body sensing data is RGB image data, image features are extracted from the RGB image data to determine the connected domain information included in the RGB image data; the connected domain information includes the position of each connected domain, the area of ​​the connected domain, the grayscale information of the pixel point, and the depth information of the pixel point. The position corresponding to the connected domain whose area of ​​the connected domain, the grayscale information of the pixel point, and the depth information of the pixel point meet the human body occlusion image characteristics is determined as the human body occlusion area.

[0107] Step 1043: Determine the to-be-matched feature points within the human body occlusion area as the human body occlusion feature points.

[0108] In yet another embodiment of the present invention, the to-be-matched feature points whose positions fall within the human body occlusion region are determined as the human body occlusion feature points.

[0109] Step 1044: Determine the occlusion ratio according to the ratio of the number of the human body occlusion feature points to the number of the feature points to be matched.

[0110] In one embodiment of the present invention, the ratio of the number of human body occlusion feature points to the number of feature points to be matched is determined as the occlusion ratio. For example, when there are 30 human body occlusion feature points and 50 feature points to be matched, the occlusion ratio is 30 / 50=60%.

[0111] Step 105: When the occlusion ratio is greater than the ratio threshold, an obstacle avoidance reminder is performed.

[0112] In one embodiment of the present invention, a ratio threshold is used to characterize the degree of correlation between human body occlusion and positioning failure. For example, the ratio threshold may be 50%, indicating that if the human body occludes half or more of the feature points to be matched, it can be considered that the decrease in feature matching degree and the resulting positioning failure are caused by human body occlusion.

[0113] In another embodiment of the present invention, when the occlusion ratio is less than the ratio threshold, it means that the main reason for the decrease in feature matching and positioning failure is not human occlusion. Therefore, the robot can be controlled to slow down to a preset value, and the staff can be notified to check on site to determine the specific reason for the robot's positioning failure and avoid collision accidents between the robot and obstacles.

[0114] In one embodiment of the present invention, the obstacle avoidance reminder method may include using various methods such as sound and light in the robot's forward area. The reminder content may be the forward area information of the robot that needs to be avoided, or it may be the relative position relationship between the robot and the obstructing person, thereby prompting the obstructing person with optional avoidance routes.

[0115] Therefore, in one embodiment of the present invention, step 105 further includes:

[0116] Step 1051: Obtain the robot's forward route information.

[0117] In one embodiment of the present invention, the forward route information includes the coordinate position of the forward route in the current environment, the relative position of the forward route with respect to the robot, and the like.

[0118] For example, the forward route information may include moving from the current position to (10, 10) or 10 meters away at 45° to the left front.

[0119] Step 1052: Determine the target indication area and target reminder information according to the front-line route information.

[0120] In one embodiment of the present invention, the target indication area is determined according to the position coordinates of the forward route in the current environment, and the target reminder information is determined according to the relative position.

[0121] For example, the target indication area is a circular area with a diameter of the line from the current position to (10, 10), and the target reminder information generated according to the relative position can be text content displayed in the form of images or audio, such as "You blocked my way, I want to turn left, please pay attention to avoid it." In another embodiment of the present invention, the target reminder information can also be a route map drawn according to the forward route information.

[0122] Step 1053: Generate multimedia content corresponding to the target reminder information; the multimedia content includes projection image content; the target indication area includes the ground area where the robot's forward route is located.

[0123] In one embodiment of the present invention, the multimedia content may include target reminder information displayed in the form of images or audio and / or video content generated according to the target reminder information.

[0124] Step 1054: Play the multimedia content in the target indication area.

[0125] In one embodiment of the present invention, considering that when the environment is noisy, the reception rate of audio content may not be ideal, and people generally check the conditions of the ground under their feet and the road they are about to walk on when walking, therefore, in order to enhance the effect of obstacle avoidance reminder, projection can be performed on the ground to indicate the area where the robot is going to move forward, so that people can perceive the robot's forward route when checking the road surface and take the initiative to avoid it.

[0126] In another embodiment of the present invention, the multimedia content may further include instruction video content, and the target instruction area may be a preset display device, such as a display screen set on the robot. By playing the instruction video content on the display screen, a person can intuitively understand the positional relationship between himself and the robot.

[0127] In another embodiment of the present invention, considering that in a specific environment, such as when the outdoor light is strong, the projected image content on the ground may not be easily recognized by the human eye, therefore, the multimedia content also includes audio content; the audio content may be the voice corresponding to the target prompt information.

[0128] In another embodiment of the present invention, before projection, a voice broadcast reminder can be used. After the audio reminder, if the feature matching degree still does not reach the matching degree threshold, the reminder intensity is increased and the projection indication is performed in the target indication area. Therefore, in another embodiment of the present invention, step 1054 also includes:

[0129] Step 541: Play the audio content in the target indication area.

[0130] In one embodiment of the present invention, an audio playback device is provided in the robot. By directing the sound outlet of the audio playback device toward the target indication area for playback, a better audio reception effect is obtained, thereby increasing the probability of avoidance.

[0131] Step 542: Determine the positioning sensor data after the audio content is played for a preset time as the post-reminder positioning sensor data.

[0132] In one embodiment of the present invention, the preset duration can be zero, that is, while playing the audio content, the positioning sensor data is continuously acquired in real time as the positioning sensor data after the reminder. Optionally, the preset duration can also be 2-3 seconds, that is, the time generally required for a person to make an evasive reaction after receiving the audio content.

[0133] Step 543: Match the post-reminder positioning sensor data with the target environment feature map to determine the post-reminder feature matching degree.

[0134] In one embodiment of the present invention, the process of step 543 is similar to the aforementioned steps 1031 - 1033 and will not be described in detail.

[0135] Step 544: When the post-reminder feature matching degree is less than the matching degree threshold, projecting the projection image content into the target indication area.

[0136] In one embodiment of the present invention, when the feature matching degree after the reminder is less than the matching degree threshold, it can be considered that the reminder strength of the voice reminder is not good and the human body still causes positioning obstacles. Therefore, projection content with stronger reminder strength is used for reminder, thereby improving the effect of obstacle avoidance reminder.

[0137] In another embodiment of the present invention, when the feature matching degree after the reminder is less than the matching degree threshold, the audio content can be continuously played while the projected image content is projected, so that the obstructing human body that causes the positioning failure is reminded by combining audio with projection, thereby further increasing the reminder intensity and improving the obstacle avoidance reminder effect.

[0138] It should be noted that, in another embodiment of the present invention, steps 101-105 of the obstacle avoidance reminder method are executed only when it is determined that the robot is in the positioning and navigation process, and when the robot is in the interactive mode, steps 101-105 are not executed, thereby avoiding the robot from making inappropriate obstacle avoidance reminders when the user stands in front of the robot to interact with it, thereby affecting the user's human-computer interaction experience.

[0139] Different from the prior art that, when a person is detected, an obstacle avoidance reminder is performed, which results in low obstacle avoidance accuracy and poor interactive experience, the robot obstacle avoidance reminder method provided in the embodiment of the present invention first determines the feature matching degree corresponding to the currently acquired positioning sensor data, and determines the positioning failure based on the feature matching degree, and only when it is determined that there is a human body in the target area based on the human body sensor data and the occlusion ratio of the person meets the ratio threshold, an obstacle avoidance reminder is performed for the cause of the positioning failure, that is, the person. This improves the accuracy of the obstacle avoidance reminder on the one hand, and avoids the impact of blind reminders on the interactive experience when the occlusion of the person is not the cause of the positioning failure on the other hand, thereby improving the interactive experience of the obstacle avoidance reminder.

[0140] Figure 2 FIG. 2 shows a schematic diagram of the structure of a robot obstacle avoidance device provided by an embodiment of the present invention. Figure 3 As shown, the device 200 includes: an acquisition module 201 , a first determination module 202 , a matching module 203 , a second determination module 204 and a reminder module 205 .

[0141] Wherein, the acquisition module 201 is used to acquire positioning sensor data and human body sensor data of the target area;

[0142] A first determination module 202, configured to determine a plurality of feature points to be matched according to the positioning sensor data;

[0143] A matching module 203 is used to match the feature points to be matched with the target environment feature map to determine the feature matching degree corresponding to the positioning sensor data;

[0144] A second determination module 204 is configured to determine the occlusion ratio of the feature point to be matched by the human body when the feature matching degree is less than a matching degree threshold and it is determined according to the human body sensing data that there is a human body in the target area;

[0145] The reminder module 205 is used to provide obstacle avoidance reminder when the occlusion ratio is greater than a ratio threshold.

[0146] In an optional manner, the target environment feature map includes a plurality of reference feature points corresponding to the target environment; the first determination module 202 is further used to: respectively calculate the distance between each of the feature points to be matched and each of the reference feature points;

[0147] Determine whether each feature point to be matched exists in the target environment feature map according to the distance;

[0148] The ratio of the number of the to-be-matched feature points that have matches to the total number of the to-be-matched feature points is determined as the feature matching degree.

[0149] In an optional manner, the second determining module 204 is further configured to:

[0150] When the human body sensing data is not empty, determining that the human body exists in the target area;

[0151] Analyzing the human body sensing data to obtain a human body occlusion area;

[0152] Determine the to-be-matched feature points within the human body occlusion area as the human body occlusion feature points;

[0153] The occlusion ratio is determined according to the ratio of the number of the human body occlusion feature points to the number of the feature points to be matched.

[0154] In an optional manner, the reminder module 205 is further used to:

[0155] Obtaining the robot's forward route information;

[0156] Determine the target indication area and target reminder information according to the front-line route information;

[0157] Generate multimedia content corresponding to the target reminder information; the multimedia content includes projection image content; the target indication area includes a ground area where the robot's forward route is located;

[0158] The multimedia content is played in the target indication area.

[0159] In an optional manner, the first determining module 202 is further configured to:

[0160] Determining whether there is projection sensor data corresponding to the projection image content generated by the last obstacle avoidance reminder in the positioning sensor data;

[0161] If so, filtering the positioning sensor data according to the projection sensor data;

[0162] Feature extraction is performed on the filtered positioning sensor data to obtain the feature points to be matched.

[0163] In an optional manner, the first determining module 202 is further configured to:

[0164] Analyzing the projection sensing data to obtain projection texture information;

[0165] Analyzing the positioning sensor data within the target indication area to obtain background texture information;

[0166] Comparing the projection texture information with the background texture information to determine a projection impact area;

[0167] Determine a plurality of original feature points according to the positioning sensor data;

[0168] The original feature points in the projection influence area are filtered to obtain the feature points to be matched.

[0169] In an optional manner, the multimedia content also includes audio content; the reminder module 205 is further used for:

[0170] Play the audio content in the target indication area;

[0171] Determining the positioning sensor data after the audio content is played for a preset time as the post-reminder positioning sensor data;

[0172] Matching the post-reminder positioning sensor data with the target environment feature map to determine the post-reminder feature matching degree;

[0173] When the post-reminder feature matching degree is less than the matching degree threshold, the projection image content is projected into the target indication area.

[0174] Different from the prior art that, when a person is detected, an obstacle avoidance reminder is performed, which results in low obstacle avoidance accuracy and poor interactive experience, the robot obstacle avoidance reminder device provided by the embodiment of the present invention first determines the feature matching degree corresponding to the currently acquired positioning sensor data, and determines the positioning failure based on the feature matching degree, and only when it is determined that there is a human body in the target area based on the human body sensor data and the occlusion ratio of the person meets the ratio threshold, an obstacle avoidance reminder is performed for the cause of the positioning failure, that is, the person. This improves the accuracy of the obstacle avoidance reminder on the one hand, and avoids the impact of blind reminders on the interactive experience when the occlusion of the person is not the cause of the positioning failure on the other hand, thereby improving the interactive experience of the obstacle avoidance reminder.

[0175] Figure 3 A schematic diagram of the structure of a robot obstacle avoidance reminder device provided in an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the robot obstacle avoidance reminder device.

[0176] like Figure 3 As shown, the robot obstacle avoidance reminder device may include: a processor (processor) 302, a communication interface (CommunicationsInterface) 304, a memory (memory) 306, and a communication bus 308.

[0177] The processor 302, the communication interface 304, and the memory 306 communicate with each other via the communication bus 308. The communication interface 304 is used to communicate with other devices such as a client or other server network elements. The processor 302 is used to execute the program 310, which can specifically execute the relevant steps in the above-mentioned embodiment of the robot obstacle avoidance reminder method.

[0178] Specifically, the program 310 may include program code including computer executable instructions.

[0179] The processor 302 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the robot obstacle avoidance reminder device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0180] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0181] Program 310 can be specifically called by processor 302 to enable the robot obstacle avoidance reminder device to perform the following operations:

[0182] Acquire positioning sensor data and human body sensor data of the target area;

[0183] Determine a plurality of feature points to be matched according to the positioning sensor data;

[0184] Matching the feature points to be matched with the target environment feature map to determine the feature matching degree corresponding to the positioning sensor data;

[0185] When the feature matching degree is less than the matching degree threshold, and it is determined according to the human body sensing data that a human body exists in the target area, determining the occlusion ratio of the human body to the feature point to be matched;

[0186] When the occlusion ratio is greater than the ratio threshold, an obstacle avoidance reminder is performed.

[0187] In an optional manner, the target environment feature map includes a plurality of reference feature points corresponding to the target environment; the program 310 is called by the processor 302 to enable the robot obstacle avoidance reminder device to perform the following operations:

[0188] Calculating the distance between each of the to-be-matched feature points and each of the reference feature points respectively;

[0189] Determine whether each feature point to be matched exists in the target environment feature map according to the distance;

[0190] The ratio of the number of the to-be-matched feature points that have matches to the total number of the to-be-matched feature points is determined as the feature matching degree.

[0191] In an optional manner, the program 310 is called by the processor 302 to enable the robot obstacle avoidance reminder device to perform the following operations:

[0192] When the human body sensing data is not empty, determining that the human body exists in the target area;

[0193] Analyzing the human body sensing data to obtain a human body occlusion area;

[0194] Determine the to-be-matched feature points within the human body occlusion area as the human body occlusion feature points;

[0195] The occlusion ratio is determined according to the ratio of the number of the human body occlusion feature points to the number of the feature points to be matched.

[0196] In an optional manner, the program 310 is called by the processor 302 to enable the robot obstacle avoidance reminder device to perform the following operations:

[0197] Obtaining the robot's forward route information;

[0198] Determine the target indication area and target reminder information according to the front-line route information;

[0199] Generate multimedia content corresponding to the target reminder information; the multimedia content includes projection image content; the target indication area includes a ground area where the robot's forward route is located;

[0200] The multimedia content is played in the target indication area.

[0201] In an optional manner, the program 310 is called by the processor 302 to enable the robot obstacle avoidance reminder device to perform the following operations:

[0202] Determining whether there is projection sensor data corresponding to the projection image content generated by the last obstacle avoidance reminder in the positioning sensor data;

[0203] If so, filtering the positioning sensor data according to the projection sensor data;

[0204] Feature extraction is performed on the filtered positioning sensor data to obtain the feature points to be matched.

[0205] In an optional manner, the program 310 is called by the processor 302 to enable the robot obstacle avoidance reminder device to perform the following operations:

[0206] Analyzing the projection sensing data to obtain projection texture information;

[0207] Analyzing the positioning sensor data within the target indication area to obtain background texture information;

[0208] Comparing the projection texture information with the background texture information to determine a projection impact area;

[0209] Determine a plurality of original feature points according to the positioning sensor data;

[0210] The original feature points in the projection influence area are filtered to obtain the feature points to be matched.

[0211] In an optional manner, the multimedia content also includes audio content; the program 310 is called by the processor 302 to enable the robot obstacle avoidance reminder device to perform the following operations:

[0212] Play the audio content in the target indication area;

[0213] Determining the positioning sensor data after the audio content is played for a preset time as the post-reminder positioning sensor data;

[0214] Matching the post-reminder positioning sensor data with the target environment feature map to determine the post-reminder feature matching degree;

[0215] When the post-reminder feature matching degree is less than the matching degree threshold, the projection image content is projected into the target indication area.

[0216] Different from the prior art that, when a person is detected, an obstacle avoidance reminder is performed, which results in low obstacle avoidance accuracy and poor interactive experience, the robot obstacle avoidance reminder device provided by the embodiment of the present invention first determines the feature matching degree corresponding to the currently acquired positioning sensor data, and determines the positioning failure based on the feature matching degree, and only when it is determined that there is a human body in the target area based on the human body sensor data and the occlusion ratio of the person meets the ratio threshold, an obstacle avoidance reminder is performed for the cause of the positioning failure, that is, the person. This improves the accuracy of the obstacle avoidance reminder on the one hand, and avoids the impact of blind reminders on the interactive experience when the occlusion of the person is not the cause of the positioning failure on the other hand, thereby improving the interactive experience of the obstacle avoidance reminder.

[0217] An embodiment of the present invention provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction runs on a robot obstacle avoidance reminder device, the robot obstacle avoidance reminder device executes the robot obstacle avoidance reminder method in any of the above method embodiments.

[0218] The executable instructions can be specifically used to enable the robot obstacle avoidance reminder device to perform the following operations:

[0219] Acquire positioning sensor data and human body sensor data of the target area;

[0220] Determine a plurality of feature points to be matched according to the positioning sensor data;

[0221] Matching the feature points to be matched with the target environment feature map to determine the feature matching degree corresponding to the positioning sensor data;

[0222] When the feature matching degree is less than the matching degree threshold, and it is determined according to the human body sensing data that a human body exists in the target area, determining the occlusion ratio of the human body to the feature point to be matched;

[0223] When the occlusion ratio is greater than the ratio threshold, an obstacle avoidance reminder is performed.

[0224] In an optional manner, the target environment feature map includes a plurality of reference feature points corresponding to the target environment; and the executable instruction enables the robot obstacle avoidance reminder device to perform the following operations:

[0225] Calculating the distance between each of the to-be-matched feature points and each of the reference feature points respectively;

[0226] Determine whether each feature point to be matched exists in the target environment feature map according to the distance;

[0227] The ratio of the number of the to-be-matched feature points that have matches to the total number of the to-be-matched feature points is determined as the feature matching degree.

[0228] In an optional manner, the executable instruction enables the robot obstacle avoidance reminder device to perform the following operations:

[0229] When the human body sensing data is not empty, determining that the human body exists in the target area;

[0230] Analyzing the human body sensing data to obtain a human body occlusion area;

[0231] Determine the to-be-matched feature points within the human body occlusion area as the human body occlusion feature points;

[0232] The occlusion ratio is determined according to the ratio of the number of the human body occlusion feature points to the number of the feature points to be matched.

[0233] In an optional manner, the executable instruction enables the robot obstacle avoidance reminder device to perform the following operations:

[0234] Obtaining the robot's forward route information;

[0235] Determine the target indication area and target reminder information according to the front-line route information;

[0236] Generate multimedia content corresponding to the target reminder information; the multimedia content includes projection image content; the target indication area includes a ground area where the robot's forward route is located;

[0237] The multimedia content is played in the target indication area.

[0238] In an optional manner, the executable instruction enables the robot obstacle avoidance reminder device to perform the following operations:

[0239] Determining whether there is projection sensor data corresponding to the projection image content generated by the last obstacle avoidance reminder in the positioning sensor data;

[0240] If so, filtering the positioning sensor data according to the projection sensor data;

[0241] Feature extraction is performed on the filtered positioning sensor data to obtain the feature points to be matched.

[0242] In an optional manner, the executable instruction enables the robot obstacle avoidance reminder device to perform the following operations:

[0243] Analyzing the projection sensing data to obtain projection texture information;

[0244] Analyzing the positioning sensor data within the target indication area to obtain background texture information;

[0245] Comparing the projection texture information with the background texture information to determine a projection impact area;

[0246] Determine a plurality of original feature points according to the positioning sensor data;

[0247] The original feature points in the projection influence area are filtered to obtain the feature points to be matched.

[0248] In an optional manner, the multimedia content also includes audio content; and the executable instructions enable the robot obstacle avoidance reminder device to perform the following operations:

[0249] Play the audio content in the target indication area;

[0250] Determining the positioning sensor data after the audio content is played for a preset time as the post-reminder positioning sensor data;

[0251] Matching the post-reminder positioning sensor data with the target environment feature map to determine the post-reminder feature matching degree;

[0252] When the post-reminder feature matching degree is less than the matching degree threshold, the projection image content is projected into the target indication area.

[0253] Different from the problem of low obstacle avoidance accuracy and poor interactive experience caused by the solution of obstacle avoidance reminder when a person is detected in the prior art, the computer storage medium provided by the embodiment of the present invention first determines the feature matching degree corresponding to the currently acquired positioning sensor data, and determines the positioning failure according to the feature matching degree, and only when it is determined according to the human body sensor data that there is a human body in the target area and the occlusion ratio of the person meets the ratio threshold, it performs obstacle avoidance reminder for the cause of the positioning failure, that is, the person. On the one hand, this improves the accuracy of the obstacle avoidance reminder, and on the other hand, avoids the impact of blind reminders on the interactive experience when the occlusion of the person is not the cause of the positioning failure, thereby improving the interactive experience of the obstacle avoidance reminder.

[0254] An embodiment of the present invention provides a robot obstacle avoidance reminder device, which is used to execute the above-mentioned robot obstacle avoidance reminder method.

[0255] An embodiment of the present invention provides a computer program, which can be called by a processor to enable a robot obstacle avoidance reminder device to execute the robot obstacle avoidance reminder method in any of the above method embodiments.

[0256] An embodiment of the present invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are run on a computer, the computer executes the robot obstacle avoidance reminder method in any of the above method embodiments.

[0257] The algorithm or display provided herein is not inherently related to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description made to specific languages ​​above is for disclosing the best mode of the present invention.

[0258] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0259] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than those expressly recited in each claim.

[0260] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0261] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A robot obstacle avoidance reminder method, characterized in that: The method is applied to a robot; the method comprises: Acquire positioning sensor data and human body sensor data of the target area; Determine a plurality of feature points to be matched according to the positioning sensor data, wherein the projection sensor data corresponding to the projection image content generated by the last obstacle avoidance reminder has been filtered out from the positioning sensor data; Matching the feature points to be matched with the target environment feature map to determine the feature matching degree corresponding to the positioning sensor data; When the feature matching degree is less than the matching degree threshold, and it is determined according to the human body sensing data that a human body exists in the target area, determining the occlusion ratio of the human body to the feature point to be matched; When the occlusion ratio is greater than the ratio threshold, an obstacle avoidance reminder is performed.

2. The method according to claim 1, characterized in that The target environment feature map includes a plurality of reference feature points corresponding to the target environment; matching the feature points to be matched with the target environment feature map to determine the feature matching degree corresponding to the positioning sensor data includes: Calculating the distance between each of the to-be-matched feature points and each of the reference feature points respectively; Determine whether each feature point to be matched exists in the target environment feature map according to the distance; The ratio of the number of the to-be-matched feature points that have matches to the total number of the to-be-matched feature points is determined as the feature matching degree.

3. The method according to claim 1, characterized in that When the feature matching degree is less than a matching degree threshold and it is determined according to the human body sensing data that a human body exists in the target area, determining the occlusion ratio of the human body to the feature point to be matched includes: When the human body sensing data is not empty, determining that the human body exists in the target area; Analyzing the human body sensing data to obtain a human body occlusion area; Determine the to-be-matched feature points within the human body occlusion area as the human body occlusion feature points; The occlusion ratio is determined according to the ratio of the number of the human body occlusion feature points to the number of the feature points to be matched.

4. The method according to claim 1, characterized in that: When the occlusion ratio is greater than the ratio threshold, an obstacle avoidance reminder is performed, including: Obtaining the robot's forward route information; Determine the target indication area and target reminder information according to the forward route information; Generate multimedia content corresponding to the target reminder information; the multimedia content includes projection image content; the target indication area includes a ground area where the robot's forward route is located; The multimedia content is played in the target indication area.

5. The method according to claim 4, characterized in that The acquiring of positioning sensor data and determining a plurality of feature points to be matched according to the positioning sensor data include: Determining whether there is projection sensor data corresponding to the projection image content generated by the last obstacle avoidance reminder in the positioning sensor data; If so, filtering the positioning sensor data according to the projection sensor data; Feature extraction is performed on the filtered positioning sensor data to obtain the feature points to be matched.

6. The method according to claim 5, characterized in that The filtering of the positioning sensor data according to the projection sensor data comprises: Analyzing the projection sensing data to obtain projection texture information; Analyzing the positioning sensor data within the target indication area to obtain background texture information; Comparing the projection texture information with the background texture information to determine a projection impact area; Determine a plurality of original feature points according to the positioning sensor data; The original feature points in the projection influence area are filtered to obtain the feature points to be matched.

7. The method according to claim 4, characterized in that The multimedia content also includes audio content; and playing the multimedia content in the target indication area further includes: Play the audio content in the target indication area; Determining the positioning sensor data after the audio content is played for a preset time as the post-reminder positioning sensor data; Matching the post-reminder positioning sensor data with the target environment feature map to determine the post-reminder feature matching degree; When the post-reminder feature matching degree is less than the matching degree threshold, the projection image content is projected into the target indication area.

8. A robot obstacle avoidance reminder device, characterized in that: The device comprises: An acquisition module is used to acquire positioning sensor data and human body sensor data of a target area; A first determination module is used to determine a plurality of feature points to be matched according to the positioning sensor data, wherein the projection sensor data corresponding to the projection image content generated by the last obstacle avoidance reminder has been filtered out from the positioning sensor data; A matching module, used to match the feature points to be matched with the target environment feature map, and determine the feature matching degree corresponding to the positioning sensor data; A second determination module is used to determine the occlusion ratio of the human body to the feature point to be matched when the feature matching degree is less than a matching degree threshold and it is determined according to the human body sensing data that there is a human body in the target area; The reminder module is used to provide obstacle avoidance reminder when the occlusion ratio is greater than a ratio threshold.

9. A robot obstacle avoidance reminder device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the robot obstacle avoidance reminder method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction runs on the robot obstacle avoidance reminder device, the robot obstacle avoidance reminder device performs the operation of the robot obstacle avoidance reminder method according to any one of claims 1 to 7.

Images