Robots, devices and methods for detecting pedestrians by robots, and readable storage media

Abstract

A robot, a method, an apparatus, and a computer-readable storage medium for detecting pedestrians are disclosed, comprising a robot memory and a processor coupled to the memory. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring multiple frames of radar data in a detection area of ​​a robot lidar, the detection area including a static pedestrian detection area and a dynamic pedestrian detection area; processing each frame of radar data in the static pedestrian detection area using a first type of algorithm and processing each frame of radar data in the dynamic pedestrian detection area using a second type of algorithm to obtain pedestrian information to be tracked in the detection area; and determining a target pedestrian based on the correlation information between the pedestrian information to be tracked in the multiple frames of radar data, thereby improving the real-time performance and accuracy of the robot's pedestrian detection.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to a robot, a device, a method for detecting pedestrians using a robot, and a readable storage medium. Background Technology

[0002] Pedestrian detection and tracking has always been a hot research topic in the field of robotics. With the rapid development of LiDAR, low-cost 2D LiDAR has gradually been applied to the market, leading to the emergence of many pedestrian detection methods based on 2D LiDAR.

[0003] In existing technologies, pedestrian detection algorithms are basically designed based on deep learning schemes. Such methods generally rely on a large amount of data, have a large computational load, and low timeliness. Summary of the Invention

[0004] This application provides a robot, a device, a method, and a readable storage medium for detecting pedestrians using a robot, which can solve the problem of low timeliness in robot pedestrian detection.

[0005] One embodiment of this application provides a robot, including: a memory and a processor coupled to the memory; the memory stores a computer program; the processor is used to execute the computer program to perform the following steps:

[0006] Acquire multiple frames of radar data from the detection area of ​​the robot's LiDAR, the detection area including a static pedestrian detection area and a dynamic pedestrian detection area;

[0007] The radar data of each frame in the static pedestrian detection area is processed by a first type of algorithm and the radar data of each frame in the dynamic pedestrian detection area is processed by a second type of algorithm to obtain the pedestrian information to be tracked in the detection area.

[0008] The target pedestrian is determined based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data.

[0009] One embodiment of this application provides a device for detecting pedestrians using a robot, comprising:

[0010] The acquisition module is used to acquire multiple frames of radar data in the detection area of ​​the robot's LiDAR, the detection area including a static pedestrian detection area and a dynamic pedestrian detection area;

[0011] The identification module is used to process each frame of radar data in the static pedestrian detection area using a first type of algorithm and to process each frame of radar data in the dynamic pedestrian detection area using a second type of algorithm, so as to obtain the pedestrian information to be tracked in the detection area.

[0012] The determination module is used to determine the target pedestrian based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data.

[0013] One embodiment of this application provides a method for a robot to detect pedestrians, including:

[0014] Acquire radar data in the detection area of ​​the robot's LiDAR, the detection area including a static pedestrian detection area and a dynamic pedestrian detection area;

[0015] Acquire multiple frames of radar data from the detection area of ​​the robot's LiDAR, the detection area including a static pedestrian detection area and a dynamic pedestrian detection area;

[0016] The radar data of each frame in the static pedestrian detection area is processed by a first type of algorithm and the radar data of each frame in the dynamic pedestrian detection area is processed by a second type of algorithm to obtain the pedestrian information to be tracked in the detection area.

[0017] The target pedestrian is determined based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data.

[0018] One aspect of this application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements a method for a robot to detect pedestrians. The method for the robot to detect pedestrians is the same as the method described above for a robot to detect pedestrians.

[0019] As can be seen from the above embodiments of this application, the robot acquires radar data in the detection area of ​​the LiDAR. The detection area is divided into a static pedestrian detection area and a dynamic pedestrian detection area. A first type of algorithm is used to process each frame of radar data in the static pedestrian detection area, and a second type of algorithm is used to process each frame of radar data in the dynamic pedestrian detection area. This allows the robot to identify the pedestrians to be tracked within the detection area. Subsequently, based on the correlation information between the pedestrian information of the pedestrians to be tracked in the multiple frames of radar data, the target pedestrian is determined. By using different processing algorithms for different areas, the overall computational load is small, the real-time performance is high, and the accuracy of identifying real pedestrians is high. Furthermore, by distinguishing different detection areas based on the characteristics of the interaction scenario, different interaction strategies can be applied based on the identified static and dynamic pedestrians, thereby improving the success rate of the interaction. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the structure of a robot provided in one embodiment of this application;

[0022] Figure 2 A flowchart illustrating a method for detecting pedestrians with a robot according to an embodiment of this application;

[0023] Figure 3 This is a schematic diagram illustrating the division of the detection area of ​​the robot in a method for detecting pedestrians provided in an embodiment of this application.

[0024] Figure 4 A flowchart illustrating a method for detecting pedestrians with a robot, provided in another embodiment of this application;

[0025] Figure 5 This is a schematic diagram of the rasterization process of the robot's grid map coordinate system in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of the structure of a robot for detecting pedestrians provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] See Figure 1 , Figure 1 This is a schematic diagram of the structure of a robot provided in one embodiment of this application. For ease of explanation, only the parts relevant to this embodiment are shown. The robot is equipped with a lidar, which serves as a pedestrian detection sensor. It scans the detection area in real time to obtain radar data. This radar data allows the robot to determine leg information within the scanned area, match pedestrians based on the leg information, and determine pedestrian positions and speeds. The robot may include a memory 100 and a processor 200 coupled to the memory 100.

[0029] The processor 200 is the core of the mobile robot's computation and control, and is the final execution unit for information processing and program execution. The memory 100 includes, for example, hard disk drive memory, non-volatile memory (such as flash memory or other electronically programmable, erasable memory used to form a solid-state drive), volatile memory (such as static or dynamic random access memory), etc., and is not limited in this embodiment.

[0030] The memory 100 stores a computer program; the processor 200 executes the computer program to implement the steps in the following method for robot pedestrian detection, see [link to relevant documentation]. Figure 2 The method includes the following steps:

[0031] S201. Acquire multiple frames of radar data in the detection area of ​​the robot's LiDAR, which includes a static pedestrian detection area and a dynamic pedestrian detection area;

[0032] The scanning detection area can be a 360° omnidirectional area centered on the robot, or it can be the area in front of the robot's movement. For ease of description, this application embodiment uses the detection area in front of the robot's movement as an example for illustration. Figure 3 As shown, the detection area in front of the robot is divided into two fan-shaped areas with different radii centered on the robot, which are designated as the static pedestrian detection area and the dynamic pedestrian detection area, respectively. The fan-shaped area with the smaller radius is the static pedestrian detection area, and the fan-shaped area with the larger radius is the dynamic pedestrian detection area.

[0033] In an optional embodiment, multiple frames of radar data of the detection area are continuously acquired sequentially.

[0034] Specifically, the static pedestrian detection area is closer to the robot than the dynamic pedestrian detection area; that is, the static pedestrian detection area is closer to the robot than the dynamic pedestrian detection area. Furthermore, the dynamic pedestrian detection area is larger than the static pedestrian detection area.

[0035] S202. Process each frame of radar data in the static pedestrian detection area using a first type of algorithm and process each frame of radar data in the dynamic pedestrian detection area using a second type of algorithm to obtain the pedestrian information to be tracked in the detection area.

[0036] Subsequently, the first type of algorithm is used to process each frame of radar data in the static pedestrian detection area, and the second type of algorithm is used to process each frame of radar data in the dynamic pedestrian detection area, so as to identify the pedestrian to be tracked and obtain the tracking information of the pedestrian to be tracked.

[0037] Optionally, the information to be tracked may specifically include the location and speed of the information to be tracked.

[0038] Optionally, the pedestrian to be tracked can be one or more, which can specifically include dynamic pedestrians and static pedestrians. The identified pedestrian to be tracked may be identified from the radar data corresponding to the static pedestrian detection area and can be called a static pedestrian, or it may be identified from the radar data corresponding to the dynamic pedestrian detection area and can be called a dynamic pedestrian. It can also include multiple pedestrians identified from the radar data corresponding to the static pedestrian detection area and the radar data corresponding to the dynamic pedestrian detection area. There is no limitation here.

[0039] In the optional scenario, since the static pedestrian detection area is closer to the robot than the dynamic pedestrian detection area, pedestrians in the static area are generally more likely to observe the robot and are in a relatively static state. Furthermore, the static pedestrian area typically does not contain static obstacles such as chairs or tables that could cause detection confusion, so the first type of algorithm can be used. However, the dynamic pedestrian detection area is relatively far from the robot. On the one hand, it may contain obstacles such as tables and chairs that could cause confusion; on the other hand, pedestrians in the dynamic area are generally in motion. Therefore, the second type of algorithm can be used, the specific algorithm of which is explained below.

[0040] S203. Based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data, the target pedestrian is determined.

[0041] In an optional embodiment, the pedestrian information to be tracked includes the pedestrian's position and speed. The target pedestrian is determined from among the pedestrians based on correlation information between the pedestrian information of the pedestrians to be tracked in consecutive multi-frame radar data. The correlation information includes the number of successful matching associations.

[0042] Optionally, when a pedestrian to be tracked is detected in the m-th frame of radar data, the position and information of the pedestrian to be tracked are used to predict the position of the pedestrian to be tracked in the next frame of radar data. The predicted position is then matched with the actual position of the pedestrian to be tracked in the next frame of radar data using the minimum distance method. That is, the distance between the predicted position and the actual position of the pedestrian to be tracked is compared with the distance between the predicted position of the pedestrian to be tracked and the distance between the predicted position of the pedestrian to be tracked and other pedestrians to be tracked. If the distance is the minimum distance, the matching association is confirmed to be successful, and the number of successful matching associations for the pedestrian to be tracked is incremented by 1. If the distance is not the minimum distance, the matching association is confirmed to be unsuccessful, and the number of successful matching associations is decremented by 1 or reduced by a preset number, or the pedestrian to be tracked is directly deleted. When the number of successful matching associations for a pedestrian to be tracked reaches the preset number, it is confirmed as a target pedestrian, that is, it is finally confirmed as a real pedestrian.

[0043] In other scenarios, if no pedestrian to be tracked is detected in the next frame of radar data, that is, the pedestrian information to be tracked is an empty set by default, then matching the predicted location with the pedestrian information to be tracked (empty set) will also be considered a failure of matching association.

[0044] In other embodiments, the associated information can also be obtained in other ways. For example, the pedestrian information to be tracked also includes the location and area of ​​the pedestrian to be tracked. The location and area of ​​the pedestrian to be tracked are compared using the radar data of two adjacent frames. For example, if the location distance is less than a preset value and the area ratio is within a preset range, the matching association is considered successful. If the location distance is not less than a preset value and the area ratio is not within a preset range, the matching association is considered to have failed.

[0045] In other embodiments, conventional methods can also be used to obtain the correlation information between the pedestrian information to be tracked in multiple frames of radar data, which is not limited here.

[0046] In this embodiment, the robot acquires radar data from the detection area of ​​a LiDAR, which is divided into a static pedestrian detection area and a dynamic pedestrian detection area. A first type of algorithm is used to process each frame of radar data in the static pedestrian detection area, and a second type of algorithm is used to process each frame of radar data in the dynamic pedestrian detection area. This allows the robot to identify the pedestrians to be tracked within the detection area. Subsequently, based on the correlation information between the pedestrian information of the pedestrians to be tracked in the multiple frames of radar data, the target pedestrian is determined. By employing different processing algorithms for different areas, the overall computational load is low, real-time performance is high, and the accuracy of identifying real pedestrians is high. Furthermore, by differentiating pedestrians in different detection areas according to the characteristics of the interaction scenario, different interaction strategies can be applied based on the identified static and dynamic pedestrians, improving the success rate of the interaction.

[0047] See Figure 4 The processor 200 calls the computer program stored in the memory 100 and executes step S202 of the robot pedestrian detection method: processing each frame of radar data in the static pedestrian detection area using a first type of algorithm, including:

[0048] Using a pre-defined clustering algorithm, such as DBSCAN or K-MEANS, radar data in the grid map coordinate system of the static pedestrian detection area are clustered into clusters. Clusters with an area greater than a preset area are identified as human legs. Each identified human leg is traversed, and the two human legs with the smallest distance that is less than a preset distance are paired. Two successfully paired human legs correspond to one pedestrian to be tracked (static pedestrian), and a single unpaired human leg corresponds to one pedestrian to be tracked (static pedestrian).

[0049] The processor 200 calls the computer program stored in the memory 100 and executes step S202 of the robot pedestrian detection method: processing each frame of radar data in the dynamic pedestrian detection area using a second type of algorithm, specifically including:

[0050] Based on the radar's installation location on the robot, determine the transformation matrix between the radar and the robot's data. Using this transformation matrix, establish a grid map coordinate system for the robot, with the origin of the grid map coordinate system being the robot's center position, and the X-axis coinciding with the central axis of the detection area. S302. Based on the radar data, obtain the attribute values ​​of each grid cell in the grid map coordinate system.

[0051] Specifically, each grid cell in the raster map coordinate system within the detection area corresponds to one pixel, see [link to relevant documentation]. Figure 5 The attribute values ​​of each grid cell are calculated frame by frame. Specifically, for each received frame of radar data, the attribute value of each grid cell is calculated. Grids containing radar data have their attribute values ​​incremented by a statistic, while grids without radar data have their attribute values ​​decremented by a statistic. For example, the initial value of each grid cell in the grid map coordinate system is 0. Within the robot's detection area, the attribute value of grids containing radar data is incremented by 1, and the attribute value of grids not containing radar data is decremented by 1. The maximum attribute value is 255, and the minimum is 0. Outside the robot's detection area, the attribute values ​​of the grid cells remain unchanged.

[0052] Furthermore, when the number of frames counting the attribute values ​​of each grid cell in the grid map coordinate system reaches a preset number of frames, such as 20 frames, dynamic pedestrian detection begins. Using a preset clustering algorithm, the radar data in the grid map coordinate system within the dynamic pedestrian detection area is clustered into groups. Clusters with an area larger than a preset area are identified as human legs. Each identified human leg is traversed, and the two legs with the smallest distance (less than a preset distance) are paired. Two successfully paired human legs correspond to one unpaired dynamic pedestrian, while a single unpaired human leg corresponds to one unpaired dynamic pedestrian. The sum of the attribute values ​​of all grid cells within a preset range of the unpaired dynamic pedestrian is calculated. If this sum is less than a preset threshold, the unpaired dynamic pedestrian is confirmed as the pedestrian to be tracked (dynamic pedestrian).

[0053] When performing dynamic pedestrian detection, since the positions of obstacles such as chairs and walls are fixed, their radar data continuously appears in the corresponding grids, and the attribute values ​​of these grids will continuously increase to a large value. However, pedestrians are usually moving dynamically, and the attribute values ​​of their corresponding grids may continuously decrease, or repeatedly increase and decrease, resulting in a smaller attribute value.

[0054] The preset range for the unidentified dynamic pedestrian refers to the preset range of the center of the unidentified dynamic pedestrian. Specifically, the center of the two legs of the unidentified dynamic pedestrian is the center of the unidentified dynamic pedestrian after successful pairing, and the center of the single leg of the unidentified dynamic pedestrian is the center of the unidentified dynamic pedestrian. For example, with the center of the unidentified dynamic pedestrian as the center, the attribute values ​​of all grids covered by a circle with a radius of 0.3 meters are summed. If the sum is less than the preset threshold, the unidentified dynamic pedestrian is identified as the pedestrian to be tracked (dynamic pedestrian).

[0055] Further, the processor 200 calls the computer program stored in the memory 100 to execute step S202 of the robot pedestrian detection method: the step of determining the target pedestrian based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data includes:

[0056] Static and dynamic pedestrians in each frame of the radar data are considered as pedestrians to be tracked. Using a preset tracking algorithm, based on the position and speed of the pedestrian to be tracked in each frame of the radar data, the predicted position of the pedestrian to be tracked in the next frame of the radar data is predicted. The predicted position is then matched with the actual position of the pedestrian to be tracked in the next frame of the radar data using minimum distance matching. If the matching is successful, the number of times the matching of the pedestrian to be tracked is successful is recorded. When the number of times the matching of the pedestrian to be tracked is successful reaches a first preset number, the pedestrian to be tracked is confirmed as the target pedestrian.

[0057] If the association fails, the pedestrian to be tracked detected in the next frame of radar data will be added as a new pedestrian. The number of successful associations of the pedestrian to be tracked will be reduced by a statistic, and the number of association failures will be recorded. When the number of association failures reaches a second preset number, it will be confirmed that the pedestrian to be tracked is not the target pedestrian.

[0058] For example, if pedestrian A, who is to be tracked in the current frame of radar data, is successfully associated with pedestrian A, who is to be tracked in the next frame of radar data, the count of successful association for A is incremented by 1. If the association fails, A is added as a new pedestrian, a new ID is assigned to A, and the count of successful association for A is recorded as 1. At the same time, the count of successful association for A is decremented by 1, and the count of failed association is incremented by 1. When the count of successful association between A and A reaches 5, they are confirmed as the target pedestrians. When the count of failed association for A reaches 5 first, A is confirmed as not being the target pedestrian, and A's record data is deleted.

[0059] Further, after the processor 200 calls the computer program stored in the memory 100 and executes step S203 of the robot pedestrian detection method—tracking and updating the positions of the static pedestrian and the dynamic pedestrian in real time, and confirming the target pedestrian based on the position changes of the static pedestrian and the dynamic pedestrian—the following steps are included:

[0060] S301. Determine the interaction strategy based on the type of the target pedestrian.

[0061] If the target pedestrian is a pedestrian to be tracked detected in the static pedestrian detection area, that is, a pedestrian to be tracked (static pedestrian) is identified in the static pedestrian detection area, then the static pedestrian closest to the robot will be used as the interaction object.

[0062] Specifically, when only one static pedestrian is detected, that static pedestrian is used as the current interaction object. When multiple static pedestrians are detected, the static pedestrian closest to the robot among the multiple static pedestrians is used as the current interaction object.

[0063] If the target pedestrian is a pedestrian to be tracked (dynamic pedestrian) detected in the dynamic pedestrian detection area, that is, a pedestrian to be tracked (dynamic pedestrian) is identified in the dynamic pedestrian detection area, then the dynamic pedestrian with the most successful associations will be used as the interaction object.

[0064] Specifically, when a single dynamic pedestrian is detected, that dynamic pedestrian is used as the current interaction object. When multiple dynamic pedestrians are detected, the dynamic pedestrian with the most successful associations is used as the current interaction object.

[0065] If no static pedestrians are detected, but a dynamic pedestrian is detected, the interaction object can be determined based on the future interaction position between the dynamic pedestrian and the robot.

[0066] Specifically, the future position of the dynamic pedestrian among the target pedestrians is calculated. Based on the robot's position and speed, it is determined whether to use this future position as an interaction location. If, based on the dynamic pedestrian's future position and the robot's position and speed, it is determined that the robot cannot meet the dynamic pedestrian at this future position, then this future position is not used as an interaction location, and the interaction may be abandoned. If, based on the dynamic pedestrian's future position and the robot's position and speed, it is determined that the robot can meet the dynamic pedestrian at this future position, and based on the currently interacting content, the estimated interaction duration is determined. The future position is then updated in real time based on the interaction duration and the robot's position and speed. The updated future position is then used to determine whether the robot can meet the dynamic pedestrian at this updated future position, and whether to use this updated future position as an interaction location. These steps are repeated until it is determined that the robot cannot meet the dynamic pedestrian at this future position; if so, this future position is not used as an interaction location, and the interaction is abandoned.

[0067] In this embodiment, the robot acquires radar data from the detection area of ​​the LiDAR, which is divided into a static pedestrian detection area and a dynamic pedestrian detection area. The radar data is converted into radar data in a grid map coordinate system and then rasterized to obtain the attribute values ​​of the converted radar data in each grid. A clustering algorithm is used to determine the human legs in the converted radar data. Static pedestrians in the static pedestrian detection area are obtained based on the human leg matching principle, and dynamic pedestrians in the dynamic pedestrian detection area are obtained based on the same human leg matching principle and the grid attribute values. By tracking and updating the positions of the static and dynamic pedestrians in real time, the robot obtains the position changes of the static and dynamic pedestrians and confirms the target pedestrian based on the position changes. This method involves low computational load, high real-time performance, and high accuracy in identifying real pedestrians. Furthermore, it distinguishes different detection areas based on the characteristics of the interaction scenario to identify pedestrians, and applies different interaction strategies to static and dynamic pedestrians within the target pedestrian group, making the interaction selection more flexible and improving the success rate of the interaction.

[0068] See Figure 6 This application provides a schematic diagram of a robot pedestrian detection device according to an embodiment. For ease of explanation, only the parts relevant to this embodiment are shown. This device can be installed in a robot. The device includes:

[0069] The acquisition module 401 is used to acquire multiple frames of radar data in the detection area of ​​the robot's LiDAR, the detection area including a static pedestrian detection area and a dynamic pedestrian detection area;

[0070] The identification module 402 is used to process each frame of radar data in the static pedestrian detection area using a first type of algorithm and to process each frame of radar data in the dynamic pedestrian detection area using a second type of algorithm, so as to obtain the pedestrian information to be tracked in the detection area.

[0071] The determination module 403 is used to determine the target pedestrian based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data.

[0072] Furthermore, the identification module 402 is also used to cluster the radar data in the static pedestrian detection area into various clusters using a preset clustering algorithm;

[0073] Clusters with an area greater than a preset area are identified as human legs;

[0074] Pair the two legs of each person whose distance is the smallest and is less than a preset distance. The two legs that are successfully paired correspond to one static pedestrian, and the single leg that is not successfully paired corresponds to one static pedestrian.

[0075] Furthermore, the device also includes:

[0076] The processing module is used to convert the acquired radar data into a pre-established grid map coordinate system based on the transformation matrix between the radar and the robot. The origin of the grid map coordinate system is the center position of the robot, and the X-axis coincides with the central axis of the detection area.

[0077] The attribute values ​​of each grid are counted on a frame-by-frame basis. For grids containing radar data, the attribute value is incremented by a statistic, and for grids not containing radar data, the attribute value is decremented by a statistic.

[0078] The identification module 402 is also used to cluster the radar data in the grid map coordinate system in the dynamic pedestrian detection area into clusters when the number of statistical frames of attribute values ​​of each grid in the grid map coordinate system reaches a preset number of frames, by using the preset clustering algorithm.

[0079] Clusters with an area greater than the preset area are identified as human legs;

[0080] Pair the two legs of each person whose distance is the smallest and is less than a preset distance. The two legs that are successfully paired correspond to one undetermined dynamic pedestrian, and the single leg that is not successfully paired corresponds to one undetermined dynamic pedestrian.

[0081] Calculate the sum of the attribute values ​​of all the grids within the preset range of the undetermined dynamic pedestrian. If the sum of the attribute values ​​is less than a preset threshold, then the undetermined dynamic pedestrian is confirmed as the dynamic pedestrian.

[0082] The determination module 403 is also used to identify the static and dynamic pedestrians in each frame of the multi-frame radar data as pedestrians to be tracked.

[0083] Based on the position and speed of the pedestrian to be tracked in each frame of radar data, predict the predicted position of the pedestrian to be tracked in the next frame of radar data;

[0084] The predicted location is correlated with the actual location of the pedestrian to be tracked in the next frame of radar data using a minimum distance matching method.

[0085] If the association is successful, the number of times the pedestrian to be tracked is successfully associated is recorded. When the number of times the pedestrian to be tracked is successfully associated reaches the first preset number, the pedestrian to be tracked is confirmed as the target pedestrian.

[0086] The determination module 403 is also used to, if the association fails, take the pedestrian to be tracked detected in the next frame of radar data as a new pedestrian, subtract a statistic from the number of successful associations of the pedestrian to be tracked, and record the number of association failures. When the number of association failures reaches a second preset number, it is confirmed that the pedestrian to be tracked is not the target pedestrian.

[0087] Furthermore, the device also includes:

[0088] The interaction strategy module is used to select the pedestrian (static pedestrian) closest to the robot as the interaction object if the target pedestrian is the pedestrian to be tracked (static pedestrian) identified in the static pedestrian detection area.

[0089] If the target pedestrian is the same pedestrian (dynamic pedestrian) identified in the dynamic pedestrian detection area, then the dynamic pedestrian with the most successful associations will be used as the interaction object.

[0090] The technical details of this embodiment are described in the foregoing embodiments and will not be repeated here.

[0091] In this embodiment, the acquisition module acquires radar data from the detection area of ​​the robot's LiDAR. The detection area is divided into a static pedestrian detection area and a dynamic pedestrian detection area. The recognition module identifies static pedestrians based on the radar data from the static pedestrian detection area and detects dynamic pedestrians based on the radar data from the dynamic pedestrian detection area. Based on multi-frame radar data, the module tracks and updates the information of the static and dynamic pedestrians to be tracked in real time to determine the target pedestrian. This method has low computational load, high real-time performance, and high accuracy in recognizing real pedestrians. Furthermore, it distinguishes different detection areas to identify pedestrians according to the characteristics of the interaction scenario, which facilitates different interaction strategies based on the identified static and dynamic pedestrians, thereby improving the success rate of interaction.

[0092] Furthermore, embodiments of the present invention also provide a computer-readable storage medium, which may be disposed in the robot of the above embodiments, and the computer-readable storage medium may be as described above. Figure 1 The memory 100 in the illustrated embodiment. A computer program is stored on this computer-readable storage medium, which, when executed by a processor, implements the aforementioned... Figure 2 and Figure 4 The method for detecting pedestrians by a robot is described in the illustrated embodiment. Furthermore, the computer storage medium can also be any medium capable of storing program code, such as a USB flash drive, external hard drive, read-only memory (ROM), RAM, magnetic disk, or optical disk.

[0093] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0094] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0095] The above is a description of the robot, the device and method for detecting pedestrians by the robot provided by the present invention, and the readable storage medium. For those skilled in the art, based on the ideas of the embodiments of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A robot, characterized in that, The robot includes: a memory and a processor coupled to the memory; the memory stores a computer program; the processor is used to execute the computer program to perform the following steps: Acquire multiple frames of radar data from the detection area of ​​the robot's LiDAR, the detection area including a static pedestrian detection area and a dynamic pedestrian detection area; The radar data of each frame in the static pedestrian detection area is processed by a first type of algorithm and the radar data of each frame in the dynamic pedestrian detection area is processed by a second type of algorithm to obtain the pedestrian information to be tracked in the detection area. The target pedestrian is determined based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data. The step of processing each frame of radar data in the static pedestrian detection area using a first type of algorithm includes: The radar data in the static pedestrian detection area are clustered into clusters using a preset clustering algorithm. Clusters with an area greater than a preset area are identified as human legs; The two legs with the smallest distance between them, which is less than a preset distance, are paired up. The two legs that are successfully paired correspond to one static pedestrian, and the single leg that is not successfully paired corresponds to one pedestrian to be tracked.

2. The robot according to claim 1, characterized in that, The static pedestrian detection area is closer to the robot than the dynamic pedestrian detection area, and the dynamic pedestrian detection area is larger than the static pedestrian detection area.

3. The robot according to claim 1, characterized in that, The process of processing each frame of radar data in the dynamic pedestrian detection area using the second type of algorithm includes: Based on the transformation matrix between the radar and the robot, the acquired radar data is converted into a pre-established grid map coordinate system; The attribute values ​​of each grid in the grid map coordinate system are counted on a frame-by-frame basis. Specifically, the attribute values ​​of grids containing radar data are incremented by a statistic, and the attribute values ​​of grids not containing radar data are decremented by a statistic. When the number of statistical frames of attribute values ​​of each grid in the grid map coordinate system reaches a preset number of frames, the radar data in the grid map coordinate system in the dynamic pedestrian detection area is clustered into various clusters by a preset clustering algorithm. Clusters with an area greater than the preset area are identified as human legs; Pair the two legs with the smallest distance that is less than a preset distance. The two legs that are successfully paired correspond to one undetermined dynamic pedestrian, and the single leg that is not successfully paired corresponds to one undetermined dynamic pedestrian. Calculate the sum of attribute values ​​of all grids within the preset range of the undetermined dynamic pedestrian. If the sum of the attribute values ​​is less than a preset threshold, then the undetermined dynamic pedestrian is identified as the pedestrian to be tracked.

4. The robot according to claim 3, characterized in that, The pedestrian information to be tracked includes the location and speed of the pedestrian to be tracked, and the association information includes the number of successful associations; The step of determining the target pedestrian based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data includes: Based on the position and speed of the pedestrian to be tracked in each frame of radar data, predict the predicted position of the pedestrian to be tracked in the next frame of radar data; The predicted location is matched and associated with the actual location of the pedestrian to be tracked in the next frame of radar data using minimum distance matching. If the association is successful, the number of times the pedestrian to be tracked is successfully associated is recorded, and when the number of times the pedestrian to be tracked is successfully associated reaches a first preset number, the pedestrian to be tracked is confirmed as the target pedestrian.

5. The robot according to claim 4, characterized in that, The processor is also configured to perform the following steps: If the association fails, the pedestrian to be tracked detected in the radar data of the next frame is added as a new pedestrian. The number of successful associations of the pedestrian to be tracked is reduced by a statistic, and the number of association failures is recorded. When the number of association failures reaches a second preset number, it is confirmed that the pedestrian to be tracked is not the target pedestrian.

6. The robot according to claim 4, characterized in that, The processor is also configured to perform the following steps: If the target pedestrian is the pedestrian to be tracked identified in the static pedestrian detection area, then the pedestrian to be tracked who is closest to the robot will be the interaction object for interaction; If the target pedestrian is the pedestrian to be tracked identified in the dynamic pedestrian detection area, then the pedestrian to be tracked with the most successful associations will be used as the interaction object for interaction.

7. A device for detecting pedestrians using a robot, characterized in that, include: The acquisition module is used to acquire multiple frames of radar data in the detection area of ​​the robot's LiDAR, the detection area including a static pedestrian detection area and a dynamic pedestrian detection area; The identification module is used to process each frame of radar data in the static pedestrian detection area using a first type of algorithm and to process each frame of radar data in the dynamic pedestrian detection area using a second type of algorithm, so as to obtain the pedestrian information to be tracked in the detection area. The determination module is used to determine the target pedestrian based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data; The identification module is used to cluster radar data in the static pedestrian detection area into clusters using a preset clustering algorithm; to identify human legs in each cluster whose area is greater than a preset area; and to pair the two human legs with the smallest distance that is less than a preset distance. The two human legs that are successfully paired correspond to one static pedestrian, and the single human leg that is not successfully paired corresponds to one pedestrian to be tracked.

8. A method for detecting pedestrians with a robot, characterized in that, include: Acquire multiple frames of radar data from the detection area of ​​the robot's LiDAR, the detection area including a static pedestrian detection area and a dynamic pedestrian detection area; The radar data of each frame in the static pedestrian detection area is processed by a first type of algorithm and the radar data of each frame in the dynamic pedestrian detection area is processed by a second type of algorithm to obtain the pedestrian information to be tracked in the detection area. The target pedestrian is determined based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data. The step of processing each frame of radar data in the static pedestrian detection area using a first type of algorithm includes: The radar data in the static pedestrian detection area are clustered into clusters using a preset clustering algorithm. Clusters with an area greater than a preset area are identified as human legs; The two legs with the smallest distance between them, which is less than a preset distance, are paired up. The two legs that are successfully paired correspond to one static pedestrian, and the single leg that is not successfully paired corresponds to one pedestrian to be tracked.

9. The method according to claim 8, characterized in that, The static pedestrian detection area is closer to the robot than the dynamic pedestrian detection area, and the dynamic pedestrian detection area is larger than the static pedestrian detection area.

10. The method according to claim 8, characterized in that, The process of processing each frame of radar data in the dynamic pedestrian detection area using the second type of algorithm includes: Based on the transformation matrix between the radar and the robot, the acquired radar data is converted into a pre-established grid map coordinate system; The attribute values ​​of each grid in the grid map coordinate system are counted on a frame-by-frame basis. Specifically, the attribute values ​​of grids containing radar data are incremented by a statistic, and the attribute values ​​of grids not containing radar data are decremented by a statistic. When the number of statistical frames of attribute values ​​of each grid in the grid map coordinate system reaches a preset number of frames, the radar data in the grid map coordinate system in the dynamic pedestrian detection area is clustered into various clusters by a preset clustering algorithm. Clusters with an area greater than the preset area are identified as human legs; Pair the two legs with the smallest distance that is less than a preset distance. The two legs that are successfully paired correspond to one undetermined dynamic pedestrian, and the single leg that is not successfully paired corresponds to one undetermined dynamic pedestrian. Calculate the sum of attribute values ​​of all grids within the preset range of the undetermined dynamic pedestrian. If the sum of the attribute values ​​is less than a preset threshold, then the undetermined dynamic pedestrian is identified as the pedestrian to be tracked.

11. The method according to claim 10, characterized in that, The pedestrian information to be tracked includes the location and speed of the pedestrian to be tracked, and the association information includes the number of successful associations; The step of determining the target pedestrian based on the correlation information between the pedestrian information to be tracked in the multi-frame radar data includes: Based on the position and speed of the pedestrian to be tracked in each frame of radar data, predict the predicted position of the pedestrian to be tracked in the next frame of radar data; The predicted location is matched and associated with the actual location of the pedestrian to be tracked in the next frame of radar data using minimum distance matching. If the association is successful, the number of times the pedestrian to be tracked is successfully associated is recorded, and when the number of times the pedestrian to be tracked is successfully associated reaches a first preset number, the pedestrian to be tracked is confirmed as the target pedestrian.

12. The method according to claim 11, characterized in that, The step of performing minimum distance matching and association between the predicted location and the actual location of the pedestrian to be tracked in the next frame of radar data includes: If the association fails, the pedestrian to be tracked detected in the radar data of the next frame is added as a new pedestrian. The number of successful associations of the pedestrian to be tracked is reduced by a statistic, and the number of association failures is recorded. When the number of association failures reaches a second preset number, it is confirmed that the pedestrian to be tracked is not the target pedestrian.

13. The method according to claim 11, characterized in that, After confirming that the pedestrian to be tracked is the target pedestrian, the following steps are included: If the target pedestrian is the pedestrian to be tracked identified in the static pedestrian detection area, then the pedestrian to be tracked who is closest to the robot will be the interaction object for interaction; If the target pedestrian is the pedestrian to be tracked identified in the dynamic pedestrian detection area, then the pedestrian to be tracked with the most successful associations will be used as the interaction object for interaction.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is used to implement a method for a robot to detect pedestrians when executed by a processor, wherein the method for a robot to detect pedestrians is the method for implementing the robot to detect pedestrians as described in any one of claims 8-13.

Images