Method for finding a person and electronic device

Abstract

Embodiments of the present application provide a person searching method and electronic equipment, and relate to the technical field of terminals. The method comprises: in response to a first instruction, determining a first room in a first map. In the process of navigating to a plurality of observation points in the first room in turn, if first human body information is collected, determining whether the orientation of a first user and the orientation of a robot device satisfy a first angle condition based on the first human body information. If not, determining a target point in the first map according to the first human body information and the accessible information corresponding to each of a plurality of positions in the first map. Navigating to the target point and collecting second human body information of the first user. In the case of determining that the first user is a target user based on the second human body information, performing an interactive operation corresponding to the first instruction. In this way, the robot can flexibly adjust its own position according to the positional relationship between the robot and the user, accurately identify the user, and thus improve the probability of finding the target user.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method for finding missing persons and an electronic device. Background Technology

[0002] With the continuous advancement of science and technology, intelligent technology is increasingly permeating people's daily lives, giving rise to intelligent robot devices that provide people with various services such as intelligent missing person search, greatly improving their quality of life.

[0003] In current implementations, some algorithms can control a robot to navigate to a designated location and collect user information by rotating 360°. The collected facial image can then be used to determine if the user is the target user. However, even with this method, it's still possible to fail to locate the target user. Summary of the Invention

[0004] This application provides a method and electronic device for locating people, applicable to the field of terminal technology, to increase the probability of finding the target user.

[0005] Firstly, embodiments of this application propose a method for locating missing persons. This method includes:

[0006] In response to a first instruction, a first room is determined in a first map, wherein the first instruction is used to indicate the search for a target user, and the first map includes multiple rooms;

[0007] During the process of navigating to multiple observation points in the first room, if the first human body information is collected, then based on the first human body information, it is determined whether the orientation of the first user and the orientation of the robot device meet the first angle condition. The first user is the user corresponding to the first human body information.

[0008] If not, then based on the first human body information and the reachability information corresponding to each of the multiple locations in the first map, the target point is determined in the first map. The reachability information is used to indicate whether the robot device can reach the location.

[0009] Navigate to the target point and collect the second human body information of the first user;

[0010] If the first user is identified as the target user based on the second human body information, the interactive operation corresponding to the first instruction is executed.

[0011] In this embodiment, by navigating sequentially to multiple observation points on the first map, the room can be scanned efficiently, and the location of potential users can be quickly determined, improving the efficiency of finding target users. Subsequently, if a valid facial image of the target user is not acquired during the initial observation, the robot's position can be dynamically adjusted based on the acquired first human body information. This flexibility allows the robot to navigate more effectively in complex environments, thereby increasing the probability of finding the target user. Furthermore, utilizing the reachability information in the first map, the robot can intelligently plan its path, avoid obstacles, and ensure safe and smooth movement.

[0012] In one possible implementation, the first human body information includes the first user's first location on the first map and the first user's first orientation;

[0013] Based on the first human body information, determine whether the orientation of the first user and the orientation of the robot device meet the first angle condition, including:

[0014] Determine the orientation angle based on the first orientation and the second orientation of the robot equipment;

[0015] The orientation angle is compared with the first angle range to determine whether the orientation of the first user and the orientation of the robot equipment meet the first angle condition.

[0016] The first angular condition includes: the included angle of orientation falls within the first angular range.

[0017] In this implementation, a first angle condition is set to determine whether the robot needs path adjustment. Calculating the orientation angle and comparing it with a preset angle range allows for a relatively accurate determination of whether the user's orientation and the robot's orientation meet the first angle condition. By adjusting the first angle range, the first angle condition can be flexibly defined to adapt to different application scenarios and needs. Furthermore, this method avoids unnecessary path adjustments, improving navigation efficiency.

[0018] In one possible implementation, the first human information includes the first user's first location in the first map;

[0019] Based on the first human body information and the reachability information corresponding to multiple locations in the first map, the target point is determined in the first map, including:

[0020] A first range is determined in the first map based on the first location, with the first location as the center and a first length as the radius;

[0021] Among the multiple locations included in the first range, candidate points are determined, and the reachability information of the candidate points is used to indicate that the robot equipment can reach the location corresponding to the candidate point;

[0022] Select the target point from the candidate points.

[0023] In this implementation, the user's location is determined using initial human information. A range centered on this location is then defined to ensure the robot can reach the user's vicinity for further facial data collection. Among multiple locations within this range, candidate points are identified. The reachability information of these candidate points indicates which locations the robot can reach, ensuring that the robot only attempts to navigate to safely accessible locations, reducing collisions and other potential risks.

[0024] In one possible implementation, the method further includes:

[0025] If there are no candidate points within the first range, a second range is determined on the first map based on the first location. The first range is centered on the first location and has a radius of the second length, which is greater than the first length.

[0026] Among the multiple locations included in the second range, candidate points are determined, and the reachability information of the candidate points is used to indicate the location corresponding to the target point that the robot device can reach;

[0027] Select the target point from the candidate points.

[0028] In this implementation, if no reachable point is found within the initial range, the system automatically expands to a larger range, increasing the probability of finding the target point and ensuring that the robot can eventually approach the user's location. By dynamically adjusting the size of the search range, the system can adapt to different environmental complexities and user location uncertainties, improving the system's flexibility and adaptability.

[0029] In one possible implementation, determining the target point from the candidate points includes:

[0030] For any candidate point, determine the angle between the first line connecting the candidate point and the first orientation corresponding to the first user. The first line is the line connecting the candidate point and the first position.

[0031] The candidate point with the smallest included angle is selected as the target point.

[0032] In this implementation, by selecting the candidate point with the smallest included angle as the target point, the robot device can collect the user's facial information more completely, thus more accurately determining whether the user is the target user.

[0033] In one possible implementation, the second human body information includes the facial image of the first user;

[0034] The method also includes:

[0035] Determine the quality information of the face image;

[0036] If the face image is determined to meet the preset conditions for face recognition based on the quality information, then the face image is compared with the preset image of the target user to determine whether the first user is the target user.

[0037] In this implementation, if the facial image is of good quality, it means that the image can accurately reflect the facial information of the first user. Then, by comparing it with a preset image of the target user, it can be determined with relatively high accuracy whether the first user is the target user.

[0038] In one possible implementation, the method further includes:

[0039] If, based on the quality information, it is determined that the face image does not meet the preset conditions for face recognition, then the first question voice is played, which is used to ask the first user whether they are the target user.

[0040] In response to the first received voice reply, determine whether the first user is the target user.

[0041] In this implementation, if the facial image is of poor quality, it means that the image cannot effectively reflect the facial information of the first user, and therefore, it is impossible to determine the identity of the first user. By playing a first questioning voice to ask the first user if they are the target user, it is possible to accurately determine whether the first user is the target user.

[0042] In one possible implementation, the method further includes:

[0043] For any one of the multiple observation points, after navigating to the observation point, the first component of the robot device is controlled to rotate by a second angle to determine whether the first human information has been collected.

[0044] The first component includes a camera unit, and the second angle is less than 360°.

[0045] In this implementation, after navigating to the observation point, rotating the camera unit at each observation point covers a wider field of view, thereby increasing the probability of detecting the user and the likelihood of acquiring initial human information. Compared to rotating 360°, selecting a second angle less than 360° for scanning reduces unnecessary rotation, saving energy and time and improving system efficiency. By limiting the rotation angle, the time required for each scan is reduced, allowing the system to complete detection and response more quickly, improving overall response speed. The size of the second angle can be adjusted according to the environment and needs to adapt to different room layouts and user positions, improving the system's adaptability and flexibility.

[0046] In one possible implementation, the method further includes:

[0047] After navigating to the observation point, determine a third range on the first map, centered on the observation point and with a radius of the third length;

[0048] Within the multiple locations included in the third range, a reference location marked as the first mark is determined, the first mark being used to indicate that the reference location has not been observed by the robotic device;

[0049] Based on the reference position, the second position of the robot device in the first map, and the second orientation of the robot device, determine the rotation angle corresponding to the reference position. The rotation angle is the angle that the robot device needs to rotate to face the reference position.

[0050] The second angle is determined based on the rotation angles corresponding to multiple reference positions and the field of view angle of the robot device.

[0051] In this implementation, by identifying unobserved reference positions in the first map and calculating the required rotation angle of the robot, the observation strategy can be optimized to ensure that uncovered areas are effectively observed. By identifying and marking unobserved reference positions, the system can more comprehensively cover the target area, reduce blind spots, and improve the comprehensiveness and accuracy of observation. Determining the second angle based on the rotation angle of the reference position and the field of view angle avoids unnecessary rotation, saves time and energy, and improves the system's resource utilization efficiency.

[0052] Secondly, embodiments of this application provide a person-finding device, which can be an electronic device, or a chip or chip system within an electronic device. The person-finding device may include a display unit and a processing unit.

[0053] When the missing person locator is an electronic device, the display unit can be a display screen. The display unit is used to perform the display step so that the electronic device implements a missing person locator method described in the first aspect or any possible implementation of the first aspect.

[0054] When the missing person locator is an electronic device, the processing unit may be a processor. The missing person locator may also include a storage unit, which may be a memory. The storage unit stores instructions, and the processing unit executes the instructions stored in the storage unit to cause the electronic device to implement a missing person locator method described in the first aspect or any possible implementation thereof.

[0055] When the missing person locator is a chip or chip system within an electronic device, the processing unit can be a processor. The processing unit executes instructions stored in the storage unit to cause the electronic device to implement a missing person locator method described in the first aspect or any possible implementation of the first aspect. The storage unit can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip within the electronic device (e.g., a read-only memory, random access memory, etc.).

[0056] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory for storing code instructions, and the processor for running the code instructions to perform the methods described in the first aspect or any possible implementation of the first aspect.

[0057] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program or instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0058] Fifthly, embodiments of this application provide a computer program product including a computer program, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0059] Sixthly, this application provides a chip or chip system including at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a circuit. The at least one processor is used to run computer programs or instructions to perform the methods described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, pins, or circuits, etc.

[0060] In one possible implementation, the chip or chip system described above in this application further includes at least one memory storing instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0061] It should be understood that the second to sixth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of an application scenario for the missing person search method provided in the embodiments of this application. Figure 1;

[0063] Figure 2 This is a schematic diagram of an application scenario for the missing person search method provided in the embodiments of this application. Figure 2 ;

[0064] Figure 3 This is a schematic diagram of an application scenario for the missing person search method provided in the embodiments of this application. Figure 3 ;

[0065] Figure 4 A flowchart illustrating a method for finding missing persons provided in an embodiment of this application;

[0066] Figure 5 This is a schematic diagram of the process for generating observation points provided in an embodiment of this application;

[0067] Figure 6 A schematic diagram of a navigation route based on observation points provided in an embodiment of this application;

[0068] Figure 7 A schematic diagram illustrating the setting of the first angle range provided in an embodiment of this application;

[0069] Figure 8 A schematic diagram illustrating the fulfillment of the first angle condition provided in an embodiment of this application;

[0070] Figure 9 A schematic diagram illustrating an embodiment of this application where the first angle condition is not met;

[0071] Figure 10 This is a flowchart illustrating the process of determining a target user, provided in an embodiment of this application.

[0072] Figure 11 This is a schematic diagram of the determination of target points provided in an embodiment of this application;

[0073] Figure 12 This is a schematic diagram of navigation to a target point provided in an embodiment of this application. Figure 1 ;

[0074] Figure 13 This is a schematic diagram of navigation to a target point provided in an embodiment of this application. Figure 2 ;

[0075] Figure 14 This is a schematic diagram of setting observation markers provided in the embodiments of this application. Figure 1 ;

[0076] Figure 15 This is a schematic diagram of setting observation markers provided in the embodiments of this application. Figure 2 ;

[0077] Figure 16 This is a schematic diagram of setting observation markers provided in the embodiments of this application. Figure 3 ;

[0078] Figure 17 A schematic diagram of calculating the second angle provided for embodiments of this application. Figure 1 ;

[0079] Figure 18 A schematic diagram of calculating the second angle provided for embodiments of this application. Figure 2 ;

[0080] Figure 19 This is a schematic diagram of the structure of the person-finding device provided in the embodiments of this application;

[0081] Figure 20 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0082] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0083] 1. Posture

[0084] In robot vision systems, human pose refers to the position and orientation of the human body in space, often referred to as posture. The definition of posture includes two key components: position and orientation.

[0085] The position (x, y, z) is used to represent the spatial coordinates (x, y, z) of the human body from the robot's perspective, where x can refer to the left and right direction (relative to the robot), y can refer to the front and back direction (relative to the robot), and z can refer to the up and down direction (ground height).

[0086] Direction describes the orientation and posture of the human body and can be represented by Euler angles (yaw, pitch, roll) or quaternions. Specifically, yaw angle represents the direction the body is facing, its rotation around the vertical axis (the angle of left or right turn), and can be calculated based on the forward / backward direction vector of the torso. Pitch angle represents the rotation around the horizontal axis (the angle of forward or backward tilt), and can be calculated based on the vector angle from the neck to the head. Roll angle represents the rotation around the body's forward / backward axis (the angle of left or right tilt), and can be calculated based on the height difference between the left and right shoulders.

[0087] 2. Point cloud data

[0088] In this embodiment, after scanning the room with a camera, point cloud data can be used to reconstruct the positions of walls and furniture, achieving 3D reconstruction of the interior space for robot navigation. Point cloud data is a way of representing 3D objects or scenes, typically consisting of a series of points with spatial positions (X, Y, Z coordinates) and possibly additional attribute information (such as color, reflection intensity, normal direction, etc.). Each point has a precise location in 3D space, and the set of these points can describe the shape, structure, or geometric features of the entire environment.

[0089] Point cloud data in three-dimensional space represents a collection of discrete points and has the following characteristics: nearby objects have dense point clouds with rich details (such as desktop textures); distant objects have sparse point clouds that show outlines (such as the diagonal walls of a room); and the point cloud colors match the real scene, for example, a blue sofa and green plants can be easily distinguished.

[0090] When point cloud data is acquired using cameras, it is typically obtained through devices such as depth cameras, LiDAR, or structured light sensors. These devices can acquire the spatial location of each point on the surface of an object or in a scene and present the data of these points in the form of a point cloud.

[0091] 3. Other terms

[0092] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and purpose. For example, "first chip" and "second chip" are used only to distinguish different chips and do not limit their order of execution. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0093] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0094] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0095] 4. Electronic equipment

[0096] In this embodiment, the electronic device includes a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and main memory. The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows.

[0097] To better understand the technical solution of this application, the relevant technologies involved in this application will be further described in detail below.

[0098] With the continuous advancement of science and technology, intelligent technology is increasingly permeating people's daily lives, greatly improving their quality of life. Intelligent robots, as an important product of intelligent technology development, have been widely applied in various fields, with service robots being particularly prominent in both home and commercial applications. With the continuous improvement of human-computer interaction technology, mobile home service robots are gradually becoming mainstream, providing users with convenient services.

[0099] The following is combined with Figures 1 to 3 Let me introduce the application scenarios of the methods for finding missing persons. Figure 1 This is a schematic diagram of an application scenario for the missing person search method provided in the embodiments of this application. Figure 1 , Figure 2 This is a schematic diagram of an application scenario for the missing person search method provided in the embodiments of this application. Figure 2 , Figure 3 This is a schematic diagram of an application scenario for the missing person search method provided in the embodiments of this application. Figure 3 .

[0100] The application scenarios for robots to find missing persons can be roughly divided into the following three scenarios. In one scenario, such as... Figure 1 As shown, assume there is currently a user A and a robot device B. User A can wake up robot device A and send instructions to it, causing robot device A to locate the target user A.

[0101] In a scenario, such as Figure 2 As shown, user A can wake up robot device A and send instructions to robot device A, causing robot device A to find target user B.

[0102] In one scenario, user A can also wake up robot device A and send instructions to it, causing robot device A to locate target user B and perform relevant operations. For example, such as... Figure 3 As shown, user A can send a command to robot device A to deliver the book "Design Aesthetics" to target user B. To execute this command, robot device A needs to locate target user B and deliver the book "Design Aesthetics" to them.

[0103] Based on the application scenarios described above, the following section will further explain the specific implementation of the missing persons search method.

[0104] In one approach, the potential location of the target user can first be predicted based on map information and the target user's historical locations. Then, this predicted location is set as a target navigation point, and the robot is guided to navigate to it. Upon reaching the target navigation point, the robot can rotate 360° to locate the target user. Once the robot acquires the user's facial information, it can then use this information to determine if the user is indeed the target user.

[0105] However, this approach may lead to the following problems: Firstly, in complex scenarios, the robot searches for the target user by rotating in a fixed position, but obstacles may obstruct the view between the robot and the user. For example, in a home environment, the user might be hiding behind a bookshelf, which could prevent the robot from effectively locating the target user. Secondly, in reality, the user might be facing away from the robot, or the user might be wearing facial accessories (such as masks, face masks, or face coverings), which could lead to errors in user identification and prevent the robot from accurately locating the target user.

[0106] To address the problems described above, this application proposes the following technical concept: First, predict the room where the target user might appear. Then, control a robot device to navigate within this room based on pre-generated observation points. During navigation, user information can be collected. Once the robot has collected user information, it can determine whether to adjust the robot's position to reach the user's vicinity based on the positional relationship between the user and the robot device, enabling the robot device to accurately identify the user. If the target user is not found in the current room, the room where the target user might appear can be predicted again, and a search operation can be performed within that room until the target user is found.

[0107] The person-finding method in this application embodiment can be executed by an electronic device equipped with a camera, or by a chip, chip system, or processor that supports the implementation of the person-finding method in the electronic device, or by a logic module or software that can implement all or part of the functions of the electronic device. This application does not impose any specific limitations on this.

[0108] The technical solutions of the embodiments of this application and how the technical solutions of the embodiments of this application solve the above-mentioned technical problems will be described in detail below with reference to the accompanying drawings and specific examples. The following specific embodiments can be implemented independently or in combination with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0109] The technical solution of this application relates to a method for finding missing persons, which will be described below. The process of the method will be explained with reference to specific embodiments. The following will be combined with… Figures 4 to 13 A detailed introduction will be given, including, Figure 4 This is a flowchart illustrating a method for finding missing persons provided in an embodiment of this application. Figure 5 This is a schematic diagram of the process for generating observation points provided in an embodiment of this application. Figure 6 This is a schematic diagram of a navigation route based on observation points provided in an embodiment of this application. Figure 7 This is a schematic diagram illustrating the setting of the first angle range provided in an embodiment of this application. Figure 8 This is a schematic diagram illustrating the fulfillment of the first angle condition in an embodiment of this application. Figure 9 This is a schematic diagram illustrating an embodiment of this application where the first angle condition is not met. Figure 10 This is a schematic diagram of the process for determining the target user provided in an embodiment of this application. Figure 11 This is a schematic diagram of determining the target point provided in an embodiment of this application. Figure 12 This is a schematic diagram of navigation to a target point provided in an embodiment of this application. Figure 1 , Figure 13 This is a schematic diagram of navigation to a target point provided in an embodiment of this application. Figure 2 .

[0110] First, you can refer to... Figure 4 To understand the technical solution of this application, such as... Figure 4 As shown, upon receiving the first instruction from the user, the robot responds by predicting the location of the target user's room to determine the first room. It then navigates to this first room, traversing it while simultaneously performing a search. During this traversal, if at least one first user is found, their identity is verified to determine if they are the target user. Otherwise, the robot continues traversing the first room and performing the search until all rooms have been successfully traversed.

[0111] When identifying the first user, if it can be determined that the first user is the target user, then relevant operations can be executed according to the first instruction. For example, this could be giving the book "Design Aesthetics" to the target user. If it is determined that the first user is not the target user, then it is necessary to determine whether the robot has already traversed the first room. If it can be determined that it has already traversed the first room, this means that the target user was not found in the current first room. Therefore, the operation of predicting the first room and searching for the target user within the first room, as well as the operation of identifying the first user in the first room, can be repeated.

[0112] exist Figure 4 Based on the introduction, the following will combine... Figure 10 The steps illustrated herein describe the missing person retrieval method provided in this application. In one implementation, the missing person retrieval method may include, for example, the following steps:

[0113] S1000: Obtain a first map of the home environment and generate observation points for each room in the first map.

[0114] For initially inactive robot devices, once the user activates the device, they can control the robot to move within the home environment via remote control or autonomous navigation mode to maximize coverage of all functional areas of the house, thereby building a basic environmental model (original map). It should be noted that due to inherent sensor errors (such as lidar ranging drift and ultrasonic sensor multipath interference) or dynamic environmental interference (such as swaying curtains and pet movement), the original map may exhibit distortions such as blurred obstacle boundaries, false alarm areas, or spatial misalignment.

[0115] In practical implementation, a first map can be obtained based on the original map of the home environment. This first map can be used to perform subsequent operations. In this embodiment, the first map can be, for example, a raster map, a domain map, or a topology map. The following description will use a raster map as an example. All locations within the home environment can be considered as a single raster map. The size of a raster cell in the raster map can be, for example, set to 5 cm x 5 cm.

[0116] In this embodiment, after the robotic device obtains the grid map, observation points can be generated for each room in the grid map. These observation points are used to guide the robotic device in navigation. After generating these observation points, they can be stored in the robotic device.

[0117] To improve the usability of raster maps, feasible regions can be pruned, invalid regions removed, and a binarized image generated. For example, the original point cloud data can be filtered based on a sensor confidence threshold (recommended to be above 85%) to remove discrete noise points in low-confidence regions. Then, morphological closing operations can be used to connect broken regions, and irregular obstacle contours can be corrected using a convex hull algorithm to generate a continuous feasible space topology map. Finally, the processed raster map can be converted into a binary representation, where white pixels (255) represent passable regions and black pixels (0) mark impassable regions.

[0118] Next, the contour skeleton of the binarized image can be extracted to determine base points in the raster map. In one implementation, a morphological thinning algorithm can first be used to reduce connected regions in the binary image to centerlines to extract the centerline or skeleton of the home environment from the binary image. Then, the skeleton pixels can be converted into a set of coordinate points, and a Delaunay triangulation algorithm can be used to construct a Delaunay triangulation network based on the skeleton points. Next, based on the Delaunay triangulation network, the circumcenter (Vino vertex) of each triangle can be calculated to generate a Vino map. Finally, specific points (such as region centers or feature points) can be extracted from the Vino map as base points.

[0119] After determining the base points in the grid map, a greedy algorithm can be used to calculate the minimum set of points covering the visible area of ​​each room, thus generating observation points for each room within the home environment. Since the robot's sensing range is limited, the observation points need to be determined based on this range. For example, the robot's effective sensing range can be set to 2.5 meters. This means that sensors (such as cameras, LiDAR, etc.) can effectively identify targets within this range; that is, for a point to be considered coverable, it must be within 2.5 meters of at least one observation point. Furthermore, the robot's effective field of view angle is conical, meaning that the robot can effectively identify targets within a specific angular range.

[0120] The basic idea of ​​a greedy algorithm is to make locally optimal choices at each step in order to eventually obtain the globally optimal solution. In this problem, the goal of the greedy strategy is to minimize the number of observation points while ensuring that the entire room is effectively covered. The steps of the greedy algorithm are briefly introduced below.

[0121] First, select an initial uncovered region, typically starting from a corner of a room or a random location. Within this uncovered region, choose a base point as the observation point, such that the coverage circle at that point covers as much of the uncovered area as possible. Then, update the uncovered region by subtracting the coverage circle of the newly selected observation point from the uncovered region. Next, repeat the process of selecting a base point as the observation point and updating the uncovered region until all regions are covered. Finally, output the set of selected observation points.

[0122] The following is combined Figure 5 A brief introduction to the process of generating observation points.

[0123] like Figure 5 As shown in (a), assuming the robot has scanned the home environment, an initial map is obtained. This initial map indicates four rooms, within which obstacles (such as tables, bookshelves, etc.) may exist, represented by the black rectangular areas shown in the diagram. Based on the above, it can be determined that a first map (e.g., a grid map) can be obtained from the initial map. Then, multiple base points can be determined in the grid map, such as... Figure 5 As shown in (b) above. Then, a greedy algorithm can be used to process multiple base points to obtain observation points, such as... Figure 5 As shown in (c) in the diagram. Figure 5 In the illustration of (c), taking room A as an example, after processing the base points in room A, four base points can be determined. Through these four base points, the robot device can observe all positions in room A.

[0124] S1001, In response to the first instruction, determine the first room in the first map.

[0125] The first map in this embodiment can be, for example, a grid map, which will be used as an example below. In practice, a grid map may include multiple rooms. Therefore, during the search for a person, the robotic device may need to search multiple rooms to determine the location of the target user. To shorten the search time, the robotic device can first predict the room where the target user is most likely to appear and perform the search operation in that room. If the target user cannot be found in that room, the prediction can be made again, and the search operation can be performed again.

[0126] In one implementation, the first room where the target user is most likely to be located can be predicted based on the target user's historical location information collected by the robotic device. For example, the room where the target user last appeared can be determined as the first room. Alternatively, the room where the target user might currently be located can be predicted based on the current time and historical usage habits. For example, assuming the current time is 12 o'clock, the robotic device can query previously collected location information of the target user. If it is found that the target user frequently appears in the kitchen at 12 o'clock, then the kitchen can be determined as the first room to search.

[0127] S1002. During the sequential navigation to multiple observation points in the first room, determine whether the first human body information has been collected. If yes, proceed to S1003. If no, continue navigation, and determine whether the first human body information has been collected during the navigation process; if the first human body information has been collected, perform subsequent operations based on the first human body information; if the first human body information has not been collected, continue navigation until the first room has been traversed. Traversing the first room can be understood as the robot device sequentially navigating to each observation point. The first human body information may include the first user's first position in the grid map and the first user's first orientation.

[0128] S1003. Based on the first human body information, determine whether the orientation of the first user and the orientation of the robot device meet the first angle condition. If yes, collect the second human body information of the first user and execute S1006. If no, execute S1004. The second human body information may include the face image of the first user.

[0129] Steps S1002 and S1003 will be described together below. In a robot vision system, once the robot successfully detects a user, its pose can be accurately acquired. Pose, as a key parameter describing the spatial state of an object, can include two core dimensions: position and orientation. Position refers to the user's three-dimensional spatial coordinates (x, y, z) in the robot's coordinate system, used to determine the absolute spatial position of the human body relative to the robot, providing basic coordinate information for motion planning.

[0130] Orientation is used to describe the human body's posture (orientation and pose), and is usually represented by Euler angles. Euler angles contain three rotational degrees of freedom: yaw, pitch, and roll. Yaw can be used to describe the human body's horizontal orientation (e.g., facing the robot / back to the robot), pitch can be used to reflect the forward / backward tilt of the human torso, and roll can be used to characterize the human body's left / right tilt.

[0131] In this embodiment, the first human body information may include the first location of the first user in the grid map and the first orientation of the first user. The first user is the user corresponding to the first human body information. The first orientation may refer to the orientation of the human body on the horizontal plane, i.e., the yaw angle.

[0132] In this embodiment, the robotic device can sequentially navigate to multiple observation points in the first room. For example, refer to... Figure 6 To understand this, assume there are four observation points in the first room: observation point 1, observation point 2, observation point 3, and observation point 4. After the robot enters the first room, it can navigate to each observation point sequentially according to a pre-planned route. This planned route helps the robot avoid obstacles and smoothly navigate from one observation point to another.

[0133] As the robotic device navigates sequentially to multiple observation points in the first room, it can continuously collect information about the first human body. If the device successfully collects this information, it can then identify the first user to determine if they are the target user.

[0134] Typically, when identifying the first user, a facial image of the first user can be captured first. Then, the captured facial image of the first user can be compared with a preset image of the target user to determine whether the first user is the target user.

[0135] In practical applications, the positional relationship between the robot and the first user can vary. In some cases, the robot may not be able to directly capture the first user's facial image, which could prevent identity verification.

[0136] Therefore, in one implementation, after acquiring the first human information, the robot first assesses whether it can directly acquire the first user's facial image based on the positional relationship between the robot and the first user. If the position is suitable and a relatively clear and complete facial image can be acquired, this facial image is directly compared with a preset image of the target user to determine the first user's identity. If a relatively clear and complete facial image cannot be acquired directly at the current position, the robot adjusts its strategy and automatically moves to a position near the first user to acquire sufficiently clear and complete facial information, thereby accurately determining whether the first user is the target user.

[0137] In this embodiment, a first angle condition can be set. If the orientation of the first user and the orientation of the robot device meet the first angle condition, it is considered that the robot device can capture a relatively clear and complete facial image of the first user. Conversely, if the orientation of the first user and the orientation of the robot device do not meet the first angle condition, it is considered that the robot device cannot capture a relatively clear and complete facial image of the first user.

[0138] In practical applications, when the orientation of the first user and the orientation of the robot device form a small angle (e.g., close to 90 degrees or less), the robot device may only be able to capture the side or back image of the first user, and cannot obtain a clear and complete image of the first user's face.

[0139] Therefore, in one implementation, the orientation angle can first be determined based on the first user's first orientation and the robot's second orientation. Then, based on the orientation angle, it can be determined whether the first user's orientation and the robot's orientation satisfy the first angle condition, thereby determining whether the robot can acquire a relatively clear and complete image of the first user's face.

[0140] In one implementation, a first angle range can be preset. If the orientation angle falls within the first angle range, the orientation of the first user and the robot device are considered to satisfy the first angle condition. Conversely, if the orientation angle does not fall within the first angle range, the orientation of the first user and the robot device are considered not to satisfy the first angle condition.

[0141] For example, it can be combined Figure 7 To understand. For example Figure 7 As shown, assume there is a robot device and a first user 1, with the right direction defined as 0° (the positive direction of the x-axis). The robot device's second orientation is 75°, and its field of view ranges from 15° to 135°. The first user 1 is located directly in front of the robot device.

[0142] Based on the above analysis, it can be determined that, for example, when the angle between the first orientation and the second orientation corresponding to the first user 1 is within the range of 0° to ±90° (0° and ±90° in the figure represent the angles described here), the robot device will have difficulty acquiring relatively complete facial information of the first user. Therefore, the first angle range can be defined as 90° to 180°, or -90° to -180°. This means that when the angle between the first orientation and the second orientation corresponding to the first user 1 is within the range of 90° to 180°, or -90° to -180°, the robot device can acquire a relatively complete facial image of the first user 1.

[0143] It's important to understand that in actual implementation, the range of the first angle corresponding to the first user may vary when the first user is not directly in front of the robot. (Reference) Figure 7 Suppose there is a first user 2 (the angle corresponding to the first user 2 is understood in a similar way to the first user 1), and it is located on the left side directly in front of the robot device. Since the robot device can see more of the field of view to the right of the first user 2, the first angle range can be adjusted, for example, it can be set to 105° to 180° or -85° to -180°.

[0144] Similarly, assuming there is a first user 3 (the angle corresponding to the first user 3 is understood in a similar way to the first user 1), and the first user 3 is located on the right side directly in front of the robot device, the robot device can see more of the left-hand field of view of the first user 3. Therefore, the first angle range can be adjusted, for example, it can be set to 85° to 180°, or -105° to -180°.

[0145] The following is combined Figure 8 The situation where the orientation of the first user and the orientation of the robot equipment satisfy the first angle condition is described.

[0146] like Figure 8 As shown in (a), assume there is a first user A, located to the left of the robot device. The first orientation of the first user A is -45°, the second orientation of the robot device is 75°, and the preset first angle range is 105° to 180°, or -85° to -180°. (Refer to...) Figure 8 It can be determined that, given the current second orientation of the robot device, theoretically, the robot device can acquire the facial image of the first user A. Therefore, it can be determined that the current orientation of the first user and the orientation of the robot device satisfy the first angle condition.

[0147] Verification was performed using the methods described above. (Refer to...) Figure 8In step (b), subtracting the second orientation 75° from the first orientation -45° yields an orientation angle of -120°. Then, comparing this orientation angle with the first angle range, it can be determined that the orientation angle falls within the first angle range, thus confirming that the orientation of the first user and the robot equipment satisfy the first angle condition.

[0148] Combined Figure 9 The following section describes the situation where the orientation of the first user and the orientation of the robot do not meet the first angle condition.

[0149] like Figure 9 As shown in (a), assume there is a first user B, who is also located on the left side directly in front of the robot device. The first orientation corresponding to the first user B is 160°, the second orientation corresponding to the robot device is 75°, and the preset first angle range is 105° to 180°, or -85° to -180°. (Refer to...) Figure 9 It can be determined that, given the current second orientation of the robot device, theoretically, the device may be unable to capture the face image of the first user B. Therefore, it can be determined that the orientation of the first user and the orientation of the robot device do not satisfy the first angle condition.

[0150] Similarly, verification can be performed using the methods described above. (Refer to...) Figure 9 In step (b), subtracting the second orientation from the first orientation of 160° yields an orientation angle of 85°. Then, comparing this orientation angle with the first angle range, it can be determined that the orientation angle does not fall within the first angle range. Therefore, it can be concluded that the orientation of the first user and the robot equipment do not meet the first angle condition.

[0151] S1004. Based on the first human body information and the reachability information corresponding to multiple locations in the grid map, determine the target point in the grid map.

[0152] In this embodiment, the reachability information corresponding to each location is used to indicate whether the robot device can reach the location. Alternatively, it can be used to indicate whether the robot device is allowed to reach the location. Or, it can be used to indicate that the robot device supports reaching the location. For example, if there are obstacles at a certain location or some locations, then the robot device cannot reach this location or these locations. In other words, the robot device is not allowed to reach this location or these locations.

[0153] Based on the above description, it can be determined that if the orientation of the first user and the orientation of the robot do not meet the first angle condition, the robot will have difficulty acquiring a clear and complete facial image of the first user. In this case, the robot may be unable to further determine whether the first user is the target user.

[0154] You can refer to this. Figure 10 In one implementation, if the orientation of the first user and the orientation of the robot meet a first angle condition, the robot can be controlled to navigate to a target point near the first user to collect the first user's facial image, thereby confirming the identity of the first user.

[0155] In this embodiment, two aspects can be considered in determining the target point: first, the distance between the target point and the first user; and second, the angle between the first line connecting the target point and the first user and the first orientation corresponding to the first user.

[0156] It's understandable that the closer the target point is to the first user, the clearer the robot's sensors or cameras will capture the first user's face when the robot is positioned at the target point. Conversely, the farther the target point is from the first user, the more likely the robot's sensors or cameras will capture a blurrier face when the robot is positioned at the target point.

[0157] Furthermore, the smaller the angle between the first connecting line and the first orientation, the closer the robot device is to the direction the first user is facing, allowing the device to capture a more complete image of the first user's face. Conversely, the larger the angle between the first connecting line and the first orientation, the farther the robot device is from the direction the first user is facing, potentially causing the device to only capture facial images of the first user from certain angles, thus failing to obtain complete facial information.

[0158] In the actual process of determining the target point, the two aspects mentioned above can be weighed. In one implementation, for example, distance can be considered to have higher priority than angle. In this implementation, a first range can first be determined on the grid map based on the first location of the first user. The first range is centered on the first location and has a radius of a first length.

[0159] Then, locations within the first range can be selected as candidate points to determine the target point from among multiple candidate points based on different included angles. In calculating the included angle, firstly, a line can be drawn connecting the candidate point to the first location to obtain the first line. Then, the angle of the first line corresponding to the candidate point can be subtracted from the first orientation corresponding to the first user, and the absolute value is taken to obtain the included angle between the first line corresponding to the candidate point and the second orientation. When determining the target point from multiple candidate points, the candidate point with the smallest included angle can be determined as the target point.

[0160] It is understandable that obstacles (such as bookshelves, refrigerators, and televisions) may exist in the multiple locations included in the first range. For the robotic device, the locations of these obstacles are not accessible. Therefore, when determining candidate points within the first range, the locations of obstacles need to be removed. In this way, the final candidate points will all be located in locations accessible to the robotic device.

[0161] When obstacles are removed, there may be situations where no candidate points exist within the first range. In this embodiment, in such cases, a second range can be redefined, and candidate points can be redefined within this second range to determine the target point. The second range has the first position as its center and a second length as its radius, where the second length is greater than the first length.

[0162] The following is combined Figure 11 The method of determining the target point is introduced.

[0163] like Figure 11 As shown in (a), assuming there is a first user A, we need to determine a target point for user A. Based on the above introduction, we can determine candidate points first based on distance, and then determine the target point based on the candidate points. (Refer to...) Figure 11 In (a), assuming the first length is set to 0.2 centimeters (cm), then the first range is a circular area with the first user A as the center and a radius of 0.2 centimeters. If there are no candidate points within the current first range, then the second range can be redefined.

[0164] refer to Figure 11 In step (a), the second range can be set as a circular area with the first user A as the center and a radius of 0.3 cm. Assume that two candidate points, candidate point 1 and candidate point 2, can be identified within the second range. Then, candidate point 1 can be connected to the first position to obtain the first connection line 1. Similarly, candidate point 2 can be connected to the first position to obtain the first connection line 2.

[0165] After obtaining the first connecting line 1 and the first connecting line 2, the included angle can be calculated based on the first connecting line and the first orientation. For example... Figure 11 As shown in (b), assuming the angle of the first connecting line 1 is 10°, the angle of the first orientation is -45°, and the angle of the first connecting line 2 is -60°, then it can be determined that the angle between the first connecting line 1 and the first orientation is 55°, and the angle between the first connecting line 2 and the first orientation is 15°. Comparing these two angles, it can be determined that the angle between the first connecting line 2 and the first orientation is the smallest, therefore, candidate point 2 can be determined as the target point.

[0166] It is understandable that, in practice, when the above situation occurs, considering that candidate point 1 and candidate point 2 are close to the first location, but candidate point 2 is located at a location that can capture a more complete facial image of the first user compared to candidate point 1, choosing candidate point 2 to capture the facial image of the first user A is the optimal choice.

[0167] It is important to understand that after determining the first range based on the first user's location, if none of the candidate points within the first range are positioned so that the robot can capture the first user's facial image, one implementation can determine the target point as the location behind (e.g., directly behind) the first user A, and control the robot to play a first inquiry voice message to determine whether the first user is the target user. The first inquiry voice message is used to ask the first user whether they are the target user.

[0168] You can refer to this. Figure 12 To understand. For example Figure 12 As shown in (a), suppose there is a robot device 1, a first user A, a dining table, and a sofa. The first user A is standing next to the dining table, and the robot device 1 is diagonally behind and to the left of the first user A. Based on the current positional relationship between the robot device 1 and the first user A, it can be determined that the orientation of the first user A and the orientation of the robot device 1 do not satisfy a first angle condition, meaning that the robot device 1 cannot capture the face image of the first user A. Therefore, based on the above description, it is understandable that a first range needs to be determined based on the first position of the first user A, and then the target point needs to be determined.

[0169] like Figure 12 As shown in (a), in this case, after determining the first range, all candidate points are located behind the first user A. For example, the target point can be determined as the position directly behind the first user A. Then, the first inquiry voice is played to determine whether the first user is the target user. (See reference...) Figure 12 In (b), the robot is controlled to navigate to the back of the first user A and interact with the first user A via voice.

[0170] In addition, combined Figure 13 The scenario where the target point is located in front of the first user is described.

[0171] like Figure 13As shown in (a), assume there is a robot device 1, a first user B, a dining table, and a sofa. First user B is facing the dining table, and robot device 1 is diagonally behind and to the left of first user B. Based on the current positional relationship between robot device 1 and first user B, the orientation of first user B and the orientation of robot device 1 do not satisfy a first angle condition, meaning robot device 1 cannot capture a facial image of first user B. Therefore, a first range can be determined based on the first position of first user B, and thus the target point can be determined.

[0172] like Figure 13 As shown in (a), in this case, after determining the first range, there are multiple candidate points in front of the first user B. At this point, for example, the candidate point with the smallest angle to the first orientation can be determined as the target point. Figure 13 As shown in (b), the robot device 1 can be controlled to navigate to the target point.

[0173] It's important to understand that the first user may also be moving during the robot's movement. If, during this process, the first user's orientation and the robot's orientation satisfy a first angle condition, then the robot can capture the first user's facial image and determine whether the first user is the target user.

[0174] If the first user is identified as the target user, the interactive operation corresponding to the first instruction can be executed. If the first user is not identified as the target user, the robot can be controlled to navigate to the next observation point. If the robot collects the first human body information during navigation to the next observation point, it can continue to collect the second human body information of the first user and determine whether the first user is the target user based on the second human body information, until the target user is found or the traversal of the first room is completed. If the target user is still not found after the traversal of the first room is completed, the first room can be re-identified, and the above operations can be repeated.

[0175] S1005. Navigate to the target point and collect the second human body information of the first user. The second human body information may include the facial image of the first user.

[0176] S1006. Determine whether the quality information of the acquired face image meets the preset conditions for face recognition. If yes, proceed to S1008. If no, proceed to S1007.

[0177] In this embodiment, the quality information of the face image is used to reflect multiple aspects of the face image's characteristics. For example, the quality information may include basic indicators such as image sharpness, brightness, and contrast, as well as information such as the angle and occlusion of the face. If the quality information determines that the face image meets the preset conditions for face recognition, then the face image can be compared with a preset image of the target user to determine whether the first user is the target user.

[0178] For example, preset conditions may include resolution ≥ 640×480 pixels (relatively clear), illumination uniformity (facial area brightness standard deviation ≤ 30), blurriness (Laplacian operator variance ≥ 150), attitude angle (pitch ± 15°, yaw ± 30°), occlusion rate ≤ 20% (including accessories such as glasses), facial feature integrity (visibility of key feature points ≥ 90%), etc. This embodiment does not limit these conditions.

[0179] S1007. Play the first inquiry voice to determine if the first user is the target user. If yes, execute the interactive operation corresponding to the first instruction. If not, continue collecting the first human body information until the target user is found or the traversal of the first room is completed.

[0180] If the facial image does not meet the preset conditions for facial recognition based on the quality information, it means that it is currently impossible to accurately determine whether the first user is the target user based on the facial image. In one implementation, a first inquiry voice can be played to ask the first user whether they are the target user. Then, the robot device responds to the received first response voice to determine whether the first user is the target user. If yes, the interactive operation corresponding to the first instruction is executed. If not, the search continues until the target user is found or the first room is traversed.

[0181] Specifically, if it is determined that the first user is not the target user, the robot can be controlled to navigate to the next observation point. If the robot collects the first human information during the navigation to the next observation point, it can continue to collect the second human information of the first user and determine whether the first user is the target user based on the second human information, until the target user is found or the traversal of the first room is completed.

[0182] S1008. Compare the facial image with a preset image of the target user to determine if the first user is the target user. If yes, execute the interactive operation corresponding to the first instruction. If no, continue collecting the first human body information until the target user is found or the traversal of the first room is completed.

[0183] In this embodiment, by navigating sequentially to multiple observation points on a grid map, the room can be scanned efficiently, and the location of potential users can be quickly determined, improving the efficiency of finding target users. Subsequently, if a valid facial image of the target user is not acquired in the initial observation, the robot's position can be dynamically adjusted based on the acquired initial human information. This flexibility allows the robot to navigate more effectively in complex environments, thereby increasing the probability of finding the target user.

[0184] When the robot device is adjusting its position, by setting a range near the location of the first user and identifying candidate points among multiple locations within this range, the robot can reach the vicinity of the user and only attempt to navigate to a location that is safe to reach, reducing collisions and other potential risks.

[0185] Furthermore, by setting a first angle range, it can be determined whether the robot needs to adjust its path. This avoids unnecessary path adjustments and improves navigation efficiency. In cases of poor facial image quality, by playing a first questioning voice to inquire whether the first user is the target user, it is possible to accurately determine whether the first user is indeed the target user.

[0186] Based on the above embodiments, it can be determined that the robot device can sequentially navigate to multiple observation points in the first room and collect first human information during this process to determine whether the target user has been found. In the person-finding method provided in this application, since the robot device's field of vision is conical, in order to ensure that the robot device can cover all locations in the first room during the person-finding process, after the robot device navigates to each observation point, the first component of the robot device can be controlled to rotate a second angle to determine whether the first human information has been collected. The first component includes a camera unit, and the second angle is less than 360°.

[0187] The following is combined Figures 14 to 18 The process for calculating the second angle is described below. Figure 14 This is a schematic diagram of setting observation markers provided in the embodiments of this application. Figure 1 , Figure 15 This is a schematic diagram of setting observation markers provided in the embodiments of this application. Figure 2 , Figure 16 This is a schematic diagram of setting observation markers provided in the embodiments of this application. Figure 3 , Figure 17 A schematic diagram of calculating the second angle provided for embodiments of this application. Figure 1 , Figure 18 A schematic diagram of calculating the second angle provided for embodiments of this application. Figure 2 .

[0188] When the robot navigates to any observation point, it can be determined that certain locations within the first room have already been observed by the robot. To improve the efficiency of finding people, the first component of the robot can be rotated by a specified angle to observe locations that the robot can observe but have not yet observed, thus enabling the search operation.

[0189] In one implementation, during the movement of the robotic device, observation markers can be set at certain locations within the first room. Specifically, observation markers for locations not yet observed by the robotic device can be set as first markers, while observation markers for locations already observed by the robotic device can be set as second markers.

[0190] In the initial state, when the robot has predicted the first room but has not yet entered it to search for people, the observation marker of each location within the first room in the grid map can be set as the first marker. Simultaneously, all locations in the grid map other than the first room can be preset as first numerical markers, indicating that these locations belong to other room areas. For example, the first numerical marker can be set to 255.

[0191] Subsequently, as the robot moves within the first room, the observation markers of locations already observed by the robot can be changed to second markers. For example, the second marker can be set to 1. In one implementation, it can be determined whether an observation marker for a location should be set to the second marker based on the robot's field of view angle and point cloud data.

[0192] Based on the above description, it can be confirmed that after scanning the room, the camera can collect point cloud data. Point cloud data can enable 3D reconstruction of the interior, allowing the robotic device to accurately understand the room's layout and the location of furniture. For example, assuming there are obstacles in the first room, the point cloud data can describe the surface shape of the obstacles using 3D coordinates.

[0193] It's understandable that when a robot is observing, its line of sight cannot penetrate obstacles to observe the area behind them. Therefore, when setting observation markers, we can first draw lines connecting the robot and the obstacles based on the point cloud data. Then, we set the observation markers at the locations on these lines that are actually observable by the robot as the second markers.

[0194] Additionally, it's important to note that the robot's observation capabilities are limited by its sensing distance and field of view angle. If an obstacle exceeds the robot's field of view angle or sensing distance, the robot will be unable to observe it. Therefore, it's sufficient to set the observation markers for blank areas within the robot's field of view and sensing distance as the second markers.

[0195] You can refer to this. Figure 14 To understand this, assume there is a robot device A with a sensing distance of 0.3 meters (m), a field of view angle of 120°, and a current field of view range of -60° to 60°. Also, assume there is an obstacle, represented by the black rectangle shown in the diagram. Based on the above analysis, it can be determined that when setting observation markers for a location, the point cloud data of the robot device and the obstacle can be connected.

[0196] Figure 14 The diagram illustrates the line connecting the robot and the obstacle point cloud data, showing the line with the largest and smallest angles. Based on the robot's sensing distance, field of view angle, these two lines, and the obstacles, it's possible to determine which locations should be set as the second marker. For example... Figure 14 As shown, the observation marker behind the obstacle remains unchanged, still designated as the first marker. The observation markers at other locations within the robot's perception distance and field of view angle are designated as the second marker.

[0197] The above combination Figure 14 This section introduces the situation of obstacles within the robot's perception distance and field of view. The following section combines... Figure 15 Let me describe what happens when obstacles are outside the sensing range of a robotic device.

[0198] like Figure 15 As shown, assume there is a robot device A and an obstacle, and the obstacle is currently outside the perception range of robot device A. (Reference) Figure 15 The position of the obstacle will not affect the robot device A's ability to identify objects within the sensing distance and field of view angle, so the observation markers of the positions within this range can be set as the second markers.

[0199] Furthermore, in real-world situations, obstacles may lie outside the robot's field of vision. The following section will discuss this further. Figure 16 Let me introduce it.

[0200] like Figure 16 As shown, assume there is currently robot device A and an obstacle, with the obstacle currently outside the field of view of robot device A. (Reference) Figure 16The position of the obstacle will not affect the robot device A's ability to identify objects within the sensing distance and field of view angle, so the observation markers of the positions within this range can be set as the second markers.

[0201] After navigating to the observation point, the rotation angle of the robot can be determined based on the reference position that the robot has not observed.

[0202] For example, in one implementation, after navigating to the observation point, a third range centered on the observation point and with a third length as its radius can be determined in the grid map.

[0203] Within the multiple locations encompassed in the third range, a reference position marked with a first marker can be determined. This first marker indicates that the reference position has not been observed by the robotic device. Subsequently, a second position of the reference position and the robotic device within the grid map can be determined. This second position is represented by grid coordinates.

[0204] In one implementation, the robot's current coordinates and point cloud data coordinates collected by the sensors can be converted into grid coordinates.

[0205] For example, suppose the origin of the raster map is [x o ,y o The grid size is r meters. The current coordinates of the robot are (x, y). r ,y r The second position (grid coordinates) corresponding to the robot device (x gr ,y gr The calculation formulas for ) can be found in Formula 1 and Formula 2 below:

[0206]

[0207] Assume the coordinates of the point cloud data are (x p ,y p ,z p The second position (x) corresponding to the point cloud data. gp ,y gp The calculation formulas for ) can be found in Formulas 3 and 4 below:

[0208]

[0209] It is understandable that, based on Formulas 3 and 4, the second position of the reference position in the raster map can be calculated from the point cloud data coordinates of the reference position. Here, the reference position is the position marked as the first observation marker.

[0210] In one implementation, after obtaining the reference position and the second position of the robot device in the grid map, the rotation angle corresponding to the reference position can be determined based on the second position corresponding to the reference position, the second position corresponding to the robot device, and the second orientation of the robot device. The rotation angle is the angle that the robot device needs to rotate to face (or be understood as facing or directly facing) the reference position.

[0211] For example, in one implementation, the formula for calculating the rotation angle can refer to the following formula five:

[0212] θ i =θ ig -β Formula Five

[0213] In Formula 5, β represents the robot's current second orientation, and θ ig The angle θ is the angle between the reference position and the second line connecting the two lines of the robot device. ig The calculation formula can be found in Formula Six below:

[0214]

[0215] In Formula 6, arctan() is the arctangent function, which can be used to calculate the angle value (in radians) corresponding to the slope of a straight line. It can be multiplied by a conversion factor. Convert radian values ​​to angle values. (x) gr ,y gr (x) represents the second position corresponding to the robot device. i ,y i ) represents the second position corresponding to the reference position.

[0216] Once the rotation angle corresponding to the reference position is obtained, the rotation angle θ can be... i Mapped to the range [0°, 359°], the mapped rotation angle θ ia The calculation formula can be found in Formula Seven below:

[0217] θ ia =(θ i +360°)mod 360° formula seven

[0218] In Formula 7, mod() is the modulo function. This means that for multiple reference positions, the rotation angle corresponding to each reference position can be obtained.

[0219] Then, the second angle can be determined based on the rotation angles corresponding to the multiple reference positions and the field of view angle of the robot device.

[0220] Based on the above description, it can be determined that the robot's field of view is conical. In practical applications, the robot's field of view angle depends on the sensor used. For example, if an RGBD sensor is used, and its horizontal field of view is ±α, then the robot's field of view angle is the second orientation ±α. For example, α can be set to 45°.

[0221] In one implementation, the minimum rotation angle θ can first be determined based on the rotation angles corresponding to multiple reference positions. min and maximum rotation angle θ max Then, based on the minimum rotation angle θ min and maximum rotation angle θ max The second angle θ is calculated by considering the angle between the field of view of the robot and the robot equipment. rotate The second angle indicates the actual angle the robot needs to rotate. The second angle θ rotate The specific definitions are as follows:

[0222] Calculate the minimum rotation angle θ using the robot's second orientation as the axis. min (excluding 0°) and maximum rotation angle θ max Each corresponds to a third angle. For example, the minimum rotation angle θ can be... min Subtract the second orientation to obtain the minimum rotation angle θ min The corresponding third angle. And, the maximum rotation angle θ. max Subtract the second orientation to obtain the maximum rotation angle θ. max The corresponding third angle.

[0223] If θ min and θ max If the corresponding third angles are all within the range of [-α, α], then the second angle θ rotate Set to 0. Otherwise, the angles of left and right rotation need to be compared to select the smaller rotation angle as the second angle.

[0224] The formula for calculating the left-hand rotation angle can be found in Formula 8 below:

[0225] θ left =θ max3 -α Formula 8 In Formula 8, θ max3 For the maximum rotation angle θ max The corresponding third angle.

[0226] The formula for calculating the right-hand rotation angle can be found in Formula Nine below:

[0227] θ right =360°-(θ) min3 +α) Formula Nine In Formula Nine, θmin3 For the minimum rotation angle θ min The corresponding third angle.

[0228] After obtaining the left-hand and right-hand rotation angles, the angle with the smallest value can be selected as the second angle. The second angle θ rotate The calculation formula can be found in Formula 10 below:

[0229] θ rotate =min(θ) left ,θ right In Formula 10, min() is the function that takes the minimum value. left For the left-hand rotation angle, θ right It is a right-hand rotation angle.

[0230] The following can be combined Figure 17 and Figure 18 The calculation of the second angle will be introduced.

[0231] like Figure 17 As shown, assume there is a robot device A with a second orientation of 285° (-75°). Also, the robot's field of view angle is 285° ± 60°, i.e., α is set to 60°.

[0232] Within the third range, the minimum rotation angle θ is determined. min For 270° and maximum rotation angle θ max The value is 315°. Therefore, the minimum rotation angle θ can be calculated. min The corresponding third angle is 270°-285°, or -15°. Maximum rotation angle θ max The corresponding third angle is 315°-285°, or 30°. The minimum rotation angle θ... min The corresponding third angle -15° and maximum rotation angle θ max The corresponding third angle of 30°, compared with [-60°, 60°], allows us to determine θ. min and θ max If the corresponding third angles are all within the range of [-α, α], then the second angle is set to 0.

[0233] Combined Figure 18 The following section describes the situations where rotation is required.

[0234] like Figure 18 As shown in (a), assume there is a robot device A with a second orientation of 75°. Also, the robot's field of view angle is 75° ± 60°, i.e., α is set to 60°.

[0235] Within the current third range, the minimum rotation angle θ can be determined. minFor 140° and maximum rotation angle θ max The value is 210°. Therefore, the minimum rotation angle θ can be calculated. min The corresponding third angle is 140°-75°, or 65°. Maximum rotation angle θ max The corresponding third angle is 210° - 75°, or 135°. The minimum rotation angle θ... min The corresponding third angle 75° and the maximum rotation angle θ max The corresponding third angle 135°, compared with [-60°, 60°], allows us to determine θ. min and θ max If the corresponding third angle is not within the range of [-α, α], then the first component of the robot needs to be rotated by a second angle θ. rotate .

[0236] refer to Figure 18 In equation (b), based on Formula 8, the left-hand rotation angle is 135° - 60°, which is 75°. And based on Formula 9, the right-hand rotation angle is 360° - 75° - 60°, which is 225°. The smallest angle is selected as the second angle, which is 75 degrees.

[0237] It's important to understand that if an obstacle exists within the third area, you can refer to... Figure 14 To understand this, the reference position of the observation marker behind the obstacle, designated as the first marker, cannot be observed regardless of how the first component of the robot is rotated. Therefore, it can be disregarded when calculating the rotation angle.

[0238] In this embodiment, by determining unobserved reference positions in the grid map and calculating the required rotation angle of the robot, it is ensured that uncovered areas are effectively observed. By identifying and marking unobserved reference positions, the system can more comprehensively cover the target area, reduce blind spots, and improve the comprehensiveness and accuracy of observation. Determining the second angle based on the rotation angle of the reference position and the field of view angle avoids unnecessary rotation, saves time and energy, and improves the system's resource utilization efficiency.

[0239] It should be noted that the module names involved in the embodiments of this application can all be defined as other names, as long as they can achieve the function of each module, and no specific restrictions are placed on the module names.

[0240] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0241] The method for finding missing persons according to the embodiments of this application has been described above. The apparatus for performing the above method, provided in the embodiments of this application, is described below. Those skilled in the art will understand that the methods and apparatus can be combined and referenced together, and the related apparatus provided in the embodiments of this application can perform the steps in the above-described method for finding missing persons.

[0242] In one implementation, this application provides a person-finding device. Figure 19 This is a schematic diagram of the structure of the person-finding device provided in the embodiments of this application.

[0243] like Figure 19 As shown, the robot device includes: a camera unit, a central controller unit, and a motion unit;

[0244] The camera unit is used to capture environmental images or videos to help the robot identify and locate target objects (such as a specific person).

[0245] The central controller unit processes information from various modules. For example, it can receive images captured by the camera unit and sensor data, and analyze them to determine the target's position, status, and relative distance. Alternatively, it can allow the robot to interact with humans via voice, further enhancing human-machine communication.

[0246] The motion unit is responsible for the robot's physical movements, controlling its movement, turning, and obstacle avoidance. It can execute specific actions based on instructions from the central controller, such as avoiding obstacles, moving towards a target person, and adjusting its speed.

[0247] Optionally, the robotic device may also include: a communication module for remote control or information transmission; and a human-machine interface module, such as a touchscreen, display, or augmented reality interface, to facilitate user operation and real-time monitoring of the robot's status.

[0248] The method for finding missing persons provided in this application can be applied to electronic devices with data processing capabilities.

[0249] In one implementation, this application provides an electronic device. Figure 20 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0250] like Figure 20 As shown, the electronic device includes: a processor, a memory, an image acquisition device, and a power device;

[0251] Image acquisition devices are used to acquire visual information from the environment, providing robots with the necessary perception capabilities. For example, they can enable devices to acquire point cloud data of the home environment, user location information, and facial images based on the field of view angle and perception distance.

[0252] The memory is used to store computer-executable instructions that can run on the processor, as well as data such as images, location data, and target information currently acquired by the sensors;

[0253] The processor executes computer execution instructions stored in memory, causing the electronic device to perform the above-described method.

[0254] Power equipment, used to drive electronic devices to move.

[0255] This application provides a chip. The chip includes a processor, which is used to call a computer program in memory to execute the technical solutions in the above embodiments. Its implementation principle and technical effects are similar to those in the related embodiments described above, and will not be repeated here.

[0256] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0257] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, Digital Versatile Discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0258] This application provides a computer program product, which includes a computer program that, when run, causes a computer to perform the above-described method.

[0259] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0260] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for locating a missing person, characterized in that, Applied to robotic equipment, including: In response to a first instruction, a first room is determined in a first map, wherein the first instruction is used to indicate the search for a target user, and the first map includes multiple rooms; During the process of navigating to multiple observation points in the first room, if the first human body information is collected, then based on the first human body information, it is determined whether the orientation of the first user and the orientation of the robot device meet the first angle condition, and the first user is the user corresponding to the first human body information; If not, then based on the first human body information and the reachability information corresponding to each of the multiple locations in the first map, a target point is determined in the first map, and the reachability information is used to indicate whether the robot device can reach the location; Navigate to the target point and collect the second human body information of the first user; If the first user is determined to be the target user based on the second human body information, the interactive operation corresponding to the first instruction is executed.

2. The method according to claim 1, characterized in that, The first human information includes the first location of the first user in the first map and the first orientation of the first user; The step of determining whether the orientation of the first user and the orientation of the robot device satisfy the first angle condition based on the first human body information includes: The orientation angle is determined based on the first orientation and the second orientation of the robot device; The orientation angle is compared with the first angle range to determine whether the orientation of the first user and the orientation of the robot device meet the first angle condition. The first angle condition includes: the included angle of orientation belongs to the first angle range.

3. The method according to claim 1 or 2, characterized in that, The first human information includes the first location of the first user in the first map; The step of determining the target point in the first map based on the first human information and the reachability information corresponding to multiple locations in the first map includes: Based on the first location, a first range is determined in the first map, the first range having the first location as the center and a first length as the radius; Among the multiple locations included in the first range, candidate points are determined, and the reachability information of the candidate points is used to indicate that the robot device can reach the location corresponding to the candidate point; The target point is determined from the candidate points.

4. The method according to claim 3, characterized in that, The method further includes: If no candidate point exists within the first range, a second range is determined in the first map based on the first location. The first range is centered at the first location and has a second length as its radius, where the second length is greater than the first length. Among the multiple locations included in the second range, candidate points are determined, and the reachability information of the candidate points is used to indicate the location corresponding to the target point that the robot device can reach; The target point is determined from the candidate points.

5. The method according to claim 3 or 4, characterized in that, Determining the target point from the candidate points includes: For any of the candidate points, determine the angle between the first line connecting the candidate point and the first orientation corresponding to the first user, where the first line is the line connecting the candidate point and the first position. The candidate point with the smallest included angle is determined as the target point.

6. The method according to any one of claims 1-5, characterized in that, The second human body information includes the facial image of the first user; The method further includes: Determine the quality information of the face image; If the quality information determines that the face image meets the preset conditions for face recognition, then the face image is compared with the preset image of the target user to determine whether the first user is the target user.

7. The method according to claim 6, characterized in that, The method further includes: If, based on the quality information, it is determined that the face image does not meet the preset conditions for face recognition, then a first inquiry voice is played, which is used to ask the first user whether it is the target user. In response to the received first voice reply, determine whether the first user is the target user.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: For any one of the plurality of observation points, after navigating to the observation point, the first component of the robot device is controlled to rotate by a second angle to determine whether the first human information has been collected. The first component includes a camera unit, and the second angle is less than 360°.

9. The method according to claim 8, characterized in that, The method further includes: After navigating to the observation point, a third range is determined in the first map, centered on the observation point and with a radius of a third length. Within the plurality of locations included in the third range, a reference location marked with a first mark is determined, the first mark indicating that the reference location has not been observed by the robotic device; Based on the reference position, the second position of the robot device in the first map, and the second orientation of the robot device, a rotation angle corresponding to the reference position is determined. The rotation angle is the angle that the robot device needs to rotate to face the reference position. The second angle is determined based on the rotation angle corresponding to each of the multiple reference positions and the field of view angle of the robot device.

10. An electronic device, characterized in that, The electronic device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 9.

11. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes computer program code that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 9.

Images