Methods, devices, robots, and storage media for determining target location

Abstract

This application provides a method, apparatus, robot, and storage medium for determining a target location, comprising: acquiring a local image collected by the robot; determining a local topology map based on the local image and a global object map corresponding to the robot's surrounding environment; determining at least one global topology map of the robot's surrounding environment based on the local topology map; determining the end nodes corresponding to each of the at least one global topology map; matching the local topology map with each of the at least one global topology map based on the end nodes corresponding to each of the at least one global topology map to obtain a set of node pairs corresponding to at least one global topology map and the local topology map; filtering a first preset number of sample node pairs from the set of node pairs; determining a second preset number of candidate pose information based on the first preset number of sample node pairs; and determining the robot's target location information based on the candidate pose information. This application can improve the efficiency of determining the robot's target location information.

Description

Technical Field

[0001] This application relates to the field of robot positioning technology, and more specifically, to a method, apparatus, robot, and storage medium for determining the location of a target. Background Technology

[0002] With the rapid development of technology, robots are widely used in people's daily lives. However, when robots are used indoors, satellite signals cannot be received, meaning GPS positioning is not feasible. Currently, without prior localization, robots rely on sensor information, such as images captured by vision sensors, to determine their pose on a global map. However, images captured by vision sensors are easily affected by lighting, rotation angles, moving objects, and are limited by the camera's field of view. This can lead to mismatches between objects viewed from different angles at the same location, resulting in low efficiency in determining the robot's target location. Therefore, improving the efficiency of robot target location determination has become a pressing issue. Summary of the Invention

[0003] In view of this, embodiments of this application propose a method, apparatus, robot, and storage medium for determining a target location to solve the existing technical problems.

[0004] According to one aspect of the embodiments of this application, a method for determining a target location is provided. The method includes: acquiring a local image collected by a robot; determining a local topology map based on the local image and a global object map corresponding to the robot's surrounding environment; determining at least one global topology map of the robot's surrounding environment based on the local topology map; determining end nodes corresponding to each of the at least one global topology map; matching the local topology map with the at least one global topology map based on the end nodes corresponding to each of the at least one global topology map to obtain a set of node pairs corresponding to the at least one global topology map and the local topology map, wherein the set of node pairs is the set of node pairs with the same corresponding labels in the local topology map and the at least one global topology map; filtering a first preset number of sample node pairs in the set of node pairs; determining a second preset number of candidate pose information based on the first preset number of sample node pairs; and determining the target location information of the robot based on the second preset number of candidate pose information.

[0005] According to one aspect of the embodiments of this application, a target location determination apparatus is provided, the apparatus comprising: a local topology map determination module, configured to acquire a local image collected by a robot, and determine a local topology map based on the local image and a global object map corresponding to the robot's surrounding environment; a global topology map determination module, configured to determine at least one global topology map of the robot's surrounding environment based on the local topology map; an end-point node determination module, configured to determine the end-point nodes corresponding to each of the at least one global topology map; a node pair set determination module, configured to match the local topology map with the at least one global topology map based on the end-point nodes corresponding to each of the at least one global topology map, to obtain a node pair set corresponding to the at least one global topology map and the local topology map, wherein the node pair set is the set of node pairs with the same corresponding labels in the local topology map and the at least one global topology map; a candidate pose information determination module, configured to filter a first preset number of sample node pairs in the node pair set, and determine a second preset number of candidate pose information based on the first preset number of sample node pairs; and a target location information determination module, configured to determine the target location information of the robot based on the second preset number of candidate pose information.

[0006] According to one aspect of the embodiments of this application, a robot is provided, the robot comprising: a body; and a control system communicating with the body, the control system including a processor and a memory communicating with the processor, the memory storing instructions, the instructions being executed on the processor to cause the processor to perform operations, the operations including: acquiring a local image collected by the robot; determining a local topology map based on the local image and a global object map corresponding to the robot's surrounding environment; determining at least one global topology map of the robot's surrounding environment based on the local topology map; determining end nodes corresponding to each of the at least one global topology map; matching the local topology map with the at least one global topology map based on the end nodes corresponding to each of the at least one global topology map to obtain a set of node pairs corresponding to the at least one global topology map and the local topology map, the set of node pairs being the set of node pairs with the same corresponding labels in the local topology map and the at least one global topology map; filtering a first preset number of sample node pairs in the set of node pairs; determining a second preset number of candidate pose information based on the first preset number of sample node pairs; and determining target position information of the robot based on the second preset number of candidate pose information.

[0007] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute the method for determining the target location as described above.

[0008] In this application, a local topology map is first determined based on local images collected by the robot and a global object map corresponding to the robot's surrounding environment. This allows for the determination of at least one global topology map of the robot's environment. Then, based on the end nodes corresponding to each location in the at least one global topology map, the local topology map is matched with the at least one global topology map to determine at least one set of node pairs corresponding to nodes with the same labels in both the global and local topology maps. Candidate pose information is then determined based on sample node pairs in this set. Finally, the robot's target position information is determined based on the candidate pose information. This application enables robot localization based on objects in the robot's surrounding environment, even in environments with limited or no precise positioning, resulting in more accurate target position information. Furthermore, it improves the efficiency of determining the robot's target position without relying on precise positioning information.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0011] Figure 1 This is a schematic diagram of the hardware structure of a robot according to an embodiment of this application.

[0012] Figure 2 This is a schematic diagram of the mechanical structure of a robot according to an embodiment of this application.

[0013] Figure 3 This is a flowchart illustrating a method for determining a target location according to an embodiment of this application.

[0014] Figure 4 This is a schematic diagram of a global object map according to an embodiment of this application.

[0015] Figure 5 This is a schematic diagram of a partial topology diagram according to an embodiment of this application.

[0016] Figure 6 This is a flowchart illustrating the specific steps of step 220 according to an embodiment of this application.

[0017] Figure 7 It is a schematic diagram of at least one global topology diagram according to an embodiment of the application.

[0018] Figure 8 This is a flowchart illustrating the specific steps of step 240 according to an embodiment of this application.

[0019] Figure 9 This is a flowchart illustrating the specific steps of step 250 according to an embodiment of this application.

[0020] Figure 10 This is a flowchart illustrating the specific steps of step 260 according to an embodiment of this application.

[0021] Figure 11 This is a flowchart illustrating a method for determining a target location according to another embodiment of this application.

[0022] Figure 12 This is a flowchart illustrating the specific steps of step 330 according to an embodiment of this application.

[0023] Figure 13 This is a block diagram of a target location determination device according to an embodiment of this application.

[0024] The accompanying drawings illustrate specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the inventive concept in any way, but rather to illustrate the concept of the invention to those skilled in the art through specific embodiments. Detailed Implementation

[0025] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0026] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0027] In the following description, the use of suffixes such as "module," "component," or "unit" to denote parts is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "component," or "unit" may be used interchangeably.

[0028] Please see Figure 1 , Figure 1This is a schematic diagram of the hardware structure of a robot 100 provided in one embodiment of this application. The robot 100 can be any of various robot types, specifically including but not limited to at least one of wheeled robots, legged robots, tracked robots, crawling robots, worm-like robots, or swimming robots. For example, the robot 100 can specifically be a legged robot, or a robot combining legged and wheeled configurations. Legged robots include monopodial robots, bipodial robots, or multipodial robots. Multipodial robots refer to robots with three or more legs; for example, a multipodial robot can specifically be a quadrupedal robot. A robot is a machine capable of performing semi-autonomous or fully autonomous tasks. Robots are not limited to humanoid machine devices and can also include robots with configurations such as dogs, horses, snakes, fish, apes, or monkeys. For example, a robot can specifically be a quadrupedal robotic horse. Figure 1 In the illustrated embodiment, robot 100 includes a mechanical unit 101, a communication unit 102, a sensing unit 103, an interface unit 104, a storage unit 105, a display unit 106, an input unit 107, a control module 110, and a power supply 111. The various components of robot 100 can be connected in any way, including wired or wireless connections. Those skilled in the art will understand that... Figure 1 The specific structure of the robot 100 shown does not constitute a limitation on the robot 100. The robot 100 may include more or fewer parts than shown. Some parts are not essential components of the robot 100 and may be omitted or combined as needed without changing the nature of the invention.

[0029] Figure 2 This is a schematic diagram of the mechanical structure of a robot according to an embodiment of this application. The following is in conjunction with... Figure 1 and Figure 2 A detailed introduction to each component of Robot 100:

[0030] Mechanical unit 101 is the hardware of robot 100. For example... Figure 1 As shown, the mechanical unit 101 may include a drive board 1011, a motor 1012, and a mechanical structure 1013, such as... Figure 2As shown, the mechanical structure 1013 may include a main body 1014, extendable legs 1015, and feet 1016. In other embodiments, the mechanical structure 1013 may also include an extendable robotic arm (not shown), a rotatable head structure 1017, a rocking tail structure 1018, a cargo-carrying structure 1019, a saddle structure 1020, a camera structure 1021, etc. It should be noted that the various component modules of the mechanical unit 101 can be one or multiple, depending on the specific situation. For example, there may be four legs 1015, and each leg 1015 may be equipped with three motors 1012, resulting in a total of twelve motors 1012.

[0031] The communication unit 102 can be used for receiving and sending signals, and can also communicate with networks and other devices. For example, it can receive instructions from a remote control or other robot 100 to move in a specific direction at a specific speed according to a specific gait, and then transmit these instructions to the control module 110 for processing. The communication unit 102 includes modules such as WiFi, 4G, 5G, Bluetooth, and infrared modules.

[0032] The sensing unit 103 is used to acquire information data about the environment surrounding the robot 100 and to monitor parameter data of various components inside the robot 100, and then sends this data to the control module 110. The sensing unit 103 includes various sensors, such as sensors for acquiring information about the surrounding environment: lidar (for remote object detection, distance determination, and / or velocity determination), millimeter-wave radar (for short-range object detection, distance determination, and / or velocity determination), cameras, infrared cameras, and Global Navigation Satellite System (GNSS). Sensors for monitoring various components inside the robot 100 include: an inertial measurement unit (IMU) (for measuring velocity, acceleration, and angular velocity values), foot sensors (for monitoring the position of the foot's contact point, foot posture, magnitude and direction of the contact force), and temperature sensors (for detecting component temperature). Other sensors that can be configured on the robot 100, such as load sensors, touch sensors, motor angle sensors, and torque sensors, are not detailed here.

[0033] The interface unit 104 can be used to receive input from external devices (e.g., data, power, etc.) and transmit the received input to one or more components within the robot 100, or it can be used to output to external devices (e.g., data, power, etc.). The interface unit 104 may include a power port, a data port (such as a USB port), a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, etc.

[0034] Storage unit 105 is used to store software programs and various data. Storage unit 105 may mainly include a program storage area and a data storage area. The program storage area may store operating system programs, motion control programs, application programs (such as text editors), etc.; the data storage area may store data generated by the robot 100 during use (such as various sensor data acquired by the sensing unit 103, log file data, etc.). Furthermore, storage unit 105 may include high-speed random access memory, and may also include non-volatile memory, such as disk storage, flash memory, or other volatile solid-state memory.

[0035] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0036] Input unit 107 can be used to receive input numerical or character information. Specifically, input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as touch screen, can collect user touch operations (such as operations performed by the user using their palm, fingers, or suitable accessories on or near touch panel 1071) and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include two parts: touch detection device 1073 and touch controller 1074. Touch detection device 1073 detects the user's touch position and the signal generated by the touch operation, and transmits the signal to touch controller 1074; touch controller 1074 receives touch information from touch detection device 1073, converts it into touch point coordinates, and sends it to control module 110, and can also receive and execute commands from control module 110. In addition to touch panel 1071, input unit 107 may also include other input devices 1072. Specifically, other input devices 1072 may include, but are not limited to, one or more of the following: remote control handles, etc., without any specific limitation here.

[0037] Furthermore, the touch panel 1071 can cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the control module 110 to determine the type of touch event. Subsequently, the control module 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Although in Figure 1In this embodiment, the touch panel 1071 and the display panel 1061 are two independent components that implement input and output functions respectively. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement input and output functions. The specific implementation is not limited here.

[0038] The control module 110 is the control center of the robot 100. It connects all the components of the robot 100 through various interfaces and lines. It controls the robot 100 as a whole by running or executing the software program stored in the storage unit 105 and calling the data stored in the storage unit 105.

[0039] Power supply 111 supplies power to various components. Power supply 111 may include a battery and a power control board. The power control board controls battery charging, discharging, and power consumption management. Figure 1 In the illustrated embodiment, power supply 111 is electrically connected to control module 110. In other embodiments, power supply 111 may also be electrically connected to sensing unit 103 (such as camera, radar, speaker, etc.) and motor 1012. It should be noted that each component may be connected to a different power supply 111, or may be powered by the same power supply 111.

[0040] Based on the above embodiments, specifically, in some embodiments, a terminal device can be used to communicate with the robot 100. When the terminal device communicates with the robot 100, it can send instruction information to the robot 100. The robot 100 can receive the instruction information through the communication unit 102 and, upon receiving the instruction information, can transmit it to the control module 110, so that the control module 110 can process the instruction information to obtain the target speed value. The terminal device includes, but is not limited to, mobile phones, tablets, servers, personal computers, wearable smart devices, and other electrical appliances with image capture capabilities.

[0041] The instruction information can be determined based on preset conditions. In one embodiment, the robot 100 may include a sensing unit 103, which can generate instruction information based on the current environment of the robot 100. The control module 110 can determine whether the current speed value of the robot 100 meets the corresponding preset conditions based on the instruction information. If it does, the robot 100 will maintain its current speed value and current gait; if it does not, the control module 110 will determine a target speed value and a corresponding target gait based on the corresponding preset conditions, thereby controlling the robot 100 to move at the target speed value and the corresponding target gait. Environmental sensors may include temperature sensors, air pressure sensors, vision sensors, and sound sensors. Instruction information may include temperature information, air pressure information, image information, and sound information. The communication method between the environmental sensors and the control module 110 can be wired or wireless. Wireless communication methods include, but are not limited to: wireless networks, mobile communication networks (3G, 4G, 5G, etc.), Bluetooth, and infrared.

[0042] Figure 3 This is a flowchart illustrating a method for determining a target location according to an embodiment of this application. This method can be executed by an electronic device with processing capabilities, such as a server, cloud server, terminal, or robot, etc., without specific limitations herein. Figure 3 As shown, taking an application to a robot as an example, the method specifically includes the following steps:

[0043] Step 210: Obtain the local images collected by the robot, and determine the local topology map based on the local images and the global object map corresponding to the robot's surrounding environment.

[0044] As one approach, when a robot is powered on and in use, its initial location in its surrounding environment cannot be accurately determined. To accurately obtain this initial location information, local images captured by the robot can be acquired. These local images can be images of relatively stable structures in the surrounding environment, captured by the robot's image acquisition device. For example, when the robot is indoors, local images of relatively stable structures such as doors, door frames, and wall grounding wires, as well as local images of infrequently moved furniture such as washing machines, refrigerators, sofas, and beds, can be acquired. This allows the robot's initial location information to be determined based on the infrequently moved furniture or relatively stable structures in the local images. Optionally, the local images can be RGBD images or other types of images; the robot's image acquisition device can be an RGBD camera or other cameras or webcams, which can be set according to actual needs and are not specifically limited here.

[0045] One approach is to create a global object map by stitching together images of all objects captured by the robot during its movement in response to global localization operations. This global object map can then be used to locate local objects within local images. Alternatively, the global object map can be obtained by stitching together images of the same objects at the same locations from all captured object-related images.

[0046] As another approach, a robot can be controlled to move around the scene requiring mapping. During this movement, the robot's odometry unit determines the odometry coordinates of each object in the acquired images. Then, across multiple frames acquired by the robot, the same object is merged into a single object based on a preset distance threshold, thereby determining a global object map. For example... Figure 4 As shown, the global object map can be determined by images collected by the robot during its movement. Figure 4 In this context, it is a global object map determined by the three-dimensional bounding boxes of objects.

[0047] One approach is to locate objects in a local image within a global object map, and then identify the object with the fewest identical object types in the local image as the central object. This central object is then used as the topological center node of the local topology graph. Other objects in the local image are designated as other nodes, and these other nodes are connected to the central node to obtain the local topology graph. Optionally, this local topology graph can be a local star topology graph or other topological structures, depending on actual needs; no specific limitations are imposed here. Figure 5 As shown, when the door is determined to be the object with the fewest number, it is designated as the central object. The central object is then connected to other objects, and the door serves as the topological center node in the local topology graph. The convex lines, concave lines, and ground lines of the walls are the other nodes. The topological center node is then connected to the other nodes to obtain the local topology graph.

[0048] Step 220: Based on the local topology graph, determine at least one global topology graph of the environment surrounding the robot.

[0049] As one approach, a global topology map refers to constructing at least one global topology map by taking the object corresponding to the center node of the local topology map as the reference and centering each object in the global object map with the same object as the object corresponding to the center node. Optionally, a distance threshold between the global topology center node and other nodes of at least one global topology map can be preset. Based on this distance threshold, objects corresponding to nodes in the global object map whose distance to the global topology center node is less than the distance threshold are identified as end nodes, thereby constructing at least one global topology map of the robot's surrounding environment.

[0050] In some embodiments, such as Figure 6 As shown, step 220 includes:

[0051] Step 221: Determine the local target node in the local topology graph.

[0052] One approach is to determine the local target node by obtaining the object labels corresponding to each topological node in the local topology map. The object label can indicate the object corresponding to the topological node in the local image. Optionally, in the local topology map, topological nodes corresponding to objects of the same type have the same object label. The object label can also indicate the rarity of the object type corresponding to each topological node, which refers to the proportion of the corresponding object type in the global object map. Optionally, the topological node with the highest rarity indicated by the object label in the local topology map can be determined as the local target node based on the object label.

[0053] Alternatively, local target nodes in the local topology map can be determined based on local images. Optionally, the local image can be a depth image captured by a depth camera; that is, an image containing the distance between the depth camera and each object. In this local image, each pixel value represents the actual distance between the depth camera and the object. Then, the object closest to the depth camera can be determined based on the average pixel value of each object in the local image. The object with the smallest average pixel value is then identified as the target object, and its topological node in the local topology map is determined as the local target node.

[0054] Step 222: Determine the global target object in the global object map, wherein the global target object and the object corresponding to the local target node are of the same type.

[0055] One approach is to first determine the category label corresponding to the local target node. This category label indicates the object category corresponding to each topological node. Then, based on this category label, global target objects of the same category as the category label are determined in the global object map. Optionally, the number of determined global target objects is at least one.

[0056] Step 223: Determine a global reference object in the global object map based on the global target object. The global reference object is an object in the global object map whose distance from the global target object is less than or equal to a first distance threshold.

[0057] As one approach, the first distance threshold can be calculated based on the distances between other topological nodes in the local topological graph and the local target node. Optionally, the first distance threshold can be determined based on the aggregate distances between objects in the local image. The geometric distances between objects in the local image can include distances between vertical lines, distances between vertical lines and horizontal lines, distances between vertical lines and bounding boxes, distances between vertical lines and planar polygons, distances between horizontal lines and planar polygons, distances between planar polygons, distances between planar polygons and bounding boxes, and distances between bounding boxes, etc. Optionally, the first distance threshold can be determined by the maximum distance between the geometric distances between objects in the local image and the object corresponding to the local target node.

[0058] The distance between vertical lines can refer to the distance between their reference points. A reference point is the intersection of the lines projected onto a Cartesian coordinate system (i.e., the horizontal plane where Z=0) along the line's direction. Optionally, the distance between the reference points can be calculated using the Euclidean distance formula. The distance between a vertical line and a horizontal line can be the projected distance from the reference point of the vertical line to the horizontal line. The distance between a vertical line and its bounding box can be the sum of the distance from the reference point of the vertical line to the center of the bounding box, the X-axis coordinate of the projection of the reference point of the vertical line onto the bounding box, and the Y-axis coordinate of the projection of the reference point of the vertical line onto the bounding box. The distance between a vertical line and a planar polygon can be the distance from the reference point of the vertical line to the center of the planar polygon. The distance between a horizontal line and its bounding box can be the projected distance from the center of the bounding box to the horizontal line and the distance from the midpoint of the horizontal line to the bounding box. The distance between the horizontal line and the planar polygon can be the projection distance from the midpoint of the planar polygon to the horizontal line; the distance between planar polygons can be the distance between the centers of the planar polygons; the distance between bounding boxes can be the distance between the centers of the bounding boxes and the projection distance from the center of one bounding box to the X-axis of another bounding box, and the projection distance from the center of one bounding box to the Y-axis of another bounding box.

[0059] As one approach, object recognition can be performed on the local image before determining the geometric distance between objects, thus facilitating the identification of each object. Optionally, bounding box extraction algorithms can be used to extract the bounding boxes of three-dimensional objects (such as sofas, appliances, cabinets, etc.), where the bounding boxes can be 3D bounding boxes. Optionally, different object labels can be set for different types of objects when performing object recognition on the image. Optionally, an object recognition model can be used to recognize the local image, where vertical and horizontal lines can be wall lines, wall edges, cabinet lines, cabinet-wall lines, sofa ground lines, etc., in the robot's surrounding environment; planar polygons can be door frames, window frames, picture frames, etc. Optionally, when recognizing objects in the local image based on the object recognition model, the coordinates of each object in the image coordinate system corresponding to the recognition result can be converted to the camera coordinate system of the image acquisition device that acquired the local image, thus facilitating the subsequent calculation of the distance between objects. Optionally, the bounding box can be an oriented bounding box (OBB). This OBB contains the corresponding object and is the smallest cuboid relative to any coordinate axis. The OBB can enclose the object as tightly as possible based on its shape characteristics, thus more closely approximating the object and significantly reducing the number of bounding boxes, thereby avoiding the need for intersection detection between a large number of bounding boxes. Optionally, the object recognition model for object recognition can be a deep learning model including at least one neural network, such as a convolutional neural network, recurrent neural network, or Hofit neural network.

[0060] Step 224: Determine at least one global topology map of the robot's surrounding environment based on the global reference object and the global target object.

[0061] One approach is to determine the distance between the global reference object and the global target object after identifying them, and then connect the global reference object and the global target object based on the distance between them to obtain a global topology graph.

[0062] Optionally, since there is at least one global target object, there is also at least one corresponding global topology graph, which can be a global star topology graph. Each global topology graph can be determined by the central object of a global target object. For example... Figure 7 As shown, when a local target node in the local topology is determined to be a door, all doors are identified in the global object map and designated as global target objects. The distances between each door and other objects are calculated. Then, global reference objects with distances less than or equal to a first distance threshold are selected in the global object map to determine at least one global topology. Figure 7 There are two global topology graphs.

[0063] Optionally, the distance between the global reference object and the global target object can also be determined by calculating the geometric distance between them. The method for calculating the geometric distance between the global reference object and the global target object can be found in step 223, which describes the calculation method for the geometric distance between objects in the local image; it will not be repeated here.

[0064] Please continue reading. Figure 3 Step 230: Determine at least one end node corresponding to each of the global topology graphs.

[0065] In one approach, an end node can be the endpoint of every link connected to the global center node (i.e., the intersection or starting point of all topological links in the global topology graph). Alternatively, an end node can also be a node corresponding to an end object in the global object map corresponding to the global topology graph.

[0066] Optionally, in the global topology graph, a corresponding node label is set for each node. The node label can be used to indicate the link corresponding to each node in the global topology graph and its position in each link. Then, by obtaining the node labels of at least one node in the global topology graph, the node whose node label indicates the end position of the corresponding node in each link of the global topology graph is determined as the end node.

[0067] Step 240: Based on the terminal nodes corresponding to at least one global topology graph, match the local topology graph with at least one global topology graph to obtain a set of node pairs corresponding to at least one global topology graph and the local topology graph. The set of node pairs is the set of node pairs with the same label in the local topology graph and at least one global topology graph.

[0068] As one approach, a label refers to the corresponding object label, used to indicate the object corresponding to the node. Optionally, matching the local topology graph with at least one global topology graph can be achieved by first aligning the central node of the local topology graph with the central node of at least one global topology graph, that is, aligning the local target node in the local topology graph with at least one global target node as a reference, and then matching each link in the local topology graph with each link in at least one global topology graph to determine the set of node pairs corresponding to at least one global topology graph and the local topology graph.

[0069] Optionally, at least one global topology graph can be a global star topology graph, and the local topology graph can be a local star topology graph. In both the global and local star topologies, a link includes its corresponding central node and its corresponding terminal node. Furthermore, the set of node pairs corresponding to the at least one global and local topology graphs can be determined based on the labels of the terminal nodes. Optionally, the central node of the local topology graph and the central node of the at least one global topology graph can also be added to the set of node pairs.

[0070] Optionally, when it is determined that the object label corresponding to the end node of the local topology graph is the same as that of the end node of at least one global topology graph, the distance between each center node and the end node in the local topology graph and at least one global topology graph can be calculated respectively. Then, based on the distance, the matching link is determined in the corresponding link in the local topology graph and the global topology graph, and the node pair corresponding to the matching link is added to the node pair set.

[0071] In some embodiments, such as Figure 8 As shown, step 240 includes:

[0072] Step 241: Match the local links in the local topology graph with the global links in at least one global topology graph to obtain the target links. The target links are links that have the same end nodes in the local topology graph and the corresponding global topology graph.

[0073] As one approach, a local link refers to each link in a local topology graph; similarly, a global link refers to each link in at least one global topology graph. Optionally, the matching method could be to first align the reference node corresponding to each local link in the local topology graph with the reference node corresponding to each global link in at least one global topology graph, and then compare the corresponding local links with other local links to achieve matching.

[0074] Optionally, the target link can be determined by the node labels of the end nodes corresponding to the local links in the local topology graph and at least one global link in the global topology graph. The node labels indicate the objects corresponding to each end node; if the objects indicated by the node labels are the same, the corresponding local link and global link can be identified as the target link.

[0075] Step 242: Obtain the first distance and the second distance of the target link.

[0076] As one approach, the first distance refers to the distance between the center node corresponding to the target link and the object corresponding to the end node in a local topology graph. It can be understood that the second distance refers to the distance between the center node corresponding to the target link and the object corresponding to the end node in at least one global topology graph. Optionally, the first and second distances can be the geometric distance between objects.

[0077] In one approach, the first distance and the second distance of the target link can be directly obtained from the local topology map and at least one global topology map. Optionally, when determining the local topology map, the distance between the local target object corresponding to the center node of the local topology map and the local reference object corresponding to the end node of each link can be calculated, and the distance between the objects corresponding to each link can be marked on the corresponding link, thereby directly obtaining the first distance of the target link from the local topology map. The specific calculation method for the distance between the local target object corresponding to the center node of the local topology map and the local reference object corresponding to the end node of each link can be found in step 223, which describes the calculation method for the geometric distance between objects in the local image, and will not be repeated here. Similarly, when determining at least one global topology map, the distance between the global target object corresponding to the center node of at least one global topology map and the global reference object corresponding to the end node of each link can be calculated, and the distance between the objects corresponding to each link can be marked on the corresponding link, thereby directly obtaining the second distance of the target link from at least one global topology map. The specific calculation method for the distance between the global target object corresponding to the central node of at least one global topology graph and the global reference object corresponding to the end node of each link can be found in step 223, which describes the calculation method for the geometric distance between objects in the local image. It will not be repeated here.

[0078] Step 243 determines the distance difference between the first distance and the second distance of the target link.

[0079] One approach is to directly subtract the second distance of the target link from its first distance to obtain the distance difference. Alternatively, to determine whether a target link in the local topology map matches a target link in at least one global topology map, the distance difference can be used. If the distance difference is less than a pre-set distance difference value, the target link in the local topology map matches a target link in at least one global topology map; if the distance difference is greater than or equal to a pre-set distance difference value, the target link in the local topology map does not match a target link in at least one global topology map.

[0080] Step 244: If the distance difference is less than the distance difference threshold, add the end node in the target link to the node pair set.

[0081] As one approach, when the distance difference is determined to be less than a distance difference threshold, it can be determined that the target link in the local topology graph matches at least one target link in the global topology graph. The corresponding end nodes of the target link in both the local and at least one global topology graph are then added to the node pair set. Optionally, the center node in the target link is also added to the corresponding node pair set. Optionally, the distance difference threshold can be set according to actual needs and is not specifically limited here.

[0082] Please continue reading. Figure 3 Step 250: Select a first preset number of sample node pairs from the node pair set, and determine a second preset number of candidate pose information based on the first preset number of sample node pairs.

[0083] As one approach, a first preset number of sample node pairs can be randomly selected from the set of node pairs using random sampling. Optionally, each group of sample node pairs in the first preset number of sample node pairs may include 3 sample node pairs, or it may include other numbers of sample node pairs. Optionally, the second preset number may be the same as or different from the first preset number, and can be set according to actual needs; no specific limitation is made here.

[0084] One approach is to determine the second preset number of candidate pose information based on the first preset number of sample node pairs. This can be achieved by determining the coordinates in the world coordinate system based on the coordinates of each of the first preset number of sample node pairs in the image coordinate system, thereby determining the second preset number of candidate pose information. Optionally, the first preset number of sample node pairs can be aligned in the world coordinate system, and the alignment error between the sample node pairs can be determined. Then, the corresponding transpose matrix can be determined based on the alignment error between the first preset number of sample node pairs and the Jacobian matrix of the transpose matrix associated with the alignment error. Finally, the second preset number of candidate pose information can be determined based on the transpose matrix.

[0085] In some embodiments, such as Figure 9 As shown, step 250 includes:

[0086] Step 251: Select a first preset number of sample node pairs from the set of node pairs.

[0087] As one approach, a first preset number of sample node pairs can be randomly selected from the set of node pairs using a random sampling method. Optionally, each group of sample node pairs in the first preset number of sample node pairs may include 3 sample node pairs, or it may include other numbers of sample node pairs.

[0088] Step 252: Determine the alignment error of each sample node pair.

[0089] One approach is to align the sample node pairs using the Levenberg-Marquardt (LM) method in least squares estimation to determine the alignment error of each sample node pair. Optionally, the alignment error of each sample node pair can be the alignment error corresponding to the object in its corresponding local topology graph or at least one global topology graph.

[0090] Optionally, since the sample node pairs are end nodes corresponding to the same object in the local topology graph and at least one global topology graph, the alignment error of the sample node pairs includes the alignment error between lines, the alignment error between planar polygons, and the alignment error between bounding boxes.

[0091] The alignment error between straight lines can be calculated using the coordinates of the corresponding straight lines of the sample nodes in the map coordinate system, specifically the coordinates of the first and second endpoints of the first and second straight lines in the map coordinate system, respectively. The first straight line refers to the object corresponding to the sample node pair in the local topology map, and the second straight line refers to the object corresponding to the sample node pair in at least one global topology map. The map coordinate system can be the coordinate system corresponding to the object map established by the robot based on objects in its surrounding environment. The alignment error can be determined by first calculating the first distance between one end of the first straight line and the second straight line, then calculating the second distance between the other end of the first straight line and the second straight line, and finally calculating the average distance based on the first and second distances. Optionally, the alignment error can be determined using the formula: To calculate the distance between one end of the first line and the second line, where d ε p is the distance between one end of the first line and the second line. a and p b These are the coordinates of the second straight line in the map coordinate system, p. i The coordinates of one end of the first straight line in the map coordinate system.

[0092] Optionally, the alignment error between planar polygons can be calculated using the coordinates of the corresponding planar polygons of the sample nodes in the map coordinate system, i.e., the coordinates of each corner point of the first and second planar polygons in the map coordinate system. The first planar polygon refers to the object corresponding to the sample node pair in the local topology map, and the second planar polygon refers to the object corresponding to the sample node pair in at least one global topology map. The distances from each corner point of the first planar polygon to the second planar polygon can be calculated first, then the average distance can be calculated based on these distances, and finally, the average distance can be determined as the alignment error between the first and second planar polygons. Optionally, it can be calculated using the formula: To calculate the distances between each corner point of the first planar polygon and the second planar polygon, where d δ p is the distance between a corner point of the first planar polygon and the second planar polygon. i ' is the coordinate of a corner point of the first planar polygon in the map coordinate system, p l p j p m These are the coordinates of the three corner points of the second planar polygon in the map coordinate system.

[0093] Optionally, the alignment error between bounding boxes can be calculated using the coordinates of the corresponding bounding boxes of the sample nodes in the map coordinate system, i.e., the coordinates of the center points of the first and second bounding boxes in the map coordinate system. The first bounding box refers to the object corresponding to the sample node pair in the local topology map, and the second bounding box refers to the object corresponding to the sample node pair in at least one global topology map. The centers of the first and second bounding boxes can be determined separately. These centers can be the centroid of the object corresponding to the first bounding box and the centroid of the object corresponding to the second bounding box, respectively, or they can be the geometric center of the first and second bounding boxes, depending on actual needs. Optionally, the distance between the centers of the first and second bounding boxes can be directly determined as the alignment error between them. Optionally, it can be calculated using the formula: d α =p i "-p j Let ' calculate the distance between the center of the first bounding box and the center of the second bounding box, where d' α p is the distance between the center of the first bounding box and the center of the second bounding box. i " represents the coordinates of the center of the first bounding box in the map coordinate system, p" j ' represents the coordinates of the center of the second bounding box in the map coordinate system.

[0094] Step 253: Obtain the transformation matrix corresponding to the robot.

[0095] In one approach, the transformation matrix corresponding to the robot is the transformation matrix of the robot's image acquisition device relative to the robot's base. This transformation matrix can be determined during the calibration of the robot's image acquisition device.

[0096] Step 254: Determine a second preset number of candidate pose information based on the alignment error and the transformation matrix.

[0097] As one approach, after determining the alignment error between each pair of sample nodes, the transpose matrix between the robot's camera coordinate system and the map coordinate system is first determined based on the Jacobian matrix of the alignment error with respect to the transpose matrix. Then, a second preset number of candidate pose information is determined based on the transpose matrix and the transformation matrix.

[0098] Optionally, the alignment error between different objects may have different Jacobian matrices for the transpose of the matrix. Specifically, the alignment error between lines may have different Jacobian matrices for the transpose of the matrix. The alignment error between two planar polygons is represented by the Jacobian matrix of the transpose of the polygon. The alignment error between bounding boxes is given by the Jacobian matrix of the transpose matrix. Where T is the transpose matrix. Optionally, the transpose matrix can be composed of a rotation matrix R and a translation vector t, i.e.

[0099] One approach is to calculate the candidate pose using the transpose and transformation matrices. Alternatively, it can be obtained using formula T. mb =T mc ·T bc -1 The candidate pose information is calculated, where T mb For candidate pose information, T mc Let T be the transpose of the matrix. bc Let T be the transformation matrix. bc -1 It is the inverse of the transpose of the matrix.

[0100] Please continue reading. Figure 3 Step 260: Determine the target position information of the robot based on the second preset number of candidate pose information.

[0101] One approach is to first filter a second preset number of candidate pose information, then calculate the confidence level of each of the filtered candidate pose information, and finally determine the candidate pose information with the highest confidence level as the target location information. Optionally, based on the determined filtered candidate pose information, the first coordinate of an object in the global object map corresponding to that object in the map coordinate system can be determined, as well as the second coordinate of the object that is the same as that object in the global object map. Then, the association error between the first and second coordinates can be determined. Finally, based on the association error and the 3sigma criterion, the matching confidence level of each object can be calculated, and the candidate pose information with the highest straightness can be determined as the target location information.

[0102] Optionally, the method for filtering the second preset number of candidate pose information may be to filter out candidate poses in the candidate pose information where the angle between the x-axis and the map x-axis and / or y-axis plane is greater than 30 degrees, or to filter out candidate pose information in the candidate pose information where there is a collision with map furniture objects, or where the projection distance to the wall ground wire is less than a certain threshold. Other filtering methods may also be used to filter out candidate pose information, which are not specifically limited here.

[0103] In some embodiments, such as Figure 10 As shown, step 260 includes:

[0104] Step 261: Determine the association error between objects with the same label in the global object map, where the distance between objects with the same label is less than or equal to a second distance threshold.

[0105] As a method, the second distance threshold can be a distance threshold set according to actual needs. Optionally, the second distance threshold can be a value equal to or less than the first distance threshold, without specific limitations here.

[0106] As one approach, "same label" refers to labels for the same object type. Association error includes association error between lines, between planar polygons, and between bounding boxes. Optionally, the method for calculating association error can refer to the method for calculating alignment error in step 252, and will not be repeated here. Optionally, when calculating association error, the coordinates of each object in the camera coordinate system can be transformed to the map coordinate system based on the candidate pose information. Then, the association error between objects with the same label can be determined based on the coordinate information of each object in the map coordinate system.

[0107] Step 262: Determine the matching confidence of the second preset number of sample nodes based on the association error.

[0108] One approach is to calculate the match confidence for each pair of objects with the same label using the 3sigma criterion. Alternatively, it can be calculated using the formula... We calculate the matching confidence of objects with the same label corresponding to each candidate pose information. Here, Bel is the matching confidence, δ is the maximum allowable error between each pair of objects with the same label, and d is the association error between each pair of objects with the same label.

[0109] Optionally, after determining the matching confidence of all objects with the same label corresponding to each candidate pose information, the average matching confidence of all objects with the same label corresponding to each candidate pose information is determined, and the average confidence is determined as the matching confidence of the corresponding candidate pose information.

[0110] Step 263: Determine the target position information from the second preset number of candidate pose information based on the matching confidence.

[0111] In one approach, to determine the target position of the robot more accurately, the candidate pose information corresponding to the highest matching confidence in the matching confidence can be determined as the target pose information, thereby enabling the determination of the target position information based on the target pose information.

[0112] Optionally, after determining the target pose information, the relative position information of an object to the robot can be determined based on the local images acquired by the robot. Then, the coordinates of the object in the map coordinate system can be transformed to the robot's corresponding coordinate system based on the target pose information. Finally, the robot's target position information can be determined based on the relative position information and the object's coordinates in the robot's corresponding coordinate system. Optionally, the target position information can be the robot's coordinates in the corresponding coordinate system (e.g., the coordinate system of the robot base relative to the map coordinate system).

[0113] In the embodiments of this application, a local topology map is first determined based on the images collected by the robot and the global object map corresponding to the robot's surrounding environment. This allows for the determination of at least one global topology map of the robot's environment based on the local topology map. Then, based on the end nodes corresponding to each location in the at least one global topology map, the local topology map is matched with the at least one global topology map to determine a set of node pairs corresponding to nodes with the same labels in both the global and local topology maps. Candidate pose information is then determined based on the sample node pairs in the node pair set. Finally, the robot's target position information is determined based on the candidate pose information. In this application's solution, even in indoor environments with limited or no precise positioning, the robot can be located based on objects in its surrounding environment, resulting in more accurate target position information. Furthermore, this method improves the efficiency of determining the robot's target position information without relying on precise positioning information (such as high-precision maps).

[0114] Figure 11 This is a flowchart illustrating a method for determining a target location according to an embodiment of this application. This method can be executed by an electronic device with processing capabilities, such as a server, cloud server, terminal, or robot, etc., without specific limitations herein. Figure 11 As shown, taking an application to a robot as an example, the method specifically includes the following steps:

[0115] Step 310: Obtain the local image captured by the robot.

[0116] Step 320 performs object recognition on the local image to obtain at least one local object in the local image.

[0117] One approach to object recognition in a local image is to use an object recognition model or an object recognition neural network to identify the local image. Optionally, at least one local object in the resulting local image can be the geometric image of the local object and its corresponding object type.

[0118] Optionally, to ensure the accuracy of object recognition results, the object recognition model or object recognition neural network needs to be trained in advance. Specifically, a sample set is pre-constructed, which includes multiple sample images and label information for each object in the sample images. The label information for each object in the sample images is used to indicate the object type and shape of the object in the sample image corresponding to the recognition task.

[0119] During training, each sample image is input into the object recognition model or object recognition neural network to extract shallow features and obtain feature maps for each sample image. Then, based on the feature maps, the sample recognition result corresponding to the recognition task of the sample image is output. It can be understood that the sample recognition result indicates the object type and shape of the object indicated by the recognition task corresponding to the sample image.

[0120] Then, based on the label information of the sample image and the sample recognition result corresponding to the sample image, the loss value of the loss function is calculated. If the loss value does not converge, the parameters of the object recognition model or object recognition neural network are adjusted in reverse. The object recognition model or object recognition neural network with adjusted parameters outputs the sample recognition result for the sample image again, and the loss value of the loss function is calculated again until the loss value converges.

[0121] For each sample image, repeat the above process until the training termination condition is met, at which point the training of the object recognition model or object recognition neural network ends. Afterwards, the object recognition model or object recognition neural network can be used online for object recognition, ensuring the accuracy of object recognition.

[0122] Step 330: Based on the global object map corresponding to the robot's surrounding environment and at least one local object, determine candidate objects in at least one local object, as well as local reference objects associated with the candidate objects.

[0123] One approach is to first determine the number of objects of the same type and the total number of objects in the global object map, then calculate the proportion of each type of object in the global object map, and then determine the rarity of the object types corresponding to all local objects in the local image based on the object type corresponding to at least one local object. This allows for the determination of candidate objects based on the rarity of the object types corresponding to all local objects in the local image. After determining the candidate objects, at least one object in the local image other than the candidate objects is determined as a local reference object.

[0124] In some embodiments, such as Figure 12 As shown, step 330 includes:

[0125] Step 331: Determine the object type corresponding to at least one local object.

[0126] As one approach, when performing object recognition on an image, the recognition result may include the object category corresponding to each local object in the local image. Optionally, the object category corresponding to at least one local object can be determined by obtaining the object label corresponding to each local object. The object label can be added to the recognition result by an object recognition model or an object recognition neural network during object recognition of the local image, and this object label is used to indicate the object category corresponding to at least one local object.

[0127] Step 332: Determine the number of objects corresponding to the object types in the global object map of the robot's surrounding environment.

[0128] One approach to determining the number of objects of each object type corresponding to at least one local object in the global map is to first determine global objects of the same object type as the object type corresponding to at least one local object in the global object map, and then determine the number of global objects corresponding to each object type, thereby determining the number of objects corresponding to each object type.

[0129] Step 333: Determine the rarity of the object type based on the quantity of objects.

[0130] One approach is to determine the rarity of each object type for at least one local object by considering the total number of objects in the global object map and the total number of objects of each object type for at least one local object. Alternatively, the rarity of each object type can be calculated using the formula n / m, thereby determining the rarity of all object types. Here, m is the total number of objects in the global object map, and n is the total number of objects of a particular type for at least one local object.

[0131] Step 334: Based on the rarity corresponding to the object type, determine at least one candidate object from the local objects.

[0132] One approach is to identify the local object with the highest rarity (i.e., the smallest percentage of its kind among all objects) as a candidate object. Optionally, if the number of local objects with the highest rarity in the local image is greater than a certain threshold, then the local object closest to the robot among these highest-rarity local objects is identified as a candidate object. Alternatively, the distance between the local object and the robot can be determined by identifying the pixel values ​​corresponding to each of the highest-rarity local objects in the local image; for example, the average pixel value of a local object in the local image can be used to determine the distance to the robot.

[0133] Please continue reading. Figure 4Step 340: Establish a local topology map based on the candidate objects and local reference objects.

[0134] One approach is to connect the candidate objects and local reference objects after identifying them, thereby establishing a topological graph. Optionally, the distances between candidate objects and local reference objects can be calculated and annotated in the local topological graph. These distances include the distances between vertical lines, between a vertical line and a horizontal line, between a vertical line and a bounding box, between a vertical line and a planar polygon, between a horizontal line and a bounding box, between a horizontal line and a planar polygon, between planar polygons, between planar polygons, between planar polygons and their bounding boxes, and between bounding boxes. Specific calculation methods can be found in step 223 and will not be elaborated upon here.

[0135] Step 350: Based on the local topology graph, determine at least one global topology graph of the environment surrounding the robot.

[0136] Step 360: Determine at least one corresponding end node of the global topology graph.

[0137] Step 370: Based on the terminal nodes corresponding to at least one global topology graph, match the local topology graph with at least one global topology graph to obtain a set of node pairs corresponding to at least one global topology graph and the local topology graph. The set of node pairs is the set of node pairs with the same label in the local topology graph and at least one global topology graph.

[0138] Step 380: Select a first preset number of sample node pairs from the node pair set, and determine a second preset number of candidate pose information based on the first preset number of sample node pairs.

[0139] Step 390: Determine the target position information of the robot based on the second preset number of candidate pose information.

[0140] The specific steps of steps 310 and 350-390 can be found in steps 210-260, and will not be repeated here.

[0141] In this embodiment, at least one local object is identified in the local image acquired by the robot. Then, the rarity of the object type corresponding to each of the at least one local object and all objects in the global object map can be determined. Candidate objects are then identified among the at least one local object based on the rarity, and local reference objects associated with the candidate objects are determined. A local topology map can then be determined based on the candidate objects and local reference objects, thereby improving the accuracy of the local topology map. This improves the accuracy when determining the global topology map and the robot's target position information based on the local topology map.

[0142] Figure 13 This is a block diagram illustrating a map generation apparatus according to an embodiment of this application, such as... Figure 13 As shown, the target location determination device 400 includes: a local topology map determination module 410, a global topology map determination module 420, an end node determination module 430, a node pair set determination module 440, a candidate pose information determination module 450, and a target location information determination module 460.

[0143] The local topology map determination module 410 is used to acquire local images collected by the robot and determine a local topology map based on the local images and the global object map corresponding to the robot's surrounding environment. The global topology map determination module 420 is used to determine at least one global topology map of the robot's surrounding environment based on the local topology map. The end-point node determination module 430 is used to determine the end-point nodes corresponding to each of the at least one global topology map. The node pair set determination module 440 is used to match the local topology map with the at least one global topology map based on the end-point nodes corresponding to each of the at least one global topology map, to obtain a set of node pairs corresponding to the at least one global topology map and the local topology map. The node pair set is the set of node pairs with the same corresponding labels in the local topology map and the at least one global topology map. The candidate pose information determination module 450 is used to filter a first preset number of sample node pairs in the node pair set and determine a second preset number of candidate pose information based on the first preset number of sample node pairs. The target position information determination module 460 is used to determine the target position information of the robot based on the second preset number of candidate pose information.

[0144] In some embodiments, the local topology map determination module 410 includes: an acquisition submodule for acquiring local images collected by the robot; an object recognition submodule for performing object recognition on the local images to obtain at least one local object in the local images; a determination submodule for determining candidate objects among the at least one local object and local reference objects associated with the candidate objects based on a global object map corresponding to the robot's surrounding environment and the at least one local object; and a local topology map determination submodule for establishing a local topology map based on the candidate objects and the local reference objects.

[0145] In some embodiments, the determining submodule includes: an object type determining unit, configured to determine the object type corresponding to the at least one local object; an object quantity determining unit, configured to determine the object quantity corresponding to the object type in a global object map corresponding to the environment surrounding the robot; a rarity determining unit, configured to determine the rarity corresponding to the object type based on the object quantity; and a candidate object determining unit, configured to determine candidate objects among the at least one local object based on the rarity corresponding to the object type.

[0146] In some embodiments, the global topology map determination module 420 includes: a local target node determination submodule, configured to determine local target nodes in the local topology map; a global target object determination submodule, configured to determine global target objects in the global object map, wherein the global target objects are of the same type as the objects corresponding to the local target nodes; a global reference object determination submodule, configured to determine global reference objects in the global object map based on the global target objects, wherein the global reference objects are objects in the global object map whose distance from the global target objects is less than or equal to a first distance threshold; and a global topology map determination submodule, configured to determine at least one global topology map of the environment surrounding the robot based on the global reference objects and the global target objects.

[0147] In some embodiments, the node pair set determination module 440 includes: a target link determination submodule, configured to match local links in the local topology graph with global links in at least one global topology graph to obtain target links, wherein the target links are links with the same end nodes in the local topology graph and the corresponding global topology graph; a distance acquisition submodule, configured to acquire a first distance and a second distance of the target link; a distance difference determination submodule, configured to determine the distance difference between the first distance and the second distance of the target link; and a node pair set determination submodule, configured to add the end nodes of the target links to the node pair set if the distance difference is less than a distance difference threshold.

[0148] In some embodiments, the candidate pose information determination module 450 includes: a sample node pair determination submodule, used to filter a first preset number of sample node pairs in the node pair set; an alignment error determination submodule, used to determine the alignment error corresponding to each of the sample node pairs; a transformation matrix acquisition submodule, used to acquire the transformation matrix corresponding to the robot; and a candidate pose information determination submodule, used to determine a second preset number of candidate pose information based on the alignment error and the transformation matrix.

[0149] In some embodiments, the target location information determination module 460 includes: an association error determination submodule, configured to determine the association error between objects with the same label in the global object map, wherein the distance between the objects with the same label is less than or equal to a second distance threshold; a matching confidence determination submodule, configured to determine the matching confidence of the second preset number of sample nodes according to the association error; and a target location information determination submodule, configured to determine the target location information from the second preset number of candidate pose information according to the matching confidence.

[0150] This application also provides a robot, which may include: a body; and a control system communicating with the body, the control system including a processor and a memory communicating with the processor, the memory storing instructions, the instructions being executed on the processor to cause the processor to perform the following methods: acquiring a local image collected by the robot; determining a local topology map based on the local image and a global object map corresponding to the robot's surrounding environment; determining at least one global topology map of the robot's surrounding environment based on the local topology map; determining the end nodes corresponding to each of the at least one global topology map; matching the local topology map with the at least one global topology map based on the end nodes corresponding to each of the at least one global topology map to obtain a set of node pairs corresponding to the at least one global topology map and the local topology map, the set of node pairs being the set of node pairs with the same corresponding labels in the local topology map and the at least one global topology map; filtering a first preset number of sample node pairs in the set of node pairs; determining a second preset number of candidate pose information based on the first preset number of sample node pairs; and determining the target position information of the robot based on the second preset number of candidate pose information.

[0151] The robot achieves the following: acquiring a local image collected by the robot and determining a local topology map based on the local image and a global object map corresponding to the robot's surrounding environment; acquiring a local image collected by the robot; performing object recognition on the local image to obtain at least one local object in the local image; determining candidate objects among the at least one local object and local reference objects associated with the candidate objects based on the global object map corresponding to the robot's surrounding environment and the at least one local object; and establishing a local topology map based on the candidate objects and the local reference objects.

[0152] The robot implements the step of determining candidate objects among at least one local object based on a global object map corresponding to the robot's surrounding environment and at least one local object, including: determining the object type corresponding to the at least one local object; determining the number of objects corresponding to the object type in the global object map corresponding to the robot's surrounding environment; determining the rarity of the object type based on the number of objects; and determining candidate objects among at least one local object based on the rarity of the object type.

[0153] The robot implements the determination of at least one global topology map of the robot's surrounding environment based on the local topology map, including: determining local target nodes in the local topology map; determining global target objects in the global object map, wherein the global target objects and the objects corresponding to the local target nodes are of the same type; determining global reference objects in the global object map based on the global target objects, wherein the global reference objects are objects in the global object map whose distance from the global target objects is less than or equal to a first distance threshold; and determining at least one global topology map of the robot's surrounding environment based on the global reference objects and the global target objects.

[0154] The robot implements the step of matching the local topology map with at least one global topology map to obtain a set of node pairs corresponding to the at least one global topology map and the local topology map, including: matching local links in the local topology map with global links in the at least one global topology map to obtain target links, wherein the target links are links with the same end nodes in the local topology map and the corresponding global topology map; obtaining a first distance and a second distance of the target links; determining the distance difference between the first distance and the second distance of the target links; and if the distance difference is less than a distance difference threshold, adding the end nodes in the target links to the set of node pairs.

[0155] The robot implements the following steps: filtering a first preset number of sample node pairs in the set of node pairs, and determining a second preset number of candidate pose information based on the first preset number of sample node pairs. This includes: filtering the first preset number of sample node pairs in the set of object pairs; determining the alignment error corresponding to each of the sample node pairs; obtaining the transformation matrix corresponding to the robot; and determining the second preset number of candidate pose information based on the alignment error and the transformation matrix.

[0156] The robot's ability to determine its target position information from the second preset number of candidate pose information includes: determining the association error between objects with the same label in the global object map, wherein the distance between objects with the same label is less than or equal to a second distance threshold; determining the matching confidence of the second preset number of sample nodes based on the association error; and determining the target position information from the second preset number of candidate pose information based on the matching confidence.

[0157] According to one aspect of this application, a computer-readable storage medium is also provided, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable storage medium carries computer-readable instructions that, when executed by a processor, implement the methods in any of the above embodiments.

[0158] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0159] Other embodiments of this application will readily conceive of by those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0160] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for determining the location of a target, characterized in that, The method includes: Acquire local images collected by the robot, and determine a local topology map based on the local images and the global object map corresponding to the robot's surrounding environment; Identify the local target node in the local topology graph; In the global object map, a global target object is determined, wherein the global target object is of the same type as the object corresponding to the local target node; Based on the global target object, a global reference object is determined in the global object map. The global reference object is an object in the global object map whose distance from the global target object is less than or equal to a first distance threshold. Based on the global reference object and the global target object, at least one global topology map of the environment surrounding the robot is determined; Determine the end nodes corresponding to each of the at least one global topology graph; Based on the terminal nodes corresponding to each of the at least one global topology graphs, the local topology graphs are matched with the at least one global topology graphs respectively to obtain a set of node pairs corresponding to the at least one global topology graphs and the local topology graphs. The set of node pairs is the set of node pairs with the same corresponding labels in the local topology graphs and the at least one global topology graphs. A first preset number of sample node pairs are selected from the set of node pairs, and a second preset number of candidate pose information is determined based on the first preset number of sample node pairs. The target position information of the robot is determined based on the second preset number of candidate pose information.

2. The method according to claim 1, characterized in that, The process of acquiring local images collected by the robot and determining a local topology map based on the local images and the global object map corresponding to the robot's surrounding environment includes: Acquire local images collected by the robot; Object recognition is performed on the local image to obtain at least one local object in the local image; Based on the global object map corresponding to the robot's surrounding environment and the at least one local object, determine candidate objects among the at least one local objects, as well as local reference objects associated with the candidate objects; A local topology graph is established based on the candidate objects and the local reference objects.

3. The method according to claim 2, characterized in that, The step of determining candidate objects among the at least one local object based on the global object map corresponding to the robot's surrounding environment and the at least one local object includes: Determine the object type corresponding to the at least one local object; In the global object map corresponding to the environment around the robot, determine the number of objects corresponding to the object type; Based on the quantity of the objects, determine the rarity of the object type. Candidate objects are determined from the at least one local object based on the rarity corresponding to the object type.

4. The method according to any one of claims 1-3, characterized in that, The step of matching the local topology graph with the at least one global topology graph to obtain a set of node pairs corresponding to the at least one global topology graph and the local topology graph includes: The local links in the local topology graph are matched with the global links in at least one global topology graph to obtain target links. The target links are links that have the same end nodes in the local topology graph and the corresponding global topology graph. Obtain the first distance and the second distance of the target link; Determine the distance difference between the first distance and the second distance of the target link; If the distance difference is less than the distance difference threshold, then the end node in the target link is added to the node pair set.

5. The method according to any one of claims 1-3, characterized in that, The step of filtering a first preset number of sample node pairs from the node pair set, and determining a second preset number of candidate pose information based on the first preset number of sample node pairs, includes: Filter the first preset number of sample node pairs from the set of node pairs; Determine the alignment error of each of the sample nodes; Obtain the transformation matrix corresponding to the robot; The second preset number of candidate pose information is determined based on the alignment error and the transformation matrix.

6. The method according to any one of claims 1-3, characterized in that, Determining the robot's target position information from the second preset number of candidate pose information includes: Determine the association error between objects with the same label in the global object map, wherein the distance between objects with the same label is less than or equal to a second distance threshold; The matching confidence level of the second preset number of sample nodes is determined based on the association error. The target position information is determined from the second preset number of candidate pose information based on the matching confidence.

7. A device for determining the location of a target, characterized in that, The device includes: The local topology determination module is used to acquire local images collected by the robot and determine the local topology based on the local images and the global object map corresponding to the robot's surrounding environment. A global topology map determination module is used to determine local target nodes in the local topology map; determine global target objects in the global object map, wherein the global target objects and the objects corresponding to the local target nodes are of the same type; determine global reference objects in the global object map based on the global target objects, wherein the global reference objects are objects in the global object map whose distance from the global target objects is less than or equal to a first distance threshold; and determine at least one global topology map of the environment surrounding the robot based on the global reference objects and the global target objects. The end node determination module is used to determine the end nodes corresponding to each of the at least one global topology graph; The node pair set determination module is used to match the local topology graph with the at least one global topology graph according to the end nodes corresponding to each of the at least one global topology graph, to obtain the node pair set corresponding to the at least one global topology graph and the local topology graph. The node pair set is the set of node pairs with the same corresponding labels in the local topology graph and the at least one global topology graph. The candidate pose information determination module is used to filter a first preset number of sample node pairs in the node pair set, and determine a second preset number of candidate pose information based on the first preset number of sample node pairs. The target position information determination module is used to determine the target position information of the robot based on the second preset number of candidate pose information.

8. A robot, characterized in that, The robot includes: body; A control system communicating with the fuselage, the control system including a processor and a memory communicating with the processor, the memory storing instructions that, when executed on the processor, cause the processor to perform operations, the operations including: Acquire local images collected by the robot, and determine a local topology map based on the local images and the global object map corresponding to the robot's surrounding environment; Identify the local target node in the local topology graph; In the global object map, a global target object is determined, wherein the global target object is of the same type as the object corresponding to the local target node; Based on the global target object, a global reference object is determined in the global object map. The global reference object is an object in the global object map whose distance from the global target object is less than or equal to a first distance threshold. Based on the global reference object and the global target object, at least one global topology map of the environment surrounding the robot is determined; Determine the end nodes corresponding to each of the at least one global topology graph; Based on the terminal nodes corresponding to each of the at least one global topology graphs, the local topology graphs are matched with the at least one global topology graphs respectively to obtain a set of node pairs corresponding to the at least one global topology graphs and the local topology graphs. The set of node pairs is the set of node pairs with the same corresponding labels in the local topology graphs and the at least one global topology graphs. A first preset number of sample node pairs are selected from the set of node pairs, and a second preset number of candidate pose information is determined based on the first preset number of sample node pairs. The target position information of the robot is determined based on the second preset number of candidate pose information.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 6.

Images