Local map generation method and apparatus, robot, computer-readable storage medium

Abstract

This specification provides a method and apparatus for generating local maps, a robot, and a computer-readable storage medium. The method includes: when the robot is inside a car, acquiring multiple positional information corresponding to the robot; determining the operating state of the car based on the multiple positional information; when the operating state of the car is stationary, acquiring target environment data corresponding to the surrounding environment of the robot, the surrounding environment including the car; and generating a local map corresponding to the robot based on the target environment data corresponding to the surrounding environment. This avoids the robot using target environment data collected during car movement to generate a local map, reducing anomalies in the local map.

Description

Technical Field

[0001] This specification relates to the field of robotics, and in particular to a method and apparatus for generating local maps, a robot, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, the application scenarios of robots are becoming increasingly widespread. For example, robots can accompany their owners in elevators, or they can use elevators to transport goods. In these application scenarios, robots need to build a local map of their surrounding environment to plan their paths and avoid being hindered by various obstacles. However, when a robot is riding an elevator, this can cause anomalies in the local map it builds when it reaches a lower floor, interfering with the robot's ability to use the local map to perform tasks such as path planning, movement, and obstacle avoidance. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this specification provides a method and apparatus for generating local maps, a robot, and a computer-readable storage medium.

[0004] According to a first aspect of the embodiments of this specification, a local map generation method is provided, applied to a robot, the method comprising: when the robot is located inside a car, acquiring multiple position information collected by the robot at multiple times; determining the operating state corresponding to the car based on the multiple position information collected at the multiple times; when the operating state corresponding to the car is a stationary state, acquiring target environment data corresponding to the surrounding environment of the robot, the surrounding environment including the car; and generating a target local map corresponding to the robot based on the target environment data corresponding to the surrounding environment.

[0005] In some embodiments, obtaining target environment data corresponding to the surrounding environment of the robot includes: obtaining original environment data corresponding to the surrounding environment and height information corresponding to the robot; determining the height range of the robot based on the height information corresponding to the robot; and filtering the original environment data based on the height range to obtain target environment data corresponding to the surrounding environment.

[0006] In some embodiments, the target environment data includes environmental image data, and the number of visual sensors that collect the environmental image data is multiple, with the multiple visual sensors respectively set at different positions of the robot.

[0007] In some embodiments, the target environment data includes environmental point cloud data, which is acquired by a depth sensor on the robot.

[0008] In some embodiments, generating a target local map corresponding to the robot based on the target environment data corresponding to the surrounding environment includes: acquiring environmental image data from the target environment data corresponding to the surrounding environment; calling a grounding wire recognition model to recognize the environmental image data to obtain the grounding wire position in the surrounding environment; determining a passable area in the surrounding environment based on the grounding wire position and the robot's position; and generating a target local map corresponding to the robot based on the passable area.

[0009] In some embodiments, the target environment data includes environmental image data and environmental point cloud data; generating a local map corresponding to the robot based on the target environment data corresponding to the surrounding environment includes: generating at least one first local map corresponding to the robot based on the environmental image data corresponding to the surrounding environment; generating at least one second local map corresponding to the robot based on the environmental point cloud data corresponding to the surrounding environment; and fusing the at least one first local map and the at least one second local map corresponding to the robot to obtain a target local map corresponding to the robot.

[0010] In some embodiments, fusing at least one first local map and at least one second local map corresponding to the robot to obtain a target local map corresponding to the robot includes: traversing a plurality of first locations in at least one second local map; for any one of the plurality of first locations, when the first height information of the first location is invalid, searching for a second location corresponding to the first location in the at least one first local map, and determining the second height information corresponding to the second location; replacing the first height information with the second height information; and determining the map obtained after traversal as the target local map corresponding to the robot.

[0011] In some embodiments, each of the plurality of position information includes horizontal position information and vertical position information; determining the operating state of the car based on the plurality of position information collected at the plurality of times includes: determining the horizontal displacement standard deviation of the robot based on the horizontal position information collected at the plurality of times; determining the vertical displacement and vertical velocity of the robot based on the vertical position information collected at the plurality of times; if the horizontal displacement standard deviation is less than a standard deviation threshold, the vertical displacement is less than a displacement threshold, and the vertical velocity is less than a velocity threshold, then the operating state of the car is determined to be a stationary state.

[0012] In some embodiments, the multiple location information is collected according to a preset frequency; the method further includes: when the operating state corresponding to the car is a motion state, shielding the target environment data corresponding to the surrounding environment of the robot, and returning to the step of obtaining the multiple location information collected by the robot at multiple times.

[0013] According to a second aspect of the embodiments of this specification, a local map generation apparatus is provided, applied to a robot. The apparatus includes: a position acquisition module, configured to acquire multiple position information collected by the robot at multiple times when the robot is located inside a car; a state determination module, configured to determine the operating state corresponding to the car based on the multiple position information collected at the multiple times; an environmental data acquisition module, configured to acquire target environment data corresponding to the surrounding environment of the robot when the operating state corresponding to the car is a stationary state, the surrounding environment including the car; and a map generation module, configured to generate a local map corresponding to the robot based on the target environment data corresponding to the surrounding environment.

[0014] According to a third aspect of the embodiments of this specification, a robot is provided, the robot including a body; a control system communicating with the body, the control system including a processor and a memory communicating with the processor, the memory storing instructions, the instructions being executed on the processor to cause the processor to perform a local map generation method for the robot according to any embodiment of this specification.

[0015] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the local map generation method for a robot as described in any embodiment of this specification.

[0016] The technical solutions provided in the embodiments of this specification may include the following beneficial effects:

[0017] In the embodiments of this specification, the operating state of the car is determined based on multiple location information collected by the robot at multiple times. When the operating state of the car is stationary, the robot will generate a local map using the target environment data corresponding to the surrounding environment, instead of using the target environment data when the car is moving. This effectively reduces the anomalies of the target local map, improves the accuracy and reliability of the target local map, and thus improves the accuracy of the robot in performing tasks such as path planning, movement, and obstacle avoidance.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this specification and, together with the specification, serve to explain the principles of this specification.

[0020] Figure 1 This is a schematic diagram of a local map corresponding to the robot shown in this specification according to an exemplary embodiment;

[0021] Figure 2 This is a flowchart illustrating a local map generation method according to an exemplary embodiment of this specification;

[0022] Figure 3 This is a flowchart illustrating a local map generation method according to another exemplary embodiment of this specification;

[0023] Figure 4 This is a flowchart illustrating a method for filtering target environmental data according to an exemplary embodiment of this specification;

[0024] Figure 5 This is a flowchart illustrating a local map generation method according to yet another exemplary embodiment of this specification;

[0025] Figure 6 This is a schematic diagram illustrating the deployment of three monocular cameras on a robot according to an exemplary embodiment of this specification;

[0026] Figure 7 This is a flowchart illustrating a local map generation method according to yet another exemplary embodiment of this specification;

[0027] Figure 8 This is a schematic diagram of the robot's reachable space.

[0028] Figure 9 This is a schematic diagram illustrating the fusion of a local map and an accessible space map according to an exemplary embodiment of this specification;

[0029] Figure 10 This is a schematic diagram of a local map generation apparatus illustrated in this specification according to an exemplary embodiment;

[0030] Figure 11 This specification is a schematic diagram of the hardware structure of a robot according to an exemplary embodiment.

[0031] Figure 12 This is a schematic diagram of the mechanical structure of a robot according to an exemplary embodiment. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0033] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0034] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0035] A robot is an automated machine that, unlike common machines, possesses intelligent capabilities similar to humans or other living beings, such as perception, planning, movement, and coordination. It is a highly flexible automated machine. Robots can assist or even replace humans in performing dangerous, arduous, and complex tasks, improving work efficiency and quality, serving human life, and expanding the scope of human activities and capabilities.

[0036] Robots can operate in elevators; for example, they can accompany their owners or transport goods. However, during elevator operation, the robot adds environmental data collected at different times and heights to its local map. This can cause anomalies in the local map when the robot reaches lower floors, interfering with the robot's ability to perform tasks such as path planning, movement, and obstacle avoidance. To address this, this specification proposes a local map generation method and apparatus, a robot, and a computer-readable storage medium to prevent the robot from using environmental data collected at different times and heights during elevator car movement to construct a local map, thereby reducing local map anomalies and improving the accuracy of the robot's path planning. The embodiments of this specification will be described in detail below.

[0037] In this specification, the robot can collect environmental data based on sensors deployed on the robot, generate a local map based on the collected environmental data, and then autonomously plan its movement route based on the local map. The environmental data includes, but is not limited to, environmental point cloud data and / or environmental image data. Environmental point cloud data can be acquired through a depth sensor, and environmental image data can be acquired through a vision sensor. For example... Figure 1 As shown, a local map can be a map within a preset range centered on the robot. This preset range can be a rectangular area as shown in the figure, a circular area with a preset radius centered on the robot, or an area of ​​other shapes. Specifically, the preset range can be a fixed range or a dynamic range, determined according to actual application needs. As the robot moves, historical information outside this range will be discarded, and newly collected environmental data will continuously update this local map. The update method can be adding new environmental data to the local map, or regenerating the local map using environmental data corresponding to the new location's range. For example, the local map can be a 3D map, which is a three-dimensional, abstract description of one or more aspects of the real world or a part of it, realistically and three-dimensionally reflecting the surrounding environment, such as a voxel map. Another example is a 2.5D map, which is a three-dimensional real-world image integrated based on three-dimensional data, intuitively simulating the scene, such as an elevation map. Compared to 3D maps, 2.5D maps contain less data, making it more conducive for the robot to quickly perform tasks based on map information.

[0038] Robots can also be equipped with sensors for acquiring position data, such as odometry or inertial measurement units (IMUs) for pose sensors. A pose sensor is a measurement unit that uses data from motion sensors to estimate how an object's position changes over time; the information it collects can include both position and orientation information. When a robot is inside an elevator car, it passively moves with the car. For example, when the car goes up or down, the robot's displacement in the direction of the car's movement will change accordingly, moving upwards or downwards. Simultaneously, the robot's displacement in a plane perpendicular to the direction of the car's movement will also change. When the car is stationary, the robot will also be stationary. Therefore, the robot can determine the car's operating status based on changes in its own position information.

[0039] like Figure 2 As shown, Figure 2 This is a flowchart illustrating a local map generation method according to an exemplary embodiment, comprising the following steps:

[0040] Step S1: When the robot is inside the car, acquire multiple position information collected by the robot at multiple times;

[0041] Step S2: Determine the corresponding operating status of the car based on the multiple location information collected at multiple times;

[0042] Step S3: When the car is in a stationary state, acquire the target environment data corresponding to the robot's surrounding environment, including the car.

[0043] Step S4: Generate a local target map for the robot based on the target environment data corresponding to the surrounding environment.

[0044] In the above embodiments, the robot generates a local target map using target environment data only when the elevator car is stationary. It does not generate a local target map when the car is moving, reducing anomalies in the local target map and improving its accuracy and reliability. This, in turn, improves the accuracy of the robot's path planning, movement, and obstacle avoidance tasks. For example, if the robot uses all the target environment data collected from inside the elevator car to generate a local map while riding an elevator from the second floor to the first floor, the generated local map will show obstacle anomalies when the elevator reaches the first floor. This prevents the robot from planning the correct path and interferes with its task execution. Therefore, according to the local map generation method in this specification, the robot does not use the collected target environment data to generate a local map while riding an elevator from the second floor to the first floor. The robot only uses the target environment data collected when the elevator is stationary to build a local map, thus reducing anomalies in the local map.

[0045] In step S1, the car can be an elevator car, which can move vertically, for example, ascending or descending. During the ascent or descent of the car, upon reaching the designated floor, the car can decelerate until it comes to a stop and the car door opens. After the car door returns from the open state to the closed state, the car can remain stationary or switch from the stationary state to the moving state.

[0046] Before executing step S1, the robot can first determine whether it is inside the elevator car. For example, environmental image data corresponding to the robot's surroundings can be collected and analyzed to determine if the robot is inside the car. Alternatively, sensors such as LiDAR can be deployed on the robot to detect the distance between the robot and objects in the surrounding environment, and the robot is determined to be inside the car when the distance between the robot and objects in the surrounding environment meets preset conditions. A semantic map built by the robot based on its surrounding environment can also be used to determine whether the robot has entered and is inside the car based on the semantic information included in the semantic map. Other methods can also be used to determine whether the robot is inside the car, which will not be elaborated here.

[0047] When the robot is inside the elevator car, multiple position information corresponding to the robot can be obtained. Different position information can be obtained at different times. When any position information is obtained, the elevator car can be in motion or in a stationary state.

[0048] In this specification, the robot is located inside the elevator car. The robot's position information can be determined with reference to the direction of the elevator car's movement. The pose information collected by the pose sensor provides the robot's horizontal position information (i.e., the robot's position on the horizontal plane, which can be represented by the robot's two-dimensional coordinates on the horizontal plane) and its vertical position information (i.e., the robot's position in the vertical direction, which can be represented by the robot's coordinates in the vertical direction). Typically, the vertical direction is the direction of the elevator car's movement (ascending or descending). Assuming (x, y) represents the robot's position information on the horizontal plane and z represents the robot's position information in the vertical direction, the position information collected by the robot at each moment can be recorded as (x, y, z), where these coordinates can be the robot's coordinates in the world coordinate system xOyz.

[0049] In step S2, the robot can determine the operating status of the car based on changes in its multiple positional information. For example... Figure 3 The diagram shown is a flowchart illustrating another method for generating a local map of a robot according to an exemplary embodiment. Based on the aforementioned embodiments, this embodiment describes a process for determining the operating state of a car based on multiple location information corresponding to the robot, including the following steps:

[0050] Step S21: Determine a first reference value based on the horizontal position information collected at multiple times. The first reference value is used to characterize the degree of change in the robot's running state in the horizontal direction.

[0051] Step S22: Determine a second reference value based on the vertical position information collected at multiple times. The second reference value is used to characterize the degree of change in the robot's running state in the vertical direction.

[0052] Step S23: If the first reference value is less than the first preset threshold and the second reference value is less than the second preset threshold, determine that the car's running state is stationary.

[0053] In some embodiments, the first reference value may include the standard deviation of horizontal displacement, which reflects the degree of displacement change of the robot in the horizontal direction. The process of calculating the standard deviation of horizontal displacement is as follows:

[0054]

[0055] in,

[0056]

[0057] l i Two-dimensional coordinates (x, y) for each of multiple different times. i ,y i The distance from the origin of the world coordinate system. Let be the average distance at each time point, 1≤i≤n, where n is the total number of different time points.

[0058] The larger the standard deviation of horizontal displacement, the greater the degree of displacement change of the robot in the horizontal direction; the smaller the standard deviation of horizontal displacement, the smaller the degree of displacement change of the robot in the horizontal direction. Therefore, a first preset threshold (i.e., the standard deviation threshold) is set, and the horizontal displacement standard deviation being less than the standard deviation threshold is used as one of the conditions for judging that the car is stationary.

[0059] In some embodiments, the second reference value may include vertical displacement and vertical velocity, which may reflect the degree of change in the robot's position in the vertical direction.

[0060] Assuming the robot's vertical displacement is denoted as ΔZ, the vertical displacement ΔZ can be calculated as follows:

[0061] ΔZ=z n ―z1

[0062] Assume the robot's vertical velocity is denoted as... The vertical velocity can then be calculated in the following way.

[0063]

[0064] Where 1≤i≤n, n is the total number of different times, z1 is the coordinate of the first recorded time, t1 is the first time, and z n Let t be the coordinate of the nth time point. n Let Δt be the nth time, and let Δt be the time elapsed from the 1st time to the nth time.

[0065] The greater the vertical displacement and the greater the vertical velocity, the greater the change in the robot's running state in the vertical direction; the smaller the vertical displacement and the smaller the vertical velocity, the smaller the change in the robot's running state in the vertical direction. Therefore, setting displacement thresholds and velocity thresholds (collectively referred to as the second preset thresholds) is one of the conditions for determining that the car is stationary.

[0066] The above examples illustrate some embodiments for determining the car's operating state. It is understood that these are merely illustrative examples and not intended to limit this specification. In other embodiments, other methods can be used to determine the car's operating state. For example, the open / closed state of the car door can be detected. When the car door is open, it can be determined that the car is stationary. As another example, environmental image data inside the car can be acquired, and the state of the floor buttons inside the car can be determined based on this environmental image data, and the operating state of the car can be determined based on the state of the floor buttons. When the state of the floor buttons changes from lit to off, it can be determined that the car is stationary. The operating state of the car can also be determined by combining the open / closed state of the car door and the state of the floor buttons. When the car door changes from open to closed, and at least one floor button is lit, the operating state of the car can be determined to be in motion. Besides the situations listed above, other methods can be used to determine the car's operating state, which will not be listed here.

[0067] If the operating state of the elevator car is determined to be stationary, steps S3 and S4 can be executed to obtain target environment data corresponding to the robot's surrounding environment, and to generate a target local map corresponding to the robot using the target environment data. For example, the target environment data can be updated to the robot's local map at a preset frequency to obtain the updated local map as the target local map. If the operating state of the elevator car is determined to be in motion, the target environment data corresponding to the robot's surrounding environment can be masked, and the process can return to the step of obtaining multiple position information collected by the robot at multiple times.

[0068] In some embodiments, the various sensors on the robot can always be operational and collect target environmental data of the surrounding environment. However, when the car is in motion, the currently collected target environmental data can be masked, and the target environmental data collected when the car is in motion will not be used in the local map generation process. In other embodiments, the various sensors on the robot can be activated to collect target environmental data when the car is stationary, and the collected target environmental data can be used in the local map generation process; when the car is in motion, the various sensors on the robot can also be controlled to stop working or stop collecting target environmental data.

[0069] The target environment data acquired in step S3 can be target environment data collected by various sensors on the robot when acquiring multiple position information in step S1, or target environment data collected by various sensors on the robot after determining that the operating state acquired in step S2 is a stationary state. In some embodiments, the target environment data includes at least one of environmental point cloud data and environmental image data.

[0070] Furthermore, when the robot travels from a high floor to a low floor in an elevator, the accumulated target environment data includes data on the elevator car's environment across multiple floors while the elevator is stationary. However, the robot only needs the target environment data for its current floor to generate a local map. Therefore, if the robot directly uses the accumulated target environment data to build a local map, the target environment data from higher floors may influence the local map of the lower floors, leading to anomalies. Additionally, the large volume of accumulated target environment data also results in a longer timeframe for the robot to generate the local map.

[0071] In this regard, such as Figure 4 As shown in the figure, this embodiment illustrates a flowchart of a method for filtering target environmental data, including the following steps:

[0072] Step S31: Obtain the raw environmental data of the surrounding environment and the corresponding height information of the robot;

[0073] Step S32: Determine the height range of the robot based on the robot's corresponding height information;

[0074] Step S33: Filter the original environmental data according to the height range to obtain the target environmental data corresponding to the surrounding environment.

[0075] For example, the raw environmental data can be raw environmental point cloud data, which may include all point cloud data collected by the depth sensor on the robot. Correspondingly, after filtering the raw environmental point cloud data, the target environmental data corresponding to the surrounding environment is also point cloud data. The filtered point cloud data only includes point cloud data within the height range corresponding to the robot's height, and does not include point cloud data outside the height range corresponding to the robot's height.

[0076] When the robot reaches a lower floor, it can directly acquire its height information through a pose sensor. The height range of the local map the robot needs to build can be set with reference to this height information, specifically the robot's height in the world coordinate system. This allows the robot to generate a local map using only point cloud data within the specified height range, reducing the amount of data processing required to build the local map and minimizing anomalies caused by mapping point cloud data from higher floors to local maps of lower floors.

[0077] In some embodiments, the height range can be calculated by adding a preset upper limit value to the robot's height information as the center value, and subtracting a preset lower limit value from the preset lower limit value to obtain the lower threshold value of the height range. The preset upper and lower limits can be fixed constants or calculated using a fixed formula. For example, if the robot's height information is 3 meters, the preset upper limit value can be 0.8 meters, and the preset lower limit value can be 0.2 meters, then the height range is 2.8 meters to 3.8 meters. As another example, if the robot's height information is 3 meters and it is on the second floor, the preset lower limit value can be 0.2 meters, and the preset upper limit value can be calculated using the following formula: K × preset floor height - robot's current height, where K is a coefficient less than 1. When K = 0.9, and the preset floor height is 4 meters, then the preset upper limit value is 0.9 × 4 - 3 = 0.6, and the height range is 2.8 meters to 3.6 meters.

[0078] The following example illustrates the specific methods for generating local maps. When the target environment data includes both environmental image data and environmental point cloud data, such as... Figure 5 As shown, the specific steps for generating a local map include the following:

[0079] Step S41: Generate at least one first local map corresponding to the robot based on the environmental image data corresponding to the surrounding environment;

[0080] Step S42: Generate at least one second local map corresponding to the robot based on the environmental point cloud data corresponding to the surrounding environment;

[0081] Step S43: Fuse at least one first local map and at least one second local map corresponding to the robot to obtain the target local map corresponding to the robot.

[0082] In some embodiments, environmental point cloud data can be acquired by a depth sensor on the robot. Since the perspective of a depth sensor is typically limited to a 360-degree field of view centered on the robot, and the robot faces movement obstacles within the cabin, it is inconvenient to acquire environmental point cloud data within a 360-degree field of view by rotating the robot body. Therefore, the robot can supplement the environmental point cloud data acquired by the depth sensor with environmental image data. Environmental image data can be acquired by a vision sensor on the robot. The different first local maps in step S41 can be acquired by the robot's vision sensor from different perspectives, and the different second local maps in step S42 can be acquired by the robot's depth sensor from different perspectives. Environmental image data acquired by the vision sensor at a certain moment can be used to generate the first local map, and environmental point cloud data acquired by the depth sensor at the same moment can be used to generate the second local map. The first and second local maps are then fused in step S43. Here, "the same moment" can refer to the same point in time, or time points differing within a preset time range.

[0083] The robot can have one or more vision sensors, with each sensor positioned at different locations on the robot. For example, each vision sensor can be a monocular vision sensor. Figure 6 As shown, the robot's 360-degree field of view can include four mutually perpendicular directions. Taking the depth sensor's field of view as the front, the field of view of the three monocular vision sensors can be left, right, and rear, respectively. These three monocular vision sensors can be called the left monocular vision sensor (or left monocular camera), the right monocular vision sensor (or right monocular camera), and the rear monocular vision sensor (or rear monocular camera). The field of view of the left and rear monocular cameras can overlap, as can the field of view of the right and rear monocular cameras. The field of view of the left and right monocular cameras can also overlap with that of the depth sensor. In this way, the overall field of view can fully cover the robot's surrounding environment. The robot can obtain target environmental data without additional rotational movements, thus building a local map more efficiently.

[0084] In some embodiments, step S43 of the foregoing embodiments specifically includes: traversing a plurality of first locations in at least one second local map; for any one of the plurality of first locations, when the first height information of the first location is invalid, searching for a second location corresponding to the first location in at least one first local map, determining the second height information corresponding to the second location, replacing the first height information with the second height information; and determining the map obtained after traversal as the target local map corresponding to the robot.

[0085] Invalid information refers to locations where the height information is empty or zero. Due to the limited field of view of the depth sensor, or because the depth sensor is obstructed, malfunctions, or has errors, some locations in the second local map may have invalid first height information. The first location and the corresponding second location represent the same location in physical space.

[0086] The robot obtains a second local map using a depth sensor and a first local map using a vision sensor. The robot can replace invalid first height information with second height information from the first local map. Since the vision sensor's field of view covers at least part of the depth sensor's blind spot, fusing the first and second local maps yields a local map with a wider field of view (i.e., the robot's target local map), which is more conducive to the robot's rational path planning.

[0087] Even when the target environment data only includes environmental image data, local maps can still be generated using other methods. For example... Figure 7 The diagram shown is a flowchart illustrating another method for generating local maps for a robot according to an exemplary embodiment, comprising the following steps:

[0088] Step S411: Obtain environmental image data from the target environment data corresponding to the surrounding environment;

[0089] Step S412: Call the grounding wire identification model to identify the environmental image data and obtain the location of the grounding wire in the surrounding environment;

[0090] Step S413: Determine the passable area in the surrounding environment based on the location of the grounding wire and the robot's location;

[0091] Step S414: Generate a local target map for the robot based on the passable area.

[0092] The methods described in this specification can be used alone to generate a target local map corresponding to the robot. Alternatively, the target local map generated in the embodiments of this specification can be used as a first local map and fused with a second local map in the aforementioned embodiments to obtain the final target local map.

[0093] In the above embodiments, the passable area can also be called the reachable space, which is used to characterize all reachable location information of the robot's surrounding environment. According to the principle of light propagation, as long as there is an obstacle within the robot's field of vision, the obstacle will form a grounding line with the ground. The visual sensor can "see" the grounding line in the distance. The connection between the grounding point and the sensor is projected onto the ground. The robot considers the projected area to be passable, which is the reachable space.

[0094] In some embodiments, the environmental image data can be two-dimensional images acquired by a vision sensor. By identifying the environmental image data, the location of the grounding wire can be determined. Based on the location of the grounding wire, a passable area corresponding to that grounding wire can be generated. Specifically, the robot can calculate the three-dimensional coordinates (x, y, y) of all grounding points within the grounding wire based on its own height information h0, the location of the grounding wire, and the mapping matrix from the pixel coordinate system to the world coordinate system. w ,y w ,z w The calculation formula is as follows:

[0095]

[0096] Where u and v are the pixel coordinates of the monocular vision sensor, x w y w z w Let R be the world coordinates, t be the rotation matrix, dX and dY be the physical dimensions of each pixel in the u-axis and v-axis directions of the pixel coordinate system, f be the camera focal length, and S be the scale factor.

[0097] and

[0098] set up M1 is the intrinsic parameter matrix of a monocular vision sensor.

[0099] M2 is the extrinsic parameter matrix of a monocular vision sensor.

[0100]

[0101] In z w When h = 0, for a calibrated vision sensor, M1 and M2 are known, and u and v are the pixel coordinates of the grounding point, which are also known. Using the above formula, solving the three equations for the three unknowns allows us to calculate the grounding point x. w y w coordinate.

[0102] In some embodiments, a passable area can be a triangular region with the robot's center and the two endpoints of one of its grounding wires as vertices, such as... Figure 8 As shown, the robot considers the area to be unobstructed and can move freely.

[0103] like Figure 9 As shown, each monocular vision sensor can acquire environmental image data within its field of view. Based on this data, a passable area (or simply reachable space map) within that field of view can be generated. The monocular vision sensor is configured as follows... Figure 6 In the scenario shown, environmental image data acquired by the right monocular camera can generate a right reachability space map, environmental image data acquired by the rear monocular camera can generate a rear reachability space map, and environmental image data acquired by the left monocular camera can generate a left reachability space map. By fusing the right, rear, and left reachability space maps with the local map obtained from the environmental point cloud data acquired by the depth sensor, a fused local map, i.e., the target local map corresponding to the robot, can be obtained.

[0104] In addition, this specification also provides a local map generation device. Figure 10 This specification is a schematic diagram illustrating the structure of a local map generation apparatus according to an exemplary embodiment. The apparatus includes the following modules:

[0105] The position acquisition module 10 is used to acquire multiple position information collected by the robot at multiple times when the robot is inside the car.

[0106] The status determination module 20 is used to determine the corresponding operating status of the car based on multiple location information collected at multiple times.

[0107] The environmental data acquisition module 30 is used to acquire target environmental data corresponding to the robot's surrounding environment when the car's corresponding running state is stationary. The surrounding environment includes the car.

[0108] The map generation module 40 is used to generate a local map for the robot based on the target environment data corresponding to the surrounding environment.

[0109] Additionally, this specification also provides a robot that includes a processor for performing the following steps:

[0110] When the robot is inside the elevator car, acquire multiple position information collected by the robot at multiple times;

[0111] The operating status of the car is determined based on the multiple location information collected at the multiple times.

[0112] When the operating state of the car is stationary, acquire the target environment data corresponding to the surrounding environment of the robot, where the surrounding environment includes the car.

[0113] A local target map corresponding to the robot is generated based on the target environment data corresponding to the surrounding environment.

[0114] In some embodiments, the processor is used to acquire raw environmental data corresponding to the surrounding environment and height information corresponding to the robot; determine the height range of the robot based on the height information corresponding to the robot; and filter the raw environmental data according to the height range to obtain target environmental data corresponding to the surrounding environment.

[0115] In some embodiments, the target environment data includes environmental image data, and the number of visual sensors acquiring the environmental image data is one or more. When there are multiple visual sensors, the multiple visual sensors are respectively set at different positions of the robot.

[0116] In some embodiments, the target environment data includes environmental point cloud data, which is acquired by a depth sensor on the robot.

[0117] In some embodiments, the processor is configured to acquire environmental image data from target environment data corresponding to the surrounding environment; call a grounding wire recognition model to recognize the environmental image data and obtain the location of the grounding wire in the surrounding environment; determine the passable area in the surrounding environment based on the location of the grounding wire and the robot's location; and generate a target local map corresponding to the robot based on the passable area.

[0118] In some embodiments, the target environment data includes environmental image data and environmental point cloud data; the processor is configured to generate at least one first local map corresponding to the robot based on the environmental image data corresponding to the surrounding environment; generate at least one second local map corresponding to the robot based on the environmental point cloud data corresponding to the surrounding environment; and fuse the at least one first local map and the at least one second local map corresponding to the robot to obtain the target local map corresponding to the robot.

[0119] In some embodiments, the processor is configured to traverse a plurality of first locations in at least one second local map; for any one of the plurality of first locations, when the first height information of the first location is invalid, to search for a second location corresponding to the first location in the at least one first local map, to determine the second height information corresponding to the second location; to replace the first height information with the second height information; and to determine the map obtained after traversal as the target local map corresponding to the robot.

[0120] In some embodiments, each of the multiple position information includes horizontal position information and vertical position information; the processor is used to determine the standard deviation of the robot's horizontal displacement based on the horizontal position information collected at multiple times; and to determine the robot's vertical displacement and vertical velocity based on the vertical position information collected at multiple times; if the standard deviation of the horizontal displacement is less than the standard deviation threshold, the vertical displacement is less than the displacement threshold, and the vertical velocity is less than the velocity threshold, then the operating state of the car is determined to be a stationary state.

[0121] In some embodiments, multiple location information is collected according to a preset frequency; the processor is also used to shield the target environment data corresponding to the robot's surrounding environment when the car's corresponding operating state is a motion state, and return to the step of obtaining multiple location information collected by the robot at multiple times.

[0122] The implementation process of the processor's functions and roles in the robot is detailed in the corresponding steps of the above method, and will not be repeated here.

[0123] Please see Figure 11 , Figure 11 This is a schematic diagram of the hardware structure of a robot 100 provided in one embodiment of this specification. The robot 100 can be any of various robot types, specifically including but not limited to at least one of wheeled robots, legged robots, tracked robots, crawling robots, worm-like robots, or swimming robots. For example, the robot 100 can specifically be a legged robot, or a robot combining legged and wheeled designs. Legged robots include monopodial robots, bipodial robots, or multipodial robots. Multipodial robots refer to robots with three or more legs; for example, a multipodial robot can specifically be a quadrupedal robot. A robot is a machine capable of performing semi-autonomous or fully autonomous tasks. Robots are not limited to humanoid robotic devices and can also include robots with configurations such as dogs, horses, snakes, fish, apes, or monkeys. For example, a robot can specifically be a quadrupedal robotic horse. Figure 11 In the illustrated embodiment, robot 100 includes a mechanical unit 101, a communication unit 102, a sensing unit 103, an interface unit 104, a storage unit 105, a display unit 106, an input unit 107, a control module 110, and a power supply 111. The various components of robot 100 can be connected in any way, including wired or wireless connections. Those skilled in the art will understand that... Figure 11 The specific structure of the robot 100 shown does not constitute a limitation on the robot 100. The robot 100 may include more or fewer parts than shown. Some parts are not essential components of the robot 100 and may be omitted or combined as needed without changing the nature of the invention.

[0124] Figure 12 This is a schematic diagram of the mechanical structure of a robot according to one embodiment of this specification. The following is in conjunction with... Figure 11 and Figure 12 A detailed introduction to each component of Robot 100:

[0125] Mechanical unit 101 is the hardware of robot 100. For example... Figure 11 As shown, the mechanical unit 101 may include a drive board 1011, a motor 1012, and a mechanical structure 1013, such as... Figure 12 As shown, the mechanical structure 1013 may include a main body 1014, extendable legs 1015, and feet 1016. In other embodiments, the mechanical structure 1013 may also include an extendable robotic arm (not shown), a rotatable head structure 1017, a rocking tail structure 1018, a cargo-carrying structure 1019, a saddle structure 1020, a camera structure 1021, etc. It should be noted that the various component modules of the mechanical unit 101 can be one or multiple, depending on the specific situation. For example, there may be four legs 1015, and each leg 1015 may be equipped with three motors 1012, resulting in a total of twelve motors 1012.

[0126] The communication unit 102 can be used for receiving and sending signals, and can also communicate with networks and other devices. For example, it can receive instructions from a remote control or other robot 100 to move in a specific direction at a specific speed according to a specific gait, and then transmit these instructions to the control module 110 for processing. The communication unit 102 includes modules such as WiFi, 4G, 5G, Bluetooth, and infrared modules.

[0127] The sensing unit 103 is used to acquire information data about the environment surrounding the robot 100 and to monitor parameter data of various components inside the robot 100, and then sends this data to the control module 110. The sensing unit 103 includes various sensors, such as sensors for acquiring information about the surrounding environment: lidar (for remote object detection, distance determination, and / or velocity determination), millimeter-wave radar (for short-range object detection, distance determination, and / or velocity determination), cameras, infrared cameras, and Global Navigation Satellite System (GNSS). Sensors for monitoring various components inside the robot 100 include: an inertial measurement unit (IMU) (for measuring velocity, acceleration, and angular velocity values), foot sensors (for monitoring the position of the foot's contact point, foot posture, magnitude and direction of the contact force), and temperature sensors (for detecting component temperature). Other sensors that can be configured on the robot 100, such as load sensors, touch sensors, motor angle sensors, and torque sensors, are not detailed here.

[0128] The interface unit 104 can be used to receive input from external devices (e.g., data, power, etc.) and transmit the received input to one or more components within the robot 100, or it can be used to output to external devices (e.g., data, power, etc.). The interface unit 104 may include a power port, a data port (such as a USB port), a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, etc.

[0129] Storage unit 105 is used to store software programs and various data. Storage unit 105 may mainly include a program storage area and a data storage area. The program storage area may store operating system programs, motion control programs, application programs (such as text editors), etc.; the data storage area may store data generated by the robot 100 during use (such as various sensor data acquired by the sensing unit 103, log file data, etc.). Furthermore, storage unit 105 may include high-speed random access memory, and may also include non-volatile memory, such as disk storage, flash memory, or other volatile solid-state memory.

[0130] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0131] Input unit 107 can be used to receive input numerical or character information. Specifically, input unit 107 may include touch panel 1071 and other input devices 1072. Touch panel 1071, also known as touch screen, can collect user touch operations (such as operations performed by the user using their palm, fingers, or suitable accessories on or near touch panel 1071) and drive corresponding connection devices according to a pre-set program. Touch panel 1071 may include two parts: touch detection device 1073 and touch controller 1074. Touch detection device 1073 detects the user's touch position and the signal generated by the touch operation, and transmits the signal to touch controller 1074; touch controller 1074 receives touch information from touch detection device 1073, converts it into touch point coordinates, and sends it to control module 110, and can also receive and execute commands from control module 110. In addition to touch panel 1071, input unit 107 may also include other input devices 1072. Specifically, other input devices 1072 may include, but are not limited to, one or more of the following: remote control handles, etc., without any specific limitation here.

[0132] Furthermore, the touch panel 1071 can cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the control module 110 to determine the type of touch event. Subsequently, the control module 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Although in Figure 11 In this embodiment, the touch panel 1071 and the display panel 1061 are two independent components that implement input and output functions respectively. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement input and output functions. The specific implementation is not limited here.

[0133] The control module 110 is the control center of the robot 100. It connects all the components of the robot 100 through various interfaces and lines. It controls the robot 100 as a whole by running or executing the software program stored in the storage unit 105 and calling the data stored in the storage unit 105.

[0134] Power supply 111 supplies power to various components. Power supply 111 may include a battery and a power control board. The power control board controls battery charging, discharging, and power consumption management. Figure 11 In the illustrated embodiment, power supply 111 is electrically connected to control module 110. In other embodiments, power supply 111 may also be electrically connected to sensing unit 103 (such as camera, radar, speaker, etc.) and motor 1012. It should be noted that each component may be connected to a different power supply 111, or may be powered by the same power supply 111.

[0135] Based on the above embodiments, specifically, in some embodiments, a terminal device can be used to communicate with the robot 100. When the terminal device communicates with the robot 100, it can send instruction information to the robot 100. The robot 100 can receive the instruction information through the communication unit 102 and, upon receiving the instruction information, can transmit it to the control module 110, so that the control module 110 can process the instruction information to obtain the target speed value. The terminal device includes, but is not limited to, mobile phones, tablets, servers, personal computers, wearable smart devices, and other electrical appliances with image capture capabilities.

[0136] The instruction information can be determined based on preset conditions. In one embodiment, the robot 100 may include a sensing unit 103, which can generate instruction information based on the current environment of the robot 100. The control module 110 can determine whether the current speed value of the robot 100 meets the corresponding preset conditions based on the instruction information. If it does, the robot 100 will maintain its current speed value and current gait; if it does not, the control module 110 will determine a target speed value and a corresponding target gait based on the corresponding preset conditions, thereby controlling the robot 100 to move at the target speed value and the corresponding target gait. Environmental sensors may include temperature sensors, air pressure sensors, vision sensors, and sound sensors. Instruction information may include temperature information, air pressure information, image information, and sound information. The communication method between the environmental sensors and the control module 110 can be wired or wireless. Wireless communication methods include, but are not limited to: wireless networks, mobile communication networks (3G, 4G, 5G, etc.), Bluetooth, and infrared.

[0137] In addition, this specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the local map generation method for the robot of any of the foregoing embodiments.

[0138] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0139] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0140] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0141] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.

[0142] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

[0143] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A method for generating a local map, characterized in that, Applied to robots, the method includes: When the robot is inside the elevator car, acquire multiple position information collected by the robot at multiple times; The operating status of the car is determined based on the multiple location information collected at the multiple times. When the operating state of the car is stationary, acquire the target environment data corresponding to the surrounding environment of the robot, where the surrounding environment includes the car. A local target map corresponding to the robot is generated based on the target environment data corresponding to the surrounding environment.

2. The method according to claim 1, characterized in that, The step of acquiring target environment data corresponding to the surrounding environment of the robot includes: Obtain the original environmental data corresponding to the surrounding environment and the height information corresponding to the robot; The height range of the robot is determined based on the height information corresponding to the robot; The original environmental data is filtered according to the height range to obtain the target environmental data corresponding to the surrounding environment.

3. The method according to claim 1, characterized in that, The step of generating a target local map corresponding to the robot based on the target environment data corresponding to the surrounding environment includes: Obtain environmental image data from the target environment data corresponding to the surrounding environment; The grounding wire identification model is invoked to identify the environmental image data, thereby obtaining the location of the grounding wire in the surrounding environment; Based on the location of the grounding wire and the robot's location, determine the passable area in the surrounding environment; A local target map corresponding to the robot is generated based on the passable area.

4. The method according to any one of claims 1 to 3, characterized in that, The target environment data includes environmental image data and environmental point cloud data; generating the target local map corresponding to the robot based on the target environment data corresponding to the surrounding environment includes: At least one first local map corresponding to the robot is generated based on the environmental image data corresponding to the surrounding environment. At least one second local map corresponding to the robot is generated based on the environmental point cloud data corresponding to the surrounding environment. The first local map and the second local map corresponding to the robot are fused to obtain the target local map corresponding to the robot.

5. The method according to claim 4, characterized in that, The step of fusing at least one first local map and at least one second local map corresponding to the robot to obtain a target local map corresponding to the robot includes: Iterate through multiple first locations in at least one second local map; For any one of the plurality of first locations, if the first height information of the first location is invalid, a second location corresponding to the first location is searched in the at least one first local map, and the second height information corresponding to the second location is determined; Replace the first altitude information with the second altitude information; The map obtained after traversal is determined as the target local map corresponding to the robot.

6. The method according to claim 1, characterized in that, Each of the multiple location information includes horizontal location information and vertical location information; The step of determining the operating state of the car based on the multiple location information collected at the multiple times includes: Based on the horizontal position information collected at the multiple time points, the standard deviation of the robot's horizontal displacement is determined. Based on the vertical position information collected at the multiple moments, the vertical displacement and vertical velocity of the robot are determined. If the standard deviation of the horizontal displacement is less than the standard deviation threshold, the vertical displacement is less than the displacement threshold, and the vertical velocity is less than the velocity threshold, then the operating state of the car is determined to be a stationary state.

7. The method according to claim 1, characterized in that, The multiple location information is collected according to a preset frequency; the method further includes: When the operating state corresponding to the car is in motion, the target environment data corresponding to the surrounding environment of the robot is masked, and the process returns to the step of obtaining multiple position information collected by the robot at multiple times.

8. A local map generation device, characterized in that, The device, applied to robots, includes: The position acquisition module is used to acquire multiple position information collected by the robot at multiple times when the robot is inside the car; The status determination module is used to determine the operating status of the car based on the multiple location information collected at the multiple times. The environmental data acquisition module is used to acquire target environmental data corresponding to the surrounding environment of the robot when the operating state of the car is stationary, and the surrounding environment includes the car. The map generation module is used to generate a local target map corresponding to the robot based on the target environment data corresponding to the surrounding environment.

9. A robot, characterized in that, The robot includes: body; A control system in communication with the fuselage, the control system comprising a processor and a memory in communication with the processor, the memory storing instructions which, when executed on the processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 7.

Images