Method for updating three-dimensional map, storage medium, electronic device, program product and robot

Abstract

The present application provides a method, storage medium, electronic device, program product and robot for updating a three-dimensional map. It relates to the field of robotics. The present application can only update the area that needs to be updated at the current moment to reduce the amount of calculation for updating the three-dimensional map. Specifically, the target area that needs to be updated can be determined first, and then the occupancy status of each three-dimensional unit in the target area can be updated based on the sensor data, so that the three-dimensional units whose occupancy status has changed in the target area block are synchronized to the three-dimensional map. Among them, the processing of sensor data in the present application can be performed in a space independent of the three-dimensional map to avoid directly affecting the three-dimensional map. At the same time, in this space, three-dimensional units are composed of a fixed number of three-dimensional blocks, and the aforementioned target area can be represented by a combination of target blocks to facilitate the rapid determination of the target area and the rapid indexing of the three-dimensional units that need to be changed.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a method for updating a three-dimensional map, a storage medium, an electronic device, a program product, and a robot. Background Art

[0002] A robot's perception of its environment has always been an important part of the field of robot control, especially in the autonomous operation of mobile robots. The autonomous operation process of such robots often requires sensing their surrounding environment and executing corresponding control strategies (such as moving, controlling robotic arm operations, etc.).

[0003] In practical robotic applications, various sensors installed on the robot can first collect sensory information and then update the three-dimensional map based on this information. However, to improve the accuracy of the three-dimensional map, robots are often equipped with multiple sensors. Processing sensor data often requires a large amount of computation, especially when updating the 3D map, which hinders the robot's rapid response to real-time data.

[0004] Therefore, how to improve the processing speed of three-dimensional maps is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a method, storage medium, electronic device, program product, and robot for updating a three-dimensional map, which can perform partial updates on the three-dimensional map to speed up the update process of the three-dimensional map.

[0006] In a first aspect, the present application provides a method for updating a three-dimensional map, which is applied to a mobile robot and includes: in response to sensor data collected at the current moment by sensors on the mobile robot, determining spatial point cloud data within a perception space based on the sensor data, wherein the perception space is configured as a three-dimensional space with the mobile robot as the origin. Determine multiple target blocks from each three-dimensional block in the perception space of the mobile robot, wherein the three-dimensional units in the perception space form a three-dimensional block in a fixed number. Determine the perception status of each three-dimensional unit of the multiple target blocks at the current moment based on the spatial point cloud data. Determine the valid unit detected by the sensor from the target block based on the perception status, and synchronize the perception status of the valid unit to the three-dimensional map of the mobile robot based on the local coordinates of the valid unit and the index relationship between the three-dimensional block and the three-dimensional unit, wherein the index relationship is determined based on the global coordinates of the three-dimensional unit.

[0007] In a second aspect, the present application provides an electronic device. The electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the three-dimensional map updating method described in the first aspect.

[0008] In a third aspect, the present application provides a computer program product, comprising a computer program / instruction, which implements the three-dimensional map updating method as described in the first aspect when the computer program / instruction processor is executed.

[0009] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program / instruction. When the computer program / instruction is executed by a processor, the three-dimensional map updating method described in the first aspect is implemented.

[0010] In a fifth aspect, the present application provides a mobile robot, which includes: a sensor for collecting sensor data of a perception space, wherein the transmitted data is used to determine spatial point cloud data of the perception space; at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for updating a three-dimensional map described in the first aspect.

[0011] The present application provides a method, storage medium, electronic device, program product and robot for updating a three-dimensional map. It relates to the field of robotics. The present application can only update the area that needs to be updated at the current moment to reduce the amount of calculation for updating the three-dimensional map. Specifically, the target area that needs to be updated can be determined first, and then the occupancy status of each three-dimensional unit in the target area can be updated based on the sensor data, so that the three-dimensional units whose occupancy status has changed in the target area block are synchronized to the three-dimensional map. Among them, the processing of sensor data in the present application can be performed in a space independent of the three-dimensional map to avoid directly affecting the three-dimensional map. At the same time, in this space, three-dimensional units are composed of a fixed number of three-dimensional blocks, and the aforementioned target area can be represented by a combination of target blocks to facilitate the rapid determination of the target area and the rapid indexing of the three-dimensional units that need to be changed. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 This is an exemplary flowchart of a three-dimensional map updating method provided in some embodiments of the present application.

[0014] Figure 2 This is an exemplary flowchart of a target block determination method provided in some embodiments of the present application.

[0015] Figure 3 This is a schematic diagram of a target block determination process provided by some embodiments of the present application.

[0016] Figure 4 This is an exemplary flowchart of an occupancy status updating method provided in some embodiments of the present application.

[0017] Figure 5 It is a schematic diagram of the effective unit determination process provided by some embodiments of the present application.

[0018] Figure 6 This is an exemplary flowchart of the index relationship determination method provided in some embodiments of the present application.

[0019] Figure 7 This is a system module diagram of a three-dimensional map updating system provided in some embodiments of the present application.

[0020] Figure 8 This is a schematic diagram of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0021] Application Overview

[0022] In related technologies, three-dimensional maps of a robot's surroundings are often presented as three-dimensional grid maps. A three-dimensional grid map is a tool for representing and processing three-dimensional spatial data. It discretizes the three-dimensional world into grids and stores relevant information in each grid, enabling modeling and analysis of complex environments.

[0023] In practical applications, each grid cell in a 3D grid map can be displayed as occupied or unoccupied, depending on whether there are obstacles in the corresponding area. This allows the 3D grid map to reflect the distribution of obstacles in space, facilitating path and task planning based on the 3D map.

[0024] The aforementioned three-dimensional grid Figure 1 It can generally be determined based on point cloud data. For example, a 3D scanning device can first acquire spatial point cloud data, then perform preprocessing on the point cloud data, such as filtering and denoising. The processed point cloud data is then converted into raster data, discretizing the continuous point cloud data into a 3D grid matrix.

[0025] It should be noted that in practical applications, the aforementioned three-dimensional grid map can also be converted into a two-dimensional grid map as needed. For example, during robot navigation, the aforementioned three-dimensional map can be filtered based on the robot's morphology to remove obstacles in the height direction that do not affect the robot's operation. The aforementioned three-dimensional map is then projected along the height direction to determine a two-dimensional grid map, and route planning or other operations are performed based on the two-dimensional grid map.

[0026] However, as the number of sensors integrated into robots increases, the aforementioned point cloud data can be acquired by different types of sensors. Determining the raster map often requires updating the 3D map based on each point cloud data point, which is computationally intensive. In particular, in real-world applications, moving obstacles may exist. To address these obstacles, the 3D map needs to be updated rapidly, but the aforementioned computational complexity makes it difficult to support such rapid updates.

[0027] The present application provides a method, storage medium, electronic device, program product and robot for updating a three-dimensional map, and relates to the field of robotics. The present application can only update the area that needs to be updated at the current moment to reduce the amount of calculation for updating the three-dimensional map. Specifically, the target area that needs to be updated can be determined first, and then the occupancy status of each three-dimensional unit in the target area can be updated based on the sensor data, so that the three-dimensional units whose occupancy status has changed in the target area block are synchronized to the three-dimensional map. Among them, the processing of sensor data in the present application can be performed in a space independent of the three-dimensional map to avoid directly affecting the three-dimensional map. At the same time, in this space, three-dimensional units are composed of a fixed number of three-dimensional blocks, and the aforementioned target area can be represented by a combination of target blocks to facilitate the rapid determination of the target area and the rapid indexing of the three-dimensional units that need to be changed.

[0028] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0029] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0030] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0032] Exemplary 3D map updating method

[0033] Figure 1 This is an exemplary flow chart of a three-dimensional map updating method (also referred to as a three-dimensional map updating method) provided in some embodiments of the present application. The three-dimensional map updating method provided in embodiments of the present application can be implemented by a mobile robot. A mobile robot may refer to a robot capable of mobility, equipped with various types of sensors to acquire sensory data (such as spatial point cloud data) about the scene in which it resides. The following description of the execution entities of each step will not be repeated.

[0034] It should be noted that the three-dimensional map update in this application is not limited to updating the data of an existing three-dimensional map. When there is no initial data in the three-dimensional map, it can also be initialized and constructed based on the method provided in this application.

[0035] like Figure 1 As shown, P100 may include the following steps:

[0036] S110 , in response to sensing data collected by sensors on the mobile robot at a current moment, determining spatial point cloud data within the perception space based on the sensing data.

[0037] S120 : Determine multiple target blocks from each three-dimensional block in the perception space of the mobile robot.

[0038] S130 : Determine, based on the spatial point cloud data, the sensor's perception of each three-dimensional unit of the plurality of target blocks at the current moment.

[0039] S140. Determine the valid unit detected by the sensor from the target block based on the perception situation, and synchronize the perception situation of the valid unit to the three-dimensional map of the mobile robot based on the local coordinates of the valid unit and the index relationship between the three-dimensional block and the three-dimensional unit.

[0040] In the aforementioned S110, the mobile robot is often equipped with multiple sensors to perceive the current environment. For example, the sensors installed on the mobile robot may include a lidar and three depth cameras in different locations. Two depth cameras may be installed on the mobile robot's arms, preferably the D405 model. One depth camera may be installed on the mobile robot's head, preferably the D415 model.

[0041] In some embodiments, sensors installed on the mobile robot can have different data collection cycles, thereby achieving sensor data fusion from multiple sensors by responding to the sensor data from different sensors in a sequential manner. That is, when a sensor on the mobile robot collects sensor data, the aforementioned method P100 can be executed immediately to complete the three-dimensional map update before the next sensor collects data.

[0042] In the aforementioned S110, the perception space may be a three-dimensional map synchronized with the three-dimensional map, a user-cached three-dimensional map that updates the map. The perception space may reflect the detection range of the sensors on the mobile robot. The perception space may be centered on the mobile robot.

[0043] In some embodiments, the sensor data collected by the aforementioned sensors can be converted into spatial point cloud data within the perception space. Spatial point cloud data can refer to point cloud data presented in the perception space. Point cloud data refers to a set of vectors in a three-dimensional coordinate system, where each point contains three-dimensional coordinates, and some may contain color information (such as RGB) or reflection intensity information.

[0044] Specifically, point cloud data is usually acquired through devices such as laser scanners, depth cameras, binocular cameras, and optical cameras. Specifically, power data can be the points where the light released by the aforementioned sensor devices is reflected in space, thereby reflecting the three-dimensional spatial situation of the current space.

[0045] In some embodiments, the spatial point cloud data in S110 may be obtained by performing coordinate conversion based on the point cloud data of the sensor. Specifically, point cloud data with the sensor as the origin may be first determined based on the sensor data, and then, based on the position of the sensor in the perception space, the point cloud data may be converted into point cloud data within the perception space (referred to as spatial point cloud data).

[0046] In the aforementioned S120, the three-dimensional unit may refer to the basic unit of the perception space. The size of the three-dimensional unit in each reference direction may be 1 standard unit. Considering that the perception space is generally resolved as a space in a three-dimensional Cartesian coordinate system, the aforementioned three-dimensional unit can also be understood as a voxel in three-dimensional space. A voxel can represent a cube in three-dimensional space, with each side having a length of 1 standard unit. Furthermore, considering that the perception space is generally aligned with a three-dimensional map, the three-dimensional unit can also be regarded as the basic building block of the three-dimensional map.

[0047] In practical applications, the actual size of the three-dimensional unit can be determined based on the actual detection accuracy. For example, when the mobile robot is an embodied robot, the length of one standard unit in the sensory space can be mapped to 1 cm in the physical space, that is, the three-dimensional unit is a cube with a side length of 1 cm.

[0048] To increase the speed of 3D map updates, this application may update only some 3D cells when updating a map. To quickly determine which 3D cells need to be updated, this application may treat fixed-size 3D cells as a 3D block based on a fixed size, thereby determining the 3D cells that need to be updated based on the 3D block that needs to be updated (referred to as the target block).

[0049] That is, the aforementioned 3D block can be a combination of a fixed number of 3D cells of a fixed size. To ensure the stability and accuracy of the 3D block's indexing of the 3D cells, the 3D cells contained in the 3D block are often continuous. For example, a block may include a continuous 8×8×8 3D cells.

[0050] A target block may refer to a 3D block that needs to be updated at the current moment. That is, the target block may be a portion of a block in the perception space. In some embodiments, the target block may be determined based on actual needs. For example, the spatial range that needs to be updated may be first determined within the perception area, and then a minimum 3D block combination encompassing that spatial range may be determined as the multiple target blocks.

[0051] In some embodiments, multiple update strategies can be configured during the 3D map update process in this application. Specifically, multiple block types with different update cycles can be pre-set. Based on the correspondence between the current moment and the update cycle, the block type corresponding to the update cycle containing the current moment is determined and used as the target block. The correspondence between block type and update cycle (or update frequency) can be determined based on actual update needs. For example, the update frequency for obstacles within the mobile robot's operating range can be 10 Hz, while the update frequency for the entire space can be 1 Hz. For another example, the update frequency for obstacles moving toward the mobile robot can be 10 Hz, while the update frequency for obstacles moving away from the mobile robot (i.e., obstacles that do not affect the robot's movement) can be 1 Hz.

[0052] Therefore, when determining the target block, a target update strategy corresponding to the sampling time including the current time can be first determined from multiple map update strategies based on the current time. The multiple map update strategies have different update cycles and update regions. Multiple target blocks can then be determined from each 3D block in the perception space based on the target update strategy.

[0053] In some embodiments, the target block includes at least the current working range of the mobile robot within the perception space, thereby updating the three-dimensional map around the mobile robot. The working range of the mobile robot can reflect the space required by the mobile robot to perform its work task at the current moment. Therefore, when updating the three-dimensional map, the spatial range required by the mobile robot itself can be updated first, so as to quickly identify obstacles (especially dynamic obstacles) within this spatial range. For more information about the working range, please refer to Figure 2 、 Figure 3 and its related descriptions.

[0054] For the convenience of subsequent description, each 3D unit contained in the target block may be referred to as a target 3D unit. After determining the target block, the sensor perception status of each target 3D unit may be determined, ie, the aforementioned S130 is executed.

[0055] In the aforementioned S130, the perception status can reflect the sensor's perception of the target three-dimensional unit at the current moment. In practical applications, the perception status can reflect at least two aspects: whether each target three-dimensional unit is detected by the sensor and whether the target unit is occupied by an obstacle when detected by the sensor.

[0056] Specifically, consider that sensors on mobile robots can perceive the external environment based on optical principles. For example, radar sensors typically perceive the surrounding environment by emitting invisible light beams, while camera sensors typically perceive the surrounding environment by directly collecting ambient light. If the light between the sensor and a specific location is blocked, the sensor cannot perceive the situation at that specific location. However, if there are no obstacles blocking the light transmission between the sensor and the specific location, the presence of an obstacle at that specific location can be determined by whether the sensor's light passes through (or stops at) that specific location.

[0057] The aforementioned specific positions can reflect each three-dimensional cell within the perceived space, and the relationship between the light and the specific positions can specifically reflect the light passing through the three-dimensional cell and the light hitting the three-dimensional cell. Light passing through a three-dimensional cell can be understood as the light at least entering the three-dimensional cell, and light hitting a three-dimensional cell can reflect the light hitting the point cloud within the three-dimensional cell (or the contour formed by it). It should be noted that light passing through a three-dimensional cell does not conflict with light hitting a three-dimensional cell; when light hits a three-dimensional cell, it also enters the three-dimensional cell.

[0058] Therefore, when updating the three-dimensional map, the light released by the sensor can be simulated in the space formed by multiple target blocks, so as to determine the perception of each three-dimensional unit based on the three-dimensional unit through which the light passes. Specifically, multiple projection light rays can be generated in multiple target blocks based on the spatial point cloud data of multiple target blocks. Then, the passage of each three-dimensional unit through which the projection light passes in the multiple target blocks is determined and stored in the storage area of ​​the corresponding three-dimensional unit to determine the perception of the three-dimensional unit. Among them, the multiple target blocks include multiple target blocks, the projection light is released from the sensor and hits the spatial point cloud data, and the perception situation includes at least the first light quantity and the second light quantity.

[0059] Specifically, during the aforementioned process, two counters can be configured within the storage area of ​​each 3D cell, one for recording the number of light rays that pass through that 3D cell (recorded as the first ray count) and the other for recording the number of light rays that hit that 3D cell (recorded as the second ray count). During the aforementioned process, when determining which 3D cells a projected light ray passes through and hits, a corresponding counting instruction can be generated to count the number of times a projected light ray passes through the corresponding counter. For each 3D cell the projected light ray passes through, the first ray count plus 1 is added to the number of times a projected light ray hits the 3D cell, and the second ray count plus 1 is added to the number of times a projected light ray hits the 3D cell.

[0060] In some embodiments, the aforementioned process can be determined based on a raycasting algorithm. The ray's release position is the sensor's position, and its release range is the space formed by multiple target blocks. For example, a raycast algorithm can be used to determine the ray and the three-dimensional cells it passes through by selecting a starting point and an end point and iterating continuously.

[0061] In S140, valid cells may refer to 3D cells sensed by the sensor. Based on the description of S130, each 3D cell within the target block may have portions not sensed by the sensor. To distinguish between these, specific data (such as the first and second light beam counts) may be written into the sensed state of a corresponding 3D cell upon detection, representing a valid cell. Cells without data written into them may be considered invalid cells.

[0062] The first and second ray counts can be re-determined each time a detection is performed and reset after data is updated. That is, when the first ray count is 0 or empty at the current moment, the corresponding three-dimensional unit is not detected by the sensor at the current moment and is an invalid unit.

[0063] Taking into account that the aforementioned perception situation is determined based on the space formed by the target block, the data can be written directly into the storage area of ​​the corresponding three-dimensional unit when determining the perception situation. After the writing is completed, in order to determine the valid unit, it is necessary to traverse each three-dimensional unit in the target area one by one. In order to reduce the difficulty of processing, the traversal can be performed through the local coordinates / local index information of the three-dimensional unit in the target block. That is, when determining the aforementioned valid unit, the storage area of ​​each three-dimensional unit in the target block can be traversed based on the local coordinates of the three-dimensional unit to determine the first light number and the second light number of each three-dimensional block. Then, based on the first light number, the valid unit and the local coordinates of the valid unit are determined from the target block, wherein the first light number of the valid unit is a non-zero value.

[0064] In some embodiments, considering that when the aforementioned light is projected, each perceived three-dimensional unit is passed by the light, then in fact (the second light quantity) can reflect the situation where the projected light ends at the three-dimensional unit (there may be an obstacle here), and (the first light quantity - the second light quantity) can reflect the situation where the projected light penetrates the three-dimensional unit (that is, there may be no obstacle here).

[0065] Therefore, when (number of second rays) > (number of first rays - number of second rays), it can be considered that the 3D cell is more likely to have an obstacle, while when (number of second rays) < (number of first rays - number of second rays), it can be considered that the 3D cell is more likely to have no obstacle. The probability of a 3D cell having an obstacle can be recorded as the occupancy probability, and the result of determining whether the 3D cell is an obstacle in the map based on the occupancy probability can be recorded as the occupancy state.

[0066] It should be noted that the aforementioned counting of the first number of rays is merely an example provided in this application. In an alternative embodiment, when projecting rays, the three-dimensional cells hit by the rays can be superimposed only on the second number of rays, rather than on the first number of rays. In this case, the first number of rays is equivalent to the aforementioned first number of rays minus the second number of rays, allowing for adaptive modification.

[0067] Based on the above description, the occupation probability change value of the effective unit at the current moment can be determined based on the first light number and the second light number of the effective unit, and then the occupation probability change value can be synchronized to the corresponding map element in the three-dimensional map of the mobile robot based on the local coordinates and index relationship of the effective unit to determine the occupation probability and occupation status of the map element.

[0068] Based on the above, considering that the aforementioned traversal process is generally based on the local coordinate traversal of the three-dimensional unit in the three-dimensional block, its global coordinates are not known after determining its actual probability change value. Therefore, when updating the three-dimensional map, the global coordinates of the valid unit in the perception space can be determined based on the local coordinates of the valid unit and the index relationship between the three-dimensional block and the three-dimensional unit, and then the corresponding data can be synchronized to the corresponding map element (i.e., the three-dimensional unit in the three-dimensional map) based on the mapping relationship between the perception space and the three-dimensional map.

[0069] In some embodiments, the index relationship between the aforementioned 3D blocks and 3D cells can be determined based on the global coordinates of the 3D cells and the fixed size of the 3D region. That is, the corresponding relationship between the blocks and 3D cells can be pre-configured before executing the aforementioned P100. Specifically, the following steps can be included:

[0070] First, the number of cells in the three-dimensional block in the reference direction is determined.

[0071] Next, each block in the perception space is determined based on the number of units, and a block list is generated.

[0072] Next, block index information of the 3D cell is determined based on the global coordinates of the 3D cell and the number of cells.

[0073] Next, a block reference position of the 3D unit corresponding to the block is determined based on the block index information and the number of units, and a local coordinate of the 3D unit in the corresponding block is determined based on the block reference position.

[0074] Finally, local index information of the 3D unit in the corresponding block is determined based on the local coordinates of the 3D unit in the corresponding area.

[0075] For example, the size of the block can be determined based on the size of the perceptual space, thereby dividing the blocks. Based on the division results, a block table (e.g., a hash table) is constructed, containing each 3D block in the perceptual space. The 3D units contained in the block are then determined based on the block size, and the correspondence between the blocks and the 3D units is written into the aforementioned block table. The corresponding 3D unit can then be indexed based on the block table. A 3D unit table can also be formed for each 3D unit, and the corresponding 3D block can be determined based on the address of each 3D unit in the table.

[0076] In some embodiments, considering that the perception space is centered on the mobile robot, the correspondence between the aforementioned blocks and 3D units can be determined directly based on a coordinate system based on 3D units (e.g., the coordinate system within the perception space). For example, 3D blocks can also be identified based on multiple reference directions. In this case, the basic unit of 3D block coordinates can be understood as a 3D block. For example, a 3D block with coordinates (1, 1, 1) represents the first block in the x-axis, the first in the y-axis, and the first in the z-axis. The coordinates of the 3D block within this space can be recorded as block index information.

[0077] Block index information can be used to index the coordinate range of the corresponding block in the perceptual space. For example, for the 3D block with block index information (1,1,1), the number of units in each direction is 8, so its coordinate range in the perceptual space is (8,8,8) to (15,15,15). The coordinate with the smallest absolute value of the 3D block in the perceptual space can be recorded as the block reference position. This block reference position can be determined directly by multiplying the block index information by the number of units in each reference direction.

[0078] For example, for a three-dimensional unit (x, y, z), the corresponding block identification information can be ([x / n x ],[y / n y ], [z / n z ]), where n x , n y , n z are the number of three-dimensional units in the block in the X, Y, and Z directions, respectively. For example, in the aforementioned block, n x , n y , nz They can all be 8, and the symbol [ ] can reflect downward evidence collection.

[0079] As an alternative embodiment, the correspondence between the aforementioned three-dimensional unit and the block can also be determined based on bitwise operations. For details, see Figure 6 The related descriptions are not repeated here.

[0080] After determining the aforementioned block index information, the local coordinates of each 3D cell within the 3D block can be determined. Here, local coordinates = global coordinates - block reference position. After determining the local coordinates, they can be vector-encoded, thereby converting them into a sequence for easier storage. For example, for a 3D block with the aforementioned block index information (1,1,1), the local coordinates of the 3D cell (9,15,8) within it can be (1,7,0), and its local index information can be 170 (which can also be converted to two for storage). Therefore, in practical applications, the aforementioned local index information can reflect the local coordinates of the 3D cell within the corresponding block and be used to determine the global coordinates of the 3D cell in combination with the block reference position.

[0081] To reflect the correspondence between three-dimensional units and three-dimensional coordinates, the aforementioned block list may include block index information of the three-dimensional block and local index information of each three-dimensional unit contained in the three-dimensional block, wherein each three-dimensional block can be reflected by the block index information, and then each three-dimensional unit within the three-dimensional block can be reflected by the aforementioned block index information.

[0082] In some embodiments, in executing the aforementioned S120, after determining the target block, all blocks except the target block in the aforementioned block list can be deleted based on the target block, thereby directly calling each 3D unit as the target 3D unit based on the deleted block list.

[0083] In the aforementioned S140, the occupancy state may reflect whether the three-dimensional cell is occupied by an obstacle. The occupancy state may generally include an idle state and an occupied state. When the three-dimensional cell is in the idle state, it may reflect that the three-dimensional cell is not occupied by an obstacle, while when the three-dimensional cell is in the occupied state, it may reflect that the three-dimensional cell is occupied by an obstacle.

[0084] In some embodiments, the aforementioned S140 may be performed based on the distribution of the spatial point cloud data in each target three-dimensional cell. For example, when the target three-dimensional space contains spatial point cloud data, the target three-dimensional cell may be determined to be occupied, otherwise it may be determined to be idle, thereby determining the occupied state.

[0085] In some embodiments, given that the aforementioned occupancy state may gradually change during the motion of the mobile robot, the occupancy probability of the target three-dimensional cell can be updated at each moment to reflect the corresponding occupancy state. If the updated occupancy probability is greater than a threshold, the cell is considered occupied; otherwise, it is considered idle.

[0086] Therefore, based on the method of updating the three-dimensional map shown in the aforementioned P100, in order to quickly determine the three-dimensional map around the robot, the present application can only update the area that needs to be updated at the current moment to reduce the amount of calculation for updating the three-dimensional map. Specifically, the target area that needs to be updated can be determined first, and then the occupancy status of each three-dimensional unit in the target area can be updated based on the sensor data, so that the three-dimensional units whose occupancy status has changed in the target area block are synchronized to the three-dimensional map. Among them, the processing of sensor data in the present application can be performed in a space independent of the three-dimensional map to avoid directly affecting the three-dimensional map. At the same time, in this space, the three-dimensional units are composed of a fixed number of three-dimensional blocks, and the aforementioned target area can be represented by a combination of target blocks to facilitate the rapid determination of the target area and the rapid indexing of the three-dimensional units that need to be changed.

[0087] In some embodiments, the difference between the sensor sensing data and the perceived space is taken into account in the aforementioned S110. To further describe the process of determining spatial point cloud data, the following uses a depth camera as an example to illustrate the process of obtaining the aforementioned spatial point cloud data.

[0088] First, the sensing data of the sensor at the current moment may be acquired, wherein the sensing data of the depth camera may be based on the depth image.

[0089] Specifically, the depth image of the depth camera can be collected through a specific data interface (ros2 topic), and the depth image of the depth camera at each moment can be collected through this data interface to determine the depth image at the current moment. The depth image can include each pixel in the perception space and its depth information.

[0090] Next, the sensing data is mapped into the perception space to serve as initial point cloud data in the perception space. When determining the initial point cloud data, the point cloud data may be determined first and then mapped.

[0091] The three-dimensional coordinates of each pixel can be calculated first: Based on the camera's intrinsic parameters (focal length, principal point, etc.) and the two-dimensional coordinates (u, v) of the pixel and the corresponding depth value d, the three-dimensional coordinates (x, y, z) of the pixel in the camera coordinate system can be calculated. The specific calculation formula is as follows:

[0092] .

[0093] Where x, y, and z are the coordinates of the camera coordinate system, (u, v) are the pixel coordinates in the depth image, d is the depth value of the pixel (usually in meters), and cx, cy, fx, and fy are the camera intrinsic parameters.

[0094] Then, based on the coordinate transformation of the mobile robot, the point cloud data in the aforementioned camera coordinate system can be converted into the coordinate system of the perception space to serve as the initial point cloud data.

[0095] Specifically, the point cloud data in the camera coordinate system can be converted to the coordinate system of the perception space based on the joint where the depth camera is located and the joint coordinate transformation information (Transform, TF) of each joint in the mobile robot. In mobile robots (especially embodied robots), each joint corresponds to a TF, which is used to describe the relative position and orientation between joints. Therefore, the point cloud data based on the joint where the depth camera is located can be mapped to the coordinate system of the perception space.

[0096] For example, for the aforementioned depth camera set at the robotic arm, the transformation matrix of the depth camera at the bottom center point of the mobile robot (or a position that can serve as the origin) can be determined based on the joint coordinate transformation information of the robotic arm, thereby determining the initial point cloud data.

[0097] Finally, the initial point cloud data is filtered based on the three-dimensional unit to determine the spatial point cloud data.

[0098] After determining the initial point cloud data, filtering can be performed based on the aforementioned 3D units (i.e., voxels with a side length of 1 cm) to reduce the point cloud density and generate a downsampled point cloud. After filtering, each 3D unit contains at most one point cloud. Specifically, the center of gravity of each 3D unit can be selected as the representative point, thereby downsampling the initial point cloud data, significantly reducing the point cloud data volume while preserving the essential shape characteristics of the original point cloud.

[0099] It should be noted that the order of some of the descriptions in the above process can be reversed. For example, after determining the point cloud data in the sensor coordinate system, filtering processing can be performed before coordinate conversion to reduce the amount of data to be processed.

[0100] Exemplary target block determination method

[0101] To further illustrate the process of determining the target block, the present application also provides an exemplary flow chart of a target block determination method ( Figure 2 ) and schematic diagram ( Figure 3 ).

[0102] like Figure 2 As shown, process P200 may include the following steps:

[0103] S210: Determine the working range of the mobile robot at the current moment based on the working state of the mobile robot.

[0104] S220 , determining a plurality of target blocks from the perception space based on the working range.

[0105] In the aforementioned S210, the working state of the mobile robot can reflect the current working status of the mobile robot. In actual work, the working state can at least reflect the current working task of the mobile robot. The working task can refer to the work that the mobile robot currently needs to complete.

[0106] The working range can reflect the range of movement of the mobile robot to perform the current task. For example, when the mobile robot needs to use the robotic arm to perform work, its working range can reflect the maximum range of movement of the robotic arm in space.

[0107] In some embodiments, in order to ensure the normal operation of the mobile robot, the aforementioned working range can also expand the activity range of performing the current working task based on a preset ratio, so as to maintain a certain perception redundancy to avoid the occurrence of emergencies.

[0108] In some embodiments, considering that the operation task of the mobile robot may involve a motion process, the aforementioned S210 may include the following sub-steps:

[0109] S211. Determine the motion status and operation task of the mobile robot at the current moment based on the working state of the mobile robot.

[0110] S212: Determine the operating range of the mobile robot when performing the operating task based on the operating task.

[0111] S213: Determine the working range based on the movement situation and the working range.

[0112] In the aforementioned S211, the motion situation can be determined based on the motion requirements, historical motion trajectory and current motion state of the mobile robot, and is mainly used to describe the directional motion trajectory of the mobile robot from the current moment to the next moment.

[0113] In S212, the operating range may refer to the space required to perform the task. Taking into account the mobile robot's motion, the operating range may include both the current operating range and the next operating range. For example, if the mobile robot is currently moving toward its destination and its robotic arm is generally not in use, its operating range will be essentially the size of its main body with some added margin for movement. If the mobile robot reaches the destination and begins operations at the next moment, its operating range may be determined based on its actual task (e.g., the maximum range of motion of the robotic arm in space).

[0114] In some embodiments, the working range can be determined based on the aforementioned motion conditions and operating range. For example, the position of the mobile robot at the next moment can be determined based on the motion conditions, and the operating range at the next moment can be determined based on the position at the next moment. Thus, based on the current operating range and the next operating range, a minimum continuous area that encompasses both areas can be determined as the working range.

[0115] In some embodiments, path fusion technology can also be used to first determine the directional motion trajectory of the mobile robot between the current moment and the next moment based on the motion situation, and then perform path fusion on the operating range based on the directional motion trajectory to determine the working range. Path fusion can be understood as an image algorithm that generates images by superimposing images along a path.

[0116] To further illustrate the scope of work process described above, see Figure 3 Among them, considering that three-dimensional images are difficult to identify, Figure 3 It is a schematic diagram on the XY plane during the target block determination process.

[0117] like Figure 3 As shown, Figure 3 The perception space 300 is based on the mobile robot 310 as the origin and is presented in blocks as basic units. Figure 3 In the partial enlarged figure A, it can be seen that a block can include 8*8 three-dimensional units in the XY plane. In practice, considering the Z axis, a block can include 8*8*8=512 three-dimensional units.

[0118] like Figure 3As shown, in the process of determining the working range, the working range 311 of the mobile robot 310 at the current moment and the directional movement path 312 between the current moment and the next moment can be determined first based on the work task. When path fusion is adopted, the working range 311 at the current moment can be mapped to the working range 313 at the next moment based on the directional movement path 312, and a closed area 314 can be formed based on the working range 311 and the working range 313. At this time, the working range can be determined based on the closed area 314, which can be the minimum continuous block combination that can cover the area. In some embodiments, in order to facilitate subsequent processing steps, the working range can be configured as the minimum rectangular range that can cover the closed area 314. Figure 3 In some embodiments, considering that the target block may be selected in the subsequent step of determining the target block, the aforementioned working range may also be configured as the aforementioned closed area 314.

[0119] In addition, the size of the working range 313 at the next moment may be different from the working range 311 at the current moment. In the current moment, the mobile robot 310 may be in the process of moving, and at the next moment, the mobile robot 310 may have completed the movement and is in the working state.

[0120] In the aforementioned S220, after determining the working range, a combination of continuous blocks that can completely cover the working range can be directly determined based on the working range to serve as multiple target blocks. In some embodiments, the target blocks can be presented in a specific shape (such as the aforementioned rectangle) to facilitate subsequent determination of occupancy status.

[0121] In some embodiments, considering that there may be dynamic obstacles moving toward the mobile robot in actual applications, in order to improve the ability to avoid dynamic obstacles, the aforementioned S220 may further include the following sub-steps:

[0122] S221. Determine at least one dynamic obstacle from the perception space.

[0123] S222: Determine a motion area of ​​at least one dynamic obstacle.

[0124] S223 , determining a plurality of target blocks from the perception space based on the motion region and the working range.

[0125] In S221, a dynamic obstacle may refer to an obstacle that moves relative to the mobile robot. For example, a dynamic obstacle may include an autonomously moving obstacle (such as a person) or a fixed obstacle that becomes relatively fixed due to the movement of the mobile robot. Specifically, when determining dynamic obstacles, obstacles that may affect the mobile robot may be identified. For example, a dynamic obstacle may be moving toward the mobile robot or located in the mobile robot's path.

[0126] In some embodiments, the aforementioned S221 can be determined based on a depth image from a depth camera. For example, a dynamic obstacle can be determined based on multiple consecutive depth images that do not contain depth information, using a target recognition algorithm and a tracking algorithm. The dynamic obstacle is then mapped into the perception space based on the depth information of the corresponding point cloud.

[0127] In S222, the motion region of the dynamic obstacle can reflect the range of the dynamic obstacle's movement from the current moment to the next moment or subsequent moments. Specifically, it can be determined based on its historical motion trajectory. Similar to the aforementioned working range, the motion region can also be represented as a rectangular range in the perception space.

[0128] In the aforementioned S223, when determining the target block, it is possible to control the target block to cover the union of the motion area and the working range. For example, the smallest rectangle covering the motion area and the working range can be determined from the perception space, and each block contained in the rectangle is used as the target block.

[0129] See Figure 3 ,like Figure 3 As shown, perception space 300 may also include dynamic obstacles 320 associated with mobile robot 310 and its motion region 321 between the current moment and the next moment. Based on motion region 321 and the aforementioned working range (the blue-edged rectangular range or closed area 314), a minimum rectangular area encompassing these two areas is determined as the target block (presented as a red-edged rectangular range).

[0130] In some embodiments, after the aforementioned range is determined, a copy of the block list may be constructed and blocks other than the target block in the block list may be removed, thereby determining the target 3D unit based on the target block.

[0131] Exemplary Occupancy Status Update Method

[0132] To further illustrate the process of updating the occupancy status, the present application also provides an exemplary flow chart of an occupancy status update method ( Figure 4 ).

[0133] like Figure 4 As shown, process P400 may include the following steps:

[0134] S410 , determining valid units, the number of first rays, and the number of second rays in the valid units from a plurality of target blocks.

[0135] S420 : Determine an occupation probability change value of the valid unit at the current moment based on the first light quantity and the second light quantity of the valid unit.

[0136] S430: Determine the occupation probability of the corresponding map element of the valid unit before the current moment.

[0137] S440: Update the occupation probability based on the occupation probability change value of the valid unit, and determine the occupation probability of the valid unit at the current moment.

[0138] S450: Synchronize the occupation probability of the valid unit at the current moment to the corresponding map element based on the local coordinates and index relationship of the valid unit, and determine the occupation state of the corresponding map element.

[0139] For the aforementioned S410 , please refer to the relevant description of P100 .

[0140] In the aforementioned S420, considering that the occupancy probability change value can reflect whether the corresponding three-dimensional unit at the current moment is more inclined to be determined as having no obstacle or having an obstacle, the occupancy probability change value can be determined according to the first light number and the second light number.

[0141] Specifically, the updated occupancy probability change (log_odds_update) can be calculated based on the relationship between the second number of rays (num_hit) and the first number of rays (num_pass). If num_hit is greater than or equal to (num_pass - num_hit), log_odds_update is set to the first change (prob_hit_log); otherwise, it is set to the second change (prob_free_log). Then, num_hit and num_pass are reset to 0, pending the next update.

[0142] The above process can be expressed as the following formula:

[0143] .

[0144] In some embodiments, when determining the occupancy status, the occupancy probability of the corresponding map element may be directly updated based on the aforementioned occupancy probability change value, thereby determining an updated occupancy probability.

[0145] In the aforementioned S430, in actual applications, the mobile robot can move within the 3D map, thereby aligning the 3D map with the perception space at the current moment. This alignment of the 3D map and the perception space does not require them to be based on the same coordinate system; it only requires a transformation relationship between the two. During application, the map element corresponding to each valid unit can be determined based on this transformation relationship.

[0146] In some embodiments, considering that valid units within the perception space may not have been detected, i.e., the map elements corresponding to the valid units may not have been configured with occupancy probabilities and occupancy states, initial configuration can be performed for these map elements in this step. Specifically, based on the sign of the aforementioned occupancy probability change value, an initial occupancy probability (e.g., 0.7) and an initial idle probability (e.g., 0.35) can be configured. After configuring the initial occupancy probability, the update in S430 can be skipped, allowing synchronization to proceed directly to S440.

[0147] When determining the map element corresponding to the valid unit, the occupancy probability stored in the map element can be called. Considering that the map element has not been updated, the occupancy probability can be the occupancy probability of the previous moment or the previous moment (denoted as P -1 (occupancy)).

[0148] In actual calculations, the occupancy probability value is generally positively correlated with the likelihood of an obstacle. That is, a higher occupancy probability value indicates a higher likelihood of an obstacle at that location. Therefore, the first change value of the occupancy probability change can be positive to reflect a positive change. Conversely, the second change value can be negative to reflect a negative change.

[0149] In some embodiments, considering that the occupancy probability may exceed the limit (such as greater than 1) during the iteration process, upper and lower limits can be set, so that when the updated result is greater than the corresponding upper and lower limits during the update, only the corresponding upper and lower limits are used.

[0150] From this, we can first determine the relationship between the change in occupancy probability (the value of log_odds_update) and the occupancy probability at the previous moment:

[0151] If log_odds_update is greater than or equal to 0 and P -1 (occupancy) is greater than or equal to the maximum occupancy probability (denoted as max_log), or log_odds_update is less than or equal to 0 and P -1 If (occupancy) is less than or equal to the minimum occupancy probability (denoted as min_log), it is returned directly without updating.

[0152] Otherwise, based on P -1 (occupancy) determines the occupancy probability at the current moment (denoted as P0(occupancy)), where P0(occupancy) can be configured as P -1(occupancy) + log_odds_update, and ensure that it is between min_log_ and max_log_. The specific formula can be as follows:

[0153] .

[0154] Among them, prob_hit_log and prob_free_log represent the logarithmic probabilities of the idle state and occupied state, and max_log and min_log define the logarithms of the maximum and minimum values ​​of the occupied probability. prob_hit_log, prob_free_log, max_log, and min_log can be empirical values.

[0155] Thus, after the occupancy probability is determined, the corresponding map element can be updated based on the occupancy probability, thereby determining the occupancy state by combining the threshold.

[0156] The specific algorithms / values ​​of each value in the above description are as follows:

[0157] . . . .

[0158] . . . . The initial value is set to 0.

[0159] In some embodiments, to further illustrate the detailed process of projecting light, the present application also provides a schematic diagram of a valid unit determination process.

[0160] like Figure 5 As shown, for the aforementioned Figure 3 The red target block is different from Figure 3 ,exist Figure 5 Each block in the figure can represent each three-dimensional unit. Figure 5 The 510 in the figure can reflect the sensor that is currently performing ray casting.

[0161] exist Figure 5 In , light can be projected according to the position of the sensor, where each light can be blocked by the point cloud inside and outside the area, thus forming a light line segment between the sensor and each position. Figure 5 For ray casting, each three-dimensional unit (indicated by a blue frame in the figure) passing through the ray can be used as a valid unit, and then subsequent synchronization is performed based on the number of first rays and the number of second rays.

[0162] Furthermore, during actual ray casting, the projection space may be larger than the space formed by the target block, so as to improve the projection effect on the three-dimensional units at the edge of the space.

[0163] It should be noted that in actual applications, determining the perception status of 3D cells within the perception space and updating the 3D map can also be configured as two independent processes, so that some steps in the aforementioned process can also be performed within the 3D map. For example, only the occupancy probability change value of a 3D cell can be synchronized to the corresponding map cell, and the occupancy probability and occupancy status of the map cell can be updated by the relevant processes of the 3D map.

[0164] Exemplary index relationship determination method

[0165] To further illustrate the index relationship between the 3D block and the 3D unit, the present application also provides a method for determining the index relationship based on bit operations ( Figure 6 ).like Figure 6 As shown, process P600 may include the following steps:

[0166] S610: Determine the number of cells of the three-dimensional block in a reference direction, and determine the bit length of the cell number based on the number of cells.

[0167] S620 , performing a right shift process on the global coordinates of the 3D unit based on the bit length, and determining the corresponding block of the 3D unit and the block index information of the corresponding block.

[0168] S630 : Determine the local coordinates of each 3D unit in the 3D block based on the block reference position of the 3D block.

[0169] S640: Determine a stacking order of each reference direction, and perform vector encoding on the local coordinates of the three-dimensional unit based on the stacking order to determine local index information of the three-dimensional unit in the corresponding block.

[0170] S650: Generate a block list based on the local index information of the three-dimensional unit in the corresponding block and the block index information.

[0171] In the aforementioned S610, to facilitate bit operations, the number of cells is a power of 2, and the bit length reflects the number of cells raised to the power n of 2. For example, a 3D block may have 2^3=8 3D cells in each direction. Therefore, the number of cells is 8, and the corresponding bit length may be 3.

[0172] In particular, the value range of each target three-dimensional unit in the local coordinates can be (0,0,0)~(( block _ num − 1), ( block _ num − 1), ( block _ num − 1)). Among them, block _ num can be a binary expression of the number of the aforementioned units, block _ num – 1 can reflect the maximum value of the local coordinate in that direction. block _ num = 1<< block _ bit For example, when the number of units is 3, the block _ num is the binary "1000" to represent the number 8, then ( block _ num − 1) is “111” in binary to represent the maximum value of the block coordinates.

[0173] Therefore, in the aforementioned S620, for the global coordinates of any three-dimensional unit vox _ index1 = [ int x , int y , int z ], then the corresponding block index information can be block _ index = [ x >> block _ bit , y >> block _ bit , z >> block _ bit ] block _ index = [ x >> block _ bit , y >> block _ bit , z >> block _ bit ]. Similar to spatial coordinates, block index information can be represented as three-dimensional coordinates in space, and each coordinate can reflect a corresponding sequence number.

[0174] Considering that in the block, the local coordinate value is 0~ ( block _ num − 1), then the corresponding local coordinates [ x , y , z ] , whose global coordinates can be represented as vox _ index1 = [ x ,y , z + block _ index ∗ block _ num.

[0175] In addition, the three-dimensional cell range included in the block can also be characterized as: block_index ∗ block_num~[block_num − 1, block_num − 1, block_num − 1] + block_index ∗ block_num.

[0176] In some embodiments, considering that the local coordinates also need to be characterized as three arrays during storage, in order to simplify the storage logic, the vector coordinates can be encoded and converted into a numerical value (denoted as local index information). Specifically, the values of the three coordinate axes can be directly arranged in sequence.

[0177] Exemplarily, for the local coordinate vox_index2 = [x, y, z], its local index information can be characterized as int vox_index_in_block = (x&(block_num − 1))<<block_bit<<block_bit + (y&(block_num − 1))<<block_bit + (z&(block_num − 1)). That is, the local index information is the coordinates of the xyz axes arranged in sequence.

[0178] [[ID=2二十二]]In addition, the aforementioned local index information can be directly converted into global coordinates, so as to execute the aforementioned step S140. Based on the local index information with the coordinates of the xyz axes arranged in sequence vox _ index _ in _ block , its corresponding global coordinates vox_ index1 can be:

[0179] z = vox_index_in_block&(block_num − 1).

[0180] y = (vox_index_in_block>>block_bit)&(block_num − 1).

[0181] x = (vox_index_in_block>>block_bit>>block_bit)&(block_num − 1).

[0182] vox_index1 = [x, y, z] + block_index * block_num.

[0183] Based on the aforementioned process, the block list may record the index information of each block and the local index information of the 3D unit of each block.

[0184] Therefore, the construction and application of index relationships based on the above method can be implemented based on bit operations, avoiding the calculation amount of positive number multiplication and division, and reducing the relevant calculation amount of coordinates in the process of updating the three-dimensional map.

[0185] Exemplary devices

[0186] Combined with the above Figures 1 to 6 , described in detail the method embodiment of the present application, and the device embodiment of the present application is described in detail below. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so that the parts not described in detail can be referred to the previous method embodiment.

[0187] The present application also provides a 3D map update system, including functional modules for implementing the 3D map update method provided in the present application. The 3D map update system can be integrated into the operating environment of an electronic device, thereby providing the electronic device with a 3D map update function.

[0188] For example Figure 7 As shown, Figure 7 3D map updating system 700 may include a data acquisition module 710 , a block determination module 720 , a state determination module 730 , and a map updating module 740 .

[0189] The data acquisition module 710 can be used to respond to the sensor data collected by the sensors on the mobile robot at the current moment, and determine the spatial point cloud data in the perception space based on the sensor data, wherein the perception space is configured as a three-dimensional space with the mobile robot as the origin.

[0190] The block determination module 720 may be configured to determine a plurality of target blocks from each three-dimensional block in the perception space of the mobile robot, wherein a fixed number of three-dimensional units in the perception space form a three-dimensional block.

[0191] The state determination module 730 may be configured to determine the perception status of each three-dimensional unit of a plurality of target blocks at a current moment based on the spatial point cloud data.

[0192] The map update module 740 can be used to determine the valid units detected by the sensor from the target block based on the perception situation, and synchronize the perception situation of the valid units to the three-dimensional map of the mobile robot based on the local coordinates of the valid units and the index relationship between the three-dimensional block and the three-dimensional unit, wherein the index relationship is determined based on the global coordinates of the three-dimensional unit.

[0193] The present application also provides an electronic device, such as Figure 8 The electronic device 800 provided in this application includes a memory 810, a processor 820, and an input / output interface 830. The memory 810, the processor 820, and the input / output interface 830 are connected via an internal connection path. The memory 810 is used to store instructions, and the processor 820 is used to execute the instructions stored in the memory 810 to control the input / output interface 830 to receive input data and information and output data such as operation results.

[0194] It should be understood that in the embodiment of the present application, the processor 820 can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiment of the present application.

[0195] The memory 810 may include a read-only memory and a random access memory, and provides instructions and data to the processor 820. A portion of the processor 820 may also include a non-volatile random access memory. For example, the processor 820 may also store information about the device type.

[0196] During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 820 or by software instructions. The three-dimensional map update method disclosed in conjunction with the embodiments of this application can be directly implemented as execution by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 810, and processor 820 reads the information in memory 810 and, in conjunction with its hardware, completes the steps of the above method.

[0197] In practical applications, the aforementioned electronic device can serve as a user terminal to implement the three-dimensional map updating method provided herein. For example, a user of the user terminal can interact with the electronic device to input the user's target timbre voice and source voice, thereby controlling the user terminal to replace the timbre of the source voice based on the user's timbre (i.e., the target timbre of the target timbre voice), thereby outputting the converted voice with the target timbre under the conditions of the source voice.

[0198] In some embodiments, the aforementioned electronic device may be presented based on actual needs of the 3D map update. For example, the aforementioned electronic device may include a mobile terminal (such as a mobile phone, tablet, notebook), a robot, etc.

[0199] The present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction processor in the computer program product provided by the present application is executed, the three-dimensional map updating method provided by the present application can be implemented.

[0200] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0201] The present application also provides a mobile robot, wherein the mobile robot may include a sensor for collecting sensory data of a perceived space, wherein the transmitted data is used to determine spatial point cloud data of the perceived space, at least one processor, and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the aforementioned method for updating a three-dimensional map (e.g., P100).

[0202] It should be understood that the term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0203] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0204] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0205] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0206] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0207] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, an operation and maintenance management (OAM), or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be read by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disk (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0208] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for updating a three-dimensional map, characterized in that: The method is applied to a mobile robot and comprises: In response to sensor data collected at a current moment by a sensor on the mobile robot, determining spatial point cloud data within a perception space based on the sensor data, wherein the perception space is configured as a three-dimensional space with the mobile robot as an origin; Determining a plurality of target blocks from respective three-dimensional blocks of a perception space of the mobile robot, wherein a fixed number of three-dimensional units in the perception space constitute the three-dimensional blocks, and the three-dimensional units are configured as basic units of the perception space; Determining, based on the spatial point cloud data, a perception status of the sensor on each three-dimensional unit of the plurality of target blocks at a current moment; Determining a valid unit detected by the sensor from the target block based on the perception situation, and synchronizing the perception situation of the valid unit into the three-dimensional map of the mobile robot based on the local coordinates of the valid unit and an index relationship between the three-dimensional block and the three-dimensional unit, wherein the index relationship is determined based on the global coordinates of the three-dimensional unit; The method further comprises: Based on the number of cells of the three-dimensional block in a reference direction; Determining each block in the perception space based on the number of units and generating a block list; Determine block index information of the three-dimensional unit based on the global coordinates of the three-dimensional unit and the number of units; Determining a block reference position of a block corresponding to the three-dimensional unit based on the block index information and the number of units, and determining a local coordinate of the three-dimensional unit in the corresponding block based on the block reference position; Based on the local coordinates of the three-dimensional cell in the corresponding area, local index information of the three-dimensional cell in the corresponding block is determined, wherein the local index information reflects the local coordinates of the three-dimensional cell in the corresponding block and is used to determine the global coordinates of the three-dimensional cell in combination with the block reference position, wherein each three-dimensional block in the block list includes the block index information of the three-dimensional block and the local index information of each three-dimensional cell contained in the three-dimensional block.

2. The method according to claim 1, characterized in that The perception status of the three-dimensional unit at the current moment includes the number of first optical fibers passing through the three-dimensional unit and the number of second optical fibers hitting the three-dimensional unit at the current moment. The determining, based on the spatial point cloud data, the perception status of the sensor on each of the three-dimensional units in the multiple target blocks at the current moment includes: generating a plurality of projection rays within the plurality of target blocks based on the spatial point cloud data of the plurality of target blocks, wherein the projection rays are released from the sensor and hit the spatial point cloud data within the plurality of target blocks; Determine the passing conditions of each three-dimensional unit through which the projection light passes through the multiple target blocks, and store them in the storage area of ​​the corresponding three-dimensional unit to determine the perception conditions of the three-dimensional unit, wherein the passing conditions of the projection light include each three-dimensional unit passed by the projection light and the three-dimensional unit hit.

3. The method according to claim 2, characterized in that The determining, based on the perception situation, a valid unit detected by the sensor from each three-dimensional unit of the target block, and synchronizing the perception situation of the valid unit into the three-dimensional map of the mobile robot based on the local coordinates of the valid unit in the target block and the index relationship between the three-dimensional block and the three-dimensional unit, includes: Traversing the storage area of ​​each three-dimensional block in the target block based on the local coordinates of the three-dimensional unit to determine the number of first rays and the number of second rays of each three-dimensional block; determining the valid unit and the local coordinates of the valid unit from the target block based on the first ray quantity, wherein the first ray quantity of the valid unit is a non-zero value; determining an occupation probability change value of the effective unit at a current moment based on the first light quantity and the second light quantity of the effective unit; The occupation probability change value is synchronized to a corresponding map element in a three-dimensional map of the mobile robot based on the local coordinates of the effective unit and the index relationship, and the occupation probability and occupation state of the map element are determined.

4. The method according to claim 3, characterized in that The step of synchronizing the occupancy probability change value to a corresponding map element in the three-dimensional map of the mobile robot based on the local coordinates of the valid unit and the index relationship, and determining the occupancy probability and occupancy state of the corresponding map element, includes: Determining an occupation probability of a corresponding map element of the valid unit before a current moment; updating the occupation probability based on the occupation probability change value of the effective unit, and determining the occupation probability of the effective unit at the current moment; Based on the local coordinates of the valid unit and the index relationship, the occupation probability of the valid unit at the current moment is synchronized to the corresponding map element, and the occupation state of the corresponding map element is determined.

5. The method according to claim 1, wherein The determining of a plurality of target blocks from each three-dimensional block of the perception space of the mobile robot includes: determining a working range of the mobile robot at a current moment based on the working state of the mobile robot; The plurality of target blocks are determined from the perception space based on the working range, wherein the plurality of target blocks completely cover the working range.

6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for updating a three-dimensional map according to any one of claims 1 to 5.

7. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction processor is executed, the method for updating a three-dimensional map according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the method for updating a three-dimensional map according to any one of claims 1 to 5 is implemented.

9. A mobile robot, characterized in that: The mobile robot comprises: a sensor for collecting sensory data of a perception space, wherein the sensory data is used to determine spatial point cloud data of the perception space; at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for updating a three-dimensional map according to any one of claims 1 to 5.

Images