Method for updating three-dimensional map, storage medium, electronic equipment, program product and robot

By collecting and processing sensing data on the robot, only the three-dimensional map areas that need to be changed are updated, which solves the problem of high computational calculations of three-dimensional maps in the prior art, and realizes rapid update and real-time perception of three-dimensional maps.

CN119085631AActive Publication Date: 2024-12-06AGIBOT INNOVATION (SHANGHAI) TECHNOLOGY CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202411190480.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-12-06
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

In the prior art, the update process of the robot's three-dimensional map is computationally expensive and it is difficult to achieve real-time rapid response, which affects the robot's rapid perception of the environment and job execution.

Method used

By collecting sensing data on the robot, the spatial point cloud data of the perceived space is determined and divided into a fixed number of three-dimensional blocks and units, only the target area that needs to be changed is updated, and the occupation in the three-dimensional map is synchronized using sensor data to update the occupancy in the three-dimensional map Status, avoid direct impact on the map and improve update efficiency.

Benefits of technology

It realizes rapid update of three-dimensional maps, reduces the amount of calculation, improves the robot's real-time perception of the environment, and enhances the efficiency of job execution.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119085631A_ABST
    Figure CN119085631A_ABST
Patent Text Reader

Abstract

The invention provides a method for updating a three-dimensional map, a storage medium, electronic equipment, a program product and a robot. Relates to the technical field of robots. According to the method, only the area needing to be updated at the current moment can be updated, so that the calculated amount of updating the three-dimensional map is reduced. Specifically, a target area needing to be updated can be determined firstly, then the occupancy state of each three-dimensional unit in the target area is updated based on sensor data, and therefore the three-dimensional units with the changed occupancy states in blocks of the target area are synchronized into a three-dimensional map. According to the method and the device, the processing of the sensor data can be executed in the space independent of the three-dimensional map, so that the three-dimensional map is prevented from being directly influenced. Meanwhile, in the space, the three-dimensional units form three-dimensional blocks in a fixed number, and the target area can be represented through the combination of the target blocks, so that the target area can be quickly determined, and the three-dimensional units needing to be changed can be quickly indexed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of robotics, and in particular to a method for updating a three-dimensional map, a storage medium, an electronic device, a program product, and a robot. Background Art

[0002] The robot's perception of its environment has always been an important part of the field of robot control, especially in the automatic operation of mobile robots. The automatic operation process of such robots often requires sensing the surrounding environment and executing corresponding control strategies (such as moving, controlling the operation of the robotic arm, etc.).

[0003] In the actual application of robots, the sensor information can be collected by various sensors installed on the robot and then the three-dimensional map can be updated based on the sensor information. However, in order to improve the accuracy of the three-dimensional map, the robot is often equipped with a variety of sensors. However, the processing of sensor data often requires a large amount of calculation, especially in the update of the three-dimensional map, which is not convenient for the robot to respond quickly to real-time data.

[0004] Therefore, how to improve the processing speed of three-dimensional maps is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] In view of this, embodiments of the present application provide a method, storage medium, electronic device, program product and robot for updating a three-dimensional map, which can perform partial updates on the three-dimensional map to speed up the update process of the three-dimensional map.

[0006] In a first aspect, the present application provides a method for updating a three-dimensional map, which is applied to a mobile robot, and includes: in response to sensor data collected at the current moment by a sensor on the mobile robot, determining spatial point cloud data in a perception space based on the sensor data, wherein the perception space is configured as a three-dimensional space with the mobile robot as the origin. Determine multiple target blocks from each three-dimensional block in the perception space of the mobile robot, wherein the three-dimensional units in the perception space form a three-dimensional block in a fixed number. Determine the perception status of each three-dimensional unit of the multiple target blocks at the current moment based on the spatial point cloud data. Determine the valid unit detected by the sensor from the target block based on the perception status, and synchronize the perception status of the valid unit to the three-dimensional map of the mobile robot based on the local coordinates of the valid unit and the index relationship between the three-dimensional block and the three-dimensional unit, wherein the index relationship is determined based on the global coordinates of the three-dimensional unit.

[0007] In a second aspect, the present application provides an electronic device. The electronic device includes: at least one processor and a memory connected to the at least one processor. The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the three-dimensional map update method described in the first aspect.

[0008] In a third aspect, the present application provides a computer program product, comprising a computer program / instruction, which, when executed by a computer program / instruction processor, implements the three-dimensional map updating method as described in the first aspect.

[0009] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program / instruction. When the computer program / instruction is executed by a processor, the three-dimensional map updating method described in the first aspect is implemented.

[0010] In a fifth aspect, the present application provides a mobile robot, comprising: a sensor for collecting sensor data of a perception space, wherein the transmitted data is used to determine spatial point cloud data of the perception space; at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for updating a three-dimensional map as described in the first aspect.

[0011] The present application provides a method, storage medium, electronic device, program product and robot for updating a three-dimensional map. It relates to the field of robotics. The present application can update only the area that needs to be updated at the current moment to reduce the amount of calculation for updating the three-dimensional map. Specifically, the target area that needs to be updated can be determined first, and then the occupancy status of each three-dimensional unit in the target area can be updated based on the sensor data, so that the three-dimensional units whose occupancy status has changed in the target area block can be synchronized to the three-dimensional map. Among them, the processing of sensor data in the present application can be performed in a space independent of the three-dimensional map to avoid directly affecting the three-dimensional map. At the same time, in this space, three-dimensional units are composed of a fixed number of three-dimensional blocks, and the aforementioned target area can be represented by a combination of target blocks to facilitate the rapid determination of the target area and the rapid indexing of the three-dimensional units that need to be changed. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 This is an exemplary flowchart of a three-dimensional map updating method provided in some embodiments of the present application.

[0014] Figure 2 It is an exemplary flow chart of a target block determination method provided in some embodiments of the present application.

[0015] Figure 3 It is a schematic diagram of a target block determination process provided by some embodiments of the present application.

[0016] Figure 4 This is an exemplary flowchart of an occupancy status updating method provided in some embodiments of the present application.

[0017] Figure 5 It is a schematic diagram of the effective unit determination process provided by some embodiments of the present application.

[0018] Figure 6 It is an exemplary flowchart of the index relationship determination method provided in some embodiments of the present application.

[0019] Figure 7 This is a system module diagram of a three-dimensional map updating system provided in some embodiments of the present application.

[0020] Figure 8 It is a schematic diagram of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0021] Application Overview

[0022] In related technologies, a three-dimensional map reflecting the robot's surrounding environment is often presented as a three-dimensional grid map. Among them, a three-dimensional grid map is a tool for representing and processing three-dimensional spatial data. It discretizes the three-dimensional world into three-dimensional grids and stores relevant information in each grid to achieve modeling and analysis of complex environments.

[0023] In practical applications, each grid in the three-dimensional grid map can be in two states: occupied or unoccupied, depending on whether there are obstacles in the corresponding area. Therefore, the three-dimensional grid map can reflect the distribution of obstacles in space, making it easier to perform path planning and operation planning based on the three-dimensional map.

[0024] The aforementioned three-dimensional grid Figure 1 Generally, it can be determined based on point cloud data. For example, point cloud data in space can be obtained by a device with 3D scanning, and then pre-processing such as filtering and denoising can be performed on the point cloud data. The processed point cloud data is then converted into raster data, that is, the continuous point cloud data is discretized into a 3D raster matrix.

[0025] It should be noted that, in practical applications, the aforementioned three-dimensional grid map can also be converted into a two-dimensional grid map according to actual needs. For example, during the navigation process of the robot, the aforementioned three-dimensional map can be filtered based on the robot's morphology to remove obstacles that will not affect the operation of the robot in the height direction, and then the aforementioned three-dimensional map is projected along the height direction to determine the two-dimensional grid map, and route planning or other operations are performed based on the two-dimensional grid map.

[0026] However, as the number of sensors integrated on the robot increases, the aforementioned point cloud data can be obtained by different types of sensors. When determining the grid map, it is often necessary to update the three-dimensional map based on each point cloud data, which requires a large amount of calculation. In particular, in practical applications, there may be moving obstacles. In order to respond to such obstacles, the three-dimensional map needs to be updated quickly, but the aforementioned amount of calculation is difficult to support the rapid update of the three-dimensional map.

[0027] The present application provides a method for updating a three-dimensional map, a storage medium, an electronic device, a program product and a robot, and relates to the field of robotics. The present application can update only the area that needs to be updated at the current moment to reduce the amount of calculation for updating the three-dimensional map. Specifically, the target area that needs to be updated can be determined first, and then the occupancy status of each three-dimensional unit in the target area can be updated based on the sensor data, so that the three-dimensional units whose occupancy status has changed in the target area block are synchronized to the three-dimensional map. Among them, the processing of sensor data in the present application can be performed in a space independent of the three-dimensional map to avoid directly affecting the three-dimensional map. At the same time, in this space, three-dimensional units are composed of a fixed number of three-dimensional blocks, and the aforementioned target area can be represented by a combination of target blocks to facilitate the rapid determination of the target area and the rapid indexing of the three-dimensional units that need to be changed.

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0030] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0032] Exemplary 3D map updating method

[0033] Figure 1 : is an exemplary flow chart of a three-dimensional map updating method (also referred to as a method for updating a three-dimensional map) provided in some embodiments of the present application. The three-dimensional map updating method provided in the embodiments of the present application can be implemented by a mobile robot. The mobile robot may refer to a robot with mobile capabilities, which is equipped with various types of sensors to obtain perception data (such as spatial point cloud data) of the scene in which it is located. The execution subject of each step will not be repeated in the following.

[0034] It should be noted that the three-dimensional map update in the present application is not limited to updating the data of the existing three-dimensional map. When there is no initial data in the three-dimensional map, it can also be initialized and constructed based on the method provided in the present application.

[0035] like Figure 1 As shown, P100 may include the following steps:

[0036] S110, in response to sensor data collected at a current moment by sensors on the mobile robot, determining spatial point cloud data within the perception space based on the sensor data.

[0037] S120, determining a plurality of target blocks from each three-dimensional block in the perception space of the mobile robot.

[0038] S130, determining the sensor's perception of each three-dimensional unit of the plurality of target blocks at the current moment based on the spatial point cloud data.

[0039] S140, determining the valid unit detected by the sensor from the target block based on the perception situation, and synchronizing the perception situation of the valid unit to the three-dimensional map of the mobile robot based on the local coordinates of the valid unit and the index relationship between the three-dimensional block and the three-dimensional unit.

[0040] In the aforementioned S110, the mobile robot is often equipped with multiple sensors to sense the current environment. Exemplarily, the sensors set on the mobile robot may include a laser radar and three depth cameras at different locations. Among them, two depth cameras may be set at the robot arms of the mobile robot, and the model may preferably be D405. One depth camera may be set at the head of the mobile robot, and the model may preferably be D415.

[0041] In some embodiments, the data collection cycles of the sensors installed on the mobile robot may be different, so as to achieve the fusion of sensor data of multiple sensors by responding to the time sequence of sensor data of different sensors. That is, when a sensor on the mobile robot collects sensor data, the aforementioned method P100 may be executed immediately to complete the update of the three-dimensional map before the next sensor collects data.

[0042] In the aforementioned S110, the perception space may be a three-dimensional map that is synchronized with the three-dimensional map and the user caches the map update status. The perception space may reflect the detection range of the sensor on the mobile robot. The mobile robot may be the origin of the perception space.

[0043] In some embodiments, the sensor data collected by the aforementioned sensors can be converted into spatial point cloud data in the perception space. The spatial point cloud data can refer to point cloud data presented in the perception space. Point cloud data refers to a set of vectors in a three-dimensional coordinate system, each point contains three-dimensional coordinates, and some may contain color information (such as RGB) or reflection intensity information (Intensity).

[0044] Specifically, point cloud data is usually obtained through devices such as laser scanners, depth cameras, binocular cameras, and optical cameras. Specifically, power data can be the points where the light released by the aforementioned sensor devices is reflected in space, thereby reflecting the three-dimensional spatial situation of the current space.

[0045] In some embodiments, in the aforementioned S110, the spatial point cloud data can be obtained by performing coordinate conversion based on the point cloud data of the sensor. That is, the point cloud data with the sensor as the origin can be first determined based on the sensor data of the sensor, and then the aforementioned point cloud data can be converted into point cloud data in the perception space (recorded as spatial point cloud data) based on the position of the sensor in the perception space.

[0046] In the aforementioned S120, the three-dimensional unit may refer to the basic unit of the perception space. Among them, the size of the three-dimensional unit in each reference direction may be 1 standard unit. Considering that the perception space is generally resolved as a space under a three-dimensional Cartesian coordinate system. Then the aforementioned three-dimensional unit can also be understood as a voxel in the three-dimensional space. Among them, the voxel can reflect a cube in the three-dimensional space, each side of which is 1 standard unit length. In addition, considering that the perception space is generally aligned with the three-dimensional map, the three-dimensional unit can also be used as the basic building block in the three-dimensional map.

[0047] In practical applications, the actual size of the three-dimensional unit can be determined according to the actual detection accuracy. For example, when the mobile robot is an embodied robot, the length of 1 standard unit in the perception space can be mapped to 1 cm in the physical space, that is, the three-dimensional unit is a cube with a side length of 1 cm.

[0048] In order to improve the update speed of the three-dimensional map, the present application may update only some three-dimensional units when updating the map. In order to quickly determine the three-dimensional units that need to be updated, the present application may treat the three-dimensional units of fixed size as a three-dimensional block based on a fixed size, thereby determining the three-dimensional units that need to be updated based on the three-dimensional block that needs to be updated (recorded as the target block).

[0049] That is, the aforementioned 3D block may be a combination of a fixed number of 3D units of a fixed size. In order to ensure the stability and accuracy of the index of the 3D block to the 3D unit, the 3D units contained in the 3D block are often continuous. For example, a block may include continuous 8×8×8 3D units.

[0050] The target block may refer to a three-dimensional block that needs to be updated at the current moment. That is, the target block may be a partial block in the perception space. In some embodiments, the target block may be determined according to actual needs. For example, the spatial range that needs to be updated may be determined in the perception area first, thereby determining the minimum three-dimensional block combination that includes the spatial range as multiple target blocks.

[0051] In some embodiments, in the present application, multiple update strategies may be configured during the process of updating the three-dimensional map. That is, multiple types of blocks with different update cycles may be pre-set, so that based on the correspondence between the current moment and the update cycle, the block type corresponding to the update cycle including the current moment is determined and used as the target block. Among them, the correspondence between the block type and the update cycle (or update frequency) may be determined according to actual update needs. For example, the update frequency for obstacles within the working range of the mobile robot may be 10 Hz, and the update frequency for the overall space may be 1 Hz. For another example, the update frequency for obstacles moving toward the mobile robot may be 10 Hz, and the update frequency for obstacles away from the mobile robot (i.e., obstacles that do not affect the movement of the robot) may be 1 Hz.

[0052] Therefore, when determining the target block, the sampling time including the target update strategy corresponding to the current time can be determined from multiple map update strategies based on the current time, wherein the update cycles and update areas of the multiple map update strategies are different. Then, multiple target blocks are determined from each three-dimensional block of the perception space based on the target update strategy.

[0053] In some embodiments, the target block at least includes the working range of the mobile robot in the perception space at the current moment, thereby updating the three-dimensional map around the mobile robot. The working range of the mobile robot can reflect the space required by the mobile robot to perform its work tasks at the current moment. Therefore, in updating the three-dimensional map, the spatial range required for the mobile robot to work can be updated quickly to quickly identify obstacles (especially dynamic obstacles) within the spatial range. For more information about the working range, please refer to Figure 2 , Figure 3 and its related description.

[0054] For the convenience of subsequent description, each 3D unit included in the aforementioned target block may be recorded as a target 3D unit. After the aforementioned target block is determined, the sensor sensing condition of each target 3D unit may be determined, that is, the aforementioned S130 is executed.

[0055] In the aforementioned S130, the perception situation can reflect the sensor's perception of the target three-dimensional unit at the current moment. In practical applications, the perception situation can reflect at least two levels of situations. That is, the perception situation can reflect whether each target three-dimensional unit is detected by the sensor and whether the target unit is occupied by an obstacle when it is detected by the sensor.

[0056] Specifically, it is considered that the sensors on the mobile robot can perceive the external environment based on optical principles. For example, radar sensors generally perceive the surrounding conditions by releasing invisible light beams, and camera sensors generally perceive the surrounding conditions directly by collecting light from the surrounding environment. When the light between the sensor and a specific position is blocked, the sensor cannot perceive the situation at that specific position. When there is no obstacle blocking the transmission of light between the sensor and the specific position, it can be determined whether there is an obstacle at that specific position based on whether the light from the sensor passes through the specific position (or stops at the specific position).

[0057] Among them, the aforementioned specific position can reflect each three-dimensional unit in the perception space, and the relationship between the light and the specific position can specifically reflect the light passing through the three-dimensional unit and the light hitting the three-dimensional unit. Among them, the light passing through the three-dimensional unit can be understood as the light at least entering the three-dimensional unit, and the light hitting the three-dimensional unit can reflect that the light hits the point cloud (or the contour formed by it) in the three-dimensional unit. It should be noted that the light passing through the three-dimensional unit does not conflict with the light hitting the three-dimensional unit. When the light hits the three-dimensional unit, the light also enters the three-dimensional unit.

[0058] Therefore, when updating the three-dimensional map, the light released by the sensor can be simulated in the space formed by multiple target blocks, so as to determine the perception of each three-dimensional unit according to the three-dimensional unit through which the light passes. Specifically, multiple projection light rays can be generated in multiple target blocks based on the spatial point cloud data of multiple target blocks. Then, the passing conditions of each three-dimensional unit through which the projection light passes in the multiple target blocks are determined, and stored in the storage area of ​​the corresponding three-dimensional unit to determine the perception of the three-dimensional unit. Among them, the multiple target blocks include multiple target blocks, the projection light is released from the sensor and hits the spatial point cloud data, and the perception situation includes at least the first light quantity and the second light quantity.

[0059] Specifically, in the above process, two counters can be configured in the storage area of ​​each three-dimensional unit to record the number of light rays passing through the three-dimensional unit (recorded as the first light ray number) and the number of light rays hitting the three-dimensional unit (recorded as the second light ray number). In the above process, when it is determined that the projected light passes through and hits the three-dimensional unit, a corresponding counting instruction can be generated to count in the corresponding counter. Among them, the number of first light rays of each three-dimensional unit passed by the projected light +1, the number of first light rays of the three-dimensional unit hit +1, and the number of second light rays +1.

[0060] In some embodiments, the aforementioned process can be determined based on a raycast algorithm. The release position of the light is the position of the sensor, and the release range is the space formed by multiple target blocks. Exemplarily, a raycast algorithm can be used to determine the light and the three-dimensional unit through which the light passes by selecting a starting point and an end point and continuously iterating.

[0061] In the aforementioned S140, the valid unit may refer to the three-dimensional unit sensed by the sensor, that is, based on the relevant description of the aforementioned S130, each three-dimensional unit in the aforementioned target block may have a part that is not sensed by the sensor. In order to distinguish the two, when the corresponding three-dimensional unit is detected, specific data (such as the aforementioned first light quantity and second light quantity) can be written in its sensing situation to characterize the valid unit, and the unit without data written can be understood as an invalid unit.

[0062] The first light quantity and the second light quantity can be re-determined each time detection is performed and reset after data update. That is, when the first light quantity at the current moment is 0 or empty, the corresponding three-dimensional unit is not detected by the sensor at the current moment and is an invalid unit.

[0063] Taking into account that the aforementioned perception situation is determined based on the space formed by the target block, the data can be written directly in the storage area of ​​the corresponding three-dimensional unit when determining the perception situation. After the writing is completed, in order to determine the valid unit, it is necessary to traverse each three-dimensional unit in the target area one by one. In order to reduce the difficulty of processing, the traversal can be performed through the local coordinates / local index information of the three-dimensional unit in the target block. That is, when determining the aforementioned valid unit, the storage area of ​​each three-dimensional unit in the target block can be traversed based on the local coordinates of the three-dimensional unit to determine the first light number and the second light number of each three-dimensional block. Then, based on the first light number, the valid unit and the local coordinates of the valid unit are determined from the target block, wherein the first light number of the valid unit is a non-zero value.

[0064] In some embodiments, considering that when the aforementioned light is projected, each three-dimensional unit perceived is passed by the light, then in fact (the second light quantity) can reflect the situation where the projected light ends at the three-dimensional unit (there may be obstacles here), and (the first light quantity - the second light quantity) can reflect the situation where the projected light penetrates the three-dimensional unit (that is, there may be no obstacles here).

[0065] Thus, when (number of second rays)>(number of first rays-number of second rays), it can be considered that the three-dimensional unit is more likely to have an obstacle, and when (number of second rays)<(number of first rays-number of second rays), it can be considered that the three-dimensional unit is more likely to have no obstacle. The possibility of the three-dimensional unit having an obstacle can be recorded as the occupancy probability, and the result of determining whether the three-dimensional unit is an obstacle in the map based on the occupancy probability can be recorded as the occupancy state.

[0066] It should be noted that the aforementioned counting of the number of first rays is only an example provided in this application. In an alternative embodiment, when projecting rays, the three-dimensional units hit by the rays can also be superimposed only on the number of second rays, rather than on the number of first rays. Then the number of first rays at this time is equivalent to the aforementioned number of first rays minus the number of second rays), so that adaptive modification can be performed.

[0067] Based on the above description, the occupation probability change value of the effective unit at the current moment can be determined based on the first light number and the second light number of the effective unit, and then the occupation probability change value can be synchronized to the corresponding map element in the three-dimensional map of the mobile robot based on the local coordinates and index relationship of the effective unit to determine the occupation probability and occupation status of the map element.

[0068] Based on the above, considering that the aforementioned traversal process is generally based on the local coordinate traversal of the three-dimensional unit in the three-dimensional block, its global coordinates are not known after determining its actual probability change value. Therefore, when updating the three-dimensional map, the global coordinates of the valid unit in the perception space can be determined based on the local coordinates of the valid unit and the index relationship between the three-dimensional block and the three-dimensional unit, and then the corresponding data can be synchronized to the corresponding map element (i.e., the three-dimensional unit in the three-dimensional map) based on the mapping relationship between the perception space and the three-dimensional map.

[0069] In some embodiments, the index relationship between the aforementioned three-dimensional block and the three-dimensional unit can be determined based on the global coordinates of the three-dimensional unit and the fixed size of the three-dimensional area. That is, the corresponding relationship between the block and the three-dimensional unit can be pre-configured before executing the aforementioned P100. Specifically, the following steps can be included:

[0070] First, the number of cells of the three-dimensional block in the reference direction is determined.

[0071] Next, each block in the perception space is determined based on the number of units, and a block list is generated.

[0072] Next, the block index information of the 3D cell is determined based on the global coordinates of the 3D cell and the cell quantity.

[0073] Next, a block reference position of the 3D unit corresponding to the block is determined based on the block index information and the number of units, and a local coordinate of the 3D unit in the corresponding block is determined based on the block reference position.

[0074] Finally, local index information of the three-dimensional unit in the corresponding block is determined based on the local coordinates of the three-dimensional unit in the corresponding area.

[0075] Exemplarily, the size of the block can be determined based on the size of the perception space, thereby dividing each block. Then, based on the division result, a block form (such as a hash table) containing each three-dimensional block in the perception space is constructed. Then, based on the size of the block, the three-dimensional units contained in the block are determined, so that the correspondence between the block and the three-dimensional unit is written into the aforementioned block form, so that the corresponding three-dimensional unit can be indexed based on the block form. Among them, each three-dimensional unit can also form a three-dimensional unit form, so as to determine the corresponding three-dimensional block based on the address of each three-dimensional unit in the form.

[0076] In some embodiments, considering that the perception space takes the mobile robot as the origin, the correspondence between the aforementioned blocks and the three-dimensional unit can be directly determined based on a coordinate system with the three-dimensional unit as the basic unit (such as the coordinate system in the aforementioned perception space). For example, a three-dimensional block can also be identified based on multiple reference directions. At this time, the basic unit of the coordinates of the three-dimensional block can be understood as a three-dimensional block. For example, a three-dimensional block with coordinates (1,1,1) represents the first block in the x-axis direction, the first block in the y-axis direction, and the first block in the z-axis direction. The coordinates of the three-dimensional block in this space can be recorded as block index information.

[0077] The block index information can be used to index the coordinate range of the corresponding block in the perception space. For example, for the three-dimensional block with the block index information (1,1,1), the number of units in each direction is 8, and the coordinate range in the perception space is (8,8,8) to (15,15,15). Among them, the coordinate with the smallest absolute value of the three-dimensional block in the perception space can be recorded as the block reference position. The block reference position can be directly determined by using the block index information * the number of units in each reference direction.

[0078] For example, for a three-dimensional unit of (x, y, z), the identification information of the corresponding block may be ([x / n x ],[y / n y ],[z / n z ]), where n x , n y , n z are the number of three-dimensional units contained in the block in the X, Y, and Z directions, respectively. For example, in the aforementioned block n x , n y , nz They can all be 8, and the symbol [] can reflect downward evidence collection.

[0079] As an alternative embodiment, the correspondence between the aforementioned three-dimensional unit and the block can also be determined similarly based on bit operations. For details, see Figure 6 And the related descriptions are not repeated here.

[0080] After determining the aforementioned block index information, the local coordinates of each three-dimensional unit in the three-dimensional block can be determined. Among them, local coordinates = global coordinates - block reference position. After determining the local coordinates, the local coordinates can be vector encoded and converted into a series for easy storage. For example, for the three-dimensional block whose aforementioned block index information is (1,1,1), the local coordinates of the three-dimensional unit (9,15,8) therein can be (1,7,0), and its local index information can be 170 (it can also be converted to two for storage). Therefore, in practical applications, the aforementioned local index information can reflect the local coordinates of the three-dimensional unit in the corresponding block and be used to determine the global coordinates of the three-dimensional unit in combination with the block reference position.

[0081] To reflect the correspondence between three-dimensional units and three-dimensional coordinates, the aforementioned block list may include block index information of the three-dimensional block and local index information of each three-dimensional unit contained in the three-dimensional block, wherein each three-dimensional block can be reflected through the block index information, and then each three-dimensional unit within the three-dimensional block can be reflected through the aforementioned block index information.

[0082] In some embodiments, in executing the aforementioned S120, after determining the target block, all blocks except the target block in the aforementioned block list may be deleted based on the target block, thereby directly calling each 3D unit as the target 3D unit based on the deleted block list.

[0083] In the aforementioned S140, the occupation state can reflect whether the three-dimensional unit is occupied by an obstacle. The occupation state can generally include an idle state and an occupied state. When the three-dimensional unit is in the idle state, it can be reflected that the three-dimensional unit is not occupied by an obstacle, and when the three-dimensional unit is in the occupied state, it can be reflected that the three-dimensional unit is occupied by an obstacle.

[0084] In some embodiments, the aforementioned S140 can be performed based on the distribution of the spatial point cloud data in each target three-dimensional unit. For example, when the target three-dimensional space contains spatial point cloud data, the target three-dimensional unit can be identified as occupied, otherwise it is idle, thereby determining the occupied state.

[0085] In some embodiments, considering that the aforementioned occupancy state may gradually change during the movement of the mobile robot, the occupancy probability of the target three-dimensional unit may be updated at each moment to reflect the corresponding occupancy state. If the updated occupancy probability is greater than a threshold, it can be considered to be in an occupied state, otherwise it is in an idle state.

[0086] Therefore, based on the method of updating the three-dimensional map shown in the aforementioned P100, in order to quickly determine the three-dimensional map around the robot, the present application can only update the area that needs to be updated at the current moment to reduce the amount of calculation for updating the three-dimensional map. Specifically, the target area that needs to be updated can be determined first, and then the occupancy status of each three-dimensional unit in the target area can be updated based on the sensor data, so that the three-dimensional units whose occupancy status has changed in the target area block are synchronized to the three-dimensional map. Among them, the processing of sensor data in the present application can be performed in a space independent of the three-dimensional map to avoid directly affecting the three-dimensional map. At the same time, in this space, three-dimensional units are composed of a fixed number of three-dimensional blocks, and the aforementioned target area can be represented by a combination of target blocks to facilitate the rapid determination of the target area and the rapid indexing of the three-dimensional units that need to be changed.

[0087] In some embodiments, the difference between the sensor sensing data and the perceived space is taken into account in the aforementioned S110. To further describe the process of determining the spatial point cloud data, the following takes a depth camera as an example to illustrate the aforementioned process of obtaining the spatial point cloud data.

[0088] First, the sensing data of the sensor at the current moment may be obtained, wherein the sensing data of the depth camera may be based on the depth image.

[0089] Specifically, the depth image of the depth camera can be collected through a specific data interface (ros2 topic), and the depth image of the depth camera at each moment can be collected through the data interface to determine the depth image at the current moment. Among them, the depth image can include each pixel in the perception space and its depth information.

[0090] Next, the sensing data is mapped into the perception space to serve as the initial point cloud data in the perception space. When determining the initial point cloud data, the point cloud data may be determined first and then mapped.

[0091] The three-dimensional coordinates of each pixel can be calculated first: based on the camera's internal parameters (focal length, principal point, etc.) and the two-dimensional coordinates of the pixel (u, v) and the corresponding depth value d, the three-dimensional coordinates (x, y, z) of the pixel in the camera coordinate system can be calculated. The specific calculation formula is as follows:

[0092]

[0093] Among them, x, y, z are the coordinates of the camera coordinate system, (u, v) are the pixel coordinates in the depth image, d is the depth value of the pixel (usually in meters), and cx, cy, fx, fy are the camera intrinsic parameters.

[0094] Then, based on the coordinate transformation of the mobile robot, the point cloud data in the aforementioned camera coordinate system can be converted to the coordinate system of the perception space to serve as initial point cloud data.

[0095] Specifically, the point cloud data in the camera coordinate system can be converted to the coordinate system of the perception space based on the joint where the depth camera is located and the joint coordinate transformation information (Transform, tf) of each joint in the mobile robot. In the mobile robot (especially the embodied robot), each joint corresponds to a tf, and these tfs are used to describe the relative position and direction between the joints. Thus, the point cloud data can be mapped to the coordinate system of the perception space based on the joint where the depth camera is located.

[0096] Exemplarily, for the aforementioned depth camera set at the robotic arm, the transformation matrix of the depth camera at the bottom center point of the mobile robot (or a position that can be used as the origin) can be determined based on the joint coordinate transformation information of the robotic arm, thereby determining the initial point cloud data.

[0097] Finally, the initial point cloud data is filtered based on the three-dimensional unit to determine the spatial point cloud data.

[0098] After determining the aforementioned initial point cloud data, filtering can be performed based on the aforementioned three-dimensional unit (i.e., a voxel with a side length of 1 cm) to reduce the density of the point cloud and generate a downsampled point cloud. Among them, after filtering, a three-dimensional unit contains at most one point cloud. Specifically, the center of gravity of each three-dimensional unit can be selected as a representative point, thereby achieving downsampling of the initial point cloud data to significantly reduce the amount of point cloud data while retaining the basic shape characteristics of the original point cloud.

[0099] It should be noted that some of the descriptions in the above process may be reversed in order. For example, after determining the point cloud data in the sensor coordinate system, filtering processing may be performed first and then coordinate conversion may be performed to reduce the amount of data to be processed.

[0100] Exemplary target block determination method

[0101] To further illustrate the process of determining the target block, the present application also provides an exemplary flow chart of a method for determining a target block ( Figure 2 ) and schematic diagram ( Figure 3 ).

[0102] like Figure 2 As shown, process P200 may include the following steps:

[0103] S210: Determine the working range of the mobile robot at the current moment based on the working state of the mobile robot.

[0104] S220, determining a plurality of target blocks from the perception space based on the working range.

[0105] In the aforementioned S210, the working state of the mobile robot can reflect the current working condition of the mobile robot. In actual work, the working state can at least reflect the current working task of the mobile robot. The working task can refer to the work that the mobile robot currently needs to complete.

[0106] The working range can reflect the range of activities of the mobile robot in performing the current task. For example, when the mobile robot needs to use a mechanical arm to perform work, its working range can reflect the maximum range of activities of the mechanical arm in space.

[0107] In some embodiments, in order to ensure the normal operation of the mobile robot, the aforementioned working range can also expand the activity range for performing the current working task based on a preset ratio, so as to maintain a certain perception redundancy to avoid the occurrence of emergencies.

[0108] In some embodiments, considering that the operation task of the mobile robot may involve a motion process, the aforementioned S210 may include the following sub-steps:

[0109] S211. Determine the movement status and operation task of the mobile robot at the current moment based on the working status of the mobile robot.

[0110] S212: Determine the operating range of the mobile robot when performing the operating task based on the operating task.

[0111] S213: Determine the working range based on the movement situation and the working range.

[0112] In the aforementioned S211, the motion situation can be determined based on the motion requirements, historical motion trajectory and current motion state of the mobile robot, and is mainly used to describe the directional motion trajectory of the mobile robot from the current moment to the next moment.

[0113] In the aforementioned S212, the operating range may refer to the space required to perform the operating task. Among them, considering the movement process of the mobile robot, the operating range may include the operating range at the current moment and the operating range at the next moment. For example, the mobile robot is in a state of moving to the destination at the current moment, and its robotic arm is generally not used, then its operating range is basically the size of its body with some movement redundancy added. If the mobile robot arrives at the destination at the next moment and starts working, its operating range can be determined based on its actual operating task (such as the maximum range of motion of the aforementioned robotic arm in space).

[0114] In some embodiments, the working range can be determined based on the aforementioned motion conditions and the operating range. For example, the position of the mobile robot at the next moment can be determined based on the motion conditions, and the operating range at the next moment can be determined based on its position at the next moment, so as to determine the minimum continuous area that can include the two areas based on the operating range at the current moment and the operating range at the next moment and serve as the working range.

[0115] In some embodiments, path fusion technology may also be used to first determine the directional motion trajectory of the mobile robot between the current moment and the next moment based on the motion situation, and then perform path fusion on the operating range based on the directional motion trajectory to determine the working range. Path fusion can be understood as an image algorithm generated by superimposing images along a path.

[0116] To further illustrate the above process of determining the scope of work, see Figure 3 Among them, considering that three-dimensional images are difficult to identify, Figure 3 It is a schematic diagram of the target block determination process in the XY plane.

[0117] like Figure 3 As shown, Figure 3 The perception space 300 is represented by a mobile robot 310 as the origin and a block as the basic unit. Figure 3 It can be seen from the partial enlarged view A that a block can include 8*8 three-dimensional units in the XY plane. In practice, considering the Z axis, a block can include 8*8*8=512 three-dimensional units.

[0118] like Figure 3 As shown, in the process of determining the working range, the working range 311 of the mobile robot 310 at the current moment and the directional movement path 312 between the current moment and the next moment can be determined based on the work task. When path fusion is adopted, the working range 311 at the current moment can be mapped to the working range 313 at the next moment based on the directional movement path 312, and a closed area 314 can be formed based on the working range 311 and the working range 313. At this time, the working range can be determined based on the closed area 314, which can be the minimum continuous block combination that can cover the area. In some embodiments, in order to facilitate subsequent processing steps, the working range can be configured as the minimum rectangular range that can cover the closed area 314. Figure 3 In some embodiments, considering that the target block may be selected in the subsequent step of determining the target block, the aforementioned working range may also be configured as the aforementioned closed area 314.

[0119] In addition, the size of the working range 313 at the next moment may be different from the working range 311 at the current moment. In the current moment, the mobile robot 310 may be in the process of moving, and in the next moment, the mobile robot 310 may have completed the movement and is in the working state.

[0120] In the aforementioned S220, after determining the working range, a combination of continuous blocks that can completely cover the working range can be directly determined based on the working range as multiple target blocks. In some embodiments, the target block can be presented in a specific shape (such as the aforementioned rectangle) to facilitate the subsequent determination of the occupancy state.

[0121] In some embodiments, considering that there may be dynamic obstacles moving toward the mobile robot in actual applications, in order to improve the ability to avoid dynamic obstacles, the aforementioned S220 may further include the following sub-steps:

[0122] S221. Determine at least one dynamic obstacle from the perception space.

[0123] S222: Determine a motion area of ​​at least one dynamic obstacle.

[0124] S223, determining a plurality of target blocks from the perception space based on the motion area and the working range.

[0125] In the aforementioned S221, the dynamic obstacle may refer to an obstacle that moves relative to the mobile robot. For example, the dynamic obstacle may be an obstacle that can move autonomously (such as a person) and a relatively predetermined fixed obstacle that is generated due to the movement of the mobile robot. In particular, when determining the dynamic obstacle, obstacles that may affect the mobile robot may be determined. For example, the dynamic obstacle moves toward the mobile robot / is located on the movement path of the mobile robot.

[0126] In some embodiments, the aforementioned S221 can be determined based on the depth image of the depth camera. For example, the dynamic obstacle can be determined based on a plurality of continuous depth images that do not contain depth information, based on a target recognition algorithm and a tracking algorithm. The dynamic obstacle is then mapped to the perception space based on the depth information of the corresponding point cloud.

[0127] In the aforementioned S222, the motion area of ​​the dynamic obstacle can reflect the range of activity of the dynamic obstacle from the current moment to the next moment or a subsequent moment. Specifically, it can be determined based on its historical motion trajectory. Similar to the aforementioned working range, the motion area can also be characterized as a rectangular range in the perception space.

[0128] In the aforementioned S223, when determining the target block, the target block can be controlled to cover the union of the motion area and the working range. For example, the minimum rectangle covering the motion area and the working range can be determined from the perception space, and each block contained in the rectangle is used as the target block.

[0129] See also Figure 3 ,like Figure 3 As shown, the perception space 300 may also include a dynamic obstacle 320 related to the mobile robot 310 and a motion area 321 between the current moment and the next moment. Based on the motion area 321 and the aforementioned working range (the rectangular range with blue edges or the closed area 314), the minimum rectangular area containing the two areas is determined as the target block (presented as a rectangular range with red edges).

[0130] In some embodiments, after the aforementioned range is determined, a copy of the block list may be constructed and other blocks except the target block in the block list may be removed, thereby determining the target 3D unit based on the target block.

[0131] Exemplary occupancy status update method

[0132] To further illustrate the process of updating the occupancy status, the present application also provides an exemplary flow chart of an occupancy status updating method ( Figure 4 ).

[0133] like Figure 4 As shown, process P400 may include the following steps:

[0134] S410 , determining valid units and the number of first rays and the number of second rays of the valid units from a plurality of target blocks.

[0135] S420: Determine an occupation probability change value of the effective unit at the current moment based on the first light quantity and the second light quantity of the effective unit.

[0136] S430: Determine the occupation probability of the corresponding map element of the valid unit before the current moment.

[0137] S440: Update the occupation probability based on the occupation probability change value of the effective unit to determine the occupation probability of the effective unit at the current moment.

[0138] S450: Synchronize the occupation probability of the valid unit at the current moment to the corresponding map element based on the local coordinates and index relationship of the valid unit, and determine the occupation state of the corresponding map element.

[0139] For the aforementioned S410, please refer to the relevant description of P100.

[0140] In the aforementioned S420, considering that the occupancy probability change value can reflect whether the corresponding three-dimensional unit at the current moment is more inclined to be judged as having no obstacle or having an obstacle, the occupancy probability change value can be determined according to the first light quantity and the second light quantity.

[0141] Specifically, the value of the occupation probability change value (recorded as log_odds_update) of this update can be calculated according to the relationship between the second number of rays (recorded as num_hit) and the first number of rays (recorded as num_pass). If num_hit is equal to (num_pass-num_hit), log_odds_update is taken as the first change value (recorded as prob_hit_log); otherwise, it is taken as the second change value (recorded as prob_free_log). Then num_hit and num_pass are reset to 0 to wait for the next update.

[0142] The above process can be expressed as the following formula:

[0143]

[0144] In some embodiments, when determining the occupancy status, the occupancy probability of the corresponding map element may be directly updated based on the aforementioned occupancy probability change value, thereby determining an updated occupancy probability.

[0145] In the aforementioned S430, in actual application, the mobile robot can move in the three-dimensional map, so as to align the three-dimensional map with the perception space at the current moment. Among them, the alignment of the three-dimensional map and the perception space does not require that the two are based on the same coordinate system, but only requires providing a conversion relationship between the two. In application, the map elements corresponding to each effective unit can be determined based on the conversion relationship.

[0146] In some embodiments, considering that the valid unit in the perception space may not have been detected, that is, the map element corresponding to the valid unit may not be configured with the occupation probability and occupation status. In this step, the map element that is not configured with the occupation probability and occupation status can be initialized. That is, based on the positive and negative values ​​of the aforementioned occupation probability change value, the initial occupation probability (which can be set to 0.7) and the initial idle probability (which can be set to 0.35) can be configured respectively. After configuring the initial occupation probability, the update of S430 can be omitted, so that synchronization can be performed directly through S440.

[0147] When determining the map element corresponding to the valid unit, the occupancy probability stored in the map element can be called. Considering that the map element has not been updated, the occupancy probability can be the occupancy probability at the last moment or the previous moment (denoted as P -1 (occupancy)).

[0148] In the actual calculation process, the value of the occupancy probability is generally positively correlated with the possibility of the existence of an obstacle. That is, the larger the value of the occupancy probability, the more likely it is that there is an obstacle at the location. Therefore, the first change value of the aforementioned occupancy probability change value can be a positive value to reflect a positive change. Conversely, the second change value can be a negative value to reflect a negative change.

[0149] In some embodiments, considering that the occupancy probability may exceed the limit (such as greater than 1) during the iteration process, upper and lower limits can be set, so that when the updated result is greater than the corresponding upper and lower limits during the update, only the corresponding upper and lower limits are used.

[0150] Therefore, we can first determine the relationship between the occupancy probability change value (log_odds_update value) and the occupancy probability at the previous moment:

[0151] If log_odds_update is greater than or equal to 0 and P -1 (occupancy) is greater than or equal to the maximum occupancy probability (referred to as max_log), or log_odds_update is less than or equal to 0 and P -1 If (occupancy) is less than or equal to the minimum occupancy probability (denoted as min_log), it is returned directly without updating.

[0152] Otherwise, based on P -1 (occupancy) determines the occupancy probability at the current time (denoted as P 0 (occupancy)), where P 0 (occupancy) can be configured as P -1 (occupancy) + log_odds_update, and ensure that it is between min_log_ and max_log_, which can be specifically expressed as follows:

[0153] P 0 (occupancy) = min[max(P ―1 (occupancy)+log_odds_update,min_log),max_log].

[0154] Among them, prob_hit_log and prob_free_log represent the logarithmic probabilities of the idle state and the occupied state, and max_log and min_log define the logarithms of the maximum and minimum values ​​of the occupied probability. Among them, prob_hit_log, prob_free_log, max_log, and min_log can be empirical values.

[0155] Thus, after the occupancy probability is determined, the corresponding map element can be updated based on the occupancy probability, thereby determining the occupancy state by combining the threshold.

[0156] The specific algorithms / values ​​of each value in the above description are as follows:

[0157] prob_hit_log=log(p_hit / 1-p_hit). prob_free_log=log(p_miss / 1-p_miss). min_log=log(p_min / 1-p_min). max_log=log(p_max / 1-p_max).

[0158] p hit =0.7. miss =0.35. min =0.12. max =0.97. 0 The initial value of (occupancy) is set to 0.

[0159] In some embodiments, in order to further illustrate the detailed process of projecting light, the present application also provides a schematic diagram of a valid unit determination process.

[0160] like Figure 5 As shown, for the aforementioned Figure 3 The red target block is different from Figure 3 ,exist Figure 5 Each block in the figure can represent each three-dimensional unit. Figure 5 The 510 in the figure can reflect the sensor currently performing ray casting.

[0161] exist Figure 5 In , light can be projected according to the position of the sensor, where each light can be blocked by the point cloud inside and outside the area, thus forming a light line segment between the sensor and each position. Figure 5 By casting rays, each three-dimensional unit (indicated by a blue frame in the figure) passing through the ray can be regarded as a valid unit, and then subsequent synchronization is performed based on the number of first rays and the number of second rays.

[0162] In addition, during actual light projection, the projection space may be larger than the space formed by the target block, so as to improve the projection effect on the three-dimensional unit at the edge of the space.

[0163] It should be noted that, in actual applications, the determination of the perception status of the three-dimensional unit in the perception space and the update of the three-dimensional map can also be configured as two independent processes, and some steps in the aforementioned process can also be performed in the three-dimensional map. For example, only the occupancy probability change value of the three-dimensional unit can be synchronized to the corresponding map unit, and the update of the occupancy probability and occupancy status in the map unit can be performed by the relevant process of the three-dimensional map.

[0164] Exemplary index relationship determination method

[0165] To further illustrate the index relationship between the 3D block and the 3D unit, the present application also provides an index relationship determination method based on bit operation ( Figure 6 ).like Figure 6 As shown, process P600 may include the following steps:

[0166] S610 , determining the number of cells of the three-dimensional block in a reference direction, and determining the bit length of the number of cells based on the number of cells.

[0167] S620 , performing right shift processing on the global coordinates of the three-dimensional unit based on the bit length, and determining the corresponding block of the three-dimensional unit and the block index information of the corresponding block.

[0168] S630 , determining the local coordinates of each three-dimensional unit in the three-dimensional block based on the block reference position of the three-dimensional block.

[0169] S640: Determine the stacking order of each reference direction, and perform vector encoding on the local coordinates of the three-dimensional unit based on the stacking order to determine the local index information of the three-dimensional unit in the corresponding block.

[0170] S650: Generate a block list based on the local index information of the three-dimensional unit in the corresponding block and the block index information.

[0171] In the aforementioned S610, for the convenience of bit operations, the number of cells is 2 to the power of n, and the bit length reflects the number of cells to the power of n based on the exponent 2. For example, a 3D block may have 2^3=8 3D cells in each direction, then the aforementioned number of cells is 8, and the corresponding bit length may be 3.

[0172] Specifically, the value range of each target three-dimensional unit in the local coordinates can be (0, 0, 0) to ((block_num - 1), (block_num - 1), (block_num - 1)). Among them, block_num can be the binary representation of the aforementioned unit quantity, and block_num - 1 can reflect the maximum value of the local coordinates in that direction. At the computational level, block_num = 1 << block_bit. Exemplarily, when the aforementioned unit quantity is 3, then the aforementioned block_num is the binary "1000" to represent the quantity 8, and (block_num - 1) is the binary "111" to represent the maximum value of the block coordinates.

[0173] Thus, in the aforementioned S620, for the global coordinate vox_index1 = [int x, int y, int z] of any three-dimensional unit, its corresponding block index information can be block_index = [x >> block_bit, y >> block_bit, z >> block_bit]block_index = [x >> block_bit, y >> block_bit, z >> block_bit]. Similar to the spatial coordinates, the block index information can be characterized as three-dimensional coordinates in space, and each coordinate can reflect the corresponding sequence number.

[0174] Considering that in the block, the value of the local coordinates is 0 to (block_num - 1), for the corresponding local coordinates [x, y, z], its global coordinate can be characterized as vox_index1 = [x, y, z] + block_index * block_num.

[0175] In addition, the range of three-dimensional units included in the block can also be characterized as: block_index * block_num to [block_num - 1, block_num - 1, block_num - 1] + block_index * block_num.

[0176] In some embodiments, considering that the local coordinates also need to be characterized as three arrays during storage, in order to simplify the storage logic, its vector coordinates can be encoded and converted into a numerical value (denoted as local index information). Specifically, the values of the three coordinate axes can be directly arranged in sequence.

[0177] Exemplarily, for the local coordinate vox_index2 = [x, y, z], its local index information can be characterized as int vox_index_in_block = (x & (block_num - 1)) << block_bit << block_bit + (y & (block_num - 1)) << block_bit + (z & (block_num - 1)). That is, the local index information is the coordinates of the xyz axes arranged in sequence.

[0178] In addition, the aforementioned local index information can be directly converted into global coordinates, so as to execute the aforementioned step S140. Based on the local index information vox_index_in_block where the coordinates of the aforementioned xyz axes are arranged in sequence, its corresponding global coordinate vox_index1 can be:

[0179] z = vox_index_in_block & (block_num - 1).

[0180] y = (vox_index_in_block >> block_bit) & (block_num - 1).

[0181] x = (vox_index_in_block >> block_bit >> block_bit) & (block_num - 1).

[0182] vox_index1 = [x, y, z] + block_index * block_num.

[0183] Based on the foregoing process, each block index information and the local index information of the three-dimensional units of each block can be recorded in the block list.

[0184] Thus, when constructing and applying the index relationship based on the foregoing method, bit operations can be used, avoiding the computational complexity of positive number multiplication and division, and reducing the computational complexity related to coordinates in the process of updating the three-dimensional map.

[0185] Exemplary Devices

[0186] As described above in conjunction with Figures 1 to 6 , the method embodiments of the present application have been described in detail. Next, the device embodiments of the present application will be described in detail. It should be understood that the descriptions of the method embodiments and the device embodiments correspond to each other. Therefore, the parts not described in detail can be referred to the previous method embodiments.

[0187] The present application also provides a 3D map updating system, including a functional module for implementing the 3D map updating method provided by the present application. The 3D map updating system can be integrated into the operating environment of an electronic device, thereby providing a 3D map updating function for the electronic device.

[0188] For example Figure 7 As shown, Figure 7 3D map updating system 700 may include a data collection module 710 , a block determination module 720 , a state determination module 730 and a map updating module 740 .

[0189] The data acquisition module 710 can be used to respond to the sensor data collected by the sensors on the mobile robot at the current moment, and determine the spatial point cloud data in the perception space based on the sensor data, wherein the perception space is configured as a three-dimensional space with the mobile robot as the origin.

[0190] The block determination module 720 may be used to determine a plurality of target blocks from each three-dimensional block in the perception space of the mobile robot, wherein a fixed number of three-dimensional units in the perception space form a three-dimensional block.

[0191] The state determination module 730 may be used to determine the perception status of each three-dimensional unit of multiple target blocks at a current moment based on the spatial point cloud data.

[0192] The map update module 740 can be used to determine the valid units detected by the sensor from the target block based on the perception situation, and synchronize the perception situation of the valid units to the three-dimensional map of the mobile robot based on the local coordinates of the valid units and the index relationship between the three-dimensional block and the three-dimensional unit, wherein the index relationship is determined based on the global coordinates of the three-dimensional unit.

[0193] The present application also provides an electronic device, such as Figure 8 The electronic device 800 provided in the present application includes a memory 810, a processor 820, and an input / output interface 830. The memory 810, the processor 820, and the input / output interface 830 are connected through an internal connection path, the memory 810 is used to store instructions, and the processor 820 is used to execute the instructions stored in the memory 810 to control the input / output interface 830 to receive input data and information, and output data such as operation results.

[0194] It should be understood that in the embodiment of the present application, the processor 820 can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs to implement the technical solution provided in the embodiment of the present application.

[0195] The memory 810 may include a read-only memory and a random access memory, and provides instructions and data to the processor 820. A portion of the processor 820 may also include a nonvolatile random access memory. For example, the processor 820 may also store information on the device type.

[0196] In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 820 or the instruction in the form of software. The three-dimensional map update method disclosed in the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in the processor for execution. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 810, and the processor 820 reads the information in the memory 810 and completes the steps of the above method in combination with its hardware.

[0197] In practical applications, the aforementioned electronic device can be used as a user terminal to implement the three-dimensional map updating method provided by the present application. For example, the user of the user terminal can interact with the electronic device to input the user's target timbre voice and source voice, thereby controlling the user terminal to replace the timbre in the source voice based on the user's timbre (i.e., the target timbre of the target timbre voice), thereby outputting the converted voice of the target timbre under the condition of the source voice.

[0198] In some embodiments, the aforementioned electronic device may be presented based on the actual needs of the three-dimensional map update. For example, the aforementioned electronic device may include a mobile terminal (such as a mobile phone, a tablet, a notebook), a robot, etc.

[0199] The present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction processor in the computer program product provided by the present application is executed, the three-dimensional map updating method provided by the present application can be implemented.

[0200] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0201] The present application also provides a mobile robot, wherein the mobile robot may include a sensor for collecting sensor data of a sensed space, wherein the transmission data is used to determine spatial point cloud data of the sensed space, at least one processor, and a memory in communication with the at least one processor. The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the aforementioned method for updating a three-dimensional map (such as P100).

[0202] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0203] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0204] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0205] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0207] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, operation and maintenance management (operation administration and maintenance, OAM) or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be read by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0208] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for updating a three-dimensional map, characterized in that: The method is applied to a mobile robot, and the method comprises: In response to sensor data collected at a current moment by a sensor on the mobile robot, determining spatial point cloud data in a perception space based on the sensor data, wherein the perception space is configured as a three-dimensional space with the mobile robot as an origin; Determine a plurality of target blocks from each three-dimensional block of the perception space of the mobile robot, wherein the three-dimensional blocks are composed of a fixed number of three-dimensional units in the perception space; Determine, based on the spatial point cloud data, the perception status of the sensor on each three-dimensional unit of the plurality of target blocks at the current moment; Based on the perception situation, the valid unit detected by the sensor is determined from the target block, and the perception situation of the valid unit is synchronized to the three-dimensional map of the mobile robot based on the local coordinates of the valid unit and the index relationship between the three-dimensional block and the three-dimensional unit, wherein the index relationship is determined based on the global coordinates of the three-dimensional unit.

2. The method according to claim 1, characterized in that The perception situation of the three-dimensional unit at the current moment includes the number of first optical fibers passing through the three-dimensional unit and the number of second optical fibers hitting the three-dimensional unit at the current moment, and determining the perception situation of the sensor on each of the three-dimensional units in the multiple target blocks at the current moment based on the spatial point cloud data includes: generating a plurality of projection rays within the plurality of target blocks based on the spatial point cloud data of the plurality of target blocks, wherein the projection rays are released from the sensor and hit the spatial point cloud data within the plurality of target blocks; Determine the passing conditions of each three-dimensional unit through which the projection light passes through the multiple target blocks, and store them in the storage area of ​​the corresponding three-dimensional unit to determine the perception conditions of the three-dimensional unit, wherein the passing conditions of the projection light include each three-dimensional unit through which the projection light passes and the three-dimensional unit hit.

3. The method according to claim 2, characterized in that The determining, based on the perception situation, from each three-dimensional unit of the target block, a valid unit detected by the sensor, and synchronizing the perception situation of the valid unit into the three-dimensional map of the mobile robot based on the local coordinates of the valid unit in the target block and the index relationship between the three-dimensional block and the three-dimensional unit, includes: Traversing the storage area of ​​each three-dimensional block in the target block based on the local coordinates of the three-dimensional unit, and determining the number of first rays and the number of second rays of each three-dimensional block; Determine the valid unit and the local coordinates of the valid unit from the target block based on the first number of rays, wherein the first number of rays of the valid unit is a non-zero value; Determine, based on the first light quantity and the second light quantity of the effective unit, an occupation probability change value of the effective unit at a current moment; The occupation probability change value is synchronized to a corresponding map element in a three-dimensional map of the mobile robot based on the local coordinates of the effective unit and the index relationship, and the occupation probability and occupation state of the map element are determined.

4. The method according to claim 3, characterized in that The step of synchronizing the occupancy probability change value to a corresponding map element in the three-dimensional map of the mobile robot based on the local coordinates of the effective unit and the index relationship, and determining the occupancy probability and occupancy state of the corresponding map element, comprises: Determining the occupation probability of the corresponding map element of the valid unit before the current moment; updating the occupation probability based on the occupation probability change value of the effective unit to determine the occupation probability of the effective unit at the current moment; Based on the local coordinates of the effective unit and the index relationship, the occupation probability of the effective unit at the current moment is synchronized to the corresponding map element, and the occupation state of the corresponding map element is determined.

5. The method according to claim 1, characterized in that The step of determining a plurality of target blocks from each three-dimensional block of the perception space of the mobile robot comprises: Determining a working range of the mobile robot at a current moment based on the working state of the mobile robot; The plurality of target blocks are determined from the perception space based on the working range, wherein the plurality of target blocks completely cover the working range.

6. The method according to any one of claims 1 to 5, characterized in that: The three-dimensional unit is configured as a basic unit of the perception space, and the method further includes: Based on the number of cells of the three-dimensional block in a reference direction; Determine each block in the perception space based on the number of units and generate a block list; Determine block index information of the three-dimensional unit based on the global coordinates of the three-dimensional unit and the number of units; Determine a block reference position of a block corresponding to the three-dimensional unit based on the block index information and the number of units, and determine a local coordinate of the three-dimensional unit in the corresponding block based on the block reference position; Based on the local coordinates of the three-dimensional unit in the corresponding area, local index information of the three-dimensional unit in the corresponding block is determined, wherein the local index information reflects the local coordinates of the three-dimensional unit in the corresponding block and is used to determine the global coordinates of the three-dimensional unit in combination with the block reference position, wherein each three-dimensional block in the block list includes the block index information of the three-dimensional block and the local index information of each three-dimensional unit contained in the three-dimensional block.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for updating a three-dimensional map according to any one of claims 1 to 6.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction processor is executed, the method for updating a three-dimensional map as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the method for updating a three-dimensional map according to any one of claims 1 to 6 is implemented.

10. A mobile robot, characterized in that: The mobile robot comprises: A sensor for collecting sensor data of a perception space, wherein the sensor data is used to determine spatial point cloud data of the perception space; at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for updating a three-dimensional map according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Methods and systems for vehicle environment map generation and updating

    CN110178048A

  • Map construction method and device, terminal equipment and storage medium

    CN114219905A

  • Map construction method and apparatus, and autonomous mobile device

    CN116182830A

  • Intelligent driving map construction method and device, computer equipment and storage medium

    CN117739946A

  • Map updating method and device, robot and storage medium

    CN118500380A