A method and device for constructing a three-dimensional map

Abstract

This application relates to the field of communication technology and discloses a method and device for constructing a three-dimensional map, which is used to construct a more accurate and complete three-dimensional map. The method comprises: a three-dimensional map construction device obtains a current global point cloud map; and, during the movement of a movable device, a plurality of image data collected by a local acquisition device provided on the movable device; and determines the real-time position information of the local acquisition device at the time of collection corresponding to each image data collected by the local acquisition device; finally, constructs the three-dimensional map based on the current global point cloud map, the plurality of image data, and the real-time position information of the local acquisition device.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication technology, and in particular to a method and device for constructing a three-dimensional map. Background Art

[0002] In areas like real-time positioning and navigation, building three-dimensional maps is a crucial prerequisite. For example, mobile robots, in order to perform intelligent actions, require accurate location information. Therefore, the integrity and accuracy of these three-dimensional maps directly impact the performance of positioning and navigation.

[0003] Currently, existing technologies primarily use simultaneous localization and mapping (SLAM) technology to construct three-dimensional maps. SLAM primarily includes visual SLAM and laser SLAM. However, visual SLAM is affected by factors such as light intensity and background texture, potentially resulting in low map accuracy. While laser SLAM can obtain relatively accurate positioning information for detected targets, it cannot capture scene information such as the target's texture and color, resulting in poor map visualization.

[0004] Therefore, how to construct a more accurate and complete three-dimensional map is a technical issue worthy of study. Summary of the Invention

[0005] The embodiments of the present application provide a method and apparatus for constructing a three-dimensional map, which are used to construct a more accurate and complete three-dimensional map and improve the accuracy of real-time positioning and navigation.

[0006] The specific technical solutions provided in the embodiments of this application are as follows:

[0007] In a first aspect, an embodiment of the present application provides a method for constructing a three-dimensional map, the method comprising: a three-dimensional map constructing device obtains a current global point cloud map; and the three-dimensional map constructing device obtains multiple image data collected by a local acquisition device provided on the movable device during the movement of the movable device; and the three-dimensional map constructing device determines the real-time posture information of the local acquisition device at the acquisition moment corresponding to each image data collected by the local acquisition device; the three-dimensional map constructing device constructs the three-dimensional map based on the current global point cloud map, the multiple image data and the real-time posture information of the local acquisition device.

[0008] In this method, by using a bird's-eye view to obtain the current global point cloud map, it can complement the local image data collected by the local acquisition device, thereby improving the integrity of the constructed three-dimensional map and enhancing the robustness of the three-dimensional map construction. Therefore, compared with the existing technical implementation of SLAM based solely on collected sensor data, which may result in fewer collected features and thus lead to inaccurate three-dimensional maps, the method provided by this application can improve the accuracy of three-dimensional map construction.

[0009] In one possible design, the three-dimensional map is constructed in a specified coordinate system, which is any one of the following coordinate systems: a first coordinate system determined by the posture of the global acquisition device at the initial moment, a second coordinate system determined by the posture of the local acquisition device at the initial moment, a world coordinate system, and a third coordinate system determined by the posture of any fixed spatial point. The three-dimensional map construction device constructs a three-dimensional map based on the current global point cloud map, the multiple image data and the real-time posture information of the local acquisition device, and can be implemented as determining the original coordinate systems in which the current global point cloud map, the multiple image data and the real-time posture information of the local acquisition device are respectively located; for an original coordinate system different from the specified coordinate system, the map data in the original coordinate system is converted based on a conversion relationship to obtain the map data in the specified coordinate system, and the conversion relationship is determined according to the posture relationship between the original coordinate system and the coordinate origin of the specified coordinate system, and the map data in the original coordinate system includes one or more of the current global point cloud map, the multiple image data and the real-time posture information of the local acquisition device; the three-dimensional map is constructed based on the current global point cloud map, the multiple image data and the real-time posture information of the local acquisition device in the specified coordinate system to obtain the three-dimensional map in the specified coordinate system.

[0010] Based on this design, a 3D map construction device can construct a 3D map in any coordinate system, facilitating real-time positioning, navigation, and the like based on the constructed 3D map. Specifically, the coordinate transformation of any spatial point in the map data between different coordinate systems can be determined based on the position of the coordinate origin of the coordinate system to obtain the transformation relationship between the different coordinate systems. Therefore, when this application is implemented, it is possible to combine the global point cloud map collected by the global acquisition device and the image data collected by the local acquisition device, thereby improving the integrity and accuracy of the constructed 3D map.

[0011] In one possible design, the relative posture of the global acquisition device and the AP is fixed, the relative posture of the local acquisition device and the STA is fixed, and the AP and the STA communicate. Exemplarily, if the original coordinate system different from the designated coordinate system is the first coordinate system, the designated coordinate system is the second coordinate system, and the conversion relationship is the first conversion relationship, the three-dimensional map construction device determines the first conversion relationship according to the following method: determining a first conversion matrix according to the relative posture of the global acquisition device and the AP, determining a second conversion matrix according to the positioning result of the AP for the STA, and determining a third conversion matrix according to the relative posture of the STA and the local acquisition device; and obtaining the first conversion relationship based on the product of the first conversion matrix, the second conversion matrix, and the third conversion matrix.

[0012] In this design, by using the AP's positioning results for STAs, the conversion relationship between the global acquisition device coordinate system and the initial local acquisition device coordinate system can be obtained, thereby achieving coordinate conversion of the global point cloud map or local image data, so as to construct a three-dimensional map based on the global point cloud map and local image data in the same coordinate system, thereby achieving a more accurate and complete three-dimensional map compared to existing technologies. Compared to existing SLAM technology-based implementations that require feature matching using adjacent sensor data at different times before determining, this application can reduce the amount of calculation and ensure accuracy based on the AP's positioning results for STAs.

[0013] In a possible design, the three-dimensional map construction device determines the acquisition time corresponding to the acquisition of each image data by the local acquisition device, and the real-time posture information of the local acquisition device can be implemented as obtaining the real-time positioning result of the AP for the STA at each acquisition time corresponding to the acquisition of each image data; based on the initial positioning result of the AP for the STA at the initial time, the initial posture information of the STA in the fourth coordinate system at the initial time, and the real-time positioning result, the real-time posture information of the STA in the fourth coordinate system at each acquisition time is obtained, wherein the coordinate origin of the fourth coordinate system is determined by the posture of the STA at the initial time; the real-time posture information of the local acquisition device in the fourth coordinate system is determined according to the real-time posture information of the STA in the fourth coordinate system and the third transformation matrix; the real-time posture information of the local acquisition device in the fourth coordinate system is coordinate-converted to obtain the real-time posture information of the local acquisition device in the specified coordinate system. In this design, the real-time pose information of the local acquisition device at each acquisition moment is obtained after the AP performs coordinate transformation on the real-time positioning result of the STA. Therefore, there is no need to rely on other image data collected in adjacent time periods. In this way, the disadvantage of the existing technology of estimating the real-time pose of the robot by feature matching of image data in adjacent time periods can be avoided, thereby improving the accuracy of the constructed three-dimensional map.

[0014] In one possible design, the three-dimensional map construction device detects loop anomalies that exist during the movement of the movable device based on the real-time positioning result of the AP for the STA.

[0015] In this design, compared with the existing loop detection implementation method, which can only determine whether the current position of the movable device is a position that has been reached before by extracting a series of features from the sensor data and then combining the bag of words method to compare the similarity between different image data, when this application is implemented, the AP performs loop detection on the STA's positioning results, which can improve the efficiency of building a three-dimensional map and improve the robustness of loop detection in weak texture environments (such as white walls).

[0016] In one possible design, before the three-dimensional map construction device obtains multiple image data collected by the local acquisition device during the movement of the movable device, the three-dimensional map construction device plans a moving path for the movable device based on the current global point cloud map; the three-dimensional map construction device generates a movement instruction according to the real-time posture information of the local acquisition device and the moving path; the three-dimensional map construction device sends the movement instruction to the controller of the movable device, so that the controller controls the movable device to move according to the movement instruction.

[0017] In this design, a moving path is planned for the movable device based on the current global point cloud map obtained, and then a moving instruction is generated for the movable device to move within the movable path range based on the real-time posture information of the local acquisition device. This can avoid blind movement of the movable device and thus improve the efficiency and accuracy of three-dimensional map construction.

[0018] In a second aspect, embodiments of the present application provide a device for constructing a three-dimensional map, comprising a transceiver unit and a processing unit. The functions performed by the transceiver unit and the processing unit may correspond to the steps performed by the device for constructing a three-dimensional map in any possible design or implementation of the first aspect.

[0019] In a third aspect, an embodiment of the present application further provides a computing device, comprising one or more processors and a memory, wherein the memory is coupled to the processor, and the memory stores computer program code, wherein the computer program code comprises computer instructions. The processor executes the computer instructions in the memory to execute the method provided in any possible design of the first aspect. Optionally, the computing device may further comprise a display screen. The display screen is used to display information to the user under the triggering of the processor, such as a global point cloud map collected by a global acquisition device, or image data collected by a local acquisition device, etc. Optionally, the computing device further comprises a communication interface, and the processor is coupled to the communication interface. The communication interface may be a transceiver or an input / output interface; when the computing device is a chip included in a network device, the communication interface may be the input / output interface of the chip. Optionally, the transceiver may be a transceiver circuit, and the input / output interface may be an input / output circuit.

[0020] In a fourth aspect, an embodiment of the present application further provides a computing device cluster, comprising at least one computing device as provided in the third aspect above.

[0021] In a fifth aspect, an embodiment of the present application further provides a computer storage medium, wherein the computer storage medium stores a computer program. When the computer program is executed by a computer, the method provided in any possible design of the first aspect is implemented.

[0022] In a sixth aspect, an embodiment of the present application further provides a computer program product, which includes: a computer program code, which, when executed by a processor of a communication device, enables the communication device to execute a method in any possible design of the first aspect above.

[0023] In a seventh aspect, an embodiment of the present application further provides a chip for reading and executing a software program stored in a memory to implement the method in any possible design of the first aspect. The memory may be connected to the chip, or the memory may be built into the chip.

[0024] For the beneficial effects of any of the second to seventh aspects, please refer to the beneficial effects of various possible designs in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of a SLAM system;

[0026] Figure 2 This is an application scenario diagram of a method for constructing a three-dimensional map in an embodiment of the present application;

[0027] Figure 3 This is an interactive schematic diagram of a method for constructing a three-dimensional map in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of a flow chart of a method for constructing a three-dimensional map in an embodiment of the present application;

[0029] Figure 5 This is a schematic structural diagram of a three-dimensional map construction device according to an embodiment of the present application;

[0030] Figure 6 This is a schematic structural diagram of another three-dimensional map construction device according to an embodiment of the present application;

[0031] Figure 7 This is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0033] The embodiments of this application can be applied to scenarios such as real-time positioning and navigation of mobile devices during autonomous movement. These mobile devices include robot vacuums, food delivery robots, delivery robots, mobile carts, and logistics carts. This application does not impose any restrictions on the specific types of mobile devices to which it can be applied.

[0034] To ensure accurate intelligent walking behavior for mobile devices, it is usually necessary to build a complete and accurate three-dimensional map. Currently, the existing technology mainly uses SLAM technology to achieve this goal. However, using SLAM technology to build three-dimensional maps may have problems such as low accuracy due to time-accumulated errors and incomplete three-dimensional maps due to blind spots.

[0035] In view of this, an embodiment of the present application provides a method for constructing a three-dimensional map, which constructs a three-dimensional map by combining a global point cloud map, multiple local image data, and real-time posture information of a local acquisition device, thereby achieving mutual blind spots between the global point cloud map and the local image data to construct a more accurate and complete three-dimensional map.

[0036] First, the system architecture to which the embodiments of the present application can be applied is introduced. Figure 1 The following is a flow chart of a SLAM system. SLAM technology can be understood as follows: a mobile robot starts from an unknown location in an unknown environment and, during movement, locates itself based on position estimation and a map. Simultaneously, it constructs an incremental map based on its own positioning, enabling autonomous positioning and navigation of the mobile robot. The SLAM system primarily includes sensor data acquisition 101, odometer 102, loop detection 103, nonlinear optimization 104, and mapping 105. The main implementation of each function is as follows:

[0037] The sensor data acquisition 101 is used to collect sensor data required for map construction. This sensor data includes, but is not limited to, image data, laser point cloud information, and inertial sensor data. For example, in visual SLAM, the sensor data can be image data collected by a visual sensor such as a camera or a video camera; in laser SLAM, the sensor data can be laser point clouds acquired by a lidar, or images derived from laser point clouds.

[0038] The odometry 102 is used to estimate the robot's posture changes and build a local three-dimensional map. Pose changes include translation and rotation information. In the prior art, laser SLAM and visual SLAM primarily estimate the robot's posture by matching and comparing sensor data from adjacent time periods. However, during the odometry calculation process, since the estimated robot's posture is based on the robot's posture changes from adjacent time periods, there may be significant time-accumulated errors, which can reduce the accuracy of the constructed three-dimensional map.

[0039] The loop detection 103 is used to determine whether the current position of the robot is a position that has been reached before. If a loop is determined to exist, the loop information is used for nonlinear optimization 104 to correct the drift of the odometer.

[0040] The nonlinear optimization 104 is used to optimize the robot posture obtained by the odometer 102 in different time periods and the loop information of the loop detection 103, mainly by optimizing the robot posture and map to obtain a globally consistent robot movement trajectory.

[0041] The mapping 105 is used to construct a complete global 3D map based on the robot's movement trajectory and the multiple local 3D maps created by the odometer 102. The constructed global 3D map includes, but is not limited to, sparse maps, dense maps, semantic maps, etc., which are not limited in this application.

[0042] Based on the content introduced in the background technology and the content introduced above, the method of constructing a map through visual SLAM is greatly affected by the accuracy of image feature matching. Under conditions of undesirable lighting and weak background texture, because fewer features are extracted from the sensor data, it may lead to problems such as inaccurate posture tracking and no map matching. The method of constructing a map through laser SLAM has the disadvantage of poor visualization, and when the robot is highly restricted, there is also the problem of incomplete map construction due to the inability to obtain information about obscured objects at high altitudes. In addition, when estimating the robot posture through the SLAM technology in the prior art, there may be a disadvantage of large time accumulation error. Therefore, the embodiment of the present application provides a method for constructing a three-dimensional map.

[0043] The embodiments of the present application can be applied to three-dimensional map construction scenarios. Specifically, the three-dimensional map construction scenario can be the initial construction of a three-dimensional map or the reconstruction of a three-dimensional map; and can be the construction of an indoor three-dimensional map or the construction of an outdoor three-dimensional map. Among them, the construction of an indoor three-dimensional map can be, for example, the construction of an indoor three-dimensional map by a sweeping robot, the construction of a warehouse three-dimensional map by a mobile vehicle, etc.; the construction of an outdoor three-dimensional map can be, for example, the construction of a park three-dimensional map by a park logistics vehicle, etc., and this application does not limit this.

[0044] It should be noted that the multiple involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. In addition, it should be understood that although the terms first, second, etc. may be used to describe each data in the embodiments of this application, these data should not be limited to these terms. These terms are only used to distinguish each data from each other.

[0045] Figure 2 This is an application scenario diagram of a three-dimensional map construction method provided in an embodiment of the present application. The application scenario may include: a global acquisition device 201, an access point (AP) 202, a workstation (STA) 203, and a local acquisition device 204.

[0046] If this application is applied to indoor 3D map construction, the global acquisition device 201 can be located at a height such as a rooftop to collect a global point cloud map of the target scene. Alternatively, if this application is applied to outdoor 3D map construction, the global acquisition device 201 can be located at a height such as a wall or utility pole. This application does not limit the configuration of the global acquisition device 201. The global acquisition device 201 can be a depth sensor, including but not limited to a laser radar based on a ranging laser, a depth camera, or a depth camera.

[0047] AP 202 is used to locate the workstation STA 203 to obtain the positioning result of the STA, thereby determining the real-time posture information of the STA. Among them, AP 202 can receive access from one or more STAs 203 based on optical communication technology, or can receive access from one or more STAs 203 through wireless technology, etc. Among them, the optical communication technology for communication between AP 202 and STA 203 includes but is not limited to light fidelity technology (light fidelity, Li-Fi), visible light communication technology (visible light communication, VLC), free-space optical communication (free-space optical communication, FSO), etc. It should be noted that AP 202 and global acquisition device 201 can be two independent devices, for example, an AP device and a depth sensor; or, AP 202 and global acquisition device 201 can also be included in an integrated device, for example, there is an integrated device that includes a unit that can realize the function of the global acquisition device 201 and a unit that can realize the function of AP202; or, the global acquisition device 201 can also be integrated into AP 202, which is not limited in this application.

[0048] The local acquisition device 204 can be provided on a mobile device or integrated in a mobile device. The local acquisition device 204 is used to collect a plurality of different image data during the movement of the mobile device. Among them, the local acquisition device 204 can be a monocular camera, a binocular camera, a red, green, and blue depth (RGBD) camera, etc., and this application does not limit this. Similar to the relative posture relationship between the global acquisition device 201 and the AP 202, when this application is implemented, the relative posture relationship between the local acquisition device 204 and the STA 203 is also not limited.

[0049] In addition, the application scenario also includes: host computer 205 (which may include Figure 2205a or 205b) and cloud server 206. The host computer 205a is used to obtain data from the global acquisition device 201 and AP 202, and process and calculate the received data to realize the display of the global point cloud map and the issuance of control instructions. For example, the host computer 205a performs coordinate conversion processing on the global point cloud map collected by the global acquisition device 201, and realizes the movement path planning of the movable device based on the global point cloud map. Similarly, the host computer 205b is used to obtain data from the local acquisition device 204 and STA203, and process and calculate the received data to realize the display of local image data and the issuance of control instructions. For example, the host computer 205b can be set in the movable device to realize autonomous movement control of the movable device and serve as a controller of the movable device. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical functional division. In actual implementation, there may be other division methods. In addition, the functional units in the various embodiments of the present application may be integrated into a processing unit, or may exist separately physically, or two or more units may be integrated into a single unit. For example, the host computer 205b may be integrated into a removable device, or may exist separately physically. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0050] The method provided in the embodiment of the present application can be implemented by the host computer 205a connected to the global acquisition device 201, or by the host computer 205b connected to the local acquisition device 204, or by the cloud server 206, or even by the coordinated division of labor between the host computer and the cloud server, that is, one or a combination of the host computer 205a, the host computer 205b or the cloud server can be used as a three-dimensional map construction device for implementing the method provided by the present application. During implementation, it can be determined based on the computing power of each device and / or the actual scenario. For example, if the computing power of the host computer 205a can support the implementation of the method provided by the present application, the method provided by the present application can be implemented by the host computer 205a, and the present application does not limit this.

[0051] It should be noted that the type and number of each device included in the application scenario may vary depending on specific practical needs. For example, there may be multiple global acquisition devices 201 installed on the roof, each responsible for collecting a global point cloud map within a region; or there may be multiple APs 202 installed on the roof.

[0052] based on Figure 2 The scene shown, Figure 3 This is a schematic diagram of an interactive method for constructing a three-dimensional map provided in an embodiment of the present application. The interactive process includes the following steps:

[0053] S301. The three-dimensional map construction device obtains the current global point cloud map.

[0054] For example, the global point cloud map can be obtained by the global acquisition device 201. During implementation, the global point cloud map represents a larger viewing angle range within the target scene for which a three-dimensional map needs to be constructed. For example, if a three-dimensional map of a room needs to be constructed to facilitate intelligent movement of a sweeping robot, the global point cloud map can be obtained from a camera in the room. In addition, since the positions of objects in the target scene may change frequently, the current global point cloud map can be obtained in real time or periodically so that changes in the target scene can be promptly known, thereby constructing a more accurate three-dimensional map that reflects the current situation.

[0055] It should be noted that the composition of the three-dimensional map construction device can be determined according to the actual scene conditions, and can be implemented by Figure 2 It is composed of one or a combination of the host computer 205a, the host computer 205b or the cloud server 206, and this application does not limit this.

[0056] In one possible implementation, the global point cloud map can be used to plan the movement path of a movable device. For example, by extracting features from the global point cloud map, the area of the walkable road in the target scene can be determined, and then the movement path for the movable device can be planned within the area of the walkable road. In addition, the three-dimensional map construction device can also determine the real-time position of the local acquisition device, thereby determining the orientation of the local acquisition device, that is, the movement direction of the movable device. Then, the three-dimensional map construction device can generate a movement instruction according to the movement direction of the movable device in the determined area of the walkable road. The three-dimensional map construction device can send the movement instruction to the controller of the movable device, and the movable device can be, for example, Figure 2 In the car, the controller can be, for example Figure 2 The upper computer 205b on the small or medium-sized vehicle can control the movable device to move according to the movement instructions through the upper computer 205b. Through this design, based on the acquired global point cloud map, the movement path of the movable device can be planned, which can avoid the blind movement of the movable device, thereby improving the efficiency and accuracy of the three-dimensional map construction.

[0057] S302: The three-dimensional map constructing device obtains a plurality of image data collected by a local collection device provided on the movable device during the movement of the movable device.

[0058] For example, since the local acquisition device can be installed on a movable device, the local acquisition device can move as the movable device moves, thereby enabling the local acquisition device to capture multiple different image data during the movement of the movable device. Specifically, the local acquisition device can periodically capture image data, for example, capturing one image data every 10 seconds; alternatively, the local acquisition device can capture one image data each time the movable device moves a specified distance, although this application is not limited to this.

[0059] S303: The three-dimensional map constructing device determines the real-time position information of the local acquisition device at the acquisition moment corresponding to each image data acquired by the local acquisition device.

[0060] In the prior art, in a SLAM system, the odometer 102 performs feature matching on adjacent image data collected by a local collection device to estimate the position change of the local collection device. However, there may be a disadvantage of a large time accumulation error.

[0061] In order to build a more accurate three-dimensional map, the positioning information of the STA can be obtained in real time based on the AP. When the present application is implemented, the relative posture of the AP and the global acquisition device is fixed, and the relative posture of the STA and the local acquisition device is fixed. Therefore, the real-time posture information of the STA can be determined by the real-time positioning result of the AP on the STA. Then, since the relative posture between the STA and the local acquisition device is fixed, the real-time posture information of the local acquisition device can be further determined based on the real-time posture information of the STA. It should be noted that in general scenarios, the postures of the global acquisition device and the AP are fixed at different times, and since the local acquisition device and the STA are set on a movable device, the posture information of the local acquisition device and the AP is changing.

[0062] S304: The three-dimensional map construction device constructs a three-dimensional map based on the current global point cloud map, the multiple image data and the determined multiple real-time posture information.

[0063] Exemplarily, according to the needs of the actual scene, a three-dimensional map can be constructed in a specified coordinate system. The specified coordinate system can be any one of the following coordinate systems: a first coordinate system determined by the posture of the global acquisition device at the initial moment (also referred to as the global acquisition device coordinate system), a second coordinate system determined by the posture of the local acquisition device at the initial moment (also referred to as the initial local acquisition device coordinate system), a world coordinate system, a third coordinate system determined by the posture of any fixed space point (for example, the AP coordinate system determined by the posture of the AP introduced in the above content, etc.). The posture includes position and posture, that is, three-dimensional coordinates and orientation. The initial moment can be represented as any moment when the movable device is in a stationary state in the target scene, or it can be any moment selected by the three-dimensional map construction device from the movable device during movement. The present application does not limit the selection of the initial moment, and is mainly for selecting a relative posture device of the global acquisition device and the local acquisition device at the same moment as the basic conversion relationship.

[0064] During the implementation process of constructing a three-dimensional map, the current global point cloud map, the plurality of image data, and the plurality of determined real-time pose information acquired by the three-dimensional map construction device may be in different coordinate systems. In order to realize the fusion construction of the three-dimensional map, these information are converted into the same coordinate system in the following steps:

[0065] Step A1: The three-dimensional map construction device determines the original coordinate systems in which the current global point cloud map, the plurality of image data, and the real-time position information of the local acquisition device are respectively located.

[0066] For example, the global point cloud map collected by the global acquisition device 201 is in the global acquisition device coordinate system. The image data collected by the local acquisition device 204 is in the real-time local acquisition device coordinate system determined by the real-time position of the local acquisition device. The real-time position information of the local acquisition device can be determined by the real-time position information of the STA, so the real-time position information of the local acquisition device is in the real-time STA coordinate system determined by the real-time position of the STA.

[0067] Step A2: For an original coordinate system different from the designated coordinate system, coordinate transformation is performed on the map data in the original coordinate system based on a transformation relationship to obtain the map data in the designated coordinate system.

[0068] The transformation relationship is determined based on the posture relationship between the original coordinate system and the coordinate origin of the designated coordinate system, and the map data in the original coordinate system includes one or more of the current global point cloud map, the multiple image data, and the real-time posture information of the local acquisition device. For example, if the designated coordinate system is the initial local acquisition device coordinate system, the original coordinate system of the global point cloud map is the global acquisition device coordinate system, a first transformation relationship from the global acquisition device coordinate system to the initial local acquisition device coordinate system is obtained, and after performing coordinate transformation on the global point cloud map in the global acquisition device coordinate system based on the first transformation relationship, a global point cloud map in the initial local acquisition device coordinate system is obtained.

[0069] Exemplarily, if the designated coordinate system is the initial local acquisition device coordinate system, the first transformation relationship for transforming the global point cloud map of the global acquisition device coordinate system to the initial local acquisition device coordinate system can be obtained through the following description.

[0070] Assume that there is any point in the space whose homogeneous coordinates in the global acquisition device coordinate system can be expressed as [X L , Y L , Z L , 1] T The homogeneous coordinates of this point in the initial local acquisition device coordinate system are expressed as [X C , Y C , Z C , 1] T , then the coordinate transformation relationship between the global acquisition device coordinate system and the initial local acquisition device coordinate system can be expressed by the following formula 1:

[0071]

[0072] Among them, the The transformation matrix from the global acquisition device coordinate system to the initial local acquisition device coordinate system is represented by the matrix R 3*3 is a rotation matrix used to represent the conversion of the orientation angle between the global acquisition device coordinate system and the initial local acquisition device coordinate system. 3*1 is a translation vector used to represent the conversion of three-dimensional coordinates between the global acquisition device coordinate system and the initial local acquisition device coordinate system.

[0073] It should be noted that if the spatial point in the initial local acquisition device coordinate system is converted to the global acquisition device coordinate system, it can be expressed by the following formula 2:

[0074]

[0075] Among them, the matrix R 3*3 ′ is the matrix R in formula 1 3*3The inverse matrix, matrix T 3*1 ′ is the matrix R 3*3 The inverse matrix of the translation vector T in formula 1 3*1 The inverse number of the product of , the specific expression can refer to the following formula 3. It can be understood that the above formula 2 is the inverse process of formula 1.

[0076] Based on the above formulas 1 and 2, we can obtain the first transformation relationship for converting any spatial point from the global acquisition device coordinate system to the initial local acquisition device coordinate system, and the second transformation relationship for converting any spatial point from the initial local acquisition device coordinate system to the global acquisition device coordinate system. This allows for convenient coordinate conversion between different coordinate systems.

[0077] In one possible implementation, the coordinate transformation relationship from the global acquisition device coordinate system to the initial local acquisition device coordinate system can be determined based on the initial positioning result of the AP for the STA at the initial moment, and can be specifically implemented by three transformation matrices as follows:

[0078] The first conversion matrix is determined based on the relative posture between the global acquisition device and the AP. Among them, based on the fact that the relative posture between the global acquisition device and the AP is fixed, it can be obtained that the first conversion matrix is a fixed matrix. It should be noted that no matter in which way the global acquisition device and the AP are implemented, such as the independent method introduced in the aforementioned content, contained in an integrated device, or the global acquisition device is contained in the AP; since the coordinate origins of the light-emitting devices (i.e., modules for optical communication) contained in the global acquisition device and the AP are different, in order to obtain a more accurate conversion relationship, the relative posture between the global acquisition device and the AP is reflected based on the coordinate origin of the global acquisition device and the coordinate origin of the light-emitting device of the AP.

[0079] For example, the first transformation matrix can be obtained by the matrix To express.

[0080] The second transformation matrix is determined according to the relative posture between AP and STA, which can be obtained by the matrix To express.

[0081] Because the STA is located on a mobile device, the relative position between the AP and the STA changes in real time as the mobile device moves. Therefore, the second transformation matrix is not a fixed matrix. In practice, the 3D map construction device can determine the relative position between the AP and the STA based on the AP's positioning of the STA, and thus determine the second transformation matrix.

[0082] For example, the positioning result of AP to STA can be obtained through matrix Positioning, based on the positioning result of the AP for the STA, the second transformation matrix can be expressed by the following formula 3:

[0083]

[0084] The third transformation matrix is determined based on the relative position between the STA and the local acquisition device. The relative position between the STA and the local acquisition device is fixed, so the third transformation matrix is also a fixed matrix.

[0085] Exemplarily, the third conversion matrix can be obtained by the same acquisition principle as the first conversion matrix To express.

[0086] Therefore, by combining the product of the first conversion matrix, the second conversion matrix, and the third conversion matrix, the conversion relationship between the global acquisition device and the local acquisition device can be obtained. The conversion relationship can be determined, for example, by the following formula 4:

[0087] In another possible implementation, when the present application is implemented, the way in which the three-dimensional map construction device determines the conversion relationship can also be implemented with the assistance of calibration objects. For example, calibration objects such as checkerboards and targets can be placed in the target scene. At any moment during the movement of the movable device, the acquisition areas of the global acquisition device and the local acquisition device contain a common field of view, which can also be understood as both being able to collect calibration object information. In this scenario, based on the feature extraction and matching algorithm in the field of computer vision technology, the coordinate set {[X C i , Y C i , Z C i |i∈calibration object]} t , and the coordinate set of the calibration object in the global acquisition device coordinate system {[X L i , Y L i , Z L i |i∈calibration object]} t Then, the iterative closest point (ICP) and perspective-n-point (PnP) registration algorithms are used to calculate the transformation relationship of the global point cloud map from the global acquisition device coordinate system to the local acquisition device coordinate system. Then, the global point cloud map is transformed based on the transformation relationship to obtain the global point cloud map in the local acquisition device coordinate system.

[0088] The following describes how to determine the real-time position information of the local acquisition device in a specified coordinate system based on the real-time positioning results of the AP for the STA.

[0089] For example, at the initial moment, the fourth coordinate system (also referred to as the initial STA coordinate system) is established with the initial posture of the STA. At this time, the STA is at the coordinate origin of the initial STA coordinate system. The initial posture information of the STA can be expressed as Among them, the matrix I 3*3 It is represented as a matrix that is consistent with the STA orientation. According to the above introduction, the third transformation matrix from STA to local acquisition device can be obtained as Then the initial pose of the local acquisition device in the initial STA coordinate system can be expressed as Furthermore, the three-dimensional map construction device can also determine the posture of STA in the AP coordinate system (i.e., the coordinate system established with the posture of AP at the initial moment as the coordinate origin) as the initial positioning result of AP for STA. To express.

[0090] During the movement, taking any moment t1 as an example, the 3D map construction device can determine the real-time position information of the local acquisition device in the initial local acquisition device coordinate system by the following steps:

[0091] Step B1: The 3D map construction device determines the real-time positioning result of the AP to the STA at time t1. Assume that To express.

[0092] Step B2: The 3D map construction device performs coordinate transformation on the STA in the AP coordinate system to obtain the real-time pose information of the STA in the initial STA coordinate system. To express.

[0093] Step B3: The three-dimensional map constructing device determines the real-time pose information of the local acquisition device in the initial STA coordinate system according to the real-time pose information of the STA in the initial STA coordinate system and the third transformation matrix.

[0094] For example, because the third transformation matrix is a fixed matrix, the relative position of the local acquisition device compared to the STA at time t1 can be determined by the initial position of the local acquisition device in the initial STA coordinate system at the initial time. At time t1, the real-time position of the local acquisition device in the initial STA coordinate system is updated to

[0095] Step B4: performing coordinate transformation on the real-time pose information of the local acquisition device in the initial STA coordinate system to obtain the real-time pose information of the local acquisition device in the initial local acquisition device coordinate system. The real-time pose information of the local acquisition device in the initial local acquisition device coordinate system can be determined by multiplying the third transformation matrix by the real-time pose information of the local acquisition device in the initial STA coordinate system, which can be expressed as follows:

[0096]

[0097] In another possible example, if the conversion relationship is obtained with the assistance of a calibration object, the real-time posture information of the local acquisition device can be determined based on the positioning result of the AP for the STA, the obtained conversion relationship, the first conversion matrix between the global acquisition device and the AP, and the second conversion matrix between the STA and the local acquisition device.

[0098] Through the above examples, the real-time posture information of the local acquisition device at each acquisition moment is obtained after the AP performs coordinate transformation on the real-time positioning result of the STA. Therefore, there is no need to rely on other image data collected in adjacent time periods. In this way, the disadvantage of possible time accumulation error in the existing technology can be avoided.

[0099] Step A3: The three-dimensional map construction device constructs a three-dimensional map based on the current global point cloud map in the specified coordinate system, the multiple image data and the real-time posture information of the local acquisition device to obtain the three-dimensional map in the specified coordinate system.

[0100] In addition, based on the positioning results of the AP for the STA, when the present application is implemented, the three-dimensional map construction device can directly determine whether there is a loop anomaly in the movement of the movable device through the positioning results of the AP for the STA when implementing loop detection. Moreover, during specific implementation, the three-dimensional map construction device can also simultaneously combine the feature matching of the image data to detect loop anomalies. Through this design, compared with the existing implementation method of loop detection, which can only extract a series of features from the sensor data and then combine the bag of words method to compare the similarity between different image data, and then determine whether the position currently moved to by the movable device is a posture that has been reached before, when the present application is implemented, the three-dimensional map construction device performs loop detection on the positioning results of the AP for the STA, which can improve the efficiency of three-dimensional map construction.

[0101] It should be noted that before the three-dimensional map construction device constructs the three-dimensional map through the method provided in the embodiment of the present application, since the embodiment of the present application needs to use STA and AP for processing, it is necessary to determine whether the STA contained in the mobile device is online. If the STA is not online, the image data collected by the local acquisition device can be directly used to perform traditional SLAM. For example, if the local acquisition device is an RGBD camera, visual SLAM can be performed based on the image data collected by the RGBD camera.

[0102] For example, based on the global point cloud map obtained by the 3D map construction device, after optimizing the obtained real-time pose information and loop anomalies of the local acquisition device based on nonlinear optimization 104, the 3D map can be constructed based on mapping 105. The processing of the nonlinear optimization 104 function in the embodiment of the present application can refer to the optimization method in the SLAM system, and this application will not elaborate on this.

[0103] In order to more clearly understand the method provided in the embodiments of the present application, Figure 4 This is a flow chart of a method for constructing a three-dimensional map provided in an embodiment of the present application. Compared to existing SLAM systems, the method provided in this application also includes the following functions: global point cloud map acquisition 401, coordinate conversion 402, movement path planning 403, movement instruction generation 404, and positioning information acquisition 405. The main implementations of the newly added functions in this application are as follows:

[0104] The global point cloud map acquisition 401 indicates that the global acquisition device acquires the current global point cloud map.

[0105] The coordinate conversion 402 represents the coordinate conversion of the map data that does not belong to the specified coordinate system in the current global point cloud map, the multiple image data, and the real-time pose information of the local acquisition device to obtain map data in the specified coordinate system. In the embodiment of the present application, the conversion of the global point cloud map from the global acquisition device coordinate system to the initial local acquisition device coordinate system is used as an example for explanation. If the implementation adopts the implementation method of converting the local image data from the local acquisition coordinate system to the global acquisition device coordinate system, the coordinate conversion 402 can be performed based on the local image data obtained by the sensor data acquisition 101.

[0106] The movement path planning 403 indicates that a movement path can be planned for the movable device based on the global point cloud map, thereby preventing the movable device from moving blindly. The movement instruction generation 404 is used to indicate the movement instructions generated for the movable device based on the movement path and the position of the local acquisition device, thereby controlling the movement of the movable device.

[0107] The positioning information acquisition 405 indicates the acquisition of the positioning result of the AP to the STA, which can be used to perform loop detection 103 to improve the efficiency of building a three-dimensional map. In addition, the acquired positioning information can also be combined with the collected sensor data and the global point cloud map. Figure 1 It serves as input information for the odometer 102 to improve the accuracy of the estimation of the real-time pose information of the local acquisition device.

[0108] Among them, according to the specific actual scenario needs, or according to the limitations of computing power, for example, the upper computer 205 cannot support the processing requirements of the computational amount of odometer 102, nonlinear optimization 104, etc. in combination with the global point cloud map, the obtained global point cloud map can also be directly used for mapping 105. In this way, the global point cloud map and the local three-dimensional map are combined for fusion, and blind spots can be complemented with each other to obtain a more complete global three-dimensional map. This application does not limit the specific fusion method of the obtained global point cloud map and traditional SLAM.

[0109] Through the content introduced in the above method embodiment, this application constructs a three-dimensional map by combining the global point cloud map collected from a bird's-eye view, and by combining the optical communication between the AP and the STA, a more accurate conversion relationship can be obtained, which provides a guarantee for the construction of the three-dimensional map and can reduce the probability of incomplete three-dimensional maps due to the existence of blind spots in local collection devices. In addition, the movement path of the movable device can also be planned through the global point cloud map, thereby avoiding the blind movement of the movable device in the target scene, thereby improving the efficiency of the three-dimensional map construction.

[0110] Based on the same technical concept as the above-mentioned three-dimensional map construction method, the embodiment of the present application also provides a three-dimensional map construction device 500, such as Figure 5 As shown, the device can be on the host computer 205, and can also be set on the cloud server 206. The three-dimensional map construction device 500 includes: a transceiver unit 501 and a processing unit 502. The device 500 can be used to implement the method described in the above method embodiment. Among them, the optional transceiver unit 501 and the processing unit 502 can be connected to each other through a communication line 503; the communication line 503 can be a peripheral component interconnect standard (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication line 503 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0111] The transceiver unit 501 is used to obtain the current global point cloud map; and the transceiver unit 501 is also used to obtain multiple image data collected by a local acquisition device provided on the movable device during the movement of the movable device; and the processing unit 502 is used to determine the real-time posture information of the local acquisition device at the acquisition moment corresponding to the acquisition of each image data by the local acquisition device; the processing unit 502 is also used to construct a three-dimensional map based on the current global point cloud map, the multiple image data and the real-time posture information of the local acquisition device.

[0112] In which, the three-dimensional map is constructed in a specified coordinate system, and the specified coordinate system is any one of the following coordinate systems: a first coordinate system determined by the posture of the global acquisition device at the initial moment, a second coordinate system determined by the posture of the local acquisition device at the initial moment, a world coordinate system, and a third coordinate system determined by the posture of any fixed space point; the processing unit 502 is used to construct a three-dimensional map based on the current global point cloud map, the multiple image data and the real-time posture information of the local acquisition device, specifically for: determining the original coordinates of the current global point cloud map, the multiple image data and the real-time posture information of the local acquisition device. system; for an original coordinate system different from the specified coordinate system, performing coordinate transformation on the map data in the original coordinate system based on a transformation relationship to obtain map data in the specified coordinate system, wherein the transformation relationship is determined according to the posture relationship between the coordinate origins of the original coordinate system and the specified coordinate system, and the map data in the original coordinate system includes one or more of the current global point cloud map, the multiple image data, and the real-time posture information of the local acquisition device; constructing a three-dimensional map based on the current global point cloud map, the multiple image data, and the real-time posture information of the local acquisition device in the specified coordinate system to obtain a three-dimensional map in the specified coordinate system.

[0113] Exemplarily, the relative posture of the global acquisition device and the access point AP is fixed, the relative posture of the local acquisition device and the workstation STA is fixed, and the AP and the STA communicate; if the original coordinate system different from the designated coordinate system is the first coordinate system, the designated coordinate system is the second coordinate system, and the conversion relationship is the first conversion relationship, the processing unit 502 is used to determine the first conversion relationship according to the following method: determine the first conversion matrix according to the relative posture of the global acquisition device and the AP, determine the second conversion matrix according to the positioning result of the AP for the STA, and determine the third conversion matrix according to the relative posture of the STA and the local acquisition device; obtain the first conversion relationship based on the product of the first conversion matrix, the second conversion matrix and the third conversion matrix.

[0114] In a possible embodiment, the processing unit 502 is used to determine the real-time posture information of the local acquisition device at the acquisition moment corresponding to each image data acquired by the local acquisition device, and is specifically used to: obtain the real-time positioning result of the AP for the STA at each acquisition moment corresponding to each image data acquired; obtain the real-time posture information of the STA in the fourth coordinate system at each acquisition moment based on the initial positioning result of the AP for the STA at the initial moment, the initial posture information of the STA in the fourth coordinate system at the initial moment, and the real-time positioning result, wherein the coordinate origin of the fourth coordinate system is determined by the posture of the STA at the initial moment; determine the real-time posture information of the local acquisition device in the fourth coordinate system according to the real-time posture information of the STA in the fourth coordinate system and the third transformation matrix; perform coordinate conversion on the real-time posture information of the local acquisition device in the fourth coordinate system to obtain the real-time posture information of the local acquisition device in the specified coordinate system.

[0115] In addition, the processing unit 502 is further configured to detect loop anomalies existing in the moving process of the movable device based on the real-time positioning result of the STA by the AP.

[0116] In one possible design, the processing unit 502 is further used to plan a moving path for the movable device based on the current global point cloud map before obtaining multiple image data collected by the local acquisition device during the movement of the movable device; generate a moving instruction according to the real-time posture information of the local acquisition device and the moving path; the transceiver unit 501 is further used to send the moving instruction to the controller of the movable device, so that the controller controls the movable device to move according to the moving instruction.

[0117] It should be noted that the division of modules in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0118] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0119] Based on the same concept as the above-mentioned three-dimensional map construction method, Figure 6 As shown, an embodiment of the present application also provides a structural schematic diagram of a three-dimensional map construction device 600. The device 600 can be used to implement the method described in the above method embodiment, and reference can be made to the description in the above method embodiment. The device 600 may include one or more processors 601. The processor 601 may be a general-purpose processor or a dedicated processor, etc. For example, it may be a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processing unit can be used to control a three-dimensional map construction device (such as a base station, terminal, or chip, etc.), execute software programs, and process data of software programs. The three-dimensional map construction device may include a transceiver unit to realize signal input (reception) and output (transmission). For example, the transceiver unit may be a transceiver, a radio frequency chip, etc.

[0120] The apparatus 600 includes one or more processors 601 , and the one or more processors 601 can implement the methods shown in the above-mentioned embodiments.

[0121] Optionally, the processor 601 may implement other functions in addition to implementing the method of the embodiment shown above.

[0122] Optionally, in one design, processor 601 may execute instructions to cause apparatus 600 to perform the method described in the above method embodiments. The instructions may be stored in whole or in part within the processor, such as instruction 603, or in whole or in part in memory 602 coupled to the processor, such as instruction 604. Instructions 603 and 604 may also be used together to cause apparatus 600 to perform the method described in the above method embodiments.

[0123] In another possible design, the device 600 may include one or more memories 602, on which instructions 604 are stored. The instructions can be executed on a processor, causing the device 600 to perform the method described in the above method embodiment. Optionally, the memory may also store data. The optional processor may also store instructions and / or data. For example, one or more memories 602 may store the corresponding relationships described in the above embodiments, or related parameters or tables involved in the above embodiments. The processor and memory may be provided separately or integrated together.

[0124] In another possible design, the device 600 may further include a transceiver 605 and an antenna 606. The processor 601 may be referred to as a processing unit and controls the device (terminal or base station). The transceiver 605 may be referred to as a transceiver, a transceiver circuit, or a transceiver unit, etc., and is configured to implement the transceiver functions of the device via the antenna 606.

[0125] It should be noted that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above-described method embodiment can be completed by the hardware integrated logic circuit 702 in the processor or by software instructions. The above-described processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above-described method.

[0126] It will be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that memory of the systems and methods described herein is intended to comprise, but not be limited to, these and any other suitable types of memory.

[0127] An embodiment of the present application further provides a computer storage medium having a computer program stored thereon. When the computer program is executed by a computer, the method for constructing a three-dimensional map of any of the method embodiments shown above is implemented.

[0128] An embodiment of the present application also provides a computer program product, which, when executed by a computer, implements the method for constructing a three-dimensional map of any of the method embodiments shown above.

[0129] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. Available media may be magnetic media (eg, floppy disks, hard disks, tapes), optical media (eg, high-density digital video discs (DVDs)), or semiconductor media (eg, solid state disks (SSDs)).

[0130] An embodiment of the present application further provides a processing device, including a processor and an interface; the processor is used to execute the three-dimensional map construction method of any of the method embodiments shown above.

[0131] It should be understood that the above-mentioned processing device can be a chip, and the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit 702, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor, which is implemented by reading the software code stored in the memory. The memory can be integrated into the processor or can be located outside the processor and exist independently.

[0132] like Figure 7 As shown, an embodiment of the present application also provides a chip 700, including an input / output interface 701 and a logic circuit 702, wherein the input / output interface 701 is used to receive / output code instructions or information, and the logic circuit 702 is used to execute code instructions or information to execute the three-dimensional map construction method of any of the method embodiments shown above.

[0133] The chip 700 can implement the functions shown in the processing unit and / or the transceiver unit in the above embodiments.

[0134] For example, the input and output interface 701 is used to obtain the current global point cloud map.

[0135] For another example, the input and output interface 701 is used to obtain a plurality of image data collected by a local collection device provided on the movable device during the movement of the movable device.

[0136] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0137] Those skilled in the art will clearly understand that, for the sake of convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0139] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0140] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0141] Through the description of the above embodiments, it will be clear to those skilled in the art that the present application can be implemented in hardware, firmware, or a combination thereof. When implemented using software, the above functions can be stored in a computer storage medium or transmitted as one or more instructions or codes on a computer storage medium. Computer storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a computer. By way of example and not limitation, the computer storage medium can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection can appropriately become a computer storage medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the fixing of the medium. As used herein, the terms "disk" and "disc" include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs use lasers to reproduce data optically. Combinations of the above should also be included within the scope of protection for computer storage media.

Claims

1. A method for constructing a three-dimensional map, characterized in that: include: The three-dimensional map construction device obtains the current global point cloud map collected by the global collection device, and the relative position of the global collection device and the access point AP is fixed; as well as, The three-dimensional map constructing device obtains image data of multiple locations collected by a local collection device set on the mobile device during the movement of the mobile device, and the mobile device is also provided with a workstation STA; and The three-dimensional map construction device determines the acquisition time corresponding to each image data acquired by the local acquisition device, and the real-time posture information of the local acquisition device, wherein the relative posture of the local acquisition device and the STA is fixed, and the real-time posture information of the local acquisition device is determined based on the real-time posture information of the STA; wherein the AP and the STA communicate, and the real-time posture information of the STA is determined by the real-time positioning result of the AP on the STA; The three-dimensional map construction device constructs the three-dimensional map based on the current global point cloud map, the image data of the multiple locations, and the real-time posture information of the local acquisition device that corresponds one-to-one to the image data of the multiple locations.

2. The method according to claim 1, characterized in that The three-dimensional map is constructed in a specified coordinate system, and the specified coordinate system is any one of the following coordinate systems: a first coordinate system determined by the posture of the global acquisition device at the initial moment, a second coordinate system determined by the posture of the local acquisition device at the initial moment, a world coordinate system, and a third coordinate system determined by the posture of any fixed spatial point; The three-dimensional map construction device constructs the three-dimensional map based on the current global point cloud map, the multiple image data and the real-time position information of the local acquisition device, including: Determine the original coordinate systems in which the current global point cloud map, the plurality of image data, and the real-time pose information of the local acquisition device are respectively located; For an original coordinate system different from the designated coordinate system, performing coordinate transformation on map data in the original coordinate system based on a transformation relationship to obtain map data in the designated coordinate system, wherein the transformation relationship is determined according to a positional relationship between the coordinate origins of the original coordinate system and the designated coordinate system, and the map data in the original coordinate system includes one or more of the current global point cloud map, the plurality of image data, and the real-time positional information of the local acquisition device; A three-dimensional map is constructed based on the current global point cloud map in the specified coordinate system, the multiple image data and the real-time posture information of the local acquisition device to obtain the three-dimensional map in the specified coordinate system.

3. The method according to claim 2, characterized in that If the original coordinate system different from the designated coordinate system is the first coordinate system, the designated coordinate system is the second coordinate system, and the conversion relationship is the first conversion relationship, the three-dimensional map construction device determines the first conversion relationship according to the following method: Determine a first conversion matrix according to the relative position between the global acquisition device and the AP, determine a second conversion matrix according to the positioning result of the AP for the STA, and determine a third conversion matrix according to the relative position between the STA and the local acquisition device; The first conversion relationship is obtained based on the product of the first conversion matrix, the second conversion matrix and the third conversion matrix.

4. The method according to claim 3, characterized in that The three-dimensional map construction device determines the acquisition time corresponding to each image data acquired by the local acquisition device, and the real-time posture information of the local acquisition device includes: Obtaining a real-time positioning result of the AP for the STA at each acquisition moment corresponding to each piece of the image data; Based on the initial positioning result of the AP for the STA at the initial moment, the initial pose information of the STA in the fourth coordinate system at the initial moment, and the real-time positioning result, obtaining the real-time pose information of the STA in the fourth coordinate system at each acquisition moment, wherein the coordinate origin of the fourth coordinate system is determined by the pose of the STA at the initial moment; Determine the real-time pose information of the local acquisition device in the fourth coordinate system according to the real-time pose information of the STA in the fourth coordinate system and the third transformation matrix; The real-time position and posture information of the local acquisition device in the fourth coordinate system is subjected to coordinate transformation to obtain the real-time position and posture information of the local acquisition device in the designated coordinate system.

5. The method according to claim 3 or 4, characterized in that The method further comprises: The three-dimensional map constructing device detects loop anomalies existing in the movement of the movable device based on the real-time positioning result of the AP on the STA.

6. The method according to any one of claims 1 to 4, characterized in that Before the three-dimensional map constructing device acquires the plurality of image data collected by the local collection device during the movement of the movable device, the method further includes: The three-dimensional map constructing device plans a moving path for the movable device based on the current global point cloud map; The three-dimensional map construction device generates a movement instruction according to the real-time position information of the local acquisition device and the movement path; The three-dimensional map constructing device sends the movement instruction to the controller of the movable device, so that the controller controls the movable device to move according to the movement instruction.

7. A three-dimensional map construction device, characterized in that: including a transceiver unit and a processing unit; The transceiver unit is used to obtain the current global point cloud map collected by the global collection device, and the relative posture of the global collection device and the access point AP is fixed; and The transceiver unit is further used to obtain image data of multiple locations collected by a local collection device set on the mobile device during the movement of the mobile device, and a workstation STA is also set on the mobile device; and The processing unit is configured to determine a capture time corresponding to each image data captured by the local acquisition device and real-time posture information of the local acquisition device, wherein the relative posture of the local acquisition device and the STA is fixed, and the real-time posture information of the local acquisition device is determined based on the real-time posture information of the STA; wherein the AP and the STA communicate, and the real-time posture information of the STA is determined based on the real-time positioning result of the AP on the STA; The processing unit is also used to construct a three-dimensional map based on the current global point cloud map, the image data of the multiple locations, and the real-time posture information of the local acquisition device corresponding one-to-one to the image data of the multiple locations.

8. The device according to claim 7, characterized in that The three-dimensional map is constructed in a specified coordinate system, and the specified coordinate system is any one of the following coordinate systems: a first coordinate system determined by the posture of the global acquisition device at the initial moment, a second coordinate system determined by the posture of the local acquisition device at the initial moment, a world coordinate system, and a third coordinate system determined by the posture of any fixed spatial point; The processing unit is configured to construct a three-dimensional map based on the current global point cloud map, the plurality of image data, and the real-time position information of the local acquisition device, specifically for: Determine the original coordinate systems in which the current global point cloud map, the plurality of image data, and the real-time pose information of the local acquisition device are respectively located; For an original coordinate system different from the designated coordinate system, performing coordinate transformation on map data in the original coordinate system based on a transformation relationship to obtain map data in the designated coordinate system, wherein the transformation relationship is determined according to a positional relationship between the coordinate origins of the original coordinate system and the designated coordinate system, and the map data in the original coordinate system includes one or more of the current global point cloud map, the plurality of image data, and the real-time positional information of the local acquisition device; A three-dimensional map is constructed based on the current global point cloud map in the specified coordinate system, the multiple image data and the real-time posture information of the local acquisition device to obtain the three-dimensional map in the specified coordinate system.

9. The device according to claim 8, characterized in that If the original coordinate system different from the designated coordinate system is the first coordinate system, the designated coordinate system is the second coordinate system, and the conversion relationship is the first conversion relationship, the processing unit is configured to determine the first conversion relationship according to the following method: Determine a first conversion matrix according to the relative position between the global acquisition device and the AP, determine a second conversion matrix according to the positioning result of the AP for the STA, and determine a third conversion matrix according to the relative position between the STA and the local acquisition device; The first conversion relationship is obtained based on the product of the first conversion matrix, the second conversion matrix and the third conversion matrix.

10. The device according to claim 9, characterized in that The processing unit is used to determine the acquisition time corresponding to each image data acquired by the local acquisition device and the real-time posture information of the local acquisition device, specifically for: Obtaining a real-time positioning result of the AP for the STA at each acquisition moment corresponding to each piece of the image data; Based on the initial positioning result of the AP for the STA at the initial moment, the initial pose information of the STA in the fourth coordinate system at the initial moment, and the real-time positioning result, obtaining the real-time pose information of the STA in the fourth coordinate system at each acquisition moment, wherein the coordinate origin of the fourth coordinate system is determined by the pose of the STA at the initial moment; Determine the real-time pose information of the local acquisition device in the fourth coordinate system according to the real-time pose information of the STA in the fourth coordinate system and the third transformation matrix; The real-time position and posture information of the local acquisition device in the fourth coordinate system is subjected to coordinate transformation to obtain the real-time position and posture information of the local acquisition device in the designated coordinate system.

11. The device according to claim 9 or 10, characterized in that The processing unit is further configured to detect loop anomalies existing in the moving process of the movable device based on the real-time positioning result of the STA by the AP.

12. The device according to any one of claims 7 to 10, characterized in that: The processing unit is further configured to plan a moving path for the movable device based on the current global point cloud map before acquiring the plurality of image data collected by the local collection device during the movement of the movable device; Generate a movement instruction according to the real-time position information of the local acquisition device and the movement path; The transceiver unit is further configured to send the movement instruction to the controller of the movable device, so that the controller controls the movable device to move according to the movement instruction.

13. A computing device, characterized in that The computing device comprises a memory and one or more processors; wherein the memory stores computer program code, and the computer program code comprises computer instructions; when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is performed.

14. A computer equipment cluster, characterized in that: The computer device cluster includes at least one computing device according to claim 13 .

15. A computer storage medium, characterized in that The computer storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 6 is performed.

Images