Point cloud acquisition method, apparatus and computer-readable storage medium

Abstract

This application provides a point cloud acquisition method, apparatus, and computer-readable storage medium, relating to the field of data processing. It reduces the steps involved in point cloud acquisition, eliminates the need for manual intervention, and improves the efficiency of point cloud acquisition. The method includes: acquiring multiple image sets and positional relationship sets of a target object; converting the depth map in the image set corresponding to each camera to a color map in the image set corresponding to each camera based on the intrinsic parameter matrix of each camera, obtaining multiple local color point clouds; processing each image set and the corresponding local color point clouds based on a point cloud registration algorithm and the positional relationship set, obtaining multiple pose transformation relationships; and determining the global point cloud of the target object based on the multiple local color point clouds and the multiple pose transformation relationships.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to point cloud acquisition methods, apparatus and computer-readable storage media. Background Technology

[0002] Holographic communication systems can transmit the global point cloud of a local object to a remote device through point cloud acquisition, so that the object can be visually reproduced on the remote device.

[0003] One existing point cloud acquisition method involves placing a marker in the space containing the target object, and then performing global point cloud acquisition of the target object by moving the marker multiple times.

[0004] This solution requires moving the markers multiple times; therefore, the existing point cloud acquisition process is cumbersome, requires manual intervention, and is inefficient. Summary of the Invention

[0005] This application provides a point cloud acquisition method, apparatus, and computer-readable storage medium, which can reduce the steps of point cloud acquisition and eliminate the need for manual intervention, thereby improving the efficiency of point cloud acquisition.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] Firstly, a point cloud acquisition method is provided, comprising: acquiring multiple image sets and positional relationship sets of a target object; the positional relationship set includes the positional relationship of multiple cameras, including a master camera and at least one slave camera; an image set includes a depth map acquired by one camera and a color map acquired by another camera; the multiple image sets include a master image set and at least one slave image set, wherein the images in the master image set are acquired by the master camera, and the images in the slave image set are acquired by the slave camera; transforming the depth map in the image set corresponding to each camera to the color map in the image set corresponding to each camera based on the intrinsic parameter matrix of each camera, thereby obtaining multiple local color point clouds; processing each image set and the local color point clouds corresponding to each image set based on a point cloud registration algorithm and the positional relationship set, thereby obtaining multiple pose transformation relationships; a pose transformation relationship is used to characterize a rotation transformation matrix and a translation transformation matrix between a slave camera and a master camera, and the point cloud registration algorithm is based on graph feature learning; and determining the global point cloud of the target object based on the multiple local color point clouds and the multiple pose transformation relationships.

[0008] Based on this scheme, multiple image sets of the target object and multiple pose transformation relationships obtained by a point cloud registration algorithm based on graph feature learning are acquired, and multiple local color point clouds are determined. The global point cloud of the target object is then determined based on these local color point clouds and multiple pose transformations. Compared with existing schemes that require placing and moving markers multiple times, the scheme in this application eliminates the need for marker placement and multiple marker movements, thereby reducing the steps involved in point cloud acquisition, avoiding manual intervention, and improving the efficiency of point cloud acquisition.

[0009] In conjunction with the first aspect, in certain embodiments of the first aspect, each map set and the corresponding local color point cloud are processed based on a point cloud registration algorithm and a set of positional relationships to obtain multiple pose transformation relationships. Specifically, this includes: processing the depth map and color map in each map set according to a first preset segmentation algorithm to obtain a target mapping relationship corresponding to each map set; the target mapping relationship is used to indicate the correspondence between points in the depth map and points in the color map in each map set; processing the local color point cloud corresponding to each map set according to the target mapping relationship and a second preset segmentation algorithm to obtain a color intermediate point cloud corresponding to each map set; determining the initial pose transformation relationship between two adjacent cameras according to the point cloud registration algorithm and the color intermediate point clouds corresponding to two adjacent cameras in the multiple color intermediate point clouds to obtain multiple initial pose transformation relationships; determining the pose transformation relationship between the second camera and the main camera in the multiple cameras according to the pose transformation relationship between the first camera and the main camera in the multiple cameras; the first camera is the camera closest to the second camera between the main camera and the second camera in the multiple set of positional relationships, and the second camera is any one of the slave cameras in the multiple cameras.

[0010] Based on this scheme, it is possible to process each graph set and the corresponding local color point cloud based on the point cloud registration algorithm and the position relationship set to obtain multiple pose transformation relationships.

[0011] In conjunction with the first aspect, in some embodiments of the first aspect, the depth map in the image set corresponding to each camera is transformed into a color map in the image set according to the intrinsic parameter matrix of each camera to obtain multiple local color point clouds. Specifically, this includes: determining the three-dimensional coordinates of the depth map according to the two-dimensional coordinates of the depth map and the intrinsic parameter matrix of the camera corresponding to the depth map; and determining multiple local color point clouds according to the rotation transformation matrix and translation transformation matrix between the color camera and the depth camera in each camera and the three-dimensional coordinates of the depth map in the image set corresponding to each camera.

[0012] Based on this scheme, it is possible to convert the depth map in the map set corresponding to each camera to the color map in the map set according to the intrinsic parameter matrix of each camera, so as to obtain multiple local color point clouds.

[0013] In conjunction with the first aspect, in certain embodiments of the first aspect, determining the global point cloud of the target object based on multiple local color point clouds and multiple pose transformation relationships specifically includes: determining a set of voxels in the three-dimensional space corresponding to the main camera; the voxel set includes multiple voxels; determining the truncated signed distance function (TSDF) value, color value, and weight value of each voxel in the voxel set corresponding to each local color point cloud, to obtain multiple initial voxel sets; deleting initial voxels in each initial voxel set whose weight values ​​differ from the depth values ​​of the corresponding depth maps, to obtain multiple intermediate voxel sets; sequentially weighting and summing the TSDF value of each intermediate voxel in each intermediate voxel set with the historical TSDF value of each intermediate voxel, sequentially weighting and summing the color value of each intermediate voxel in each intermediate voxel set with the historical color value of each intermediate voxel, and sequentially weighting and summing the weight value of each intermediate voxel in each intermediate voxel set with the historical weight value of each intermediate voxel, to obtain the initial global point cloud of the target object; and determining the global point cloud of the target object based on the implicit surface extraction algorithm and the initial global point cloud of the target object.

[0014] Based on this scheme, it is possible to determine the global point cloud of a target object based on multiple local color point clouds and multiple pose transformation relationships.

[0015] Secondly, a point cloud acquisition device is provided to implement the point cloud acquisition method of the first aspect described above. The point cloud acquisition device includes modules, units, or means corresponding to the above method. These modules, units, or means can be implemented in hardware, software, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

[0016] In conjunction with the second aspect, in some embodiments of the second aspect, the point cloud acquisition device includes: an acquisition module and a processing module; the acquisition module is used to acquire multiple image sets and positional relationship sets of the target object; the positional relationship set includes the positional relationship of multiple cameras, the multiple cameras include a master camera and at least one slave camera, an image set includes a depth map acquired by one camera and a color map acquired by one camera, the multiple image sets include a master image set and at least one slave image set, the images in the master image set are acquired by the master camera, and the images in the slave image set are acquired by the slave camera; the processing module is used to convert the depth map in the image set corresponding to each camera to the color map in the image set corresponding to each camera according to the intrinsic parameter matrix of each camera, to obtain multiple local color point clouds; the processing module is also used to process each image set and the local color point cloud corresponding to each image set based on the point cloud registration algorithm and the positional relationship set, to obtain multiple pose transformation relationships; a pose transformation relationship is used to characterize a rotation transformation matrix and a translation transformation matrix between a slave camera and a master camera, the point cloud registration algorithm is based on graph feature learning; the processing module is also used to determine the global point cloud of the target object according to the multiple local color point clouds and the multiple pose transformation relationships.

[0017] In conjunction with the second aspect, in some embodiments of the second aspect, the processing module is further configured to process each map set and the corresponding local color point cloud based on a point cloud registration algorithm and a set of positional relationships to obtain multiple pose transformation relationships. Specifically, this includes: processing the depth map and color map in each map set according to a first preset segmentation algorithm to obtain a target mapping relationship corresponding to each map set; the target mapping relationship is used to indicate the correspondence between points in the depth map and points in the color map in each map set; processing the corresponding local color point cloud of each map set according to the target mapping relationship and the second preset segmentation algorithm to obtain a color intermediate point cloud corresponding to each map set; determining the initial pose transformation relationship between two adjacent cameras based on the point cloud registration algorithm and the color intermediate point clouds corresponding to two adjacent cameras in the multiple color intermediate point clouds to obtain multiple initial pose transformation relationships; determining the pose transformation relationship between the second camera and the main camera in the multiple cameras based on the pose transformation relationship between the first camera and the main camera in the multiple cameras; the first camera is the camera closest to the second camera between the main camera and the second camera in the multiple set of positional relationships, and the second camera is any one of the slave cameras in the multiple cameras.

[0018] In conjunction with the second aspect, in some embodiments of the second aspect, the processing module is used to transform the depth map in the image set corresponding to each camera into a color map in the image set according to the intrinsic parameter matrix of each camera, to obtain multiple local color point clouds. Specifically, this includes: determining the three-dimensional coordinates of the depth map according to the two-dimensional coordinates of the depth map and the intrinsic parameter matrix of the camera corresponding to the depth map; and determining multiple local color point clouds according to the rotation transformation matrix and translation transformation matrix between the color camera and the depth camera in each camera and the three-dimensional coordinates of the depth map in the image set corresponding to each camera.

[0019] In conjunction with the second aspect, in some embodiments of the second aspect, the processing module is further configured to determine the global point cloud of the target object based on multiple local color point clouds and multiple pose transformation relationships, specifically including: determining a set of voxels in the three-dimensional space corresponding to the main camera; the voxel set includes multiple voxels; determining the truncated signed distance function (TSDF) value, color value, and weight value of each voxel in the voxel set corresponding to each local color point cloud, to obtain multiple initial voxel sets; deleting initial voxels in each initial voxel set whose weight values ​​differ from the depth values ​​of the corresponding depth maps, to obtain multiple intermediate voxel sets; sequentially weighting and summing the TSDF value of each intermediate voxel in each intermediate voxel set with the historical TSDF value of each intermediate voxel, sequentially weighting and summing the color value of each intermediate voxel in each intermediate voxel set with the historical color value of each intermediate voxel, and sequentially weighting and summing the weight value of each intermediate voxel in each intermediate voxel set with the historical weight value of each intermediate voxel, to obtain the initial global point cloud of the target object; and determining the global point cloud of the target object based on the implicit surface extraction algorithm and the initial global point cloud of the target object.

[0020] Thirdly, a point cloud acquisition apparatus is provided, comprising: at least one processor and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the method provided by the first aspect and any possible implementation thereof.

[0021] Fourthly, a computer-readable storage medium is provided, wherein when instructions in the computer-readable storage medium are executed by a processor of a point cloud acquisition device, the point cloud acquisition device is enabled to perform the method provided by the first aspect and any possible implementation thereof.

[0022] Fifthly, a computer program product containing instructions is provided that, when run on a computer, enables the computer to perform the methods provided in the first aspect and any possible implementation thereof.

[0023] In a sixth aspect, a chip system is provided, comprising: a processor and an interface circuit; the interface circuit being configured to receive a computer program or instructions and transmit them to the processor; the processor being configured to execute the computer program or instructions to cause the chip system to perform the methods provided in the first aspect and any possible embodiments thereof.

[0024] The technical effects of any one of the second to sixth aspects can be found in the technical effects of the different embodiments of the first aspect described above, and will not be repeated here. Attached Figure Description

[0025] Figure 1 This application provides a schematic diagram of the architecture of a point cloud acquisition system.

[0026] Figure 2 A flowchart illustrating a point cloud acquisition method provided in this application;

[0027] Figure 3 A flowchart illustrating yet another point cloud acquisition method provided in this application;

[0028] Figure 4 A flowchart illustrating yet another point cloud acquisition method provided in this application;

[0029] Figure 5 A flowchart illustrating yet another point cloud acquisition method provided in this application;

[0030] Figure 6 This is a schematic diagram of the structure of a point cloud acquisition device provided in this application;

[0031] Figure 7 This is a structural schematic diagram of another point cloud acquisition device provided in this application. Detailed Implementation

[0032] In the description of this application, unless otherwise stated, "multiple" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0033] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0034] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being better or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0035] It is understood that the term "embodiment" used throughout the specification means that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, various embodiments throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0036] It is understood that in this application, "when," "if," and "if" all refer to the corresponding processing that will be carried out under certain objective circumstances, and are not limited to a specific time, nor do they require that there must be a judgment action when implemented, nor do they imply any other limitations.

[0037] It is understood that some optional features in the embodiments of this application can be implemented independently in certain scenarios without relying on other features, such as the current solution on which they are based, to solve the corresponding technical problems and achieve the corresponding effects. Alternatively, they can be combined with other features as needed in certain scenarios. Correspondingly, the apparatus given in the embodiments of this application can also implement these features or functions, which will not be elaborated here.

[0038] In this application, unless otherwise specified, the same or similar parts between the various embodiments can be referred to each other. In the various embodiments and implementation methods of the various embodiments in this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments and between the implementation methods of the various embodiments are consistent and can be mutually referenced. The technical features in different embodiments and between the implementation methods of the various embodiments can be combined according to their inherent logical relationships to form new embodiments, implementation methods, implementation methods, or implementation approaches. The following embodiments of this application do not constitute a limitation on the scope of protection of this application.

[0039] Figure 1 This is a schematic diagram of the architecture of a point cloud acquisition system provided in this application. The technical solutions of the embodiments of this application can be applied to... Figure 1The point cloud acquisition system shown is as follows: Figure 1 As shown, the point cloud acquisition system 10 includes a point cloud acquisition device 11 and an electronic device 12.

[0040] The point cloud acquisition device 11 is directly or indirectly connected to the electronic device 12. This connection can be wired or wireless, and this embodiment of the application does not limit the connection.

[0041] The point cloud acquisition device 11 can be used to receive data from the electronic device 12.

[0042] Electronic device 12 can be used to send data to point cloud acquisition device 11.

[0043] It should be noted that the point cloud acquisition device 11 and the electronic device 12 can be independent devices or integrated into the same device; this application does not make any specific limitation in this regard.

[0044] When the point cloud acquisition device 11 and the electronic device 12 are integrated into the same device, the communication method between the point cloud acquisition device 11 and the electronic device 12 is the same as the communication between internal modules of the device. In this case, the communication process between the two is the same as the communication process between the point cloud acquisition device 11 and the electronic device 12 when they are independent of each other.

[0045] In the following embodiments provided in this application, the point cloud acquisition device 11 and the electronic device 12 are described as being set up independently of each other.

[0046] In practical applications, the point cloud acquisition method provided in this application embodiment can be applied to the point cloud acquisition device 11, or to the devices included in the point cloud acquisition device 11.

[0047] The point cloud acquisition method provided in this application embodiment will be described below with reference to the accompanying drawings, taking the application of the point cloud acquisition method to the point cloud acquisition device 11 as an example.

[0048] Figure 2 This application provides a flowchart illustrating a point cloud acquisition method, as shown below. Figure 2 As shown, the method includes the following steps:

[0049] S201, The point cloud acquisition device acquires multiple sets of graphs and sets of positional relationships of the target object.

[0050] The location relationship set includes the location relationships of multiple cameras, including a master camera and at least one slave camera. The map set includes a depth map acquired by one camera and a color map acquired by another camera. The multiple map sets include a master map set and at least one slave map set. The maps in the master map set are acquired by the master camera, and the maps in the slave map set are acquired by the slave camera.

[0051] It should be noted that the target object can be a movable object, such as a human body, or it can be a stationary object, such as a table. This application does not impose any specific restrictions on this.

[0052] Taking three cameras as an example, the three cameras can be positioned in an equilateral triangle, an isosceles triangle, or a scalene triangle. Of course, the number of cameras and their positional relationships can also vary; this application does not impose any specific limitations on these.

[0053] As one possible implementation method, combined with Figure 1 The point cloud acquisition device receives a message from an electronic device, which includes multiple sets of graphs and positional relationships of the target object. The point cloud acquisition device then obtains these multiple sets of graphs and positional relationships of the target object from the message.

[0054] As another possible implementation method, combined with Figure 1 The point cloud acquisition device receives a first message from the electron and a second message from each of the multiple cameras. The first message includes a set of positional relationships, and the second message includes a set of graphs of the target objects corresponding to each camera. The point cloud acquisition device obtains the set of positional relationships from the first message and obtains each set of graphs of the target objects from each second message, thus obtaining multiple sets of graphs of multiple target objects.

[0055] S202. The point cloud acquisition device converts the depth map in the map set corresponding to each camera to the color map in the map set corresponding to each camera according to the intrinsic parameter matrix of each camera, and obtains multiple local color point clouds.

[0056] As one possible implementation, firstly, the point cloud acquisition device determines the three-dimensional coordinates of the depth map based on the two-dimensional coordinates of the depth map and the intrinsic parameter matrix of the camera corresponding to the depth map; then, the point cloud acquisition device determines multiple local color point clouds based on the rotation transformation matrix and translation transformation matrix between the color camera and the depth camera in each camera and the three-dimensional coordinates of the depth map in the image set corresponding to each camera.

[0057] It should be noted that for a detailed description of the possible implementation, please refer to the relevant description in the following sections of the specific implementation of this application, which will not be repeated here.

[0058] S203. The point cloud acquisition device processes each map set and the corresponding local color point cloud based on the point cloud registration algorithm and the position relationship set to obtain multiple pose transformation relationships.

[0059] Among them, a pose transformation relationship is used to characterize a rotation transformation matrix and a translation transformation matrix between the main camera and the secondary camera. The point cloud registration algorithm is based on graph feature learning.

[0060] As one possible implementation, the point cloud acquisition device processes the depth map and color map in each map set according to a first preset segmentation algorithm to obtain the target mapping relationship corresponding to each map set; processes the local color point cloud corresponding to each map set according to the target mapping relationship corresponding to each map set and a second preset segmentation algorithm to obtain multiple intermediate color point clouds; determines the initial pose transformation relationship between two adjacent cameras according to the point cloud registration algorithm and the intermediate color point clouds corresponding to two adjacent cameras in the multiple intermediate color point clouds to obtain multiple initial pose transformation relationships; and determines the pose transformation relationship between the second camera and the main camera in the multiple cameras according to the pose transformation relationship between the first camera and the main camera in the multiple cameras.

[0061] It should be noted that for a detailed description of the possible implementation, please refer to the relevant description in the following sections of the specific implementation of this application, which will not be repeated here.

[0062] S204. The point cloud acquisition device determines the global point cloud of the target object based on multiple local color point clouds and multiple pose transformation relationships.

[0063] As one possible implementation, the point cloud acquisition device determines the voxel set in the 3D space corresponding to the main camera; determines the truncated signed distance function (TSDF) value, color value, and weight value of each voxel in the voxel set corresponding to each local color point cloud, obtaining multiple initial voxel sets; deletes initial voxels in each initial voxel set whose weight values ​​differ from the depth values ​​of the corresponding depth maps, obtaining multiple intermediate voxel sets; sequentially weights and sums the TSDF value of each intermediate voxel in each intermediate voxel set with the historical TSDF value of each intermediate voxel, sequentially weights and sums the color value of each intermediate voxel in each intermediate voxel set with the historical color value of each intermediate voxel, and sequentially weights and sums the weight value of each intermediate voxel in each intermediate voxel set with the historical weight value of each intermediate voxel, obtaining the initial global point cloud of the target object; and determines the global point cloud of the target object based on the implicit surface extraction algorithm and the initial global point cloud of the target object.

[0064] It should be noted that for a detailed description of the possible implementation, please refer to the relevant description in the following sections of the specific implementation of this application, which will not be repeated here.

[0065] Based on this scheme, multiple image sets of the target object and multiple pose transformation relationships obtained by a point cloud registration algorithm based on graph feature learning are acquired, and multiple local color point clouds are determined. The global point cloud of the target object is then determined based on these local color point clouds and multiple pose transformations. Compared with existing schemes that require placing and moving markers multiple times, the scheme in this application eliminates the need for marker placement and multiple marker movements, thereby reducing the steps involved in point cloud acquisition, avoiding manual intervention, and improving the efficiency of point cloud acquisition.

[0066] The above is a general description of the point cloud acquisition method provided in this application. The following will provide a further explanation of the point cloud acquisition method provided in this application with reference to the accompanying drawings.

[0067] Figure 3 A flowchart illustrating another point cloud acquisition method provided in this application is shown below. Figure 3 As shown, S203 provided in the specific embodiments of this application may specifically include the following steps:

[0068] S301, The point cloud acquisition device processes the depth map and color map in each map set according to the first preset segmentation algorithm to obtain the target mapping relationship corresponding to each map set.

[0069] The target mapping relationship is used to indicate the correspondence between points in the depth map and points in the color map in each map set.

[0070] It should be noted that, taking the human body as the target object as an example, the first preset segmentation algorithm can be a human body segmentation algorithm, and the target mapping relationship can be a human body index map.

[0071] As one possible implementation, taking the first preset segmentation algorithm as the human body segmentation algorithm and the target mapping relationship as the human body index map as an example, the point cloud acquisition device performs human body detection on the depth map in each map set through the human body segmentation algorithm to obtain the initial human body index map, and converts the initial human body index map to the color map of the map set to obtain the human body index map corresponding to the map set.

[0072] S302, The point cloud acquisition device processes the local colored point cloud corresponding to each graph set according to the target mapping relationship and the second preset segmentation algorithm, and obtains the colored intermediate point cloud corresponding to each graph set.

[0073] It should be noted that the second preset segmentation algorithm can be a point cloud segmentation algorithm.

[0074] As one possible implementation, taking the second preset segmentation algorithm as a point cloud segmentation algorithm as an example, the point cloud acquisition device segments the local colored point cloud corresponding to each graph set according to the target mapping relationship and the point cloud segmentation algorithm, and obtains the colored intermediate point cloud corresponding to each graph set.

[0075] S303, the point cloud acquisition device determines the initial pose transformation relationship between two adjacent cameras based on the point cloud registration algorithm and the color intermediate point clouds corresponding to two adjacent cameras in multiple color intermediate point clouds, and obtains multiple initial pose transformation relationships.

[0076] As one possible implementation, the point cloud acquisition device randomly downsamples the intermediate point clouds corresponding to two adjacent cameras in multiple intermediate color point clouds to 1024 points each, and then inputs them into a point cloud registration algorithm based on graph feature learning. The point cloud feature extraction module of the point cloud registration algorithm extracts point cloud features from both point clouds separately, generating 1024-dimensional point cloud features for each point cloud. The point cloud registration algorithm inputs the generated point cloud features into its graph generation module to generate an adjacency matrix for each point cloud, representing the edges within a point cloud and the confidence score of each edge. The point cloud registration algorithm then inputs the two generated graphs into a graph matching module to generate a soft correspondence matrix between the two graphs, i.e., the confidence score that a point in one graph correctly matches a point in another graph. Specifically, when the two graphs first pass through the graph matching module, the point cloud registration algorithm uses a cross-graph convolution to learn the relationships between the two graphs to enhance node features, and then iterates through the graph generation and graph matching modules again using the node features output from the cross-graph convolution. Finally, the point cloud registration model outputs a soft correspondence matrix between the two point clouds. The point cloud acquisition device transforms the soft correspondence matrix into a hard correspondence matrix, thus obtaining the original rotation and translation transformations. Finally, the point cloud acquisition device precisely adjusts the original rotation and translation transformations using either the iterative closest point (ICP) algorithm or the normal-distributions transform (NDT) algorithm to obtain the initial pose transformation relationship.

[0077] S304, The point cloud acquisition device determines the pose transformation relationship between the second camera and the main camera among multiple cameras based on the pose transformation relationship between the first camera and the main camera among multiple cameras.

[0078] Among them, the first camera is the camera that is closest to the second camera among the multiple sets of positional relationships between the main camera and the second camera, and the second camera is any one of the multiple cameras that is a slave camera.

[0079] As one possible implementation, taking multiple cameras as four cameras, which are designated as main camera 1, slave camera 2, slave camera 3, and slave camera 4 in a clockwise direction, and the positional relationship of the four cameras is rectangular, with the second camera being slave camera 3, the camera closest to slave camera 3 between the main camera and slave camera 3 is camera 2. The point cloud acquisition device determines the pose transformation relationship between camera 3 and the main camera based on the pose transformation relationship between camera 2 and the main camera.

[0080] It should be noted that the specific implementation scheme of this possible method can refer to existing schemes, and will not be described in this application.

[0081] Based on this scheme, it is possible to process each graph set and the corresponding local color point cloud based on the point cloud registration algorithm and the position relationship set to obtain multiple pose transformation relationships.

[0082] Figure 4 A flowchart illustrating another point cloud acquisition method provided in this application is shown below. Figure 4 As shown, S202 provided in the specific embodiments of this application may specifically include the following steps:

[0083] S401, the point cloud acquisition device determines the three-dimensional coordinates of the depth map based on the two-dimensional coordinates of the depth map and the intrinsic parameter matrix of the camera corresponding to the depth map.

[0084] As one possible implementation, the point cloud acquisition device determines the two-dimensional coordinates of the depth map, the intrinsic parameter matrix of the camera corresponding to the depth map, and the three-dimensional coordinates of the depth map, satisfying the following relationship:

[0085]

[0086] Where (u, v) are the two-dimensional coordinates of the depth map, (x1, y1, z1) are the three-dimensional coordinates of the depth map, and K is the intrinsic parameter matrix of the corresponding camera.

[0087] S402, the point cloud acquisition device determines multiple local color point clouds based on the rotation transformation matrix and translation transformation matrix between the color camera and the depth camera in each camera, as well as the three-dimensional coordinates of the depth map in the image set corresponding to each camera.

[0088] As one possible implementation, the point cloud acquisition device determines the three-dimensional coordinates of the depth map, the rotation transformation matrix and translation transformation matrix of the camera corresponding to the depth map, and the local color point cloud corresponding to the depth map, satisfying the following relationship:

[0089]

[0090] Where (x2, y2, z2) represents the local color point cloud corresponding to the depth map, and (x1, y1, z1) represents the 3D coordinates of the depth map. R is the rotation transformation matrix of the camera corresponding to the depth map, and t is the translation transformation matrix of the camera corresponding to the depth map.

[0091] Based on this scheme, it is possible to convert the depth map in the map set corresponding to each camera to the color map in the map set according to the intrinsic parameter matrix of each camera, so as to obtain multiple local color point clouds.

[0092] Figure 5 A flowchart illustrating another point cloud acquisition method provided in this application is shown below. Figure 5 As shown, S204 provided in the specific embodiments of this application may specifically include the following steps:

[0093] S501, the point cloud acquisition device determines the set of voxels in the three-dimensional space corresponding to the main camera.

[0094] The voxel set includes multiple voxels.

[0095] As one possible implementation, the point cloud acquisition device divides the three-dimensional space corresponding to the main camera into multiple cubes of equal size, which are voxels, thus obtaining a voxel set.

[0096] S502, The point cloud acquisition device determines the TSDF value, color value and weight value of each voxel in the voxel set corresponding to each local color point cloud, and obtains multiple initial voxel sets.

[0097] As one possible implementation, with a total of four local color point clouds, the point cloud acquisition device determines the TSDF value, color value, and weight value of each voxel in the voxel set corresponding to each local color point cloud, thus obtaining four initial voxel sets.

[0098] S503, the point cloud acquisition device deletes the initial voxels in each initial voxel set whose weight values ​​are different from the depth values ​​of the corresponding depth map, and obtains multiple intermediate voxel sets.

[0099] As one possible implementation, the point cloud acquisition device compares the weight value of each initial voxel in each initial voxel set with the depth value of the depth map corresponding to that initial voxel set, deletes the initial voxels in each initial voxel set whose weight values ​​are different from the depth values ​​of the depth map corresponding to that initial voxel set, and obtains multiple intermediate voxel sets.

[0100] S504. The point cloud acquisition device sequentially adds the TSDF value of each intermediate voxel in each intermediate voxel set to the historical TSDF value of each intermediate voxel, sequentially adds the color value of each intermediate voxel in each intermediate voxel set to the historical color value of each intermediate voxel, and sequentially adds the weight value of each intermediate voxel in each intermediate voxel set to the historical weight value of each intermediate voxel, thereby obtaining the initial global point cloud of the target object.

[0101] It should be noted that the weights for the weighted sum can be 0.5 for historical values ​​and 0.5 for non-historical values, or 0.4 for historical values ​​and 0.6 for non-historical values. This application does not impose any specific restrictions on this.

[0102] As one possible implementation, taking color values ​​as an example, with four intermediate voxel sets, and the weighted summation weights being 0.5 for historical values ​​and 0.5 for non-historical values, if the color value of intermediate voxel 1 in an intermediate voxel set is 1, and its historical color value is 2, the point cloud acquisition device will use 0.5 × 1 + 0.5 × 2 = 1.5 as the preliminary color value of intermediate voxel 1. If this intermediate voxel set is the last set to be summed, then the point cloud acquisition device will use 1.5 as the final color value of intermediate voxel 1. Similarly, the point cloud acquisition device obtains the initial global point cloud of the target object.

[0103] S505, the point cloud acquisition device determines the global point cloud of the target object based on the implicit surface extraction algorithm and the initial global point cloud of the target object.

[0104] As one possible implementation, the point cloud acquisition device uses the initial global point cloud input values ​​of the target object in an implicit surface extraction algorithm to obtain the global point cloud of the target object.

[0105] Based on this scheme, it is possible to determine the global point cloud of a target object based on multiple local color point clouds and multiple pose transformation relationships.

[0106] The above mainly describes the solution provided by the embodiments of this application from the perspective of a point cloud acquisition device executing a point cloud acquisition method. To achieve the above functions, the point cloud acquisition device includes hardware structures and / or software modules corresponding to each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0107] This application embodiment can divide the point cloud acquisition device into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation. Furthermore, "module" here can refer to an application-specific integrated circuit (ASIC), a circuit, a processor and memory executing one or more software or firmware programs, integrated logic circuits, and / or other devices that can provide the above functions.

[0108] When using functional module division Figure 6 A schematic diagram of a point cloud acquisition device is shown. Figure 6 As shown, the point cloud acquisition device 60 includes an acquisition module 601 and a processing module 602.

[0109] In some embodiments, the point cloud acquisition device 60 may further include a storage module ( Figure 6 (Not shown in the image) is used to store program instructions and data.

[0110] The acquisition module 601 is used to acquire multiple image sets and positional relationship sets of the target object. The positional relationship set includes the positional relationship of multiple cameras, including a master camera and at least one slave camera. Each image set includes a depth map acquired by one camera and a color map acquired by another camera. The multiple image sets include a master image set and at least one slave image set, where the images in the master image set are acquired by the master camera and the images in the slave image set are acquired by the slave camera. The processing module 602 is used to convert the depth map in the image set corresponding to each camera to the color map in the image set corresponding to each camera based on the intrinsic parameter matrix of each camera, thereby obtaining multiple local color point clouds. The processing module 602 is also used to process each image set and the local color point cloud corresponding to each image set based on the point cloud registration algorithm and the positional relationship set, thereby obtaining multiple pose transformation relationships. Each pose transformation relationship is used to characterize the rotation transformation matrix and translation transformation matrix between a slave camera and a master camera. The point cloud registration algorithm is based on graph feature learning. The processing module 602 is also used to determine the global point cloud of the target object based on the multiple local color point clouds and the multiple pose transformation relationships.

[0111] Optionally, the processing module 602 is further configured to process each map set and the corresponding local color point cloud based on a point cloud registration algorithm and a set of positional relationships to obtain multiple pose transformation relationships. Specifically, this includes: processing the depth map and color map in each map set according to a first preset segmentation algorithm to obtain a target mapping relationship corresponding to each map set; the target mapping relationship is used to indicate the correspondence between points in the depth map and points in the color map in each map set; processing the local color point cloud corresponding to each map set according to the target mapping relationship and a second preset segmentation algorithm to obtain a color intermediate point cloud corresponding to each map set; determining the initial pose transformation relationship between two adjacent cameras based on the point cloud registration algorithm and the color intermediate point clouds corresponding to two adjacent cameras in the multiple color intermediate point clouds to obtain multiple initial pose transformation relationships; determining the pose transformation relationship between the second camera and the main camera in the multiple cameras based on the pose transformation relationship between the first camera and the main camera in the multiple cameras; the first camera is the camera closest to the second camera between the main camera and the second camera in the multiple set of positional relationships, and the second camera is any one of the slave cameras in the multiple cameras.

[0112] Optionally, the processing module 602 is used to transform the depth map in the image set corresponding to each camera into a color map in the image set according to the intrinsic parameter matrix of each camera, so as to obtain multiple local color point clouds. Specifically, it includes: determining the three-dimensional coordinates of the depth map according to the two-dimensional coordinates of the depth map and the intrinsic parameter matrix of the camera corresponding to the depth map; determining multiple local color point clouds according to the rotation transformation matrix and translation transformation matrix between the color camera and the depth camera in each camera and the three-dimensional coordinates of the depth map in the image set corresponding to each camera.

[0113] Optionally, the processing module 602 is further configured to determine the global point cloud of the target object based on multiple local color point clouds and multiple pose transformation relationships, specifically including: determining the voxel set in the three-dimensional space corresponding to the main camera; the voxel set includes multiple voxels; determining the truncated signed distance function (TSDF) value, color value, and weight value of each voxel in the voxel set corresponding to each local color point cloud, to obtain multiple initial voxel sets; deleting initial voxels in each initial voxel set whose weight values ​​differ from the depth values ​​of the corresponding depth maps, to obtain multiple intermediate voxel sets; sequentially weighting and summing the TSDF value of each intermediate voxel in each intermediate voxel set with the historical TSDF value of each intermediate voxel, sequentially weighting and summing the color value of each intermediate voxel in each intermediate voxel set with the historical color value of each intermediate voxel, and sequentially weighting and summing the weight value of each intermediate voxel in each intermediate voxel set with the historical weight value of each intermediate voxel, to obtain the initial global point cloud of the target object; and determining the global point cloud of the target object based on the implicit surface extraction algorithm and the initial global point cloud of the target object.

[0114] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0115] When the functions of the above modules are implemented in hardware... Figure 7 A schematic diagram of a point cloud acquisition device is shown. Figure 7 As shown, the point cloud acquisition device 70 includes a processor 701, a memory 702, and a bus 703. The processor 701 and the memory 702 can be connected via the bus 703.

[0116] The processor 701 is the control center of the point cloud acquisition device 70. It can be a single processor or a collective term for multiple processing elements. For example, the processor 701 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0117] As one embodiment, processor 701 may include one or more CPUs, for example Figure 7 CPU 0 and CPU 1 are shown in the diagram.

[0118] The memory 702 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0119] As one possible implementation, the memory 702 can exist independently of the processor 701. The memory 702 can be connected to the processor 701 via a bus 703 and is used to store instructions or program code. When the processor 701 calls and executes the instructions or program code stored in the memory 702, it can implement the point cloud acquisition method provided in this application embodiment.

[0120] In another possible implementation, the memory 702 can also be integrated with the processor 701.

[0121] Bus 703 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0122] It should be pointed out that, Figure 7 The structure shown does not constitute a limitation on the point cloud acquisition device 70. Except... Figure 7 In addition to the components shown, the point cloud acquisition device 70 may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0123] As an example, combined Figure 6 The functions implemented by the acquisition module 601 and the processing module 602 in the point cloud acquisition device 60 are the same as those of the acquisition module 601 and the processing module 602. Figure 7 The processor 701 in it has the same function.

[0124] Optional, such as Figure 7 As shown, the point cloud acquisition device 70 provided in this application embodiment may further include a communication interface 704.

[0125] Communication interface 704 is used to connect with other devices via a communication network. This communication network can be Ethernet, a wireless access network, a wireless local area network (WLAN), etc. Communication interface 704 may include a receiving unit for receiving data and a transmitting unit for transmitting data.

[0126] In one possible implementation, the communication interface 704 in the point cloud acquisition device 70 provided in this application embodiment can also be integrated into the processor 701, and this application embodiment does not specifically limit this.

[0127] As a possible product form, the point cloud acquisition device of this application embodiment can also be implemented using the following: one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits capable of performing the various functions described throughout this application.

[0128] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional units is used as an example. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the device can be divided into different functional units to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0129] This application also provides a computer-readable storage medium storing a computer program or instructions thereon, which, when executed, causes a computer to perform the various steps in the method flow shown in the above method embodiments.

[0130] Embodiments of this application provide a computer program product containing instructions that, when executed on a computer, cause the computer to perform the various steps in the method flow shown in the above-described method embodiments.

[0131] This application provides a chip system, including: a processor and an interface circuit; the interface circuit is used to receive computer programs or instructions and transmit them to the processor; the processor is used to execute the computer programs or instructions so that the chip system performs each step in the method flow shown in the above method embodiments.

[0132] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), registers, hard disks, optical fibers, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing, or any other form of computer-readable storage medium in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in a purpose-specific ASIC. In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0133] Since the point cloud acquisition device, computer-readable storage medium, and computer program product provided in this embodiment can be applied to the point cloud acquisition method provided in this embodiment, the technical effects that can be obtained can also be referred to the above method embodiments. The embodiments of this application will not be repeated here.

[0134] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0135] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A point cloud acquisition method, characterized in that, The method comprises: acquiring a plurality of image sets and a position relationship set of a target object; the position relationship set comprises position relationships of a plurality of cameras, the plurality of cameras comprising a main camera and at least one slave camera, one image set comprising a depth image captured by one camera and a color image captured by the one camera, the plurality of image sets comprising a main image set and at least one slave image set, the images in the main image set being captured by the main camera, and the images in the slave image set being captured by the slave camera; converting the depth images in the image set corresponding to each camera into the color images in the image set corresponding to the each camera according to an intrinsic matrix of the each camera, to obtain a plurality of local color point clouds; processing each image set and the local color point cloud corresponding to the each image set based on a point cloud registration algorithm and the position relationship set, to obtain a plurality of pose transformation relationships; one pose transformation relationship is used to represent a rotation transformation matrix and a translation transformation matrix between one slave camera and the main camera, and the point cloud registration algorithm is based on image feature learning; determining a voxel set of a three-dimensional space corresponding to the main camera; the voxel set comprises a plurality of voxels; determining a truncated signed distance function (TSDF) value, a color value and a weight value of each voxel in the voxel set corresponding to each local color point cloud, to obtain a plurality of initial voxel sets; deleting an initial voxel in each initial voxel set whose weight value is different from a depth value of a corresponding depth image, to obtain a plurality of intermediate voxel sets; sequentially performing weighted addition of a TSDF value of each intermediate voxel in each intermediate voxel set and a historical TSDF value of the each intermediate voxel, sequentially performing weighted addition of a color value of each intermediate voxel in each intermediate voxel set and a historical color value of the each intermediate voxel, and sequentially performing weighted addition of a weight value of each intermediate voxel in each intermediate voxel set and a historical weight value of the each intermediate voxel, to obtain an initial global point cloud of the target object; determining a global point cloud of the target object according to an implicit surface extraction algorithm and the initial global point cloud of the target object.

2. The method of claim 1, wherein, The processing of each image set and the local color point cloud corresponding to the each image set based on the point cloud registration algorithm and the position relationship set to obtain a plurality of pose transformation relationships specifically comprises: processing the depth images and the color images in the each image set according to a first preset segmentation algorithm, to obtain a target mapping relationship corresponding to the each image set; the target mapping relationship is used to indicate a corresponding relationship between points of the depth images and points of the color images in the each image set; processing the local color point cloud corresponding to the each image set according to the target mapping relationship corresponding to the each image set and a second preset segmentation algorithm, to obtain a color intermediate point cloud corresponding to the each image set; determining an initial pose transformation relationship between two adjacent cameras according to the point cloud registration algorithm and the color intermediate point clouds corresponding to the two adjacent cameras in the color intermediate point cloud, to obtain a plurality of initial pose transformation relationships. determine a pose transformation relationship between a second camera in the plurality of cameras and the main camera according to a pose transformation relationship between a first camera in the plurality of cameras and the main camera, the first camera being a camera closest to the second camera in the plurality of position relationship sets, and the second camera being any one slave camera in the plurality of cameras.

3. The method of claim 1, wherein, The method further includes: converting a depth map in the graph set corresponding to each camera into a color map in the graph set corresponding to each camera according to an intrinsic matrix of each camera, to obtain a plurality of local color point clouds. The method further includes: determining a three-dimensional coordinate of the depth map according to a two-dimensional coordinate of the depth map and an intrinsic matrix of a camera corresponding to the depth map. The method further includes: determining the plurality of local color point clouds according to a rotation transformation matrix and a translation transformation matrix between a color camera and a depth camera in each camera, and the three-dimensional coordinate of the depth map in the graph set corresponding to each camera.

4. A point cloud acquisition apparatus, characterized by comprising: The point cloud acquisition device includes: an acquisition module and a processing module. The acquisition module is configured to acquire a plurality of graph sets and a position relationship set of a target object, the position relationship set including position relationships of a plurality of cameras, the plurality of cameras including a main camera and at least one slave camera, one graph set including a depth map acquired by one camera and a color map acquired by the one camera, the plurality of graph sets including a main graph set and at least one slave graph set, graphs in the main graph set being acquired by the main camera, and graphs in the slave graph set being acquired by the slave camera. The processing module is configured to convert a depth map in a graph set corresponding to each camera into a color map in the graph set corresponding to each camera according to an intrinsic matrix of each camera, to obtain a plurality of local color point clouds. The processing module is further configured to process each graph set and a local color point cloud corresponding to each graph set based on a point cloud registration algorithm and the position relationship set, to obtain a plurality of pose transformation relationships, one pose transformation relationship being used to represent a rotation transformation matrix and a translation transformation matrix between one slave camera and the main camera, and the point cloud registration algorithm being based on graph feature learning. The processing module is further configured to determine a voxel set of a three-dimensional space corresponding to the main camera, the voxel set including a plurality of voxels. The processing module is further configured to determine a truncated signed distance function (TSDF) value, a color value and a weight value of each voxel in the voxel set corresponding to each local color point cloud, to obtain a plurality of initial voxel sets. The processing module is further configured to delete an initial voxel in each initial voxel set, the initial voxel having a weight value different from a depth value of a corresponding depth map, to obtain a plurality of intermediate voxel sets. The processing module is further configured to sequentially perform weighted addition of a TSDF value of each intermediate voxel in each intermediate voxel set and a historical TSDF value of the intermediate voxel, sequentially perform weighted addition of a color value of each intermediate voxel in each intermediate voxel set and a historical color value of the intermediate voxel, and sequentially perform weighted addition of a weight value of each intermediate voxel in each intermediate voxel set and a historical weight value of the intermediate voxel, to obtain an initial global point cloud of the target object. The global point cloud of the target object is determined according to an implicit surface extraction algorithm and an initial global point cloud of the target object.

5. The point cloud acquisition apparatus according to claim 4, characterized in that, The processing module is further configured to process each image set and a corresponding local color point cloud of each image set based on a point cloud registration algorithm and the set of position relationships, to obtain a plurality of pose transformation relationships, specifically including: processing the depth image and the color image in each image set according to a first preset segmentation algorithm, to obtain a target mapping relationship corresponding to each image set; the target mapping relationship is used to indicate the corresponding relationship between the points of the depth image and the points of the color image in each image set; processing the local color point cloud corresponding to each image set according to the target mapping relationship corresponding to each image set and a second preset segmentation algorithm, to obtain a color intermediate point cloud corresponding to each image set; determining an initial pose transformation relationship between two adjacent cameras according to the point cloud registration algorithm and the color intermediate point clouds corresponding to the two adjacent cameras, to obtain a plurality of initial pose transformation relationships; determining a pose transformation relationship between a second camera and the main camera in the plurality of cameras according to the pose transformation relationship between the first camera and the main camera in the plurality of cameras; the first camera is the camera closest to the second camera in the set of position relationships between the main camera and the second camera, and the second camera is any one of the plurality of cameras.

6. The point cloud acquisition apparatus of claim 4, wherein, The processing module is configured to convert the depth image in the image set corresponding to each camera into the color image in the image set according to the intrinsic matrix of each camera, to obtain a plurality of local color point clouds, specifically including: determining the three-dimensional coordinates of the depth image according to the two-dimensional coordinates of the depth image and the intrinsic matrix of the camera corresponding to the depth image; determining a plurality of local color point clouds according to the rotation transformation matrix and the translation transformation matrix between the color camera and the depth camera in each camera and the three-dimensional coordinates of the depth image in the image set corresponding to each camera.

7. A point cloud acquisition apparatus, characterized by comprising: The point cloud acquisition device includes a processor coupled with a memory, and the memory is used to store programs or instructions, when the programs or instructions are executed by the processor, the point cloud acquisition device executes the method in any one of claims 1 to 3.

8. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instructions are executed to make the computer execute the method in any one of claims 1 to 3.

Images