Three-dimensional imaging method and system based on point cloud-image

Abstract

The invention relates to a point cloud-image-based three-dimensional imaging method and system, and belongs to the technical field of optical imaging. Comprising the following steps: inputting an image, and carrying out two-dimensional image instance segmentation on the image to obtain a two-dimensional mask; extracting a point cloud corresponding to the two-dimensional mask from the three-dimensional point cloud data of the image, processing the point cloud and performing three-dimensional modeling to obtain a three-dimensional research target; triangulation is carried out on the three-dimensional research target, and discrete point cloud is converted into a grid structure; projecting the grid structure into a two-dimensional coordinate system, and searching intersection points of edges of triangles in the grid structure and image edge information; wherein the image edge information is obtained by performing edge detection on the two-dimensional research target; the two-dimensional research target is obtained by extracting a two-dimensional mask; and adding the intersection points into the point cloud structure and performing triangulation to complete three-dimensional imaging. According to the method, the three-dimensional space information and the texture features can be obtained at the same time, the difference between the point cloud and the image data scale is reduced, and the loss of the texture information is reduced.

Description

Technical Field

[0001] The present invention relates to the field of optical imaging technology, and in particular to a point cloud-image-based three-dimensional imaging method and system. Background Art

[0002] 3D imaging technology can provide a comprehensive description of real-world objects by combining LiDAR and optical cameras. This technology combines the depth information of point clouds with the texture information of images to provide both spatial information and surface texture of objects. However, due to the difference in scale between point cloud and image data, traditional processing methods face challenges in achieving accurate 3D reconstruction and texture mapping. To address this problem, researchers have developed a point cloud-image pre-fusion technique that fuses point cloud generation and image capture at the initial stage to better preserve the depth information and image texture of each pixel. This technique not only improves the overall performance and robustness of the system, but also has broad application prospects in areas such as autonomous driving, robot navigation, and intelligent monitoring.

[0003] Despite this, existing optical 3D imaging methods often cannot simultaneously obtain 3D spatial information and texture features in the imaging results, and the difference in scale between point cloud and image data has not been effectively solved. In addition, in the 3D imaging method based on point cloud and image registration, due to the discreteness of the point cloud coordinate distribution and the continuity of the image coordinate distribution, a large amount of image texture information is lost during the imaging process, especially the edge features in the image. At the same time, since it is impossible to obtain depth information pixel by pixel, the imaging results of the point cloud and the image are similar, making it impossible for existing methods to fully retain the original texture features when completing the 3D imaging task. Summary of the invention

[0004] To this end, the technical problem to be solved by the present invention is to overcome the inability of the prior art to simultaneously obtain three-dimensional spatial information and texture features, the difference in scale between point cloud and image data, and the loss of texture information, and propose a three-dimensional imaging method and system based on point cloud-image.

[0005] In a first aspect, in order to solve the above technical problems, the present invention provides a three-dimensional imaging method based on point cloud-image, comprising:

[0006] An image is input, and two-dimensional image instance segmentation is performed on the image to obtain a two-dimensional mask;

[0007] Extracting a point cloud corresponding to the two-dimensional mask from the three-dimensional point cloud data of the image, processing the point cloud and performing three-dimensional modeling to obtain a three-dimensional research target;

[0008] Triangulating the three-dimensional research target and converting the discrete point cloud into a grid structure;

[0009] Projecting the grid structure into a two-dimensional coordinate system, and finding the intersection of the edges of the triangles in the grid structure and the image edge information; wherein the image edge information is obtained by performing edge detection on the two-dimensional research target; and the two-dimensional research target is obtained by extracting the two-dimensional mask;

[0010] The intersection points are added to the point cloud structure and triangulated to complete the three-dimensional imaging.

[0011] In one embodiment of the present invention, a method of extracting a point cloud corresponding to the two-dimensional mask from the three-dimensional point cloud data of the image, processing the point cloud and performing three-dimensional modeling to obtain a three-dimensional research target is as follows:

[0012] The point cloud is projected from the three-dimensional coordinate system to the two-dimensional coordinate system. The projection formula is:

[0013]

[0014] Among them, (x i ,y i ,z i ) is the 3D coordinate of a point in the point cloud, (u i ,v i ) is the coordinate of the entire point cloud projected into the two-dimensional coordinate system, K is the intrinsic parameter matrix, R is the rotation matrix, and t is the translation vector;

[0015] Compare the 2D coordinate area covered by the 2D mask and (u i ,v i ), judge (u i ,v i ) is within the area of ​​the two-dimensional mask; if it is within the area of ​​the two-dimensional mask, then find (u i ,v i ) corresponds to (x i ,y i ,z i ), obtain all three-dimensional point clouds projected in the two-dimensional mask;

[0016] According to the clustering state of the points in the point cloud, the cluster with the smallest x value in the x-axis direction is determined as the three-dimensional research target.

[0017] In one embodiment of the present invention, the triangulation method is:

[0018] For each triangle in the grid structure, the area inside the two-dimensional mask and the area outside the two-dimensional mask are calculated using the following formula:

[0019]

[0020] Among them, S n,i is the area within the two-dimensional mask, S n,o is the area outside the two-dimensional mask, T n is one of the triangles in the grid structure, mask represents the two-dimensional mask, and A is the integration object of the double integral.

[0021] In one embodiment of the present invention, for each triangle in the grid structure, the area inside the two-dimensional mask and the area outside the two-dimensional mask are calculated, and when the area inside the two-dimensional mask is not greater than the area outside the two-dimensional mask, the triangle is deleted.

[0022] In one embodiment of the present invention, the method for performing two-dimensional image instance segmentation on the image is a GroundedSAM method.

[0023] In one embodiment of the present invention, the edge detection is Canny edge detection.

[0024] In a second aspect, in order to solve the above technical problem, the present invention provides a computer program product, which includes a computer program. When the computer program is run, the above point cloud-image-based three-dimensional imaging method is executed.

[0025] In a third aspect, in order to solve the above technical problems, the present invention provides a point cloud-image-based three-dimensional imaging system, comprising:

[0026] A two-dimensional mask acquisition module, used for inputting an image, performing two-dimensional image instance segmentation on the image, and obtaining a two-dimensional mask;

[0027] A three-dimensional research target acquisition module, used to extract a point cloud corresponding to the two-dimensional mask from the three-dimensional point cloud data of the image, process the point cloud and perform three-dimensional modeling to obtain a three-dimensional research target;

[0028] A triangulation module, used for triangulating the three-dimensional research target and converting the discrete point cloud into a grid structure;

[0029] An intersection acquisition module is used to project the grid structure into a two-dimensional coordinate system and find the intersection of the edges of the triangles in the grid structure and the image edge information; wherein the image edge information is obtained by performing edge detection on the two-dimensional research target; and the two-dimensional research target is obtained by extracting the two-dimensional mask;

[0030] The three-dimensional imaging module is used to add the intersection point to the point cloud structure and perform triangulation to complete the three-dimensional imaging.

[0031] In a fourth aspect, in order to solve the above-mentioned technical problem, the present invention provides a 3D scanner, including the above-mentioned point cloud-image-based three-dimensional imaging system.

[0032] In a fifth aspect, in order to solve the above-mentioned technical problem, the present invention provides a medical device, including the above-mentioned 3D scanner.

[0033] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0034] The three-dimensional imaging method and system based on point cloud-image described in the present invention adopt edge detection and triangulation technology, and find a unit scale through point cloud to ensure that the image texture within each scale unit is highly similar. This innovative strategy effectively alleviates the mismatch problem between point cloud and image data in terms of order of magnitude, reduces the loss of texture information, and achieves clear and sharp three-dimensional imaging. At the same time, the present invention makes full use of point cloud data and image edge information, synchronously obtains three-dimensional space information and texture features, improves the accuracy and realism of imaging, and provides a solid foundation for high-quality three-dimensional reconstruction. In addition, extracting the point cloud corresponding to the two-dimensional mask from the three-dimensional point cloud data of the image can filter out the three-dimensional point cloud corresponding to the two-dimensional mask from the original three-dimensional point cloud, ensuring the integrity and accuracy of the research target in three-dimensional space. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0036] Figure 1 A flow chart of a three-dimensional imaging method based on point cloud-image in a preferred embodiment of the present invention;

[0037] Figure 2 Schematic diagram of a method for acquiring point clouds within a research area based on a two-dimensional mask in a preferred embodiment of the present invention;

[0038] Figure 3 A schematic diagram of Delauney triangulation correction based on a two-dimensional mask in a preferred embodiment of the present invention;

[0039] Figure 4 A result diagram of a practical application of a point cloud-image-based three-dimensional imaging system in a preferred embodiment of the present invention;

[0040] Figure 5 This is another result diagram of the practical application of a point cloud-image based three-dimensional imaging system in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0042] Reference Figure 1 As shown, an embodiment of the present invention provides a point cloud-image-based three-dimensional imaging method, comprising:

[0043] Input an image, perform two-dimensional image instance segmentation on the image, and obtain a two-dimensional mask;

[0044] Extract the point cloud corresponding to the two-dimensional mask from the three-dimensional point cloud data of the image, process the point cloud and perform three-dimensional modeling to obtain the three-dimensional research target;

[0045] Triangulate the three-dimensional research target and convert the discrete point cloud into a mesh structure;

[0046] Project the grid structure into a two-dimensional coordinate system, and find the intersection of the edges of the triangles in the grid structure and the image edge information; the image edge information is obtained by edge detection of the two-dimensional research target; the two-dimensional research target is obtained by extracting the two-dimensional mask; at this time, the image texture corresponding to each triangle is approximated, and each triangle is a unit scale;

[0047] The intersection points are added to the point cloud structure and triangulated to complete the three-dimensional imaging.

[0048] The embodiment of the present invention provides a point cloud-image-based 3D imaging method, which uses edge detection and triangulation technology, and finds a unit scale through the point cloud, so that the image texture within each unit scale is approximate, thereby alleviating the problem of information loss caused by the mismatch between the point cloud and the image data in order of magnitude, and further achieving clear and sharp 3D imaging. At the same time, the method makes full use of the point cloud data and image edge information, synchronously obtains 3D space information and texture features, improves the accuracy and realism of imaging, and provides a solid foundation for high-quality 3D reconstruction.

[0049] In order to efficiently extract specific 2D and 3D research targets from large-scale scenes, traditional processing methods rely on instance segmentation algorithms for images and point clouds, respectively. However, this method has limitations, mainly because the accumulation of small errors in the processing of multiple neural networks may lead to inconsistent results, and the demand for computing resources is also high. The Grounded SAM (Grounded Spatial Attention Module, referred to as Grounded SAM, Chinese meaning is a text-based full-scene visual model) method effectively reduces the risk of error accumulation, simplifies the processing flow, and reduces the dependence on high computing resources by reducing the steps that rely on multiple neural networks. The Grounded SAM method combines two methods, Grounding DINO and Segment Anything Model (SAM), in which Grounding DINO is responsible for understanding text prompts and locating the corresponding targets in the image, while SAM is responsible for instance segmentation and generating masks. This method not only speeds up data processing by directly mapping 2D masks to 3D space, but also ensures the accuracy of the conversion from 2D to 3D, thereby improving the accuracy of research target extraction. Therefore, in the embodiment of the present invention, the method for performing two-dimensional image instance segmentation on an image is preferably the Grounded SAM method.

[0050] Specifically, in the instance segmentation process, the detection results of Grounding DINO are used as input data, and SAM performs segmentation operations on the image based on these detection results to generate masks for each instance. These masks fit the target instances in the image closely, ensuring the accuracy of the segmentation. Through this coherent segmentation process, one or more two-dimensional masks will be obtained, which represent the target instances segmented in the image. Finally, these masks are used to extract the required image areas, which are the two-dimensional research targets. This method not only improves the efficiency of segmentation, but also enhances the accuracy and reliability of the results.

[0051] After obtaining the 2D mask, the next step is to map the 2D mask into 3D space using the precise projection relationship between the image and the point cloud. This process involves associating the 2D image data with the corresponding 3D point cloud data, so as to accurately locate and extract the point cloud data within the study area in 3D space. Through this mapping, the 3D point cloud corresponding to the 2D mask can be screened out from the original 3D point cloud, ensuring the integrity and accuracy of the research target in 3D space. This method effectively converts 2D information into 3D information, providing rich 3D data support for subsequent analysis and application.

[0052] Specifically, refer to Figure 2 , Figure 2 The figure is a schematic diagram of the method for obtaining point clouds in the research area based on a two-dimensional mask. The point cloud corresponding to the two-dimensional mask is extracted from the three-dimensional point cloud data of the image, and the point cloud is processed and three-dimensionally modeled to obtain the three-dimensional research target. Among them, the processing of the point cloud includes smoothing and denoising to optimize the data quality. The denoising step is dedicated to eliminating outliers and noise in the point cloud, while the smoothing step is used to reduce small-scale fluctuations in the data, providing a continuous and accurate surface representation for subsequent three-dimensional modeling, analysis and visualization. The method for obtaining the three-dimensional research target is:

[0053] S201. According to the projection relationship between the point cloud and the image, the point cloud is projected from the three-dimensional coordinate system to the two-dimensional coordinate system. The projection formula is:

[0054]

[0055] Among them, (x i ,y i ,z i ) is the 3D coordinate of a point in the point cloud; (u i ,v i ) is the coordinate of the entire point cloud projected into the two-dimensional coordinate system; K is the intrinsic parameter matrix, which is determined by the camera that collects the image; R is the rotation matrix; t is the translation vector, which is determined by the pose of the camera and the radar or other equipment that collects the point cloud.

[0056] S202, comparing the two-dimensional coordinate area covered by the two-dimensional mask and (u i ,v i ), judge (u i ,v i ) is within the area of ​​the two-dimensional mask.

[0057] S203, if it is within the area of ​​the two-dimensional mask, then search for (u i ,v i ) corresponds to (x i ,y i ,z i ) to obtain all three-dimensional point clouds projected in the two-dimensional mask, where the same point can be searched consistently according to the index order in the entire array.

[0058] S204. According to the clustering state of the points in the point cloud, determine that the cluster with the smallest x value in the x-axis direction is the three-dimensional research target.

[0059] In three-dimensional space, points in the depth direction (usually the x-axis) of the point cloud may fall within the two-dimensional mask area when projected onto a two-dimensional plane. However, not all of these projected points belong to the object under study. According to the characteristics of point cloud acquisition, only the points of the objects closest to the acquisition device will be visible in the two-dimensional mask, while objects farther away may be blocked. Therefore, in step S204, it is necessary to use the depth information and occlusion relationship of the point cloud to accurately screen out valid points belonging to the object under study to ensure the accuracy and relevance of the data.

[0060] After acquiring the three-dimensional research target, it is triangulated to generate a mesh for three-dimensional modeling. Delaunay triangulation is a widely used method that can quickly generate meshes and ensure that the generated mesh is hole-free, which means that the generated mesh is a complete surface without internal holes. However, it should be noted that the mesh generated by Delaunay triangulation may form a convex hull, which may cause the generated mesh to not completely match the shape of the actual object. The convex hull refers to the smallest convex polygon or convex polyhedron that can enclose all points, and the surface of the actual object may be non-convex, so directly using Delaunay triangulation may cause some geometric distortion. To solve this problem, it is usually necessary to optimize and adjust the results of Delaunay triangulation to better match the shape of the actual object. This may include removing some triangles that do not conform to the surface of the object, or rearranging and connecting the triangles to generate a more accurate and smooth mesh model. In addition, other triangulation algorithms can be used, such as Constrained Delaunay Triangulation, which allows triangulation under certain constraints to better fit the actual boundaries of the object. The embodiment of the present invention preferably modifies the Delauney triangulation by using the obtained two-dimensional mask and formula (1).

[0061] Specifically, refer to Figure 3 , Figure 3 The following is a schematic diagram of Delauney triangulation correction based on a two-dimensional mask. The specific method of triangulation and its correction is:

[0062] For each triangle in the grid structure, the area inside the two-dimensional mask and the area outside the two-dimensional mask are calculated using the following formula:

[0063]

[0064] Among them, S n,i is the area within the two-dimensional mask, S n,o is the area outside the two-dimensional mask, T nis one of the triangles in the grid structure, mask represents the two-dimensional mask, and A is the double integral integration object. When the area inside the two-dimensional mask is not larger than the area outside the two-dimensional mask, that is, S n,i ≤S n,o , delete this triangle.

[0065] Furthermore, the grid structure is projected into a two-dimensional coordinate system. In order to accurately align the three-dimensional grid with the two-dimensional image data, it is first necessary to perform edge detection on the two-dimensional research target and extract the edge information in the image. Secondly, the edges of all triangles are extracted from the grid structure of the three-dimensional model. By calculating the intersection points of these edges with the image edges, the correspondence between the three-dimensional structure and the two-dimensional image is determined. This method improves the accuracy and reliability of three-dimensional reconstruction and image analysis.

[0066] Among them, in the selection of edge detection algorithms, the Canny algorithm stands out due to its excellent performance. It can not only accurately identify the real edges in the image, but also accurately locate the edge pixels. Through the preprocessing of the Gaussian filter, the Canny algorithm significantly reduces the influence of image noise, thereby improving the accuracy of edge detection. In addition, the Canny algorithm provides a flexible parameter adjustment mechanism, including the standard deviation and dual threshold setting of the Gaussian filter, so that users can optimize the edge detection results according to specific application scenarios. Its original design intention is to ensure the high accuracy of the detection results by minimizing the edges of false detection and false positioning, so that the detected edges are as close to the actual edges as possible. Therefore, the embodiment of the present invention preferably uses the Canny edge detection method to obtain image edge information.

[0067] Embodiment 2

[0068] This embodiment provides a computer program product, which includes a computer program. When the computer program is run, the point cloud-image-based three-dimensional imaging method described in the first embodiment is executed.

[0069] Embodiment 3

[0070] Based on the same inventive concept, this embodiment provides a point cloud-image-based three-dimensional imaging system, and its principle of solving the problem is similar to the point cloud-image-based three-dimensional imaging method provided in Example 1, and the repeated parts are not repeated here.

[0071] An embodiment of the present invention provides a point cloud-image based three-dimensional imaging system, comprising:

[0072] A two-dimensional mask acquisition module is used to input an image, perform two-dimensional image instance segmentation on the image, and obtain a two-dimensional mask;

[0073] A three-dimensional research target acquisition module is used to extract the point cloud corresponding to the two-dimensional mask from the three-dimensional point cloud data of the image, process the point cloud and perform three-dimensional modeling to obtain the three-dimensional research target;

[0074] The triangulation module is used to triangulate the three-dimensional research target and convert the discrete point cloud into a mesh structure;

[0075] The intersection acquisition module is used to project the grid structure into a two-dimensional coordinate system and find the intersection of the edges of the triangles in the grid structure and the image edge information; wherein the image edge information is obtained by edge detection of the two-dimensional research target; and the two-dimensional research target is obtained by two-dimensional mask extraction;

[0076] The 3D imaging module is used to add intersection points to the point cloud structure and perform triangulation to complete 3D imaging.

[0077] Reference Figure 4 and Figure 5 , Figure 4 and Figure 5 Two result graphs generated by the point cloud-image 3D imaging system in actual application. It can be clearly seen from the figure that the system provided by the embodiment of the present invention can not only capture 3D spatial information and texture features at the same time, but also effectively reduce the scale difference between point cloud and image data, thereby preventing the loss of texture information.

[0078] Embodiment 4

[0079] This embodiment provides a 3D scanner, including a point cloud-image-based three-dimensional imaging system provided in Embodiment 3.

[0080] Embodiment 5

[0081] This embodiment provides a medical device, including a 3D scanner provided in Embodiment 4.

[0082] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0083] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0084] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0086] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A three-dimensional imaging method based on point cloud-image, characterized in that: include: An image is input, and two-dimensional image instance segmentation is performed on the image to obtain a two-dimensional mask; Extracting a point cloud corresponding to the two-dimensional mask from the three-dimensional point cloud data of the image, processing the point cloud and performing three-dimensional modeling to obtain a three-dimensional research target; Triangulating the three-dimensional research target and converting the discrete point cloud into a grid structure; Projecting the grid structure into a two-dimensional coordinate system, and finding the intersection of the edges of the triangles in the grid structure and the image edge information; wherein the image edge information is obtained by performing edge detection on the two-dimensional research target; and the two-dimensional research target is obtained by extracting the two-dimensional mask; The intersection points are added to the point cloud structure and triangulated to complete the three-dimensional imaging.

2. A point cloud-image-based three-dimensional imaging method according to claim 1, characterized in that: A method of extracting a point cloud corresponding to the two-dimensional mask from the three-dimensional point cloud data of the image, processing the point cloud and performing three-dimensional modeling to obtain a three-dimensional research target is as follows: The point cloud is projected from the three-dimensional coordinate system to the two-dimensional coordinate system. The projection formula is: Among them, (x i ,y i , z i ) is the 3D coordinate of a point in the point cloud, (u i , v i ) is the coordinate of the entire point cloud projected into the two-dimensional coordinate system, K is the intrinsic parameter matrix, R is the rotation matrix, and t is the translation vector; Compare the 2D coordinate area covered by the 2D mask and (u i , v i ), judge (u i , v i ) is within the area of ​​the two-dimensional mask; if it is within the area of ​​the two-dimensional mask, then find (u i , v i ) corresponds to (x i ,y i , z i ), obtain all three-dimensional point clouds projected in the two-dimensional mask; According to the clustering state of the points in the point cloud, the cluster with the smallest x value in the x-axis direction is determined as the three-dimensional research target.

3. The point cloud-image-based three-dimensional imaging method according to claim 1, characterized in that: The triangulation method is: For each triangle in the grid structure, the area inside the two-dimensional mask and the area outside the two-dimensional mask are calculated using the following formula: Among them, S n,i is the area within the two-dimensional mask, S n,o is the area outside the two-dimensional mask, T n is one of the triangles in the grid structure, mask represents the two-dimensional mask, and A is the integration object of the double integral.

4. The point cloud-image-based three-dimensional imaging method according to claim 3, characterized in that: For each triangle in the grid structure, the area inside the two-dimensional mask and the area outside the two-dimensional mask are calculated, and when the area inside the two-dimensional mask is not greater than the area outside the two-dimensional mask, the triangle is deleted.

5. The point cloud-image-based three-dimensional imaging method according to claim 1, characterized in that: The method for performing two-dimensional image instance segmentation on the image is the Grounded SAM method.

6. The point cloud-image-based three-dimensional imaging method according to claim 1, characterized in that: The edge detection is Canny edge detection.

7. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is run, the method according to any one of claims 1 to 6 is executed.

8. A three-dimensional imaging system based on point cloud-image, characterized in that: include: A two-dimensional mask acquisition module, used for inputting an image, performing two-dimensional image instance segmentation on the image, and obtaining a two-dimensional mask; A three-dimensional research target acquisition module, used to extract a point cloud corresponding to the two-dimensional mask from the three-dimensional point cloud data of the image, process the point cloud and perform three-dimensional modeling to obtain a three-dimensional research target; A triangulation module, used for triangulating the three-dimensional research target and converting the discrete point cloud into a grid structure; An intersection acquisition module is used to project the grid structure into a two-dimensional coordinate system and find the intersection of the edges of the triangles in the grid structure and the image edge information; wherein the image edge information is obtained by performing edge detection on the two-dimensional research target; and the two-dimensional research target is obtained by extracting the two-dimensional mask; The three-dimensional imaging module is used to add the intersection points to the point cloud structure and perform triangulation to complete the three-dimensional imaging.

9. A 3D scanner, characterized in that: Including a point cloud-image based three-dimensional imaging system as described in claim 8.

10. A medical device, characterized in that: A 3D scanner comprising the method described in claim 9.

Images