Three-dimensional scanning method, scanning device, computing device, and non-transitory storage medium

Abstract

The application discloses a three-dimensional scanning method, a scanning device, a computing device and a nonvolatile storage medium. The method comprises the following steps: acquiring a plurality of meshes generated by a scanning device through multi-round scanning of an object, aligning the plurality of meshes to the same coordinate system; splicing the plurality of aligned meshes to obtain a spliced mesh, determining the visibility of each patch in the spliced mesh corresponding to a texture image acquired in the multi-round scanning; and rendering the patch using at least one visible texture image. The application solves the technical problem that the background and the scanned object are prone to misalignment when the multi-project files generated by multi-round scanning in a dynamic environment are fused in the related art.

Description

Technical Field

[0001] This application relates to the field of three-dimensional imaging technology, and more specifically, to a three-dimensional scanning method, scanning device, computing device, and non-volatile storage medium. Background Technology

[0002] In related technologies, 3D reconstruction of the same object requires multiple rounds of 3D scanning to obtain multiple frames of point clouds and textures. Texture and point cloud fusion typically employs a direct fusion strategy, where the texture image obtained from each scan is fused onto the final constructed 3D mesh model based on the corresponding point cloud data. This process relies on comparing the point cloud frame depth with the mesh rendering depth to determine whether to select a texture, aiming to avoid fusion of areas with excessively large depth differences, thereby reducing misalignment. However, this technical approach exhibits significant limitations in dynamic environments with multiple rounds of scanning, easily leading to misalignment between the background and the scanned object. Summary of the Invention

[0003] This application provides a three-dimensional scanning method, scanning device, computing device, and non-volatile storage medium to at least solve the technical problem in related technologies where the background and scanned object are easily misaligned when merging multiple engineering files generated by multiple rounds of scanning in a dynamic environment.

[0004] According to one aspect of the embodiments of this application, a three-dimensional scanning method is provided, comprising: acquiring multiple meshes generated by a scanning device performing multiple scans on an object; aligning the multiple meshes to the same coordinate system; stitching the aligned multiple meshes to obtain a stitched mesh; determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in the multiple scans; and rendering the facet using at least one visible texture image.

[0005] Optionally, it also includes: aligning multiple meshes to the same coordinate system based on the relative pose transformation relationship between the multiple meshes; and aligning the texture image to the same coordinate system.

[0006] Optionally, the same coordinate system is the coordinate system of the grid generated by any one of the multiple scans.

[0007] Optionally, each round of scanning includes multiple scanning processes, in which a point cloud image and a texture image of the object are acquired using a scanning device in each scanning process, and a mesh is generated based on the multiple point cloud images acquired in each round of scanning.

[0008] Optionally, the multi-round scan is performed in at least two spatial environments, which are at least partially different from each other.

[0009] Optionally, the multi-round scan includes a first round scan performed in a first spatial environment and a second round scan performed in a second spatial environment, and the multiple grids include a first grid generated by the first round scan and a second grid generated by the second round scan, wherein the first spatial environment and the second spatial environment are at least partially different.

[0010] Optionally, determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in multiple scans includes: for a first facet of the current mesh in the stitched mesh, in response to a second facet of at least one other mesh being searchable around the first facet, determining that the first facet is visible to the first texture image corresponding to the first facet and the second texture image corresponding to the second facet; for a first facet of the current mesh in the stitched mesh, in response to a second facet of no other mesh being searchable around the first facet, determining that the first facet is visible only to the first texture image corresponding to the first facet.

[0011] Optionally, determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in multiple scans includes: for a first facet of the current mesh in the stitched mesh, determining the positional relationship between the first facet and a second facet of at least one other mesh; and determining the visibility of the first facet to the second texture image corresponding to the second facet based on the positional relationship.

[0012] Optionally, determining the visibility of the first patch to the second texture image corresponding to the second patch based on the positional relationship includes: determining the first patch center and the first normal vector of the first patch, and the second patch center and the second normal vector of the second patch; determining the center distance between the first patch center and the second patch center, and determining the normal cosine value between the first normal vector and the second normal vector; determining that the first patch is not visible to the second texture image corresponding to the second patch in response to the center distance being greater than a preset distance threshold, or the normal cosine value being less than a preset cosine value threshold; and determining the visibility of the first patch to the second texture image corresponding to the second patch based on the projection relationship between the second patch center and the first patch in response to the center distance not being greater than the preset distance threshold and the normal cosine value not being less than the preset cosine value threshold.

[0013] Optionally, determining the visibility of the second texture image corresponding to the second patch to the first patch based on the projection relationship between the center of the second patch and the first patch includes: projecting the center of the second patch onto a target plane along the direction of the second normal vector to obtain a projection intersection point, wherein the target plane is a plane centered on the center of the first patch and with the direction of the first normal vector as its normal; determining that the first patch is visible to the second texture image corresponding to the second patch in response to the projection intersection point being located inside the first patch; and determining that the first patch is not visible to the second texture image corresponding to the second patch in response to the projection intersection point not being located inside the first patch.

[0014] Optionally, rendering a face using at least one visible texture image includes: determining one or more faces corresponding to the visible texture image; and projecting and rendering the one or more faces based on relevant parameters of the visible texture image, wherein the relevant parameters include at least one of the following: pose parameters of the texture image and texture camera intrinsics.

[0015] According to another aspect of the embodiments of this application, a scanning device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes a three-dimensional scanning method when it runs.

[0016] According to another aspect of the embodiments of this application, a computing device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes a three-dimensional scanning method during runtime.

[0017] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device containing the non-volatile storage medium executes a three-dimensional scanning method by running the computer program.

[0018] In this embodiment, multiple meshes generated by multiple scans of an object using a scanning device are acquired and aligned to the same coordinate system. The aligned meshes are then stitched together to obtain a stitched mesh. The visibility of each facet in the stitched mesh corresponding to the texture image acquired in the multiple scans is determined. The facets are rendered using at least one visible texture image. By accurately calculating the visibility of the facets, the goal of ensuring that the texture of the scanned object can be accurately stitched to the corresponding facets in different scan cycles is achieved. This solves the technical problem in related technologies where the background and the scanned object are easily misaligned when merging multiple project files generated by multiple scans in a dynamic environment. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1A This is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a three-dimensional scanning method according to an embodiment of this application;

[0021] Figure 1B This is a schematic diagram of the structure of a scanning device for implementing a three-dimensional scanning method according to an embodiment of this application;

[0022] Figure 2This is a schematic diagram of a three-dimensional scanning method flow according to an embodiment of this application;

[0023] Figure 3A This is a schematic diagram illustrating an example of a scanning environment according to an embodiment of this application;

[0024] Figure 3B This is a schematic diagram illustrating another example of a scanning environment provided according to an embodiment of this application;

[0025] Figure 4A This is a schematic diagram illustrating the effect of individual texture fusion in an engineering project 1 according to an embodiment of this application;

[0026] Figure 4B This is a schematic diagram illustrating the effect of individual texture blending in Project 2 according to an embodiment of this application;

[0027] Figure 5A This is a schematic diagram of a direct texture fusion result of Project 1 + Project 2 according to an embodiment of this application;

[0028] Figure 5B This is a schematic diagram of a multi-engineered texture fusion result provided according to an embodiment of this application;

[0029] Figure 6 This is a schematic diagram of a three-dimensional scanning device provided according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] When performing a full-range scan of the same object, multiple scan rounds are often required. For example, the object's exposed outer surface in the environment may be scanned first. Then, the object may be flipped or inverted, and the newly exposed outer surface may be scanned. The results of multiple scan rounds are then stitched together to generate a 3D mesh of the object. Finally, a suitable point cloud frame (image) is determined based on the point cloud frame and the depth information of the 3D mesh from each scan, and the corresponding texture frame (image) is stitched onto the 3D mesh. In some examples, the scanning environment may change. When the scanning environment changes, such as when the object moves from one room to another, even if the depth information of the point cloud frame and the 3D mesh is similar, differences in background texture may cause significant misalignment after texture fusion, affecting the visual effect of the final model. In addition, if the position of the scanned object changes slightly in different scan rounds, it is difficult to accurately align and fuse textures, resulting in texture misalignment of the target object.

[0033] To address the aforementioned issues, this application provides a solution that accurately selects textures for fusion by determining the poses between meshes obtained from multiple scans and the visibility of textures to inner mesh patches, thereby preventing misalignment between the texture background and the target object. This will be described in detail below.

[0034] According to an embodiment of this application, a method embodiment for three-dimensional scanning is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] The method embodiments provided in this application can be executed in a mobile terminal, computer terminal, scanning device or similar computing device. Figure 1A A hardware structure block diagram of a computer terminal (or electronic device) for implementing a 3D scanning method is shown. Figure 1A As shown, the computer terminal 100 (or electronic device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1AThe structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, the computer terminal 100 may also include... Figure 1A The more or fewer components shown, or having the same Figure 1A The different configurations shown.

[0036] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element within the computer terminal 100 (or electronic device). As involved in the embodiments of this application, the data processing circuit serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the three-dimensional scanning method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned three-dimensional scanning method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 100 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 100. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0039] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 100 (or electronic device).

[0040] Figure 1BA schematic diagram of an exemplary scanning device 10 is shown. In the depicted embodiment, the scanning device 10 includes a frame structure 20, an imaging module 30 located within the frame structure 20, and a display screen 40. The imaging modules 30 may be arranged in an interleaved manner, such that the fields of view of each imaging module at least partially overlap. In some embodiments, the imaging module 30 may include five cameras, namely a first camera 31, a second camera 32, a third camera 33, a fourth camera 34, and a fifth camera 35. The imaging module 30 may also include two sets of light projectors, namely a first light projector group 36 and a second light projector group 37, each light projector group may include one or more light projectors, for example, three light projectors. Each light projector may include a light source, a collimating lens, and a diffractive optical element. The light source is configured to emit a light beam toward the lens. The light beam, collimated by the collimating lens, propagates to the diffractive optical element, which replicates the focused light beam to form a speckle pattern or fringe pattern and projects it onto the scanned object / area. The speckle image or fringe pattern is reflected back from the scanned object / area and acquired by the imaging module 30 for further processing by a processing chip (not shown) to obtain the three-dimensional information of the scanned object / area. In other embodiments, the light projector may be an image projector, such as a digital micromirror device, a liquid crystal display projector, or an organic electroluminescent display projector.

[0041] In some embodiments, the light projectors of the first light projector group 36 may include a single identical light source, such as a light source emitting infrared light, white light, blue light, or other visible monochromatic light. In other embodiments, multiple light projectors of the first light projector group 36 are configured to emit light with wavelengths between 405 nm and 1100 nm, for example, all are configured to emit near-infrared light with wavelengths between 780 nm and 1100 nm. In still other embodiments, the light projectors of the first light projector group 36 may include multiple different light sources, for example, one or two light projectors include a light source emitting infrared light, for example, one or two light projectors include a light source emitting near-infrared light with wavelengths between 780 nm and 1100 nm, and other light projectors include a light source emitting blue light or white light. For example, the first light projector of the first light projector group 36 includes a first light source emitting near-infrared light, the second light projector of the first light projector group 36 includes a second light source emitting white light, and the third light projector of the first light projector group 36 includes a third light source emitting blue light; or the first projector of the first light projector group 36 includes a first light source emitting near-infrared light, the second light projector of the first light projector group 36 includes a second light source emitting near-infrared light, and the third light projector of the first light projector group 36 includes a third light source emitting white / blue light. In some examples, the multiple projectors of the first light projector group 36 can be implemented as separate units, or as separate units in a region.

[0042] In some embodiments, the light projectors of the second light projector group 37 may include a single identical light source, such as a light source emitting infrared light, white light, blue light, or other visible monochromatic light. In other embodiments, multiple light projectors of the second light projector group 37 are configured to emit light with wavelengths between 405 nm and 1100 nm, for example, all configured to emit blue light with wavelengths between 435 nm and 480 nm. In still other embodiments, the light projectors of the second light projector group 37 may include multiple different light sources, for example, one or two light projectors include a light source emitting blue light, for example, one or two light projectors include a light source emitting blue light with wavelengths between 435 nm and 480 nm, and other light projectors include a light source emitting near-infrared light or white light. For example, the first light projector of the second light projector group 37 includes a first light source emitting blue light, the second light projector of the second light projector group 37 includes a second light source emitting white light, and the third light projector of the second light projector group 37 includes a third light source emitting near-infrared light; or the first projector of the second light projector group 37 includes a first light source emitting blue light, the second light projector of the second light projector group 37 includes a second light source emitting blue light, and the third light projector of the second light projector group 37 includes a third light source emitting white / near-infrared light. In some examples, the multiple projectors of the second light projector group 37 can be implemented as separate units, or as separate units in a region.

[0043] In some embodiments, the first light projector group 36 and the second light projector 37 can be configured to be selectively turned on or off to adapt to different scanning modes of the scanning device 10.

[0044] The first camera 31, the second camera 32, the third camera 33, and the fourth camera 34 are typically monochrome (e.g., black and white) cameras, and their types depend on the types of light sources(s) used in the first light projector group 36 and the second light projector group 37. In some embodiments, the first camera 31, the second camera 32, the third camera 33, and the fourth camera 34 may be monochrome, visible spectrum, or near-infrared cameras, and the light projector of the first light projector group 36 is an infrared light projector or a near-infrared light projector, while the light projector of the second light projector group 37 is a blue light projector.

[0045] In some embodiments, a first camera 31 and a second camera 32 are used together as a first camera combination, and a third camera 33 and a fourth camera 34 are used together as a second camera combination. The first camera 31 and the second camera 32 are configured to have the same focal length, e.g., a closer focal length or a farther focal length, and the third camera 33 and the fourth camera 34 are configured to have the same focal length, e.g., a farther focal length or a closer focal length, and the focal lengths of the first camera combination and the second camera combination are not the same. For example, the first camera combination or the second camera combination can be selected to capture the reflections on the object from either a set of light projectors based on the adapted focal length, and the light projector set used can also be selected based on, for example, the scanning speed, scanning accuracy required by the environment, and the characteristics of the object.

[0046] In some embodiments, the first camera 31, the second camera 32, the third camera 33, and the fourth camera 34 may use any suitable shutter technology, including but not limited to rolling shutters, global shutters, mechanical shutters, and optical liquid crystal display (LCD) shutters. In some embodiments, the fifth camera 35 may be a color camera (also known as a texture camera). The texture camera may use any suitable shutter technology, including but not limited to rolling shutters, global shutters, mechanical shutters, and optical liquid crystal display (LCD) shutters. In some embodiments, the first camera 31, the second camera 32, the third camera 33, the fourth camera 34, and the fifth camera 35 may have similar configurations to improve matching confidence and speed. In some embodiments, the imaging module 30 may also be able to use fewer projector groups, such as only one projector group, and fewer cameras, such as only two cameras, to complete the acquisition of reflected light and color textures, for example, omitting the second projector group 37 (first projector group 36), and omitting the third camera 33 (first camera 31) and the fourth camera 34 (second camera 32).

[0047] like Figure 1B As shown, the first camera 31, the second camera 32, and the first light projector group 36 can be located side-by-side on one surface of the frame structure 20, with the first camera 31, the second camera 32, and the first light projector group 36 spaced apart from each other and all facing directly forward of the surface. The third camera 33, the fourth camera 34, and the second light projector group 37 can be located side-by-side on one surface of the frame structure 20, with the third camera 33, the fourth camera 34, and the second light projector group 37 spaced apart from each other and all facing directly forward of the surface.

[0048] A data connection (such as a serial communication connection) between the scanning device 10 and one or more computer processors (not shown) allows the transmission of data collected by the first camera 31, second camera 32, third camera 33, fourth camera 34, and fifth camera 35, enabling it to be processed to derive 3D measurements of the surface of the scanned object / object. The one or more computer processors may be implemented in a remote computing system (electronic device), or alternatively, may be part of the scanning device 10 itself.

[0049] For example, each light projector group may include a single light projector, or may have two or more (e.g., three) light projectors. The light projectors may be configured to project visible or invisible light, coherent or incoherent light. In some embodiments, the light projector may include one or more light sources consisting of lasers (e.g., vertical-cavity surface-emitting lasers (VCSELs), edge-emitting lasers (EELs), solid-state lasers, and semiconductor lasers) and / or one or more LEDs (or OLEDs).

[0050] A light projector can be configured to project a structured light pattern consisting of multiple light sheets arranged side-by-side. When the light sheets are projected onto the surface of an object, they can appear as elongated light stripes. These elongated light stripes are non-intersecting and, in some embodiments, can be substantially parallel to each other. In some embodiments, the light projector can be a programmable light projector unit capable of projecting more than one light pattern. For example, the light projector can be configured to project different structured line patterns. In some embodiments, the light projector can emit light with wavelengths between 405 nm and 1100 nm.

[0051] In some examples, two images of an object can be captured simultaneously using a first camera 31, a second camera 32, or a third camera 33 and a fourth camera 34. Image processing can be applied, for example, to computational methods implemented by one or more processors, or to computational methods implemented, for example, by electronic devices, to derive 3D measurements of the surface of the scanned object / object.

[0052] In some examples, the second camera 32 can capture the texture of the object while the first camera 31, the second camera 32, or the third camera 33, the fourth camera 34 are capturing the object image, and the texture can be applied to a computational method, for example, implemented by one or more processors, or to a computational method, for example, implemented by an electronic device, to map onto a 3D measurement of the surface of the scanned object / object.

[0053] In some examples, using a membrane / film with bandpass filter functionality fixed to the lens of a camera (e.g., first camera 31, second camera 32, third camera 33, and fourth camera 34) can match the wavelength of (multiple) projectors, which can help reduce light source interference from ambient light and other projectors.

[0054] In some examples, a calibration plate, such as a single plate or a set of plates whose true geometric distance values ​​have been measured in advance using high-precision methods such as photogrammetry, is used to measure the intrinsic and extrinsic parameters of the first camera 31, the second camera 32, the third camera 33, the fourth camera 34, and the fifth camera 35. The measurement process typically involves a series of consecutive image acquisitions by the calibration cameras of the scanning device 10 after adjusting the calibration plate to different positions, and the calculation of the spatial position and orientation of the first camera 31, the second camera 32, the third camera 33, the fourth camera 34, and the fifth camera 35 by identifying the positions of reference (marked) points / regions / lines in the calibration images, thereby completing the calibration of the intrinsic and extrinsic parameters of the first camera 31, the second camera 32, the third camera 33, the fourth camera 34, and the fifth camera 35.

[0055] In some examples, fill lights (38, 39) composed of light-emitting diodes, for example, are used to illuminate the first camera 31, the second camera 32, the third camera 33, and the fourth camera 34. The fill lights may include one or more, configured to surround the cameras (e.g., uniformly surround), and may be configured to emit different types of light, such as white light, infrared light, blue light, or ultraviolet light. In other examples, the fifth camera 35 may also be illuminated using, for example, a white fill light.

[0056] In some examples, display screen 40 is configured to display a 3D model of the object captured by imaging module 30. Display screen 40 may be positioned opposite to imaging module 30. Display screen 40 may include, for example, touch functionality or physical buttons / touch areas attached to the area surrounding display screen 40, allowing users to move, zoom the 3D model of the object, or turn display screen 40 on / off using touch. In some examples, display screen 40 may be a liquid crystal display (LCD), organic light-emitting diode (OLED), active-matrix organic light-emitting diode (AMOLED), flexible light-emitting diode (FLED), Mini LED, Micro LED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc.

[0057] Under the above operating environment, this application provides a three-dimensional scanning method. Figure 2 This is a schematic diagram of a three-dimensional scanning method flow according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0058] Step S202: Obtain multiple grids generated by the scanning device through multiple rounds of scanning of the object, and align the multiple grids to the same coordinate system;

[0059] By acquiring multiple meshes generated through multiple rounds of scanning and aligning them to the same coordinate system, coordinate system matching between different meshes was achieved, laying the foundation for subsequent stitching and texture fusion.

[0060] Step S204: After stitching and aligning multiple grids, a stitched grid is obtained. The visibility of each facet in the stitched grid corresponding to the texture image obtained in multiple scans is determined.

[0061] It's understandable that each scan generates multiple point cloud images and corresponding texture images. Related techniques typically fuse the texture images into appropriate positions on the final mesh based on the matching between the point cloud images and the stitched mesh. This matching can include positional matching and depth matching. Specifically, first, a point cloud image with accurate positioning is selected from the multiple scans based on the part of the mesh to be rendered. Then, the depth of the accurately positioned point cloud image is compared with the depth of the part of the mesh to be rendered. When the depth is suitable, the texture image corresponding to that accurately positioned point cloud image is selected and fused with the part of the mesh to be rendered, thus achieving the effect of texture mapping.

[0062] In other words, when multiple scans are performed on the same object, there may be multiple point cloud images that match in both position and depth. However, because the background may change during multiple scans, the texture images corresponding to these multiple point cloud images may be different. Related technologies often choose to fuse these multiple texture images corresponding to the multiple point cloud images onto a stitched mesh, which results in a noticeable background misalignment displayed on a single stitched mesh.

[0063] A stitched mesh is formed by stitching and aligning the meshes, and then the visibility of the texture image corresponding to each facet in the stitched mesh is determined across all scanning rounds. This process requires precise determination of whether a facet should select a texture from the texture image of a particular scanning round, avoiding texture mismatch caused by background changes.

[0064] Step S206: Render the face using at least one visible texture image.

[0065] Based on the visibility information of the facets, at least one visible texture image is used to render the facets, ensuring the consistency and accuracy of the textures. Even when the environment changes significantly, since only the facets in the stitching mesh are selected for texture stitching, background changes can be ignored to achieve accurate alignment of texture blending.

[0066] In other words, when fusing meshes from multiple scans to generate a stitched mesh, the object / object portion of the stitched mesh can originate from the meshes from multiple scans. For example, the object / object in the stitched mesh might come from two different scans. For instance, the first part of the object / object in the stitched mesh might come from the first scan mesh, and the second part might come from the second scan mesh. Then, the texture image from the first scan is used to render the first part, and the texture image from the second scan is used to render the second part.

[0067] It is understandable that by determining the visibility between the facets of the first part and the texture image of the first scan, and the visibility between the facets of the second part and the texture image of the second scan, the texture image can be correctly selected to render the facets of objects in the stitched mesh.

[0068] In multi-round scans where the background changes, to avoid background misalignment, the background portion of the stitched mesh typically originates from any one of the scan rounds—for example, all from the first scan round or all from the second scan round. Then, a texture image from the same scan round is used to render the background portion of the stitched mesh. For instance, based on the visibility between the background patches and the texture image of the corresponding scan round, the appropriate texture image is selected to render the background patches in the stitched mesh.

[0069] In some examples, the final result will be a textured version of a stitched mesh, where the images of objects are from multiple scans, while the images of the background are from any single scan.

[0070] In some examples, each scan can generate at least one project file, which retains the mesh of the object generated in that scan, such as at least a portion of the object's mesh. Combining multiple project files from multiple scans enables omnidirectional, discontinuous, and moving 3D reconstruction of the object. Through these steps, by accurately calculating the visibility of textures across facets, the consistency of the target object's texture across different scan cycles is ensured. This solves the technical problem in related technologies where misalignment between the background and the scanned object easily occurs when merging multiple project files generated from multiple scans in dynamic environments.

[0071] The three-dimensional scanning method in steps S202 to S206 of the embodiments of this application will be further described below.

[0072] First, acquire multiple grids generated by the scanning device scanning the object in multiple rounds.

[0073] In some embodiments of this application, each round of scanning includes multiple scanning processes. A point cloud image and a texture image of the object are acquired using a scanning device in each scanning process, and a mesh is generated based on the multiple point cloud images acquired in each round of scanning.

[0074] Specifically, the scanning device can acquire a point cloud image and a corresponding texture image of the object in each scanning process. Then, a mesh is generated based on the set of point cloud images acquired in each round of scanning.

[0075] It should be noted that in some embodiments of this application, multi-round scanning is performed in at least two spatial environments, and these at least two spatial environments are at least partially different. That is, multi-round scanning in the embodiments of this application is performed in at least two spatial environments that are partially different. For example, during the multi-round scanning process, the spatial layout around the scanned object changes or becomes completely different.

[0076] For example, in some embodiments of this application, multi-round scanning may include a first round of scanning performed in a first spatial environment and a second round of scanning performed in a second spatial environment. Multiple meshes include a first mesh generated by the first round of scanning and a second mesh generated by the second round of scanning, wherein the first spatial environment and the second spatial environment are at least partially different. Subsequently, the multiple meshes generated by the multi-round scanning need to be aligned to the same coordinate system. The specific steps are as follows.

[0077] In some embodiments of this application, the following steps are also included: aligning multiple meshes to the same coordinate system according to the relative pose transformation relationship between multiple meshes; and aligning the texture image to the same coordinate system.

[0078] Specifically, the common parts of multiple scans can be used as a reference to calculate the relative pose transformation relationship between the meshes obtained from multiple scans, unify all meshes to the same coordinate system, thereby ensuring that the models of different scan cycles can be accurately aligned; at the same time, all texture images are also aligned to the same coordinate system.

[0079] For example, consider two rounds of scanning conducted in at least two partially different spatial environments. Assume the first round of scanning reconstructs multiple point cloud images into a grid m1, corresponding to a local coordinate system g1, and the second round reconstructs multiple point cloud images into a grid m2, corresponding to a local coordinate system g2. Then, a feature point matching algorithm can be used to find corresponding points between the two grids, and the relative pose transformation relationship T between the two grids can be calculated based on these corresponding points. g1g2 Then, the poses of all single frames in the mesh m1 obtained from the second scan are transformed to the local coordinate system of the first scan. Assume that a single frame in m2 is cj, and its pose is T. g2cj The updated pose is T. g1g2 ×T g2cj .

[0080] It should be noted that, in some embodiments of this application, the same coordinate system can be the coordinate system of the mesh generated by any one of the multiple scan rounds. In the example above, the local coordinates g1 corresponding to the first scan round are used as the coordinate system for alignment. In reality, it could also be the local coordinates g2 corresponding to the second scan round, or the coordinate system corresponding to any one of the subsequent scan rounds. This application does not limit this.

[0081] Through the above steps, all the scanned meshes are aligned to the same coordinate system, ensuring the consistency of the mesh models in spatial position and direction across different scan cycles. This allows them to be stitched together into a complete mesh (also known as a 3D model), providing a solid foundation for subsequent patch visibility analysis. This enables the comparison and matching between patches to be performed within the same reference frame, improving the accuracy and reliability of patch visibility analysis.

[0082] After aligning multiple grids to the same coordinate system, a stitched grid can be obtained by stitching the aligned grids together. Then, it is necessary to determine the visibility of each patch in the stitched grid corresponding to the texture image obtained in multiple scans, that is, to determine whether the patch is visible in different scan cycles, as follows.

[0083] In some embodiments of this application, determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in multiple scans includes: for a first facet of the current mesh in the stitched mesh, in response to a second facet of at least one other mesh being searchable around the first facet, determining that the first facet is visible to the first texture image corresponding to the first facet and the second texture image corresponding to the second facet; for a first facet of the current mesh in the stitched mesh, in response to a second facet of no other mesh being searchable around the first facet, determining that the first facet is visible only to the first texture image corresponding to the first facet.

[0084] In this embodiment, visibility can be determined by performing nearest neighbor search matching on the facets in the stitched network. Specifically, when determining the visibility of each facet in the stitched mesh corresponding to the texture image obtained in multiple scans, for any first facet f1 of the current mesh (taking mesh m1 obtained in the first scan as an example), if at least one second facet f2 from other meshes (e.g., mesh m2 obtained in the second scan) can be found around it, then f1 is determined to be visible to the texture images corresponding to f1 and f2 respectively; otherwise, if no f2 can be found around f1, then f1 is only considered to be visible to its own texture image.

[0085] It is understood that the definition of a visible texture image described in this article is that a visible texture image can be rendered onto the corresponding texture image. Typically, a visible texture image is visible to multiple texture images in a mesh (e.g., a mesh generated in a single scan, rather than multiple meshes generated in multiple scans), meaning that different parts of the visible texture image will be applied to the texture images at their respective locations.

[0086] The embodiments described herein ensure that when texture mapping is performed on a stitched mesh from multiple scans, the mesh facets from each scan are only visible to the texture image of that scan and not to the mesh facets from other scans, thereby preventing texture images from other scans from being incorrectly mapped onto the current mesh facets.

[0087] By using spatial proximity analysis and inter-patch referencing, this embodiment of the application can accurately identify which texture images are used in the final mesh fusion process. Background texture images are obviously invisible to any patch of the stitched mesh and therefore will not be fused to the patch, effectively avoiding texture mismatch problems caused by environmental changes. This results in more realistic and believable rendering results and avoids the impact of background misalignment on object texture definition. Furthermore, the visibility between patches can be analyzed more precisely based on the spatial relationship between different patches, as described below.

[0088] In some embodiments of this application, determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in multiple scans includes: for a first facet of the current mesh in the stitched mesh, determining the positional relationship between the first facet and a second facet of at least one other mesh; and determining the visibility of the first facet to the second texture image corresponding to the second facet based on the positional relationship.

[0089] Specifically, the visibility of a facet can be determined by combining Euclidean distance and normal cosine value, ensuring that only truly visible faces are used for texture blending. The specific steps are as follows.

[0090] In some embodiments of this application, determining the visibility of the second texture image corresponding to the second patch of the first patch based on the positional relationship includes the following steps: determining the first patch center and the first normal vector of the first patch, and the second patch center and the second normal vector of the second patch; determining the center distance between the first patch center and the second patch center, and determining the normal cosine value between the first normal vector and the second normal vector; determining that the first patch is not visible to the second texture image corresponding to the second patch in response to the center distance being greater than a preset distance threshold, or the normal cosine value being less than a preset cosine value threshold; and determining the visibility of the first patch to the second texture image corresponding to the second patch based on the projection relationship between the second patch center and the first patch in response to the center distance not being greater than the preset distance threshold and the normal cosine value not being less than the preset cosine value threshold.

[0091] Specifically, for the current mesh (taking mesh m1 obtained from the first scan as an example), the first patch center fcs1 and the first normal vector fns1 corresponding to each first patch f1 are determined; for at least one mesh from another mesh (e.g., mesh m2 obtained from the second scan), the second patch center fcs2 and the second normal vector fns2 corresponding to each second patch f2 are determined; then, a kd-tree can be built based on fcs1. By constructing the kd-tree and searching for the nearest point of each of the multiple second patch centers fcs2 of the second mesh m2 relative to the first patch center fcs1, the center distance (Euclidean distance) ed and the cosine value cd of the angle between the two are calculated. If the center distance ed is greater than a preset distance threshold (e.g., 1 mm) or the cosine value of cd is less than a preset cosine value threshold (e.g., 0.866), it can be determined that the first patch is not visible to the texture image corresponding to the second patch; otherwise, it is further checked whether the intersection of the patch projections is located inside the first patch, as follows.

[0092] In some embodiments of this application, determining the visibility of the second texture image corresponding to the second patch to the first patch based on the projection relationship between the center of the second patch and the first patch includes: projecting the center of the second patch onto a target plane along the direction of the second normal vector to obtain a projection intersection point, wherein the target plane is a plane centered on the center of the first patch and with the direction of the first normal vector as its normal; determining that the first patch is visible to the second texture image corresponding to the second patch in response to the projection intersection point being located inside the first patch; and determining that the first patch is not visible to the second texture image corresponding to the second patch in response to the projection intersection point not being located inside the first patch.

[0093] Specifically, the center of the second patch is projected onto a target plane along its normal vector direction. This target plane is referenced to the center of the first patch, and its normal vector is the same as that of the first patch. If the intersection point of the projections falls inside the first patch, the first patch is considered visible to the texture image corresponding to the second patch, thus allowing texture rendering. Conversely, if the intersection point does not fall inside the first patch, the first patch is considered invisible to the texture image of the second patch, avoiding incorrect texture mapping.

[0094] After determining the visibility of each patch corresponding to a texture image, the patch can be rendered using at least one visible texture image, as follows.

[0095] In some embodiments of this application, rendering a face using at least one visible texture image includes the following steps: determining one or more faces corresponding to the visible texture image; and performing projection rendering on the one or more faces based on relevant parameters of the visible texture image, wherein the relevant parameters include at least one of the following: pose parameters of the texture image and texture camera intrinsic parameters.

[0096] Specifically, referring to the preceding description, for each facet in the stitched mesh, the visible texture image is first identified and acquired, thus obtaining a set of visible texture images. Each visible texture image in this set is visible to one or more faces. Subsequently, one or more corresponding faces are acquired based on the visible texture images in the set, and projection rendering is performed on one or more faces using these visible texture images. The order of projection rendering can be determined based on the acquisition order of the visible texture images. During the projection rendering process, the pose parameters of the texture images and camera intrinsic parameters are comprehensively considered to ensure the accuracy of texture mapping. This process significantly improves the quality of texture fusion, especially when dealing with dynamic environments, effectively avoiding misalignment between the background and the scanned object, making the final 3D model more accurate and possessing good visual effects.

[0097] It is understandable that for a patch with multiple visible texture images, sequential fusion can be achieved based on the order in which the texture images are acquired, ensuring that the stitched mesh always displays the most recently acquired texture image. In practical applications, the solution presented in this application demonstrates significantly better performance compared to related technologies in multi-engineering texture fusion. For example, ... Figure 3A As shown, the desktop environment of Project 1 is illustrated during the first round of scanning of the keyboard, as follows: Figure 3B The image shows the ground environment of Project 2 during the second scan performed on the same keyboard. Due to the change in environment, the background and lighting conditions differed between the two scans.

[0098] In some embodiments, the keyboard mesh in Project 1 is partially removed, and the texture fusion effects of the two scans performed separately are as follows: Figure 4A and Figure 4B As shown, since the texture fusion process only involves a single scan, the texture fusion effect is natural and there is no background misalignment. Figure 4B As shown, because the background part of the second round of scanning was removed and only the keyboard part was retained, both the 3D model and the texture fusion only show the keyboard.

[0099] However, if the methods in related technologies are used to directly fuse the texture images obtained from multiple scans onto the mesh (for example, replacing the deleted portion of Project 1 with the mesh of Project 2, and using the texture images of Project 1 + Project 2 for texture fusion), without considering the spatial environment differences between different scan rounds, it will cause misalignment between the background and the scanned object during texture fusion, such as... Figure 5A As shown, the background is clearly disordered, affecting the final rendering effect; however, the solution in this application, through precise visibility judgment and texture fusion strategy, ensures that the result of multi-project texture fusion is more realistic and reliable, such as... Figure 5BAs shown, the texture on each mesh facet is rendered by the texture image visible to it, avoiding the influence of background misalignment on the definition of object texture, making the final rendering result more accurate and natural.

[0100] This application's solution optimizes the correspondence between texture images and patches from multiple scans by accurately calculating the visibility of patches. This avoids background misalignment caused by changes in the scanning environment, ensuring the consistency of the target object's texture across different scan cycles. Even when the object moves slightly or the environment changes significantly, high-quality texture fusion can be achieved, significantly improving the accuracy and consistency of texture fusion in 3D scanning and greatly enhancing the visual effect and application value of 3D models.

[0101] According to an embodiment of this application, an embodiment of a three-dimensional scanning device is also provided. Figure 6 This is a schematic diagram of the structure of a three-dimensional scanning device according to an embodiment of this application. Figure 6 As shown, the device includes:

[0102] The grid alignment module 600 is used to acquire multiple grids generated by the scanning device during multiple rounds of scanning of the object, and to align the multiple grids to the same coordinate system.

[0103] The visibility analysis module 602 is used to stitch together multiple aligned meshes to obtain a stitched mesh, and to determine the visibility of each facet in the stitched mesh corresponding to the texture image acquired in multiple scans;

[0104] Texture rendering module 604 is used to render a face using at least one visible texture image.

[0105] Optionally, it also includes: aligning multiple meshes to the same coordinate system based on the relative pose transformation relationship between the multiple meshes; and aligning the texture image to the same coordinate system.

[0106] Optionally, the same coordinate system is the coordinate system of the grid generated by any one of the multiple scans.

[0107] Optionally, each round of scanning includes multiple scanning processes, in which a point cloud image and a texture image of the object are acquired using a scanning device in each scanning process, and a mesh is generated based on the multiple point cloud images acquired in each round of scanning.

[0108] Optionally, the multi-round scan is performed in at least two spatial environments, which are at least partially different from each other.

[0109] Optionally, the multi-round scan includes a first round scan performed in a first spatial environment and a second round scan performed in a second spatial environment, and the multiple grids include a first grid generated by the first round scan and a second grid generated by the second round scan, wherein the first spatial environment and the second spatial environment are at least partially different.

[0110] Optionally, determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in multiple scans includes: for a first facet of the current mesh in the stitched mesh, in response to a second facet of at least one other mesh being searchable around the first facet, determining that the first facet is visible to the first texture image corresponding to the first facet and the second texture image corresponding to the second facet; for a first facet of the current mesh in the stitched mesh, in response to a second facet of no other mesh being searchable around the first facet, determining that the first facet is visible only to the first texture image corresponding to the first facet.

[0111] Optionally, determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in multiple scans includes: for a first facet of the current mesh in the stitched mesh, determining the positional relationship between the first facet and a second facet of at least one other mesh; and determining the visibility of the first facet to the second texture image corresponding to the second facet based on the positional relationship.

[0112] Optionally, determining the visibility of the first patch to the second texture image corresponding to the second patch based on the positional relationship includes: determining the first patch center and the first normal vector of the first patch, and the second patch center and the second normal vector of the second patch; determining the center distance between the first patch center and the second patch center, and determining the normal cosine value between the first normal vector and the second normal vector; determining that the first patch is not visible to the second texture image corresponding to the second patch in response to the center distance being greater than a preset distance threshold, or the normal cosine value being less than a preset cosine value threshold; and determining the visibility of the first patch to the second texture image corresponding to the second patch based on the projection relationship between the second patch center and the first patch in response to the center distance not being greater than the preset distance threshold and the normal cosine value not being less than the preset cosine value threshold.

[0113] Optionally, determining the visibility of the second texture image corresponding to the second patch to the first patch based on the projection relationship between the center of the second patch and the first patch includes: projecting the center of the second patch onto a target plane along the direction of the second normal vector to obtain a projection intersection point, wherein the target plane is a plane centered on the center of the first patch and with the direction of the first normal vector as its normal; determining that the first patch is visible to the second texture image corresponding to the second patch in response to the projection intersection point being located inside the first patch; and determining that the first patch is not visible to the second texture image corresponding to the second patch in response to the projection intersection point not being located inside the first patch.

[0114] Optionally, rendering a face using at least one visible texture image includes: determining one or more faces corresponding to the visible texture image; and projecting and rendering the one or more faces based on relevant parameters of the visible texture image, wherein the relevant parameters include at least one of the following: pose parameters of the texture image and texture camera intrinsics.

[0115] It should be noted that each module in the above-mentioned 3D scanning device can be a program module (for example, a set of program instructions to implement a certain function) or a hardware module. For the latter, it can be manifested in the following forms, but is not limited to them: each of the above modules is manifested as a processor, or the functions of each of the above modules are implemented by a processor.

[0116] It should be noted that the three-dimensional scanning device provided in this embodiment can be used to perform... Figure 2 The three-dimensional scanning method shown above is also applicable to the embodiments of this application, and will not be repeated here.

[0117] This application embodiment also provides a scanning device, including: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the following three-dimensional scanning method: acquiring multiple meshes generated by the scanning device performing multiple rounds of scanning on an object, aligning the multiple meshes to the same coordinate system; stitching the aligned multiple meshes to obtain a stitched mesh, determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in the multiple rounds of scanning; and rendering the facet using at least one visible texture image.

[0118] It should be noted that the scanning device described in this embodiment has the ability to independently execute 3D scanning methods. That is, the scanning device itself is not only responsible for performing multiple scans of the object and generating multiple meshes, but also for processing these mesh data and performing complex alignment, stitching, and patch visibility analysis. Data processing and scanning are both completed on the same device.

[0119] This application embodiment also provides a computing device, including: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the following three-dimensional scanning method: acquiring multiple meshes generated by the scanning device performing multiple rounds of scanning on an object, aligning the multiple meshes to the same coordinate system; stitching the aligned multiple meshes to obtain a stitched mesh, determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in the multiple rounds of scanning; and rendering the facet using at least one visible texture image.

[0120] It should be noted that the computing device described in this embodiment does not directly participate in the scanning process, but is connected to the scanning device and undertakes the data processing tasks in the 3D scanning method. After the scanning device completes multiple scans and generates multiple meshes, these mesh data are transmitted to the computing device. The program running on the computing device processes the received mesh data, unifies it into the same coordinate system, performs mesh stitching, and performs patch visibility analysis. Finally, the computing device uses the analysis results, i.e., those texture images that are determined to be visible, to render the patches and generate a high-precision 3D model. These processing steps are transferred to an external device connected to the scanning device, which can be a more powerful computer or server, reflecting the externalization of inter-device collaboration and data processing, which is beneficial for handling more complex tasks with larger data volumes. That is, the computing device is mainly responsible for data processing, while the scanning device focuses on data acquisition. This design highlights the externalization of data processing and computing tasks, allowing for the use of more powerful computing resources to improve processing speed and model quality.

[0121] This application embodiment also provides a non-volatile storage medium, which includes a stored computer program. The device containing the non-volatile storage medium executes the following three-dimensional scanning method by running the computer program: acquiring multiple meshes generated by the scanning device through multiple rounds of scanning of an object, aligning the multiple meshes to the same coordinate system; stitching the aligned multiple meshes to obtain a stitched mesh, determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in the multiple rounds of scanning; and rendering the facet using at least one visible texture image.

[0122] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the three-dimensional scanning method described in various embodiments of this application: acquiring multiple meshes generated by a scanning device performing multiple scans on an object, aligning the multiple meshes to the same coordinate system; stitching the aligned multiple meshes to obtain a stitched mesh, determining the visibility of each facet in the stitched mesh corresponding to a texture image acquired in the multiple scans; and rendering the facet using at least one visible texture image.

[0123] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0124] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0126] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0128] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0129] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A three-dimensional scanning method, comprising: Multiple grids generated by a scanning device scanning an object in multiple rounds are acquired, and the multiple grids are aligned to the same coordinate system, wherein the multiple rounds of scanning are performed in at least two spatial environments, and the at least two spatial environments are at least partially different; The multiple meshes, after being stitched and aligned, form a stitched mesh. The visibility of each facet in the stitched mesh corresponding to the texture image acquired in the multiple scans is determined, including: for a first facet of the current mesh in the stitched mesh, in response to the presence of at least one other mesh's second facet around the first facet, determining that the first facet is visible to both the first texture image corresponding to the first facet and the second texture image corresponding to the second facet; or, in response to the presence of no other mesh's second facet around the first facet, determining that the first facet is only visible to the first texture image corresponding to the first facet; and The patch is rendered using at least one visible texture image.

2. The method according to claim 1, further comprising: Based on the relative pose transformation relationship between the multiple meshes, align the multiple meshes to the same coordinate system; as well as Align the texture images to the same coordinate system.

3. The method according to claim 2, wherein, The same coordinate system refers to the coordinate system of the grid generated by any one of the multiple scans.

4. The method according to claim 1, wherein, Each round of scanning includes multiple scanning processes. In each scanning process, the scanning device acquires a point cloud image and a texture image of the object, and generates a mesh based on the multiple point cloud images acquired in each round of scanning.

5. The method according to claim 1, wherein, The multi-round scan includes a first round scan performed in a first spatial environment and a second round scan performed in a second spatial environment. The multiple grids include a first grid generated by the first round scan and a second grid generated by the second round scan, wherein the first spatial environment and the second spatial environment are at least partially different.

6. The method according to claim 1, wherein, Determining the visibility of each facet in the stitched mesh corresponding to the texture image acquired in the multi-round scan includes: For the first facet of the current grid in the spliced ​​mesh, determine the positional relationship between the first facet and at least one second facet of another grid; Based on the positional relationship, the visibility of the first patch to the second texture image corresponding to the second patch is determined.

7. The method according to claim 6, wherein, Determining the visibility of the first patch to the second texture image corresponding to the second patch based on the positional relationship includes: Determine the first patch center and first normal vector of the first patch, and the second patch center and second normal vector of the second patch; Determine the center distance between the center of the first patch and the center of the second patch, and determine the normal cosine value between the first normal vector and the second normal vector; In response to the center distance being greater than a preset distance threshold, or the normal cosine value being less than a preset cosine value threshold, it is determined that the first patch is not visible to the second texture image corresponding to the second patch; In response to the fact that the center distance is not greater than the preset distance threshold and the normal cosine value is not less than the preset cosine value threshold, the visibility of the first patch to the second texture image corresponding to the second patch is determined based on the projection relationship between the center of the second patch and the first patch.

8. The method according to claim 7, wherein, Based on the projection relationship between the center of the second patch and the first patch, the visibility of the first patch to the second texture image corresponding to the second patch is determined, including: The center of the second patch is projected onto the target plane along the direction of the second normal vector to obtain the projection intersection point, wherein the target plane is a plane with the center of the first patch as its center and the direction of the first normal vector as its normal; In response to the projection intersection being located inside the first patch, it is determined that the first patch is visible to the second texture image corresponding to the second patch; and In response to the fact that the projection intersection is not located inside the first patch, it is determined that the first patch is not visible to the second texture image corresponding to the second patch.

9. The method according to claim 1, wherein, Rendering the patch using at least one visible texture image includes: Identify one or more patches corresponding to a visible texture image; Based on the relevant parameters of the visible texture image, projection rendering is performed on the one or more patches, wherein the relevant parameters include at least one of the following: the pose parameters of the texture image, and the intrinsic parameters of the texture camera.

10. A scanning device, comprising: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when executed, performs the three-dimensional scanning method according to any one of claims 1 to 9.

11. A computing device, comprising: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when executed, performs the three-dimensional scanning method according to any one of claims 1 to 9.

12. A non-volatile storage medium, the non-volatile storage medium comprising a stored computer program, wherein, The device containing the non-volatile storage medium executes the three-dimensional scanning method according to any one of claims 1 to 9 by running the computer program.

Images