Three-dimensional scanning method, three-dimensional scanning equipment and three-dimensional scanning system

Abstract

The invention discloses a three-dimensional scanning method, three-dimensional scanning equipment and a three-dimensional scanning system.The three-dimensional scanning method comprises the steps that a three-dimensional grid model and a first texture map of an object to be measured are obtained, and the first texture map and the three-dimensional grid model have a first three-dimensional mapping relation; processing the first texture map based on a super-resolution algorithm to obtain a second texture map, the second texture map and the first texture map having a two-dimensional mapping relationship; determining a second three-dimensional mapping relation between the second texture map and the three-dimensional grid model based on the first three-dimensional mapping relation and the two-dimensional mapping relation; and performing texture mapping on the three-dimensional grid model based on the second three-dimensional mapping relation and the second texture mapping. The three-dimensional scanning equipment can realize the three-dimensional scanning method. The three-dimensional scanning system can realize the three-dimensional scanning method. Through the setting, the definition of the texture map can be improved without using a camera with high resolution.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional scanning technology, and in particular to a three-dimensional scanning method, a three-dimensional scanning device, and a three-dimensional scanning system. Background Art

[0002] In the prior art, two cameras are generally set up to obtain images of different positions of the object to be measured, and then feature matching is performed on the image obtained by the high-resolution camera and the image obtained by the low-resolution camera. Then, the image obtained by the high-resolution camera replaces the image obtained by the low-resolution camera to achieve map replacement, thereby obtaining a texture map with higher clarity.

[0003] However, the above-mentioned texture replacement method depends on the resolution of the image acquired by the camera. A camera with a higher resolution needs to be used to acquire images of different positions of the object to be measured in order to obtain a texture map with higher definition. Summary of the Invention

[0004] In order to address the deficiencies of the prior art, the purpose of the present application is to provide a three-dimensional scanning method, a three-dimensional scanning device, and a three-dimensional scanning system, which can improve the clarity of texture mapping without using a high-resolution camera.

[0005] To achieve the above objectives, this application adopts the following technical solutions:

[0006] A three-dimensional scanning method, comprising:

[0007] A three-dimensional mesh model and a first texture map of the object to be measured are obtained, wherein the first texture map and the three-dimensional mesh model have a first three-dimensional mapping relationship; the first texture map is processed based on a super-resolution algorithm to obtain a second texture map, wherein the second texture map and the first texture map have a two-dimensional mapping relationship; based on the first three-dimensional mapping relationship and the two-dimensional mapping relationship, a second three-dimensional mapping relationship between the second texture map and the three-dimensional mesh model is determined; and based on the second three-dimensional mapping relationship and the second texture map, texture mapping is performed on the three-dimensional mesh model.

[0008] Furthermore, obtaining a three-dimensional mesh model and a first texture map of the object to be measured includes:

[0009] Acquire first three-dimensional visual data and second three-dimensional visual data, where the first three-dimensional visual data and the second three-dimensional visual data are collected at least for the object to be measured; based on the first three-dimensional visual data, obtain a three-dimensional point cloud model of the object to be measured, and mesh the three-dimensional point cloud model to obtain a three-dimensional mesh model; based on the second three-dimensional visual data, obtain a first texture map of the object to be measured.

[0010] Furthermore, the second three-dimensional visual data includes a first texture image set and a second texture image set, the images in the second texture image set can constitute a first texture map, and the image clarity of the second texture image set is higher than the image clarity of the first texture image set; the images in the first texture image set have a third three-dimensional mapping relationship with the three-dimensional mesh model; and the three-dimensional scanning method further includes:

[0011] Based on the third three-dimensional mapping relationship and the matching relationship between the images in the first texture image set and the images in the second texture image set, a first three-dimensional mapping relationship between the first texture map and the three-dimensional mesh model is determined.

[0012] Furthermore, the first texture map includes multiple first texture images, and the multiple first texture images are at least divided by the first texture map; processing the first texture map based on the super-resolution algorithm to obtain the second texture map includes: processing the multiple first texture images separately through the super-resolution algorithm to obtain multiple second texture maps, each second texture map has a two-dimensional mapping relationship with a corresponding first texture image.

[0013] Furthermore, processing the first texture map based on a super-resolution algorithm to obtain a second texture map includes:

[0014] The first texture map is processed based on a super-resolution algorithm to obtain a second texture image; and the second texture image is divided into a plurality of second texture maps.

[0015] Furthermore, the first three-dimensional mapping relationship is calculated by one of orthogonal projection, perspective projection, affine projection, spherical projection, cylindrical projection and cube mapping.

[0016] Furthermore, the three-dimensional scanning method further includes:

[0017] The three-dimensional coordinates of the three-dimensional mesh model and the first two-dimensional coordinates of the first texture map are obtained; the first three-dimensional mapping relationship is the three-dimensional mapping relationship between the three-dimensional coordinates and the first two-dimensional coordinates.

[0018] Furthermore, calculating the two-dimensional mapping relationship includes:

[0019] Obtain first two-dimensional coordinates of the first texture map; obtain a magnification factor between the first texture map and the second texture map; calculate second two-dimensional coordinates of the second texture map based on the first two-dimensional coordinates and the magnification factor; calculate a two-dimensional mapping relationship based on the first two-dimensional coordinates and the second two-dimensional coordinates.

[0020] To achieve the above objectives, this application adopts the following technical solutions:

[0021] A three-dimensional scanning device includes a memory and a processor. The memory stores program instructions. When the processor executes the program instructions stored in the memory, the three-dimensional scanning method is implemented.

[0022] To achieve the above objectives, this application adopts the following technical solutions:

[0023] A three-dimensional scanning system includes a three-dimensional scanning device and a computer device. The three-dimensional scanning is used to collect the above-mentioned first three-dimensional visual data and second three-dimensional visual data; the computer device is used to execute the above-mentioned three-dimensional scanning method.

[0024] After obtaining the three-dimensional mesh model and the first texture map of the object to be measured, the above-mentioned three-dimensional scanning method, three-dimensional scanning device and three-dimensional scanning system improve the resolution of the first texture map through a super-resolution algorithm and obtain a second texture map, and then map the second texture map to the three-dimensional mesh model, thereby improving the clarity of the texture map without the need for a high-resolution camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a hardware structure block diagram of the three-dimensional scanning device according to an embodiment of the present application.

[0026] Figure 2 This is a flow chart of a three-dimensional scanning method according to an embodiment of the present application.

[0027] Figure 3 This is a specific flow chart of step S1 of the three-dimensional scanning method according to an embodiment of the present application.

[0028] Figure 4 This is a specific flow chart of step S12 of the three-dimensional scanning method according to an embodiment of the present application.

[0029] Figure 5 This is a specific flow chart of step S13 of the three-dimensional scanning method according to an embodiment of the present application.

[0030] Figure 6 This is a specific flow chart of the first step S2 of the three-dimensional scanning method according to an embodiment of the present application.

[0031] Figure 7 This is a specific flow chart of the second step S2 of the three-dimensional scanning method according to an embodiment of the present application.

[0032] Figure 8 This is a flowchart of calculating a two-dimensional mapping relationship according to an embodiment of the present application.

[0033] Figure 9 This is a structural block diagram of a three-dimensional scanning system according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the specific implementation of the present application will be clearly and completely described below in conjunction with the drawings in the implementation of the present application.

[0035] It should be noted that the words “first”, “second” and similar terms used in the specification and claims of this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as “one” or “a” do not indicate a quantity limitation, but rather indicate the presence of at least one. “Multiple” means at least two. Unless otherwise indicated, words such as “include” or “comprise” mean that the elements or objects appearing before “include” or “comprises” include the elements or objects listed after “include” or “comprises” and their equivalents, and do not exclude other elements or objects. Words such as “connected” or “connected” are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0036] As used in this specification and the appended claims, the singular forms "a," "an," "said," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0037] The method embodiment provided in this embodiment can be executed in a three-dimensional scanning device 100, a three-dimensional scanning system 200 or similar devices. Figure 1 1 is a hardware structure diagram of a three-dimensional scanning device 100 for executing a three-dimensional scanning method according to an embodiment of the present application. Figure 1 As shown, the three-dimensional scanning device 100 may include one or more ( Figure 1 (only one is shown) memory 11 and processor 12.

[0038] The memory 11 stores program instructions, such as software programs and modules of application software, such as a computer program corresponding to a three-dimensional scanning method in this embodiment. The processor 12 executes the computer program stored in the memory 11 to execute various functional applications and data processing, thereby implementing the above-mentioned method. The memory 11 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some embodiments, the memory 11 may further include a memory 11 remotely located relative to the processor 12, and these remote memories 11 may be connected to the three-dimensional scanning device 100 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0039] The processor 12 is used to execute the program instructions stored in the memory 11. The processor 12 may include, but is not limited to, a microcontroller unit (MCU) or a field programmable gate array (FPGA). The three-dimensional scanning device 100 may also include a transmission device 13 and an input / output device 14 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned three-dimensional scanning device 100. For example, the three-dimensional scanning device 100 may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0040] The transmission device 13 is used to receive or transmit data via a network. This network may include a wireless network provided by the communications provider of the 3D scanning device 100. In one embodiment, the transmission device 13 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 13 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] This embodiment provides a three-dimensional scanning method. Figure 2 This is a flow chart of a three-dimensional scanning method according to an embodiment of the present application, which is used to improve the clarity of an image during texture mapping.

[0042] like Figure 2 As shown, the three-dimensional scanning method of the present application includes the following steps:

[0043] S1 obtains a three-dimensional mesh model and a first texture map of the object to be measured, wherein the first texture map and the three-dimensional mesh model have a first three-dimensional mapping relationship.

[0044] In this application, a 3D mesh model is a model of the object to be measured that has not been texture mapped after 3D scanning, that is, a 3D model that only includes the shape features of the object to be measured. The first texture map is a texture image representing the surface of the object to be measured. The relative position between the 3D mesh model and the first texture map can be obtained through the first 3D mapping relationship.

[0045] like Figure 2 and Figure 3 As shown, obtaining the three-dimensional mesh model and the first texture map of the object to be measured in step S1 includes the following steps:

[0046] S11 acquires first three-dimensional visual data and second three-dimensional visual data, wherein the first three-dimensional visual data and the second three-dimensional visual data are collected for at least the object to be measured. The first three-dimensional visual data can be used to obtain a three-dimensional mesh model of the object to be measured, and the second three-dimensional visual data can be used to obtain a first texture map of the object to be measured.

[0047] Exemplarily, in the present application, the first three-dimensional visual data and the second three-dimensional visual data can be acquired separately. For example, the first three-dimensional visual data is acquired by acquiring the object to be measured through a binocular camera, and the second three-dimensional visual data is acquired by acquiring the object to be measured through a color camera. Alternatively, the binocular camera and the color camera are integrated into a system (e.g., a binocular color camera), so that the first three-dimensional visual data and the second three-dimensional visual data are acquired simultaneously through the above-mentioned integrated system. In summary, the present application does not limit the method of acquiring the first three-dimensional visual data and the second three-dimensional visual data.

[0048] As an exemplary implementation, during texture mapping, the object to be measured is photographed using a color camera and a binocular camera. The color camera is used to capture a texture image of the surface of the object to be measured, i.e., the first texture map of the present application. The binocular camera is used to capture an image of the object to be measured and, based on the depth information of the captured image, obtain a three-dimensional mesh model of the object to be measured.

[0049] S12 obtains a 3D point cloud model of the object to be measured based on the first 3D visual data, and meshes the 3D point cloud model to obtain a 3D mesh model. Through the above settings, a 3D mesh model of the object to be measured can be obtained to facilitate subsequent texture mapping.

[0050] In this embodiment, a binocular camera shoots the object to be measured to collect multiple frames of left and right images, aligns the left and right images of each frame through epipolar correction, and then uses deep learning to calculate the view difference of the left and right images of each frame to obtain the disparity value of each pixel in the left and right images of each frame. The above disparity value is then converted into depth information based on the intrinsic parameters and baseline distance of the binocular camera. The coordinates of each pixel in the left and right images of each frame are then combined with the depth information to obtain a single-frame point cloud contained in each frame of the left and right images. The single-frame point clouds in each frame of the image are spliced ​​together to obtain a global point cloud. Finally, the global point cloud is fused to form a three-dimensional point cloud model, and then the three-dimensional point cloud model is meshed to obtain a three-dimensional mesh model of the object to be measured.

[0051] Since the color camera and the binocular camera move synchronously, the color camera and the binocular camera are at the same shooting position when capturing the same frame of image, and the relative position of the color camera and the binocular camera can be calculated by the processor 12, or the relative position of the color camera and the binocular camera is known, thereby enabling the three-dimensional mesh model and the first texture map to have a first three-dimensional mapping relationship.

[0052] As an embodiment, the first three-dimensional mapping relationship of the present application is calculated using one of orthogonal projection, perspective projection, affine projection, spherical projection, cylindrical projection, and cube mapping. Through the above configuration, an appropriate projection method can be selected based on the actual acquisition requirements of the first three-dimensional mapping relationship, thereby facilitating improved efficiency and accuracy in acquiring the first three-dimensional mapping relationship.

[0053] Exemplarily, orthogonal projection is a projection method that keeps the size of an object unchanged. When obtaining the first three-dimensional mapping relationship through orthogonal projection, the first texture map is projected into a rectangular area of ​​a constant size in the three-dimensional mesh model, thereby obtaining the first three-dimensional mapping relationship between the first texture map and the three-dimensional mesh model.

[0054] For example, perspective projection is a projection method that simulates the way human eyes observe objects. Objects closer to the observer appear larger than objects farther away. Perspective projection can improve the realism of the first texture map projected onto the three-dimensional mesh model.

[0055] Exemplarily, affine projection is a projection method used to process the rotation, translation and scaling of objects in three-dimensional space. Through affine projection, the first texture map can be projected onto a plane on the three-dimensional mesh model, and the first texture map can be transformed in position on the above-mentioned plane, thereby improving the diversity of the positions where the first texture map is projected onto the three-dimensional mesh model.

[0056] For example, spherical projection is a projection method that maps a three-dimensional sphere to a two-dimensional plane. When performing spherical projection, a spherical coordinate system is established, and longitude and latitude are used to determine the position of points on the sphere on the projection plane. The points in the spherical projection are described by radius, horizontal angle (longitude) and vertical angle (latitude), thereby improving the description accuracy of the points projected by the first texture map on the three-dimensional grid model, which is conducive to improving the accuracy of the first three-dimensional mapping relationship.

[0057] For example, cylindrical projection is a projection method that maps a three-dimensional sphere onto a two-dimensional plane. Cylindrical projection also uses the longitude and latitude described above to determine the position of a point on the sphere on the projection plane. Depending on actual projection requirements, either spherical projection or cylindrical projection can be selected to obtain the first three-dimensional mapping relationship between the first texture map and the three-dimensional mesh model, thereby increasing the diversity of methods for obtaining the first three-dimensional mapping relationship.

[0058] For example, cube mapping is a texture mapping method used to render the interior of a cube. Cube mapping typically uses a six-sided cube to represent the texture, and maps texture coordinates to the cube's different faces. Cube mapping can improve the accuracy of mapping a first texture map to a cube-shaped three-dimensional mesh model, thereby improving the accuracy of obtaining a first three-dimensional mapping relationship between the first texture map and the three-dimensional mesh model.

[0059] More specifically, if Figure 4 As shown, the three-dimensional scanning method further includes:

[0060] S121 obtains the three-dimensional coordinates of the three-dimensional mesh model and the first two-dimensional coordinates of the first texture map; the first three-dimensional mapping relationship is the three-dimensional mapping relationship between the three-dimensional coordinates and the first two-dimensional coordinates.

[0061] In this embodiment, the three-dimensional coordinates of the three-dimensional mesh model are set to (x, y, z), and the two-dimensional coordinates of the first texture map are set to (u, v), that is, the first two-dimensional coordinates are (u, v). The first three-dimensional mapping relationship between the first texture map and the three-dimensional mesh model can be expressed as

[0062] (u,v)=f(x,y,z) (Formula 1).

[0063] Wherein, f is the projection relationship between the first texture map and the three-dimensional mesh model, that is, the projection relationship f can be calculated by one of the above-mentioned orthogonal projection, perspective projection, affine projection, spherical projection, cylindrical projection and cube mapping.

[0064] S13 obtains a first texture map of the object to be measured based on the second three-dimensional visual data. Through the above settings, the first texture map of the object to be measured can be obtained, so as to facilitate the subsequent acquisition of the second texture map.

[0065] In the present application, the second three-dimensional visual data includes a texture image of the surface of the object to be measured and the color of its texture, thereby obtaining a first texture map of the object to be measured. It should be noted that when obtaining the second three-dimensional visual data, the first texture map can be directly obtained by capturing the texture of the surface of the object to be measured using a color camera, or the first texture map can be obtained by map replacement, and this application does not impose any restrictions on this.

[0066] For example, a texture replacement method is used for description. The second three-dimensional visual data includes a first texture image set and a second texture image set. The images in the second texture image set can constitute the first texture map, and the image clarity of the second texture image set is higher than that of the first texture image set. Through the above setting, the image clarity of the first texture map can be improved, thereby reducing the number of first texture maps that require super-resolution algorithm processing, thereby improving the efficiency of texture mapping; alternatively, the clarity of the texture map can be further improved based on the image clarity of the first texture map.

[0067] In this embodiment, the first texture image set and the second texture image set are sets of texture images of the object to be measured collected by different devices. The first texture image set and the first visual three-dimensional data can be data collected by the same device.

[0068] Exemplarily, the resolution of the device used to capture the second texture image set is higher than the resolution of the device used to capture the first texture image set. For example, the device used to capture the second texture image set may be a single-lens reflex camera or a mobile phone, while the device used to capture the first texture image set may be a color camera built into a scanner.

[0069] First, after the device acquires a first texture image set and first 3D visual data, binocular reconstruction is achieved by texture matching the images in the first 3D visual data to form a single-frame point cloud. Second, the single-frame point clouds are spliced ​​together to form a point cloud model, which is then meshed to form a 3D mesh model. Finally, the 3D mesh model can be mapped using the first texture image set, or it can be left unmapped and subsequently mapped using the second texture image set.

[0070] Specifically, the images in the first texture image set and the three-dimensional mesh model have a third three-dimensional mapping relationship. The images in the first texture image set and the three-dimensional mesh model are acquired by the same device. Therefore, when the devices capture the same frame of image, the relative positions of the images in the first texture image set and the three-dimensional mesh model can be calculated by the processor 12, or the relative positions of the images in the first texture image set and the three-dimensional mesh model are known, thereby enabling the images in the first texture image set and the three-dimensional mesh model to have the third three-dimensional mapping relationship.

[0071] like Figure 5 As shown, based on the third three-dimensional mapping relationship between the images in the first texture image set and the three-dimensional mesh model, the three-dimensional scanning method of the present application further includes:

[0072] S131 determines a first three-dimensional mapping relationship between the first texture map and the three-dimensional mesh model based on the third three-dimensional mapping relationship and the matching relationship between the images in the first texture image set and the images in the second texture image set. Since the images in the first texture image set and the images in the second texture image set are images of the same object to be measured, texture matching can be performed between the images in the first texture image set and the images in the second texture image set to facilitate subsequent texture mapping. It should be noted that the matching relationship between the images in the first texture image set and the images in the second texture image set can be obtained through texture matching.

[0073] like Figure 2 As shown, S2 processes the first texture map based on a super-resolution algorithm to obtain a second texture map, and the second texture map and the first texture map have a two-dimensional mapping relationship.

[0074] A super-resolution (SR) algorithm is an algorithm that increases the resolution of an image through computation. Therefore, compared to the first texture map, the super-resolution algorithm can increase the resolution of the second texture map image, thereby increasing the resolution of the image used for texture mapping, thereby improving the clarity of the texture map.

[0075] In this embodiment, the first texture map is a texture image in two-dimensional space. When the first texture map is processed using a super-resolution algorithm, only the resolution of the first texture map is increased. That is, the second texture map obtained by the super-resolution algorithm is also a texture image in two-dimensional space. Therefore, a two-dimensional mapping relationship exists between the first texture map and the second texture map. This two-dimensional mapping relationship can be used to obtain the coordinates of the second texture map in three-dimensional space, facilitating the subsequent determination of the relative position between the second texture map and the three-dimensional mesh model. It should be noted that this two-dimensional mapping relationship has been described below and will not be repeated here.

[0076] In one embodiment, the first texture map includes multiple first texture images, each of which is formed by at least segmenting the first texture map. That is, in this embodiment, the multiple first texture images are segmented before the first texture map is subjected to super-resolution algorithm calculations. It should be noted that the first texture map can be directly acquired by capturing the surface texture of the object to be measured using a color camera, or it can be acquired through a texture replacement method. During processing, if the clarity of a portion of the first texture map meets actual usage requirements, there is no need to perform super-resolution processing on that portion of the first texture map. In other words, only that portion of the first texture image can be super-resolution processed. This reduces the image area required to be processed in the first texture map, thereby improving the efficiency of the super-resolution algorithm in processing the first texture map, and thus improving the efficiency of texture mapping. Furthermore, segmenting the first texture map into multiple first texture images for processing can reduce the image area required to be processed by the super-resolution algorithm in a single pass, thereby reducing the processing burden on the software and / or hardware when processing the first texture map in a single pass using the super-resolution algorithm. For example, this can reduce the workload of the processor 12 when processing the first texture map in a single pass using the super-resolution algorithm.

[0077] like Figure 2 and Figure 6 As shown, in this embodiment, based on the above-mentioned multiple first texture images, step S2 of this application includes the following steps:

[0078] S21 processes the plurality of first texture images separately using a super-resolution algorithm, and obtains a plurality of second texture maps, each of which has a two-dimensional mapping relationship with a corresponding first texture image. In the above process of processing the first map using the super-resolution algorithm, the first texture map is first divided into a plurality of first texture images, and then the first texture images are processed using the super-resolution algorithm. If the clarity of any first texture image can meet the use requirements of the texture map, the first texture image does not need to be processed as described above; if the clarity of any first texture image cannot meet the use requirements of the texture map, the first texture image is processed as described above, thereby reducing the number of first texture images that need to be processed, thereby facilitating an increase in the rate of obtaining the second texture map.

[0079] Each first texture image is a texture image in two-dimensional space. When the first texture images are processed using a super-resolution algorithm, only the resolution of each first texture image is increased. That is, the multiple second texture maps obtained by the super-resolution algorithm are also texture images in two-dimensional space. Therefore, a two-dimensional mapping relationship exists between the first texture image and the second texture map. The pose of the second texture map in three-dimensional space can be determined using this two-dimensional mapping relationship and the first three-dimensional mapping relationship.

[0080] like Figure 2 and Figure 7 As shown, as another implementation, the step of processing the first texture map based on the super-resolution algorithm in step S2 of the present application to obtain the second texture map includes:

[0081] S21 processes the first texture map based on a super-resolution algorithm to obtain a second texture image.

[0082] S22 divides the second texture image into a plurality of second texture maps.

[0083] Through the above configuration, the clarity of each second texture map can meet the texture map usage requirements, thereby improving the clarity of the texture map. Furthermore, the above configuration allows the second texture map to be obtained through a single super-resolution algorithm calculation, thereby reducing the complexity of the texture mapping process and improving texture mapping efficiency.

[0084] The first texture map is a texture image in two-dimensional space. When the super-resolution algorithm is applied to the first texture map, only the resolution of the first texture map is increased. That is, the second texture image obtained by the super-resolution algorithm is also a texture image in two-dimensional space. Therefore, a two-dimensional mapping relationship exists between the second texture image and the first texture map. Furthermore, because the multiple second texture maps are segmented from the second texture image, the poses of the multiple second texture maps in three-dimensional space can be determined using this two-dimensional mapping relationship and the first three-dimensional mapping relationship.

[0085] like Figure 2 As shown, S3 determines a second three-dimensional mapping relationship between the second texture map and the three-dimensional mesh model based on the first three-dimensional mapping relationship and the two-dimensional mapping relationship.

[0086] The second three-dimensional mapping relationship can be used to obtain the relative position between the second texture map and the three-dimensional mesh model, thereby facilitating the subsequent mapping of the second texture map to the three-dimensional mesh model, i.e., facilitating the subsequent texture mapping of the three-dimensional mesh model. It should be noted that the calculation formula for the second three-dimensional mapping relationship is described below and will not be repeated here.

[0087] Exemplarily, in the present application, the processor 12 can obtain a first three-dimensional mapping relationship by calculating the relative posture of the color camera and the binocular camera, and calculate a two-dimensional mapping relationship by changing the resolution between the first texture map and the second texture map, or enable the processor 12 to directly obtain the first three-dimensional mapping relationship and the two-dimensional mapping relationship stored in the memory 11 (at this time, the first three-dimensional mapping relationship and the two-dimensional mapping relationship are pre-stored in the memory 11). This application does not impose any restrictions on this.

[0088] S4 performs texture mapping on the three-dimensional mesh model based on the second three-dimensional mapping relationship and the second texture map. Through the above settings, the second texture map with increased resolution can be mapped to the three-dimensional mesh model through the second three-dimensional mapping relationship, thereby improving image clarity during texture mapping.

[0089] In summary, in the present application, a three-dimensional mesh model and a first texture map of the object to be measured are obtained, and then the first texture map is processed by a super-resolution algorithm to obtain a second texture map with higher resolution. Finally, based on the first three-dimensional mapping relationship between the first texture map and the three-dimensional mesh model, and the two-dimensional mapping relationship between the first texture map and the second texture map, a second three-dimensional mapping relationship between the second texture map and the three-dimensional mesh model can be obtained, so that the second texture map can be mapped to the three-dimensional mesh model, thereby improving the clarity of the texture map without the need for a high-resolution camera.

[0090] like Figure 8 As shown, calculating the two-dimensional mapping relationship includes the following steps:

[0091] Q1 obtains the first two-dimensional coordinate of the first texture map.

[0092] Q2 obtains the magnification factor between the first texture map and the second texture map.

[0093] Q3 calculates the second two-dimensional coordinates of the second texture map based on the first two-dimensional coordinates and the magnification factor. Q4 calculates the two-dimensional mapping relationship based on the first two-dimensional coordinates and the second two-dimensional coordinates. Through the above settings, the two-dimensional mapping relationship between the first texture map and the second texture map can be obtained using the specific first two-dimensional coordinates and the second two-dimensional coordinates, thereby improving the accuracy of the two-dimensional mapping relationship.

[0094] The first two-dimensional coordinate, i.e., the two-dimensional coordinate of the first texture map mentioned above, is (u, v). After the first texture map is subjected to super-resolution processing, the second texture map is obtained. At this time, the two-dimensional coordinate of the second texture map is (u', v'), i.e., the second two-dimensional coordinate is (u', v'). Assume that after the first texture map is super-resolution processed, the second texture map is magnified n times compared to the first texture map, i.e., the magnification is n. At this time, the two-dimensional mapping relationship between the second texture map and the first texture map can be expressed as:

[0095] (u', ​​v') = n·(u, v) (Formula 2)

[0096] It should be noted that, in some embodiments, n=2 k (k∈N + ).

[0097] With the above settings, when the two-dimensional coordinates of the first texture map are known, the two-dimensional coordinates of the second texture map can be obtained according to the magnification n, thereby obtaining a two-dimensional mapping relationship between the second texture map and the first texture map.

[0098] At the same time, through the above formula 1 and formula 2, the expression of the second three-dimensional mapping relationship between the second texture map and the three-dimensional mesh model can be obtained as follows:

[0099] (u', ​​v') = n·f (x, y, z) (Formula 3).

[0100] That is, formula 1 can be used to obtain a first three-dimensional mapping relationship between the first texture map and the three-dimensional mesh model, and formula 2 can be used to obtain a two-dimensional mapping relationship between the first texture map and the second texture map. Formula 3 can be derived from formulas 1 and 2, that is, a second three-dimensional mapping relationship between the second texture map and the second texture map is obtained, thereby improving the clarity of the texture map without using a high-resolution camera.

[0101] like Figure 9 As shown, a three-dimensional scanning system 200 is also provided in this embodiment, which includes a three-dimensional scanning device 100 and a computer device 21. The three-dimensional scanning device 100 is used to collect the above-mentioned first three-dimensional visual data and the second three-dimensional visual data, and the computer device 21 is used to execute the above-mentioned three-dimensional scanning method.

[0102] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the claims appended to this application.

Claims

1. A three-dimensional scanning method, characterized in that: Acquire a three-dimensional mesh model and a first texture map of the object to be measured, wherein the first texture map and the three-dimensional mesh model have a first three-dimensional mapping relationship; Processing the first texture map based on a super-resolution algorithm to obtain a second texture map, wherein the second texture map has a two-dimensional mapping relationship with the first texture map; Determining a second three-dimensional mapping relationship between the second texture map and the three-dimensional mesh model based on the first three-dimensional mapping relationship and the two-dimensional mapping relationship; Based on the second three-dimensional mapping relationship and the second texture map, texture mapping is performed on the three-dimensional mesh model.

2. The three-dimensional scanning method according to claim 1, characterized in that: The step of obtaining a three-dimensional mesh model and a first texture map of the object to be measured includes: Acquire first three-dimensional visual data and second three-dimensional visual data, where the first three-dimensional visual data and the second three-dimensional visual data are collected at least for the object to be measured; Acquire a three-dimensional point cloud model of the object to be measured based on the first three-dimensional visual data, and mesh the three-dimensional point cloud model to obtain the three-dimensional mesh model; Based on the second three-dimensional visual data, a first texture map of the object to be measured is obtained.

3. The three-dimensional scanning method according to claim 2, characterized in that: The second three-dimensional visual data includes a first texture image set and a second texture image set, wherein images in the second texture image set can constitute the first texture map, and the image definition in the second texture image set is higher than the image definition in the first texture image set; The images in the first texture image set have a third three-dimensional mapping relationship with the three-dimensional mesh model; The three-dimensional scanning method further includes: The first three-dimensional mapping relationship between the first texture map and the three-dimensional mesh model is determined based on the third three-dimensional mapping relationship and the matching relationship between the images in the first texture image set and the images in the second texture image set.

4. The three-dimensional scanning method according to claim 1, characterized in that: The first texture map includes a plurality of first texture images, and the plurality of first texture images are formed by at least dividing the first texture map; The step of processing the first texture map based on a super-resolution algorithm to obtain a second texture map includes: The plurality of first texture images are processed respectively by the super-resolution algorithm to obtain a plurality of second texture maps, each of the second texture maps having the two-dimensional mapping relationship with a corresponding one of the first texture images.

5. The three-dimensional scanning method according to claim 1, characterized in that: The step of processing the first texture map based on a super-resolution algorithm to obtain a second texture map includes: Processing the first texture map based on the super-resolution algorithm to obtain a second texture image; The second texture image is divided into a plurality of second texture maps.

6. The three-dimensional scanning method according to claim 1, characterized in that: The first three-dimensional mapping relationship is calculated by one of orthogonal projection, perspective projection, affine projection, spherical projection, cylindrical projection and cube mapping.

7. The three-dimensional scanning method according to claim 1 or 6, characterized in that: The three-dimensional scanning method further includes: Obtaining three-dimensional coordinates of the three-dimensional mesh model and first two-dimensional coordinates of the first texture map; The first three-dimensional mapping relationship is a three-dimensional mapping relationship between the three-dimensional coordinates and the first two-dimensional coordinates.

8. The three-dimensional scanning method according to claim 1, wherein: Calculating the two-dimensional mapping relationship includes: Obtaining first two-dimensional coordinates of the first texture map; Obtaining a magnification factor between the first texture map and the second texture map; Calculating second two-dimensional coordinates of the second texture map based on the first two-dimensional coordinates and the magnification; The two-dimensional mapping relationship is calculated based on the first two-dimensional coordinates and the second two-dimensional coordinates.

9. A three-dimensional scanning device, characterized in that: include: a memory storing program instructions; A processor, wherein when the processor executes the program instructions stored in the memory, the three-dimensional scanning method according to any one of claims 1 to 8 is implemented.

10. A three-dimensional scanning system, characterized in that: include: A three-dimensional scanning device, wherein the three-dimensional scanning is used to collect the first three-dimensional visual data and the second three-dimensional visual data as claimed in claim 2 or 3; A computer device, wherein the computer device is used to execute the three-dimensional scanning method according to any one of claims 1 to 8.

Images