Methods, apparatus and handheld digitizing devices for object digitization and facial scanning

Abstract

This application provides a method, apparatus, and handheld digitization device for object digitization and facial scanning. The method includes acquiring a spectral image of an object under multi-node illumination; acquiring a response function of the spectral image; recovering the reflectance spectrum of the object based on the response function and a reflectance database; acquiring a fine point cloud of the object; and digitizing the object based on the fine point cloud and the recovered reflectance spectrum. This application can quickly recover the true reflectance spectrum and high-resolution three-dimensional shape of an object, thereby creating a realistic digital avatar for application in digital fields such as the metaverse. Based on this, this application can effectively acquire the reflectance spectrum and three-dimensional information of a face for personal facial health monitoring and beauty enhancement.

Description

Technical Field

[0001] This application belongs to the field of digital technology, and in particular relates to a method, apparatus and handheld digital device for object digitization and facial scanning. Background Technology

[0002] Information technology, developed under the impetus of computer technology, is the technology of using electronic information to describe and process real-world affairs. The concept of digitization, which has emerged in recent years, is considered an upgraded version of informatization. The biggest difference between digitization and informatization lies in the scope and degree of informatization. Digitization expands the informatization of things from simple management information to comprehensive digital and data descriptions. In digitization, all aspects of an object need to be digitized—holographically digitized—and accessed and manipulated based on the data. In other words, object digitization refers to converting physical objects in reality into digital form and applying them in various digital applications. Summary of the Invention

[0003] The purpose of this application is to provide a method, apparatus, and handheld digitization device for object digitization and facial scanning, in order to solve the technical problem of how to truly realize object digitization.

[0004] In a first aspect, this application provides a method for digitizing an object, the method comprising: acquiring a spectral image of the object under a multi-node illumination source; acquiring a response function of the spectral image; recovering the reflection spectrum of the object based on the response function and a reflection database; acquiring a fine point cloud of the object; and digitizing the object based on the fine point cloud and the recovered reflection spectrum of the object.

[0005] In one implementation of the first aspect, the multi-node lighting source includes multiple node light sources, each of which is either a spectral light source with a single color channel or a spectral light source with N color channels. .

[0006] In one implementation of the first aspect, the node light source within the frontal area of ​​the object is provided with a spectral light source with N color channels. .

[0007] In one implementation of the first aspect, the spectral light source includes an LED light source and an LED collimator, a microlens array, a projection mirror, and a semi-transparent mirror arranged sequentially along the light transmission direction; the light beam emitted by the LED light source passes sequentially through the LED collimator, the microlens array, the projection mirror, and the semi-transparent mirror before being incident perpendicularly on the object.

[0008] In one implementation of the first aspect, the LED light source is a circular light source in which four LEDs of the same color channel are symmetrically arranged.

[0009] In one implementation of the first aspect, obtaining the spectral image of the object under the illumination source includes: controlling the spectral light source with N color channels to light up sequentially, and obtaining the multi-node spectral image of the object under the N color channels.

[0010] In one implementation of the first aspect, the response function of the spectral image is: ,in, It is the spectral response of each pixel in a spectral image with N color channels. Let N be the emission spectrum of the spectral light source with N color channels, where N∈[3,15]. S is the reflection spectrum of the object, and S is the number of wavelength points corresponding to the N color channels, with the wavelength point range being [350, 800].

[0011] In one implementation of the first aspect, the method of acquiring the reflection database includes: acquiring multiple sample objects of the same type as the object; acquiring sample reflection spectra of the multiple sample objects based on a spectral light source with N color channels, where N∈[3,15]; acquiring multiple spectral clusters based on the spectral similarity of the multiple sample reflection spectra; acquiring M representative reflection spectra based on the multiple spectral clusters; and using the result of multiplying the M representative reflection spectra with the emission spectra of the spectral light source with N color channels as the reflection data of the reflection database.

[0012] In one implementation of the first aspect, recovering the reflectance spectrum of the object based on the response function and the reflectance database includes: comparing each pixel in the spectral image of N color channels with the reflectance data to generate a weight matrix; and recovering the reflectance spectrum of the object based on the weight matrix.

[0013] In one implementation of the first aspect, recovering the reflectance spectrum of the object based on the weight matrix includes:

[0014] The restored reflectance spectrum of the object is as follows:

[0015]

[0016] in, Let be the p-th principal component coefficient of the object, and , .

[0017] In one implementation of the first aspect, obtaining the fine point cloud of the object includes: obtaining the coarse point cloud of the object; obtaining the surface normal of the object based on the multi-node illumination source; and obtaining the fine point cloud based on the coarse point cloud and the surface normal.

[0018] In one implementation of the first aspect, obtaining the coarse point cloud of the object includes: obtaining multi-angle images of the object; calculating a basic matrix based on feature matching between the multi-angle images; calculating a camera matrix based on the basic matrix; and obtaining the coarse point cloud of the object based on the camera matrix.

[0019] In one implementation of the first aspect, obtaining the surface normal of the object based on the multi-node illumination source includes: obtaining a first brightness value and a second brightness value of the object; and obtaining the surface normal based on a comparison between the first brightness value and the second brightness value.

[0020] In one implementation of the first aspect, the brightness of the object under illumination by multiple node light sources of the same light intensity is used as the first brightness value.

[0021] In one implementation of the first aspect, the brightness of the object under illumination by multiple node light sources of different intensities is used as a second brightness value; wherein the light intensity of the node light source is controlled by a light intensity function.

[0022] In one implementation of the first aspect, the plurality of said node light sources are distributed in a light cage shape.

[0023] In one implementation of the first aspect, the plurality of node light sources are distributed in a linear shape, the linear shape including linearity in the X direction and linearity in the Y direction.

[0024] In one implementation of the first aspect, the brightness of the object under illumination by multiple node light sources of different intensities is used as a second brightness value; wherein the light intensity of the node light source is controlled by a light intensity function and a distance function.

[0025] In one implementation of the first aspect, the plurality of node light sources are distributed in a planar shape.

[0026] In one implementation of the first aspect, the light intensity function is:

[0027]

[0028] in, Let k be the light intensity of the light source at the i-th node, and k be a constant. Let be the angle of the i-th node relative to the object.

[0029] In one implementation of the first aspect, the distance function is:

[0030]

[0031] Where d is the distance between the i-th node light source and the object.

[0032] In one implementation of the first aspect, obtaining a fine point cloud based on the coarse point cloud and the surface normal includes: obtaining the normal of the coarse point cloud; correcting the coarse point cloud based on the surface normal and the normal of the coarse point cloud to obtain a corrected point cloud; iteratively performing the correction operation until the surface normal and the normal of the corrected point cloud are compared and a preset condition is met; wherein, the correction operation is to correct the corrected point cloud based on the surface normal and the normal of the corrected point cloud to obtain a new corrected point cloud.

[0033] In one implementation of the first aspect, obtaining a fine point cloud based on the rough point cloud and the surface normal includes: obtaining the normal of the rough point cloud; optimizing the normal based on the bi Laplace equation; wherein the surface normal is used as a von Neumann boundary condition.

[0034] Secondly, this application provides an object digitization device, the device comprising: an image acquisition module for acquiring a spectral image of an object under a multi-node illumination source; a response module for acquiring a response function of the spectral image; a spectral restoration module for restoring the reflection spectrum of the object based on the response function and a reflection database; a geometric restoration module for acquiring a fine point cloud of the object; and a digitization module for digitizing the object based on the fine point cloud and the restored reflection spectrum of the object.

[0035] In one implementation of the second aspect, the device includes an illumination module for providing the multi-node illumination source; and the illumination module for controlling the light intensity of the multi-node illumination source.

[0036] In one implementation of the second aspect, the image acquisition module is further configured to acquire multi-angle images of the object.

[0037] Thirdly, this application provides a facial scanning method, the method comprising digitally scanning the face using the object digitization method as described in the first aspect to obtain the reflectance spectrum and three-dimensional shape of the face; and evaluating the facial cosmetic effect by obtaining at least one of the following indicators based on the reflectance spectrum of the face: melanin concentration, epidermal surface thickness, blood volume, and oxygen content.

[0038] In one implementation of the third aspect, the method further includes: evaluating facial health based on the reflectance spectrum and three-dimensional shape of the face.

[0039] In one implementation of the third aspect, the method further includes: evaluating the matching effect of the cosmetic and the face based on the reflectance spectrum of the face and the reflectance spectrum of the cosmetic.

[0040] Fourthly, this application provides a facial scanning device, the device including the object digitization device as described in the second aspect.

[0041] Fifthly, this application provides a handheld digitizing device, the device including the object digitizing apparatus as described in the second aspect; wherein the image acquisition module is further configured to acquire the spatial position and orientation of a multi-node illumination source relative to the object.

[0042] In one implementation of the fifth aspect, the lighting module controls the light intensity of the multi-node lighting source based on the spatial position and orientation of the multi-node lighting source relative to the object.

[0043] As described above, the object digitization method and apparatus, facial scanning method and apparatus, and handheld digitization device described in this application have the following beneficial effects:

[0044] 1) This application provides a set of lighting source designs composed of different spectra and constructs a response function, which can quickly recover the true reflection spectrum of the object and the high-resolution three-dimensional shape of the object, create a real digital avatar, and then apply it to digital fields such as the metaverse.

[0045] 2) This application can effectively realize facial digitization and conduct facial health monitoring and beauty treatments based on it, find the best cosmetics with spectral matching under any light, effectively ensure the health of the human face, and provide more effective results for health care and beauty.

[0046] 3) This application provides a handheld digitizing device that can effectively digitize physical objects in any scenario, improving the convenience and speed of digitization. Attached Figure Description

[0047] Figure 1 The flowchart shown is an embodiment of the object digitization method described in this application.

[0048] Figure 2 The diagram shown is a structural schematic of the spectral light source described in one embodiment of this application.

[0049] Figure 3The diagram shown is a structural schematic of an LED light source described in one embodiment of this application.

[0050] Figure 4 The diagram shows the wavelength range covered by the spectral light source for each color channel as described in the embodiments of this application.

[0051] Figure 5 The diagram shown is an illumination schematic of the multi-node lighting source described in an embodiment of this application.

[0052] Figure 6 The flowchart shown is an embodiment of the object digitization method described in this application.

[0053] Figure 7 The flowchart shown is an embodiment of the object digitization method described in this application.

[0054] Figure 8 This diagram shows the distribution shape of the plurality of node light sources described in the embodiments of this application.

[0055] Figures 9a to 9b The diagram illustrates the difference between the actual spectral curve and the measured spectral curve obtained by scanning facial skin using the facial scanning method described in the embodiments of this application.

[0056] Figure 10 The diagram shown is a structural schematic of an embodiment of the object digitization device described in this application.

[0057] Figure 11 The diagram shown is a structural schematic of the electronic device of this application in one embodiment. Detailed Implementation

[0058] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0059] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0060] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. If the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0061] Object digitization refers to converting physical objects in the real world into digital forms and applying them in various digital applications. Currently, object digitization is often achieved by constructing point clouds of objects to create high-resolution 3D shapes, and then applying these 3D digital avatars to applications such as games and the web. Taking the metaverse as an example, it encompasses various aspects such as economy, politics, culture, and technology, representing an advanced version of the digital society. Under metaverse technology, all things and affairs in the real world exist and operate based on digital space; nothing is not virtual, and nothing is not real, achieving a complete fusion of the virtual and the real. It can be said that the metaverse extends humanity's existence from the real world to the digital world, representing a new form of human existence. However, in the current digitization process, there is often a lack of attention to the spectral density of physical objects, resulting in a certain degree of distortion in the spectral density of digital objects, making it difficult to fully match the real world. This affects the application of many digital scenarios. Furthermore, in the fields of health and medical aesthetics, object digitization methods are also used to digitize the human face to better analyze skin composition indicators for problems or to evaluate the effects of cosmetic procedures. However, due to the distortion of the skin's spectrum, the results of medical or cosmetic assessments are affected to some extent, making it difficult to effectively guarantee the health of the skin.

[0062] To at least address the aforementioned technical problems, embodiments of this application provide a method, apparatus, and handheld digitization device for object digitization and facial scanning, which solve the technical problem of how to truly achieve object digitization.

[0063] Please see Figure 1 As shown, the object digitization method provided in this application includes:

[0064] S1. Obtain the spectral image of the object under a multi-node lighting source.

[0065] To restore the true reflectance spectrum of an object under different spectral illuminations and ensure the authenticity of the digital object, the multi-node illumination source designed in this application includes multiple node light sources to illuminate the object. Each node light source is equipped with a single color channel spectral light source or a spectral light source with N color channels. .

[0066] Furthermore, the node light source within the frontal area of ​​the object is equipped with a spectral light source with N color channels. .

[0067] It should be noted that the spectral light source of a single color channel should be included in N color channels. That is, a multi-node illumination source actually provides illumination for N color channels, of which some node light sources (outside the front view of the object) only provide a single color from the N color channels.

[0068] In some embodiments, please refer to Figure 2 As shown, the spectral light source for each color channel includes an LED light source 21 and an LED collimator 22, a microlens array 23, a projection mirror 24, and a semi-transparent mirror 25 arranged sequentially along the light transmission direction. The light beam emitted by the LED light source 21 passes sequentially through the LED collimator 22, the microlens array 23, the projection mirror 24, and the semi-transparent mirror 25 before being incident perpendicularly on the object.

[0069] For further details, please refer to Figure 3 As shown, the LED light source is a circular light source with four LEDs of the same color channel symmetrically arranged to provide the best spatial uniformity. That is, the spectral light source for each color channel is provided by the four circular light sources with four LEDs of the same color channel symmetrically arranged. Therefore, the spectral light source designed in this application can ensure that any light point in the illuminated area receives uniform light: the same illuminance in space and the same spectral composition. For example, point A and point B are two areas in the illuminated area. They have the same irradiance for any color channel of the spectral light source.

[0070] It should be noted that if a spectral light source with N color channels is set at a node, then LEDs of N colors can be integrated onto the same LED light source 21. That is, four LEDs of each color are arranged symmetrically, and LEDs of different colors together form a symmetrically arranged circular light source. Figure 3 For example, an LED light source 21 integrating four colors is shown, wherein each of the four colors has four identical LEDs arranged symmetrically. At this time, the LED light source 21 at a node can be controlled to emit light of the four colors, which then pass through the same LED collimator 22, microlens array 23, projection mirror 24, and semi-transparent mirror 25.

[0071] For further details, please refer to Figure 4As shown, the spectral light source for each color channel covers a specific wavelength range (with peak wavelength and spectral width). When providing a multi-node illumination source, several spectral light sources for different color channels can be selected according to the different needs of the object, thereby performing spectral recovery of the object's surface.

[0072] Furthermore, in some embodiments, white spectral light sources with different spectral shapes can be selectively used. Unlike spectral light sources with other color channels, white spectral light sources can extend to the entire visible light spectrum (400-700 nm) or even wider, thus enabling better spectral recovery of objects in some scenarios.

[0073] It should be noted that this application does not impose any restrictions on the specific color channel selection required for each type of object.

[0074] by Figure 5 For example, in the above embodiment, the multi-node illumination source illuminates the object. At this time, the spectral light sources controlling N color channels (spectral light sources of the same color channel on different nodes) are sequentially lit, acquiring the spectral image of the object in the N color channels. Please refer to... Figure 5 As shown, in some embodiments, a camera lens can be positioned behind each node light source to acquire spectral images at the corresponding node location (acquiring spectral images of multiple nodes of the object under illumination by spectral light sources with different color channels). In other embodiments, a single camera lens can be moved to acquire spectral images at multiple node locations. That is, for each node (corresponding to different illumination positions of the object), multiple spectral images of the object (at different shooting positions) are acquired under illumination by spectral light sources with different color channels. Of course, the camera lens can also be positioned in front of the object to acquire only the spectral image within the frontal area. This application does not impose any restrictions on the acquisition position of the spectral images.

[0075] Furthermore, the camera lens can be either a monochrome camera lens or a color camera lens; this application makes no restrictions on this.

[0076] S2. Obtain the response function of the spectral image.

[0077] For a spectral image, each pixel possesses complete spectral information. Therefore, by recovering the spectral information of each pixel in the spectral image of an object, the complete spectrum of the corresponding region of the object in the spectral image can be obtained. To this end, for each pixel in a spectral image with N color channels, the following response function is obtained:

[0078]

[0079] in, It is the spectral response of each pixel in a spectral image with N color channels. It is the emission spectrum of the spectral light source with N color channels. , It is the reflection spectrum of the object, and S is the number of wavelength points corresponding to the N color channels, with the wavelength point range being [350, 800].

[0080] As mentioned above, the spectral light source for each color channel covers a specific wavelength range (with peak wavelength and spectral intensity). Therefore, in the above formula, for each value of N, there is a corresponding range of wavelength points.

[0081] S3. Recover the reflection spectrum of the object based on the response function and the reflection database.

[0082] After obtaining the response function of the spectral image, each pixel in the spectral image can be reconstructed based on the reflection database by comparing the response function with the reflection database, thereby obtaining a true and unbiased reflection spectrum. Therefore, the reflection database is actually a database of reflection spectra of objects of the same type as the target object. That is, objects of the same type have similar spectral shapes, or their spectral variations follow certain patterns. Therefore, by simply establishing a database of reflection spectra of objects of the same type, the target spectrum can be accurately reconstructed by comparing this database with the obtained target object.

[0083] Please see Figure 6 As shown, the methods for obtaining the reflection database include:

[0084] S31. Obtain multiple sample objects of the same type as the object.

[0085] S32. Obtain the sample reflectance spectra of multiple sample objects based on a spectral light source with N color channels. .

[0086] Specifically, multiple sample objects are illuminated by a multi-node illumination source with the same design, and the reflectance spectra of the corresponding sample objects are obtained. That is, when digitizing an object, a reflectance database of similar sample objects must be established beforehand. In this case, the same multi-node illumination source design should be used, including ensuring that the color channels of the spectral sources on the multiple nodes are consistent.

[0087] S33. Obtain multiple spectral clusters based on the spectral similarity of the reflectance spectra of multiple samples.

[0088] S34. Obtain M representative reflectance spectra based on multiple spectral clusters.

[0089] In some embodiments, a total of 3500 sample reflectance spectra are acquired. Since the reflectance spectra of multiple sample objects are similar, the 3500 spectra can be clustered using PCA or K-means methods to obtain 6 spectral clusters. Then, within each spectral cluster, 50 sample emission spectra are selected, and these 50 emission spectra should be uniformly distributed within the cluster. Thus, for the 6 spectral clusters, a total of M=300 representative reflectance spectra can be obtained, which can represent the reflectance spectra of all samples.

[0090] S35. The result of multiplying the M representative reflection spectra with the emission spectra of the N color channels of the spectral light source is used as the reflection data of the reflection database.

[0091] Taking M=300 as an example, multiplying the 300 representative reflectance spectra with the emission spectra of the spectral light source from N color channels yields the reflectance data in the reflectance database. Therefore, by clustering and selecting M representative reflectance spectra, the computational load can be significantly reduced, the efficiency of spectral recovery can be improved, and the M representative reflectance spectra can be guaranteed to represent the true spectral information of the sample object, which helps to accurately recover the target spectrum of the object.

[0092] Based on the pixel response functions described above, the reflectance data should actually be the true spectral information of each pixel of the sample object. Therefore, by comparing each pixel (response function) in the spectral image of N color channels with the reflectance data, a weight matrix can be generated, and the reflectance spectrum of the object can be recovered using the weight matrix.

[0093] Specifically, the weight matrix Wi is obtained based on the weighted least squares method.

[0094] Specifically, the reflectance spectrum of the object is recovered based on the weight matrix, and the recovered reflectance spectrum of the object is as follows:

[0095]

[0096] in, Let be the p-th principal component coefficient of the object, and , .

[0097] Since the spectral information of an object can be considered as a linear sum of its principal components—that is, the p principal components of the object actually contain its spectral information—recovering each principal component yields the fully recovered reflectance spectrum of the object. The number of principal components (the range of values ​​for p) should be determined using PCA (Principal Component Analysis), meaning that the value of p will differ for different types of objects. The specific number of principal components can be determined when obtaining representative reflectance spectra, based on the spectral analysis of multiple sample objects.

[0098] S4. Obtain the fine point cloud of the object.

[0099] For details, please refer to Figure 7 As shown, obtaining the fine point cloud of the object includes:

[0100] S41. Obtain the rough point cloud of the object.

[0101] In some embodiments, obtaining the coarse point cloud of the object includes:

[0102] 1) Obtain multi-angle images of the object.

[0103] 2) Calculate the basic matrix based on feature matching between the multi-angle images.

[0104] 3) Calculate the camera matrix based on the aforementioned basic matrix.

[0105] 4) Obtain the rough point cloud of the object based on the camera matrix.

[0106] Specifically, images of the object are acquired from multiple angles, i.e., multi-angle images are obtained. These images are then processed. First, two initial images are selected, and feature point detection can be performed on both images using SIFT, SURF, or ORB to find corresponding matching points. Next, the RANSAC method is used to estimate the optimal fundamental matrix and the index of the correct points, calculating the fundamental matrix for different image pairs. Then, the camera matrix (rotation matrix and translation vector) is calculated from the fundamental matrix. Finally, triangulation is used to calculate 3D points. Based on this, new images are gradually added, feature points are extracted from the new images, and they are matched with previous images to obtain new matching relationships. The camera pose of the new image relative to the previous images is estimated. Typically, PNP (Perspective-n-Point) is used. Under a known camera matrix, the 3D points at that angle are compared with the corresponding image point coordinates to estimate the shooting information at that angle. Triangulation is then performed using the new matching pairs and the estimated pose (rotation matrix and translation vector) to generate new 3D points. These newly generated 3D points are then merged with the previous point cloud to construct a larger sparse point cloud. This process is repeated for images from all angles to finally obtain the coarse point cloud.

[0107] In the above embodiments, it can be implemented using OpenCV.

[0108] In other embodiments, other methods may be selected to obtain the coarse point cloud of the object, such as the Time-of-Flight (TOF) principle, etc., and this application does not impose any restrictions on this.

[0109] It should be noted that when acquiring multi-angle images, it is only necessary to acquire multiple images from different angles. This application does not impose any restrictions on the color and intensity of the light source.

[0110] S42. Obtain the surface normal of the object based on the multi-node lighting source.

[0111] Specifically, obtaining the surface normal of the object based on the multi-node illumination source includes: obtaining a first brightness value and a second brightness value of the object, and obtaining the surface normal based on the comparison result of the first brightness value and the second brightness value.

[0112] In some embodiments, the brightness of the object under illumination by a plurality of nodal light sources with the same light intensity is used as the first brightness value.

[0113] In some embodiments, the brightness of the object under illumination by multiple nodal light sources of different intensities is used as a second brightness value; wherein the light intensity of the nodal light source is controlled by a light intensity function.

[0114] Specifically, the light intensity function is:

[0115]

[0116] in, Let k be the light intensity of the light source at the i-th node, and k be a constant. Let be the angle of the i-th node relative to the object.

[0117] In the above embodiments, the multiple node light sources are distributed in a light cage shape or a linear shape, and the linear shape includes linearity in the X direction and linearity in the Y direction. When the multiple node light sources are distributed in a light cage shape or a linear shape, the light sources can illuminate the object in an orderly manner from various directions, ensuring that the object receives uniform light from all angles.

[0118] Figure 8 The diagram shows multiple node light sources distributed in either a light cage shape or a linear shape in the X direction. Please refer to [link / reference]. Figure 8 As shown, taking the X direction as an example, all node light sources are set to the same light intensity, and the object at this time ( Figure 8 The brightness of the object (displayed as a face) is recorded as the first brightness value. Then, the light intensity of all node light sources is adjusted according to the aforementioned light intensity function, thereby achieving a gradual brightening or darkening of the light source from left to right (gradient lighting), and the object at this time (…) is… Figure 8 The brightness of the face (displayed in the image) is recorded as the second brightness value. Finally, the first brightness value and the second brightness value are compared to obtain the surface normal.

[0119] In other embodiments, the brightness of the object under illumination by multiple nodal light sources of different intensities is used as a second brightness value; wherein the light intensity of the nodal light source is controlled by a light intensity function and a distance function.

[0120] Specifically, the distance function is:

[0121]

[0122] Where d is the distance between the i-th node light source and the object.

[0123] In the above embodiments, the multiple node light sources are distributed in a planar shape. The light sources can also illuminate the object in an orderly manner from various directions of the planar shape, ensuring that the object receives uniform light from all angles.

[0124] Specifically, when multiple node light sources are distributed in a planar shape, all node light sources are still set to the same light intensity, and the object at this time ( Figure 8The brightness of the face (displayed in the image) is recorded as the first brightness value. Subsequently, when designing gradient lighting, the light intensity of all nodal light sources needs to be adjusted according to direction and distance. That is, the light intensity of all nodal light sources is adjusted according to the aforementioned light intensity function and distance function, thereby achieving a gradual brightening or dimming of the light source from left to right. At this time, the light intensity of all nodal light sources is actually consistent with that when distributed in a light cage shape. Finally, the first brightness value and the second brightness value are compared to obtain the surface normal.

[0125] S43. Obtain a fine point cloud based on the rough point cloud and the surface normal.

[0126] Specifically, obtaining a fine point cloud based on the coarse point cloud and the surface normal includes:

[0127] S431. Obtain the normal of the rough point cloud.

[0128] S432. Based on the surface normal and the normal of the rough point cloud, the rough point cloud is corrected to obtain a corrected point cloud.

[0129] S433. Iteratively execute the correction operation until the surface normal and the normal of the corrected point cloud are compared and a preset condition is met; wherein, the correction operation is to correct the corrected point cloud based on the surface normal and the normal of the corrected point cloud to obtain a new corrected point cloud.

[0130] In the above embodiment, after obtaining the coarse point cloud of the object in step S41, further optimization is needed to obtain a fine point cloud, thereby creating a realistic digital object. Therefore, using the coarse point cloud obtained in step S41 as a base, the normal of the coarse point cloud is obtained. Then, the high-frequency portion of the surface normal obtained in step S42 is added to the normal of the coarse point cloud, and combined with the coarse point cloud to calculate a more refined point cloud, i.e., a corrected point cloud. Afterward, the normal of the corrected point cloud is obtained, and the surface normal and the normal of the corrected point cloud are compared to determine if a preset threshold condition is met. If not, the same process is continued, adding the high-frequency portion of the surface normal to the normal of the corrected point cloud, and combining it with the corrected point cloud to calculate a more refined point cloud, i.e., a new corrected point cloud. This process is iterated until the final fine point cloud is obtained.

[0131] In other embodiments, a fine point cloud can be obtained by acquiring the normals of the coarse point cloud and optimizing them based on the bi Laplace equation. Here, the surface normals are used as von Neumann boundary conditions.

[0132] In the above embodiments, the normals of the coarse point cloud or the modified point cloud can be obtained using the PCA method. First, the covariance matrix of the coarse point cloud or the modified point cloud is calculated, and then the eigenvectors and eigenvalues ​​of the covariance matrix are calculated. Finally, the eigenvector corresponding to the smallest eigenvalue is selected as the estimated normal direction.

[0133] In other embodiments, other methods may be selected to obtain the normals of the coarse point cloud or the corrected point cloud, such as surface reconstruction, etc., and this application does not impose any restrictions on this.

[0134] S5. Digitize the object based on the fine point cloud and the reflected spectrum of the recovered object.

[0135] Specifically, after obtaining the fine point cloud and the restored reflectance spectrum of the object, any object can be digitized based on both and copied in a 3D or 2D graphics viewing portal (such as a computer, monitor, or XR headset).

[0136] Because it combines the 3D and spectral information of the object, the digital object digitized using the object digitization method provided in this application has the highest accuracy and can be widely used in digital fields such as metaverse and VR / AR. It can also be used to detect human skin and determine the health of the skin.

[0137] Therefore, this application embodiment also provides a facial scanning method, which can digitally scan the face using the object digitization method provided in the above embodiment to obtain the reflectance spectrum and three-dimensional shape of the face.

[0138] In some embodiments, after obtaining the reflectance spectrum and three-dimensional shape of the face, at least one of the following indicators—melanin concentration, epidermal surface thickness, blood volume, and oxygen content—can be obtained based on the reflectance spectrum of the face to evaluate the facial cosmetic effect.

[0139] Specifically, after facial cosmetic procedures, the desired effect requires precise measurement of facial shape details. This can be obtained through accurate changes in three-dimensional shape information. Furthermore, indicators such as melanin concentration, epidermal surface thickness, blood volume, and oxygen content can be obtained based on reflectance spectra. Therefore, by analyzing changes in three-dimensional and spectral information before and after the cosmetic procedure, the effectiveness can be assessed.

[0140] Figure 9a and Figure 9b The diagram illustrates the difference between the actual spectral curve and the measured spectral curve obtained by scanning facial skin using the facial scanning method provided in this application embodiment. Figure 9aThe difference between the actual spectral curve and the measured spectral curve is not significant. Figure 9a The melanin concentration, blood volume, epidermal surface thickness, and oxygen content obtained based on the true reflectance spectrum were 38.83%, 4.19%, 42.51 μm, and 10.25%, respectively. Figure 9b The actual spectral curve and the measured spectral curve differ significantly. Figure 9b The melanin concentration, blood volume, epidermal surface thickness, and oxygen content obtained based on the true reflectance spectrum were 6.33%, 7.75%, 46.35 μm, and 85.66%, respectively. This demonstrates that the facial scanning method described in this application can better obtain the true spectral curve of the skin and the accurate values ​​of melanin concentration, epidermal surface thickness, blood volume, and oxygen content, enabling a better evaluation of facial cosmetic effects.

[0141] In some embodiments, facial health can be evaluated based on the obtained reflectance spectrum and three-dimensional shape of the face.

[0142] Taking dark circles and complexion as examples, by scanning the face to obtain the skin's reflectance spectrum information, the causes of dark circles can be identified. Furthermore, after obtaining three-dimensional information (precise microstructure), facial health can be evaluated in a timely manner based on daily changes, preventing the occurrence of diseases such as skin cancer.

[0143] As described above, the facial scanning method of this application can better obtain the true spectral curve of the skin, thereby enabling timely monitoring of facial health and preventing the occurrence of diseases such as skin cancer.

[0144] In some embodiments, after obtaining the reflectance spectrum of the face, the reflectance spectrum of the cosmetic can also be obtained, and the matching effect between the cosmetic and the face can be evaluated based on this.

[0145] Specifically, after scanning the face to obtain its reflectance spectrum, the cosmetic product can also be scanned using the object digitization method provided in the above embodiments to obtain its reflectance spectrum. In this way, under any light source, the higher the similarity between the reflectance spectrum of the face and the reflectance spectrum of the cosmetic product, the higher the degree of matching between the cosmetic product and the face.

[0146] The scope of protection for the object digitization method and / or facial scanning method described in the embodiments of this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0147] This application also provides an object digitization device and / or a facial scanning device. The object digitization device can implement the object digitization method described in this application, and the facial scanning device can implement the facial scanning method described in this application. However, the devices for implementing the methods described in this application include, but are not limited to, the device structures listed in this embodiment. All structural modifications and substitutions of the prior art made based on the principles of this application are included within the protection scope of this application.

[0148] Please see Figure 10 As shown in the illustration, this application also provides an object digitization device, the device comprising:

[0149] Image acquisition module 41 is used to acquire the spectral image of the object under a multi-node illumination source.

[0150] The response module 42 is used to obtain the response function of the spectral image.

[0151] The spectral recovery module 43 is used to recover the reflection spectrum of the object based on the response function and the reflection database.

[0152] The geometric restoration module 44 is used to acquire the fine point cloud of the object.

[0153] The digitization module 45 is used to digitize the object based on the fine point cloud and the reflected spectrum of the recovered object.

[0154] Furthermore, the image acquisition module 41 is also used to acquire multi-angle images of the object.

[0155] Please continue reading. Figure 10 As shown, the object digitization device of this application also includes an illumination module 46. The illumination module 46 is used to provide the multi-node illumination source and to control the light intensity of the multi-node illumination source.

[0156] Specifically, the multi-node lighting source provided by the lighting module 46 includes multiple node light sources, each of which is a spectral light source with a single color channel or a spectral light source with N color channels. .

[0157] Furthermore, the node light source within the frontal area of ​​the object is equipped with a spectral light source with N color channels. .

[0158] The spectral light source includes an LED light source and an LED collimator, a microlens array, a projection mirror, and a semi-transparent mirror arranged sequentially along the light transmission direction. The light beam emitted by the LED light source passes sequentially through the LED collimator, the microlens array, the projection mirror, and the semi-transparent mirror before being incident perpendicularly on the object.

[0159] Furthermore, the LED light source is a circular light source with four LEDs of the same color channel arranged symmetrically to provide the best spatial uniformity.

[0160] Furthermore, multiple nodal light sources can be distributed in the shape of a light cage, a linear shape, or a planar shape. When distributed in a linear shape, it includes linearity in the X direction and linearity in the Y direction.

[0161] When multiple node light sources are distributed in a light cage shape or a linear shape, the light intensity of each node light source is controlled by a light intensity function to adjust the light intensity of the node light source according to different angles (with the object as a reference).

[0162] When multiple nodal light sources are distributed in a planar shape, the light intensity of each nodal light source is controlled by a light intensity function and a distance function, so as to adjust the light intensity of the nodal light source according to different angles and distances (with the object as a reference).

[0163] The light intensity function is:

[0164]

[0165] in, Let k be the light intensity of the light source at the i-th node, and k be a constant. Let be the angle of the i-th node relative to the object.

[0166] The distance function is:

[0167]

[0168] Where d is the distance between the i-th node light source and the object.

[0169] Specifically, when an object needs to be digitized, the object is placed in a lighting area and illuminated by a multi-node lighting source provided by the lighting module 46. At this time, the lighting module 46 controls N (… The spectral light sources in each of the N color channels are sequentially illuminated, and the image acquisition module 41 acquires the spectral image of the object in N color channels. Then, the response module 42 acquires the response function of the spectral image, and the spectral restoration module 43 restores the reflection spectrum of the object based on the response function and the reflection database. Simultaneously, the illumination module 46 controls the light intensity of multiple node light sources to provide the gradient illumination required for geometric restoration, and the image acquisition module 41 also acquires multi-angle images of the object so that the geometric restoration module 44 can ultimately acquire the fine point cloud of the object. Finally, the digitization module 45, after acquiring the three-dimensional and spectral information of the object, can achieve digitization.

[0170] It should be noted that the structure and principle of the image acquisition module 41, response module 42, spectral restoration module 43, geometric restoration module 44, digitization module 45 and illumination module 46 can be referred to the content of the above method embodiment, and will not be repeated here.

[0171] This application also provides a facial scanning device, which includes the object digitization device provided in the above embodiments. The specific structure and principle of the facial scanning device can be referred to the above description and will not be repeated here.

[0172] This application also provides a handheld digitizing device, which includes the object digitizing device provided in the above embodiments.

[0173] The image acquisition module is also used to acquire the spatial position, orientation, and distance of the multi-node lighting source relative to the object.

[0174] The lighting module controls the light intensity of the multi-node lighting source based on the spatial position and orientation of the multi-node lighting source relative to the object.

[0175] Specifically, taking a mobile phone as an example, the phone's camera is the image acquisition module. The lighting module can be installed in mobile phone accessories or used as a flashlight, becoming part of the phone. This application does not impose any restrictions on this. Taking the digitization of a face using a mobile phone as an example, the principle is as follows:

[0176] Hold the phone face-to-face and illuminate it using the multi-node light source provided by the flashlight. At this time, the flashlight controls N (…). A spectral light source with N color channels is sequentially illuminated, and a camera captures spectral images of the face across these N color channels. The response function of the spectral image is then obtained, and the reflectance spectrum of the object is reconstructed based on the response function and a reflectance database. Simultaneously, the phone is moved to create a virtual light source distribution shape. The phone's IMU / camera can be used to determine the phone's (flashlight's) position and orientation in space, as well as its distance relative to the face. For example, when the phone moves around the face, forming a light cage or linear shape, the flashlight's brightness can be controlled based on position and orientation (light intensity function) to provide gradient illumination for geometric reconstruction. Conversely, when the phone moves around the face, forming a planar shape, the flashlight's brightness can be controlled based on position, orientation, and distance (light intensity function and distance function) to provide gradient illumination for geometric reconstruction. The camera also captures images from multiple angles, allowing for the acquisition of a fine point cloud of the object based on the gradient illumination and the multi-angle images. Finally, after acquiring the object's 3D and spectral information, digitization is achieved.

[0177] In other embodiments, a drone / robotic arm-based system can also be used to scan along a pre-designed path.

[0178] As can be seen, the object digitization and facial scanning method, apparatus, and handheld digitization device provided in this application can quickly recover the true reflectance spectrum and high-resolution three-dimensional shape of an object, creating a realistic digital avatar, which can then be applied to digital fields such as the metaverse. Furthermore, this application can effectively achieve facial digitization and, based on this, perform facial health monitoring and cosmetic procedures, finding the best cosmetics with spectral matching under any lighting conditions, effectively ensuring the health of the human face. Finally, this application provides a handheld digitization device that can effectively digitize physical objects in any scenario, improving the convenience and speed of digitization.

[0179] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0180] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0181] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0182] This application also provides a computer-readable storage medium. Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)).

[0183] This application also provides an electronic device. The electronic device includes a processor and a memory.

[0184] The memory is used to store computer programs.

[0185] The memory includes various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card, or optical disk.

[0186] The processor is connected to the memory and is used to execute the computer program stored in the memory to enable the electronic device to perform the above-described object digitization method and / or facial scanning method.

[0187] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0188] like Figure 11 As shown, the electronic device of this application is embodied in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: one or more processors or processing units 51, memory 52, and bus 53 connecting different system components (including memory 52 and processing unit 51).

[0189] Bus 53 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0190] Electronic devices typically include a variety of computer-readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, and removable and non-removable media.

[0191] Memory 52 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 521 and / or cache memory 522. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 523 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 11 Not shown; usually referred to as a "hard drive"). Although Figure 11As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 53 via one or more data media interfaces. Memory 52 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0192] A program / utility 524 having a set (at least one) of program modules 5241 may be stored, for example, in memory 52. ​​Such program modules 5241 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 5241 typically perform the functions and / or methods described in the embodiments of this application.

[0193] The electronic device can also communicate with one or more external devices (e.g., keyboard, pointing device, display, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., network interface card, modem, etc.). This communication can be performed through input / output (I / O) interface 54. Furthermore, the electronic device can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 55. Figure 11 As shown, network adapter 55 communicates with other modules of the electronic device via bus 53. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0194] This application embodiment may also provide a computer program product comprising one or more computer instructions. When the computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in this application embodiment are generated. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0195] When the computer program product is executed by a computer, the computer performs the method described in the foregoing method embodiments. The computer program product can be a software installation package; when the foregoing method is required, the computer program product can be downloaded and executed on the computer.

[0196] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0197] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for digitizing objects, characterized in that, The method includes: Acquire the spectral image of an object with N color channels under a multi-node illumination source; the node light sources within the frontal area of ​​the object are equipped with spectral light sources with N color channels. ; Obtain the response function of the spectral image; The process of recovering the reflectance spectrum of the object based on the response function and the reflectance database includes: comparing the spectral response of each pixel in the spectral image with N color channels with the reflectance data to generate a weight matrix; and recovering the reflectance spectrum of the object based on the weight matrix. Obtain the fine point cloud of the object; The object is digitized based on the refined point cloud and the recovered reflectance spectrum of the object; wherein, the implementation methods for obtaining the reflectance database include: Obtain multiple sample objects of the same type as the object being described; The sample reflectance spectra of multiple sample objects are obtained based on a spectral light source with N color channels, where N∈[3,15]; Multiple spectral clusters are obtained based on the spectral similarity of the reflectance spectra of the multiple samples; M representative reflectance spectra are obtained based on multiple spectral clusters; The reflection data of the reflection database is obtained by multiplying the M representative reflection spectra with the emission spectra of the N color channels of the spectral light source.

2. The object digitization method according to claim 1, characterized in that, The spectral light source includes an LED light source and an LED collimator, a microlens array, a projection mirror, and a semi-transparent mirror arranged sequentially along the light transmission direction; the light beam emitted by the LED light source passes sequentially through the LED collimator, the microlens array, the projection mirror, and the semi-transparent mirror before being incident perpendicularly on the object.

3. The object digitization method according to claim 2, characterized in that, The LED light source is a circular light source with four LEDs of the same color channel arranged symmetrically.

4. The object digitization method according to claim 1, characterized in that, Obtaining the spectral image of the object under the illumination source includes: The spectral light sources of N color channels are controlled to light up sequentially, thereby acquiring a multi-node spectral image of the object in the N color channels.

5. The object digitization method according to claim 4, characterized in that, The response function of the spectral image is: in, It is the spectral response of each pixel in a spectral image with N color channels. It is the emission spectrum of the spectral light source with N color channels. It is the reflection spectrum of the object, and S is the number of wavelength points corresponding to the N color channels, with the wavelength point range being [350, 800].

6. The object digitization method according to claim 1, characterized in that, Recovering the reflectance spectrum of the object based on the weight matrix includes: The restored reflectance spectrum of the object is as follows: in, Let be the coefficient of the p-th principal component of the reflectance spectrum of the object, and , .

7. The object digitization method according to claim 1, characterized in that, Obtaining the fine point cloud of the object includes: Obtain the rough point cloud of the object; The surface normal of the object is obtained based on the multi-node lighting source; A fine point cloud is obtained based on the rough point cloud and the surface normal.

8. The object digitization method according to claim 7, characterized in that, Obtaining the coarse point cloud of the object includes: Obtain multi-angle images of the object; The basic matrix is ​​calculated based on feature matching between the multi-angle images; Calculate the camera matrix based on the aforementioned basic matrix; The rough point cloud of the object is obtained based on the camera matrix.

9. The object digitization method according to claim 7, characterized in that, Obtaining the surface normal of the object based on the multi-node illumination source includes: Obtain the first brightness value and the second brightness value of the object; The surface normal is obtained based on the comparison between the first brightness value and the second brightness value.

10. The object digitization method according to claim 9, characterized in that, The brightness of the object under illumination by multiple nodal light sources of the same intensity is taken as the first brightness value.

11. The object digitization method according to claim 9, characterized in that, The brightness of the object under illumination by multiple nodal light sources of different intensities is taken as a second brightness value; wherein, the light intensity of the nodal light source is controlled by a light intensity function.

12. The object digitization method according to claim 11, characterized in that, The multiple node light sources are distributed in a light cage shape.

13. The object digitization method according to claim 11, characterized in that, The multiple node light sources are distributed in a linear shape, which includes linearity in the X direction and linearity in the Y direction.

14. The object digitization method according to claim 9, characterized in that, The brightness of the object under illumination by multiple nodal light sources with different light intensities is taken as the second brightness value; wherein, the light intensity of the nodal light source is controlled by a light intensity function and a distance function.

15. The object digitization method according to claim 14, characterized in that, The multiple node light sources are distributed in a planar shape.

16. The object digitization method according to claim 11 or 14, characterized in that, The light intensity function is: in, Let k be the light intensity of the light source at the i-th node, and k be a constant. Let be the angle of the i-th node relative to the object.

17. The object digitization method according to claim 14, characterized in that, The distance function is: Where d is the distance between the i-th node light source and the object.

18. The object digitization method according to claim 7, characterized in that, Obtaining a fine point cloud based on the rough point cloud and the surface normal includes: Obtain the normals of the coarse point cloud; Based on the surface normal and the normal of the rough point cloud, the rough point cloud is corrected to obtain a corrected point cloud. The correction operation is performed iteratively until the surface normal and the normal of the corrected point cloud are compared and a preset condition is met; wherein, the correction operation is to correct the corrected point cloud based on the surface normal and the normal of the corrected point cloud to obtain a new corrected point cloud.

19. The object digitization method according to claim 7, characterized in that, Obtaining a fine point cloud based on the rough point cloud and the surface normal includes: Obtain the normals of the coarse point cloud; The normal is optimized based on the bi Laplace equation, wherein the surface normal is used as a Neumann boundary condition.

20. An object digitization device, characterized in that, The device includes: The image acquisition module is used to acquire the spectral image of an object with N color channels under a multi-node illumination source; the node light sources within the frontal area of ​​the object are equipped with spectral light sources with N color channels. ; A response module is used to obtain the response function of the spectral image; The spectral restoration module is used to restore the reflectance spectrum of the object based on the response function and the reflectance database, including: comparing the spectral response of each pixel in the spectral image of N color channels with the reflectance data to generate a weight matrix; and restoring the reflectance spectrum of the object based on the weight matrix. The geometric restoration module is used to acquire the fine point cloud of the object. The digitization module is used to digitize the object based on the fine point cloud and the recovered reflectance spectrum of the object; wherein, the implementation method for obtaining the reflectance database includes: Obtain multiple sample objects of the same type as the object being described; The sample reflectance spectra of multiple sample objects are obtained based on a spectral light source with N color channels, where N∈[3,15]; Multiple spectral clusters are obtained based on the spectral similarity of the reflectance spectra of the multiple samples; M representative reflectance spectra are obtained based on multiple spectral clusters; The reflection data of the reflection database is obtained by multiplying the M representative reflection spectra with the emission spectra of the N color channels of the spectral light source.

21. The object digitization device according to claim 20, characterized in that, The device includes a lighting module for providing the multi-node lighting source; and The lighting module is used to control the light intensity of the multi-node lighting source.

22. The object digitization device according to claim 20, characterized in that, The image acquisition module is also used to acquire multi-angle images of the object.

23. A facial scanning method, characterized in that, The method uses the object digitization method as described in any one of claims 1-19 to digitally scan the face and obtain the reflectance spectrum and three-dimensional shape of the face; as well as The facial cosmetic effect is evaluated by obtaining data on at least one of the following indicators based on the facial reflectance spectrum: melanin concentration, epidermal surface thickness, blood volume, and oxygen content.

24. The facial scanning method according to claim 23, characterized in that, The method also includes: evaluating facial health based on the reflectance spectrum and three-dimensional shape of the face.

25. The facial scanning method according to claim 23, characterized in that, The method further includes: evaluating the matching effect of the cosmetics with the face based on the reflectance spectrum of the face and the reflectance spectrum of the cosmetics.

26. A facial scanning device, characterized in that, The apparatus includes an object digitization apparatus as described in any one of claims 20-22.

27. A handheld digital device, characterized in that, The device includes the object digitization apparatus as described in any one of claims 20-22; wherein... The image acquisition module is also used to acquire the spatial position and orientation of the multi-node lighting source relative to the object.

28. The handheld digital device according to claim 27, characterized in that, The lighting module controls the light intensity of the multi-node lighting source based on the spatial position and orientation of the multi-node lighting source relative to the object.

Images