Object digitization and face scanning method and device and handheld digitization equipment

Abstract

The invention provides an object digitization and face scanning method and device and handheld digitization equipment. The method comprises the following steps: acquiring a spectral image of an object under a multi-node illumination light source; acquiring a response function of the spectral image; recovering a reflection spectrum of the object based on the response function and a reflection database; obtaining a fine point cloud of the object; and digitizing the object object based on the fine point cloud and the recovered reflection spectrum of the object object. According to the method, the real reflection spectrum and the high-resolution three-dimensional shape of the object can be quickly recovered, so that a real digital body is created, and the method is applied to the digital fields such as the universe. On the basis, the reflection spectrum and the three-dimensional information of the face can be effectively acquired, so that personal face health monitoring and beautifying are carried out.

Description

Technical Field

[0001] The present application belongs to the field of digital technology, and in particular, relates to an object digitization and face scanning method, device and handheld digital equipment. Background Art

[0002] Information technology is a technology developed under the promotion of computer technology that uses electronic information to describe and process real-world affairs. The concept of digitization that has emerged in recent years is considered to be an upgraded version of informatization. The biggest difference between digitization and informatization is the scope and degree of informatization. Digitalization extends the informatization of things from simple management information to comprehensive digital and data descriptions. In digitization, all aspects of the object need to be digitized - holographic digitization, and accessed based on data. In other words, object digitization refers to converting physical objects in reality into digital form and applying them in various digital applications. Summary of the invention

[0003] The purpose of the present application is to provide an object digitization and face scanning method, apparatus and handheld digitization device, which are used to solve the technical problem of how to truly realize object digitization.

[0004] In a first aspect, the present application provides an object digitization method, the method comprising: obtaining a spectral image of an object under a multi-node lighting source; obtaining a response function of the spectral image; restoring a reflection spectrum of the object based on the response function and a reflection database; obtaining a fine point cloud of the object; and digitizing the object based on the fine point cloud and the restored reflection spectrum of the object.

[0005] In an implementation of the first aspect, the multi-node lighting light source includes a plurality of node light sources, each of the node light sources is provided with a spectral light source of a single color channel or a spectral light source of N color channels, N∈[3, 15].

[0006] In an implementation of the first aspect, a spectral light source with N color channels is provided on the node light source within the front range of the object, where N∈[3, 15].

[0007] In an implementation of the first aspect, the spectral light source includes an LED light source and an LED collimator, a microlens array, a projection mirror, and a semi-transparent and semi-reflective mirror arranged in sequence along a light transmission direction; the light beam emitted by the LED light source passes through the LED collimator, the microlens array, the projection mirror, and the semi-transparent and semi-reflective mirror in sequence, and then is vertically incident on the object.

[0008] In an implementation of the first aspect, the LED light source is a circular light source in which four LEDs of the same color channel are symmetrically arranged.

[0009] In an implementation of the first aspect, acquiring a spectral image of the object under the illumination light source includes: controlling the spectral light sources of N color channels to light up in sequence, and acquiring spectral images of multiple nodes of the object under the N color channels.

[0010] In an implementation of the first aspect, the response function of the spectral image is: RE[Nx1]=ALED[NxS*RefSpec[Sx1], wherein RE[Nx1] is each pixel in the spectral image of N color channels, ALED[NxS] is the emission spectrum of the spectral light source of N color channels, N∈[3,15], RefSpec[Sx1] is the reflectance spectrum of the object, S is the wavelength corresponding to the N color channels, S∈[350,800].

[0011] In an implementation of the first aspect, the implementation method of obtaining a reflection database includes: obtaining multiple sample objects of the same type as the object; obtaining sample reflection spectra of the multiple sample objects based on a spectral light source of N color channels, N∈[3, 15]; obtaining multiple spectral clustering clusters based on spectral similarity of the multiple sample reflection spectra; obtaining M representative reflection spectra based on the multiple spectral clustering clusters; and taking a result of multiplying the M representative reflection spectra with an emission spectrum of the spectral light source of N color channels as reflection data of the reflection database.

[0012] In an implementation of the first aspect, restoring the reflection spectrum of the object based on the response function and the reflection database includes: comparing each pixel in the spectral image of N color channels with the reflection data to generate a weight matrix; and restoring the reflection spectrum of the object based on the weight matrix.

[0013] In an implementation manner of the first aspect, the weight matrix is ​​obtained based on the weighted least squares method:

[0014]

[0015] Among them, the R i It is the multiplication result of the i-th representative reflection spectrum and the spectrum light source of N color channels, i∈[1,M].

[0016] In an implementation of the first aspect, restoring the reflection spectrum of the object based on the weight matrix includes:

[0017] The restored reflectance spectrum of the object is:

[0018] Spectest=∑a test-pRefSpec[Sx1]

[0019] Among them, a test-p is the pth principal component of the object, and a test-p =T·RE[Nx1], T=X[p×M]·W i [M×M]·R T [M×N]·inv[R N×M W iM×M R T M×N ]=[p×N].

[0020] In an implementation of the first aspect, obtaining a fine point cloud of the object includes: obtaining a rough point cloud of the object; obtaining a surface normal of the object based on the multi-node illumination light source; and obtaining a fine point cloud based on the rough point cloud and the surface normal.

[0021] In an implementation of the first aspect, obtaining a rough point cloud of the object includes: obtaining multi-angle images of the object; calculating a basic matrix based on feature matching between the multi-angle images; calculating a camera matrix based on the basic matrix; and obtaining a rough point cloud of the object based on the camera matrix.

[0022] In an implementation of the first aspect, obtaining the surface normal of the object based on the multi-node lighting source includes: obtaining a first brightness value and a second brightness value of the object; and obtaining the surface normal based on a comparison result of the first brightness value and the second brightness value.

[0023] In an implementation manner of the first aspect, the brightness of the object under illumination by a plurality of node light sources of the same light intensity is used as the first brightness value.

[0024] In an implementation of the first aspect, the brightness of the object under the illumination of a plurality of node light sources of different light intensities is used as a second brightness value; wherein the light intensity of the node light source is controlled by a light intensity function.

[0025] In an implementation manner of the first aspect, the plurality of node light sources are distributed in a light cage shape.

[0026] In an implementation manner of the first aspect, the plurality of node light sources are distributed in a linear shape, and the linear shape includes an X-direction linear shape and a Y-direction linear shape.

[0027] In an implementation of the first aspect, the brightness of the object under the illumination of a plurality of node light sources of different light intensities is used as a second brightness value; wherein the light intensity of the node light source is controlled by a light intensity function and a distance function.

[0028] In an implementation manner of the first aspect, the plurality of node light sources are distributed in a planar shape.

[0029] In an implementation manner of the first aspect, the light intensity function is:

[0030] L i = k·theta(i)

[0031] Among them, L i is the light intensity of the light source at the ith node, k is a constant, and theta(i) is the angle of the ith node relative to the object.

[0032] In an implementation manner of the first aspect, the distance function is:

[0033] L i =d 2

[0034] Wherein, d is the distance between the i-th node light source and the object.

[0035] In an implementation of the first aspect, obtaining a fine point cloud based on the rough point cloud and the surface normal includes: obtaining the normal of the rough point cloud; correcting the rough point cloud based on the surface normal and the normal of the rough point cloud to obtain a corrected point cloud; iteratively performing the correction operation until the surface normal is compared with the normal of the corrected point cloud and satisfies a preset condition; wherein the correction operation is to correct the corrected point cloud based on the surface normal and the normal of the corrected point cloud to obtain a new corrected point cloud.

[0036] In an implementation of the first aspect, obtaining a fine point cloud based on the rough point cloud and the surface normal includes: obtaining a normal of the rough point cloud; optimizing the normal based on a bi Laplace equation; wherein the surface normal is used as a Neumann boundary condition.

[0037] In a second aspect, the present application provides an object digitization device, comprising: an image acquisition module, used to acquire a spectral image of an object under a multi-node lighting source; a response module, used to acquire a response function of the spectral image; a spectral recovery module, used to restore the reflection spectrum of the object based on the response function and a reflection database; a geometric recovery module, used to acquire a fine point cloud of the object; and a digitization module, used to digitize the object based on the fine point cloud and the restored reflection spectrum of the object.

[0038] In an implementation manner of the second aspect, the device includes a lighting module, which is used to provide the multi-node lighting light source; and the lighting module is used to control the light intensity of the multi-node lighting light source.

[0039] In an implementation of the second aspect, the image acquisition module is further used to acquire multi-angle images of the object.

[0040] In a third aspect, the present application provides a facial scanning method, the method comprising digitally scanning the face using the object digitization method as described in the first aspect to obtain a reflectance spectrum and a three-dimensional shape of the face; and obtaining at least one indicator data of a facial melanin concentration index, an epidermal surface thickness index, a blood volume index, and an oxygen content index based on the reflectance spectrum of the face to evaluate the facial beauty effect.

[0041] In an implementation of the third aspect, the method further includes: evaluating facial health based on the reflectance spectrum and three-dimensional shape of the face.

[0042] In an implementation of the third aspect, the method further includes: evaluating a matching effect between the cosmetics and the face based on a reflectance spectrum of the face and a reflectance spectrum of the cosmetics.

[0043] In a fourth aspect, the present application provides a facial scanning device, comprising the object digitization device as described in the second aspect.

[0044] In a fifth aspect, the present application provides a handheld digitizing device, which includes the object digitizing apparatus as described in the second aspect; wherein the image acquisition module is also used to acquire the spatial position and direction of the multi-node lighting light source relative to the object.

[0045] In an implementation of the fifth aspect, the lighting module controls the light intensity of the multi-node lighting light source according to the spatial position and direction of the multi-node lighting light source relative to the object.

[0046] As described above, the object digitization method and apparatus, the face scanning method and apparatus, and the handheld digitization device described in the present application have the following beneficial effects:

[0047] 1) This application provides a set of lighting source designs composed of different spectra and constructs a response function, so as to quickly restore the real reflection spectrum of the object and the high-resolution three-dimensional shape of the object, create a real digital avatar, and then apply it to digital fields such as the metaverse.

[0048] 2) This application can effectively realize facial digitization, and based on this, perform facial health monitoring and beauty, find the best cosmetics that match the spectrum under any light, effectively ensure the health of the human face, and provide more effective effects for health care and beauty.

[0049] 3) The present application provides a handheld digitizing device that can effectively digitize physical objects in any scenario, thereby improving the convenience and speed of digitization. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Shown is a flow chart of an object digitization method according to an embodiment of the present application.

[0051] Figure 2 Shown is a schematic structural diagram of a spectral light source described in an embodiment of the present application.

[0052] Figure 3 Shown is a schematic structural diagram of an LED light source described in an embodiment of the present application.

[0053] Figure 4 Shown is a schematic diagram of the wavelength range covered by the spectral light source of each color channel described in an embodiment of the present application.

[0054] Figure 5 Shown is a lighting schematic diagram of a multi-node lighting source described in an embodiment of the present application.

[0055] Figure 6 Shown is a flow chart of an object digitization method according to an embodiment of the present application.

[0056] Figure 7 Shown is a flow chart of an object digitization method according to an embodiment of the present application.

[0057] Figure 8 A schematic diagram of the distribution shape of the multiple node light sources described in the embodiment of the present application is shown.

[0058] Figure 9a to Figure 9b The figure shows a schematic diagram of the difference between a real spectral curve and a measured spectral curve obtained by scanning facial skin using the facial scanning method described in an embodiment of the present application.

[0059] Fig.10 Shown is a schematic structural diagram of an object digitization device according to an embodiment of the present application in one embodiment.

[0060] Fig.11 Shown is a schematic structural diagram of an electronic device of the present application in one embodiment. DETAILED DESCRIPTION

[0061] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0062] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0063] In addition, in this application, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0064] Object digitization refers to converting physical objects in reality into digital form and applying them in various digital applications. At present, object digitization is often achieved by constructing point clouds of object objects, creating high-resolution three-dimensional shapes, and applying three-dimensional digital avatars to games, Web and other applications. Taking the metaverse as an example, it encompasses all aspects of economics, politics, culture, science and technology, and is an advanced version of the digital society. Under the metaverse technology, all things and affairs in the real world exist and proceed based on digital space. Nothing is not virtual, nothing is not real, and the virtual and the real are fully integrated. It can be said that the metaverse enables human beings to expand their living conditions from the real world to the digital world, which is a new form of human survival. However, in the process of digitization, there is often a lack of attention to the spectrum of physical objects, resulting in a certain degree of distortion in the spectrum of digital objects, which is difficult to fully match with the real world, which affects the application of many digital scenes. In addition, in the fields of health care and medical beauty, object digitization methods will also be used to digitize the human face, so as to better analyze whether there are problems with the composition indicators of the skin, or evaluate the effect of medical beauty. However, due to the distortion of the skin's spectrum, the results of medical or cosmetic assessments are affected to a certain extent, making it difficult to effectively ensure the health of the skin.

[0065] In order to at least solve the above-mentioned technical problems, the embodiments of the present application provide an object digitization and face scanning method, apparatus and handheld digitization device, which are used to solve the technical problem of how to truly realize object digitization.

[0066] See also Figure 1 As shown, the object digitization method provided in the embodiment of the present application includes:

[0067] S1. Obtain a spectral image of an object under a multi-node lighting source.

[0068] In order to restore the real reflection spectrum of the object under different spectral illumination to ensure the authenticity of the digital object, the multi-node illumination light source designed in the present application includes multiple node light sources to illuminate the object. Each of the node light sources is provided with a spectrum light source of a single color channel or a spectrum light source of N color channels, N∈[3,15].

[0069] Furthermore, the node light source within the front range of the object is provided with a spectral light source of N color channels, N∈[3, 15].

[0070] It should be noted that the spectral light source of a single color channel should be included in the N color channels. That is, the multi-node lighting light source actually provides illumination of N color channels, wherein some node light sources (not within the front range of the object) only provide a single color in the N color channels.

[0071] In some embodiments, see Figure 2 As shown, the spectrum light source of each color channel includes an LED light source 21 and an LED collimator 22, a micro lens array 23, a projection mirror 24, and a semi-transparent mirror 25 arranged in sequence along the light transmission direction. The light beam emitted by the LED light source 21 passes through the LED collimator 22, the micro lens array 23, the projection mirror 24, and the semi-transparent mirror 25 in sequence, and then vertically incident on the object.

[0072] For further information, see Figure 3 As shown, the LED light source is a circular light source with four LEDs of the same color channel arranged symmetrically to provide the best spatial uniformity. That is, the spectral light source of each color channel is provided correspondingly by four circular light sources with four LEDs of the same color channel arranged symmetrically. Therefore, the spectral light source designed in the present application can ensure that any light point on the illuminated area can receive uniform light: the same illumination in space and the same spectral components. For example, point A and point B are two areas in the illumination area. They have the same irradiance for the spectral light source of any color channel.

[0073] It should be noted that if a spectral light source with N color channels is set on a node, the LEDs of N colors can be integrated into the same LED light source 21, that is, four LEDs of each color are arranged symmetrically, and LEDs of different colors together form a symmetrically arranged circular light source. Figure 3 For example, an LED light source 21 with four colors integrated into one is shown, wherein each of the four colors has four identical LEDs arranged symmetrically. At this time, the LED light source 21 on a node can be controlled to emit four colors of light, and pass through the same LED collimator 22, micro lens array 23, projection mirror 24, and semi-transparent and semi-reflective mirror 25.

[0074] For further information, see Figure 4 As shown, the spectral light source of each color channel covers a specific wavelength range (with a peak wavelength and a spectral width). When providing a multi-node illumination light source, spectral light sources of several color channels can be selected according to different needs of the object, so as to perform spectral restoration of the surface of the object.

[0075] In addition, in some embodiments, a white spectrum light source with different spectral shapes can be selectively used. Unlike the spectrum light sources of other color channels, the white spectrum light source can be extended to the full visible light (400-700 nanometers) or even wider, so that the spectrum restoration of the object can be better achieved in some scenes.

[0076] It should be noted that this application does not impose any restrictions on the specific color channel selection required for each object.

[0077] by Figure 5 For example, the multi-node illumination light source provided in the above embodiment is irradiated on the object. At this time, the spectral light sources of N color channels (spectral light sources of the same color channel on different nodes) are controlled to light up in sequence to obtain the spectral image of the object under N color channels. Figure 5 As shown, in some embodiments, a camera lens can be set behind each node light source to obtain a spectral image at the corresponding node position (obtaining spectral images of multiple nodes of the object under the illumination of spectral light sources of different color channels). In other embodiments, a single camera lens can also be selected to move to obtain spectral images at multiple node positions. That is, for each node (corresponding to different lighting positions of the object), there are multiple spectral images of the object (different shooting positions) obtained under the illumination of spectral light sources of different color channels. Of course, the camera lens can also be placed in front of the object to obtain only the spectral image within the front range. With regard to the acquisition position of the spectral image, this application does not impose any restrictions on this.

[0078] In addition, the camera lens may be a monochrome camera lens or a color camera lens, and this application does not impose any limitation on this.

[0079] S2. Obtaining a response function of the spectral image.

[0080] For a spectral image, each pixel has a complete spectral information. Therefore, if the spectral information of each pixel in the spectral image of the object is restored, the complete spectrum of the corresponding area of ​​the object and the spectral image can be obtained. To this end, for each pixel in the spectral image of N color channels, the following response function is obtained:

[0081] RE[Nx1]=ALED[NxS*RefSpec[Sx1]

[0082] Wherein, RE[Nx1] is each pixel in the spectral image of N color channels, ALED[NxS] is the emission spectrum of the spectral light source of N color channels, N∈[3,15], RefSpec[Sx1] is the reflectance spectrum of the object, S is the wavelength corresponding to the N color channels, S∈[350,800].

[0083] As mentioned above, the spectral light source of each color channel covers a specific wavelength range (with peak wavelength and spectral luminosity). Therefore, in the above formula, for each value of N, there is a corresponding range of values ​​of S.

[0084] S3. Restoring the reflection spectrum of the object based on the response function and the reflection database.

[0085] After obtaining the response function of the spectral image, by comparing the response function with the reflection database, each pixel in the spectral image can be restored based on the reflection database, so as to obtain a true and unbiased reflection spectrum. Therefore, the reflection database is actually a reflection spectrum database of objects of the same type as the object. Taking the object object as a yellow face as an example, the spectral shape of the yellow face has similarity, that is, the spectral variation of the yellow face has a certain regularity. For this reason, it is only necessary to establish a reflection spectrum database of the yellow face, and then use the database to compare with the acquired object to produce an accurate restoration of the target spectrum.

[0086] See also Figure 6 As shown, the implementation method of obtaining the reflection database includes:

[0087] S31. Acquire multiple sample objects of the same type as the target object.

[0088] Specifically, taking the faces of yellow people as an example, the faces of multiple yellow people are used as sample objects.

[0089] S32. Acquire sample reflectance spectra of a plurality of the sample objects based on a spectral light source with N color channels, where N∈[3, 15].

[0090] Specifically, a multi-node illumination light source of the same design is used to illuminate multiple sample objects, and sample reflection spectra corresponding to the multiple sample objects are obtained. That is, when it is necessary to digitize the face of a yellow person, a reflection database of the face of a yellow person needs to be established in advance. At this time, the same multi-node illumination light source design should be used, including that the color channels of the spectral light sources on multiple node light sources should be kept consistent.

[0091] S33: Acquire multiple spectral clusters based on the spectral similarity of the multiple sample reflection spectra.

[0092] S34. Obtain M representative reflection spectra based on the multiple spectral clusters.

[0093] In some embodiments, a total of 3500 sample reflection spectra are obtained. Since the reflection spectra of multiple sample objects are similar, the 3500 spectra can be clustered by the PCA method or the K means method to obtain 6 spectral clustering clusters. Afterwards, 50 sample emission spectra are selected in each spectral clustering cluster, and the 50 sample emission spectra should be evenly distributed in the cluster. Thus, for the 6 spectral clustering clusters, a total of M=300 representative reflection spectra can be obtained, and these representative reflection spectra can represent all sample reflection spectra.

[0094] S35 . Use the multiplication result of the M representative reflection spectra and the emission spectra of the spectrum light sources of N color channels as the reflection data of the reflection database.

[0095] Taking M=300 as an example, by multiplying the 300 representative reflection spectra and the emission spectra of the spectral light source of N color channels, the reflection data in the reflection database can be obtained. Therefore, by clustering and selecting M representative reflection spectra, the amount of data calculation can be greatly reduced, the efficiency of spectrum recovery can be improved, and the M representative reflection spectra can be guaranteed to represent the real spectrum information of the sample object, which is helpful for the accurate recovery of the target spectrum of the object.

[0096] According to the above pixel response function, in fact, the reflection data should be the real spectral information of each pixel of the sample object. Therefore, by comparing each pixel (response function) in the spectral image of N color channels with the reflection data, a weight matrix can be generated, and the reflection spectrum of the object can be restored through the weight matrix.

[0097] Specifically, the weight matrix is ​​obtained based on the weighted least squares method:

[0098]

[0099] Among them, the R i It is the multiplication result of the i-th representative reflection spectrum and the spectrum light source of N color channels, i∈[1,M].

[0100] Specifically, the reflectance spectrum of the object is restored based on the weight matrix, and the restored reflectance spectrum of the object is:

[0101] Spectest=∑a test-p RefSpec[Sx1]

[0102] Among them, a test-p is the pth principal component of the object, and a test-p =T·RE[Nx1], T=X[p×M]·W i [M×M]·R T [M×N]·inv[R N×M W iM×M R T M×N ]=[p×N].

[0103] Since the spectral information of the object can be considered as the linear sum of the principal components of the object, that is, the p principal components of the object actually contain the spectral information of the object. Therefore, each principal component is restored, and finally the fully restored reflectance spectrum of the object can be obtained. Among them, for the number of principal components (the value range of p), the principal component analysis should be performed using the PCA method, that is, for different types of object, the value of p is actually not the same. For the specific number of principal components, it can be determined when obtaining a representative reflectance spectrum, that is, based on the spectral analysis of multiple sample objects.

[0104] S4. Obtain a fine point cloud of the object.

[0105] For details, please refer to Figure 7 As shown, obtaining the fine point cloud of the object includes:

[0106] S41, obtaining a rough point cloud of the object.

[0107] In some embodiments, obtaining a rough point cloud of the object includes:

[0108] 1) Acquire multi-angle images of the object.

[0109] 2) Calculating a basic matrix based on feature matching between the multi-angle images.

[0110] 3) Calculate the camera matrix based on the basic matrix.

[0111] 4) Acquire a rough point cloud of the object based on the camera matrix.

[0112] Specifically, the image of the object is obtained at multiple angles, that is, multi-angle images are obtained, and then the images at multiple angles are processed. First, two initial pictures can be selected, and feature point detection of the two images can be realized by SIFT, SURF or ORB to find the corresponding matching points. After that, the RANSAC method is used to estimate the best basic matrix and the index of the correct point, and the basic matrix of different image pairs is obtained, and then the camera matrix (rotation matrix and translation vector) is calculated by the basic matrix. Finally, the three-dimensional point is calculated by triangulation. Based on this, new pictures are gradually added, feature points are extracted from the new images and matched with the previous images to obtain a new matching relationship, and the camera pose of the new image relative to the previous image is estimated. Usually, pnp (Perspective-n-Point) is used under a known camera matrix, and the three-dimensional points at this angle and the corresponding image point coordinates are used to estimate the shooting information of the angle, and the new matching pairs and the estimated pose (rotation matrix and translation vector) are used for triangulation to generate new three-dimensional points, and the newly generated three-dimensional points are merged with the previous point cloud to construct a larger sparse point cloud, so as to increase the images of all angles and finally obtain the rough point cloud.

[0113] In the above embodiment, it can be implemented by OpenCV.

[0114] In other embodiments, other methods may be selected to obtain a rough point cloud of the object, such as the TOF principle, etc., and this application does not impose any limitation on this.

[0115] It should be noted that when acquiring multi-angle images, it is only necessary to acquire multiple images at different angles. This application does not impose any restrictions on the color and intensity of the light source.

[0116] S42: Acquire the surface normal of the object based on the multi-node lighting light source.

[0117] Specifically, obtaining the surface normal of the object based on the multi-node illumination light source includes: obtaining a first brightness value and a second brightness value of the object, and obtaining the surface normal based on a comparison result of the first brightness value and the second brightness value.

[0118] In some embodiments, the brightness of the object under the illumination of a plurality of node light sources of the same light intensity is used as the first brightness value.

[0119] In some embodiments, the brightness of the object under the illumination of a plurality of node light sources of different light intensities is used as the second brightness value; wherein the light intensity of the node light source is controlled by a light intensity function.

[0120] Specifically, the light intensity function is:

[0121] L i = k·theta(i)

[0122] Among them, L i is the light intensity of the light source at the ith node, k is a constant, and theta(i) is the angle of the ith node relative to the object.

[0123] In the above embodiment, the plurality of node light sources are distributed in a light cage shape, or in a linear shape, and the linear shape includes a linear shape in the X direction and a linear shape in the Y direction. When the plurality of node light sources are distributed in a light cage shape or a linear shape, the light sources can be orderly illuminated to the object from all directions, ensuring that the object can receive uniform light from all angles.

[0124] Figure 8 Displays multiple node light sources distributed in a light cage shape or a linear shape in the X direction. Figure 8 As shown, taking the X direction as an example, all node light sources are set to the same light intensity, and the object at this time ( Figure 8 The brightness of the face in the image is recorded as the first brightness value. Then, the light intensity of all node light sources is adjusted accordingly according to the above light intensity function, so that the light source gradually becomes brighter or darker from left to right (gradient lighting), and the object at this time ( Figure 8 The brightness of the portion (shown as a face) is recorded as a second brightness value, and finally the first brightness value is compared with the second brightness value to obtain the surface normal.

[0125] In some other embodiments, the brightness of the object under the illumination of a plurality of node light sources of different light intensities is used as the second brightness value; wherein the light intensity of the node light source is controlled by a light intensity function and a distance function.

[0126] Specifically, the distance function is:

[0127] L i =d 2

[0128] Wherein, d is the distance between the i-th node light source and the object.

[0129] In the above embodiment, the plurality of node light sources are distributed in a plane shape. The light sources can also shine on the object in an orderly manner from all directions of the plane, ensuring that the object can receive uniform light from all angles.

[0130] Specifically, when multiple node light sources are distributed in a plane shape, all node light sources are still set to the same light intensity, and the object at this time ( Figure 8The brightness of the part (shown as a face in the figure) is recorded as the first brightness value. Later, when designing gradient lighting, the light intensity of all node light sources needs to be adjusted according to the direction and distance. That is, the light intensity of all node light sources is adjusted accordingly according to the above-mentioned light intensity function and distance function, thereby achieving that the light source gradually becomes brighter or darker from left to right. At this time, the light intensity of all node light sources is actually consistent with when distributed in the shape of a light cage. Finally, the first brightness value is compared with the second brightness value to obtain the surface normal.

[0131] S43: Acquire a fine point cloud based on the rough point cloud and the surface normal.

[0132] Specifically, obtaining a fine point cloud based on the rough point cloud and the surface normal includes:

[0133] S431. Obtain the normal of the rough point cloud.

[0134] S432: Based on the surface normal and the normal of the rough point cloud, correct the rough point cloud to obtain a corrected point cloud.

[0135] S433, iteratively performing a correction operation until the surface normal is compared with the normal of the correction point cloud and a preset condition is satisfied; wherein the correction operation is to correct the correction point cloud based on the surface normal and the normal of the correction point cloud to obtain a new correction point cloud.

[0136] In the above embodiment, after the rough point cloud of the object is obtained in step S41, it needs to be further optimized to obtain a fine point cloud, and then create a real digital object. Therefore, the rough point cloud obtained in step S41 is used as a basis to obtain the normal of the rough point cloud, and then the high-frequency part of the surface normal obtained in step S42 is added to the normal of the rough point cloud, and a more detailed point cloud, that is, a corrected point cloud, is calculated by combining with the rough point cloud. Afterwards, the normal of the corrected point cloud is obtained, and the surface normal and the normal of the corrected point cloud are compared to determine whether the preset threshold condition is met. If not, continue to follow the same process as above, add the high-frequency part of the surface normal to the normal of the corrected point cloud, and calculate a more detailed point cloud, that is, a new corrected point cloud, by combining with the corrected point cloud. This iteration is performed until the final fine point cloud is obtained.

[0137] In some other embodiments, the fine point cloud can be obtained by obtaining the normal of the rough point cloud and optimizing the normal based on the bi Laplace equation, wherein the surface normal is used as the Neumann boundary condition.

[0138] In the above embodiment, the PCA method can be used to obtain the normal of the rough point cloud or the modified point cloud. First, the covariance matrix of the rough point cloud or the modified point cloud is calculated, and the eigenvectors and eigenvalues ​​of the covariance matrix are calculated, and finally the eigenvector corresponding to the minimum eigenvalue is selected as the estimated normal direction.

[0139] In other embodiments, other methods may be selected to obtain the normals of the rough point cloud or the corrected point cloud, such as surface reconstruction, etc., and the present application does not impose any limitation on this.

[0140] S5. Digitize the object based on the fine point cloud and the restored reflectance spectrum of the object.

[0141] Specifically, after obtaining the fine point cloud and the restored reflectance spectrum of the object, any object can be digitized based on the two and reproduced in a 3D or 2D graphics viewing portal (such as a computer, display or XR headset).

[0142] Since the 3D information and spectral information of the object are combined, the digital object digitized by the object digitization method provided in the embodiment of the present application has the highest accuracy, can be widely used in digital fields such as the metaverse, VR / AR, etc., and can also be used to detect human skin and determine the health of the skin.

[0143] To this end, an embodiment of the present application also provides a facial scanning method, which can digitally scan the face through the object digitization method provided in the above embodiment to obtain the reflection spectrum and three-dimensional shape of the face.

[0144] In some embodiments, after obtaining the reflectance spectrum and three-dimensional shape of the face, at least one indicator data of the facial melanin concentration index, epidermal surface thickness index, blood volume index and oxygen content index can be obtained based on the reflectance spectrum of the face to evaluate the facial beauty effect.

[0145] Specifically, after a facial beauty treatment, the beauty effect requires accurate measurement of facial shape details. This can be obtained through accurate changes in three-dimensional shape information. In addition, melanin concentration index, epidermal surface thickness index, blood volume index and oxygen content index can also be obtained based on the reflection spectrum. Therefore, the beauty effect can be evaluated through the changes in three-dimensional information and spectral information before and after beauty treatment.

[0146] Figure 9a and Figure 9b The diagram shows a method of scanning facial skin using the facial scanning method provided in an embodiment of the present application to obtain a difference between a real spectral curve and a measured spectral curve. Figure 9aThere is little difference between the real spectrum curve and the measured spectrum curve. Figure 9a The melanin concentration index obtained based on the true reflectance spectrum is 38.83%, the blood volume index is 4.19%, the epidermal surface thickness index is 42.51um, and the oxygen content index is 10.25%. Figure 9b There is a big difference between the real spectrum curve and the measured spectrum curve. Figure 9b The melanin concentration index obtained based on the real reflection spectrum is 6.33%, the blood volume index is 7.75%, the epidermal surface thickness index is 46.35um, and the oxygen content index is 85.66%. It can also be seen that for white people and the elderly, the facial scanning method of this application can better obtain the real spectrum curve of the skin, and obtain the real values ​​of the melanin concentration index, epidermal surface thickness index, blood volume index and oxygen content index, which can better evaluate the facial beauty effect.

[0147] In some embodiments, after obtaining the reflectance spectrum and three-dimensional shape of the face, the facial health can be evaluated based on this.

[0148] Taking dark circles and complexion as an example, by scanning the face to obtain the skin's reflectance spectrum information, the cause of dark circles can be determined. And after obtaining three-dimensional information (precise microstructure), facial health can be evaluated in a timely manner based on daily changes to avoid the occurrence of diseases such as skin cancer.

[0149] As described above, for white people and the elderly, the facial scanning method of the present application can better obtain the true spectral curve of the skin, thereby timely monitoring facial health and avoiding the occurrence of diseases such as skin cancer.

[0150] In some embodiments, after obtaining the reflectance spectrum of the face, the reflectance spectrum of the cosmetics may also be obtained, and based on this, the matching effect of the cosmetics and the face may be evaluated.

[0151] Specifically, after scanning the face to obtain the reflection spectrum of the face, the cosmetics can also be scanned based on the object digitization method provided in the above embodiment to obtain the reflection spectrum of the cosmetics. In this way, it can be achieved that under any light source, the higher the similarity between the reflection spectrum of the face and the reflection spectrum of the cosmetics, the higher the matching degree of the cosmetics and the face.

[0152] The protection scope of the object digitization method and / or facial scanning method described in the embodiments of the present application is not limited to the execution order of the steps listed in the embodiments. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present application are included in the protection scope of the present application.

[0153] The embodiments of the present application also provide an object digitization device and / or a facial scanning device. The object digitization device can implement the object digitization method described in the present application, and the facial scanning device can implement the facial scanning method described in the present application. However, the implementation device of the method described in the present application includes but is not limited to the device structure listed in the present embodiment. All structural deformations and replacements of the prior art made according to the principles of the present application are included in the protection scope of the present application.

[0154] See also Fig.10 As shown, the embodiment of the present application further provides an object digitization device, the device comprising:

[0155] The image acquisition module 41 is used to acquire a spectral image of an object under a multi-node illumination light source.

[0156] The response module 42 is used to obtain the response function of the spectral image.

[0157] The spectrum recovery module 43 is used to recover the reflection spectrum of the object based on the response function and the reflection database.

[0158] The geometry recovery module 44 is used to obtain a fine point cloud of the object.

[0159] The digitization module 45 is configured to digitize the object based on the fine point cloud and the restored reflectance spectrum of the object.

[0160] Furthermore, the image acquisition module 41 is also used to acquire multi-angle images of the object.

[0161] Please continue reading Fig.10 As shown, the object digitizing device of the present application further includes a lighting module 46. The lighting module 46 is used to provide the multi-node lighting light source and to control the light intensity of the multi-node lighting light source.

[0162] Specifically, the multi-node lighting light source provided by the lighting module 46 includes a plurality of node light sources, each of which is provided with a spectral light source of a single color channel or a spectral light source of N color channels, where N∈[3, 15].

[0163] Furthermore, the node light source within the front range of the object is provided with a spectral light source of N color channels, N∈[3, 15].

[0164] The spectral light source includes an LED light source and an LED collimator, a microlens array, a projection mirror, and a semi-transparent and semi-reflective mirror arranged in sequence along the light transmission direction. The light beam emitted by the LED light source passes through the LED collimator, the microlens array, the projection mirror, and the semi-transparent and semi-reflective mirror in sequence, and then vertically incident on the object.

[0165] Furthermore, the LED light source is a circular light source in which four LEDs of the same color channel are symmetrically arranged to provide the best spatial uniformity.

[0166] Furthermore, the multiple node light sources can be distributed in a light cage shape, a linear shape, and a plane shape. When distributed in a linear shape, it includes an X-direction linear shape and a Y-direction linear shape.

[0167] When multiple node light sources are distributed in a light cage shape or a linear shape, the light intensity of each node light source is controlled by a light intensity function to adjust the light intensity of the node light source according to different angles (with the object as a reference).

[0168] When multiple node light sources are distributed in a planar shape, the light intensity of each node light source is controlled by a light intensity function and a distance function to adjust the light intensity of the node light source according to different angles and distances (with the object as a reference).

[0169] Wherein, the light intensity function is:

[0170] L i = k·theta(i)

[0171] Among them, L i is the light intensity of the light source at the ith node, k is a constant, and theta(i) is the angle of the ith node relative to the object.

[0172] Wherein, the distance function is:

[0173] L i =d 2

[0174] Wherein, d is the distance between the i-th node light source and the object.

[0175] Specifically, when an object needs to be digitized, the object is placed in the lighting area and illuminated by the multi-node lighting source provided by the lighting module 46. At this time, the lighting module 46 controls the spectral light sources of N (N∈[3, 15]) color channels to light up in sequence, and the image acquisition module 41 acquires the spectral image of the object under N color channels. Afterwards, the response module 42 acquires the response function of the spectral image, and the spectral recovery module 43 restores the reflective spectrum of the object based on the response function and the reflection database. At the same time, the lighting module 46 can control the light intensity of multiple node light sources to provide the gradient lighting required for geometric recovery, and the image acquisition module 41 will also acquire multi-angle images of the object, so that the geometric recovery module 44 can finally acquire the fine point cloud of the object. Finally, after acquiring the three-dimensional information and spectral information of the object, the digitization module 45 can achieve digitization.

[0176] It should be noted that the structures and principles of the image acquisition module 41, the response module 42, the spectrum recovery module 43, the geometry recovery module 44, the digitization module 45 and the illumination module 46 can refer to the contents of the above-mentioned method embodiment and will not be repeated here.

[0177] The embodiment of the present application also provides a face scanning device, which includes the object digitizing device provided in the above embodiment. The specific structure and principle of the face scanning device can be referred to the above content, and will not be repeated here.

[0178] An embodiment of the present application further provides a handheld digitizing device, which includes the object digitizing apparatus provided in the above embodiment.

[0179] The image acquisition module is also used to acquire the spatial position and direction of the multi-node illumination light source relative to the object and the distance relative to the object.

[0180] The lighting module controls the light intensity of the multi-node lighting light source according to the spatial position and direction of the multi-node lighting light source relative to the object.

[0181] Specifically, taking a mobile phone as an example, the camera of the mobile phone is the image acquisition module. The lighting module can be installed in the mobile phone accessories, or it can be a part of the mobile phone as a flashlight. This application does not impose any restrictions on this. Taking the use of a mobile phone to digitize a face as an example, the principle is:

[0182] Place the phone in front of the face and illuminate it with the multi-node lighting source provided by the flashlight. At this time, the flashlight controls the spectral light sources of N (N∈[3,15]) color channels to light up in sequence, and the camera captures the spectral image of the face under N color channels. Afterwards, the response function of the spectral image is obtained, and the reflectance spectrum of the object is restored based on the response function and the reflection database. At the same time, the phone is moved to create a virtual light source distribution shape. Among them, the IMU / camera of the phone can be used to determine the position and direction of the phone (flashlight) in space, as well as the distance relative to the face. For example, when the phone moves around the face to form a light cage shape or a linear shape, the brightness of the flashlight can be controlled according to the position and direction (light intensity function) to provide the gradient lighting required for geometric restoration. When the phone moves around the face to form a planar shape, the brightness of the flashlight can be controlled according to the position and direction and distance (light intensity function and distance function) to provide the gradient lighting required for geometric restoration. At the same time, the camera will also collect images from multiple angles to obtain a fine point cloud of the object based on gradient lighting and multi-angle images. Finally, after obtaining the three-dimensional information and spectral information of the object, digitization can be achieved.

[0183] In other embodiments, a drone / robotic arm based system may be used to scan along a designed and predetermined path.

[0184] It can be seen that the object digitization and facial scanning method, device and handheld digitization device provided by this application can quickly restore the real reflection spectrum of the object and the high-resolution three-dimensional shape of the object, create a real digital avatar, and then apply it to digital fields such as the metaverse. In addition, this application can effectively realize facial digitization, and based on this, perform facial health monitoring and beauty, find the best cosmetics that match the spectrum under any light, and effectively ensure the health of the human face. Finally, this application provides a handheld digitization device that can effectively digitize physical objects in any scenario, improving the convenience and speed of digitization.

[0185] In the several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules or units, which can be electrical, mechanical or other forms.

[0186] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0187] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0188] The present application embodiment also provides a computer-readable storage medium. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by a program to instruct the processor, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state hard disk (SSD)), etc.

[0189] The embodiment of the present application further provides an electronic device, wherein the electronic device includes a processor and a memory.

[0190] The memory is used to store computer programs.

[0191] The memory includes: ROM, RAM, disk, USB flash drive, memory card or CD and other media that can store program codes.

[0192] The processor is connected to the memory and is used to execute the computer program stored in the memory so that the electronic device executes the above-mentioned object digitization method and / or facial scanning method.

[0193] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0194] like Fig.11 As shown, the electronic device of the present application is in the form of a general computing device. The components of the electronic device may include but are not limited to: one or more processors or processing units 51, a memory 52, and a bus 53 connecting different system components (including the memory 52 and the processing unit 51).

[0195] Bus 53 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0196] Electronic devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0197] The memory 52 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 521 and / or cache memory 522. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 523 may be used to read and write non-removable, non-volatile magnetic media ( Fig.11 not shown, usually called a "hard drive"). Although Fig.11Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical medium) may be provided. In these cases, each drive may be connected to the bus 53 via one or more data medium interfaces. The memory 52 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.

[0198] A program / utility 524 having a set (at least one) of program modules 5241 may be stored, for example, in the memory 52, such program modules 5241 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 5241 generally perform the functions and / or methods of the embodiments described herein.

[0199] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, displays, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via input / output (I / O) interface 54. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 55. Fig.11 As shown, the network adapter 55 communicates with other modules of the electronic device via the bus 53. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0200] The present application embodiment may also provide a computer program product, the computer program product including one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer or data center.

[0201] When the computer program product is executed by a computer, the computer executes the method described in the above method embodiment. The computer program product may be a software installation package, and when the above method is required, the computer program product may be downloaded and executed on a computer.

[0202] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0203] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for digitizing an object, characterized in that: The method comprises: Acquire a spectral image of the object under a multi-node lighting source; Acquiring a response function of the spectral image; Restoring the reflection spectrum of the object based on the response function and the reflection database; Acquire a fine point cloud of the object; The object is digitized based on the fine point cloud and the restored reflectance spectrum of the object.

2. The object digitization method according to claim 1, characterized in that: The multi-node lighting source includes a plurality of node light sources, each of which is provided with a spectrum light source of a single color channel or a spectrum light source of N color channels. N∈[3，15]。 3. The object digitization method according to claim 2, characterized in that: The node light source within the front range of the object is provided with a spectral light source of N color channels, N∈[3, 15].

4. The object digitization method according to claim 2, characterized in that: The spectral light source includes an LED light source and an LED collimator, a microlens array, a projection mirror, and a semi-transparent and semi-reflective mirror arranged in sequence along the light transmission direction; the light beam emitted by the LED light source passes through the LED collimator, the microlens array, the projection mirror, and the semi-transparent and semi-reflective mirror in sequence, and then is vertically incident on the object.

5. The object digitization method according to claim 4, characterized in that: The LED light source is a circular light source in which four LEDs of the same color channel are symmetrically arranged.

6. The object digitization method according to claim 3, characterized in that: Acquiring a spectral image of an object under an illumination source includes: The spectral light sources of the N color channels are controlled to light up in sequence, and spectral images of multiple nodes of the object under the N color channels are acquired.

7. The object digitization method according to claim 6, characterized in that: The response function of the spectral image is: RE[Nx1]=ALED[NxS*RefSpec[Sx1] Wherein, RE[Nx1] is each pixel in the spectral image of N color channels, ALED[NxS] is the emission spectrum of the spectral light source of N color channels, N∈[3, 15], RefSpec[Sx1] is the reflectance spectrum of the object, S is the wavelength corresponding to the N color channels, S∈[350, 800].

8. The object digitization method according to claim 7, characterized in that: The implementation methods for obtaining the reflection database include: Acquire multiple sample objects of the same type as the subject object; Acquire sample reflectance spectra of a plurality of sample objects based on a spectral light source of N color channels, N∈[3,15]; Acquire a plurality of spectral clusters based on spectral similarity of a plurality of the sample reflectance spectra; Acquire M representative reflection spectra based on the plurality of spectral clusters; The multiplication result of the M representative reflection spectra and the emission spectra of the spectrum light sources of N color channels is used as the reflection data of the reflection database.

9. The object digitization method according to claim 8, characterized in that: Restoring the reflection spectrum of the object based on the response function and the reflection database includes: Compare each pixel in the spectral image of N color channels with the reflectance data to generate a weight matrix; The reflectance spectrum of the object is restored based on the weight matrix.

10. The object digitization method according to claim 9, characterized in that: The weight matrix is ​​obtained based on the weighted least squares method: Among them, the R i It is the multiplication result of the i-th representative reflection spectrum and the spectrum light source of N color channels, i∈[1,M].

11. The object digitization method according to claim 10, characterized in that: Restoring the reflection spectrum of the object based on the weight matrix includes: The restored reflectance spectrum of the object is: spectest=∑a test-p ·Refspec[sx1] Among them, a test-p is the pth principal component of the object, and a test-p =T·RE[Nx1], T=X[p×M]·W i [M×M]·R T [M×N]·inv[R N×M W iM×M R T M×N ]=[p×N]。 12. The object digitization method according to claim 2, characterized in that: Acquiring a fine point cloud of the object includes: Acquire a rough point cloud of the object; Acquiring a surface normal of the object based on the multi-node illumination light source; A fine point cloud is obtained based on the coarse point cloud and the surface normal.

13. The object digitization method according to claim 12, characterized in that: Acquiring a rough point cloud of the object includes: Acquire multi-angle images of the object; Calculating a basic matrix based on feature matching between the multi-angle images; Calculate a camera matrix based on the basic matrix; A rough point cloud of the object is acquired based on the camera matrix.

14. The object digitization method according to claim 12, characterized in that: Acquiring the surface normal of the object based on the multi-node illumination light source includes: Acquire a first brightness value and a second brightness value of the object; The surface normal is obtained based on a comparison result of the first brightness value and the second brightness value.

15. The object digitization method according to claim 14, characterized in that: The brightness of the object under the illumination of multiple node light sources of the same light intensity is used as the first brightness value.

16. The object digitization method according to claim 14, characterized in that: The brightness of the object under the illumination of the node light sources of multiple different light intensities is used as a second brightness value; wherein the light intensity of the node light source is controlled by a light intensity function.

17. The object digitization method according to claim 16, characterized in that: The plurality of node light sources are distributed in a light cage shape.

18. The object digitization method according to claim 16, characterized in that: The plurality of node light sources are distributed in a linear shape, and the linear shape includes an X-direction linear shape and a Y-direction linear shape.

19. The object digitization method according to claim 14, characterized in that: The brightness of the object under the illumination of the node light sources of multiple different light intensities is used as a second brightness value; wherein the light intensity of the node light source is controlled by a light intensity function and a distance function.

20. The object digitization method according to claim 19, characterized in that: The plurality of node light sources are distributed in a plane shape.

21. The object digitization method according to claim 16 or 19, characterized in that: The light intensity function is: L i =k·theta(i) Among them, L i is the light intensity of the light source at the ith node, k is a constant, and theta(i) is the angle of the ith node relative to the object.

22. The object digitization method according to claim 19, characterized in that: The distance function is: L=d 2 Wherein, d is the distance between the i-th node light source and the object.

23. The object digitization method according to claim 12, characterized in that: Acquiring a fine point cloud based on the rough point cloud and the surface normal includes: Obtaining a normal of the rough point cloud; Based on the surface normal and the normal of the rough point cloud, the rough point cloud is corrected to obtain a corrected point cloud; The correction operation is iteratively performed until the surface normal is compared with the normal of the correction point cloud and satisfies a preset condition; wherein the correction operation is to correct the correction point cloud based on the surface normal and the normal of the correction point cloud to obtain a new correction point cloud.

24. The object digitization method according to claim 12, characterized in that: Acquiring a fine point cloud based on the rough point cloud and the surface normal includes: Obtaining a normal of the rough point cloud; The normal is optimized based on the bi Laplace equation; wherein the surface normal is used as a Neumann boundary condition.

25. An object digitizing device, characterized in that: The device comprises: An image acquisition module, used to acquire a spectral image of an object under a multi-node lighting source; A response module, used for obtaining a response function of the spectral image; A spectrum recovery module, used for recovering the reflection spectrum of the object based on the response function and the reflection database; A geometry recovery module, used for obtaining a fine point cloud of the object; A digitization module is used to digitize the object based on the fine point cloud and the restored reflectance spectrum of the object.

26. The object digitizing device according to claim 25, characterized in that: The device comprises a lighting module, the lighting module is used to provide the multi-node lighting light source; and The lighting module is used to control the light intensity of the multi-node lighting light sources.

27. The object digitizing device according to claim 25, characterized in that: The image acquisition module is also used to acquire multi-angle images of the object.

28. A facial scanning method, characterized in that: The method digitally scans the face by the object digitization method according to any one of claims 1 to 24 to obtain the reflectance spectrum and three-dimensional shape of the face; as well as At least one index data of the facial melanin concentration index, epidermal surface thickness index, blood volume index and oxygen content index is obtained based on the facial reflectance spectrum to evaluate the facial beauty effect.

29. The facial scanning method according to claim 28, characterized in that: The method also includes: evaluating facial health based on the reflectance spectrum and three-dimensional shape of the face.

30. The facial scanning method according to claim 28, characterized in that: The method further includes: evaluating the matching effect of the cosmetics and the face based on the reflectance spectrum of the face and the reflectance spectrum of the cosmetics.

31. A facial scanning device, characterized in that: The device comprises an object digitizing device as claimed in any one of claims 25-27.

32. A handheld digitizing device, characterized in that: The device comprises an object digitizing apparatus as claimed in any one of claims 25 to 27; wherein, The image acquisition module is also used to acquire the spatial position and direction of the multi-node lighting source relative to the object.

33. The handheld digitizing device according to claim 32, characterized in that: The lighting module controls the light intensity of the multi-node lighting light sources according to the spatial position and direction of the multi-node lighting light sources relative to the object.

Images