Polarized light scanning method and system based on LED virtual shooting digital assets

Abstract

The invention relates to the technical field of object scanning, in particular to a polarized light scanning method and system based on LED virtual shooting digital assets, and the method comprises the steps: obtaining each alternative feature point in a non-highlight polarization image in different cross polarization directions corresponding to different shooting angles, and the SIFT vector of the alternative feature point; according to the local area of each alternative feature point, analyzing a neighborhood gray scale difference condition, a diagonal feature obvious degree and a polarization characteristic similar condition to determine a texture feature, and further selecting all target feature points; and according to the gray value, the texture feature and the SIFT vector of each target feature point, target feature point matching between different polarization images is carried out to obtain a fusion image, and then the LED virtual shooting digital assets of the object are constructed. According to the method, feature point selection and feature point matching in the process of scanning the object by the polarized light are improved, so that the construction accuracy of the LED virtual shooting digital assets of the object is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of object scanning, and in particular to a polarized light scanning method and system based on LED virtual shooting of digital assets, which uses polarized light to eliminate highlights and obtain a non-highlight texture map by installing polarizers on a light source and a camera lens. Background Art

[0002] Traditional polarized light scanning technology usually installs polarizers on the light source (flash) and camera lens, and uses the characteristics of the polarizer to adjust the polarization angle so that only light with a specific polarization angle enters the camera, thereby eliminating the high light reflection on the surface of the object and obtaining a texture map without highlights. Specifically, the texture information of the object is revealed by analyzing the changes in the polarization state of reflection and transmission. In the creation and application of LED (Lighting Emitting Diode) virtual digital assets, it can enhance the realism and three-dimensionality of the image, avoid the adverse effects caused by highlights during the scanning process, and provide a more realistic visual effect, that is, improve the construction quality of digital assets. Since the polarization-related parameters of the shooting equipment need to be adjusted in the process of completing the construction of virtual digital assets by polarized light scanning of objects, the same shooting direction will present polarized images with different cross-polarization directions. By fusing different polarized images, the acquisition of LED virtual digital assets can be achieved.

[0003] In the process of achieving image fusion in existing polarized light scanning, the SIFT algorithm is often used to extract and match image feature points. Due to the changes in polarization-related parameters during the shooting process, the features of pixel points at the same position of the scanned object in images with different polarizations will be significantly different, making the SIFT vector of the feature point insufficient to clearly describe the key point information in the image. The SIFT feature vectors at the same position will have large differences in different polarization images, and there is a possibility of feature point matching errors, which in turn affects the accuracy of the digital asset construction results of the photographed object. Summary of the invention

[0004] In order to solve the technical problem that the digital assets constructed by the above-mentioned existing polarized light scanning method have deviations, the purpose of the present invention is to provide an improved polarized light scanning method. Specifically, polarizers are installed on the flash light and the camera lens, and the angle of the polarizers is adjusted to eliminate the high light reflection on the surface of the object. On the basis of obtaining a texture map without highlights, a polarized light scanning method and system based on LED virtual shooting of digital assets are proposed to solve the problem of low accuracy of feature point matching during image fusion. The technical solutions adopted are as follows:

[0005] An embodiment of the present invention provides a polarized light scanning method and system based on LED virtual shooting of digital assets, the method comprising the following steps:

[0006] In the process of polarized light scanning an object, polarizers are first installed on both the light source and the camera lens, and then the angle of the polarizer is adjusted to obtain polarized images without highlights in different cross-polarization directions corresponding to different shooting angles, and then each candidate feature point in the polarized image and its SIFT vector are obtained;

[0007] According to the local area of ​​each candidate feature point, the neighborhood grayscale difference, the degree of diagonal feature prominence and the similarity of polarization characteristics are analyzed to determine the texture feature of each candidate feature point, and then all target feature points are selected based on the texture feature;

[0008] According to the gray value, texture feature and SIFT vector of each target feature point in the polarization image of each cross-polarization direction, the target feature points between different polarization images are matched to obtain the fused images corresponding to all cross-polarization directions;

[0009] Construct an LED virtual shot digital asset of the object by fusing images from each shooting angle.

[0010] Furthermore, the method of analyzing the neighborhood grayscale difference, the degree of diagonal feature prominence, and the similarity of polarization characteristics according to the local area of ​​each candidate feature point to determine the texture feature of each candidate feature point includes:

[0011] For any candidate feature point, a window area is constructed with the candidate feature point as the center, and a first texture noticeable factor of the candidate feature point is determined according to the gray value of each pixel point in the window area of ​​the candidate feature point;

[0012] According to the gray level co-occurrence matrix at different angles in the window area of ​​the candidate feature point, the diagonal feature vector of the candidate feature point is obtained; according to the element discreteness and element size of the diagonal feature vector, the second texture obvious factor of the candidate feature point is determined;

[0013] The texture saliency index of the candidate feature point is determined by combining the first texture saliency factor and the second texture saliency factor of the candidate feature point; and the texture feature of the candidate feature point is constructed based on the texture saliency index and the diagonal feature vector of the candidate feature point.

[0014] Furthermore, the step of determining the first texture noticeability factor of the candidate feature point according to the grayscale value of each pixel in the window area of ​​the candidate feature point includes:

[0015] According to the grayscale value, all the pixels in the window area of ​​the candidate feature points are divided into two categories, and the grayscale average value of each category is calculated;

[0016] The absolute value of the difference between the grayscale average values ​​of the two categories is used as the first texture obvious factor of the candidate feature point.

[0017] Furthermore, the diagonal feature vectors of the candidate feature points are obtained according to the gray level co-occurrence matrices at different angles in the window area of ​​the candidate feature points, including:

[0018] Set several angles, and obtain the grayscale co-occurrence matrix of each angle based on the grayscale value of each pixel in the window area of ​​the candidate feature points;

[0019] Determine the diagonal pixel points in the gray level co-occurrence matrix at each angle by using a diagonal function, and then calculate the binary norm of the gray value of the diagonal pixel points in the gray level co-occurrence matrix;

[0020] The bi-norm is normalized to obtain a normalized value, and then a vector composed of the normalized values ​​obtained at each angle is used as a diagonal feature vector.

[0021] Furthermore, matching target feature points between different polarization images is performed according to the grayscale value, texture feature and SIFT vector of each target feature point in the polarization image of each cross-polarization direction to obtain fused images corresponding to all cross-polarization directions, including:

[0022] For any target feature point, all pixels in the window area of ​​the target feature point are clustered according to the grayscale value according to the number of cluster categories, and the image matrix of each cluster window corresponding to the target feature point is obtained;

[0023] Determine the texture feature matching degree between two target feature points according to each clustering window image matrix, texture feature and SIFT vector corresponding to any target feature point in the polarization images of any two cross-polarization directions;

[0024] Target feature points are matched according to the texture feature matching degree to obtain fused images corresponding to all cross-polarization directions.

[0025] Furthermore, clustering all pixel points in the window area of ​​the target feature point according to the grayscale value according to the number of cluster categories to obtain each cluster window image matrix corresponding to the target feature point includes:

[0026] For any number of clustering categories, all pixels in the window area of ​​the target feature point are clustered according to the grayscale value to obtain the initial clustering window area; according to the preset marking rules, the pixels belonging to the same cluster cluster in the initial clustering window area are marked with the same category number; according to the category number of each pixel in the initial clustering window area, the clustering window image matrix of the number of clustering categories is obtained.

[0027] Furthermore, the method of determining the degree of texture feature matching between two target feature points according to each clustering window image matrix, texture feature and SIFT vector corresponding to any target feature point in the polarization images of any two cross-polarization directions includes:

[0028] For any two polarization images with cross polarization directions, one of them is used as a template image and the other is used as an image to be matched. One target feature point is selected from the image to be matched and the template image to form a target feature point pair.

[0029] Determine a first texture feature similarity factor of the target feature point pair according to the number of each cluster category of the target feature point pair and the cluster window image matrix of each cluster category number;

[0030] According to the similarity between the diagonal feature vectors in the texture features of the target feature point pair and the similarity between the SIFT vectors, a second texture feature similarity factor of the target feature point pair is obtained;

[0031] The first texture feature similarity factor and the second texture feature similarity factor of the target feature point pair are combined to determine the texture feature matching degree of the target feature point pair.

[0032] Furthermore, the first texture feature similarity factor of the target feature point pair is determined according to the number of each clustering category of the target feature point pair and the clustering window image matrix of each clustering category number, including

[0033] The calculation formula of the first texture feature similarity factor of the target feature point pair is:

[0034] Where L1(X,Y) represents the first texture feature similarity factor of the target feature point pair, X represents one of the target feature points in the target feature point pair, Y represents the other target feature point in the target feature point pair, K represents the number of cluster categories, K′ represents the maximum number of cluster categories, norm represents the linear normalization function, and f K (X) represents the clustering window image matrix when the number of clustering categories of the target feature point X is K, f K (Y) represents the clustering window image matrix when the number of clustering categories of the target feature point Y is K, and ‖‖ represents the binary norm.

[0035] Furthermore, matching target feature points according to the texture feature matching degree to obtain fused images corresponding to all cross-polarization directions includes:

[0036] Determine the texture feature matching degree between any target feature point in the image to be matched and each target feature point in the template image, and use the target feature point corresponding to the maximum texture feature matching degree in the template image as the matching feature point of the target feature point in the image to be matched;

[0037] According to each target feature point in the image to be matched and its matching feature point, the image to be matched and the template image are fused to obtain a first fused image;

[0038] The first fused image is used as a template image, and image fusion processing based on feature point matching is performed according to each target feature point in the template image and any other polarization image in the cross-polarization direction to obtain a second fused image;

[0039] The second fused image is used as a template image again, and image fusion processing based on feature point matching is performed according to each target feature point in the template image and any other polarization image in the cross-polarization direction to obtain a third fused image;

[0040] The iteration is continued until a fused image corresponding to all cross polarization directions is obtained; wherein the polarization images of other cross polarization directions refer to polarization images that are not involved in image fusion.

[0041] An embodiment of the present invention also provides a polarized light scanning system for digital assets based on LED virtual photography, including a processor and a memory, wherein the processor is used to process instructions stored in the memory to implement a polarized light scanning method for digital assets based on LED virtual photography.

[0042] The present invention has the following beneficial effects:

[0043] The present invention provides a polarized light scanning method and system for digital assets based on LED virtual photography. The basic data set is first acquired. During the process of polarized light scanning the object, the polarizer needs to be rotated to acquire polarized images without highlights in different cross-polarization directions corresponding to different shooting angles, and then each candidate feature point and its SIFT vector in the polarized image are acquired. The acquired candidate feature points and their SIFT vectors provide data support for subsequent image data analysis, and are also convenient for combining the polarized light scanning method for eliminating highlights by installing a polarizer with an improved feature point selection and matching algorithm. This method not only eliminates the influence of highlight reflection and improves the quality of texture information of the acquired polarized image, but is also beneficial for the subsequent preliminary enhancement of the accuracy of feature point matching based on high-quality texture information. This combination method can effectively solve the problem of low feature point matching accuracy caused by changes in highlights and polarization angles in the prior art, and greatly improves the construction accuracy of digital assets for LED virtual photography. When rotating the polarizer, the changes in different cross-polarization directions or polarization parameters will affect the presentation of the polarization image. Specifically, the pixels at the same position of the scanned object have significant feature differences in the polarization images of different cross-polarization directions, which makes the SIFT vector of the feature point at the same position insufficient to clearly describe the key point information in the image. Therefore, it is necessary to analyze the neighborhood grayscale differences, the degree of diagonal feature significance, and the similarity of polarization characteristics based on the local area of ​​each candidate feature point to determine the texture features, and then select all the target feature points based on the texture features. Determining the target feature points can effectively eliminate the adverse effects brought by the significant SIFT vector features of the pixels, that is, eliminate the interference of feature points with unclear texture details. The existing feature point matching ignores the texture differences of the scanned object images at different polarization angles when performing feature point matching fusion based on the target feature points, that is, different polarized lights have different reflections and projections on the same position, which makes the texture features have certain differences, which are manifested as the grayscale changes of the pixels in the polarization image. Therefore, it is necessary to correct the matching conditions of the feature points. Specifically, the influence of multiple factors is taken into account in the matching analysis, that is, the target feature points between different polarization images are matched according to the grayscale value, texture features and SIFT vector of each target feature point in the polarization image of each cross-polarization direction, and the fused images corresponding to all cross-polarization directions are obtained; the fused image determined by improving the feature point matching process has higher image quality and stronger authenticity, which reduces the adverse effects of changes in cross-polarization directions or polarization parameters on the generated fused image to a certain extent, and further improves the accuracy of constructing the object's LED virtual shooting digital assets. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 This is a flow chart of a polarized light scanning method for digital assets based on LED virtual photography according to an embodiment of the present invention;

[0046] Figure 2 This is a flowchart of the implementation of step S2 in one embodiment of the present invention;

[0047] Figure 3 This is a flowchart for implementing step S3 in one embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the technical solutions proposed by the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0049] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0050] The application scenarios targeted by the present invention may be:

[0051] In the process of constructing LED virtual shooting digital assets by scanning objects with polarized light, the fusion result of polarization images in different cross-polarization directions in the same shooting direction achieved by the traditional SIFT algorithm has low accuracy, which leads to deviations in the three-dimensional model constructed based on the fused image in each shooting direction. That is, the obtained digital assets deviate from the actual situation and have low accuracy.

[0052] In order to improve the accuracy of the obtained digital assets, this embodiment provides a polarized light scanning method based on LED virtual shooting of digital assets, such as Figure 1 As shown, the following steps are included:

[0053] S1, in the process of polarized light scanning the object, first install polarizers on both the light source and the camera lens, and then adjust the angle of the polarizer to obtain polarized images without highlights in different cross-polarization directions corresponding to different shooting angles, and then obtain each candidate feature point in the polarized image and its SIFT vector.

[0054] The above step S1 can be implemented through steps S11 to S12 (not shown in the figure):

[0055] S11, in the process of polarized light scanning an object, firstly install polarizing plates on both the light source and the camera lens, and then adjust the angle of the polarizing plates to obtain polarized images without highlights in different cross-polarization directions corresponding to different shooting angles.

[0056] Here, the polarization image can represent the surface image of the photographed object; for the polarization image without highlights, cross polarization is used to eliminate reflected light or scattered light, so it can eliminate specular reflected light (i.e., highlights) to the greatest extent, and only retain diffuse reflected light, which makes the acquired image have no highlight area, retains the real texture information of the surface of the object, and thus improves the clarity of the polarization image; the shooting angle of the polarized light scanning object and the number and size of the cross polarization directions of the rotating polarizer can be set by the implementer according to the specific actual situation, without specific limitation, wherein the cross polarization direction refers to the direction that is 90 degrees to the polarization direction, if different incident polarization directions (such as horizontal, vertical, oblique, etc.) are considered, then there can be multiple different cross polarization directions, that is, each incident polarization direction will have a corresponding cross polarization direction, and the size and number of the cross polarization directions can be set by the implementer according to the specific actual situation, without specific limitation. It is worth noting that when setting the shooting angle, it is necessary to ensure that the complete surface of the object is covered, so as to facilitate the subsequent construction of the LED virtual shooting digital asset of the object, so that the constructed digital asset has integrity.

[0057] Specifically, first, when scanning an object with polarized light, prepare a 500W flashlight with a polarizer, then mount the flashlight on the camera, and also mount a polarizer on the camera lens. By adjusting the relative angles of the light source polarizer and the lens polarizer, light in the cross-polarization direction enters the camera, thereby eliminating the high light reflection on the surface of the object; during the shooting and scanning process, the polarizer is rotated for each shooting angle to change the polarization angle, and polarized images without highlights in different cross-polarization directions corresponding to different shooting angles are obtained. Among them, multiple polarized images contain the texture information of the object in different polarization states, which provides rich data to be analyzed for subsequent feature point extraction and matching.

[0058] It is worth noting that when collecting polarization images, additional polarization scanning related technical methods can be added to further improve the effect of the obtained polarization scanning, such as:

[0059] Dynamic polarization control, specifically the introduction of an adjustable intelligent polarizer control system, automatically and dynamically adjusts the polarizer angle according to the material and light conditions of the object surface, thereby optimizing the polarized light scanning effect. This automated adjustment can reduce human errors and improve scanning efficiency and accuracy; multi-spectral polarization scanning, specifically combining a multi-spectral light source, increasing the wavelength range of polarized light, and using polarized light of different spectra to scan the material properties of the object in more detail, which will enhance the resolution of different materials, especially the reconstruction of complex textures and multi-layered materials; dual polarization path integration, specifically combining cross polarization with parallel polarization to simultaneously obtain high-reflection and low-reflection information, which can more accurately separate the objects during the scanning process. Optical properties (such as specular reflection and diffuse reflection) to more comprehensively build virtual assets; fully automated scanning process, specifically combining polarized light scanning with the object's automatic rotation table to form a fully automated multi-angle polarized light scanning system. By pre-setting different scanning paths and angles, the system can automatically complete the complete scan of the object without human operation, thereby reducing manual intervention and improving production efficiency; real-time feedback and quality assessment, specifically adding real-time data feedback and quality assessment modules. During the scanning process, the system will dynamically adjust the light source and camera parameters according to the quality and polarization information of each frame of the image to ensure the best quality of the scanned data, which can be achieved by calculating indicators such as clarity and contrast of the polarized image.

[0060] It should be noted that, since the subsequent steps need to analyze the light intensity characteristics of the polarization image, the polarization image recorded in this embodiment is an image that has been grayscaled, that is, the grayscale value of each pixel in the polarization image can be directly obtained. Among them, the implementation process of grayscale processing of the polarization image is a prior art, which is not within the scope of protection of the present invention and will not be elaborated here.

[0061] S12, based on the polarization image of each cross polarization direction corresponding to each shooting angle, obtaining each candidate feature point and its SIFT vector in the polarization image.

[0062] Here, the feature points obtained by the SIFT detection algorithm are called candidate feature points. The reason is that in the process of performing polarized light scanning on the object to complete the construction of virtual digital assets, it is necessary to change the cross-polarization direction of the polarizer in the polarization camera or adjust the relevant parameters of the polarization camera. This is manifested in the fact that the pixel points at the same position of the object in the polarization image have significant differences in their texture features in polarization images with different cross-polarization directions, resulting in the SIFT vector of the feature point being insufficient to clearly describe the key point information in the image. Therefore, it is necessary to select some feature points from the feature points obtained by the SIFT detection algorithm for feature point matching analysis. Therefore, the feature points initially obtained are called candidate feature points here.

[0063] Specifically, by using the SIFT (Scale Invariant Feature Transform) detection algorithm to process each polarized image, each feature point in each polarized image and the SIFT vector of each feature point can be obtained, and the obtained feature points are used as candidate feature points. Among them, the implementation process of the SIFT detection algorithm is a prior art, which is not within the protection scope of the present invention and will not be elaborated in detail here.

[0064] It should be noted that by installing a polarizer on the camera and adjusting the angle of the polarizer to eliminate highlights during the object scanning process, eliminating highlights can make the obtained polarized image without highlights have more stable texture features, which can provide more reliable basic data for feature point extraction and matching, facilitate the subsequent acquisition of high-quality fused images, and thus help to build more accurate LED virtual shooting digital assets.

[0065] So far, this embodiment has obtained basic data for subsequent image data analysis, namely, each candidate feature point and its SIFT vector in the polarization image without highlights in each cross-polarization direction corresponding to each shooting angle.

[0066] S2, according to the local area of ​​each candidate feature point, the neighborhood grayscale difference, the degree of diagonal feature prominence and the similarity of polarization characteristics are analyzed to determine the texture feature of each candidate feature point, and then all target feature points are selected based on the texture feature.

[0067] First of all, it should be noted that in order to exclude as much as possible the feature points that have different features of the same texture in the polarization image due to different cross-polarization directions, that is, to overcome the defect that the texture features of the pixels at the same position in different polarization images are quite different, it is necessary to select the candidate feature points with more significant texture information in the polarization image as the feature points for the final matching analysis, that is, the target feature points. The reason for determining the target feature points is that when the texture feature information of a feature point in the polarization image is relatively small, it means that the light intensity feature at the position of the feature point is very similar to the light intensity feature of the surrounding pixels. When the angle of polarized light changes, the texture information of the feature point in the polarization image will show obvious changes. At this time, whether the texture information is enhanced or weakened, it will lead to an increase in the difference in the SIFT vectors of the feature point at the same object position. Therefore, it is necessary to select feature points with more obvious texture information from all the candidate feature points in the polarization image.

[0068] In this embodiment, the texture features include a diagonal feature vector of a local area of ​​a candidate feature point and a texture saliency index. The diagonal feature vector is composed of a number of diagonal feature values, which are used to characterize the directional characteristics of the texture in the local area of ​​the candidate feature point. The texture saliency index can be obtained by comprehensive analysis of the diagonal feature value and the grayscale value of the local area of ​​the candidate feature point. Among them, the texture features of each candidate feature point can be determined, and the target feature point with more significant texture information can be selected from all the candidate feature points.

[0069] The above step S2 can be Figure 2 The steps shown achieve:

[0070] S21, for any candidate feature point, construct a window area with the candidate feature point as the center, and determine a first texture noticeability factor of the candidate feature point according to the gray value of each pixel point in the window area of ​​the candidate feature point.

[0071] Here, the first texture obvious factor can characterize the difference in pixel grayscale within the neighborhood of the candidate feature point. The greater the difference in pixel grayscale within the neighborhood, the more obvious the texture feature of the candidate feature point, so the quantified index can be used as one of the elements for determining the texture significant index. The reason for analyzing the first texture obvious factor is that for the feature point position with significant texture of the scanned object, the polarization light characteristics will affect the light intensity around the feature point with significant texture features, which is manifested as a significant difference in grayscale around the feature point at the corresponding position in the polarization image, so the light intensity characteristics can be characterized by the grayscale value; while for the feature points in the normal background area, the polarization light characteristics have little effect on the surrounding pixels, which is manifested as a high consistency in the grayscale value of the feature point and the surrounding pixels. Based on the above description, the interference of feature points with less texture details can be eliminated to a certain extent by determining the first texture obvious factor.

[0072] First, for any candidate feature point, a window area is constructed with the candidate feature point as the center.

[0073] In this embodiment, for the convenience of description and understanding, one candidate feature point is selected in the polarization image of each cross-polarization direction at each shooting angle for subsequent texture feature analysis. Since the analysis of texture features is usually implemented based on the image features of pixel points in a local area, for the selected candidate feature points, a window area of ​​a preset size is constructed with the candidate feature point as the center as the local area of ​​the candidate feature point. As an example, the size of the window area can be set to 9×9. Of course, the implementer can also set the size of the window area according to the specific size of the polarization image, which is not limited here.

[0074] Secondly, according to the gray value of each pixel point in the window area of ​​the candidate feature point, the first texture obvious factor of the candidate feature point is determined, and the specific implementation steps may include:

[0075] According to the grayscale value, all the pixels in the window area of ​​the candidate feature point are divided into two categories, and the grayscale average value of each category is calculated; the absolute value of the difference between the grayscale average values ​​of the two categories is used as the first texture obvious factor of the candidate feature point. As an example, the Otsu threshold can be used to classify all the pixels in the window area of ​​the candidate feature point. Among them, the implementation process of the Otsu threshold is a prior art, which is not within the protection scope of the present invention and will not be elaborated here.

[0076] It should be noted that the greater the difference in the average grayscale values ​​of the two categories in the window area of ​​the candidate feature point, the more obvious the texture feature is, indicating that there is an obvious difference in light intensity near the position of the candidate feature point.

[0077] S22, obtaining a diagonal feature vector of the candidate feature point according to the gray level co-occurrence matrix at different angles in the window area of ​​the candidate feature point; and determining a second texture noticeability factor of the candidate feature point according to the discreteness and size of the elements of the diagonal feature vector.

[0078] Here, the second texture obvious factor can characterize the size and discreteness of the diagonal eigenvalues ​​of each angle that constitutes the diagonal eigenvector. The diagonal eigenvalues ​​can reflect the texture direction characteristics of the local area of ​​the candidate feature points, which is specifically manifested as the distribution of pixel pairs with the same grayscale values ​​on the diagonal in the grayscale co-occurrence matrix of the local area. The larger the diagonal eigenvalue, the more pixel pairs with the same grayscale values ​​on the diagonal, and thus the more significant the texture characteristics of the polarization scan in the corresponding diagonal direction.

[0079] The above step S22 can be implemented through steps S221 to S224 (not shown in the figure):

[0080] S221, setting several angles, and obtaining a grayscale co-occurrence matrix for each angle based on the grayscale value of each pixel in the window area of ​​the candidate feature point.

[0081] As an example, first set 4 angles, namely 0, and Of course, implementers can also set other numbers of angle sizes according to specific actual conditions, which is not limited here.

[0082] In this embodiment, the diagonal of the grayscale co-occurrence matrix at a certain angle represents the number of pixel pairs with the same grayscale value. The more the number of pixel pairs with the same grayscale value, the higher the similarity of the polarized light intensity in the direction of the angle. Therefore, the grayscale value on the diagonal can characterize the directional characteristics of the texture. The larger the index obtained by quantifying the grayscale value on the diagonal, the more significant the texture characteristics of the polarization scan in the diagonal direction. Among them, the acquisition process of the grayscale co-occurrence matrix is ​​a prior art and is not within the scope of protection of the present invention, and will not be elaborated here.

[0083] S222, using a diagonal function to determine the diagonal pixel points in the gray level co-occurrence matrix at each angle, and then calculating the binary norm of the gray value of the diagonal pixel points in the gray level co-occurrence matrix.

[0084] S223, normalizing the binary norm to obtain a normalized value, and then taking a vector composed of the normalized values ​​obtained at each angle as a diagonal feature vector.

[0085] In this embodiment, the normalized value obtained at each angle is called a diagonal eigenvalue. As an example, the calculation formula for the diagonal eigenvalue of the hth angle may be:

[0086] In the formula, S(θ h ) represents the diagonal eigenvalue of the hth angle of the candidate feature point, ‖‖ represents the second norm, diag represents the diagonal function, G(θ h ) represents the gray level co-occurrence matrix of the h-th angle of the candidate feature point.

[0087] In the calculation formula of the diagonal eigenvalue, the diagonal eigenvalue S(θ h ) is larger, indicating that the texture features of the candidate feature point window area at the hth angle are more significant, and the candidate feature point is more likely to be selected for feature point matching, which can effectively improve the accuracy of feature point matching; ‖diag(G(θ h ))‖ can characterize the diagonal texture features of the hth angle in the window area, ‖diag(G(θ h ))‖ is larger, the more significant the diagonal texture feature is, the more obvious the texture feature of the candidate feature point is, and the denominator of the diagonal eigenvalue‖G(θ h )‖ represents the texture feature of the entire window area at the h-th angle, which is used to realize the normalization processing of the diagonal eigenvalues, and can also be replaced by the non-diagonal texture feature of the h-th angle.

[0088] After obtaining the diagonal texture feature value of each angle, all diagonal texture feature values ​​are combined into a feature vector in the order of angle from small to large, which is used to represent the diagonal features of the window area and is recorded as the diagonal feature vector, that is,

[0089]

[0090] S224, determining a second texture noticeability factor of the candidate feature point according to the discreteness and size of the elements of the diagonal feature vector.

[0091] As an example, the average value of all elements in the diagonal feature vector is calculated, and the standard deviation of all elements in the diagonal feature vector is calculated, and the product of the average value and the standard deviation is used as the second texture obvious factor of the candidate feature point. Of course, the average value can be replaced by calculating the cumulative value, and the standard deviation can be replaced by calculating the variance.

[0092] It should be noted that when calculating the second texture conspicuousness factor, two aspects of the texture features of the diagonal eigenvector are analyzed, namely, the diagonal feature significance and the polarization characteristic difference. The polarization characteristic difference refers to the consistency between the diagonal eigenvalues ​​at various angles in the local area of ​​the candidate feature points.

[0093] It is worth noting that if the diagonal features of the local area of ​​the candidate feature points are not obvious, that is, most of the pixels in the local area do not appear as texture feature details of the polarized light scan, but reflect the normal background of the scanned object, and there are also more pixels with the same grayscale value in the background in the grayscale co-occurrence matrix, which makes the obtained diagonal eigenvalues ​​also larger, that is, the texture features of the local area obtained by the grayscale co-occurrence matrix appear as the background, rather than the details of the polarized light scanned object, so the window corresponding to the pixels with significant diagonal features is not necessarily the area with rich details in the polarized image.

[0094] The reason for analyzing the difference in polarization characteristics is that the detailed texture of the scanned object will cause obvious changes in the polarization characteristics of the reflected light of the polarized light in different directions, such as polarization degree, polarization angle, light intensity, etc. Therefore, when the texture details are richer in the local area, there are obvious differences in the polarization characteristics in different directions in the polarization image; on the contrary, the local area with fewer details does not have obvious texture features, and it appears as a flat area, and the reflected light in different directions of the flat area is basically the same, so the polarization characteristics of the reflected light remain consistent, which is reflected in the high similarity of the polarization characteristics in various directions of the polarization image; and since light intensity is one of the polarization characteristics in the polarization image, in the polarization image, the light intensity is represented by the gray value of the pixel point, and the texture characteristics of the scanned object represented by the light intensity in the window can be represented by the diagonal features obtained above, therefore, the difference in polarization characteristics can be represented by the discreteness between the diagonal feature values ​​in each direction, that is, it is necessary to determine the standard deviation of all elements in the diagonal feature vector.

[0095] S23, determining a texture saliency index of the candidate feature point by combining the first texture saliency factor and the second texture saliency factor of the candidate feature point; constructing a texture feature of the candidate feature point based on the texture saliency index and the diagonal feature vector of the candidate feature point.

[0096] In this embodiment, the first texture obvious factor, the second texture obvious factor and the texture significant index are all positively correlated. The larger the first texture obvious factor and the second texture obvious factor are, the larger the texture significant index is. In this embodiment, the texture feature of each candidate feature point includes two factors, one is the diagonal feature of the local area of ​​the candidate feature point, and the other is the texture feature significance of the candidate feature point.

[0097] As an example, the product of the first texture noticeable factor and the second texture noticeable factor is first calculated, and then the product is normalized to obtain a normalized value, and the normalized value is used as the texture noticeable index of the candidate feature point. Among them, the normalization method can be maximum and minimum normalization, and its implementation process is a prior art, which is not within the protection scope of the present invention and will not be elaborated in detail here.

[0098] Among them, the calculation formula of the texture saliency index of the candidate feature points can be:

[0099] p=δZ×E(S)×a; wherein p represents the initial texture saliency index of the candidate feature point, δZ represents the first texture saliency factor of the candidate feature point, E(S) represents the average value of all elements in the diagonal feature vector, a represents the standard deviation of all elements in the diagonal feature vector, and E(S)×a represents the third texture saliency factor of the candidate feature point.

[0100] In the formula, p ′ represents the texture saliency index of the candidate feature point, p represents the initial texture saliency index of the candidate feature point, and p max represents the maximum initial texture saliency index corresponding to all candidate feature points, p min Represents the minimum initial texture saliency index corresponding to all candidate feature points.

[0101] It should be noted that the numerical accuracy of the texture saliency index determined from multiple aspects is higher. Determining the texture saliency index with higher numerical accuracy helps to subsequently select feature points with stronger reliability and richer detail features for matching processing, thereby improving the image quality of the fused image at each shooting angle obtained by the polarization scanning object.

[0102] S24, selecting all target feature points in the polarization image according to the texture saliency index of each candidate feature point in the polarization image of each cross-polarization direction.

[0103] In this embodiment, the larger the texture saliency index is, the higher the texture saliency of the candidate feature point is, the higher the similarity in texture performance of the pixel points at corresponding positions in the polarization images of different cross-polarization directions at the same shooting angle corresponding to the candidate feature point is, and the greater the possibility of subsequent feature point matching for the corresponding candidate feature point is.

[0104] As an example, the texture saliency indicators of all candidate feature points in the same polarization image are arranged in order from large to small to obtain a texture saliency indicator sequence; the candidate feature points corresponding to the first 75% of the texture saliency indicators in the texture saliency indicator sequence are selected as the target feature points in the corresponding polarization image.

[0105] In another example, a significant index threshold is set, and candidate feature points whose texture significant index is greater than or equal to the significant index threshold are all taken as target feature points. The significant index threshold can be taken as an empirical value of 0.6.

[0106] Among them, the expression for selecting the target feature point can be:

[0107] M={x:p ′ (x)>0.6}; where M represents the target feature point set, x represents the candidate feature point, and p ′ (x) represents the texture saliency index of the candidate feature point.

[0108] It should be noted that the target feature points determined by the texture saliency index can avoid the situation where only the SIFT feature of the feature points is significant, and can effectively exclude feature points with unclear texture details, because such feature points are likely to appear as feature points in a polarization image of a certain cross-polarization direction, but when the cross-polarization direction changes, the feature points at the corresponding position are likely to appear as feature points with unclear texture.

[0109] So far, this embodiment obtains target feature points in polarization images with different cross-polarization directions at different shooting angles.

[0110] S3, matching target feature points between different polarization images according to the gray value, texture feature and SIFT vector of each target feature point in the polarization image of each cross-polarization direction, and obtaining fused images corresponding to all cross-polarization directions.

[0111] It should be noted that during the shooting of virtual digital assets, the polarizer needs to be adjusted to achieve shooting of the scanned object in different cross-polarization directions. Different cross-polarization directions may cause a certain degree of change in texture details, which in turn causes the same scanning position to present different features under different polarization angles. If the feature points extracted at this time are directly used for matching, there is a high probability that the feature points will be matched incorrectly.

[0112] The texture difference of the scanned object image under different cross-polarization directions is mainly reflected in the difference in reflection or projection of polarized light in different directions on the same position, which makes the texture features have certain differences. This difference is reflected through the polarization characteristics of polarized light, mainly manifested as the grayscale change of pixels in the polarization image. In addition, the texture feature difference of the polarization image of the feature point at the same scanning position under different cross-polarization directions is mainly reflected in the large difference in the significance of the texture feature. The difference in the significance of the texture feature is due to the difference in the polarization state at the same position, which leads to the difference in the performance of local light, and then the grayscale difference between the pixels in different polarization images and the surrounding pixels is different, resulting in matching errors.

[0113] The diagonal features obtained in the above steps are to extract the grayscale pairs of the same grayscale value in the grayscale co-occurrence matrix, which can only show the same or different grayscale values, without considering the grayscale pairs of pixels with grayscale differences. Therefore, when using conventional feature matching methods to match and analyze the target feature points, there is still the possibility of inaccurate matching results. For example, the grayscale pair in the polarization image of a cross-polarization direction at the same position of the scanned object is (3,3), while the grayscale pair in the polarization image of another cross-polarization direction is (5,5). Therefore, this embodiment needs to correct the process of matching the target feature points based only on the SIFT vector through the grayscale value and texture features of the target feature points, so as to obtain high-quality and true fused images corresponding to all cross-polarization directions corresponding to the same shooting angle through accurate matching results.

[0114] The above step S3 can be Figure 3 The steps shown achieve:

[0115] S31, for any target feature point, cluster all pixel points in the window area of ​​the target feature point according to a number of clustering categories and grayscale values ​​to obtain each clustering window image matrix corresponding to the target feature point.

[0116] In this embodiment, each target feature point has a corresponding number of clustering window image matrices. The clustering window image matrix is ​​obtained in order to quantify the grayscale difference between the local area constructed by the pixel point and the surrounding pixel points in different polarization images. By clustering all the pixel points in the window area of ​​each target feature point, a clustering window image matrix corresponding to the target feature point can be obtained. The elements in the clustering window image matrix are the category mark numbers corresponding to the pixel grayscale, that is, the category number. Using the category number instead of the grayscale value can ignore the difference caused by the grayscale value to a certain extent.

[0117] The above step S31 can be implemented by the following steps:

[0118] For any number of clustering categories, cluster all pixels in the window area of ​​the target feature point according to the grayscale value to obtain the initial clustering window area; according to the preset marking rule, mark the pixels belonging to the same cluster cluster in the initial clustering window area with the same category number; according to the category number of each pixel in the initial clustering window area, obtain the clustering window image matrix of the number of clustering categories. Among them, the preset marking rule refers to setting the category number size and category number marking principle of the clustering cluster, which can be set by the implementer according to the specific actual situation without specific limitation. In addition, the size and number of clustering categories can also be set by the implementer according to the specific actual situation without specific limitation.

[0119] As an example, the number of cluster categories can be 2, 3, 4 and 5. When the number of cluster categories is 2, all pixels in the window area of ​​the target feature point are divided into two categories according to the gray value of each pixel in the window area of ​​the target feature point, and the gray average values ​​of the two categories are calculated; the category number of the category with a larger gray average value is set to 1, and the category number of the category with a smaller gray average value is set to 0, and each pixel in the window area of ​​the target feature point is marked with a corresponding category number, and the category number of each pixel in the window area can be used to form a cluster window image matrix when the number of cluster categories is 2. Among them, the matrix is ​​the same size as the window area, and the pixels corresponding to the elements in the matrix have the same characteristics as the pixels in the window area in a one-to-one comparison.

[0120] S32, determining the degree of texture feature matching between two target feature points according to each clustering window image matrix, texture feature and SIFT vector corresponding to any target feature point in the polarization images of any two cross-polarization directions.

[0121] In this embodiment, the texture feature matching degree is used to indicate the similarity of the diagonal features, SIFT vectors and local area grayscale between two target feature points located in different polarization images. The numerical accuracy of the texture feature matching degree determined by different factors is higher, which can overcome the adverse effects of polarizer rotation to a certain extent, thereby avoiding the matching errors caused by subsequent target feature point matching. In addition, the greater the texture feature matching degree, the greater the possibility that the two target feature points are a matching point pair with each other. By determining the texture feature matching degree, the corresponding matching feature points can be determined for the target feature points in different polarization images.

[0122] The above step S32 can be implemented through steps S321 to S324 (not shown in the figure):

[0123] S321, for any two polarization images with cross polarization directions, use one of them as a template image and the other as an image to be matched, and select any one target feature point from the image to be matched and the template image to form a target feature point pair.

[0124] In this embodiment, there are multiple polarization images of cross polarization directions in the same shooting direction, and the objects for target feature point matching can only be polarization images of two cross polarization directions, so only two polarization images of cross polarization directions can be selected first, and after image fusion of target feature point matching, image fusion analysis is performed based on the fused image and the remaining polarization image, and then image fusion analysis is performed again based on the newly obtained fused image and the remaining polarization image, and the analysis is continuously iterated until all polarization images of cross polarization directions in the same shooting angle are fused into one image. Among them, the calculation process of the texture feature matching degree of each target feature point pair is the same. In order to reduce unnecessary description and facilitate understanding, the texture feature matching degree is determined by taking one of the target feature point pairs as an example.

[0125] S322, determining a first texture feature similarity factor of the target feature point pair according to the number of each cluster category of the target feature point pair and the clustering window image matrix of each cluster category number.

[0126] Here, the first texture feature similarity factor may characterize the local grayscale similarity between two target feature points in different polarization images, which is specifically determined by performing data analysis on the number of cluster categories and the cluster window image matrix of each cluster analysis.

[0127] As an example, the calculation formula of the first texture feature similarity factor of the target feature point pair is:

[0128] Where L1(X,Y) represents the first texture feature similarity factor of the target feature point pair, X represents one of the target feature points in the target feature point pair, Y represents the other target feature point in the target feature point pair, K represents the number of cluster categories, K′ represents the maximum number of cluster categories, norm represents the linear normalization function, and f K (X) represents the clustering window image matrix when the number of clustering categories of the target feature point X is K, f K (Y) represents the clustering window image matrix when the number of clustering categories of the target feature point Y is K, and ‖‖ represents the binary norm.

[0129] In the calculation formula of the first texture feature similarity factor, the larger the number of cluster categories K is, the more grayscale categories the clustering window image matrix has, and the higher the clustering accuracy is; ‖f K (X)-f K(Y)‖ represents the difference between the two clustering window image matrices in different polarization images when the number of clustering categories is K. The smaller the difference between the two clustering window image matrices, the stronger the neighborhood similarity of the target feature points. are both large, indicating that the texture features of the target feature point pair are more similar, and the local grayscale similarity between the two target feature points is also higher; the first texture feature similarity factor L1(X, Y) reflects the similarity of texture or details of the local area of ​​the target feature point pair at different scales. The larger the first texture feature similarity factor, the higher the consistency of the texture and details of the local area at different scales, and the greater the possibility that the target feature point pair belongs to the same area of ​​the scanned object, which means that the matching degree of the target feature point pair should be higher.

[0130] S323, obtaining a second texture feature similarity factor of the target feature point pair according to the similarity between diagonal feature vectors and the similarity between SIFT vectors in the texture features of the target feature point pair.

[0131] Here, the second texture feature similarity factor is determined from two aspects, namely, the similarity between the diagonal feature vectors and the similarity between the SIFT vectors. In the existing feature point matching, only the similarity between the SIFT vectors is matched, but due to the image characteristics of the polarized light scanning, the existing feature point matching results have deviations, so it is necessary to combine the similarity between the diagonal feature vectors and the first texture feature similarity factor to correct the similarity between the SIFT vectors to obtain a matching analysis index that is more in line with the actual situation, that is, to determine the degree of texture feature matching. At this time, after determining the first texture feature similarity factor, it is necessary to determine the second texture feature similarity factor of the target feature point pair.

[0132] As an example, the calculation formula of the second texture feature similarity factor of the target feature point pair can be:

[0133] Where L2(X,Y) represents the second texture feature similarity factor of the target feature point pair, S(X) represents the diagonal feature vector of the target feature point X, S(Y) represents the diagonal feature vector of the target feature point Y, SIFT(X) represents the SIFT vector of the target feature point X, and SIFT(Y) represents the SIFT vector of the target feature point Y.

[0134] It should be noted that the elements of the grayscale co-occurrence matrix, clustering window image matrix, diagonal eigenvector and SIFT vector in this embodiment are not all zero, so the values ​​at the denominator position of the diagonal eigenvalues, the first texture feature similarity factor and the second texture feature similarity factor recorded above do not have the possibility of being zero.

[0135] S324: Determine the texture feature matching degree of the target feature point pair by combining the first texture feature similarity factor and the second texture feature similarity factor of the target feature point pair.

[0136] In this embodiment, the first texture feature similarity factor, the second texture feature similarity factor and the texture feature matching degree are positively correlated. The larger the first texture feature similarity factor and the second texture feature similarity factor, the more matching the texture features of the target feature point pair. Among them, the greater the texture feature matching degree of the target feature point pair, the greater the possibility that the target feature point pair belongs to the same scanning area position, and the higher the corresponding matching degree.

[0137] As an example, the product of the first texture feature similarity factor and the second texture feature similarity factor is used as the texture feature matching degree of the target feature point pair.

[0138] In another example, a value obtained by adding the first texture feature similarity factor and the second texture feature similarity factor is used as the texture feature matching degree of the target feature point pair.

[0139] S33, matching target feature points according to the texture feature matching degree to obtain fused images corresponding to all cross polarization directions.

[0140] Here, each shooting angle has its corresponding fused image, and fused images of different shooting angles can be used to realize the construction of digital assets. The fused image is realized on the basis of feature point matching, so it is necessary to determine a most matching target feature point in the template image for the target feature point in the image to be matched according to the degree of texture feature matching, and then use the image fusion algorithm based on feature point matching to complete the fusion of polarized images in different cross-polarization directions. Among them, the implementation process of the image fusion algorithm is a prior art and is not within the scope of protection of the present invention, and will not be elaborated here.

[0141] The above step S33 can be implemented through steps S331 to S333 (not shown in the figure):

[0142] S331, determining the degree of texture feature matching between any target feature point in the image to be matched and each target feature point in the template image, and taking the target feature point corresponding to the maximum texture feature matching degree in the template image as the matching feature point of the target feature point in the image to be matched.

[0143] In this embodiment, the greater the texture feature matching degree, the more the texture features of the two target feature points in the different polarization images match, so the target feature point corresponding to the maximum texture feature matching degree in the template image can be used as the matching feature point of the corresponding target feature point in the image to be matched. The matching feature point is determined to achieve the fusion of two polarization images with different cross polarization directions.

[0144] Specifically, for any target feature point in the image to be matched, refer to the calculation process of the texture feature matching degree of the above-mentioned target feature point pair, so that the target feature point and each target feature point in the template image form a target feature point pair, and calculate the texture feature matching degree; determine the maximum value from all the texture feature matching degrees, and use the target feature point in the template image corresponding to the maximum value as the matching feature point of the target feature point in the image to be matched, and then perform the above-mentioned matching process on each target feature point in the image to be matched to determine the matching feature point of each target feature point in the image to be matched.

[0145] The expression for determining the matching feature point of the target feature point in the image to be matched can be:

[0146] Y X = argmax Y math(X,Y); where Y X Represents the matching feature points of the target feature points in the image to be matched, X represents the target feature points in the image to be matched, Y represents the target feature points in the template image, math(X,Y) represents the degree of texture feature matching between the target feature points in the image to be matched and the target feature points in the template image, and argmax represents the independent variable that makes a function reach the maximum value.

[0147] S332, according to each target feature point in the image to be matched and its matching feature point, perform image fusion on the image to be matched and the template image to obtain a first fused image.

[0148] In this embodiment, after obtaining each target feature point and its matching feature point in the image to be matched, the image to be matched and the template image can be fused using an image fusion algorithm based on feature point matching, thereby obtaining a fused image.

[0149] Specifically, first, based on each target feature point and its matching feature point in the image to be matched, the homography matrix between the target feature point and its matching feature point is calculated by the RANSAC (Random Sample Consensus) algorithm to improve the robustness of the matching; secondly, the homography matrix is ​​used to transform the image to be matched so that it is aligned with the template image; then, based on the alignment, the image to be matched and the template image are fused together by a simple averaging method or a more complex fusion technique, such as the Laplacian pyramid; finally, the fused image is smoothed to eliminate possible ghosting or unnatural seams.

[0150] As another specific implementation, according to each target feature point and its matching feature point in the image to be matched, feature matching enhanced by deep learning is used, that is, matching and fusing feature points using a deep learning algorithm to replace or enhance the traditional SIFT method, which can more accurately separate the optical properties of the object (such as specular reflection and diffuse reflection, etc.) during the scanning process to more comprehensively construct virtual assets. Among them, the implementation process of deep learning enhancement is a prior art and is not within the scope of protection of the present invention, and will not be elaborated here.

[0151] S333, taking the first fused image as a template image, performing image fusion processing based on feature point matching according to each target feature point in the template image and the polarization image of any other cross-polarization direction, to obtain a second fused image; again taking the second fused image as a template image, performing image fusion processing based on feature point matching according to each target feature point in the template image and the polarization image of any other cross-polarization direction, to obtain a third fused image; and continuously iterating until a fused image corresponding to all cross-polarization directions is obtained.

[0152] In this embodiment, after the polarization images of two cross-polarization directions are fused together, the fused image is used as a new template image, and the remaining polarization images of any other cross-polarization directions are used as images to be matched. Referring to the above steps S31 to S33, the image fusion is performed again, and the fused image is obtained again, and the fusion analysis is performed again with the remaining polarization images of any other cross-polarization directions as a new template image, until all the polarization images of the cross-polarization directions are fused into one image, and the fused images corresponding to all the cross-polarization directions can be obtained. Among them, the polarization images of other cross-polarization directions refer to the polarization images that are not involved in the image fusion.

[0153] So far, this embodiment has obtained the fused images corresponding to all cross polarization directions at each shooting angle.

[0154] S4, constructs the LED virtual shooting digital asset of the object through the fusion image of each shooting angle.

[0155] In this embodiment, a three-dimensional model of the scanned object can be constructed based on the fused image of each shooting angle, and the obtained three-dimensional model is used as the LED virtual shooting digital asset of the object. Of course, the implementer can also directly use the fused image of each shooting angle as the final LED virtual shooting digital asset, which is not specifically limited in this embodiment.

[0156] Specifically, based on the fused images from each shooting angle, a multi-view geometry method is used to perform 3D reconstruction in combination with depth information to obtain a 3D model; in order to improve the authenticity of the 3D model, the generated 3D model is optimized, including texture mapping, lighting adjustment, etc.; finally, the constructed 3D model is imported into the required application, such as a game engine, virtual reality environment, etc.

[0157] Thus, this embodiment obtains LED virtual photographed digital assets with higher authenticity and accuracy by improving the feature point selection and feature point matching in the polarized light scanning technology process.

[0158] This embodiment also provides a polarized light scanning system for digital assets based on LED virtual photography, including a processor and a memory, wherein the processor is used to process instructions stored in the memory to implement a polarized light scanning method for digital assets based on LED virtual photography as described above.

[0159] The present invention provides a polarized light scanning method and system for digital assets based on LED virtual photography. The basic data set is first acquired. During the process of polarized light scanning the object, the polarizer needs to be rotated to acquire polarized images without highlights in different cross-polarization directions corresponding to different shooting angles, and then each candidate feature point and its SIFT vector in the polarized image are acquired. The acquired candidate feature points and their SIFT vectors provide data support for subsequent image data analysis, and are also convenient for combining the polarized light scanning method for eliminating highlights by installing a polarizer with an improved feature point selection and matching algorithm. This method not only eliminates the influence of highlight reflection and improves the quality of texture information of the acquired polarized image, but is also beneficial for the subsequent preliminary enhancement of the accuracy of feature point matching based on high-quality texture information. This combination method can effectively solve the problem of low feature point matching accuracy caused by changes in highlights and polarization angles in the prior art, and greatly improves the construction accuracy of digital assets for LED virtual photography. When rotating the polarizer, the changes in different cross-polarization directions or polarization parameters will affect the presentation of the polarization image. Specifically, the pixels at the same position of the scanned object have significant feature differences in the polarization images of different cross-polarization directions, which makes the SIFT vector of the feature point at the same position insufficient to clearly describe the key point information in the image. Therefore, it is necessary to analyze the neighborhood grayscale differences, the degree of diagonal feature significance, and the similarity of polarization characteristics based on the local area of ​​each candidate feature point to determine the texture features, and then select all the target feature points based on the texture features. Determining the target feature points can effectively eliminate the adverse effects brought by the significant SIFT vector features of the pixels, that is, eliminate the interference of feature points with unclear texture details. The existing feature point matching ignores the texture differences of the scanned object images at different polarization angles when performing feature point matching fusion based on the target feature points, that is, different polarized lights have different reflections and projections on the same position, which leads to certain differences in texture features, which are manifested as grayscale changes of pixels in the polarization image. Therefore, it is necessary to correct the matching conditions of the feature points. Specifically, the influence of multiple factors is taken into account during the matching analysis, that is, the target feature points between different polarization images are matched according to the grayscale value, texture features and SIFT vector of each target feature point in the polarization image of each cross-polarization direction, and the fused images corresponding to all cross-polarization directions are obtained; the fused image determined by improving the feature point matching has higher image quality and stronger authenticity, which reduces the adverse effects of changes in cross-polarization directions or polarization parameters on the generated fused image to a certain extent, and further improves the accuracy of constructing the object's LED virtual shooting digital assets.

[0160] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A polarized light scanning method for digital assets based on LED virtual photography, characterized in that: The following steps are involved: In the process of polarized light scanning an object, polarizers are first installed on both the light source and the camera lens, and then the angle of the polarizer is adjusted to obtain polarized images without highlights in different cross-polarization directions corresponding to different shooting angles, and then each candidate feature point in the polarized image and its SIFT vector are obtained; According to the local area of ​​each candidate feature point, the neighborhood grayscale difference, the degree of diagonal feature prominence and the similarity of polarization characteristics are analyzed to determine the texture feature of each candidate feature point, and then all target feature points are selected based on the texture feature; According to the gray value, texture feature and SIFT vector of each target feature point in the polarization image of each cross-polarization direction, the target feature points between different polarization images are matched to obtain the fused images corresponding to all cross-polarization directions; Construct an LED virtual shot digital asset of the object by fusing images from each shooting angle.

2. The polarized light scanning method based on LED virtual shooting of digital assets according to claim 1, characterized in that: The method of analyzing the neighborhood grayscale difference, the degree of diagonal feature prominence, and the similarity of polarization characteristics of each candidate feature point in the local area to determine the texture feature of each candidate feature point includes: For any candidate feature point, a window area is constructed with the candidate feature point as the center, and a first texture noticeable factor of the candidate feature point is determined according to the gray value of each pixel point in the window area of ​​the candidate feature point; According to the gray level co-occurrence matrix at different angles in the window area of ​​the candidate feature point, the diagonal feature vector of the candidate feature point is obtained; according to the element discreteness and element size of the diagonal feature vector, the second texture obvious factor of the candidate feature point is determined; The texture saliency index of the candidate feature point is determined by combining the first texture saliency factor and the second texture saliency factor of the candidate feature point; and the texture feature of the candidate feature point is constructed based on the texture saliency index and the diagonal feature vector of the candidate feature point.

3. The polarized light scanning method based on LED virtual shooting of digital assets according to claim 2, characterized in that: The step of determining the first texture noticeability factor of the candidate feature point according to the grayscale value of each pixel in the window area of ​​the candidate feature point comprises: According to the grayscale value, all the pixels in the window area of ​​the candidate feature points are divided into two categories, and the grayscale average value of each category is calculated; The absolute value of the difference between the grayscale average values ​​of the two categories is used as the first texture obvious factor of the candidate feature point.

4. The polarized light scanning method based on LED virtual shooting of digital assets according to claim 2, characterized in that: The diagonal feature vectors of the candidate feature points are obtained according to the gray level co-occurrence matrices at different angles in the window area of ​​the candidate feature points, including: Set several angles, and obtain the grayscale co-occurrence matrix of each angle based on the grayscale value of each pixel in the window area of ​​the candidate feature points; Determine the diagonal pixel points in the gray level co-occurrence matrix at each angle by using a diagonal function, and then calculate the binary norm of the gray value of the diagonal pixel points in the gray level co-occurrence matrix; The bi-norm is normalized to obtain a normalized value, and then a vector composed of the normalized values ​​obtained at each angle is used as a diagonal feature vector.

5. The polarized light scanning method based on LED virtual shooting of digital assets according to claim 2, characterized in that: According to the gray value, texture feature and SIFT vector of each target feature point in the polarization image of each cross-polarization direction, target feature point matching between different polarization images is performed to obtain fused images corresponding to all cross-polarization directions, including: For any target feature point, all pixels in the window area of ​​the target feature point are clustered according to the grayscale value according to the number of cluster categories, and the image matrix of each cluster window corresponding to the target feature point is obtained; Determine the texture feature matching degree between two target feature points according to each clustering window image matrix, texture feature and SIFT vector corresponding to any target feature point in the polarization images of any two cross-polarization directions; Target feature points are matched according to the texture feature matching degree to obtain fused images corresponding to all cross-polarization directions.

6. The polarized light scanning method based on LED virtual shooting of digital assets according to claim 5, characterized in that: The method clusters all pixel points in the window area of ​​the target feature point according to the grayscale value according to the number of cluster categories to obtain each cluster window image matrix corresponding to the target feature point, including: For any number of clustering categories, all pixels in the window area of ​​the target feature point are clustered according to the grayscale value to obtain the initial clustering window area; according to the preset marking rules, the pixels belonging to the same cluster cluster in the initial clustering window area are marked with the same category number; according to the category number of each pixel in the initial clustering window area, the clustering window image matrix of the number of clustering categories is obtained.

7. The polarized light scanning method based on LED virtual shooting of digital assets according to claim 6, characterized in that: The method of determining the degree of texture feature matching between two target feature points according to each clustering window image matrix, texture feature and SIFT vector corresponding to any target feature point in the polarization images of any two cross-polarization directions comprises: For any two polarization images with cross polarization directions, one of them is used as a template image and the other is used as an image to be matched. One target feature point is selected from the image to be matched and the template image to form a target feature point pair. Determine a first texture feature similarity factor of the target feature point pair according to the number of each cluster category of the target feature point pair and the cluster window image matrix of each cluster category number; According to the similarity between the diagonal feature vectors in the texture features of the target feature point pair and the similarity between the SIFT vectors, a second texture feature similarity factor of the target feature point pair is obtained; The first texture feature similarity factor and the second texture feature similarity factor of the target feature point pair are combined to determine the texture feature matching degree of the target feature point pair.

8. The polarized light scanning method based on LED virtual shooting of digital assets according to claim 7, characterized in that: The first texture feature similarity factor of the target feature point pair is determined according to the number of each clustering category of the target feature point pair and the clustering window image matrix of each clustering category number, including The calculation formula of the first texture feature similarity factor of the target feature point pair is: Where L1(X,Y) represents the first texture feature similarity factor of the target feature point pair, X represents one of the target feature points in the target feature point pair, Y represents the other target feature point in the target feature point pair, K represents the number of cluster categories, K′ represents the maximum number of cluster categories, norm represents the linear normalization function, and f K (X) represents the clustering window image matrix when the number of clustering categories of the target feature point X is K, f K (Y) represents the clustering window image matrix when the number of clustering categories of the target feature point Y is K, and ‖‖ represents the binary norm.

9. The polarized light scanning method for digital assets based on LED virtual photography according to claim 7, characterized in that: The matching of target feature points according to the texture feature matching degree to obtain fused images corresponding to all cross-polarization directions includes: Determine the texture feature matching degree between any target feature point in the image to be matched and each target feature point in the template image, and use the target feature point corresponding to the maximum texture feature matching degree in the template image as the matching feature point of the target feature point in the image to be matched; According to each target feature point in the image to be matched and its matching feature point, the image to be matched and the template image are fused to obtain a first fused image; The first fused image is used as a template image, and image fusion processing based on feature point matching is performed according to each target feature point in the template image and any other polarization image in the cross-polarization direction to obtain a second fused image; The second fused image is used as a template image again, and image fusion processing based on feature point matching is performed according to each target feature point in the template image and any other polarization image in the cross-polarization direction to obtain a third fused image; The iteration is continued until a fused image corresponding to all cross polarization directions is obtained; wherein the polarization images of other cross polarization directions refer to polarization images that are not involved in image fusion.

10. A polarized light scanning system based on LED virtual shooting of digital assets, characterized in that: It comprises a processor and a memory, wherein the processor is used to process instructions stored in the memory to implement a polarized light scanning method for digital assets based on LED virtual photography as described in any one of claims 1-9.

Images