Polarized Light Scanning Method and System for LED Virtual Shooting Digital Assets

Abstract

The present invention relates to the technical field of object scanning, and particularly relates to a polarized light scanning method and system for LED virtual shooting digital assets, including: obtaining each candidate feature point and its SIFT vector in polarization images without highlights with different cross-polarization directions corresponding to different shooting angles; determining texture features according to the neighborhood gray-scale difference, the obviousness of diagonal features, and the polarization characteristic difference in the local area of each candidate feature point, and then selecting all target feature points; performing matching of target feature points between different polarization images according to the gray-scale value, texture features, and SIFT vectors of each target feature point to obtain a fused image, and further constructing the LED virtual shooting digital assets of the object. The present invention improves the construction accuracy of the LED virtual shooting digital assets of the object by improving the selection and matching of feature points in the process of scanning the object with polarized light.

Description

Technical Field

[0001] The present invention relates to the technical field of object scanning, and specifically relates to a polarized light scanning method and system for LED virtual shooting digital assets that obtains a high - light - free texture map by installing polarizers on a light source and a camera lens and using polarized light to eliminate highlights. Background Art

[0002] Traditional polarized light scanning technology usually installs polarizers on a light source (flashlight) and a camera lens. By using the characteristics of the polarizers and adjusting the polarization angle, only light with a specific polarization angle can enter the camera, thereby eliminating the specular reflection on the object surface and obtaining a high - light - free texture map. Specifically, the texture information of the object is revealed by analyzing the changes in the polarization states of reflection and transmission. In the creation and application of LED (Lighting Emitting Diode) virtual digital assets, it can enhance the realism and three - dimensional sense of the image, avoid the adverse effects caused by highlights during the scanning process, provide a more realistic visual effect, that is, improve the construction quality of digital assets. Since during the process of constructing virtual digital assets by performing polarized light scanning on an object, it is necessary to adjust the polarization - related parameters of the shooting device. Polarization images with different cross - polarization directions will be presented in the same shooting direction. By fusing different polarization images, the acquisition of LED virtual digital assets can be achieved.

[0003] However, in the process of realizing image fusion in existing polarized light scanning, the SIFT algorithm is often used to extract and match image feature points. Due to the changes in polarization - related parameters during the shooting process, there will be significant differences in the features of pixel points at the same position of the scanned object in different polarization images, making the SIFT vectors of the feature points insufficient to clearly describe the key point information in the image. There are large differences in the SIFT feature vectors at the same position in different polarization images, and there is a possibility of incorrect feature point matching, which in turn affects the accuracy of the digital asset construction result of the photographed object. Summary of the Invention

[0004] In order to solve the technical problem that the digital assets constructed by the above - mentioned existing polarized light scanning method are deviated, the purpose of the present invention is to provide an improved polarized light scanning method. Specifically, on the basis of installing polarizers on a flash lamp and a camera lens and adjusting the angles of the polarizers to eliminate the specular reflection on the object surface and obtain a high - light - free texture map, aiming at the problem of low accuracy of feature point matching in the process of image fusion, a polarized light scanning method and system for LED virtual shooting digital assets are proposed. The specific technical solutions adopted are as follows:

[0005] An embodiment of the present invention provides a polarized light scanning method and system for LED virtual shooting digital assets. The method includes the following steps:

[0006] During the process of scanning an object with polarized light, first install polarizers on both the light source and the camera lens, and then adjust the angles of the polarizers to obtain polarization images without highlights with different cross-polarization directions corresponding to different shooting angles, and further obtain each candidate feature point and its SIFT vector in the polarization image;

[0007] Analyze the neighborhood gray-scale difference, the obviousness of diagonal features, and the polarization characteristic difference in the local area of each candidate feature point to determine the texture feature of each candidate feature point, and further select all target feature points based on the texture feature;

[0008] Match the target feature points between different polarization images according to the gray-scale values, texture features, and SIFT vectors of each target feature point in the polarization image of each cross-polarization direction to obtain the fused images corresponding to all cross-polarization directions;

[0009] Construct the LED virtual shooting digital asset of the object through the fused images of each shooting angle.

[0010] Further, the analyzing the neighborhood gray-scale difference, the obviousness of diagonal features, and the polarization characteristic difference in the local area of each candidate feature point to determine the texture feature of each candidate feature point includes:

[0011] For any candidate feature point, construct a window area centered on the candidate feature point, and determine the first texture obviousness factor of the candidate feature point according to the gray-scale values of each pixel point in the window area of the candidate feature point;

[0012] Obtain the diagonal feature vector of the candidate feature point according to the gray-level co-occurrence matrix of different angles in the window area of the candidate feature point; determine the second texture obviousness factor of the candidate feature point according to the element dispersion situation and element size situation of the diagonal feature vector;

[0013] Combine the first texture obviousness factor and the second texture obviousness factor of the candidate feature point to determine the texture significant index of the candidate feature point; based on the texture significant index and the diagonal feature vector of the candidate feature point, constitute the texture feature of the candidate feature point.

[0014] Further, the determining the first texture obviousness factor of the candidate feature point according to the gray-scale values of each pixel point in the window area of the candidate feature point includes:

[0015] Divide all pixel points in the window area of the candidate feature point into two categories according to the gray-scale values, and calculate the gray-scale average value of each category;

[0016] Take the absolute value of the difference between the gray-scale average values of the two categories as the first texture obviousness factor of the candidate feature point.

[0017] Further, obtaining the diagonal feature vector of the alternative feature point according to the gray-level co-occurrence matrices of different angles within the window area of the alternative feature point includes:

[0018] Set several angles, and based on the gray values of each pixel point within the window area of the alternative feature point, obtain the gray-level co-occurrence matrix of each angle;

[0019] Use the diagonal function to determine the diagonal pixel points in the gray-level co-occurrence matrix of each angle, and then calculate the second norm of the gray values of the diagonal pixel points in the gray-level co-occurrence matrix;

[0020] Perform normalization processing on the second norm to obtain a normalized value, and then use the vector composed of the normalized values obtained for each angle as the diagonal feature vector.

[0021] Further, performing target feature point matching between different polarization images according to the gray values, texture features, and SIFT vectors of each target feature point in the polarization images of each cross-polarization direction, and obtaining the fused images corresponding to all cross-polarization directions includes:

[0022] For any target feature point, according to several numbers of clustering categories, cluster all pixel points within the window area of the target feature point according to the gray values to obtain each clustering window image matrix corresponding to the target feature point;

[0023] According to each clustering window image matrix, texture feature, and SIFT vector corresponding to any target feature point in the polarization images of any two cross-polarization directions, determine the texture feature matching degree between the two target feature points;

[0024] Perform target feature point matching according to the texture feature matching degree to obtain the fused images corresponding to all cross-polarization directions.

[0025] Further, the step of clustering all pixel points within the window area of the target feature point according to the gray values according to several numbers of clustering categories to obtain each clustering window image matrix corresponding to the target feature point includes:

[0026] For any number of clustering categories, cluster all pixel points within the window area of the target feature point according to the gray values to obtain an initial clustering window area; according to the preset marking rule, mark the pixel points belonging to the same clustering cluster in the initial clustering window area with the same category number; according to the category numbers of each pixel point within the initial clustering window area, obtain the clustering window image matrix of this number of clustering categories.

[0027] Further, determining the texture feature matching degree between two target feature points according to each clustering window image matrix, texture feature, and SIFT vector corresponding to any target feature point in the polarization images in any two cross-polarization directions includes:

[0028] For polarization images in any two cross-polarization directions, taking one of them as the template image and the other as the image to be matched, and randomly selecting one target feature point from both the image to be matched and the template image to form a pair of target feature points;

[0029] Determining the first texture feature similarity factor of the pair of target feature points according to the number of each clustering category of the pair of target feature points and the clustering window image matrix of each clustering category number;

[0030] Obtaining the second texture feature similarity factor of the pair of target feature points according to the similarity between diagonal feature vectors in the texture features of the pair of target feature points and the similarity between SIFT vectors;

[0031] Combining the first texture feature similarity factor and the second texture feature similarity factor of the pair of target feature points to determine the texture feature matching degree of the pair of target feature points.

[0032] Further, determining the first texture feature similarity factor of the pair of target feature points according to the number of each clustering category of the pair of target feature points and the clustering window image matrix of each clustering category number includes

[0033] The calculation formula for the first texture feature similarity factor of the pair of target feature points is:

[0034] ; where represents the first texture feature similarity factor of the pair of target feature points, X represents one of the target feature points of the pair of target feature points, Y represents the other target feature point of the pair of target feature points, K represents the number of clustering categories, represents the maximum value of the number of clustering categories, norm represents the linear normalization function, represents the clustering window image matrix when the number of clustering categories of target feature point X is K, represents the clustering window image matrix when the number of clustering categories of target feature point Y is K, represents the two-norm.

[0035] Further, performing target feature point matching according to the texture feature matching degree to obtain a fused image corresponding to all cross-polarization directions includes:

[0036] Determine the texture feature matching degree between any target feature point in the image to be matched and each target feature point in the template image, and use the target feature point corresponding to the maximum texture feature matching degree in the template image as the matching feature point of this target feature point in the image to be matched;

[0037] According to each target feature point in the image to be matched and its matching feature point, perform image fusion on the image to be matched and the template image to obtain a first fused image;

[0038] Use the first fused image as the template image, and perform image fusion processing based on feature point matching according to each target feature point in the template image and a polarization image in any other cross-polarization direction to obtain a second fused image;

[0039] Again, use the second fused image as the template image, and perform image fusion processing based on feature point matching according to each target feature point in the template image and a polarization image in any other cross-polarization direction to obtain a third fused image;

[0040] Iterate continuously until a fused image corresponding to all cross-polarization directions is obtained; wherein, the polarization image in the other cross-polarization direction refers to the polarization image that has not participated in image fusion.

[0041] An embodiment of the present invention also provides a polarized light scanning system for LED virtual shooting of digital assets, including a processor and a memory, and the processor is used to process the instructions stored in the memory to implement a polarized light scanning method for LED virtual shooting of digital assets.

[0042] The present invention has the following beneficial effects:

[0043] The present invention provides a polarized light scanning method and system for LED virtual shooting digital assets. First, a basic data set is obtained. During the process of scanning an object with polarized light, the polarizer needs to be rotated to obtain polarization images without highlights with different cross-polarization directions corresponding to different shooting angles, and then each candidate feature point and its SIFT vector in the polarization image are obtained. The obtained candidate feature points and their SIFT vectors provide data support for subsequent image data analysis, and at the same time facilitate the combination of the polarized light scanning method with a polarizer installed to eliminate highlights and an improved feature point selection and matching algorithm. It not only eliminates the influence of specular reflection, improves the texture information quality of the obtained polarization images, but also is conducive to initially enhancing the accuracy of feature point matching based on high-quality texture information. This combination method can effectively solve the problem of low accuracy of feature point matching caused by highlights and polarization angle changes in the prior art, and greatly improves the construction accuracy of LED virtual shooting digital assets. When rotating the polarizer, the changes in different cross-polarization directions or polarization parameters will affect the presentation effect of the polarization image. Specifically, there are significant feature differences in the pixel points at the same position of the scanned object in the polarization images with different cross-polarization directions, making the SIFT vectors of the feature points at the same position insufficient to clearly describe the key point information in the image. Therefore, it is necessary to analyze the neighborhood gray difference, diagonal feature obviousness, and polarization characteristic difference in the local area based on each candidate feature point to determine the texture features, and then select all target feature points based on the texture features. Determining the target feature points can effectively eliminate the adverse effects brought by the significant SIFT vector features of pixel points, that is, eliminate the interference of feature points with unclear texture details. When performing feature point matching and fusion based on target feature points in the existing feature point matching, the texture differences in the images of the scanned object at different polarization angles are ignored, that is, different polarized lights have differences in reflection and projection at the same position, resulting in certain differences in texture features, manifested as the gray value changes of pixel points in the polarization image. Therefore, it is necessary to correct the matching conditions of feature points, specifically, taking into account the influence of multiple factors during the matching analysis, that is, based on the gray value, texture features, and SIFT vectors of each target feature point in the polarization image of each cross-polarization direction, performing target feature point matching between different polarization images to obtain the fusion images corresponding to all cross-polarization directions; the image quality of the fusion images determined by improving the feature point matching process is higher and the authenticity is stronger, reducing to a certain extent the adverse effects caused by the change of cross-polarization direction or the change of polarization parameters on the generated fusion images, and further improving the construction accuracy of the LED virtual shooting digital assets of the object. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0045] Figure 1 It is a flowchart of a polarization light scanning method for LED virtual shooting digital assets according to an embodiment of the present invention;

[0046] Figure 2 It is a flowchart for implementing step S2 in an embodiment of the present invention;

[0047] Figure 3 It is a flowchart for implementing step S3 in an embodiment of the present invention. Specific Embodiments

[0048] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following will, in conjunction with the accompanying drawings and preferred embodiments, detail the specific embodiments, structures, features, and effects of the technical solutions proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0050] The application scenarios targeted by the present invention can be:

[0051] In the process of constructing LED virtual shooting digital assets by polarizing light to scan an object, the accuracy of the fusion result of polarization images with different cross-polarization directions in the same shooting direction achieved by using the traditional SIFT algorithm is low. As a result, there are deviations in the three-dimensional models constructed based on the fusion images in each shooting direction, that is, the obtained digital assets deviate from the real situation and the accuracy is low.

[0052] To improve the accuracy of the obtained digital assets, this embodiment provides a polarization light scanning method for LED virtual shooting digital assets, as Figure 1 shown, including the following steps:

[0053] S1. During the process of scanning an object with polarized light, first attach polarizers to both the light source and the camera lens, and then adjust the angles of the polarizers to obtain polarization images without highlights with different cross-polarization directions corresponding to different shooting angles, and further obtain each candidate feature point and its SIFT vector in the polarization images.

[0054] The above step S1 can be implemented through steps S11 to S12 (not shown in the figure):

[0055] S11. During the process of scanning an object with polarized light, first attach polarizers to both the light source and the camera lens, and then adjust the angles of the polarizers to obtain polarization images without highlights with different cross-polarization directions corresponding to different shooting angles.

[0056] Here, the polarization image can represent the surface image of the object being photographed; regarding the polarization image without highlights, cross-polarization is used to eliminate reflected light or scattered light, so it can eliminate specular reflection light (i.e., highlights) to the greatest extent, only retaining diffuse reflection light, which makes the obtained image have no highlight area and retains the true texture information of the object surface, thereby improving the clarity of the polarization image; the shooting angles of scanning the object with polarized light and the number and size of the cross-polarization directions of rotating the polarizer can be set by the implementer according to the specific actual situation without specific limitation. Among them, the cross-polarization direction refers to the direction that is 90 degrees to the polarization direction. If different incident polarization directions (such as horizontal, vertical, oblique, etc.) are considered, then there can be multiple different cross-polarization directions, that is, each incident polarization direction will have a corresponding cross-polarization direction, and the size and number of cross-polarization directions can be set by the implementer according to the specific actual situation without specific limitation. It should be noted that when setting the shooting angles, it is necessary to ensure that the entire surface of the object is covered to facilitate the subsequent construction of the LED virtual shooting digital assets of the object, so that the constructed digital assets are complete.

[0057] Specifically, first, when scanning an object with polarized light, first prepare a 500W flash with a polarizer, then mount the flash on the camera, and at the same time attach a polarizer to the camera lens. By adjusting the relative angles of the light source polarizer and the lens polarizer, the light with the cross-polarization direction enters the camera, thereby eliminating the high-light reflection on the object surface; during the shooting and scanning process, rotate the polarizer for each shooting angle to change the polarization angle and obtain polarization images without highlights with different cross-polarization directions corresponding to different shooting angles. Among them, multiple polarization images contain the texture information of the object in different polarization states, which provides rich data to be analyzed for subsequent feature point extraction and matching.

[0058] It should be noted that relevant technical methods of polarization scanning can also be additionally added during the acquisition of polarization images to further improve the obtained polarization scanning effect, such as:

[0059] Dynamic polarization control, specifically introducing an adjustable intelligent polarizer control system that automatically adjusts the polarizer angle according to the material of the object surface and the light conditions, thereby optimizing the polarization light scanning effect. This automated adjustment can reduce human error and improve the efficiency and accuracy of scanning; multi-spectral polarization scanning, specifically combining multi-spectral light sources to increase the wavelength range of polarized light, and using polarized light of different spectra to scan the material properties of the object in more detail, which will enhance the resolution ability for different materials, especially for the reconstruction of complex textures and multi-layered materials; dual-polarization path integration, specifically combining cross-polarization and parallel polarization to simultaneously obtain high-reflection and low-reflection information. This method can more accurately separate the optical properties of the object (such as specular reflection and diffuse reflection) during the scanning process to more comprehensively construct virtual assets; fully automated scanning process, specifically combining polarization light scanning with an automatic rotating stage of the object to form a fully automated multi-angle polarization light scanning system. By presetting different scanning paths and angles, the system can automatically complete the complete scanning of the object without human operation, thereby reducing manual intervention and improving production efficiency; real-time feedback and quality assessment, specifically adding a real-time data feedback and quality assessment module. During the scanning process, the system will dynamically adjust the light source and camera parameters according to the quality and polarization information of each frame of the image to ensure that the quality of the scanning data reaches the best. This can be achieved by calculating indicators such as the clarity and contrast of the polarization image.

[0060] It should be noted that since the subsequent steps require analyzing the light intensity characteristics of the polarization image, the polarization images recorded in this embodiment are images that have been grayscale processed, that is, the grayscale values of each pixel point in the polarization image can be directly obtained. Among them, the implementation process of grayscale processing the polarization image is prior art and not within the protection scope of the present invention, so it will not be elaborated in detail here.

[0061] S12. Based on the polarization images of each cross-polarization direction corresponding to each shooting angle, obtain each candidate feature point and its SIFT vector in the polarization image.

[0062] Here, the feature points obtained by the SIFT detection algorithm are called candidate feature points. The reason is that during the process of constructing virtual digital assets by performing polarization light scanning on the object, it is necessary to change the cross-polarization direction of the polarizer in the polarization camera or adjust the relevant parameters of the polarization camera. This is manifested in the polarization image as the texture features of the pixel points at the same position of the object being significantly different in the polarization images of different cross-polarization directions, resulting in the SIFT vectors of the obtained feature points being insufficient to clearly describe the key point information in the image. Therefore, it is necessary to select some feature points from the feature points obtained by the SIFT detection algorithm for feature point matching analysis. Therefore, the initially obtained feature points are called candidate feature points here.

[0063] Specifically, by using the SIFT (Scale Invariant Feature Transform) detection algorithm to process each polarized image, each feature point in each polarized image and the SIFT vector of each feature point can be obtained, and the obtained feature points are used as candidate feature points. Among them, the implementation process of the SIFT detection algorithm is a prior art and is not within the protection scope of the present invention, and will not be elaborated here.

[0064] It should be noted that by installing a polarizer on the camera and adjusting the angle of the polarizer to eliminate highlights during the object scanning process, eliminating highlights can make the obtained polarized image without highlights have more stable texture features, which can provide more reliable basic data for feature point extraction and matching, facilitate the subsequent acquisition of high-quality fused images, and thus help to build more accurate LED virtual shooting digital assets.

[0065] So far, this embodiment has obtained basic data for subsequent image data analysis, namely, each candidate feature point and its SIFT vector in the polarization image without highlights in each cross-polarization direction corresponding to each shooting angle.

[0066] S2, according to the local area of each candidate feature point, the neighborhood grayscale difference, the degree of diagonal feature prominence and the polarization characteristic difference are analyzed to determine the texture feature of each candidate feature point, and then all target feature points are selected based on the texture feature.

[0067] First of all, it should be noted that in order to exclude as much as possible the feature points that have different features of the same texture in the polarization image due to different cross-polarization directions, that is, to overcome the defect that the texture features of the pixels at the same position in different polarization images are quite different, it is necessary to select the candidate feature points with more significant texture information in the polarization image as the feature points for the final matching analysis, that is, the target feature points. The reason for determining the target feature points is that when the texture feature information of a feature point in the polarization image is relatively small, it means that the light intensity feature at the position of the feature point is very similar to the light intensity feature of the surrounding pixels. When the angle of polarized light changes, the texture information of the feature point in the polarization image will show obvious changes. At this time, whether the texture information is enhanced or weakened, it will lead to an increase in the difference in the SIFT vectors of the feature point at the same object position. Therefore, it is necessary to select feature points with more obvious texture information from all the candidate feature points in the polarization image.

[0068] In this embodiment, the texture features include the diagonal feature vector of the local area of the alternative feature point and the texture significance index. The diagonal feature vector consists of several diagonal feature values, and the diagonal feature value is used to characterize the direction feature of the texture in the local area of the alternative feature point. The texture significance index can be obtained through comprehensive analysis of the diagonal feature value and the gray value of the local area of the alternative feature point. Among them, to determine the texture features of each alternative feature point, target feature points with more significant texture information can be selected from all alternative feature points.

[0069] The above step S2 can be achieved by Figure 2 the steps shown as follows:

[0070] S21. For any alternative feature point, a window area is constructed with the alternative feature point as the center. According to the gray value of each pixel point in the window area of the alternative feature point, the first texture distinctiveness factor of the alternative feature point is determined.

[0071] Here, the first texture distinctiveness factor can characterize the pixel gray value difference situation within the neighborhood range of the alternative feature point. The greater the pixel gray value difference within the neighborhood, the more obvious the texture feature of the alternative feature point. Therefore, the index obtained by quantifying it can be used as one of the elements for determining the texture significance index. The reason for analyzing the first texture distinctiveness factor is that for the position of the feature point with significant texture of the scanned object, since the polarization light characteristic will affect the light intensity around the feature point with significant texture feature, it is manifested as a significant gray value difference around the corresponding position feature point in the polarization image. So, the light intensity feature can be characterized by the size of the gray value. For the feature points in the normal background area, the polarization light characteristic has little influence on the surrounding pixel points, and it is manifested as a high consistency between the gray value of the feature point and its surrounding pixel points. Based on the above description, by determining the first texture distinctiveness factor, the interference of feature points with insufficient texture details can be excluded to a certain extent.

[0072] First, for any alternative feature point, a window area is constructed with the alternative feature point as the center.

[0073] In this embodiment, for the convenience of description and understanding, an alternative feature point is randomly selected from the polarization images in each cross-polarization direction at each shooting angle for subsequent texture feature analysis. Since the analysis of texture features is usually realized based on the image features of pixel points in the local area, for the selected alternative feature point, a window area with a preset size is constructed with the alternative feature point as the center as the local area of the alternative feature point. As an example, the size of the window area can be set to , of course, the implementer can also set the size of the window area according to the specific size of the polarization image, and no limitation is made here.

[0074] Secondly, according to the gray value of each pixel point in the window area of the candidate feature point, the first texture obvious factor of the candidate feature point is determined, and the specific implementation steps may include:

[0075] According to the grayscale value, all the pixels in the window area of the candidate feature point are divided into two categories, and the grayscale average value of each category is calculated; the absolute value of the difference between the grayscale average values of the two categories is used as the first texture obvious factor of the candidate feature point. As an example, the Otsu threshold can be used to classify all the pixels in the window area of the candidate feature point. Among them, the implementation process of the Otsu threshold is a prior art, which is not within the protection scope of the present invention and will not be elaborated here.

[0076] It should be noted that the greater the difference in the average grayscale values of the two categories in the window area of the candidate feature point, the more obvious the texture feature is, indicating that there is an obvious difference in light intensity near the position of the candidate feature point.

[0077] S22, obtaining a diagonal feature vector of the candidate feature point according to the gray level co-occurrence matrix at different angles in the window area of the candidate feature point; and determining a second texture noticeability factor of the candidate feature point according to the discreteness and size of the elements of the diagonal feature vector.

[0078] Here, the second texture obvious factor can characterize the size and discreteness of the diagonal eigenvalues of each angle that constitutes the diagonal eigenvector. The diagonal eigenvalues can reflect the texture direction characteristics of the local area of the candidate feature points, which is specifically manifested as the distribution of pixel pairs with the same grayscale values on the diagonal in the grayscale co-occurrence matrix of the local area. The larger the diagonal eigenvalue, the more pixel pairs with the same grayscale values on the diagonal, and thus the more significant the texture characteristics of the polarization scan in the corresponding diagonal direction.

[0079] The above step S22 can be implemented through steps S221 to S224 (not shown in the figure):

[0080] S221, setting several angles, and obtaining a grayscale co-occurrence matrix for each angle based on the grayscale value of each pixel in the window area of the candidate feature point.

[0081] As an example, first set 4 angles, namely 0, , and Of course, the implementer can also set other numbers of angle sizes according to the actual situation, which is not limited here.

[0082] In this embodiment, the more pixel pairs with the same gray value are represented on the diagonal of the gray-level co-occurrence matrix at a certain angle, the higher the similarity of the light intensity polarized in the direction of this angle. Therefore, the gray value on the diagonal can characterize the directional feature of the texture. The larger the index obtained by quantifying the gray value on the diagonal, the more significant the texture feature of the polarization scan in the diagonal direction. Among them, the process of obtaining the gray-level co-occurrence matrix is a prior art and not within the protection scope of the present invention, so it will not be elaborated in detail here.

[0083] S222. Use the diagonal function to determine the diagonal pixels in the gray-level co-occurrence matrix at each angle, and then calculate the second norm of the gray values of the diagonal pixels in the gray-level co-occurrence matrix.

[0084] S223. Perform normalization processing on the second norm to obtain a normalized value, and then use the vector composed of the normalized values obtained at each angle as the diagonal feature vector.

[0085] In this embodiment, the normalized value obtained at each angle is called the diagonal eigenvalue. As an example, the calculation formula for the diagonal eigenvalue at the h-th angle can be:

[0086] ; in the formula, represents the diagonal eigenvalue at the h-th angle of the alternative feature point, represents the calculation of the second norm, represents the diagonal function, represents the gray-level co-occurrence matrix at the h-th angle of the alternative feature point.

[0087] In the calculation formula of the diagonal eigenvalue, the larger the diagonal eigenvalue , the more significant the texture feature of the window area of the alternative feature point at the h-th angle, and the greater the possibility that the alternative feature point is selected for feature point matching, which can effectively improve the accuracy of feature point matching; can characterize the diagonal texture feature at the h-th angle within the window area, the larger it is, the more significant the diagonal texture feature, the more obvious the texture feature of the alternative feature point, and the denominator of the diagonal eigenvalue represents the texture feature of the entire window area at the h-th angle, which is used to implement the normalization processing of the diagonal eigenvalue, and it can also be replaced by the non-diagonal texture feature at the h-th angle.

[0088] After obtaining the diagonal texture feature values at each angle, all the diagonal texture feature values are formed into a feature vector in ascending order of the angles, which is used to represent the diagonal feature of the window area, denoted as the diagonal feature vector, that is .

[0089] S224. Determine the second texture distinctiveness factor of the candidate feature points according to the element discretization and element magnitude of the diagonal eigenvector.

[0090] As an example, calculate the average value of all elements in the diagonal eigenvector and calculate the standard deviation of all elements in the diagonal eigenvector, and use the product of the average value and the standard deviation as the second texture distinctiveness factor of the candidate feature points. Of course, the average value can be replaced by calculating the cumulative value, and the standard deviation can be replaced by calculating the variance.

[0091] It should be noted that when calculating the second texture distinctiveness factor, two aspects of the texture features of the diagonal eigenvector are analyzed, that is, the diagonal feature significance and the polarization characteristic difference. The polarization characteristic difference refers to the consistency representation of the diagonal feature values in each angular direction within the local area of the candidate feature points.

[0092] It is worth noting that if the diagonal feature of the local area of the candidate feature point is not obvious, that is, most of the pixel points in the local area do not show the texture feature details of polarized light scanning, but reflect the normal background of the scanned object, and there are also relatively many pixel points with the same gray value in the background in the gray-level co-occurrence matrix, which makes the obtained diagonal feature values also relatively large. That is, the texture feature of the local area obtained through the gray-level co-occurrence matrix is the background, rather than the details of the object scanned by polarized light. Therefore, the window corresponding to the pixel points with significant diagonal features is not necessarily the area with rich details in the polarization image.

[0093] The reason for analyzing the polarization characteristic difference is that the detailed texture of the scanned object will cause obvious changes in the polarization characteristics of the reflected light of polarized light in different directions, such as polarization degree, polarization angle, light intensity, etc. Therefore, when the texture details in the local area are richer, there are obvious differences in the polarization characteristics in different directions in the polarization image; on the contrary, in the local area with fewer details, due to the lack of obvious texture features, it appears as a flat area, and the reflected light in different directions is basically the same, so the polarization characteristics of the reflected light are consistent, which is reflected in the high similarity of the polarization characteristics in each direction of the polarization image; and since the light intensity is one of the polarization characteristics in the polarization image, the light intensity in the polarization image is represented by the gray value of the pixel points, and the texture feature of the scanned object represented by the light intensity within the window can be represented by the diagonal feature obtained above. Therefore, the difference in polarization characteristics can be represented by the discreteness between the diagonal feature values in each direction, that is, it is necessary to determine the standard deviation of all elements in the diagonal eigenvector.

[0094] S23. Combine the first texture distinctiveness factor and the second texture distinctiveness factor of the candidate feature points to determine the texture significance index of the candidate feature points; based on the texture significance index of the candidate feature points and the diagonal eigenvector, construct the texture feature of the candidate feature points.

[0095] In this embodiment, the first texture distinctiveness factor, the second texture distinctiveness factor and the texture significance index are all positively correlated. The larger the first texture distinctiveness factor and the second texture distinctiveness factor are, the larger the texture significance index is. In this embodiment, the texture feature of each alternative feature point includes two factors. One is the diagonal feature of the local area of the alternative feature point, and the other is the texture feature significance of the alternative feature point.

[0096] As an example, first calculate the product of the first texture distinctiveness factor and the second texture distinctiveness factor, and then perform normalization processing on the product to obtain a normalized value, and use the normalized value as the texture significance index of the alternative feature point. Among them, the normalization method can be maximum-minimum normalization, and its implementation process is prior art and not within the protection scope of the present invention, so it will not be elaborated in detail here.

[0097] Among them, the calculation formula of the texture significance index of the alternative feature point can be:

[0098] ; in the formula, p represents the initial texture significance index of the alternative feature point, represents the first texture distinctiveness factor of the alternative feature point, represents the average value of all elements in the diagonal feature vector, represents the standard deviation of all elements in the diagonal feature vector, represents the third texture distinctiveness factor of the alternative feature point.

[0099] ; in the formula, represents the texture significance index of the alternative feature point, p represents the initial texture significance index of the alternative feature point, represents the maximum initial texture significance index corresponding to all alternative feature points, represents the minimum initial texture significance index corresponding to all alternative feature points.

[0100] It should be noted that the numerical accuracy of the texture significance index determined from multiple aspects is higher. By determining the texture significance index with higher numerical accuracy, it is helpful to subsequently select more reliable and more detailed feature points for matching processing, thereby improving the image quality of the fused image of each shooting angle obtained by polarized scanning of the object.

[0101] S24. Select all target feature points in the polarization image according to the texture significance index of each alternative feature point in the polarization image of each cross-polarization direction.

[0102] In this embodiment, the larger the texture significance index, the higher the texture significance of the candidate feature points. The higher the similarity of the texture performance of the pixel points corresponding to the candidate feature points at the corresponding positions in the polarization images of different cross-polarization directions at the same shooting angle, the greater the possibility of subsequent feature point matching for the corresponding candidate feature points.

[0103] As an example, arrange the texture significance indices of all candidate feature points in the same polarization image in descending order to obtain a texture significance index sequence; select the candidate feature points corresponding to the top 75% of the texture significance indices in the texture significance index sequence as the target feature points in the corresponding polarization image.

[0104] In another example, set a significance index threshold, and regard all candidate feature points with a texture significance index greater than or equal to the significance index threshold as target feature points. Among them, the significance index threshold can take an empirical value of 0.6.

[0105] Among them, the expression for selecting target feature points can be:

[0106] ; in the formula, M represents the set of target feature points, x represents the candidate feature points, represents the texture significance index of the candidate feature points.

[0107] It should be noted that the target feature points determined by the texture significance index can avoid the situation of only focusing on the SIFT features of the feature points, and can effectively exclude the feature points with unclear texture details, because such feature points are very likely to appear as feature points in the polarization image in a certain cross-polarization direction, but when the cross-polarization direction changes, the corresponding position feature points are very likely to appear as feature points with unclear texture.

[0108] So far, this embodiment has obtained the target feature points in the polarization images of different cross-polarization directions at different shooting angles.

[0109] S3. According to the gray value, texture feature, and SIFT vector of each target feature point in the polarization image of each cross-polarization direction, perform target feature point matching between different polarization images to obtain a fused image corresponding to all cross-polarization directions.

[0110] It should be noted that during the shooting of virtual digital assets, it is necessary to adjust the polarizer to achieve shooting of the scanned object in different cross-polarization directions. Different cross-polarization directions may cause certain changes in texture details, and then cause different features to appear at the same scanning position at different polarization angles. If the feature points extracted at this time are directly used for matching, there is a high possibility of incorrect feature point matching.

[0111] The texture differences in scanning the object images under different cross-polarization directions are mainly reflected in the differences in the reflection or projection of polarized light in different directions on the same position, resulting in certain differences in texture features. Such differences are manifested through the polarization characteristics of polarized light, mainly as the gray-scale changes of pixel points in the polarization image. Moreover, the texture feature differences of the feature points at the same scanning position in the polarization images under different cross-polarization directions are mainly reflected in the significant differences in texture feature saliency. The significant differences in texture features are due to the differences in the polarization states at the same position, resulting in differences in the performance of local light, and further causing different gray-scale differences between the pixel points and the surrounding pixel points in different polarization images, thus generating matching errors.

[0112] The diagonal features obtained in the above steps are to extract the gray-scale pairs with the same gray-scale value in the gray-level co-occurrence matrix. It can only show whether the gray-scale values are the same or different, without considering the gray-scale pairs of pixel points with gray-scale differences. Therefore, when using conventional feature matching methods to perform matching analysis on target feature points, there is still a possibility of inaccurate matching results. For example, the gray-scale pair of a certain position of the scanned object in the polarization image in one cross-polarization direction is (3, 3), while the gray-scale pair in the polarization image in another cross-polarization direction is (5, 5). Therefore, in this embodiment, it is necessary to correct the process of matching target feature points based only on SIFT vectors through the gray-scale values and texture features of the target feature points, so as to obtain high-quality and real fusion images corresponding to all cross-polarization directions at the same shooting angle through accurate matching results.

[0113] The above step S3 can be achieved through Figure 3 the steps shown below:

[0114] S31. For any target feature point, according to several clustering category numbers, cluster all the pixel points in the window area of the target feature point according to the gray-scale values, and obtain each clustering window image matrix corresponding to the target feature point.

[0115] In this embodiment, each target feature point has several corresponding clustering window image matrices. Obtaining the clustering window image matrix is to quantify the gray-scale difference situation between the local area constructed by the pixel points and their surrounding pixel points in different polarization images. Each time all the pixel points in the window area of a pair of target feature points are clustered, a clustering window image matrix corresponding to the target feature point can be obtained. The elements in the clustering window image matrix are the category label serial numbers corresponding to the pixel gray-scales, that is, category numbers. Using category numbers to replace gray-scale values can, to a certain extent, ignore the differences brought by gray-scale values.

[0116] The above step S31 can be achieved through the following steps:

[0117] For any number of clustering categories, all the pixel points within the window area of the target feature point are clustered according to the gray value to obtain an initial clustering window area; according to the preset marking rule, the pixel points belonging to the same clustering cluster in the initial clustering window area are marked with the same category number; according to the category number of each pixel point within the initial clustering window area, a clustering window image matrix with this number of clustering categories is obtained. Among them, the preset marking rule refers to setting the size of the category number of the clustering cluster and the category number marking principle, which can be set by the implementer according to the specific actual situation and is not specifically limited. Moreover, the size and number of the clustering categories can also be set by the implementer according to the specific actual situation and are not specifically limited.

[0118] As an example, the number of clustering categories can be 2, 3, 4, and 5. When the number of clustering categories is 2, all the pixel points within the window area of the target feature point are divided into two categories according to the gray value of each pixel point within the window area of the target feature point, and the average gray values of the two categories are calculated; the category number of the category with the larger average gray value is set to 1, the category number of the category with the smaller average gray value is set to 0, and each pixel point within the window area of the target feature point is marked with the corresponding category number. The clustering window image matrix with the number of clustering categories being 2 can be formed by the category numbers of each pixel point within the window area. Among them, the matrix has the same size as the window area, and the pixel points corresponding to the elements in the matrix have the feature of one-to-one correspondence in position with the pixel points within the window area.

[0119] S32. Determine the texture feature matching degree between two target feature points according to each clustering window image matrix, texture feature, and SIFT vector corresponding to any target feature point in the polarization images with any two cross-polarization directions.

[0120] In this embodiment, the texture feature matching degree is used to represent the similarity degree of the diagonal feature, SIFT vector, and local region gray value between two target feature points located in different polarization images. The numerical accuracy of the texture feature matching degree determined by different factors is higher, which can overcome the adverse effects brought by the rotation of the polarizer to a certain extent, and thus avoid the matching error generated by subsequent target feature point matching; in addition, the larger the texture feature matching degree, the greater the possibility that the two target feature points are matching point pairs. By determining the texture feature matching degree, the corresponding matching feature points can be determined for the target feature points in different polarization images.

[0121] The above step S32 can be implemented through steps S321 to S324 (not shown in the figure):

[0122] S321. For any two polarization images with cross-polarization directions, take one as the template image and the other as the image to be matched, and select an object feature point from each of the image to be matched and the template image to form an object feature point pair.

[0123] In this embodiment, there are multiple polarization images with cross-polarization directions in the same shooting direction, while the objects for object feature point matching can only be two polarization images with cross-polarization directions. Therefore, only two polarization images with cross-polarization directions can be randomly selected first. After performing image fusion of object feature point matching on them, image fusion analysis is then performed based on the fused image and the remaining polarization images. Subsequently, image fusion analysis is performed again based on the newly obtained fused image and the remaining polarization images, and iterative analysis is continuously carried out until all polarization images with cross-polarization directions at the same shooting angle are fused into one image. Among them, the calculation process of the texture feature matching degree of each object feature point pair is the same. To reduce unnecessary descriptions and facilitate understanding, the texture feature matching degree is determined by taking one object feature point pair as an example.

[0124] S322. Determine the first texture feature similarity factor of the object feature point pair according to the number of each clustering category of the object feature point pair and the clustering window image matrix of the number of each clustering category.

[0125] Here, the first texture feature similarity factor can characterize the local gray-level similarity degree between two object feature points in different polarization images, and it is specifically determined by performing data analysis through the number of clustering categories and the clustering window image matrix of each clustering analysis.

[0126] As an example, the calculation formula for the first texture feature similarity factor of the object feature point pair is:

[0127] ; in the formula, represents the first texture feature similarity factor of the object feature point pair, X represents one object feature point of the object feature point pair, Y represents the other object feature point of the object feature point pair, K represents the number of clustering categories, represents the maximum value of the number of clustering categories, norm represents the linear normalization function, represents the clustering window image matrix when the number of clustering categories of object feature point X is K, represents the clustering window image matrix when the number of clustering categories of object feature point Y is K, represents the two-norm.

[0128] In the calculation formula of the first texture feature similarity factor, the larger the number of clustering categories K, the more gray-level categories the clustering window image matrix has, and the higher the clustering accuracy; It represents the difference between two clustering window image matrices in different polarization images when the number of clustering categories is K. The smaller the difference between the two clustering window image matrices, the stronger the neighborhood similarity of the target feature points. If the neighborhood similarity is relatively large under different numbers of clustering categories, it indicates that the texture feature similarity of the target feature point pairs is relatively large, and the local gray-level similarity degree between the two target feature points is relatively high. The first texture feature similarity factor reflects the similarity of the texture or details of the local area of the target feature point pairs at different scales. The larger the first texture feature similarity factor, the higher the consistency of the texture and details of the local area at different scales, and the greater the possibility that the target feature point pairs belong to the same area of the scanned object, thus indicating that the matching degree of the target feature point pairs should be higher.

[0129] S323. According to the similarity between the diagonal feature vectors and the similarity between the SIFT vectors in the texture features of the target feature point pairs, the second texture feature similarity factor of the target feature point pairs is obtained.

[0130] Here, the second texture feature similarity factor is determined from two aspects, namely the similarity between the diagonal feature vectors and the similarity between the SIFT vectors. Currently, when performing feature point matching, only the similarity between the SIFT vectors is used for matching. However, affected by the image features of polarized light scanning, the existing feature point matching results have deviations. Therefore, it is necessary to combine the similarity between the diagonal feature vectors and the first texture feature similarity factor to correct the similarity between the SIFT vectors in order to obtain a matching analysis index that more conforms to the actual situation, that is, to determine the texture feature matching degree. At this time, after determining the first texture feature similarity factor, it is necessary to then determine the second texture feature similarity factor of the target feature point pairs.

[0131] As an example, the calculation formula for the second texture feature similarity factor of the target feature point pairs can be:

[0132] ; where represents the second texture feature similarity factor of the target feature point pairs, represents the diagonal feature vector of the target feature point X, represents the diagonal feature vector of the target feature point Y, represents the SIFT vector of the target feature point X, represents the SIFT vector of the target feature point Y.

[0133] It should be noted that in this embodiment, all elements of the gray-level co-occurrence matrix, the clustering window image matrix, the diagonal eigenvector, and the SIFT vector are not all zero. Therefore, there is no possibility that the values in the denominator positions of the above-mentioned diagonal eigenvalue, the first texture feature similarity factor, and the second texture feature similarity factor are zero.

[0134] S324. Combine the first texture feature similarity factor and the second texture feature similarity factor of the target feature point pair to determine the texture feature matching degree of the target feature point pair.

[0135] In this embodiment, the first texture feature similarity factor, the second texture feature similarity factor, and the texture feature matching degree all show a positive correlation. The larger the first texture feature similarity factor and the second texture feature similarity factor, the more matching the texture features of the target feature point pair. Among them, the greater the texture feature matching degree of the target feature point pair, the greater the possibility that the target feature point pair belongs to the same scanning area position, and the higher the corresponding matching degree.

[0136] As an example, take the product of the first texture feature similarity factor and the second texture feature similarity factor as the texture feature matching degree of the target feature point pair.

[0137] Another example is to take the value after adding the first texture feature similarity factor and the second texture feature similarity factor as the texture feature matching degree of the target feature point pair.

[0138] S33. Perform target feature point matching according to the texture feature matching degree to obtain the fused images corresponding to all cross-polarization directions.

[0139] Here, each shooting angle has its corresponding fused image, and the fused images of different shooting angles can be used to realize the construction of digital assets. The fused image is realized based on feature point matching. Therefore, it is necessary to determine a most matching target feature point in the template image for the target feature point in the image to be matched according to the texture feature matching degree, and then use the image fusion algorithm based on feature point matching to complete the polarization image fusion of different cross-polarization directions. Among them, the implementation process of the image fusion algorithm is the prior art and is not within the protection scope of the present invention, so it will not be elaborated in detail here.

[0140] The above step S33 can be realized through steps S331 to S333 (not shown in the figure):

[0141] S331. Determine the texture feature matching degree between any target feature point in the image to be matched and each target feature point in the template image, and take the target feature point corresponding to the maximum texture feature matching degree in the template image as the matching feature point of this target feature point in the image to be matched.

[0142] In this embodiment, the greater the degree of texture feature matching indicates that the texture features of two target feature points in different polarization images are more matched. Therefore, the target feature point corresponding to the maximum texture feature matching degree in the template image can be used as the matching feature point of the corresponding target feature point in the image to be matched. Among them, determining the matching feature point is to realize the fusion of polarization images with two different cross-polarization directions.

[0143] Specifically, for any target feature point in the image to be matched, referring to the calculation process of the texture feature matching degree of the above-mentioned target feature point pairs, make this target feature point form a target feature point pair with each target feature point in the template image, and calculate the texture feature matching degree; determine the maximum value from all the texture feature matching degrees, and use the target feature point in the template image corresponding to the maximum value as the matching feature point of this target feature point in the image to be matched. Furthermore, perform the above matching process on each target feature point in the image to be matched to determine the matching feature points of each target feature point in the image to be matched.

[0144] Among them, the expression for determining the matching feature point of the target feature point in the image to be matched can be:

[0145] ; In the formula, represents the matching feature point of the target feature point in the image to be matched, X represents the target feature point in the image to be matched, Y represents the target feature point in the template image, represents the texture feature matching degree between the target feature point in the image to be matched and the target feature point in the template image, represents the independent variable that makes a certain function obtain the maximum value.

[0146] S332. According to each target feature point in the image to be matched and its matching feature point, perform image fusion on the image to be matched and the template image to obtain the first fused image.

[0147] In this embodiment, after obtaining each target feature point in the image to be matched and its matching feature point, the image fusion algorithm based on feature point matching can be used to realize the fusion of the image to be matched and the template image, so as to obtain the fused image.

[0148] Specifically, first, based on each target feature point and its matching feature point in the image to be matched, the homography matrix between the target feature point and its matching feature point is calculated by the RANSAC (Random Sample Consensus) algorithm to improve the robustness of the matching. Secondly, the image to be matched is transformed using the homography matrix to align it with the template image. Then, based on the alignment, the image to be matched and the template image are fused together through a simple averaging method or a more complex fusion technique such as the Laplacian pyramid. Finally, the fused image is smoothed to eliminate possible ghosting or unnatural seams.

[0149] As another specific embodiment, according to each target feature point and its matching feature point in the image to be matched, deep learning-enhanced feature matching is used, that is, a deep learning algorithm is used for feature point matching and fusion, replacing or enhancing the traditional SIFT method, which can more accurately separate the optical properties of an object (such as specular reflection and diffuse reflection, etc.) during the scanning process to more comprehensively construct virtual assets. Among them, the implementation process of deep learning enhancement is prior art and not within the scope of protection of the present invention, so it will not be elaborated in detail here.

[0150] S333. Take the first fused image as the template image, and perform image fusion processing based on feature point matching according to each target feature point in the template image and any polarized image in any other cross-polarization direction to obtain a second fused image. Then take the second fused image as the template image, and perform image fusion processing based on feature point matching according to each target feature point in the template image and any polarized image in any other cross-polarization direction to obtain a third fused image. Iterate continuously until a fused image corresponding to all cross-polarization directions is obtained.

[0151] In this embodiment, after fusing the polarized images in two cross-polarization directions together, the fused image is used as a new template image, and any polarized image in any other remaining cross-polarization direction is used as the image to be matched. Referring to the implementation processes of steps S31 to S33 above, image fusion is performed again to obtain a fused image again, which is used as a new template image to perform fusion analysis with any polarized image in any other remaining cross-polarization direction until all polarized images in all cross-polarization directions are fused into one image, and a fused image corresponding to all cross-polarization directions can be obtained. Among them, the polarized images in other cross-polarization directions refer to the polarized images that have not participated in the image fusion.

[0152] So far, this embodiment has obtained the fused images corresponding to all cross-polarization directions at each shooting angle.

[0153] S4. Construct the LED virtual shooting digital asset of the object through the fused images at each shooting angle.

[0154] In this embodiment, a three-dimensional model of the scanned object can be constructed based on the fused images of each shooting angle, and the obtained three-dimensional model is used as the LED virtual shooting digital asset of the object. Of course, the implementer can also directly use the fused images of each shooting angle as the final LED virtual shooting digital asset, and this embodiment does not make specific limitations.

[0155] Specifically, based on the fused images of each shooting angle, a multi-view geometry method is adopted, combined with depth information for three-dimensional reconstruction to obtain a three-dimensional model; in order to improve the authenticity of the three-dimensional model, the generated three-dimensional model is optimized, including texture mapping, lighting adjustment, etc.; finally, the constructed three-dimensional model is imported into the required applications, such as game engines, virtual reality environments, etc.

[0156] So far, in this embodiment, by improving the feature point selection and feature point matching in the process of polarized light scanning technology, LED virtual shooting digital assets with higher authenticity and accuracy are obtained.

[0157] This embodiment also provides a polarized light scanning system based on LED virtual shooting digital assets, including a processor and a memory. The processor is used to process the instructions stored in the memory to implement a polarized light scanning method based on LED virtual shooting digital assets as described above.

[0158] The present invention provides a polarized light scanning method and system based on LED virtual shooting digital assets. First, a basic data set is acquired. During the process of scanning an object with polarized light, the polarizer needs to be rotated to obtain polarization images without highlights with different cross-polarization directions corresponding to different shooting angles, and then each candidate feature point and its SIFT vector in the polarization image are obtained. The obtained candidate feature points and their SIFT vectors provide data support for subsequent image data analysis, and at the same time facilitate the combination of the polarized light scanning method with a polarizer installed to eliminate highlights and an improved feature point selection and matching algorithm. It not only eliminates the influence of specular reflection, improves the quality of the texture information of the obtained polarization image, but also helps to initially enhance the accuracy of feature point matching based on high-quality texture information. This combination method can effectively solve the problem of low accuracy of feature point matching caused by highlights and polarization angle changes in the prior art, and greatly improves the construction accuracy of LED virtual shooting digital assets. When rotating the polarizer, the changes in different cross-polarization directions or polarization parameters will affect the presentation effect of the polarization image. Specifically, there are significant feature differences in the pixel points at the same position of the scanned object in the polarization images with different cross-polarization directions, so that the SIFT vectors of the feature points at the same position are not sufficient to clearly describe the key point information in the image. Therefore, it is necessary to analyze the neighborhood gray difference, the obviousness of diagonal features, and the polarization characteristic difference in the local area of each candidate feature point to determine the texture features, and then select all target feature points based on the texture features. Determining the target feature points can effectively eliminate the adverse effects brought by the significant SIFT vector features of pixel points, that is, eliminate the interference of feature points with unclear texture details. When performing feature point matching fusion based on target feature points in the existing feature point matching, the texture differences in the images of the scanned object at different polarization angles are ignored, that is, different polarized lights have differences in reflection and projection at the same position, and thus there are certain differences in texture features, manifested as the gray value change of pixel points in the polarization image. Therefore, it is necessary to correct the matching conditions of feature points, specifically, to take into account the influence of multiple factors during the matching analysis, that is, to perform target feature point matching between different polarization images according to the gray value, texture features, and SIFT vectors of each target feature point in the polarization image of each cross-polarization direction to obtain a fused image corresponding to all cross-polarization directions; the image quality of the fused image determined by improving the feature point matching is higher and the authenticity is stronger, which reduces the adverse effects caused by the change of cross-polarization direction or the change of polarization parameters on the generated fused image to a certain extent, and further improves the construction accuracy of the LED virtual shooting digital assets of the object.

[0159] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A polarized light scanning method for LED virtual shooting digital assets, characterized in that, Including the following steps: During the process of scanning an object with polarized light, first install polarizers on both the light source and the camera lens, and then adjust the angles of the polarizers to obtain polarization images without highlights with different cross-polarization directions corresponding to different shooting angles, and further obtain each candidate feature point and its SIFT vector in the polarization image; Analyze the neighborhood gray difference, the obviousness of the diagonal feature, and the polarization characteristic difference in the local area of each candidate feature point to determine the texture feature of each candidate feature point, and further select all target feature points based on the texture feature; the polarization characteristic difference is represented by the discreteness between the diagonal feature values in each direction; Match the target feature points between different polarization images according to the gray value, texture feature, and SIFT vector of each target feature point in the polarization image of each cross-polarization direction to obtain the fused images corresponding to all cross-polarization directions; Construct the LED virtual shooting digital asset of the object through the fused images of each shooting angle; The analyzing the neighborhood gray difference, the obviousness of the diagonal feature, and the polarization characteristic difference in the local area of each candidate feature point to determine the texture feature of each candidate feature point includes: For any candidate feature point, construct a window area centered on the candidate feature point, and determine the first texture obviousness factor of the candidate feature point according to the gray value of each pixel point in the window area of the candidate feature point; Obtain the diagonal feature vector of the candidate feature point according to the gray level co-occurrence matrix of different angles in the window area of the candidate feature point; determine the second texture obviousness factor of the candidate feature point according to the element discreteness and element size of the diagonal feature vector; Combine the first texture obviousness factor and the second texture obviousness factor of the candidate feature point to determine the texture significant index of the candidate feature point; based on the texture significant index and the diagonal feature vector of the candidate feature point, constitute the texture feature of the candidate feature point; The matching the target feature points between different polarization images according to the gray value, texture feature, and SIFT vector of each target feature point in the polarization image of each cross-polarization direction to obtain the fused images corresponding to all cross-polarization directions includes: For any target feature point, cluster all pixel points in the window area of the target feature point according to the gray value according to a number of clustering category numbers to obtain each clustering window image matrix corresponding to the target feature point; Determine the texture feature matching degree between two target feature points according to each clustering window image matrix, texture feature, and SIFT vector corresponding to any target feature point in the polarization images of any two cross-polarization directions; Match the target feature points according to the texture feature matching degree to obtain the fused images corresponding to all cross-polarization directions.

2. The polarization light scanning method for LED virtual shooting digital assets according to claim 1, characterized in that The determining the first texture obviousness factor of the candidate feature point according to the gray value of each pixel point in the window area of the candidate feature point includes: Divide all pixel points in the window area of the candidate feature point into two categories according to the gray value, and calculate the gray average value of each category; Take the absolute value of the difference between the gray average values of the two categories as the first texture obviousness factor of the candidate feature point.

3. A polarized light scanning method for LED virtual shooting digital assets according to claim 1, characterized in that, Obtaining a diagonal feature vector of an alternative feature point according to gray-level co-occurrence matrices at different angles within a window region of the alternative feature point, including: Setting a plurality of angles, and obtaining gray-level co-occurrence matrices at each angle based on the gray-level values of each pixel point within the window region of the alternative feature point; Using a diagonal function to determine diagonal pixel points in the gray-level co-occurrence matrix at each angle, and further calculating the two-norm of the gray-level values of the diagonal pixel points in the gray-level co-occurrence matrix; Performing normalization processing on the two-norm to obtain a normalized value, and further using a vector composed of the normalized values obtained at each angle as the diagonal feature vector.

4. A polarized light scanning method for LED virtual shooting digital assets according to claim 1, characterized in that, Clustering all pixel points within the window region of a target feature point according to gray-level values according to a plurality of clustering category numbers, to obtain respective clustering window image matrices corresponding to the target feature point, including: For any one clustering category number, clustering all pixel points within the window region of the target feature point according to gray-level values to obtain an initial clustering window region; according to a preset marking rule, marking pixel points belonging to the same clustering cluster in the initial clustering window region with the same category number; and obtaining a clustering window image matrix of this clustering category number according to the category numbers of each pixel point within the initial clustering window region.

5. A polarized light scanning method for LED virtual shooting digital assets according to claim 4, characterized in that, Determining the texture feature matching degree between two target feature points according to respective clustering window image matrices, texture features, and SIFT vectors corresponding to any one target feature point in polarization images in any two cross-polarization directions, including: For polarization images in any two cross-polarization directions, taking one of them as a template image and the other as an image to be matched, and randomly selecting one target feature point from the image to be matched and the template image to form a target feature point pair; Determining a first texture feature similarity factor of the target feature point pair according to the respective clustering category numbers of the target feature point pair and the clustering window image matrices of the respective clustering category numbers; Obtaining a second texture feature similarity factor of the target feature point pair according to the similarity between diagonal feature vectors in the texture features of the target feature point pair and the similarity between SIFT vectors; Combining the first texture feature similarity factor and the second texture feature similarity factor of the target feature point pair to determine the texture feature matching degree of the target feature point pair.

6. A polarized light scanning method for LED virtual shooting digital assets according to claim 5, characterized in that Determining a first texture feature similarity factor of the target feature point pair according to the respective clustering category numbers of the target feature point pair and the clustering window image matrices of the respective clustering category numbers, including The calculation formula for the first texture feature similarity factor of the target feature point pair is: ; wherein, represents the first texture feature similarity factor of the target feature point pair, X represents one of the target feature points of the target feature point pair, Y represents the other target feature point of the target feature point pair, K represents the number of clustering categories, represents the maximum value of the number of clustering categories, norm represents the linear normalization function, represents the clustering window image matrix when the number of clustering categories of the target feature point X is K, represents the clustering window image matrix when the number of clustering categories of the target feature point Y is K, represents the two-norm.

7. A polarized light scanning method for LED virtual shooting digital assets according to claim 5, characterized in that, Performing target feature point matching according to the texture feature matching degree to obtain a fused image corresponding to all cross-polarization directions, including: Determining the texture feature matching degree between any one target feature point in the image to be matched and each target feature point in the template image, and taking the target feature point corresponding to the maximum texture feature matching degree in the template image as the matching feature point of this target feature point in the image to be matched; Performing image fusion on the image to be matched and the template image according to each target feature point in the image to be matched and its matching feature point to obtain a first fused image; Take the first fused image as the template image, and perform image fusion processing based on feature point matching according to each target feature point in the template image and the polarization image in any other cross-polarization direction to obtain the second fused image; Again, take the second fused image as the template image, and perform image fusion processing based on feature point matching according to each target feature point in the template image and the polarization image in any other cross-polarization direction to obtain the third fused image; Iterate continuously until a fused image corresponding to all cross-polarization directions is obtained; wherein, the polarization image in the other cross-polarization direction refers to the polarization image that has not participated in image fusion.

8. A polarized light scanning system based on LED virtual shooting digital assets, characterized in that, It includes a processor and a memory, and the processor is used to process the instructions stored in the memory to implement a polarization light scanning method for LED virtual shooting digital assets as described in any one of claims 1-7.

Images