A method and system for site three-dimensional image acquisition and immersive generation

Abstract

The application discloses a kind of for site three-dimensional image acquisition and immersive generation method and system, it is related to the image acquisition and modeling technical field of site building, the three-dimensional image acquisition method for site includes: obtaining the three-dimensional point cloud data generated after three-dimensional scanner scans site building;While obtaining the three-dimensional point cloud data, the reflection spectrum data of the surface of the remains building under the illumination of multiple different wavelengths is obtained at each scanning point;The reflection spectrum data of each scanning point is compared and processed, at least one low reflectivity point is extracted, and the reflection spectrum similarity of each low reflectivity point and each scanning point in adjacent area is calculated.Combination of point cloud and spectral data not only presents the three-dimensional structure and color distribution of building, but also can identify the area of pattern loss or fading by analyzing reflectivity, realize the comprehensive diagnosis and display of site building.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of site image acquisition and modeling generation, and particularly relates to a method and system for three-dimensional image acquisition and immersive generation of a site. BACKGROUND

[0002] In the process of three-dimensional space data scanning and recording of a site building, a scanning device such as a three-dimensional laser scanner is usually used to scan and record the geometric spatial structure of the site building. If image information on the site needs to be acquired, such as the acquisition of mural patterns on the site and the combination of the mural patterns with the constructed three-dimensional model, the existing technology is mostly to scan the mural on the site building by region through a camera, and then generate a complete mural by using image stitching technology and paste it on the three-dimensional model.

[0003] The absence of mural patterns is not only manifested as color changes on the image, but more often due to environmental factors such as weathering, dampness, etc. The mural pigments age and appear pattern peeling. Taking the Guge Dynasty site in Tibet as an example, the buildings of the Guge Dynasty mostly use local sandstone and shale, and the pigments used in the murals of the buildings mostly use cinnabar, lapis lazuli and ocher. These rock materials are prone to surface loosening and peeling under the long-term weathering effect, which in turn affects the overall peeling of the mural pigments attached thereto. Tibet is located in a plateau environment, and the ultraviolet radiation intensity is high. Cinnabar and other mineral pigments will gradually fade and decompose under ultraviolet radiation, accelerating the aging process of the pigments and causing the absence of mural patterns.

[0004] The existing three-dimensional scanning technology can only provide geometric information of the building surface and is difficult to identify pattern absence areas caused by material degradation. SUMMARY

[0005] I) Technical problems to be solved

[0006] The present application provides a method and system for three-dimensional image acquisition and immersive generation of a site to solve the problem that the existing scanning technology can only provide surface geometric information and is difficult to identify pattern absence areas caused by material degradation.

[0007] II) Technical solutions

[0008] To achieve the above object, the present application provides the following technical solutions: a method for three-dimensional image acquisition of a site, comprising:

[0009] acquiring three-dimensional point cloud data generated after scanning a site building by a three-dimensional scanner;

[0010] acquiring reflectance spectrum data of the surface of the site building under illumination of multiple different wavelengths at each scanning point while acquiring the three-dimensional point cloud data.

[0011] The reflection spectrum data of each of the scanning points is compared and processed to extract at least one low reflectivity point, and the reflection spectrum similarity of each of the low reflectivity points and each scanning point in the adjacent area is calculated; wherein the low reflectivity point represents that the reflectivity of a certain scanning point is lower than the reflectivity of other scanning points in the corresponding color light wave band; the calculation of the reflection spectrum similarity is specifically comparing the reflectivity information of the low reflectivity point and each scanning point in its neighborhood in different color wave bands, and calculating the reflection spectrum similarity; and

[0012] The low reflectivity point and each scanning point in its neighborhood with high reflection spectrum similarity together constitute a complete area, and the area is labeled in the three-dimensional point cloud data.

[0013] Further, the three-dimensional point cloud data generated after the site building is scanned by the three-dimensional scanner includes:

[0014] At each scanning position, a Cartesian coordinate system is constructed with the position of the three-dimensional scanner as the origin; wherein the three axes in the coordinate system represent the horizontal direction, the vertical direction and the distance information of the scanned target to the three-dimensional scanner in the scanning range respectively;

[0015] The three-dimensional scanner calculates the distance of each laser reflection point on the site surface to the three-dimensional scanner by emitting pulsed laser and calculating the time of flight of the laser reflected from the building surface;

[0016] The distance data of each laser reflection point on the site surface is combined with the corresponding horizontal angle and vertical angle to calculate the three-dimensional coordinates of the point in the Cartesian coordinate system, each of the three-dimensional coordinate points is recorded as independent three-dimensional point cloud data, and the point cloud model of the entire site surface is constructed together.

[0017] Further, while the three-dimensional scanning is being performed, a color image of the current site surface is captured by a camera, and for each scanning point in the three-dimensional point cloud data, at least one pixel value in the color image is matched; specifically,

[0018] In the three-dimensional coordinate system, each three-dimensional point corresponds to at least one two-dimensional pixel coordinate;

[0019] The two-dimensional pixel coordinates are converted into the coordinates of the three-dimensional points by setting the internal parameters of the camera;

[0020] For each of the three-dimensional points, the corresponding image pixel is found, the color value of the pixel is obtained and assigned to the three-dimensional point to form three-dimensional point cloud data with color information.

[0021] Further, while the three-dimensional scanning is being performed, a plurality of different wavelengths of light are selected to illuminate the surface of the site building, and reflectance spectrum data at different wavelengths is acquired at each scanning point, the reflectance spectrum data containing reflectance information within different color bands.

[0022] Further, after comparative analysis of the reflectance spectrum data of each scanning point, a plurality of low reflectance points P i are marked i ; i , y i , z i ) in the three-dimensional coordinate system;

[0023] For each marked low reflectance point P i , a neighborhood with the low reflectance point P i as the center point is defined, and the reflectance information of each scanning point P j in the neighborhood is extracted;

[0024] For each scanning point, the reflectance spectrum data collected forms a spectrum vector X, the spectrum vector X containing reflectance within a plurality of color bands:

[0025]

[0026] wherein, represents the reflectance of the scanning point at wavelength λ i ;

[0027] The spectrum vector i of the low reflectance point P j and the spectrum vector of a scanning point P low in the neighborhood are calculated, and the cosine similarity cosθ between the two spectrum vectors is calculated:

[0028]

[0029] wherein, X·X low represents the inner product of two vectors:

[0030]

[0031] ||X|| and ||X j || are the norms (i.e. vector lengths) of the two vectors, respectively.

[0032] The value of cosθ is close to 1, indicating that the spectrum vectors are highly similar, and that the reflectance information within different color bands of the scanning point P i in the neighborhood and the low reflectance point P j are highly similar.

[0033] Further, if there is a scanning point P j with low reflectivity in the neighborhood of the center point P i with similar reflectivity information, the scanning point P j is taken as a new center point, and the expansion continues.

[0034] The neighborhood is redefined and the reflectance spectrum similarity calculation is performed for the new center point until the cosine similarity cosθ between the new center point and all scanning points in the neighborhood is lower than the set cosine similarity threshold, and the expansion stops.

[0035] After the expansion stops, all low reflectivity points P i in the expanded area and scanning points P j with similar reflectivity information in their neighborhoods are collectively formed into a complete area, which is defined as a pattern color missing area of the site surface, and the pattern color missing area is marked in the three-dimensional point cloud data.

[0036] A method for immersive generation of three-dimensional images of a site, comprising:

[0037] After point cloud denoising, point cloud alignment and point cloud simplification are performed on the three-dimensional point cloud data with image color information obtained after scanning the site building, the simplified point cloud data is converted into a triangular mesh, i.e., a surface structure is generated through the adjacency relationship between points.

[0038] The image color information in the three-dimensional point cloud is mapped to the generated triangular mesh.

[0039] The reflectance spectrum data of the site building surface reflecting multiple different wavelengths of light obtained at each scanning point is combined with the surface properties of each triangular mesh patch, and the area formed by the low reflectivity points in the reflectance spectrum and the scanning points with similar reflectivity information in their neighborhoods is marked in the triangular mesh.

[0040] A system for collecting three-dimensional images of a site, comprising:

[0041] The data acquisition end is configured to:

[0042] obtain three-dimensional point cloud data generated after scanning the site building with a three-dimensional scanner;

[0043] obtain three-dimensional point cloud data generated after scanning the site building with a three-dimensional scanner;

[0044] acquire reflectance spectrum data of the site building surface under multiple different wavelengths of light at each scanning point while acquiring the three-dimensional point cloud data;

[0045] The reflection spectrum data of each scanning point is compared and processed to extract at least one low reflectivity point, and the reflection spectrum similarity of each low reflectivity point and each scanning point in the adjacent area is calculated.

[0046] The low reflectivity point and each scanning point with high reflection spectrum similarity in the adjacent area are collectively formed into a complete area, and the area is marked in the three-dimensional point cloud data.

[0047] A three-dimensional image immersive generation system for a site includes:

[0048] The data generation end is configured to:

[0049] After the three-dimensional point cloud data with image color information obtained after scanning the site building is subjected to point cloud noise reduction, point cloud alignment and point cloud simplification, the simplified point cloud data is converted into a triangular mesh, i.e., a surface structure is generated through the adjacency relationship between points;

[0050] The image color information in the three-dimensional point cloud is mapped to the generated triangular mesh;

[0051] The reflection spectrum data of the site building surface reflection of multiple different wavelengths of light acquired at each scanning point is combined with the surface properties of each triangular mesh patch, and the area formed by the low reflectivity point in the reflection spectrum and the scanning points with similar reflectivity information in the adjacent area is marked in the triangular mesh.

[0052] III) Advantages:

[0053] Compared with the prior art, the present application has the following advantages:

[0054] The present application generates a site building three-dimensional model with geometric, color and spectral information by fusing three-dimensional point cloud, color image and spectral data. Based on the principle that mineral pigments (such as cinnabar, lapis lazuli and ocher) have specific spectral reflection characteristics in normal state, and the reflectivity of missing or faded areas will significantly decrease in multiple color bands, the area with significant spectral difference is found and marked as a pattern missing or pigment degradation area. Through the combination of three-dimensional point cloud and spectral data, not only the three-dimensional structure and color distribution of the building are presented, but also the pattern missing or faded area is identified through the analysis of reflectivity, realizing the comprehensive diagnosis and display of the site building. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 A flowchart of a three-dimensional image acquisition method for a site provided by an embodiment of the present application;

[0056] Figure 2A flowchart of a method for immersive generation of three-dimensional images of archaeological sites provided in an embodiment of the present invention;

[0057] Figure 3 This invention provides a method for acquiring three-dimensional images of archaeological sites by scanning and capturing a portion of the two-dimensional color image generated from the surface of the archaeological site.

[0058] Figure 4 The embodiments of the present invention provide a pair Figure 3 The image at point A shown identifies a low reflectivity point, and after comparing its spectral characteristics with those of the scan points in its neighborhood, the area with missing pattern color is marked.

[0059] Figure 5 A comparison diagram of the reflectance spectra of normal and severely degraded areas of cinnabar pigment under the same illumination conditions and reflection distance, provided in an embodiment of the present invention.

[0060] Figure 6 A comparison of the reflectance spectra of normal and severely degraded regions of lapis lazuli pigment under the same illumination conditions and reflection distance, provided in an embodiment of the present invention.

[0061] Figure 7 This is a comparison of the reflectance spectra of normal and severely degraded areas of ochre pigment under the same illumination conditions and reflection distance, provided in an embodiment of the present invention.

[0062] Figure 8 The present invention provides a method and system for immersive generation of three-dimensional images of archaeological sites, which generates a triangular mesh model based on three-dimensional point cloud data. Detailed Implementation

[0063] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0064] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0065] In the process of scanning and recording the three-dimensional spatial data of the site building, the geometric spatial structure of the site building is usually scanned and recorded by means of a scanning device such as a three-dimensional laser scanner. If image information on the site needs to be collected, for example, mural patterns on the site need to be collected and combined in the constructed three-dimensional model, the existing technology mostly scans the entire mural by a camera in different regions, and then generates a complete mural by using image stitching technology and pastes it on the three-dimensional model.

[0066] The absence of mural patterns is not only the absence of color on the image, but more often the absence of mural patterns caused by the degradation of painting materials. Due to environmental factors such as weathering and humidity, mural pigments may peel off or age. The existing three-dimensional scanning technology can only provide surface geometric information and is difficult to identify the pattern missing area caused by material degradation.

[0067] Taking the ancient Guge Dynasty site in Tibet as an example, the buildings of the Guge Dynasty mostly use local sandstone and shale, and the pigments used in the murals of the buildings mostly use cinnabar, lapis lazuli and ocher. The rock materials used in the buildings are prone to surface loosening and peeling under the long-term weathering effect, which in turn affects the mural pigments attached thereto. The pore structure of sandstone and shale makes them more easily absorb moisture, and when the temperature changes or is exposed to the wind, the evaporation of water causes the stone to expand and contract, thereby accelerating the peeling of the mural surface pigments.

[0068] In addition, cinnabar is used as a red pigment, and its main component is mercury sulfide. In a humid environment, cinnabar is prone to oxidation, gradually changing color and even powdering, causing the red area of the mural to fade or peel off. Lapis lazuli is used as a blue pigment, and its main component is a composite of oligoclase and other minerals. Lapis lazuli is sensitive to humidity, and long-term exposure to moisture can cause it to decompose, fade in color, and lose patterns as the substrate peels off. Ocher is used as a brown or reddish-brown pigment. Ocher, as a natural iron oxide pigment, will undergo a chemical reaction under humid and acidic conditions, causing the color of the pigment to fade or peel off. Tibet is located in a highland environment with high radiation intensity, and cinnabar and other mineral pigments will gradually fade or decompose under ultraviolet radiation, which will accelerate the aging process of the pigments and cause the loss of patterns. In the high-sand area of the Guge Dynasty site, the mural surface has been eroded by fine particles for a long time, causing the pigments to gradually peel off.

[0069] The method and system for three-dimensional image acquisition and immersive generation provided by the embodiment of the present application can identify the areas where the pigments on the surface have peeled off or aged by using multi-reflectance spectral data, and can mark the areas where the patterns and colors are missing in the subsequent data generation and transformation model process, so as to provide a reference for repair and immersive display.

[0070] Specifically, referring to Figure 1 , Figure 1 A flowchart of a method for three-dimensional image acquisition of a historical site is provided in embodiments of the present application, and specifically:

[0071] Step 101: Obtain three-dimensional point cloud data generated after scanning the historical site building with a three-dimensional scanner. In some embodiments of the present application, the historical site building is scanned by a laser scanner, and the distance from each laser reflection point on the building surface to the scanner is obtained by measuring the time of flight of the laser from the scanner to the surface reflection return, to generate three-dimensional point cloud data. A spatial coordinate system is constructed, and the three-dimensional coordinates (x, y, z) of each laser reflection point record the position of the point in the scanning area, forming a point cloud model in three-dimensional space, accurately reflecting the geometric structure of the building.

[0072] Specifically, at each scanning position, a Cartesian coordinate system is established with the position of the three-dimensional scanner as the coordinate origin. The three axes in the coordinate system are used to represent the horizontal direction, vertical direction of the scanner in the scanning range, and the distance (depth information) from the three-dimensional scanner to the reflection point on the target historical site surface. Among them:

[0073] X-axis: horizontal scanning direction;

[0074] Y-axis: vertical scanning direction;

[0075] Z-axis: depth information from the laser scanner to the scanned target.

[0076] The three-dimensional laser scanner emits a pulsed laser beam to the surface of the historical site building, then receives the laser reflected from the surface, and determines the distance from each reflection point to the scanner by calculating the time of flight of the laser from emission to return.

[0077] At the same time, the laser scanner records the horizontal angle and vertical angle at the time of laser emission, which respectively represent the orientation of the reflection point relative to the scanner. By using the horizontal angle, vertical angle and measured distance information, the spherical coordinate is converted to Cartesian coordinate. The three-dimensional coordinates (x, y, z) of each point can be converted by the following formula:

[0078]

[0079] z=D cosθ

[0080] where D is the distance, θ is the vertical angle, is the horizontal angle.

[0081] The three-dimensional coordinates of each reflection point are recorded as an independent three-dimensional point. As the three-dimensional scanner scans different positions of the site building multiple times, these three-dimensional points gradually accumulate, and eventually a complete point cloud model of the entire site surface is constructed.

[0082] In some embodiments of the present application, while using the laser scanner, a RGB camera is used to take color images of the site surface, obtaining the color information (RGB value) of each pixel point.

[0083] Each pixel value of the color image is matched with the corresponding point in the three-dimensional point cloud data. Through the internal and external parameter correction of the camera, the two-dimensional pixel coordinates (u, v) in the color image are converted into points in the three-dimensional space, and each three-dimensional point is assigned its corresponding color information.

[0084] Specifically, each laser reflection point (that is, a point in the three-dimensional point cloud) has corresponding three-dimensional coordinates (x, y, z) in the Cartesian coordinate system. Because the image captured by the camera is two-dimensional, each pixel point in the image also has its corresponding two-dimensional pixel coordinates (u, v).

[0085] More specifically, through the internal and external parameters of the camera, the two-dimensional pixel coordinates corresponding to each three-dimensional point are determined. The internal parameters describe the optical characteristics of the camera, mainly including focal length, principal point position, and radial distortion coefficient, etc. The external parameters are used to describe the position and orientation of the camera in the three-dimensional space, including the rotation matrix and the displacement vector, which are used to convert the points in the three-dimensional space into two-dimensional pixels in the camera's view.

[0086] In some embodiments of the present application, through the use of the internal and external parameters of the camera, the three-dimensional coordinates (x, y, z) can be mapped to the two-dimensional pixel coordinates (u, v) of the camera image plane.

[0087]

[0088] Where K is the internal parameter matrix of the camera, R is the rotation matrix, and t is the displacement vector.

[0089] After mapping the three-dimensional points to the two-dimensional image pixel coordinates, the color value (such as the RGB value) of the pixel is obtained.

[0090] The obtained color value is assigned to the three-dimensional point. In this way, each three-dimensional point not only carries its spatial coordinates, but also carries the color information of the point.

[0091] In summary, each point in the three-dimensional point cloud data is assigned corresponding color information to form a three-dimensional point cloud model with color. The three-dimensional point cloud data generated by this method not only accurately describes the geometric shape of the site building, but also retains the color details on the surface, achieving a more realistic and detailed digital expression.

[0092] Step 102: While acquiring the three-dimensional point cloud data, acquire the reflection spectrum data of the site building surface under multiple different wavelength illuminations at each scanning point. In some implementable embodiments of the present application, by setting a spectral camera, the reflection spectrum data of each scanning point under different wavelength illuminations is acquired synchronously. The spectral camera measures each pixel in multiple wavebands (such as ultraviolet, visible light, infrared) and records its reflectivity at different wavelengths. These spectral data can help distinguish different areas of materials, especially when color changes or material degradation are difficult to identify with the naked eye, spectral analysis can provide more details.

[0093] It should be noted that, in order to avoid the interference of different illuminations on the work of the RGB camera and the spectral camera, in some embodiments of the present application, a uniform white light LED light source is used to avoid the influence of the flash on the spectral camera, and a standard red, green and blue filter is used in front of the RGB camera to separate color information. A synchronous triggering mechanism is used to ensure that the three-dimensional laser scanner, the RGB camera and the spectral camera can collect data at the same time point, that is, during the scanning process, the three-dimensional laser scanner generates point cloud data, while the RGB camera and the spectral camera capture relevant color images and spectral data respectively.

[0094] In the embodiment of the ancient Guge Dynasty site in Tibet, the following are different painting pigments and related reflection characteristics:

[0095] Cinnabar: The main component is mercury sulfide (such as HgS).

[0096] In the visible spectrum, cinnabar usually has high reflectivity in the red light band (about 620-750 nm), and low reflectivity in the blue-green light band.

[0097] With reference to Figure 5 , Figure 5 The reflection spectrum comparison chart of the normal area and the severely degraded area of the cinnabar pigment under the same illumination conditions and reflection distances provided by the embodiment of the present application.

[0098] In the normal state, the reflection spectrum of cinnabar (mercury sulfide, HgS) shows strong reflection in the red light band of visible light (about 620-750 nm); there is obvious absorption in the green band (about 500-550 nm) and the blue band (about 450-490 nm).

[0099] The Guge Dynasty site in Tibet is located in a highland area with high intensity of ultraviolet radiation and high frequency of natural disasters such as wind and sand. Sulfurized mercury (vermillion) will gradually transform into oxidized mercury under the action of ultraviolet rays and oxygen. This change alters the chemical structure of vermillion, and the reflection characteristics of oxidized mercury are different from those of sulfurized mercury. The reflection ability of oxidized mercury in the red light band of visible light decreases significantly.

[0100] Moreover, due to the influence of wind and sand weather, the pigment particles will decompose or become finer. The size of the pigment particles directly affects their spectral reflection characteristics. Larger particles can more effectively reflect light in a specific wave band, while when the vermillion pigment particles become finer, the reflectivity in the red light band decreases, which means that the intensity of the reflection peak decreases.

[0101] It can be understood that the reflection peak of vermillion pigment in a region with a large degree of degradation decreases, especially the reflectivity in the red light band decreases. These changes destroy the original spectral characteristics of vermillion, resulting in a decrease in the reflection ability in the red light band. Therefore, by detecting and comparing the regions with a significant decrease in reflectivity in the red light band, it can be determined that these regions are the faded and missing regions of the pattern color.

[0102] Lapis lazuli (blue): mainly composed of a complex of feldspar and other minerals.

[0103] Lapis lazuli has high reflectivity in the blue and violet light bands (about 450-490 nm) and has a distinct reflection peak in the reflection spectrum, usually showing strong blue color.

[0104] For specific reference Figure 6 , Figure 6 The reflection spectrum comparison chart of normal and severely degraded regions of lapis lazuli pigment under the same illumination conditions and reflection distance is provided by the embodiments of the present application.

[0105] The spectral reflection characteristics of lapis lazuli in a normal state show strong reflection in the blue band (about 450-490 nm), while in the red band (about 620-750 nm) and the green band (about 500-550 nm), there is obvious absorption. This is determined by the crystal structure of the main component lazurite in lapis lazuli, especially the absorption and reflection characteristics of the sulfide ions inside.

[0106] However, in the pattern color missing or degraded region, the blue reflection peak decreases and the reflectivity decreases. This phenomenon is mainly due to the fact that ultraviolet rays can destroy the chemical bonds of the pigment. Under the influence of ultraviolet rays, the colored components in lapis lazuli will undergo photodegradation, which weakens its reflection performance in the blue light band. As the photodegradation progresses, the reflection peak in the blue band decreases, and due to the destruction of the surface chemical structure by photodegradation, the reflection characteristics also change and the reflectivity decreases.

[0107] It can be understood that in the pattern color missing or degradation area, the degradation of the lapis lazuli pigment can cause the reflection peak of the blue band to decrease, and the reflectivity to decrease significantly, so by detecting and comparing the area where the reflectivity in the blue band decreases significantly, it can be determined that it is the area where the pattern color fades and is missing.

[0108] Ochre (red-brown): the main component is natural iron oxide mineral.

[0109] The reflectivity in the red and green light bands (about 580-640 nm) is relatively high, especially in the red band area.

[0110] Specific reference Figure 7 , Figure 7 The reflection spectrum comparison chart of the normal area and the severely degraded area of the ochre pigment under the same illumination conditions and reflection distance provided by the embodiment of the present application.

[0111] In the normal state, the optical properties of ochre are mainly reflected in the red and yellow bands (about 580-640 nm) with relatively high reflectivity, and in the blue band (about 450-490 nm) with relatively low absorption. The red and yellow reflection of ochre is mainly due to the chemical properties of iron oxide. However, when the pattern color is missing or the ochre is degraded, the absorption in the red band decreases significantly, which indicates that the structure of the ochre pigment has changed.

[0112] In the area greatly affected by the environment, iron oxide is prone to further chemical reaction to form other forms of iron oxide, such as hydrated iron oxide (FeO(OH)) or other secondary minerals. These newly formed compounds have different spectral reflection characteristics compared with iron oxide (hematite), resulting in a decrease in reflectivity in the red and yellow bands, thereby exhibiting a decrease in reflectivity. This chemical transformation causes the pigment to gradually fade or lose its red and yellow appearance. Although iron oxide is relatively stable, its surface structure is damaged under the influence of ultraviolet light for a long time, affecting the light reflection performance of the pigment, which exhibits a significant decrease in reflectivity in the red band (about 620-720 nm).

[0113] It can be understood that in the pattern color missing or degradation area, the degradation of the lapis lazuli pigment can cause the reflection peak of the blue band to decrease, and the reflectivity to decrease significantly, so by detecting and comparing the area where the reflectivity in the blue band decreases significantly, it can be determined that it is the area where the pattern color fades and is missing.

[0114] Step 103: comparing the reflection spectrum data of each scanning point, extracting at least one low reflectivity point, and calculating the reflection spectrum similarity between each low reflectivity point and each scanning point in the adjacent area.

[0115] On the basis of the above, in some embodiments of the present application, the acquired spectral data is processed to extract low reflectivity scanning points, which are defined as points whose reflectivity in multiple color light bands (such as red, green, blue bands, etc.) is significantly lower than that of other scanning points.

[0116] Specifically, the reflectance spectral data of each scanning point at different wavelengths is analyzed to find scanning points whose reflectivity in multiple color bands (such as red, green, blue bands, etc.) is significantly lower than that of other scanning points in the same region. In some embodiments provided by the present application, a threshold value is set Only when a certain pixel meets the low reflectivity in all key bands (red, green, and blue), can it be marked as a low reflectivity point, and the condition meets the formula:

[0117]

[0118] For each detected low reflectivity point, it is marked as P i , and its position coordinates (x i , y i , z i ) in the three-dimensional coordinate system are recorded. These scanning points are used as potential pattern fading, missing, or serious degradation of drawing materials, and the range of the missing area is confirmed through subsequent similarity analysis.

[0119] Regarding the similarity analysis, specifically, for each scanning point, the spectral similarity will measure the reflectivity of the pixel at a series of wavelengths. The spectral data collected from each scanning point will form a vector, and the spectral vector contains the reflectivity information of multiple bands. The spectral vector is defined as:

[0120]

[0121] where X is the spectral vector of a certain scanning point;

[0122] represents the reflectivity of the pixel at wavelength λ i .

[0123] Taking the murals of the ancient Guge Dynasty in Tibet as an example, the three common pigments are cinnabar, lapis lazuli, and ocher. Their reflectivity in different bands is different, so the reflectivity values of their spectral vectors in each band are significantly different. When the murals fade or are missing, the reflectivity will significantly decrease at all wavelengths, forming a low reflectivity spectral vector.

[0124] More specifically, when the pigments in the murals degrade or the patterns are missing, each component of the spectral vector, i.e., the reflectivity value of each band, such as the red light band of cinnabar, the blue light band of lapis lazuli, and the red and yellow light band of ocher, will become significantly lower.

[0125] The low reflectivity point P was extracted above. i Based on this, with each low reflectivity point P i Define a neighborhood containing several surrounding scan points, with the center point as the center. The size of this neighborhood can be set according to actual needs, usually determined by the scan density and the spatial distribution of mural features.

[0126] For reference Figure 3 , Figure 3 A color image generated from a scan of a mural at the ruins of the Guge Kingdom in Tibet. Figure 3 Point A in the middle, refer to Figure 4 A low reflectivity point P was identified at point A. i Now we define a point P with low reflectivity. i Using the neighborhood centered at a point, and taking the furthest distance between three scan points on a straight line as the radius, we focus on local features and extract each scan point P within the neighborhood. j The reflectance spectral characteristics, including the spectral vector X of each scan point, are analyzed for spectral similarity. Based on the spectral similarity, scan points in the neighborhood that belong to pattern missing or faded areas are identified.

[0127] Specifically, in some feasible embodiments of the present invention, cosine similarity is used to analyze the similarity of spectral vectors between various scanning points. Cosine similarity only considers the direction between spectral vectors. In the analysis of mural patterns, although the reflectance of different scanning points has certain differences, their relative proportions in each band are more important. Therefore, cosine similarity can more effectively capture the shape changes of the spectral curve, rather than the absolute reflectance value, thus avoiding errors introduced by differences in reflectance magnitude.

[0128] Furthermore, the spectral data from mural scans typically includes multiple bands, forming high-dimensional spectral vectors. Cosine similarity performs well in processing high-dimensional vectors, quickly calculating the similarity between vectors while maintaining high computational efficiency regardless of the number of bands. In the analysis of mural patterns, the reflectance of adjacent scan points fluctuates due to environmental factors (such as changes in illumination or instrument noise). Cosine similarity focuses on measuring the directional differences of vectors, thus exhibiting good robustness to these minute fluctuations and not allowing small variations in reflectance to affect similarity judgments. Because cosine similarity focuses more on the similarity of spectral curve shapes, it can effectively identify scan points with spectral curves similar to low-reflectance points in local areas. This characteristic helps to quickly find other affected scan points in missing or degraded areas, thus aiding in the identification of missing regions.

[0129] In multi-band (such as visible light and near-infrared light) scanning, the spectral reflectance of each scanning point is composed of reflectance at different wavelengths, forming a spectral vector. Let P be a scanning point in the neighborhood j The spectral vector of P is The spectral vector of P i is The cosine similarity is calculated by the following formula:

[0130]

[0131] Where X·X low represents the inner product of two vectors:

[0132]

[0133] ||X|| and ||X low || are the norms (i.e. vector lengths) of the two vectors.

[0134] When the value of cosθ is close to 1, it indicates that the spectral vectors of the two scanning points are highly similar, meaning that the scanning point in the neighborhood is also a low reflectance point, indicating that it is in a pattern missing or faded area.

[0135] When the value of cosθ is close to 0, it indicates that the spectral vectors are quite different, meaning that the scanning point in the neighborhood is less similar to the low reflectance point, and may still maintain the original pattern.

[0136] When the value of cosθ is close to -1, it indicates that the directions of the two spectral vectors are completely opposite.

[0137] Step 104: The low reflectance point and each scanning point in its neighborhood with high reflectance similarity form a complete area, and the area is marked in the three-dimensional point cloud data.

[0138] Based on the calculation of cosine similarity, the newly detected high similarity point is continuously taken as a new center point P' i , the neighborhood is further expanded, and the similarity analysis is repeated until the complete missing area is found.

[0139] More specifically, the spectral vector of each scanning point P' i in the neighborhood of the new center point P' j is calculated for cosine similarity, and the calculated similarity value, i.e. the value of cosθ, is compared with the set cosine similarity threshold T. In the embodiment of the present application, the cosine similarity threshold T is used to judge the similarity between the new center point P' i and other scanning points P' jThe size of T can be set according to actual operation requirements, and is not specifically limited here.

[0140] If the new center point P' i is similar to a certain scanning point P' j in the neighborhood: cosθ>T: the scanning point P' j is considered to have similar spectral characteristics with the new center point P' i , and is added to the region to be analyzed, and the point is set as the new center point P' i .

[0141] If cosθ≤T: the scanning point P' j is no longer added to the region to be analyzed.

[0142] Select a new center point (such as point P' j ), and repeat the above steps to calculate the similarity of other scanning points in the neighborhood of the new center point:

[0143] Continue to expand: if the similarity value is still greater than the threshold T, continue to expand the neighborhood.

[0144] Stop expanding: the similarity of all scanning points in the neighborhood is less than or equal to the threshold T, indicating that the boundary of the region has been determined.

[0145] Once the expansion stops, record all scanning points similar to the initial low reflectivity point, which constitute the missing or faded region.

[0146] Through this cosine similarity-based region expansion method, the missing or faded region in the mural can be effectively identified, and its boundary can be accurately defined and marked in the collected and processed three-dimensional point cloud data. This method can help identify the range of murals that need to be repaired in practical applications, so that relevant researchers can better understand and maintain historical heritage.

[0147] A three-dimensional image immersive generation method for a site provided in an embodiment of the application is used to model and process data collected by scanning the site building, as shown in the flowchart of Figure 2 , and specifically includes:

[0148] Step 201: After the three-dimensional point cloud data with image color information obtained by scanning the site building is subjected to point cloud denoising, point cloud alignment and point cloud simplification, the simplified point cloud data is converted into a triangular mesh, i.e. a surface structure is generated through the adjacency relationship between points. In some implementable embodiments of the present application, the three-dimensional point cloud data with image color information obtained by scanning is subjected to denoising, because some noise points may be generated in the scanning process, which do not belong to the surface of the object and need to be removed by an algorithm. If multiple angles are used when scanning an object or scene, the point cloud data at different angles needs to be aligned (for example, using the ICP algorithm) to generate a complete model. Each point of the point cloud data is independent, and there is no direct connection between points (such as the edges between triangular facets), so it cannot directly represent the surface of the object, but represents a series of discrete points, which can be converted into a continuous surface model by a surface reconstruction algorithm (such as Poisson Surface Reconstruction), as shown in Figure 8 . Figure 8 a triangular mesh model generated according to a three-dimensional point cloud data.

[0149] It can be understood that the simplified three-dimensional point cloud data is constructed into a triangular mesh according to the adjacency relationship between points. The generation of the triangular mesh makes the geometric structure of the site building surface more coherent, and can better display the details through the facets, and realizes the digital reconstruction of the site surface structure through the triangular mesh, so that it has a realistic geometric shape and is suitable for immersive experience.

[0150] Step 202: Map the image color information in the three-dimensional point cloud to the generated triangular mesh.

[0151] Specifically, the color information (obtained from a color image) in the original three-dimensional point cloud is accurately mapped to the generated triangular mesh facets, so that each mesh facet not only carries geometric information but also has surface color properties, ensuring that the model visually presents a realistic appearance of the site.

[0152] Step 203: Combine the reflection spectrum data of the site building surface at each scanning point with the surface properties of each triangular mesh facet, and mark the area formed by the low reflectivity points in the reflection spectrum and the scanning points in the neighborhood thereof having similar reflectivity information in the triangular mesh.

[0153] Specifically, the reflection spectrum characteristics of each mesh facet are analyzed in combination with the previously collected spectral data of the site surface reflecting light of different wavelengths. The spectral data provide additional information of the surface material, and the missing area of the mural pattern color can be analyzed and identified by comparing the reflectivity of multiple color bands.

[0154] It can be understood that through the above-mentioned region expansion method based on cosine similarity, the pattern missing or faded regions in the mural are identified, the boundaries thereof are accurately defined, and the pattern missing or faded regions are marked on the three-dimensional model for display or for researchers to refer to analysis.

[0155] On the basis of the above, it can be further understood that the triangular mesh model with color information and spectral markers after the above processing is imported into an immersive display system, such as a virtual reality (VR) or augmented reality (AR) system, a user can view the mural pattern missing or faded regions marked after spectral analysis in three-dimensional space, immerse in the real state of the site building, and understand the details of the degradation, damage, etc. of the surface material.

[0156] Through the combination of point cloud and spectral data, not only the three-dimensional geometric structure of the site building can be reconstructed, but also the pattern color missing condition can be identified through spectral analysis.

[0157] The above is only a preferred embodiment of the present application, and is not used to limit the present application, the patent protection scope of the present application is subject to the claims, any equivalent structural changes made by using the contents of the specification and drawings of the present application should be included in the protection scope of the present application.

Claims

1. A method for acquiring three-dimensional images of archaeological sites, characterized in that, include: Acquire 3D point cloud data generated after scanning the ruins with a 3D scanner; While acquiring the three-dimensional point cloud data, reflectance spectral data of the surface of the ruins under multiple different wavelengths of light were acquired at each scanning point. The reflectance spectral data of each scan point are compared to extract at least one low reflectance point. The reflectance spectral similarity between each low reflectance point and each scan point in the adjacent region is calculated. A low reflectance point indicates that the reflectance of a certain scan point is lower than the reflectance of other scan points in the corresponding color light bands. The calculation of the reflectance spectral similarity specifically involves comparing the reflectance information of the low reflectance point with that of each scan point in its neighborhood under different color bands to calculate their reflectance spectral similarity. as well as The low reflectivity point and each scan point with high reflectivity spectrum similarity in their neighborhood constitute a complete region. After comparing and analyzing the reflectivity spectrum data of each scan point, multiple low reflectivity points are marked, and the position coordinates of each low reflectivity point are calculated. For each marked low reflectivity point, a neighborhood centered on the low reflectivity point is defined, and the reflectivity information of each scan point in the neighborhood is extracted. For each scan point, the collected reflectivity spectrum data is used to form a spectral vector, which contains reflectivity in multiple color bands. The cosine similarity between the spectral vector of the low reflectivity point and the spectral vector of a certain scan point in its neighborhood is calculated. If a scan point within the neighborhood has low reflectivity and similar reflectivity information to a low-reflectivity point, then that scan point is designated as the new center point, and expansion continues. The neighborhood of the new center point is redefined, and reflectivity spectral similarity is calculated until the cosine similarity between the new center point and all scan points within the neighborhood is lower than a set cosine similarity threshold, at which point expansion stops. After expansion stops, all low-reflectivity points within the expanded area and scan points within their neighborhoods with similar reflectivity information are combined to form a complete region. This region is defined as the pattern color missing region on the surface of the site and marked in the three-dimensional point cloud data.

2. The method for acquiring three-dimensional images of archaeological sites according to claim 1, characterized in that, The acquisition of 3D point cloud data generated after scanning the ruins with a 3D scanner includes: At each scanning position, a Cartesian coordinate system is constructed with the location of the 3D scanner as the origin; wherein, the three axes in the coordinate system represent the horizontal direction, the vertical direction, and the distance information from the scanned target to the 3D scanner within the scanning range, respectively. The 3D scanner calculates the distance from each laser reflection point on the site surface to the 3D scanner by emitting pulsed lasers and calculating the flight time of the lasers reflected from the building surface. The distance data of each laser reflection point on the surface of the site is combined with the corresponding horizontal and vertical angles to calculate the three-dimensional coordinates of the point in the Cartesian coordinate system. Each three-dimensional coordinate point is recorded as an independent three-dimensional point cloud data, and together they are used to construct a point cloud model of the entire surface of the site.

3. The method for acquiring three-dimensional images of archaeological sites according to claim 2, characterized in that, While performing the 3D scan, a color image of the current site surface is captured by a camera. For each scan point in the 3D point cloud data, at least one pixel value from the color image is matched; specifically, In a three-dimensional coordinate system, each three-dimensional point corresponds to at least one two-dimensional pixel coordinate; By setting the camera's intrinsic parameters, the two-dimensional pixel coordinates are converted into the coordinates of three-dimensional points; For each of the three-dimensional points, its corresponding image pixel is found, the color value of the pixel is obtained and then assigned to the three-dimensional point to form three-dimensional point cloud data with color information.

4. A method for acquiring three-dimensional images of archaeological sites according to claim 2, characterized in that, While performing the three-dimensional scan, multiple different wavelengths of light are selected to illuminate the surface of the ruins building, and reflectance spectral data at different wavelengths are acquired at each scanning point. The reflectance spectral data contains reflectance information in different color bands.

5. A method for acquiring three-dimensional images of archaeological sites according to claim 4, characterized in that, Mark multiple low reflectivity points Calculate each low reflectivity point Position coordinates in a three-dimensional coordinate system The spectral vector Reflectance included across multiple color bands: ； in, This indicates that the scan point is at a wavelength Reflectance at that location; Calculate the low reflectivity point spectral vector a scan point in its neighborhood spectral vector cosine similarity : ； in, Represents the dot product of two vectors: ； and These are the magnitudes of the two vectors, respectively. A value close to 1 indicates a high degree of similarity in the spectral vectors, suggesting that the scan point in the neighborhood is similar. With low reflectivity point The reflectance information is highly similar across different color bands.

6. A method for immersive generation of 3D images of archaeological sites, characterized in that, A method for acquiring three-dimensional images of a historical site as described in any one of claims 1 to 5, comprising: After performing point cloud denoising, point cloud alignment, and point cloud simplification on the 3D point cloud data with image color information obtained after scanning the ruins, the simplified point cloud data is converted into a triangular mesh, that is, the surface structure is generated by the adjacency relationship between points. The image color information in the 3D point cloud is mapped onto the generated triangular mesh; The reflectance spectral data of multiple wavelengths of light reflected from the surface of the ruins at each scanning point are combined with the surface properties of each triangular mesh facet, and the region formed by the low reflectance points in the reflectance spectrum and the scanning points with similar reflectance information in their neighborhood is marked in the triangular mesh.

7. A system for acquiring three-dimensional images of archaeological sites, characterized in that, Performing a method for acquiring three-dimensional images of a historical site as described in any one of claims 1 to 5, comprising: The data acquisition terminal is configured as follows: Acquire 3D point cloud data generated after scanning the ruins with a 3D scanner; While acquiring the three-dimensional point cloud data, reflectance spectral data of the surface of the ruins under multiple different wavelengths of light were acquired at each scanning point. The reflectance spectral data of each scan point are compared to extract at least one low reflectance point, and the similarity of the reflectance spectrum of each low reflectance point with that of each scan point in the adjacent region is calculated; and The low reflectivity point and each scan point with high reflectance spectrum similarity in their neighborhood are collectively formed into a complete region, and the region is marked in the three-dimensional point cloud data.

8. A system for immersive generation of 3D images of archaeological sites, characterized in that, Performing the immersive generation method for three-dimensional images of archaeological sites as described in claim 6 includes: The data generation end is configured as follows: After performing point cloud denoising, point cloud alignment, and point cloud simplification on the 3D point cloud data with image color information obtained after scanning the ruins, the simplified point cloud data is converted into a triangular mesh, that is, the surface structure is generated by the adjacency relationship between points. The image color information in the 3D point cloud is mapped onto the generated triangular mesh; The reflectance spectral data of multiple wavelengths of light reflected from the surface of the ruins at each scanning point are combined with the surface properties of each triangular mesh facet, and the region formed by the low reflectance points in the reflectance spectrum and the scanning points with similar reflectance information in their neighborhood is marked in the triangular mesh.

Images