Method and system for site three-dimensional image acquisition and immersive generation

Abstract

The invention discloses a three-dimensional image acquisition and immersive generation method and system for a ruins, and relates to the technical field of image acquisition and modeling of ruins buildings, and the three-dimensional image acquisition method for the ruins comprises the steps: obtaining three-dimensional point cloud data generated after the ruins buildings are scanned by a three-dimensional scanner; while obtaining the three-dimensional point cloud data, obtaining reflection spectrum data of the surface of the relics building under illumination of a plurality of different wavelengths at each scanning point; and comparing the reflection spectrum data of each scanning point, extracting at least one low-reflectivity point, and calculating the reflection spectrum similarity between each low-reflectivity point and each scanning point in the adjacent region. Through the combination of the point cloud and the spectral data, the three-dimensional structure and color distribution of the building are presented, and through the analysis of the reflectivity, the pattern missing or fading area can be identified, so that the comprehensive diagnosis and display of the site building can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of site image acquisition and generation modeling, and in particular to a method and system for acquisition and immersive generation of three-dimensional images of sites. Background Art

[0002] In the process of scanning and recording the three-dimensional spatial data of the ruins buildings, the geometric spatial structure of the ruins buildings is usually scanned and recorded with the help of scanning devices such as three-dimensional laser scanners. On the basis of constructing the geometric model, if it is necessary to collect image information on the ruins, such as collecting mural patterns on the ruins and combining them with the constructed three-dimensional model, most of the existing technologies use cameras to scan the murals on the ruins buildings in different areas, and then use image stitching technology to generate a complete mural and fit it to the three-dimensional model.

[0003] The loss of mural patterns is not only manifested in color changes on the images, but more often due to environmental factors such as weathering and humidity, the aging of mural pigments and the peeling of patterns. Taking the ruins of the Guge Dynasty in Tibet as an example, the buildings of the Guge Dynasty mostly use local sandstone and shale, and the pigments used in the murals in the buildings are mostly cinnabar, lapis lazuli and ochre. These rock materials are prone to loosening and peeling off under long-term weathering, which in turn affects the overall peeling of the mural pigments attached to them; Tibet is located in a plateau environment with high ultraviolet radiation intensity. Cinnabar and other mineral pigments will gradually fade and decompose under ultraviolet radiation, which accelerates the aging process of the pigments and causes the loss of mural patterns.

[0004] Existing 3D scanning technology can only provide geometric information of the building surface, and it is difficult to identify areas where patterns are missing due to material degradation. Summary of the invention

[0005] 1) Technical issues solved

[0006] The present invention provides a method and system for collecting and immersively generating three-dimensional images of ruins, so as to solve the problem that the existing scanning technology can only provide surface geometric information and is difficult to identify pattern missing areas caused by material degradation.

[0007] 2) Technical solution

[0008] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for collecting three-dimensional images of ruins, comprising:

[0009] Obtaining three-dimensional point cloud data generated by scanning the ruins building with a three-dimensional scanner;

[0010] While acquiring the three-dimensional point cloud data, at each scanning point, the reflectance spectrum data of the surface of the ruins building under multiple different wavelengths of light are acquired;

[0011] Comparative processing is performed on the reflectance spectrum data of each of the scanning points, at least one low reflectance point is extracted, and the reflectance spectrum similarity between each of the low reflectance points and each scanning point in an adjacent area is calculated; wherein the low reflectance point indicates that the reflectance of a certain scanning point in multiple color light bands is lower than the reflectance of other scanning points in the corresponding color light bands; the calculation of the reflectance spectrum similarity is specifically to compare the reflectance information of the low reflectance point and each scanning point in its neighborhood in different color bands, and calculate the reflectance spectrum similarity; and

[0012] The low reflectivity point and each scanning point in its neighborhood with high reflectivity spectrum similarity together form a complete area, and the area is marked in the three-dimensional point cloud data.

[0013] Furthermore, the step of obtaining three-dimensional point cloud data generated by scanning the ruins building with a three-dimensional scanner includes:

[0014] At each scanning position, a Cartesian coordinate system is constructed with the position of the three-dimensional scanner as the origin; wherein the three axes in the coordinate system respectively represent the horizontal direction, the vertical direction and the distance information from the scanned object to the three-dimensional scanner within the scanning range;

[0015] The three-dimensional scanner emits pulsed lasers and calculates the flight time of the lasers reflected from the building surface, thereby calculating the distance from each laser reflection point on the surface of the site to the three-dimensional scanner;

[0016] The distance data of the laser reflection point on the surface of each site is combined with the corresponding horizontal angle and vertical angle to calculate the three-dimensional coordinates of the point in the Cartesian coordinate system. Each of the three-dimensional coordinate points is recorded as independent three-dimensional point cloud data and together they form a point cloud model of the entire site surface.

[0017] Furthermore, while performing the three-dimensional scanning, a color image of the current site surface is captured by a camera, and for each scanning point in the three-dimensional point cloud data, at least one pixel value in the color image is matched; specifically,

[0018] In the three-dimensional coordinate system, each three-dimensional point corresponds to at least one two-dimensional pixel coordinate;

[0019] By setting the intrinsic parameters of the camera, the two-dimensional pixel coordinates are converted into the coordinates of the three-dimensional point;

[0020] For each of the three-dimensional points, its corresponding image pixel is found, and the color value of the pixel is obtained and then assigned to the three-dimensional point to form three-dimensional point cloud data with color information.

[0021] Furthermore, while performing the three-dimensional scanning, a plurality of lights of different wavelengths are selected to illuminate the surface of the ruins building, and reflection spectrum data at different wavelengths are acquired at each scanning point, wherein the reflection spectrum data includes reflectivity information in different color bands.

[0022] Further, after comparing and analyzing the reflectance spectrum data of each scanning point, a plurality of low reflectance points P are marked. i , calculate each low reflectivity point P i The position coordinates (x i ,y i , z i );

[0023] For each marked low reflectivity point P i , define a point P with low reflectivity i is the neighborhood of the center point, and each scanning point P in the neighborhood is extracted j Reflectivity information;

[0024] For each scanning point, the collected reflectance spectrum data is formed into a spectrum vector X, which contains the reflectance in multiple color bands:

[0025]

[0026] in, Indicates that the scanning point is at wavelength λ i The reflectivity at

[0027] Calculate the low reflectivity point P i The spectrum vector With a scanning point P in its area j The spectrum vector The cosine similarity cosθ is:

[0028]

[0029] Among them, X·X low Represents the inner product of two vectors:

[0030]

[0031] ||X|| and ||X low || are the moduli of the two vectors (i.e., the length of the vector);

[0032] The value of cosθ is close to 1, indicating that the spectral vectors are highly similar, indicating that the scan point P in the neighborhood j With low reflectivity point P i The reflectance information in different color bands is highly similar.

[0033] Furthermore, if there is a scanning point P in the neighborhood j The reflectivity is low and close to the low reflectivity point P i have similar reflectivity information, then the scanning point P j As the new said center point, continue to expand;

[0034] Redefine the neighborhood for the new center point and perform the reflection spectrum similarity calculation until the cosine similarity cosθ between the new center point and all scanning points in the neighborhood is lower than the set cosine similarity threshold value and then stop expanding;

[0035] After the expansion stops, all low reflectivity points P in the expanded area i and the scanning point P with similar reflectivity information in its neighborhood j , together constitute a complete area, this area is defined as the pattern color missing area on the surface of the ruins, and the pattern color missing area is marked in the three-dimensional point cloud data.

[0036] A method for immersive generation of three-dimensional images of ruins, comprising:

[0037] After performing point cloud denoising, point cloud alignment and point cloud simplification on the three-dimensional point cloud data with image color information obtained after scanning the ruins buildings, the simplified point cloud data is converted into a triangular mesh, that is, the surface structure is generated through the adjacent relationship between points;

[0038] Mapping the image color information in the three-dimensional point cloud to the generated triangular mesh;

[0039] The reflectance spectrum data of multiple different wavelengths of light reflected by the surface of the ruins building obtained at each scanning point is combined with the surface properties of each triangular mesh facet, and the area composed of the low reflectance points in the reflectance spectrum and the scanning points with similar reflectance information in their neighborhood are marked in the triangular mesh.

[0040] A three-dimensional image acquisition system for ruins, comprising:

[0041] The data collection end is configured as:

[0042] Obtaining three-dimensional point cloud data generated by scanning the ruins building with a three-dimensional scanner;

[0043] Obtaining three-dimensional point cloud data generated by scanning the ruins building with a three-dimensional scanner;

[0044] While acquiring the three-dimensional point cloud data, at each scanning point, the reflectance spectrum data of the surface of the ruins building under multiple different wavelengths of light are acquired;

[0045] Comparing the reflection spectrum data of each scanning point, extracting at least one low reflectivity point, and calculating the reflection spectrum similarity between each low reflectivity point and each scanning point in an adjacent area; and

[0046] The low reflectivity point and each scanning point in its neighborhood with high reflectivity spectrum similarity together form a complete area, and the area is marked in the three-dimensional point cloud data.

[0047] A system for generating immersive three-dimensional images of ruins, comprising:

[0048] The data generation end is configured as:

[0049] After performing point cloud denoising, point cloud alignment and point cloud simplification on the three-dimensional point cloud data with image color information obtained after scanning the ruins buildings, the simplified point cloud data is converted into a triangular mesh, that is, the surface structure is generated through the adjacent relationship between points;

[0050] Mapping the image color information in the three-dimensional point cloud to the generated triangular mesh;

[0051] The reflectance spectrum data of multiple different wavelengths of light reflected by the surface of the ruins building obtained at each scanning point is combined with the surface properties of each triangular mesh facet, and the area composed of the low reflectance points in the reflectance spectrum and the scanning points with similar reflectance information in their neighborhood are marked in the triangular mesh.

[0052] 3) Beneficial effects:

[0053] Compared with the prior art, the invention has the following beneficial effects:

[0054] The present invention generates a three-dimensional model of the ruins building with geometric, color and spectral information by fusing three-dimensional point cloud, color image and spectral data. Based on the principle that mineral pigments (such as cinnabar, lapis lazuli and ochre) have specific spectral reflectance characteristics in normal state and the reflectivity of missing or faded areas will be significantly reduced in multiple color bands, the area with significant spectral difference is found and marked as pattern missing or pigment degraded area. Through the combination of three-dimensional point cloud and spectral data, not only the three-dimensional structure and color distribution of the building are presented, but also the areas with missing or faded patterns can be identified by analyzing the reflectivity, so as to realize the comprehensive diagnosis and display of the ruins building. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A flowchart of a method for collecting three-dimensional images of ruins provided by an embodiment of the present invention;

[0056] Figure 2A flowchart of a method for immersive generation of three-dimensional images of ruins provided by an embodiment of the present invention;

[0057] Figure 3 A partial two-dimensional color image generated by scanning and capturing the surface of a site in a three-dimensional image acquisition method for a site provided by an embodiment of the present invention;

[0058] Figure 4 The embodiment of the present invention provides Figure 3 In the image at point A shown in the figure, a low reflectivity point is identified, and after the spectral characteristics are compared with the scanned points in its neighborhood, the color missing area of ​​the pattern is marked;

[0059] Figure 5 A comparison diagram of the reflection spectra of a normal area and a severely degraded area of ​​cinnabar pigment under the same lighting conditions and reflection distance provided by an embodiment of the present invention;

[0060] Figure 6 A comparison diagram of the reflection spectra of a normal area and a severely degraded area of ​​lapis lazuli pigment under the same lighting conditions and reflection distance provided by an embodiment of the present invention;

[0061] Figure 7 A comparison diagram of the reflection spectra of a normal area and a severely degraded area of ​​ochre pigment under the same lighting conditions and reflection distance provided by an embodiment of the present invention;

[0062] Figure 8 A triangular mesh model generated by a method and system for immersive generation of three-dimensional images of ruins according to an embodiment of the present invention is generated based on three-dimensional point cloud data. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0064] In the description of the present invention, it should be understood that the terms "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0065] In the process of scanning and recording the three-dimensional spatial data of the ruins buildings, the geometric spatial structure of the ruins buildings is usually scanned and recorded with the help of scanning devices such as three-dimensional laser scanners. On the basis of constructing the geometric model, if it is necessary to collect image information on the ruins, such as the need to collect mural patterns on the ruins and combine them with the constructed three-dimensional model, most of the existing technologies use a camera to scan the entire mural in different areas, and then use image stitching technology to generate a complete mural and fit it to the three-dimensional model.

[0066] The loss of mural patterns is not only manifested as the loss of color on the image, but more often it is caused by the degradation of painting materials. Due to environmental factors such as weathering and humidity, the mural pigments will peel off or age. Existing 3D scanning technology can only provide surface geometry information, and it is difficult to identify the pattern missing areas caused by material degradation.

[0067] Take the ruins of the Guge Dynasty in Tibet as an example. The buildings of the Guge Dynasty were mostly made of local sandstone and shale, and the pigments used in the murals in the buildings were mostly cinnabar, lapis lazuli and ochre. The rock materials used in the buildings are prone to loosening and peeling off under long-term weathering, which in turn affects the mural pigments attached to them. The porous structure of sandstone and shale makes them more likely to absorb moisture. When the temperature changes or is exposed to the wind, the evaporation of water causes the stone to expand and contract, which accelerates the peeling of the pigment on the surface of the murals.

[0068] In addition, cinnabar is used as a red pigment, and its main component is mercury sulfide. In a humid environment, cinnabar is prone to oxidation, gradually changing color or even powdering, causing the red area of ​​the mural to fade or peel off. Lapis lazuli is used as a blue pigment, and its main component is a complex of labradorite and other minerals. Lapis lazuli is sensitive to humidity. Long-term exposure to moisture will cause it to decompose and fade, and the pattern will be lost as the base peels off. Ochre is used as a brown or reddish brown pigment. As a natural iron oxide pigment, ochre will react chemically under humid and acidic conditions, causing the color of the pigment to fade or fall off. In addition, Tibet is located in a plateau environment with high radiation intensity. Cinnabar and other mineral pigments will gradually fade or decompose under ultraviolet rays, which will accelerate the aging process of the pigment and cause the pattern to be lost. In high-wind and sandy areas such as the ruins of the Guge Dynasty, the surface of the murals has been eroded by fine particles for a long time, causing the pigment to gradually fall off.

[0069] The embodiments of the present invention provide a method and system for collecting and immersively generating three-dimensional images of ruins. Since pigment degradation caused by weathering and humidity exhibits different reflectivity changes at different wavelengths, the areas on the surface where pigment has peeled off or aged can be identified through multi-reflection spectral data, and the areas where pattern color is missing can be marked in the subsequent data generation conversion model process for restoration reference and immersive display.

[0070] Specifically, refer to Figure 1 , Figure 1 A flowchart of a method for collecting three-dimensional images of ruins provided by an embodiment of the present invention, specifically:

[0071] Step 101: Obtain the three-dimensional point cloud data generated after scanning the ruins building with a three-dimensional scanner. In some embodiments of the present invention, the ruins building is scanned by a laser scanner, and the flight time of the laser from the scanner to the surface reflection is measured to obtain the distance from each laser reflection point on the building surface to the scanner, and generate three-dimensional point cloud data. Construct a spatial coordinate system, and the three-dimensional coordinates (x, y, z) of each laser reflection point record the position of the point in the scanning area, forming a point cloud model in three-dimensional space, accurately reflecting the geometric structure of the building.

[0072] Specifically, at each scanning position, a Cartesian coordinate system is established with the position of the 3D scanner as the coordinate origin. The three axes in the coordinate system are used to represent the horizontal direction and vertical direction of the scanner within the scanning range, and the distance (depth information) from the 3D scanner to the reflection point on the surface of the target site. Among them:

[0073] X axis: horizontal scanning direction;

[0074] Y axis: vertical scanning direction;

[0075] Z axis: Depth information from the laser scanner to the scanned object.

[0076] The 3D laser scanner emits a pulsed laser beam to the surface of the site building, then receives the laser reflected from the surface and determines the distance from each reflection point to the scanner by calculating the flight time of the laser from emission to return.

[0077] At the same time, the laser scanner records the horizontal angle and vertical angle when the laser is emitted, which represent the orientation of the reflection point relative to the scanner. The horizontal angle, vertical angle and measured distance information are converted to Cartesian coordinates using spherical coordinates. The three-dimensional coordinates (x, y, z) of each point can be converted using the following formula:

[0078]

[0079] z=D cosθ

[0080] Where D is the distance, θ is the vertical angle, is the horizontal angle.

[0081] The 3D coordinates of each reflection point are recorded as an independent 3D point. As the 3D scanner scans different locations of the ruins multiple times, these 3D points are gradually accumulated, and finally a complete point cloud model of the entire ruins surface is constructed.

[0082] In some feasible embodiments of the present invention, while using a laser scanner, an RGB camera is synchronously used to capture a color image of the surface of the ruins to obtain the color information (RGB value) of each pixel.

[0083] Each pixel value of the color image is matched with the corresponding point in the 3D point cloud data. Through the calibration of the camera's internal and external parameters, the 2D pixel coordinates (u, v) in the color image are converted to points in 3D space, and each 3D point is assigned its corresponding color information.

[0084] Specifically, each laser reflection point (that is, a point in the three-dimensional point cloud) has a corresponding three-dimensional coordinate (x, y, z) in the Cartesian coordinate system. Because the image captured by the camera is two-dimensional, most of the pixels in the image also have their corresponding two-dimensional pixel coordinates (u, v).

[0085] More specifically, the 2D pixel coordinates corresponding to each 3D point are determined through the camera's intrinsic and extrinsic parameters. The intrinsic parameters describe the camera's optical properties, mainly including focal length, principal point position, and radial distortion coefficient, while the extrinsic parameters are used to describe the camera's position and orientation in 3D space, including rotation matrices and displacement vectors, which are used to convert points in 3D space into 2D pixels from the camera's perspective.

[0086] In some feasible embodiments of the present invention, by using the intrinsic and extrinsic parameters of the camera, the three-dimensional coordinates (x, y, z) can be mapped to the two-dimensional pixel coordinates (u, v) of the camera image plane.

[0087]

[0088] Among them, K is the intrinsic parameter matrix of the camera, R is the rotation matrix, and t is the displacement vector.

[0089] After mapping the three-dimensional point to the two-dimensional image pixel coordinates, the color value (such as RGB value) of the pixel is obtained.

[0090] The obtained color value is assigned to the 3D point. In this way, each 3D point carries not only its spatial coordinates but also the color information of the point.

[0091] In summary, each point in the 3D point cloud data is assigned corresponding color information to form a 3D point cloud model with color. The 3D point cloud data generated by this method can not only accurately describe the geometric form of the ruins building, but also retain the color details on the surface, achieving a more realistic and detailed digital expression.

[0092] Step 102: While acquiring the three-dimensional point cloud data, obtain the reflectance spectrum data of the surface of the ruins building under multiple different wavelengths of light at each scanning point. In some feasible embodiments of the present invention, by setting a spectral camera, the reflectance spectrum data of each scanning point under different wavelengths of light is synchronously acquired. The spectral camera measures each pixel in multiple bands (such as ultraviolet, visible light, infrared) and records its reflectivity at different wavelengths. These spectral data can help distinguish different areas of the material, especially when the color changes or material degradation are difficult to identify with the naked eye, spectral analysis can provide more details.

[0093] It should be noted that, in order to avoid different lighting interference with the work of the RGB camera and the spectral camera, in some embodiments of the present invention, a uniform white light LED light source is used to avoid the influence of the flash on the spectral camera, and standard red, green, and blue filters are used in front of the RGB camera to separate color information. A synchronous trigger mechanism is used to ensure that the 3D laser scanner, RGB camera, and spectral camera can collect data at the same time point, that is, during the scanning process, while the 3D laser scanner generates point cloud data, the RGB camera and spectral camera respectively capture the relevant color image and spectral data.

[0094] In this example from the Guge ruins in Tibet, the following are the different paint pigments and their associated reflective properties:

[0095] Cinnabar: The main component is mercury sulfide (such as HgS).

[0096] In the visible spectrum, cinnabar usually has high reflectivity in the red light band (about 620-750nm) and lower reflectivity in the blue-green light band.

[0097] Specific reference Figure 5 , Figure 5 A comparison diagram of the reflection spectra of a normal area and a severely degraded area of ​​cinnabar pigment under the same lighting conditions and reflection distance provided by an embodiment of the present invention.

[0098] Under normal conditions, the reflectance spectrum of cinnabar (mercury sulfide, HgS) shows strong reflection in the red band of visible light (about 620-750nm); there is obvious absorption in the green band (about 500-550nm) and the blue band (about 450-490nm).

[0099] The ruins of the Guge Dynasty in Tibet are located on a plateau with high intensity of ultraviolet radiation and frequent natural disasters such as sandstorms. Under the action of ultraviolet rays and oxygen, part of mercury sulfide (cinnabar) will gradually turn into mercury oxide. This change changes the chemical structure of cinnabar. The reflective properties of mercury oxide are different from those of mercury sulfide, and its reflective ability in the visible red light band is significantly reduced.

[0100] And due to the influence of windy and sandy weather, the pigment particles will decompose or refine. The size of the pigment particles directly affects its spectral reflectance characteristics. Larger particles can more effectively reflect light in a specific band. When the cinnabar pigment particles are refined, the reflectivity of the red light band will decrease, which means that the intensity of the reflection peak will weaken.

[0101] It is understandable that the reflection peak of cinnabar pigment in the area with a greater degree of degradation is reduced, especially the reflectivity in the red light band is weakened. These changes destroy the original spectral characteristics of cinnabar and cause the reflection ability in the red light band to decrease. Therefore, by detecting and comparing the areas where the reflectivity in the red light band is significantly reduced, it can be determined that they are areas where the pattern color has faded and is missing.

[0102] Lapis lazuli (blue): The main component is a complex of labradorite and other minerals.

[0103] Lapis lazuli has high reflectivity in the blue and violet light bands (about 450-490nm), with obvious reflection peaks in the reflection spectrum, and usually shows a strong blue color.

[0104] Specific reference Figure 6 , Figure 6 A comparison diagram of the reflection spectra of a normal area and a severely degraded area of ​​lapis lazuli pigment under the same lighting conditions and reflection distance provided by an embodiment of the present invention.

[0105] The spectral reflectance characteristics of lapis lazuli in normal state show strong reflection in the blue band (about 450-490nm), and obvious absorption in the red band (about 620-750nm) and green band (about 500-550nm). This is determined by the crystal structure of the main component of lapis lazuli, blue quartz, especially the absorption and reflection characteristics of the sulfide ions inside it.

[0106] However, in the area where the pattern is missing or degraded, the blue reflection peak is lowered and the reflectivity decreases. This is mainly because ultraviolet rays will destroy the chemical bonds of the pigment. Under the influence of ultraviolet rays, the colored components in lapis lazuli will undergo photodegradation, weakening its reflective performance in the blue light band. As this photodegradation proceeds, the reflection peak in the blue band decreases. At the same time, due to the destruction of the surface chemical structure by photodegradation, the reflection characteristics also change and the reflectivity decreases.

[0107] It can be understood that in areas where the pattern color is missing or degraded, the degradation of the lapis lazuli pigment will cause the reflection peak in the blue band to decrease and the reflectivity to drop significantly. Therefore, by detecting and comparing the areas where the reflectivity in the blue light band drops significantly, it can be determined that they are areas where the pattern color has faded and is missing.

[0108] Ochre (reddish brown): The main component is natural iron oxide mineral.

[0109] The reflectivity is higher in the red and green light bands (about 580-640nm), especially in the red band region where it has strong reflection.

[0110] Specific reference Figure 7 , Figure 7 A comparison diagram of the reflection spectra of a normal area and a severely degraded area of ​​ochre pigment under the same lighting conditions and reflection distance provided by an embodiment of the present invention.

[0111] Under normal conditions, the optical properties of ochre are mainly reflected in the high reflectivity in the red and yellow bands (about 580-640nm), and the low absorbance in the blue band (about 450-490nm). The red and yellow reflection of ochre is mainly attributed to the chemical properties of iron oxide. However, when the pattern color is missing or the ochre is degraded, the absorbance in the red band decreases significantly, indicating that the structure of the ochre pigment has changed.

[0112] In areas that are more affected by the environment, iron oxide is prone to further chemical reactions to form other forms of iron oxides, such as hydrated iron oxide (FeO(OH)) or other secondary minerals. These newly formed compounds have different spectral reflectance characteristics compared to iron oxide (hematite), resulting in a decrease in its reflectivity in the red and yellow bands, which manifests as a decrease in reflectivity. This chemical transformation causes the pigment to gradually fade or lose its red and yellow appearance. Although iron oxide is relatively stable, its surface structure is destroyed under the influence of ultraviolet rays for a long time, affecting the light reflection performance of the pigment, which is manifested as a significant decrease in reflectivity in the red band (approximately 620-720nm).

[0113] It can be understood that in areas where the pattern color is missing or degraded, the degradation of the ochre pigment will cause the reflection peaks in the red and yellow bands to decrease, and the reflectivity will drop significantly. Therefore, by detecting and comparing the areas where the reflectivity in the blue light band drops significantly, it can be determined that they are areas where the pattern color is faded and missing.

[0114] Step 103: performing comparison processing on the reflection spectrum data of each scanning point, extracting at least one low reflectivity point, and calculating the reflection spectrum similarity between each low reflectivity point and each scanning point in an adjacent area.

[0115] On the basis of the above, in some embodiments of the present invention, the acquired spectral data is processed to extract scanning points with low reflectivity. The low reflectivity points are defined herein as points whose reflectivity in multiple color light bands (such as red, green, blue bands, etc.) is significantly lower than that of other scanning points.

[0116] Specifically, the reflectance spectrum data of each scanning point at different wavelengths is analyzed to find the reflectance at multiple color bands (such as red, green, blue bands, etc.) that is significantly lower than other scanning points in the same area. In some embodiments provided by the present invention, the threshold is set. Only when a pixel meets the low reflectivity in all key bands (red, green, blue) can it be marked as a low reflectivity point. The conditional formula is:

[0117]

[0118] For each detected low reflectivity point, marked as P i , record its position coordinates in the three-dimensional coordinate system (x i ,y i , z i ), taking these scan points as potential areas of pattern fading, missing, or severe degradation of painting materials, and confirming the extent of the missing area through subsequent similarity analysis.

[0119] Regarding similarity analysis, specifically, for each scanning point, the spectral camera measures the reflectivity of the pixel at a series of wavelengths. The spectral data collected from each scanning point forms a vector, which contains reflectivity information of multiple bands. The spectral vector is defined as:

[0120]

[0121] Where X is the spectrum vector of a certain scanning point;

[0122] Indicates that the pixel is at wavelength λ i The reflectivity at .

[0123] Take the murals of the Guge Dynasty ruins in Tibet as an example. The three common pigments are cinnabar, lapis lazuli and ochre. Their reflectivity in different bands varies, so their spectral vectors have significant differences in reflectivity values ​​in each band. When the murals fade or disappear, the reflectivity will be significantly reduced at all wavelengths, forming a low reflectivity spectral vector.

[0124] More specifically, when the pigments in the murals degrade or the patterns are missing, each component of the spectral vector, that is, the reflectance value of each band, such as the red light band of cinnabar, the blue light band of lapis lazuli, and the red and yellow light bands of ochre, will become significantly lower.

[0125] In the above, the low reflectivity point P is extracted i Based on each low reflectivity point P i As the center point, a neighborhood including several surrounding scanning points is defined. The size of the neighborhood can be set according to actual needs, usually determined by the scanning density and the spatial distribution of mural features.

[0126] Reference Figure 3 , Figure 3 A color image of a mural scanned from the ruins of the Guge Dynasty in Tibet. Figure 3 At A in the reference Figure 4 , a low reflectivity point P is identified at A i Now define a point with low reflectivity P i The area with the center point as the radius is the farthest distance between the three scanning points on the straight line, so as to focus on the local features and extract each scanning point P in the neighborhood. j The reflectance spectral characteristics of the image are taken into consideration, including the spectral vector X of each scanning point, and the spectral similarity analysis is performed to identify the scanning points in the neighborhood that are pattern missing or faded based on the spectral similarity.

[0127] Specifically, in some feasible embodiments of the present invention, cosine similarity is used to analyze the similarity of spectral vectors between various scanning points. Cosine similarity only considers the direction between spectral vectors. In the analysis of mural patterns, although the reflectivity of different scanning points is different, its relative proportion in each band is more important. Therefore, cosine similarity can more effectively capture the shape change of the spectral curve, rather than the absolute reflectivity value, avoiding the error introduced by the difference in reflectivity.

[0128] In addition, the spectral data of mural scans usually include multiple bands, forming high-dimensional spectral vectors. Cosine similarity has a good performance in processing high-dimensional vectors and can quickly calculate the similarity between vectors. Regardless of the number of bands, it can maintain a high computational efficiency. In the analysis of mural patterns, the reflectivity of adjacent scanning points fluctuates due to environmental factors (such as changes in illumination or instrument noise). Cosine similarity focuses on measuring the directional differences of vectors, so it has good robustness to these tiny fluctuations and will not be affected by small changes in reflectivity. Similarity judgment. Since cosine similarity pays more attention to the similarity of the shape of the spectral curve, it can effectively identify scanning points that are similar to the spectral curve of low-reflectivity points in local areas. This feature can help quickly find other affected scanning points in missing or degraded areas, which helps to determine the scope of the missing area.

[0129] Under multi-band (such as visible light and near-infrared light) scanning, the spectral reflectance of each scanning point is composed of reflectances at different wavelengths to form a spectral vector. Now set a scanning point P in the neighborhood j The spectral vector is Low reflectivity point P i The spectral vector is Calculated by the following formula:

[0130]

[0131] Among them: X·X low Represents the inner product of two vectors:

[0132]

[0133] ||X|| and ||X low || are the moduli (i.e. vector lengths) of the two vectors respectively.

[0134] The value of cosθ is close to 1, indicating that the spectral vectors of the two scanning points are highly similar, indicating that the scanning point in the neighborhood is also a point with lower reflectivity, indicating that it is in the pattern missing or faded area.

[0135] The value of cosθ is close to 0, indicating that the spectral vector is quite different, which means that the similarity between the scan point and the low reflectivity point in the neighborhood is low, and the original pattern may still be maintained.

[0136] The value of cosθ is close to -1, indicating that the directions of the two spectral vectors are completely opposite.

[0137] Step 104: The low reflectivity point and each scanning point in its neighborhood with high reflectivity spectrum similarity together form a complete area, and the area is marked in the three-dimensional point cloud data.

[0138] Based on the above cosine similarity calculation, the newly detected high similarity points are continuously used as the new center point P′ i , further expand the neighborhood, and repeat the similarity analysis until the complete missing region is found.

[0139] More specifically, for the new center point P′ i Each scan point P′ in the neighborhood of j The spectral vector of is used to perform cosine similarity calculation, and the calculated similarity value, i.e., the value of cosθ, is compared with the set cosine similarity threshold T. In the embodiment of the present invention, the cosine similarity threshold T is used to determine the new center point P′. i and other scan points P′ in the neighborhood jThe spectral similarity of the spectrum, the size of the T value can be set according to actual operation requirements and is not specifically limited here.

[0140] If the new center point P′ i and a scan point P′ in the neighborhood j Similarity: cosθ>T: the scan point P′ j Considered as the new center point P′ i Points with similar spectral characteristics are added to the area to be analyzed and set as the new center point P′ i .

[0141] If cosθ≤T: do not scan point P′ j Add the area to be analyzed.

[0142] Select a new center point (such as point P′ j ), and repeat the above steps to calculate the similarity of other scan points in the neighborhood of the new center point:

[0143] Continue to expand: If the similarity value is still greater than the threshold T, continue to expand the neighborhood.

[0144] Stop expansion: The similarity of all scan points in the domain is less than or equal to the threshold T, indicating that the boundary of the area has been determined.

[0145] Once expansion has stopped, record all scan points that are similar to the initial low reflectivity points that constitute the area where the loss or fading exists.

[0146] Through this cosine similarity-based region expansion method, missing or faded areas in the murals can be effectively identified, their boundaries can be accurately defined, and they can be marked in the collected and processed 3D point cloud data. In practical applications, this method can help identify and mark the areas of murals that need to be restored, allowing relevant researchers to better understand and maintain historical heritage.

[0147] In an embodiment of the present invention, a method for immersive generation of three-dimensional images of ruins is provided, which is used to model the data collected by scanning the ruins buildings. Figure 2 The flowchart shown specifically includes:

[0148] Step 201: After performing point cloud denoising, point cloud alignment and point cloud simplification on the three-dimensional point cloud data with image color information obtained after scanning the ruins building, the simplified point cloud data is converted into a triangular mesh, that is, the surface structure is generated through the adjacent relationship between points. In some feasible embodiments of the present invention, the three-dimensional point cloud data with image color information obtained by scanning is denoised, because some noise points may be generated during the scanning process. These points do not belong to the surface of the object and need to be removed by an algorithm. If multiple viewing angles are used when scanning an object or scene, the point cloud data at different angles need to be aligned (for example, using the ICP algorithm) to generate a complete model. Each point of the point cloud data is independent, and there is no direct connection between points (such as edges between triangular facets). Therefore, it cannot directly represent the surface of an object, but represents a series of discrete points. The point cloud can be converted into a continuous surface model through a surface reconstruction algorithm (such as Poisson Surface Reconstruction). Figure 8 As shown, Figure 8 A triangular mesh model generated based on a 3D point cloud data.

[0149] It is understandable that the simplified 3D point cloud data is used to construct a triangular mesh according to the adjacency relationship between points. The generation of the triangular mesh makes the geometric structure of the surface of the ruins more coherent and can better display the details through the facets. The digital reconstruction of the surface structure of the ruins is achieved through the triangular mesh, making it have a realistic geometric form and suitable for immersive experience.

[0150] Step 202: Map the image color information in the three-dimensional point cloud to the generated triangular mesh.

[0151] Specifically, the color information in the original 3D point cloud (obtained from the color image) is accurately mapped to the generated triangular mesh facets. In this way, each mesh facet not only carries geometric information but also has surface color attributes, ensuring that the model visually presents the true appearance of the ruins.

[0152] Step 203: The reflectance spectrum data of the ruins building surface reflecting multiple different wavelengths of light obtained at each scanning point is combined with the surface properties of each triangular mesh facet, and the area formed by the low reflectance points in the reflectance spectrum and the scanning points with similar reflectance information in their neighborhood are marked in the triangular mesh.

[0153] Specifically, the spectral data of light reflected from the ruins surface at different wavelengths collected previously were combined to analyze the reflectance spectral characteristics of each mesh patch. The spectral data provides additional information about the surface material, and the color missing areas of the mural pattern can be analyzed and identified by comparing the reflectance of multiple color bands.

[0154] It can be understood that, through the above-mentioned cosine similarity-based region expansion method, the areas in the mural where the pattern is missing or faded are identified, their boundaries are accurately defined, and these areas where the pattern is missing or faded are marked on the three-dimensional model for display or reference analysis by researchers.

[0155] Based on the above, it can be extended to understand that by importing the triangular mesh model with color information and spectral markings after the above processing into an immersive display system, such as a virtual reality (VR) or augmented reality (AR) system, the user can view the missing or faded areas of the mural patterns marked after spectral analysis in three-dimensional space, immersively experience the real state of the ruins building, and understand the details such as the degradation and damage of its surface materials.

[0156] By combining point cloud and spectral data, we can not only reconstruct the three-dimensional geometric structure of the ruins buildings, but also identify the missing colors of the patterns through spectral analysis.

[0157] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. The patent protection scope of the present invention shall be based on the claims. All equivalent structural changes made using the contents of the description and drawings of the present invention should also be included in the protection scope of the present invention.

Claims

1. A method for collecting three-dimensional images of ruins, characterized in that: include: Obtaining three-dimensional point cloud data generated by scanning the ruins building with a three-dimensional scanner; While acquiring the three-dimensional point cloud data, at each scanning point, the reflectance spectrum data of the surface of the ruins building under multiple different wavelengths of light are acquired; Comparative processing is performed on the reflectance spectrum data of each scanning point, at least one low reflectance point is extracted, and the reflectance spectrum similarity between each low reflectance point and each scanning point in the adjacent area is calculated; wherein the low reflectance point indicates that the reflectance of a certain scanning point in multiple color light bands is lower than the reflectance of other scanning points in the corresponding color light bands; the calculation of the reflectance spectrum similarity is specifically to compare the reflectance information of the low reflectance point and each scanning point in its neighborhood in different color bands, and calculate the reflectance spectrum similarity; as well as The low reflectivity point and each scanning point in its neighborhood with high reflectivity spectrum similarity together form a complete area, and the area is marked in the three-dimensional point cloud data.

2. A method for collecting three-dimensional images of ruins according to claim 1, characterized in that: The step of obtaining three-dimensional point cloud data generated by scanning the ruins building with a three-dimensional scanner includes: At each scanning position, a Cartesian coordinate system is constructed with the position of the three-dimensional scanner as the origin; wherein the three axes in the coordinate system respectively represent the horizontal direction, the vertical direction and the distance information from the scanned object to the three-dimensional scanner within the scanning range; The three-dimensional scanner emits pulsed lasers and calculates the flight time of the lasers reflected from the building surface, thereby calculating the distance from each laser reflection point on the surface of the ruins to the three-dimensional scanner; The distance data of the laser reflection point on the surface of each site is combined with the corresponding horizontal angle and vertical angle to calculate the three-dimensional coordinates of the point in the Cartesian coordinate system. Each of the three-dimensional coordinate points is recorded as independent three-dimensional point cloud data and together they form a point cloud model of the entire site surface.

3. The method for collecting three-dimensional images of ruins according to claim 2, characterized in that: While performing the three-dimensional scanning, a color image of the current site surface is captured by a camera, and for each scanning point in the three-dimensional point cloud data, at least one pixel value in the color image is matched; specifically, In the three-dimensional coordinate system, each three-dimensional point corresponds to at least one two-dimensional pixel coordinate; By setting the intrinsic parameters of the camera, the two-dimensional pixel coordinates are converted into the coordinates of the three-dimensional point; For each of the three-dimensional points, its corresponding image pixel is found, and the color value of the pixel is obtained and then assigned to the three-dimensional point to form three-dimensional point cloud data with color information.

4. The method for collecting three-dimensional images of ruins according to claim 2, characterized in that: While performing the three-dimensional scanning, a plurality of lights of different wavelengths are selected to illuminate the surface of the ruins building, and reflection spectrum data at different wavelengths are acquired at each scanning point, wherein the reflection spectrum data includes reflectivity information in different color bands.

5. The method for collecting three-dimensional images of ruins according to claim 4, characterized in that: After comparing and analyzing the reflectance spectrum data of each scanning point, a plurality of low reflectance points P are marked. i , calculate each low reflectivity point P i The position coordinates (x i ,y i , z i ); For each marked low reflectivity point P i , define a point P with low reflectivity i is the neighborhood of the center point, and each scanning point P in the neighborhood is extracted j Reflectivity information; For each scanning point, the collected reflectance spectrum data is formed into a spectrum vector X, which contains the reflectance in multiple color bands: in, Indicates that the scanning point is at wavelength λ i The reflectivity at Calculate the low reflectivity point P i The spectrum vector With a scanning point P in its area j The spectrum vector The cosine similarity cosθ is: Among them, X·X low Represents the inner product of two vectors: ||X|| and ||X low || are the moduli of the two vectors (i.e., vector lengths); The value of cosθ is close to 1, indicating that the spectral vectors are highly similar, indicating that the scan point P in the neighborhood j With low reflectivity point P i The reflectance information in different color bands is highly similar.

6. A method for collecting three-dimensional images of ruins according to claim 5, characterized in that: If there is a scanning point P in the neighborhood j The reflectivity is low and close to the low reflectivity point P i have similar reflectivity information, then the scanning point P j As the new said center point, continue to expand; Redefine the neighborhood for the new center point and perform the reflection spectrum similarity calculation until the cosine similarity cosθ between the new center point and all scanning points in the neighborhood is lower than the set cosine similarity threshold value and then stop expanding; After the expansion stops, all low reflectivity points P in the expanded area i and the scanning point P with similar reflectivity information in its neighborhood j , together constitute a complete area, this area is defined as the pattern color missing area on the surface of the ruins, and the pattern color missing area is marked in the three-dimensional point cloud data.

7. A method for immersive generation of three-dimensional images of ruins, characterized in that: include: After performing point cloud denoising, point cloud alignment and point cloud simplification on the three-dimensional point cloud data with image color information obtained after scanning the ruins buildings, the simplified point cloud data is converted into a triangular mesh, that is, the surface structure is generated through the adjacent relationship between points; Mapping the image color information in the three-dimensional point cloud to the generated triangular mesh; The reflectance spectrum data of multiple different wavelengths of light reflected by the surface of the ruins building obtained at each scanning point is combined with the surface properties of each triangular mesh facet, and the area composed of the low reflectance points in the reflectance spectrum and the scanning points with similar reflectance information in their neighborhood are marked in the triangular mesh.

8. A three-dimensional image acquisition system for ruins, characterized in that: include: The data collection end is configured as: Obtaining three-dimensional point cloud data generated by scanning the ruins building with a three-dimensional scanner; While acquiring the three-dimensional point cloud data, at each scanning point, the reflectance spectrum data of the surface of the ruins building under multiple different wavelengths of light are acquired; Comparing the reflection spectrum data of each scanning point, extracting at least one low reflectivity point, and calculating the reflection spectrum similarity between each low reflectivity point and each scanning point in an adjacent area; and The low reflectivity point and each scanning point in its neighborhood with high reflectivity spectrum similarity together form a complete area, and the area is marked in the three-dimensional point cloud data.

9. A system for generating immersive three-dimensional images of ruins, characterized in that: include: The data generation end is configured as: After performing point cloud denoising, point cloud alignment and point cloud simplification on the three-dimensional point cloud data with image color information obtained after scanning the ruins buildings, the simplified point cloud data is converted into a triangular mesh, that is, the surface structure is generated through the adjacent relationship between points; Mapping the image color information in the three-dimensional point cloud to the generated triangular mesh; The reflectance spectrum data of multiple different wavelengths of light reflected by the surface of the ruins building obtained at each scanning point is combined with the surface properties of each triangular mesh facet, and the area composed of the low reflectance points in the reflectance spectrum and the scanning points with similar reflectance information in their neighborhood are marked in the triangular mesh.

Images