Method for automatically generating orthographic expansion drawing of arched top mural based on three-dimensional fine color model

Abstract

The invention discloses a three-dimensional fine color model-based arched top mural orthographic expansion automatic generation method, which comprises the following steps of: constructing a three-dimensional fine color model of an arched top mural, segmenting the three-dimensional fine color model, independently partitioning different reference surfaces, and carrying out geometric fitting on the three-dimensional fine color model to obtain geometric parameters of the three-dimensional fine color model; the three-dimensional fine color model is subjected to line scanning orthographic expansion, in the line scanning orthographic expansion process, the three-dimensional fine color model is subjected to one-time pose transformation every time scanning is conducted, a fringe projection drawing is generated, and pose transformation parameters are related to geometric parameters; and the fringe projection drawings generated by the line scanning orthographic projection are spliced in sequence according to the sequence obtained by line scanning, and an orthographic expanded drawing of the arched top mural is generated. According to the method, the conventional curved surface expansion projection thinking is broken through, and the geometric property of the model is fully utilized to unfold the three-dimensional fine color model, so that the obtained two-dimensional orthographic expansion drawing is smaller in deformation, more uniform and higher in efficiency.

Description

Technical Field

[0001] The invention belongs to the field of digital protection of ancient buildings, aims at automatically unfolding curved surface paintings in the cultural heritage of ancient buildings, and relates to a method for automatically generating an orthographic unfolding diagram of a dome ceiling mural based on a three-dimensional fine color model. Background Art

[0002] As a treasure of cultural heritage, murals carry profound historical heritage and inestimable research value. With the development of hardware equipment and computer technology, the digitization of murals, especially the fine three-dimensional digitization, has become the primary content of mural technology protection. The digital products generated by data mining have served the fine preservation, digital display, 3D restoration and other aspects. Among them, the orthophoto of the mural is a very important digital achievement. The orthophoto of the mural is a high-precision, distortion-free two-dimensional image generated by photogrammetry and geometric correction of the mural. It accurately reflects the true form and proportion of the mural by eliminating the deformation caused by perspective and curved surface. The orthophoto of the mural is of great value in the fields of archaeology, art restoration, historical research, etc. At present, there are two main methods for obtaining the orthophoto of the mural: one is the correction method based on a single photo, which usually involves a series of digital image processing technologies, aiming to remove the perspective distortion from a single photographic image so that each part of the image matches the real size and shape of the actual object, thereby obtaining the orthophoto of the mural. However, for special murals that are extremely complex or not flat, this method cannot achieve the best results; the other is directional orthophoto projection based on three-dimensional color models, which combines three-dimensional modeling and high-precision color capture technology to generate high-quality orthophotos of murals. The core of this method is to use three-dimensional scanning technology to capture the geometry and surface details of the murals, while combining high-resolution color photography technology to record the color and texture of the murals. However, this method consumes a lot of manpower, material and financial resources throughout the process, and in some cases, the location of the murals may be difficult to access. For cultural heritage protection professionals, it is necessary to weigh the cost, benefit and feasibility of this method. The above method is feasible for most flat murals, but for murals like dome-shaped murals, their orthophotos will cause great geometric and texture deformation, which is not suitable for display and further research. Summary of the invention

[0003] An object of the present invention is to solve at least the above problems and / or disadvantages and to provide at least the advantages which will be described hereinafter.

[0004] Another object of the present invention is to provide a method for automatically generating an orthographic expansion diagram of a dome ceiling mural based on a three-dimensional fine color model.

[0005] To this end, the technical solution provided by the present invention is:

[0006] A method for automatically generating an orthographic development diagram of a dome ceiling mural based on a three-dimensional fine color model comprises the following steps:

[0007] 1) Collect 3D laser scanning point clouds of ancient building dome murals and build 3D detailed color models of the dome murals.

[0008] 2) segmenting the three-dimensional fine color model of the dome ceiling mural, dividing different reference planes into separate blocks, and performing geometric fitting on the three-dimensional fine color model to obtain geometric parameters of the three-dimensional fine color model of the dome ceiling mural;

[0009] 3) performing line scanning orthophoto expansion on the three-dimensional fine color model of the dome ceiling mural. During the line scanning orthophoto expansion process, each time the three-dimensional fine color model of the dome ceiling mural is scanned, a posture transformation is performed on the three-dimensional fine color model of the dome ceiling mural to generate a fringe projection image, and the posture transformation parameters are related to the geometric parameters of the three-dimensional fine color model of the dome ceiling mural;

[0010] The fringe projection images generated by the line scanning orthographic projection are stitched together in the order in which the line scanning is acquired to generate an orthographic unfolded image of the dome ceiling mural.

[0011] Preferably, in the method for automatically generating an orthographic development diagram of a dome ceiling mural based on a three-dimensional fine color model, step 3) comprises the following steps:

[0012] 3.1) Performing corresponding quaternion transformation according to the positional relationship between the camera and the three-dimensional fine color model to adjust the model posture: First, the three-dimensional fine color model of the dome ceiling mural of the model is imported into the world coordinate system to obtain the world coordinates, and then the quaternion transformation is used to adjust the model posture. The quaternion calculation process includes:

[0013] First, the axial direction of the cylinder is obtained through the geometric parameters of the three-dimensional fine color model of the dome ceiling mural as the current orientation of the model. The cylinder axis vector and the world coordinate system Z axis vector are recorded as A and B respectively. The rotation angle θ is obtained by calculating the angle between the current orientation vector and the target orientation vector, which is realized by the vector dot product formula:

[0014]

[0015] At the same time, the vector product is used to find the rotation axis, which is perpendicular to the plane defined by the current heading vector and the target heading vector. The rotation axis axis is calculated by the cross product:

[0016] axis=A×B (4)

[0017] The three-dimensional fine color model of the dome ceiling mural is rotated by quaternion. Assuming there is a unit quaternion q=w+xi+yj+zk, where w, x, y, z are real numbers and i, j, k are imaginary units, the unit quaternion is calculated using the rotation axis and rotation angle:

[0018]

[0019] Where (x, y, z) is the three-dimensional vector of the rotation axis, and θ is the rotation angle;

[0020] Convert this quaternion into a rotation matrix using the following formula:

[0021]

[0022] Each time a scan is performed, the three-dimensional fine color model undergoes posture transformation according to the preset rotation axis to generate a fringe projection image.

[0023] Preferably, in the method for automatically generating an orthographic development image of a dome ceiling mural based on a three-dimensional fine color model, in step 3.1), the three-dimensional fine color model of the dome ceiling mural is first transformed in three-dimensional space according to the following matrix and then imported into the world coordinate calculation:

[0024]

[0025] Among them, r ij Represents the element of the rotation matrix, t x ,t y and t z represent the translation components along the x, y, and z axes respectively.

[0026] Preferably, in the method for automatically generating an orthographic development diagram of a dome ceiling mural based on a three-dimensional fine color model, in step 3.1), the method for calculating the world coordinates is:

[0027] P world =M n ·M n-1 ·…·M1·P local (2)

[0028] Among them, M i is the transformation matrix of the i-th node, P local is a point in the local coordinate system, P world is a point in the world coordinate system.

[0029] Preferably, in the method for automatically generating an orthographic development diagram of a dome ceiling mural based on a three-dimensional fine color model, step 3) further comprises the following steps:

[0030] 3.2) Adjustment of the camera viewport: Calculate the width and height of the camera viewport according to the required resolution, and then input them into the viewport transformation matrix to automatically adjust the size of the camera viewport. The viewport transformation matrix is:

[0031] M final =M viewport *M NDC (9)

[0032] in,

[0033]

[0034] n, d, c, respectively represent the position of coordinate mapping,

[0035] Map coordinates in the range [0,1] to screen coordinates:

[0036]

[0037] Among them, left and bottom define the position of the lower left corner of the camera viewport in screen coordinates, width and height are the width and height of the camera viewport, near and far define the depth range, depth = far-near.

[0038] Preferably, in the method for automatically generating an orthographic development diagram of a dome ceiling mural based on a three-dimensional fine color model, step 3) further comprises the following steps:

[0039] 3.3) Line scanning orthographic projection: by rotating the three-dimensional fine color model of the dome ceiling mural, the camera projects along the surface of the three-dimensional fine color model in a line scanning manner to obtain a set of fringe projection images, each image corresponding to the projection of the three-dimensional fine color model at different angles, and each time the three-dimensional fine color model of the dome ceiling mural undergoes a posture transformation;

[0040] 3.4) Fringe projection image stitching: For the stitching of quasi-cylindrical fringe projection images, firstly, the coordinates of the fringe projection images are matched, and then the edges of each fringe image are processed to align them correctly. Then, the edges of adjacent fringe images are fused. After all fringe images are aligned and fused, they are finally stitched into a complete orthophoto development image.

[0041] For the splicing of spherical fringe projection images, a fan-shaped splicing method is used to fix the coordinates of the fan-shaped center point of the fringe projection image to keep it stationary, and the rotation splicing is performed with the fan-shaped center point as the axis, and the adjacent images are registered to make the texture continuous, and the expanded orthophoto image is output. During the splicing process, the rotation angle increment of the fringe projection image during fan-shaped splicing is equal to the rotation angle increment of the three-dimensional fine color model during line scanning;

[0042] Until the orthophoto of the three-dimensional fine color model is completed.

[0043] Preferably, in the method for automatically generating an orthographic development image of a dome ceiling mural based on a three-dimensional fine color model, the step 2) of obtaining geometric parameter information of the three-dimensional fine color model of the dome ceiling mural comprises:

[0044] For quasi-cylindrical surfaces: Establish the error equation

[0045] f=(x-x1) 2 +(y-y1) 2 +(z-z1) 2 -

[0046] [a(x-x1)+b(y-y1)+c(z-z1)] 2 -r 2 (10)

[0047] Where, f is the error equation; (x1, y1, z1) is the coordinate of the center of the fitted circle; (a, b, c) is the initial value of the cylinder axis vector; r is the radius of the cylinder;

[0048] make:

[0049] f={(x-x1) 2 +(y-y1) 2 +(z-z1) 2 -[a(x-x1)+b(y-y1)+c(z-z1)] 2 -r 2} / 2r(11)

[0050] Linearize f:

[0051]

[0052] The error equation can be listed as (that is, taking the partial derivative with respect to x1):

[0053]

[0054] in:

[0055]

[0056] Write the error equation in matrix form to minimize V and make it close to 0:

[0057] V n×1 =B n×7 X 7×1 -L n×1 (15)

[0058] Among them, V, B, X, and L are all matrices, and the lower right corner mark is the number of rows and columns of the matrix.

[0059] Preferably, in the method for automatically generating an orthographic development image of a dome ceiling mural based on a three-dimensional fine color model, the step 2) of obtaining geometric parameter information of the three-dimensional fine color model of the dome ceiling mural comprises: for a spherical surface:

[0060] Initialization: Select an initial sphere center position (a, b, c) and radius r;

[0061] Iterative optimization: Use numerical optimization algorithms to minimize RSS.

[0062]

[0063] Among them, f(x,β) is used to describe the relationship between y and x. The r in the formula i Represents the residual of the i-th observation, that is, the actual value y i and the model prediction value f(x i ,β), β is the parameter vector of the model, expressed as (a, b, c, r), which is used to describe the relationship between the independent variable x and the dependent variable y;

[0064] Convergence judgment: When the change of parameter update is less than the preset threshold or reaches the preset number of iterations, stop the iteration; Result output: Output the final parameters (a, b, c, r) as the parameters of the best fitting ball.

[0065] The present invention has at least the following beneficial effects:

[0066] The present invention uses a three-dimensional fine color model to obtain a surface painted two-dimensional orthophoto expansion map. First, according to the geometric characteristics of the three-dimensional fine color model, the model is geometrically fitted accordingly to obtain the geometric parameters of the model; secondly, the viewport texture is rendered using the model geometric features combined with line scanning to obtain a strip projection map, and the texture deformation is controlled to a minimum in the single curvature direction; finally, in the process of splicing the cylindrical fringe projection map, image registration is required to ensure that they can be correctly aligned and finally spliced ​​into a complete orthophoto expansion map. In the process of splicing the spherical fringe projection map, it is necessary to use a fan projection method to align the fringe projection map to achieve visual consistency and continuity. The final fan-shaped orthophoto expansion map after splicing is obtained. The present invention breaks the previous thinking of curved surface expansion projection, and makes full use of the geometric properties of the model to expand the three-dimensional fine color model, so that the obtained two-dimensional orthophoto expansion map is less deformed, more uniform, and more efficient.

[0067] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 The present invention provides a technical roadmap for automatically generating an orthophoto of a dome ceiling mural based on a three-dimensional fine color model in an embodiment of the present invention.

[0069] Figure 2 Schematic diagram of model quaternion transformation in an embodiment of the present invention.

[0070] Figure 3 It is a roadmap of the quasi-cylindrical line scanning technology in the embodiment of the present invention.

[0071] Figure 4 Schematic diagram of quasi-cylindrical line scanning in an embodiment of the present invention.

[0072] Figure 5 A schematic diagram of cylindrical rectangular splicing in an embodiment of the present invention is shown.

[0073] Figure 6 This is a spherical line scanning technology roadmap in an embodiment of the present invention.

[0074] Figure 7 Schematic diagram of spherical line scanning in an embodiment of the present invention.

[0075] Figure 8 This is a technical roadmap for spherical fan-shaped splicing in an embodiment of the present invention.

[0076] Fig. 9 A schematic diagram of spherical fan-shaped splicing in an embodiment of the present invention is shown.

[0077] Fig.10 A comparison chart showing the results of different cylindrical expansion methods in an embodiment of the present invention is shown.

[0078] Fig.11 A comparison chart showing the results of different spherical surface expansion methods in an embodiment of the present invention is shown; wherein a is a Gaussian projection, b is a Mercator projection, c is an orthographic projection, and d is the method of this paper.

[0079] Fig.12 A comparative analysis diagram of spherical quantitative data in an embodiment of the present invention is shown; and Fig.11 Corresponding from top to bottom. DETAILED DESCRIPTION

[0080] The present invention will be further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0081] It should be understood that terms such as “having”, “including” and “comprising” used herein do not specify the existence or addition of one or more other elements or combinations thereof.

[0082] The present invention provides a method for automatically generating an orthographic development diagram of a dome ceiling mural based on a three-dimensional fine color model, comprising:

[0083] Step 1: Collect the 3D laser scanning point cloud of the 3D fine color model of the ancient building, and perform downsampling and denoising processing to build a 3D fine color model of the ancient building, hereinafter referred to as the model.

[0084] Step 2: Segment the ancient building's three-dimensional fine color model, separate different reference surfaces into blocks, and perform geometric fitting on the reference surfaces according to their geometric characteristics to obtain the model's geometric parameters. Specifically, it includes:

[0085] Ⅰ. The three-dimensional fine color model of the ancient building is divided according to different reference planes. Each reference plane has its own corresponding geometric shape. Different operations are performed on different reference planes according to their different geometric characteristics.

[0086] Ⅱ. According to the geometric characteristics of each reference surface, different fitting methods are selected to perform geometric fitting on it to obtain the geometric parameter information of the model.

[0087] III. Visualize the three-dimensional fine color model in the OSG three-dimensional rendering engine and ortho-expand the three-dimensional fine color model.

[0088] Optionally, the reference plane is determined based on the normal information of the model point cloud. For example, when going from one wall to another wall, the normal direction of the point cloud will change greatly (the normal direction of the vertical wall will change by about 90°).

[0089] The different geometric characteristics of different reference surfaces mentioned above refer to whether the reference surface is a plane, a cylinder or a sphere. Different operations are performed on different reference surfaces, which means plane fitting for plane-like surfaces, cylinder fitting for cylinder-like surfaces and spherical fitting for spherical-like surfaces, to obtain the geometric parameters of the model. This will be described in detail later.

[0090] Step 3, set the initial pose, set the rotation angle increment, set the stitching parameters, and use the OSG camera to perform line scanning orthographic projection on the three-dimensional fine color model (the model needs to add two clipping planes in the world coordinate system, and the model is first clipped to reduce the complexity of data processing, and optimize the allocation of system resources to ensure the maximum computational efficiency.). The stitching parameters described here include two aspects. First, the stitching parameters of the cylindrical expansion are the left side of the lower left corner of the fringe projection image, and the left side of the lower left corner of the fringe projection image is incremented each time, and each increment is the width of a fringe projection image; for example, the width of the fringe projection image is doubled by the coordinates of the lower left corner of the second fringe projection image, and the width of the fringe projection image is increased by (n-1) fringe projection images for the nth fringe projection image, ensuring seamless connection between fringe projection images. Second, the splicing parameters of the spherical surface expansion, when the fan-shaped fringe projection images are spliced, the coordinates of the center of each fringe projection image are kept unchanged. Each time the fringe projection image is spliced, the fringe projection image is rotated by a small angle (the angle here is equal to the angle of rotation of the spherical model each time the line scan is performed), and the angle is also increased each time to ensure the continuity of the fan-shaped splicing. In general, the splicing parameters are the increase in coordinates and angles during the splicing process of the fringe projection images.

[0091] Specifically, step three includes:

[0092] S1. Set the quaternion angle transformation and adjust the model posture so that the camera viewport faces the model front view.

[0093] S2. During the online scanning orthophoto expansion process, each time the model is scanned, the posture is transformed according to the preset rotation axis to generate a fringe projection image.

[0094] S3. Stitching the fringe projection images generated by the line scanning orthographic projection, and stitching them in sequence according to the order of line scanning acquisition (the stitching of the spherical model needs to adopt the fan-shaped stitching method).

[0095] Specifically, it includes: 3.1) Importing the model into the world coordinate system, obtaining the world coordinates, and using the quaternion transformation to adjust the model's position and posture: Assume that there is a unit quaternion (that is, its norm is 1) q = w + xi + yj + zk, where w, x, y, z are real numbers and i, j, k are imaginary units. It represents a rotation of θ radians around the unit vector (x, y, z). This quaternion can be calculated from the rotation axis and rotation angle: Where (x, y, z) is the three-dimensional vector of the rotation axis and θ is the rotation angle.

[0096] 3.2) Viewport matrix transformation: The viewport size is calculated based on the resolution, and the model scanning area is divided into blocks so that the generated orthophoto image reaches the required resolution;

[0097] 3.3) Line Scan Orthographic Projection: In this process, we rotate the model by a small angle so that the camera projects along the surface of the model in a line scan manner. In this way, we obtain a set of fringe projection images, each image corresponding to the projection of the model at a different angle. These fringe projection images are obtained by precisely controlling the camera rotation angle and line scanning method, ensuring the continuity and integrity of the projection.

[0098] 3.4) Fringe projection image stitching: For the stitching of cylindrical fringe projection images, first, process the edges of each fringe image to ensure that they can be correctly aligned; then, fuse the edges of adjacent fringe images to ensure seamless and natural stitching; after all fringe images are aligned and fused, they are finally stitched into a complete orthophoto expansion image. For the stitching of spherical fringe projection images, it is necessary to use the fan-shaped stitching method. In the process of the previous line scan, the increment of the rotation angle of each line scan model is equal to the increment of the rotation angle when the fringe projection image is fan-shaped stitched; in this way, it is ensured that the texture information of the fringe projection image can be accurately rendered when stitching. Finally, the high-resolution orthophoto image after expansion is output.

[0099] In one embodiment of the present invention, preferably, in step 3.1), the transformation is performed in three-dimensional space according to the following matrix:

[0100]

[0101] Among them, r ij Represents the elements of the rotation matrix, t x ,t y and t z represent the translation components along the x, y, and z axes respectively.

[0102] In one embodiment of the present invention, preferably, in step 3.1), the method for calculating the world coordinates is:

[0103] P world =M n ·M n-1 ·…·M1·P local (2)

[0104] Among them, M i is the transformation matrix of the i-th node, P local is a point in the local coordinate system, P world is a point in the world coordinate system.

[0105] In one embodiment of the present invention, as a preferred embodiment, in step 3.1), first, the axial direction of the cylinder is obtained by fitting the geometric parameters obtained by the point cloud, that is, the current orientation of the model. The cylinder axis vector and the world coordinate system Z axis vector are denoted as A and B respectively. The rotation angle θ can be obtained by calculating the angle between the current orientation vector and the target orientation vector, which can be achieved by the vector dot product formula:

[0106]

[0107] The cross product can be used to find the rotation axis, which is perpendicular to the plane defined by the current heading vector and the target heading vector. The rotation axis can be calculated by the cross product:

[0108] axis=A×B(4)

[0109] After determining the rotation axis and rotation angle, the model is rotated by quaternion. Assume there is a unit quaternion (that is, its norm is 1) q = w + xi + yj + zk, where w, x, y, z are real numbers and i, j, k are imaginary units. It represents a rotation of θ radians around the unit vector (x, y, z). This quaternion can be calculated from the rotation axis and rotation angle:

[0110]

[0111] Where (x, y, z) is the three-dimensional vector of the rotation axis and θ is the rotation angle.

[0112] Convert this quaternion into a rotation matrix using the following formula:

[0113]

[0114] In one embodiment of the present invention, preferably, in step 3.2), the output resolution is adjusted based on the viewport transformation to meet the high resolution requirement:

[0115] The viewport transform is the process of mapping normalized device coordinates (NDC) to window or screen coordinates. The input parameters tell the rendering pipeline how to map NDC to screen coordinates. NDC is a cube that goes from -1 to 1 in all directions, and the viewport transform maps this cube to a rectangular area of ​​the screen.

[0116]

[0117] n, d, c, respectively represent the position of the coordinate mapping.

[0118] Map coordinates in the range [0,1] to screen coordinates:

[0119]

[0120] Among them, left and bottom define the position of the lower left corner of the viewport in screen coordinates, width and height are the width and height of the viewport, near and far define the depth range, depth = far-near.

[0121] The final viewport transformation matrix is ​​the product of these two matrices:

[0122] M final =M viewport *M NDC (9)

[0123] According to the resolution we need, we can calculate the width and height of the viewport and then input them into the transformation matrix to achieve the adaptive viewport size.

[0124] In one embodiment of the present invention, as a preferred step 3.3), the model is subjected to line scanning orthographic projection. In this process, the model is rotated at a small angle so that the camera projects along the surface of the model in a line scanning manner. In this way, the image in the frame buffer object is temporarily stored in the CPU, and a set of fringe projection images are obtained, each image corresponding to the projection of the model at different angles. The acquisition of these fringe projection images is achieved by precisely controlling the camera rotation angle and line scanning method, ensuring the continuity and integrity of the projection.

[0125] In one embodiment of the present invention, as a preference, step 3.4), stitching after the cylindrical line scan orthophoto expansion: first, process the edges of each fringe image to ensure that they can be correctly aligned. For example, the stitching parameters for the cylindrical expansion are the left side of the lower left corner of the fringe projection image, and the left side of the lower left corner of the fringe projection image is incremented each time, and each increment is the width of a fringe projection image; for example, the lower left corner coordinate of the second fringe projection image doubles the width of the fringe projection image, and the nth fringe projection image increases the width of the fringe projection image by (n-1) fringe projection images, thereby ensuring seamless connection between fringe projection images. This involves edge detection and image registration technology to facilitate the identification of features that can be aligned; then, the adjacent fringe images are registered to ensure seamless and natural stitching; after all fringe images are processed, they are finally stitched into a complete orthophoto expansion image.

[0126] In one embodiment of the present invention, as a preference, step 3.4), stitching after orthophoto expansion of spherical line scan: when performing projection image stitching, a fan-shaped stitching method needs to be adopted. During the model rotation process, the angle increment of each rotation should be equal to the rotation angle increment when the fringe projection image is stitched, and then the adjacent images are aligned to ensure texture continuity. For example, for the stitching parameters of spherical expansion, when the fan-shaped fringe projection image is stitched, the coordinates of the center of the fan circle of each fringe projection image are kept unchanged. Each time the fringe projection image is stitched, the fringe projection image is rotated by a small angle (the angle here is equal to the angle of rotation of the spherical model each time the line scan is performed), and the angle is also increased each time, ensuring the continuity of the fan-shaped stitching. In this way, it is ensured that each projection image can be accurately aligned when stitching until the orthophoto expansion image of the entire model is completed.

[0127] The present invention discloses a method for automatically generating an orthophoto expansion map of a dome ceiling mural based on a three-dimensional fine color model, comprising: obtaining a three-dimensional fine color model of an ancient building; segmenting the three-dimensional fine color model according to different reference surfaces, and dividing each reference surface into blocks separately; performing geometric fitting on the three-dimensional fine color model according to the geometric characteristics of the reference surfaces of the separate blocks, and obtaining the geometric parameter information of the model; inputting the model and the model geometric parameter information into an algorithm, performing corresponding quaternion transformation according to the positional relationship between the camera and the model, and adjusting the model posture; inversely calculating the size of the viewport according to the required resolution, and adjusting the viewport; setting the model rotation angle increment in the process of line scanning orthophoto projection according to the size of the viewport, and generating a fringe projection map; finally, obtaining a two-dimensional high-resolution orthophoto map of the three-dimensional fine color model by matrix splicing (a method of using fan-shaped splicing for a spherical surface). The present invention breaks the previous thinking of curved surface expansion projection, and makes full use of the geometric properties of the model to expand the three-dimensional fine color model, so that the obtained two-dimensional orthophoto expansion map is less deformed, more uniform, and more efficient.

[0128] Specifically, the present invention provides a method for automatically generating an orthographic development diagram of a dome ceiling mural based on a three-dimensional fine color model, comprising:

[0129] Step 1: Collect the 3D laser scanning point cloud of the 3D fine color model of the ancient building, and perform downsampling and denoising to build the 3D fine color model of the ancient building. The experimental object in this experiment is the dome in the 3D fine color model.

[0130] Step 2: After obtaining the three-dimensional fine color model, segment each reference surface, perform geometric fitting on the model according to the geometric characteristics of the reference surface, and obtain the model geometric parameter information. The details are as follows:

[0131] Quasi-cylinder: Establish the error equation.

[0132] f=(x-x1) 2+(y-y1) 2 +(z-z1) 2 -[a(x-x1)+b(y-y1)+c(z-z1)] 2 -r 2 (10)

[0133] Among them, f is the error equation; (x1, y1, z1) are the coordinates of the fitted center of the circle; (a, b, c) are the initial values ​​of the cylinder axis vector. The direction of the axis vector can be used to obtain the position information of the cylinder. When the model posture is transformed later, it is used as a reference for rotating the model until the axis is parallel to the Z axis; r is the radius of the cylinder.

[0134] make:

[0135] f={(x-x1) 2 +(y-y1) 2 +(z-z1) 2 -[a(x-x1)+b(y-y1)+c(z-z1)] 2 -r 2} / 2r(11)

[0136] Linearize f:

[0137]

[0138] The error equation can be listed as (that is, taking the partial derivative with respect to x1):

[0139]

[0140] in:

[0141]

[0142] Write the error equation in matrix form to minimize V and make it close to 0:

[0143] V n×1 =B n×7 X 7×1 -L n×1 (15)

[0144] Among them, V, B, X, and L are all matrices, and the lower right corner mark is the number of rows and columns of the matrix.

[0145] Spherical surface:

[0146] 1. Initialization: Select an initial sphere center position (a, b, c) and radius r;

[0147] 2. Iterative optimization: Use numerical optimization algorithms (such as gradient descent, Gauss-Newton method, Loewenberg-Marquardt method, etc.)

[0148] To iteratively update the parameters (a, b, c, r) to reduce RSS (Residual Sum of Squares)

[0149] It is an indicator used to measure the degree of model fit, which represents the sum of squares of the differences between the model prediction value and the actual observation value);

[0150]

[0151] f(x,β) is used to describe the relationship between y and x. The r in the formula i Represents the residual of the i-th observation, that is, the actual value y i and the model predicted value f(x i ,β). β is the parameter vector of the model, usually expressed as (a, b, c, r), which is used to describe the relationship between the independent variable x and the dependent variable y. The goal of optimization is to minimize RSS by updating these parameters, thereby improving the prediction accuracy of the model.

[0152] 3. Convergence judgment: When the change of parameter update is less than a certain threshold or reaches the preset number of iterations, stop the iteration.

[0153] 4. Result output: Output the final parameters (a, b, c, r) as the parameters of the best fitting sphere.

[0154] Step 3: Adjust the posture of the 3D fine color model. The camera viewport needs to face the model to generate an orthophoto. Use quaternion transformation to adjust the posture of the model to obtain an orthophoto. Figure 2 .

[0155] 3.1) Import the model into the world coordinate system, obtain the world coordinates, and use the quaternion transformation to adjust the model's position and posture: Assume there is a unit quaternion (that is, its norm is 1) q = w + xi + yj + zk, where w, x, y, z are real numbers and i, j, k are imaginary units. It represents a rotation of θ radians around the unit vector (x, y, z). This quaternion can be calculated from the rotation axis and rotation angle: Where (x, y, z) is the three-dimensional vector of the rotation axis and θ is the rotation angle.

[0156] First, the three-dimensional fine color model of the dome ceiling mural is transformed in three-dimensional space according to the following matrix:

[0157]

[0158] Among them, r ij Represents the element of the rotation matrix, t x ,t y and t zrepresent the translation components along the x, y, and z axes respectively.

[0159] The calculation method of the world coordinates is:

[0160] P world =M n ·M n-1 ·…·M1·P local (2)

[0161] Among them, M i is the transformation matrix of the i-th node, P local is a point in the local coordinate system, P world is a point in the world coordinate system.

[0162] First, the geometric parameters obtained by point cloud fitting are used to obtain the axial direction of the cylinder, which is the current orientation of the model. The cylinder axis vector and the world coordinate system Z axis vector are denoted as A and B respectively. The rotation angle θ can be obtained by calculating the angle between the current orientation vector and the target orientation vector, which can be achieved using the vector dot product formula:

[0163]

[0164] The cross product can be used to find the rotation axis, which is perpendicular to the plane defined by the current heading vector and the target heading vector. The rotation axis can be calculated by the cross product:

[0165] axis=A×B(4)

[0166] After determining the rotation axis and rotation angle, the model is rotated by quaternion. Assume there is a unit quaternion (that is, its norm is 1) q = w + xi + yj + zk, where w, x, y, z are real numbers and i, j, k are imaginary units. It represents a rotation of θ radians around the unit vector (x, y, z). This quaternion can be calculated from the rotation axis and rotation angle:

[0167]

[0168] Where (x, y, z) is the three-dimensional vector of the rotation axis and θ is the rotation angle.

[0169] Convert this quaternion into a rotation matrix using the following formula:

[0170]

[0171] 3.2) Viewport matrix transformation: The viewport size is calculated based on the resolution, and the model scanning area is divided into blocks so that the generated orthophoto image reaches the required resolution; the output resolution can be adjusted according to the viewport transformation to meet the high-resolution requirements.

[0172] The viewport transform is the process of mapping normalized device coordinates (NDC) to window or screen coordinates. The input parameters tell the rendering pipeline how to map NDC to screen coordinates. NDC is a cube that goes from -1 to 1 in all directions, and the viewport transform maps this cube to a rectangular area of ​​the screen.

[0173]

[0174] n, d, c, respectively represent the position of the coordinate mapping.

[0175] Map coordinates in the range [0,1] to screen coordinates:

[0176]

[0177] Among them, left and bottom define the position of the lower left corner of the viewport in screen coordinates, width and height are the width and height of the viewport, near and far define the depth range, depth = far-near.

[0178] The final viewport transformation matrix is ​​the product of these two matrices:

[0179] M final =M viewport *M NDC (9)

[0180] According to the resolution we need, we can calculate the width and height of the viewport and then input them into the transformation matrix to achieve the adaptive viewport size.

[0181] 3.3) Line Scan Orthographic Projection: In this process, we rotate the model by a small angle so that the camera projects along the surface of the model in a line scan manner. In this way, the images in the frame buffer object are temporarily stored in the CPU, and we get a set of fringe projection images, each image corresponding to the projection of the model at a different angle. These fringe projection images are obtained by precisely controlling the camera rotation angle and line scanning method, ensuring the continuity and integrity of the projection.

[0182] 3.4) Fringe projection image stitching: For the stitching of cylindrical fringe projection images, they are finally stitched into a complete orthophoto expansion image. For the stitching of spherical fringe projection images, the fan-shaped stitching method is required. In the process of the previous line scanning, the increment of the rotation angle of each line scanning model is equal to the increment of the rotation angle of the fringe projection image fan-shaped stitching to ensure texture continuity; in this way, the texture information of the fringe projection image can be accurately rendered when stitching. Finally, the high-resolution orthophoto image after expansion is output.

[0183] Step 4: Set the line scan viewport size, line scan rotation angle, and stitching parameters to finally get a complete orthophoto.

[0184] Ⅰ. The projection transformation from 3D to 2D mainly uses the RTT function in OSG to implement stripe projection. RTT is a technique for rendering scenes to textures instead of directly to the screen. This method can be used to create stripe patterns and then map them to cylindrical objects for unfolding. The RTT function is a technique for rendering scenes to textures. It can be used to implement some advanced image processing effects, such as dynamic reflections, shadows, post-processing effects, etc. In OSG, the RTT function is applied to the Camera object. It includes the following steps:

[0185] (1) Create a texture object for rendering: Use the osg::Texture2D class to create a 2D texture object and set the texture's size, format and other properties.

[0186] (2) Create a rendering target object: Use the osg::Camera class to create a rendering target object osg::RenderTexture. Set the texture object created previously as the color buffer of the rendering target object.

[0187] (3) Set the rendering target object to the camera: Use the camera's setRenderTarget function to set the rendering target object to the camera, so that the camera renders the scene to the texture instead of the screen.

[0188] (4) Rendering using a render target object: Add the camera and render target object to the scene graph and specify the camera as the active camera. Then call the render function of the renderer to render the scene, and the scene can be rendered to the texture.

[0189] Through the RTT function, the result of rendering to the texture can be used for subsequent image processing operations, such as applying the texture to the material of the model, as an environment texture, etc. Using OSG's RTT function, you can achieve a variety of advanced image effects, and it has good scalability and flexibility.

[0190] Ⅱ. Line scanning orthographic projection is performed on the cylindrical model. We rotate the model at a small angle so that the camera projects along the surface of the model in a line scanning manner. In this way, we obtain a set of fringe projection images, each image corresponding to the projection of the model at a different angle. The acquisition of these fringe projection images is achieved by precisely controlling the camera rotation angle and line scanning method, ensuring the continuity and integrity of the projection. Figure 3 , Figure 4 .

[0191] Optionally, in each iteration, set the model's transformation matrix by calling the mt->setMatrix() method to achieve the rotation of the camera around the Z axis. Create a new osg::Group object root and add the child node of the transformation matrix mt to the scene graph to build the current rendering scene. Get the resolution of the current screen through sg::GraphicsContext and output it to the console. Calculate the bounding sphere bs of the model and get its radius r. Output the radius to the console. Define the camera's position eye, the observation target center (the center point of the model), and the up direction up. Set the viewing matrix through the camera->setViewMatrixAsLookAt() method to determine the camera's viewing angle and orientation. Add the camera as scene data to the viewer for rendering.

[0192] III. Stitching of quasi-cylindrical fringe projection images: Match the adjacent fringe images according to the coordinates of the lower left corner with the coordinates where they should be placed to ensure seamless and natural stitching, and finally stitch them into a complete orthographic expansion image. Figure 5 .

[0193] Optionally, first, create a new osg::Geode object geode to store the image drawing data. In each iteration, create a new osg::DrawPixels object bitmapk. Add bitmapk to the geode so that it can be drawn in the scene. Initialize an osgViewer::Viewer object viewer and set the geode to the scene data for rendering. Set up an osgGA::TrackballManipulator for the viewer to allow the user to control the camera view by dragging. Create a new osg::Image object image to store the rendering results of the current frame. Append the rendering results to the camera's color buffer. Use the osgDB::writeImageFile() method to save the current rendered image to the specified path.

[0194] IV. Line scan cylindrical orthographic projection is performed on the model obtained after the longitude zoning of the quasi-spherical surface. The process is the same as that of the quasi-cylindrical surface. After obtaining the orthographic expansion map of the zoning area, the middle part is taken to obtain the stripe projection map. And add a clipping plane in osg to block the model to achieve the purpose of clipping. Figure 6 , Figure 7 .

[0195] V. The fan-shaped stitching of the spherical surface needs to determine the rotation angle of the fan-shaped stitching fringe projection image according to the model rotation angle during the line scanning process. During the model rotation process, the angle increment of each rotation should be equal to the rotation angle increment of the fringe projection image. The fringe projection image is registered to achieve visual consistency and continuity. Figure 8 , Fig. 9 .

[0196] Optionally, during the process of sector-shaped stitching of the spherical model, the coordinates of the center point of the sector of the fringe projection image are obtained, kept stationary, and rotated and stitched around this point. The angle of rotation during the stitching process is equal to the angle of rotation during the model line scanning. This ensures that the fringe projection image is perfectly stitched with no texture missing and no texture overlap.

[0197] VI. The method of the present invention is compared with the existing method for obtaining two-dimensional orthophoto images of three-dimensional fine color models of ancient buildings and map projection methods, and the analysis and comparison of quantitative data are carried out. Fig.10 , Fig.11 , Fig.12 .

[0198] There are two main methods for obtaining orthophotos of murals: one is a correction method based on a single photo, which usually involves a series of digital image processing techniques, aiming to remove perspective distortion from a single photographic image so that each part of the image matches the real size and shape of the actual object, thereby obtaining an orthophoto of the mural. The other is a directional orthophoto projection based on a three-dimensional color model, which combines three-dimensional modeling and high-precision color capture technology to generate high-quality orthophotos of murals.

[0199] The present invention proposes a high-resolution orthophoto unfolding method based on a three-dimensional fine color model. The main technologies include: (1) construction of a three-dimensional fine color model of an ancient building: using data from on-site scanning to construct a three-dimensional model; (2) segmenting the three-dimensional fine color model according to different reference planes, and each reference plane is divided into separate blocks. (3) geometric fitting: after the different reference planes are divided into separate blocks, the corresponding geometric fitting is performed on the three-dimensional curved surface to be unfolded to obtain the geometric parameter information of the geometric model. (4) automatic unfolding: adding the three-dimensional fine color model and the parameter information obtained by geometric fitting to the algorithm; adjusting the positional relationship between the model and the camera through quaternion transformation; then inversely calculating the viewport size according to the required resolution size and adjusting it; then setting the rotation angle increment of the model each time during the line scanning orthophoto projection process according to the viewport size, performing line scanning orthophoto projection, and temporarily storing the generated fringe projection image in the CPU; finally, selecting different splicing methods (rectangular splicing and fan splicing) according to different geometric models, splicing the fringe projection image, and obtaining the unfolded two-dimensional orthophoto image. Technical solution: The overall goal is to use the geometric characteristics of the three-dimensional fine color model to carry out two-dimensional expansion of the model surface painting. Based on the separate reference surface model obtained by segmentation, first, geometric fitting is performed on the model according to its geometric properties to obtain the geometric parameter information of the model; secondly, the expansion algorithm is applied to the model for line scanning orthophoto projection, and the viewport size is inversely calculated through the required resolution to ensure the resolution of the generated orthophoto image. The stripe projection image generated by the line scanning orthophoto projection will be temporarily stored in the CPU; finally, the parameters set for the line scanning correspond to the parameters set during splicing, and the temporarily stored stripe projection images are taken and spliced ​​in the order of line scanning orthophoto projection. Models with different geometric properties select different splicing methods (rectangular splicing and fan-shaped splicing), and finally a high-resolution two-dimensional orthophoto image is obtained after the three-dimensional fine color model is expanded.

[0200] The number of modules and processing scales described here are used to simplify the description of the present invention. Applications, modifications and variations of the present invention are obvious to those skilled in the art.

[0201] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A method for automatically generating an orthophoto of a dome ceiling mural based on a three-dimensional fine color model, characterized in that: The steps include: 1) Collect 3D laser scanning point clouds of ancient building dome murals and build 3D detailed color models of the dome murals. 2) segmenting the three-dimensional fine color model of the dome ceiling mural, dividing different reference planes into separate blocks, and performing geometric fitting on the three-dimensional fine color model to obtain geometric parameters of the three-dimensional fine color model of the dome ceiling mural; 3) performing line scanning orthophoto expansion on the three-dimensional fine color model of the dome ceiling mural. During the line scanning orthophoto expansion process, each time the three-dimensional fine color model of the dome ceiling mural is scanned, a posture transformation is performed on the three-dimensional fine color model of the dome ceiling mural to generate a fringe projection image, and the posture transformation parameters are related to the geometric parameters of the three-dimensional fine color model of the dome ceiling mural; The fringe projection images generated by the line scanning orthographic projection are stitched together in the order in which the line scanning is acquired to generate an orthographic unfolded image of the dome ceiling mural.

2. The method for automatically generating an orthophoto of a dome ceiling mural based on a three-dimensional fine color model according to claim 1, characterized in that: Step 3) includes the following steps: 3.1) Performing corresponding quaternion transformation according to the positional relationship between the camera and the three-dimensional fine color model to adjust the model posture: First, the three-dimensional fine color model of the dome ceiling mural is imported into the world coordinate system to obtain the world coordinates, and then the quaternion transformation is used to adjust the model posture. The quaternion calculation process includes: First, the axial direction of the cylinder is obtained through the geometric parameters of the three-dimensional fine color model of the dome ceiling mural as the current orientation of the model. The cylinder axis vector and the world coordinate system Z axis vector are recorded as A and B respectively. The rotation angle θ is obtained by calculating the angle between the current orientation vector and the target orientation vector, which is realized by the vector dot product formula: At the same time, the vector product is used to find the rotation axis, which is perpendicular to the plane defined by the current heading vector and the target heading vector. The rotation axis axis is calculated by the cross product: axis=A×B (4) The three-dimensional fine color model of the dome ceiling mural is rotated by quaternion. Assuming there is a unit quaternion q=w+xi+yj+zk, where w, x, y, z are real numbers and i, j, k are imaginary units, the unit quaternion is calculated using the rotation axis and rotation angle: Where (x, y, z) is the three-dimensional vector of the rotation axis, and θ is the rotation angle; Convert this quaternion into a rotation matrix using the following formula: Each time a scan is performed, the three-dimensional fine color model undergoes a posture transformation according to the preset rotation axis and the rotation angle to generate a fringe projection image.

3. The method for automatically generating an orthophoto of a dome ceiling mural based on a three-dimensional fine color model as claimed in claim 2, characterized in that: In step 3.1), the three-dimensional fine color model of the dome ceiling mural is first transformed in three-dimensional space according to the following matrix and then imported into the world coordinate calculation: Among them, r ij Represents the element of the rotation matrix, t x ,t y and t z represent the translation components along the x, y, and z axes respectively.

4. The method for automatically generating an orthophoto of a dome ceiling mural based on a three-dimensional fine color model as claimed in claim 2, characterized in that: In step 3.1), the method for calculating the world coordinates is: P world =M n ·M n-1 ·…·M1·P local (2) Among them, M i is the transformation matrix of the i-th node, P local is a point in the local coordinate system, P world is a point in the world coordinate system.

5. The method for automatically generating an orthophoto of a dome ceiling mural based on a three-dimensional fine color model according to claim 1, characterized in that: Step 3) also includes the following steps: 3.2) Adjustment of the camera viewport: Calculate the width and height of the camera viewport according to the required resolution, and then input them into the viewport transformation matrix to automatically adjust the size of the camera viewport. The viewport transformation matrix is: M final =M viewport *M NDC (9) in, n, d, c, respectively represent the position of coordinate mapping, Map coordinates in the range [0,1] to screen coordinates: Among them, left and bottom define the position of the lower left corner of the camera viewport in screen coordinates, width and height are the width and height of the camera viewport, near and far define the depth range, depth = far-near.

6. The method for automatically generating an orthophoto of a dome ceiling mural based on a three-dimensional fine color model according to claim 1, characterized in that: Step 3) also includes the following steps: 3.3) Line scanning orthographic projection: by rotating the three-dimensional fine color model of the dome ceiling mural, the camera projects along the surface of the three-dimensional fine color model in a line scanning manner to obtain a set of fringe projection images, each image corresponding to the projection of the three-dimensional fine color model at different angles, and each time the three-dimensional fine color model of the dome ceiling mural undergoes a posture transformation; 3.4) Fringe projection image stitching: For the stitching of quasi-cylindrical fringe projection images, firstly, the coordinates of the fringe projection images are matched, and then the edges of each fringe image are processed to align them correctly. Then, the edges of adjacent fringe images are fused. After all fringe images are aligned and fused, they are finally stitched into a complete orthophoto development image. For the splicing of spherical fringe projection images, a fan-shaped splicing method is used to fix the coordinates of the fan-shaped center point of the fringe projection image to keep it stationary, and the rotation splicing is performed with the fan-shaped center point as the axis, and the adjacent images are registered to make the texture continuous, and the expanded orthophoto image is output. During the splicing process, the rotation angle increment of the fringe projection image during fan-shaped splicing is equal to the rotation angle increment of the three-dimensional fine color model during line scanning; Until the orthophoto of the three-dimensional fine color model is completed.

7. The method for automatically generating an orthophoto of a dome ceiling mural based on a three-dimensional fine color model according to claim 1, characterized in that: The geometric parameter information of the three-dimensional fine color model of the dome ceiling mural obtained in step 2) includes: For quasi-cylindrical surfaces: Establish the error equation f=(x-x1) 2 +(y-y1) 2 +(z-z1) 2 - [a(x-x1)+b(y-y1)+c(z-z1)] 2 -r 2 (10) Where, f is the error equation; (x1, y1, z1) is the coordinate of the center of the fitted circle; (a, b, c) is the initial value of the cylinder axis vector; r is the radius of the cylinder; make: f={(x-x1) 2 +(y-y1) 2 +(z-z1) 2 -[a(x-x1)+b(y-y1)+c(z-z1)] 2 -r 2 } / 2r(11) Linearize f: The error equation can be listed as (that is, taking the partial derivative with respect to x1): in: Write the error equation in matrix form to minimize V and make it close to 0: V n×1 =B n×7 X 7×1 -L n×1 (15) Among them, V, B, X, and L are all matrices, and the lower right corner mark is the number of rows and columns of the matrix.

8. The method for automatically generating an orthophoto of a dome ceiling mural based on a three-dimensional fine color model according to claim 1, characterized in that: The geometric parameter information of the three-dimensional fine color model of the dome ceiling mural obtained in step 2) includes: for a spherical surface: Initialization: Select an initial sphere center position (a, b, c) and radius r; Iterative optimization: Use numerical optimization algorithms to minimize RSS. Among them, f(x,β) is used to describe the relationship between y and x. The r in the formula i Represents the residual of the i-th observation, that is, the actual value y i and the model predicted value f(x i ,β), β is the parameter vector of the model, expressed as (a, b, c, r), which is used to describe the relationship between the independent variable x and the dependent variable y; Convergence determination: When the change in parameter update is less than the preset threshold or reaches the preset number of iterations, the iteration stops; Result output: Output the final parameters (a, b, c, r) as the parameters of the best fitting sphere.

Images