Cultural relic spectral image splicing method and system based on grid homography transformation

Abstract

The invention provides a cultural relic spectral image splicing method and system based on grid homography transformation, and the method comprises the steps: introducing an orthographic color panoramic image as a reference image, and taking a space plane where the reference image is located as a reference coordinate system for the projection transformation of a spectral image; carrying out SIFT feature registration on the reference image and the spectral image to obtain SIFT feature matching points, and screening and supplementing the SIFT feature matching points by using LBP textures to obtain accurate feature matching point pairs; grid homography transformation is adopted, the offset position of a target grid pixel point of the spectral image is calculated with the distance between the feature matching points as the weight, and projection transformation is conducted on the spectral image according to position offset information to achieve positioning; and selecting a reference sub-lens spectral image to sequentially carry out histogram matching on the other sub-lens spectral images to realize equalization, and fusing the equalized sub-lens cultural relic spectral images by adopting weighted fusion to obtain reliable panoramic spectral image data.

Description

Technical Field

[0001] The invention belongs to the technical field of digital image processing, and in particular to a method and system for splicing cultural relic spectral images by grid homography transformation. Background Art

[0002] Due to the large field of view of the three-dimensional Guanyin cultural relic, in the process of spectral image data acquisition and shooting, the spectral image sequence can only be acquired by multiple lenses at the expense of wide viewing angle, and the spectral camera viewpoint moves a large range due to the shooting environment, resulting in serious parallax problems. At present, there are two main methods to solve the parallax problem in image stitching technology: one is to improve the accuracy of image registration by improving the image registration method, and obtain the homography correspondence that satisfies most feature registration points, but this method has certain limitations for cultural relic spectral images with large parallax and many depth of field planes. The other is to add linear guidance during the image homography deformation process. This method has a good application effect on natural scenes with certain linear structures, but it will cause certain structural deformation for cultural relic spectral images with complex textures. Therefore, it is necessary to use a specific image stitching technology to stitch and fuse multi-lens high-resolution, narrow-viewing-angle spectral images to obtain panoramic spectral images for subsequent research. Summary of the invention

[0003] In view of the defects or shortcomings of the above-mentioned prior art, the object of the present invention is to provide a method and system for stitching cultural relics spectral images based on grid homography transformation.

[0004] To achieve the above object, the present invention provides the following technical solution: a method for splicing cultural relics spectral images by grid homography transformation, comprising the following steps;

[0005] S1 introduces the orthophotod color panoramic image as the reference image, and uses the spatial plane where the reference image is located as the reference coordinate system for the spectral image projection transformation;

[0006] S2 performs SIFT feature registration on the reference image and the spectral image to obtain SIFT feature matching points, and uses LBP texture to screen and supplement the SIFT feature matching points to obtain accurate feature matching point pairs;

[0007] S3 uses grid homography transformation, takes the distance between feature matching points as weight, calculates the offset position of the target grid pixel point of the spectral image, and then performs projection transformation on the spectral image according to the position offset information to achieve positioning;

[0008] S4 selects a reference split-lens spectral image and performs histogram matching on the remaining split-lens spectral images in turn to achieve equalization, and uses weighted fusion to fuse the equalized split-lens cultural relic spectral images to obtain panoramic spectral image data.

[0009] Furthermore, in S2, SIFT feature matching points p of the reference image and the spectral image are extracted. i and p′ i , the FLANN algorithm is used to match the SIFT feature points of the sub-lens cultural relic spectrum image and the reference image to obtain the feature matching point pair

[0010] Use LBP features to determine the Hamming distance of SIFT feature matching point pairs, discard feature matching points with normalized Hamming distance greater than 0.2, and use feature matching point pairs with Hamming distance less than or equal to 0.2 to establish matching relationships, so as to filter and supplement SIFT feature matching results and obtain accurate feature matching point pairs.

[0011] Furthermore, in S2, the LBP features are specifically:

[0012] The neighborhood of the target pixel (x, y) is defined as (N, R), which is a set of N sampling points on a circle with a radius of R around the target pixel (x, y), as shown in the following formula (1):

[0013]

[0014] Among them, g c is the gray value of the target pixel (x, y), g n is the gray value of the nth neighborhood. When the neighborhood does not fall on integer coordinates, its intensity value is determined by bilinear interpolation. s(·) is the sign function defined as formula (2):

[0015]

[0016] Furthermore, in S3, the split-lens spectrum image is projected and transformed into the reference coordinate system. At this time, the split-lens spectrum image is divided into uniform grid units, and the center of each grid unit is taken as p * , give p * Different distance weights for each matching point around Specifically:

[0017]

[0018] Among them, δ is the proportionality coefficient, p i is the i-th matching point {p i ,p′ i}, from pixel p * The closer the feature matching points are, the higher the weight will be.

[0019] Furthermore, in S3, the global homography matrix is ​​calculated for the accurate feature matching point pairs, and the calculation method is as shown in formula (4):

[0020] p′=Hp (4)

[0021] Where H is the global homography transformation matrix between the cultural relic spectral image and the reference image;

[0022] Linearize equation (4) to obtain:

[0023]

[0024] Where h = [h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 ] T , is obtained by vectorizing H, and the matrix on the left side of equation (5) is recorded as A. Given an estimated h, the algebraic least squares solution can be solved to obtain equation (6):

[0025]

[0026] So H can be obtained by Reconstructed, that is, any pixel p in the spectral image I * It can be transformed into p′ in the reference image I′ by projection through equation (7): * Location;

[0027] p * =Hp * (7)

[0028] The distance weight is introduced into equation (6) to update the vectorized global homography transformation matrix, as shown in equation (9):

[0029]

[0030] The global homography transformation matrix H with offset position information is reconstructed using formula (9): * , and then use the global homography transformation matrix H with offset position information according to formula (10) * Perform projection transformation on the split-lens spectrum image to achieve the split-lens spectrum image:

[0031] p′ * =H * p * (10).

[0033] Furthermore, in S4, the brightness and contrast of the multiple lens spectrum images are made uniform by using a histogram matching method. Specifically, a lens spectrum image corrected by a reflector is used as a reference to calculate the probability density p of the input gray level z. z (z) is the specified probability density; the remaining lens spectral images are taken as the target images to be processed, and the probability density function p of the target image input gray level r is calculated. r (r), the transformation function G(z) can be obtained by formula (11):

[0034]

[0035] Where w, t are pseudo variables of integration, G(z) is the mapping from z to s;

[0036] The histogram matching of the target image should be a mapping from s to z, which is calculated by formula (12):

[0037] z=G -1 (s) (12)

[0038] The above transformation is performed on each pixel of the target image to obtain a target image with the same probability density as the reference image.

[0039] Furthermore, in S4, the spectral image of each sub-lens cultural relic after equalization is defined as f ij (x, y), f(x, y) is the stitched panoramic image, and the fusion method is defined as:

[0040]

[0041] Among them, i represents the number of rows, j is the number of columns, and w ij It represents the weight coefficient of each lens image in the overlapping area and is solved using a central symmetric function.

[0042] The present invention provides a cultural relic spectral image stitching system based on grid homography transformation, comprising:

[0043] A coordinate system acquisition module is used to introduce an orthophoto color panoramic image as a reference image, and to use the spatial plane where the reference image is located as a reference coordinate system for the projection transformation of the spectral image;

[0044] The feature matching point pair acquisition module performs SIFT feature registration on the reference image and the spectral image to obtain SIFT feature matching points. The SIFT feature matching points are screened and supplemented using LBP texture to obtain accurate feature matching point pairs.

[0045] The split-lens spectral image positioning module uses grid homography transformation and the distance between feature matching points as weight to calculate the offset position of the target grid pixel point of the spectral image, and then performs projection transformation on the spectral image according to the position offset information to achieve positioning;

[0046] The panoramic spectral image data acquisition module is used to select the reference split-lens spectral image and perform histogram matching on the remaining split-lens spectral images in turn to achieve equalization, and use weighted fusion to fuse the equalized split-lens cultural relic spectral images to obtain the panoramic spectral image data.

[0047] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for splicing cultural relics spectral images using a grid homography transformation are implemented.

[0048] The present invention also provides a computer-readable storage medium, comprising a computer program, which, when executed by a processor, implements the steps of the method for stitching cultural relics spectral images using a grid homography transformation.

[0049] Compared with the prior art, the present invention has at least the following beneficial effects:

[0050] The present invention provides a method for splicing cultural relic spectral images by grid homography projection transformation. By introducing an orthophoto color panoramic image as a reference image, the cumulative error and parallax problem of sequence splicing can be effectively avoided. The SIFT-LBP feature detection method is insensitive to image scaling, translation, rotation, illumination change and affine transformation, and is suitable for feature matching of cultural relic spectral images and reference images. The SIFT-LBP feature matching algorithm is used to screen and supplement the SIFT feature matching using LBP texture to obtain accurate feature matching point pairs; then the grid homography projection transformation method adjusts the projection transformation position of the target grid with the distance weight of the feature matching point, aligns each depth of field plane, and can effectively solve the artifacts and image structure deformation problems caused by the parallax problem; finally, the spectral images are fused by the histogram matching weighted fusion method to obtain a cultural relic spectral panoramic image with correct structural information. The method of the present invention can not only eliminate the artifacts of the overlapping parts of the spliced ​​images, but also avoid the wrong scaling relationship, and maintain the structural information of the cultural relic when the number of shots is large. Comprehensive evaluation shows that the proposed method has the best effect in the splicing of cultural relic spectral images, and can provide reliable spectral panoramic data for the research and protection analysis of cultural relics. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is the overall framework flow chart of the cultural relic spectral image stitching method based on grid homography projection transformation.

[0052] Figure 2 (a) is the split-lens image to be stitched, Figure 2 (b) is a schematic diagram of grid division. Figure 2 (c) is the positioning map after grid homography projection transformation.

[0053] Figure 3 (a)-(d) are the images of the Liao Dynasty painted Guanyin statue to be stitched together, and (e) is the stitching result of the spectral image of the Liao Dynasty painted Guanyin statue.

[0054] Figure 4 (a)-(h) in the middle are the images of the mural of "The Invincible Great Wrathful Vajra King" on the eastern part of the south wall of Dule Temple to be spliced ​​together, and (i) is the splicing result of the mural of "The Invincible Great Wrathful Vajra King" on the eastern part of the south wall of Dule Temple.

[0055] Figure 5 A schematic diagram of a computer device provided by an embodiment of the present invention.

[0056] Figure 6 The block diagram of a chip provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0058] like Figure 1 As shown, the present invention proposes a method for mosaicking cultural relics spectral images by grid homography transformation, which is mainly divided into four parts: establishing a reference coordinate system, SIFT-LBP image registration, grid homography transformation, and histogram matching image fusion. The specific steps are as follows:

[0059] Step 1) Introduce an orthophoto color panoramic image as a reference image, and use the spatial plane where the reference image is located as the reference coordinate system for the projection transformation of the spectral image. As the number of lenses increases, the stitching error will gradually accumulate, seriously affecting the structural information of the stitched image. In order to avoid the cumulative error caused by the stitching of image sequences, an orthophoto color panoramic image is introduced as a reference image. A reference coordinate system is established on the plane where the reference image is located, and the spectral image is projected and transformed to the reference coordinate system. For each split-lens spectral image, only one conversion calculation to the reference coordinate is completed, which can effectively avoid the problem of cumulative error.

[0060] Step 2) Perform SIFT feature registration on the reference image and the spectral image, use LBP texture to filter and supplement SIFT feature matching points, and obtain accurate feature matching point pairs. The specific steps are as follows:

[0061] a) Denote the split-shot spectral image and the reference image as I and I′ respectively, and then extract the SIFT feature matching points p between the reference image and the split-shot spectral image i and p′i The FLANN algorithm is used to match the SIFT feature matching points of the split-lens spectrum image and the reference image to obtain the feature matching point pair.

[0062] b) Use the LBP operator to calculate the relationship between the neighborhood points and the feature matching points. The specific calculation method is as shown in formula (1). Define the neighborhood of the target pixel (x, y) as (N, R), which is a set of N sampling points on a circle with a radius of R around the pixel (x, y). Since the depth of field of the Guanyin cultural relics is rich, the range of the feature neighborhood varies greatly, so the radius is taken as 1 here, and the number of sampling points N = 8. The specific calculation method is as follows:

[0063]

[0064] where g c is the gray value of the target pixel (x, y), g n is the grayscale value of the nth neighborhood. When the neighborhood does not fall on integer coordinates, its intensity value is determined by bilinear interpolation, and s(·) is the sign function defined as in Equation (2).

[0065] c) Use LBP features to determine the Hamming distance of SIFT feature matching point pairs to determine whether the point is matched correctly. The specific calculation method is as follows:

[0066] d(p i ,p′ i ) = ∑(p i [j]≠p′ i [j]) (3)

[0067] Where j represents the number of bits of the binary operator, and the Hamming distance d(p i ,p′ i ). Since the research on cultural relic spectral data requires pixel accuracy, feature matching points with normalized Hamming distance greater than 0.2 are discarded, and only d(p i ,p′ i )≤0.2 to establish a matching relationship between feature matching point pairs, so as to filter and supplement the SIFT feature matching results and obtain accurate feature matching point pairs.

[0068] Step 3), such as Figure 2 As shown in the figure, the grid homography transformation is adopted, and the distance between the feature matching points is used as the weight to calculate the offset position of the target grid pixel point of the spectral image. Then, the spectral image is projected according to the position offset information. The specific calculation method is as follows:

[0069] a) Obtain accurate feature matching point pairs through step 2) and calculate the global homography matrix. The calculation method is as shown in formula (4).

[0070] p′=Hp (4)

[0071] Where H is the global homography transformation matrix between the artifact spectral image and the reference image.

[0072] The global homography transformation matrix H can be solved according to DLT (Direct Linear Transformation), which is a method of matching feature points. The calculation results show that ~ means that they are equal in proportion. Formula (4) can be linearized into formula (5):

[0073]

[0074] where h = [h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 ] T , is obtained by vectorizing the global homography transformation matrix H. The matrix on the left side of equation (5) is recorded as A. Given an estimated h, the algebraic least squares solution can be solved, as shown in equation (6):

[0075]

[0076] So H can be obtained by Reconstructed, that is, any pixel p in the split-lens spectral image I * It can be projected to p′ in the reference image I′ by the following method * Position, as shown in formula (7).

[0077] p′ * =Hp * (7)

[0078] b) If Figure 2 As shown, the split-lens spectrum image I is projected and transformed into the reference coordinate system, so the split-lens spectrum image I is evenly divided into a grid of c1*c2 units, and the center of each unit is taken as p * , give pixel p * Different distance weights for each matching point around As shown in formula (8):

[0079]

[0080] Where δ is the proportionality coefficient, p iis the i-th matching point {p i ,p′ i}, from the pixel point p * The closer the feature matching points are, the greater the distance weight will be. The higher the p * When continuous translation is performed in the image domain, the calculated global homography transformation matrix also changes smoothly, denoted as H * At this time, formula (6) can be added with distance weight to obtain h * , as shown in formula (9):

[0081]

[0082] According to formula (8), H * , and then perform projection transformation on the spectral image according to formula (9) to obtain the offset position of each lens spectral image in the reference coordinate.

[0083] p′ * =H * p * (10)

[0084] Step 4) The histogram matching weighted fusion method is used to fuse the split-lens spectral images to smooth the image overlapping areas and provide reliable panoramic spectral image data for cultural relics protection and research.

[0085] The brightness and contrast of multiple lens spectrum images are unified by using histogram matching method. The histogram distribution of the two images is calculated, and the probability density p of the input gray level z is calculated by taking a lens spectrum image corrected by a reflector as the reference image. z (z), which is the specified probability density. The remaining lens spectral images are taken as the target images to be processed, and the probability density function p of the target image input gray level r is calculated. r (r), the transformation function G(z) can be obtained by formula (11).

[0086]

[0087] Where w, t are integration pseudo variables, and G(z) is the mapping from z to s. The histogram matching of the target image should be the mapping from s to z, which is calculated by equation (12):

[0088] z=G -1 (s) (12)

[0089] The transformation of formula (12) is performed on each pixel of the target image to obtain a target image with the same probability density as the reference image. After obtaining the spectral images of each lens with balanced brightness, the spectral images of each lens are defined as f ij(x, y), f(x, y) is the stitched panoramic image, and the fusion method is defined as:

[0090]

[0091] Where i is the number of rows, j is the number of columns, and w is the number of columns. ij Indicates that the weight coefficients of each lens image in the overlapping area satisfy ∑ ij w ij = 1 constraint, the centrosymmetric function can be used to solve it, and the cosine function is selected here. The weighted fusion method can make the grayscale value transition of the cultural relic spectral image natural and eliminate the influence of exposure error on the splicing effect of the cultural relic spectral image.

[0092] Based on the above method, an embodiment of the present invention further provides a cultural relic spectral image stitching system based on grid homography transformation, comprising:

[0093] A coordinate system acquisition module is used to introduce an orthophoto color panoramic image as a reference image, and to use the spatial plane where the reference image is located as a reference coordinate system for the projection transformation of the spectral image;

[0094] The feature matching point pair acquisition module performs SIFT feature registration on the reference image and the spectral image to obtain SIFT feature matching points. The SIFT feature matching points are screened and supplemented using LBP texture to obtain accurate feature matching point pairs.

[0095] The split-lens spectral image positioning module uses grid homography transformation and the distance between feature matching points as weight to calculate the offset position of the target grid pixel point of the spectral image, and then performs projection transformation on the spectral image according to the position offset information to achieve positioning;

[0096] The panoramic spectral image data acquisition module is used to select the reference split-lens spectral image and perform histogram matching on the remaining split-lens spectral images in turn to achieve equalization, and use weighted fusion to fuse the equalized split-lens cultural relic spectral images to obtain the panoramic spectral image data.

[0097] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions to implement corresponding method flows or corresponding functions; the processor described in the embodiment of the present invention can be used for the operation of a method for splicing cultural relics spectral images with grid homography transformation, including:

[0098] Step 1), introducing an orthophotod color panoramic image as a reference image, and using the spatial plane where the reference image is located as a reference coordinate system for the spectral image projection transformation;

[0099] Step 2), perform SIFT feature registration on the reference image and the spectral image, use LBP texture to screen and supplement SIFT feature matching points, and obtain accurate feature matching point pairs;

[0100] Step 3), using grid homography transformation, taking the distance between feature matching points as weight, calculate the offset position of the target grid pixel point of the spectral image, and then perform projection transformation on the spectral image according to the position offset information;

[0101] Step 4) uses the histogram matching weighted fusion method to fuse the split-lens spectral images to smooth the image overlapping areas and provide reliable panoramic spectral image data for cultural relics protection and research.

[0102] Figure 5 A schematic diagram of a computer device provided by an embodiment of the present invention.

[0103] See also Figure 5, the terminal device is a computer device, and the computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, a method for stitching cultural relics spectral images by grid homography transformation in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the system for stitching cultural relics spectral images by grid homography transformation in the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0104] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 5 This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0105] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0106] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0107] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0108] Figure 6 The block diagram of a chip provided according to an embodiment of the present invention.

[0109] See also Figure 6 , the terminal device is a chip, and the chip 600 of this embodiment includes a processor 622, which may be one or more, and a memory 632 for storing a computer program executable by the processor 622. The computer program stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 622 may be configured to execute the computer program to execute the above-mentioned method for stitching artifact spectral images using a grid homography transformation.

[0110] In addition, the chip 600 may further include a power supply component 626 and a communication component 650, wherein the power supply component 626 may be configured to perform power management of the chip 600, and the communication component 650 may be configured to implement communication, for example, wired or wireless communication, of the chip 600. In addition, the chip 600 may further include an input / output interface 658. The chip 600 may operate based on an operating system stored in the memory 632.

[0111] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (Non-VolatileMemory), such as at least one disk memory.

[0112] The processor may load and execute one or more instructions stored in a computer-readable storage medium to implement the corresponding steps of the method for stitching cultural relics spectral images based on a grid homography transformation in the above embodiment; the processor may load and execute the following steps of one or more instructions in the computer-readable storage medium:

[0113] Step 1), introducing an orthophotod color panoramic image as a reference image, and using the spatial plane where the reference image is located as a reference coordinate system for the spectral image projection transformation;

[0114] Step 2), perform SIFT feature registration on the reference image and the spectral image, use LBP texture to screen and supplement SIFT feature matching points, and obtain accurate feature matching point pairs;

[0115] Step 3), using grid homography transformation, taking the distance between feature matching points as weight, calculate the offset position of the target grid pixel point of the spectral image, and then perform projection transformation on the spectral image according to the position offset information;

[0116] Step 4) uses the histogram matching weighted fusion method to fuse the split-lens spectral images to smooth the image overlapping areas and provide reliable panoramic spectral image data for cultural relics protection and research.

[0117] In order to verify the application effect of the mural color restoration method based on the pigment fading spectral color difference model proposed in the present invention in actual situations, the application effect tests were conducted on a Liao Dynasty colored sculpture of Guanyin in Dule Temple and the mural of "Invincible Great Wrathful Vidyaraja" on the eastern part of the south wall of Dule Temple. For the three-dimensional Guanyin statue cultural relic, the SIFT-LBP feature registration method was used to register the reference image and the split-lens spectral image. Then the homography was transformed through the grid homography, and finally the panoramic sound and sound cultural relic spectral image was obtained through the histogram equalization weighted fusion method.

[0118] like Figure 3 As shown, in the results obtained by the method of the present invention, the complex textures of the planes with different depths of field in the overlapping area fit perfectly, and the geometric structures of other prominent points are also well protected in the grid homography transformation.

[0119] like Figure 4 As shown, for the verification effect of a flat mural without depth of field change, in the results obtained by the method of the present invention, even if the distance span is large, the arm structure and collar of the character are not stretched or bent, and the overall visual effect is natural.

[0120] In general, the present invention is based on the method of splicing cultural relics spectral images based on grid homography projection transformation, introduces orthophoto color panoramic images as reference images, and avoids the problem of cumulative errors based on single-lens spectral images. Through the SIFT-LBP feature matching algorithm, the SIFT feature matching is screened and supplemented by LBP texture to obtain accurate feature matching point pairs; then the grid homography transformation is used, and the distance between feature matching points is used as the weight to calculate the offset position of the target grid pixel point to reduce the problem of feature relative position offset; finally, the histogram matching weighted fusion method is used to fuse the split-lens spectral images to smooth the image overlapping area, which can not only eliminate the artifacts of the overlapping part of the spliced ​​image, but also avoid the wrong scaling relationship, and maintain the structural information of the cultural relics when the number of split-lens is large. Comprehensive evaluation shows that the proposed method has the best effect in the splicing of cultural relics spectral images, and can provide reliable spectral panoramic data for the research and protection analysis of cultural relics.

[0121] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0122] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0123] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0124] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0125] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0127] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0128] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0131] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for splicing cultural relics spectral images based on grid homography transformation, characterized in that: The steps include: S1 introduces the orthophotod color panoramic image as the reference image, and uses the spatial plane where the reference image is located as the reference coordinate system for the spectral image projection transformation; S2 performs SIFT feature registration on the reference image and the spectral image to obtain SIFT feature matching points, and uses LBP texture to screen and supplement the SIFT feature matching points to obtain accurate feature matching point pairs; S3 uses grid homography transformation, takes the distance between feature matching points as weight, calculates the offset position of the target grid pixel point of the spectral image, and then performs projection transformation on the spectral image according to the position offset information to achieve positioning; S4 selects a reference split-lens spectral image and performs histogram matching on the remaining split-lens spectral images in turn to achieve equalization, and uses weighted fusion to fuse the equalized split-lens cultural relic spectral images to obtain panoramic spectral image data.

2. The method for splicing cultural relics spectral images by grid homography transformation as claimed in claim 1, characterized in that: In S2, SIFT feature matching points p of the reference image and spectral image are extracted. i and p i ′ , the FLANN algorithm is used to match the SIFT feature points of the sub-lens cultural relic spectrum image and the reference image to obtain the feature matching point pair Use LBP features to determine the Hamming distance of SIFT feature matching point pairs, discard feature matching points with normalized Hamming distance greater than 0.2, and use feature matching point pairs with Hamming distance less than or equal to 0.2 to establish matching relationships, so as to filter and supplement SIFT feature matching results and obtain accurate feature matching point pairs.

3. The method for splicing cultural relics spectral images by grid homography transformation as claimed in claim 2, characterized in that: In S2, the LBP features are specifically: The neighborhood of the target pixel (x, y) is defined as (N, R), which is a set of N sampling points on a circle with a radius of R around the target pixel (x, y), as shown in the following formula (1): Among them, g c is the gray value of the target pixel (x, y), g n is the gray value of the nth neighborhood. When the neighborhood does not fall on integer coordinates, its intensity value is determined by bilinear interpolation. s(·) is the sign function defined as formula (2):

4. The method for splicing cultural relics spectral images by grid homography transformation as claimed in claim 1, characterized in that: In S3, the split-lens spectrum image is projected and transformed into the reference coordinate system. At this time, the split-lens spectrum image is divided into uniform grid cells, and the center of each grid cell is taken as p * , give p * Different distance weights for each matching point around Specifically: Among them, δ is the proportionality coefficient, p i is the i-th matching point {p i ,p′ i }, from pixel p * The closer the feature matching points are, the higher the weight will be.

5. The method for splicing cultural relics spectral images by grid homography transformation as claimed in claim 4, characterized in that: In S3, The global homography matrix is ​​calculated for the accurate feature matching point pairs. The calculation method is as follows: p′=Hp (4) Where H is the global homography transformation matrix between the cultural relic spectral image and the reference image; Linearize equation (4) to obtain: Where h = [h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 ] T , is obtained by vectorizing H, and the matrix on the left side of equation (5) is recorded as A. Given an estimated h, the algebraic least squares solution can be solved to obtain equation (6): Therefore, H can be obtained by Reconstructed, that is, any pixel p in the spectral image I * It can be transformed into p′ in the reference image I′ by projection through equation (7): * Location; p * =Hp * (7) The distance weight is introduced into equation (6) to update the vectorized global homography transformation matrix, as shown in equation (9): The global homography transformation matrix H with offset position information is reconstructed using formula (9): * , and then use the global homography transformation matrix H with offset position information according to formula (10) * Perform projection transformation on the split-lens spectrum image to achieve the split-lens spectrum image: p′ * =H * p * (10)。 6. The method for splicing cultural relics spectral images by grid homography transformation as claimed in claim 1, characterized in that: In S4, the brightness and contrast of multiple lens spectrum images are unified by using the histogram matching method. Specifically, a lens spectrum image corrected by a reflector is used as a reference to calculate the probability density p of the input gray level z. z (z) is the specified probability density; the remaining lens spectral images are taken as the target images to be processed, and the probability density function p of the target image input gray level r is calculated. r (r), the transformation function G(z) can be obtained by formula (11): Where w, t are pseudo variables of integration, G(z) is the mapping from z to s; The histogram matching of the target image should be a mapping from s to z, which is calculated by formula (12): z=G -1 (s) (12) The above transformation is performed on each pixel of the target image to obtain a target image with the same probability density as the reference image.

7. The method for splicing cultural relics spectral images by grid homography transformation as claimed in claim 6, characterized in that: In S4, the spectral image of each lens artifact after equalization is defined as f ij (x, y), f(x, y) is the stitched panoramic image, and the fusion method is defined as: Among them, i represents the number of rows, j is the number of columns, and w ij It represents the weight coefficient of each lens image in the overlapping area and is solved using a central symmetric function.

8. A cultural relic spectral image stitching system based on grid homography transformation, characterized in that: include: A coordinate system acquisition module is used to introduce an orthophoto color panoramic image as a reference image, and to use the spatial plane where the reference image is located as a reference coordinate system for the projection transformation of the spectral image; The feature matching point pair acquisition module performs SIFT feature registration on the reference image and the spectral image to obtain SIFT feature matching points. The SIFT feature matching points are screened and supplemented using LBP texture to obtain accurate feature matching point pairs. The split-lens spectral image positioning module uses grid homography transformation and the distance between feature matching points as weight to calculate the offset position of the target grid pixel point of the spectral image, and then performs projection transformation on the spectral image according to the position offset information to achieve positioning; The panoramic spectral image data acquisition module is used to select the reference split-lens spectral image and perform histogram matching on the remaining split-lens spectral images in turn to achieve equalization, and use weighted fusion to fuse the equalized split-lens cultural relic spectral images to obtain the panoramic spectral image data.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for stitching cultural relics spectral images by grid homography transformation described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for stitching cultural relics spectral images by grid homography transformation described in any one of claims 1 to 7 are implemented.

Images