Method and device for restoring and displaying historical relics
Through high-resolution high color spectrum technology and multi-dimensional image processing, the pattern outline and color system of the green tile building are restored, which solves the problems of painting damage and data acquisition interference, provides detailed data support, and realizes the repair of the old as it is old effect.
Patent Information
- Application Number
- CN202411061451.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-26
AI Technical Summary
The colored patterns of the green tile building are damaged due to chemical corrosion and repair process, resulting in distortion of the pattern color system and outline, lack of original drawing comparison, and construction equipment interferes with data collection, making it difficult to achieve the effect of repairing the old as before.
Using high-resolution high-color spectrum technology and image recognition algorithm, the pattern outline and color spectrum information of the mural are restored through multi-dimensional pattern cube array and neural recognition model, and the multi-dimensional recognition network and image weighting technology are used to paint high-resolution images. processing and analysis.
The original pattern outline and color system of the painted are effectively restored, providing a detailed data foundation, supporting the construction team to achieve the goal of repairing the old as before, reducing construction interference, and improving the accuracy and reliability of data collection.
Smart Images

Figure CN119027346A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of non-destructive restoration of the original appearance of cultural relics by high color spectrum analysis, and in particular to a method and device for restoring and displaying historical cultural relics. Background Art
[0002] The Green Tile Building is located in a certain city campus. The foundation was laid on July 7, 1930 and the construction was completed in October 1933 with an investment of about 780,000 yuan. The building is a four-story palace-style beam-column structure in the traditional revival style. The exterior treatment varies. The plane is symmetrical in the left, right, front and back. The exterior wall is made of artificial stone, the roof is single-eaved with green glazed tiles, red columns, painted brackets, platform railings, granite steps, red steps, etc. The whole building is magnificent and beautiful. The building faces south and is divided into three sections: left, middle and right. The roof is also divided into three parts. The width is 25 meters in the middle; the left and right wings are 20 meters each; the width is about 93 meters, the depth is about 66 meters, and the height is about 31.4 meters. It covers an area of 3,063 square meters, and the total area of each floor of the building is 8,982 square meters. The first floor of its exterior is a platform surrounded by railings. The upper three floors are beam-column structures. The roof is covered with green glazed tiles and reinforced concrete structures. The Green Tile Building has now been listed as the fifth batch of cultural relics protection units in a certain city.
[0003] In order to serve the latest renovation and record and maintain the current building structure and various types of painting status, it is necessary to collect and save multi-source data of the building facade, as well as construct and display real textures and detailed multi-dimensional models.
[0004] During the comprehensive renovation of the Green Tile Building, it was discovered that there were original paintings under the paint layer. After careful peeling and cleaning by the construction workers, they were vaguely visible, but due to the corrosion of cement, lime and other chemical substances, and the impact of the peeling and cleaning process, the paintings have been damaged to varying degrees. Due to historical reasons, there are few original drawings of this building, and the ancient rice paper rubbing method cannot meet the needs of fine restoration.
[0005] Therefore, the following problems exist in the renovation of the existing Green Tile Building:
[0006] The color of the pattern has been seriously distorted: the Green Tile Building has been damaged since its construction, resulting in a large number of original paintings being covered with concrete, lime, chemical pigments, etc. The surface of the paintings has undergone chemical changes, and its color and outline can no longer be clearly identified, let alone restored on the original basis.
[0007] The outline of the pattern can no longer be discerned: In order to restore the old to its original state, data collection needs to analyze the pattern color and pattern line outline.
[0008] No original drawings for comparison: The original drawings of the Green Tile Building are severely missing, the painted patterns are difficult to verify, and the few newspaper patterns that exist cannot be used as a basis for comparison.
[0009] Construction has a significant impact on data collection: Currently, facilities, equipment and dust under construction have a significant impact on data collection. Summary of the invention
[0010] In order to overcome the problems existing in the related art, the disclosed embodiments of the present invention provide a method and device for restoring and displaying historical relics.
[0011] The technical solution is as follows: A method for restoring and displaying historical relics, the method comprising:
[0012] S1, high color spectrum data acquisition: Use high resolution and high color spectrum technology to collect full spectrum data of the murals, and restore the mural outline and color spectrum information through image recognition, enhancement algorithm and spectrum fusion technology;
[0013] S2, processing of high-resolution images and high-color spectrum images;
[0014] S3, based on the processed image, high color spectrum analysis is performed to identify the ground objects using the image plus high color spectrum information.
[0015] Further, in step S1, high color spectrum data collection includes: indoor painted area image collection, restoration and modeling;
[0016] Collection, restoration and modeling of painted images of interior halls.
[0017] Furthermore, the processing of painting the high-resolution image and the high-color spectrum image in step S2 includes:
[0018] S201, feature extraction, target recognition and enhanced image analysis and processing of high-definition images;
[0019] S202, the high color spectrum image is subjected to pattern, spectrum, color spectrum information recognition and fusion technology to obtain the pattern outline, color spectrum segment and color level information of the mural.
[0020] Furthermore, the pattern outline, color spectrum and color level information of the mural obtained in step S202 include:
[0021] Step 1: Generate an array of multidimensional pattern cubes using a multidimensional pattern profile or color level recognition method, and randomly divide the multidimensional pattern cube array into a recognition array, a fusion array, and a test array;
[0022] Step 2: the recognition array is used to update the pattern outline, color spectrum and color level information parameters of the mural for multiple recognitions, while the fusion array is used to fuse the performance of the recognition arrays and select the array with the best recognition;
[0023] Step three, select a test array to verify the effectiveness of the recognition array, and obtain the results of distinguishing the pattern outline, color spectrum and color level information of the mural.
[0024] Furthermore, the step 1 specifically includes:
[0025] Select the center pixel z from the original spot i The m×m neighboring pixels generate an array of multidimensional patch cubes {z1,z2,...,z n}∈O m×m×e ; If the target pixel is at the edge of the image, the missing adjacent pixel values are set to zero; In the multidimensional pattern contour or color level recognition method, m is the patch size, the patch size is set to 216, and e is the number of color bands; The multidimensional pattern cube set is randomly divided into the recognition array U s , fusion array U l and the test array U c , the corresponding image label vector is divided into and Only the pattern outline or color level information around the target pixel is used.
[0026] Furthermore, the multidimensional pattern contour or color level recognition method in step 1 is divided into three objects: color spectrum segment, pattern contour and color level information, which are respectively used to capture the color spectrum feature map, the pattern contour feature map and the color level information feature, and the three acquired features are fused for differentiated display; wherein, the color spectrum segment includes the color spectrum area and the color spectrum focus area; the pattern contour includes the pattern contour area and the pattern contour focus area; the color level information includes the color level information area and the color level focus area.
[0027] Furthermore, the following models are used to identify and fuse arrays in step 2:
[0028] (1) Multidimensional recognition network: Multidimensional recognition network is a common element in neural recognition models based on 3D image cubes; g ×m g ×e g Size of t g Feature map, a multi-dimensional recognition network layer, contains a size of γ g+1 ×γ g+1 ×h g+1 K g+1 channels, generating a size of m g+1 ×m g+1 ×e g+1 t g+1 Output feature map; the i-th output of the (g+1)-th multidimensional recognition network layer is calculated as:
[0029]
[0030]
[0031] in, is the j-th input feature map of the (g+1) layer, is the output of the discriminator of layer g; D(·) and Var(·) represent the expected and variance functions of the input respectively; V i g+1 and They represent the weights and biases of the (g+1)-layer multidimensional recognition network, * is the 3D recognition operation, and L(·) is the processing function that introduces the network non-smoothing processor;
[0032] (2) Concentrated connection: The concentrated area is the basic unit in pattern recognition. The output of the lth concentrated area is calculated as:
[0033] z l =V l [z0,z1,...,z l-1 ];
[0034] Among them, V l is a region containing the recognition layer, the processing layer, and the discriminator layer, z0,z1,...,z l-1 The more connections there are, the more image flows there are in the concentrated network. A concentrated network with S layers has S(S+1) / 2 connections, while a traditional recognition network with the same number of layers has only S direct connections.
[0035] (3) Attention strategy:
[0036] Pattern contour attention mapping U∈O c×c It is directly from the initial input F∈O c×m×m Calculated, where m×m is the size of the input area, c represents the number of input channels; replace F with F T Perform matrix multiplication to obtain the channel attention mapping U∈O c×c , connect the processing layer as follows:
[0037]
[0038] Among them, z ji represents the influence of the ith channel on the jth channel; F T The result of matrix multiplication with F can be restored to O c ×m×m ; The reconstructed result is weighted by the parameter of the pattern outline size γ, and the input A is added to obtain the final color spectrum attention map D∈O c×m×m :
[0039]
[0040] Among them, γ is initialized to grayscale and learned step by step; the final graph D contains the weighted sum of all channel features, which is used to describe a dependency relationship and enhance the discriminability of features;
[0041] Color spectrum or color level attention area: Given an input feature F∈O c×m×m , two recognition layers are used to generate new color spectrum feature maps I and color level features J, where {I,J}∈O c×m×m ; I and J can be restored to O c×t , where t = m × m is the number of pixels; matrix multiplication is performed between I and J, additional processing layers are added, and the color spectrum or color level attention feature map P∈O is calculated. t×t :
[0042]
[0043] Among them, P ji Represents the influence of the i-th pixel to the j-th pixel; the closer the feature representations of two pixels are, the stronger the correlation between the pixels is;
[0044] The initial input feature F is sent to the recognition layer at the same time to obtain a new feature map M∈O c×m×m Can be restored to O c×t ; in M and P T The matrix multiplication operation is performed between them, and the result can be restored to O c×m×m :
[0045]
[0046] Among them, the initial value of χ is 1, and more weights are gradually learned and assigned; all positions and original features are added with certain weights to obtain the final feature D∈O c×m×m , so the context information on the color spectrum or color level value is modeled as D;
[0047] (4) Restorable multidimensional recognition: Restorable multidimensional recognition dynamically adjusts the restoration area of the receptive field according to the actual situation of the image. The input features of size C×H×W are passed through a multidimensional recognition network of size m×v×r to generate difference features of size 3N×C×H×W, where N=m×v×r is the size of the sampling grid; it has 3N values along the channel dimension, and the values represent the restorable values of the multidimensional sampling grid; the learned difference features are used for the restoration of the sampling grid of the multidimensional recognition network to generate a multidimensional sampling grid; the multidimensional sampling grid is used to generate the output features;
[0048] Multidimensionality is expressed by the following formula:
[0049]
[0050] Among them, Δm t Represents the difference corresponding to the t-th value in the m×v×r identification sampling grid, using BIM to generate accurate values; the BIM method formula is:
[0051]
[0052] (5) Processing function: The processing function used in multidimensional recognition is the fusion function. The formula of the fusion function is:
[0053]
[0054] Where z represents the input; the fusion function has an upper bound of unbounded and a lower bound of [≈-0.31,∞]; the differential coefficient of the fusion function is defined as:
[0055]
[0056] in,
[0057] (6) Selection of optimal weights. During the recognition process, the model with the highest accuracy on the fusion array is selected as the output. If the accuracy on the fusion array is consistent, the model with the smallest loss on the fusion array is selected as the output. The best model is saved for each recognition. If the model recognized next time is better, the last saved model is replaced, otherwise it is not replaced.
[0058] Furthermore, in step 3, based on the processed image, high color spectrum analysis is performed, and identifying ground objects using image high color spectrum information includes:
[0059] The dataset F consists of N labeled pixels {b1,b2,…,b t}∈Β 1×1×m where m is the high color spectrum band, and the corresponding ground feature category label set is I = {e1, e2, …, e t}∈B 1×1×v , where q is the number of mural categories.
[0060] Furthermore, in step 3, the quantitative indicator used to measure the difference between the predicted result and the true value is the indicator difference function, which is defined as:
[0061]
[0062] in, represents the pattern outline, color spectrum and color level information label vector of the mural predicted by the model, Q = [Q1, Q2, ..., Q L ] represents the pattern outline, color spectrum and color level information label vector of the real mural.
[0063] Another object of the present invention is to provide a historical relics restoration and display device, the device comprising:
[0064] The high color spectrum data acquisition module area is used to collect full-spectrum data of murals through high-resolution and high-color spectrum technology, and to restore the mural outline and color spectrum information through image recognition, enhancement algorithm and spectrum fusion technology.
[0065] Image processing module area, used for processing high-resolution images and high-color spectrum images.
[0066] The recognition model area is used to perform high color spectrum analysis based on the processed image and recognize the ground objects by using the image and high color spectrum information.
[0067] In combination with all the above technical solutions, the beneficial effects of the present invention are as follows:
[0068] Preservation of original images before renovation: In order to preserve the original image data of the building, the current construction status, the record of the renovation process and provide the most reliable basis for the design required for the restoration of the painting, multi-source data collection and preservation of the indoor painted area of the building, as well as the construction and display of real textures and fine multi-dimensional models are carried out. Records are preserved for irreversible states.
[0069] Identify painted outlines and color schemes: Through new technology acquisition and computer cloud computing platforms, multi-spectral and color-level analysis is performed to form a true pattern texture, which is seamlessly integrated with the multi-dimensional model of the building, providing the design team with a detailed and reliable data foundation and basis to guide construction. BRIEF DESCRIPTION OF THE DRAWINGS
[0070] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure;
[0071] Figure 1 is a flow chart of a method for restoring and displaying historical relics provided by an embodiment of the present invention;
[0072] Figure 2 It is a processing flow chart of a high-resolution image and a high-color spectrum image provided by an embodiment of the present invention;
[0073] Figure 3 It is a diagram of the effect of the atlas information display (image cube) provided by an embodiment of the present invention;
[0074] Figure 4 It is a conventional photo in the original color area and the later destroyed color area with similar colors provided by the embodiment of the present invention;
[0075] Figure 5It is a rendering of the processing result based on the atlas information of the present invention in the original color area and the later damaged color area with similar colors provided by the embodiment of the present invention;
[0076] Figure 6 The embodiment of the present invention provides a conventional photo without multi-spectral data fusion, image enhancement and edge extraction;
[0077] Figure 7 The embodiment of the present invention provides a spectrum enhancement effect diagram after multi-spectral data fusion, image enhancement and edge extraction;
[0078] Figure 8 It is a (partial) effect diagram for displaying color spectrum information of different bands provided by an embodiment of the present invention;
[0079] Fig. 9 is an image of a damaged object acquired by a conventional high-definition camera provided by an embodiment of the present invention;
[0080] Fig.10 It is a rendering of enhanced display (pseudo-color) of atlas information provided by an embodiment of the present invention;
[0081] Fig.11 It is a color display effect diagram (test) based on atlas information provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0082] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific implementation disclosed below.
[0083] Embodiment 1, as Figure 1 As shown, the method for restoring and displaying historical relics provided by the embodiment of the present invention includes:
[0084] S1, high color spectrum data acquisition: Use high-resolution high color spectrum technology to collect full spectrum data of the murals, and restore the mural outline and color spectrum information through image recognition, enhancement algorithm and spectrum fusion technology.
[0085] Includes: image acquisition, restoration and modeling of indoor painted areas;
[0086] Collection, restoration and modeling of painted images of the third floor lobby;
[0087] S2, processing of high-resolution images and high-color spectrum images; e.g. Figure 2 As shown, including:
[0088] S201, image analysis and processing such as feature extraction, target recognition and enhancement of high-definition images;
[0089] S202, using the high color spectrum image to perform information recognition and fusion technology such as pattern spots, patterns, and color spectra to obtain the pattern outline, color spectrum segments, and color level information of the mural;
[0090] S3, based on the processed image, high color spectrum analysis is performed to identify the ground objects using the image plus high color spectrum information.
[0091] Exemplarily, obtaining the pattern outline, color spectrum and color level information of the mural in step S202 includes:
[0092] Step 1: Generate an array of multidimensional pattern cubes using a multidimensional pattern profile or color level recognition method, and randomly divide the multidimensional pattern cube array into a recognition array, a fusion array, and a test array;
[0093] Step 2: the recognition array is used to update the pattern outline, color spectrum and color level information parameters of the mural for multiple recognitions, while the fusion array is used to fuse the performance of the recognition arrays and select the array with the best recognition;
[0094] Step three, select a test array to verify the effectiveness of the recognition array, and obtain the results of distinguishing the pattern outline, color spectrum and color level information of the mural.
[0095] Preferably, the step 1 specifically includes:
[0096] Select the center pixel z from the original spot i The m×m neighboring pixels generate an array of multidimensional patch cubes {z1,z2,...,z n}∈O m×m×e ; If the target pixel is at the edge of the image, the missing adjacent pixel values are set to zero; In the multidimensional pattern contour or color level recognition method, m is the patch size, the patch size is set to 216, and e is the number of color bands; The multidimensional pattern cube set is randomly divided into the recognition array U s , fusion array U l and the test array U c , the corresponding image label vector is divided into and Only the pattern outline or color level information around the target pixel is used.
[0097] Preferably, the multidimensional pattern contour or color level recognition method in step one is divided into three objects: color spectrum segment, pattern contour and color level information, which are respectively used to capture the color spectrum feature map, the pattern contour feature map and the color level information feature, and the three acquired features are fused for differentiated display; wherein, the color spectrum segment includes the color spectrum area and the color spectrum focus area; the pattern contour includes the pattern contour area and the pattern contour focus area; the color level information includes the color level information area and the color level focus area.
[0098] Preferably, the following model is used to identify the array and fuse the array in step 2:
[0099] (1) Multidimensional recognition network: Multidimensional recognition network is a common element in neural recognition models based on 3D image cubes; g ×m g ×e g Size of t g Feature map, a multi-dimensional recognition network layer, contains a size of γ g+1 ×γ g+1 ×h g+1 K g+1 channels, generating a size of m g+1 ×m g+1 ×e g+1 t g+1 Output feature map; the i-th output of the (g+1)-th multidimensional recognition network layer is calculated as:
[0100]
[0101]
[0102] in, is the j-th input feature map of the (g+1) layer, is the output of the discriminator of layer g; D(·) and Var(·) represent the expected and variance functions of the input respectively; V i g+1 and They represent the weights and biases of the (g+1)-layer multidimensional recognition network, * is the 3D recognition operation, and L(·) is the processing function that introduces the network non-smoothing processor;
[0103] (2) Concentrated connection: The concentrated area is the basic unit in pattern recognition. The output of the lth concentrated area is calculated as:
[0104] z l =V l [z0,z1,...,z l-1 ];
[0105] Among them, V lis a region containing the recognition layer, the processing layer, and the discriminator layer, z0,z1,...,z l-1 The more connections there are, the more image flows there are in the concentrated network. A concentrated network with S layers has S(S+1) / 2 connections, while a traditional recognition network with the same number of layers has only S direct connections.
[0106] (3) Attention strategy:
[0107] Pattern contour attention mapping U∈O c×c It is directly from the initial input F∈O c×m×m Calculated, where m×m is the size of the input area, c represents the number of input channels; replace F with F T Perform matrix multiplication to obtain the channel attention mapping U∈O c×c , connect the processing layer as follows:
[0108]
[0109] Among them, z ji represents the influence of the ith channel on the jth channel; F T The matrix multiplication result with F can be restored to O c ×m×m ; The reconstructed result is weighted by the parameter of the pattern outline size γ, and the input A is added to obtain the final color spectrum attention map D∈O c×m×m :
[0110]
[0111] Among them, γ is initialized to grayscale and learned step by step; the final graph D contains the weighted sum of all channel features, which is used to describe a dependency relationship and enhance the discriminability of features;
[0112] Color spectrum or color level attention area: Given an input feature F∈O c×m×m , two recognition layers are used to generate new color spectrum feature maps I and color level features J, where {I,J}∈O c×m×m ; I and J can be restored to O c×t , where t = m × m is the number of pixels; matrix multiplication is performed between I and J, additional processing layers are added, and the color spectrum or color level attention feature map P∈O is calculated. t×t :
[0113]
[0114] Among them, P ji Represents the influence of the i-th pixel to the j-th pixel; the closer the feature representations of two pixels are, the stronger the correlation between the pixels is;
[0115] The initial input feature F is sent to the recognition layer at the same time to obtain a new feature map M∈O c×m×m Can be restored to O c×t ; in M and P T The matrix multiplication operation is performed between them, and the result can be restored to O c×m×m :
[0116]
[0117] Among them, the initial value of χ is 1, and more weights are gradually learned and assigned; all positions and original features are added with certain weights to obtain the final feature D∈O c×m×m , so the context information on the color spectrum or color level value is modeled as D;
[0118] (4) Restorable multidimensional recognition: Restorable multidimensional recognition dynamically adjusts the restoration area of the receptive field according to the actual situation of the image. The input features of size C×H×W are passed through a multidimensional recognition network of size m×v×r to generate difference features of size 3N×C×H×W, where N=m×v×r is the size of the sampling grid; it has 3N values along the channel dimension, and the values represent the restorable values of the multidimensional sampling grid; the learned difference features are used for the restoration of the sampling grid of the multidimensional recognition network to generate a multidimensional sampling grid; the multidimensional sampling grid is used to generate the output features;
[0119] Multidimensionality is expressed by the following formula:
[0120]
[0121] Among them, Δm t Represents the difference corresponding to the t-th value in the m×v×r identification sampling grid, using BIM to generate accurate values; the BIM method formula is:
[0122]
[0123] (5) Processing function: The processing function used in multidimensional recognition is the fusion function. The formula of the fusion function is:
[0124]
[0125] Where z represents the input; the fusion function has an upper bound of unbounded and a lower bound of [≈-0.31,∞]; the differential coefficient of the fusion function is defined as:
[0126]
[0127] in,
[0128] (6) Selection of optimal weights. During the recognition process, the model with the highest accuracy on the fusion array is selected as the output. If the accuracy on the fusion array is consistent, the model with the smallest loss on the fusion array is selected as the output. The best model is saved for each recognition. If the model recognized next time is better, the last saved model is replaced, otherwise it is not replaced.
[0129] Preferably, in step 3, high color spectrum analysis is performed based on the processed image, and identifying the ground object by using the image and high color spectrum information includes:
[0130] The dataset F consists of N labeled pixels {b1,b2,…,b t}∈Β 1×1×m where m is the high color spectrum band, and the corresponding ground feature category label set is I = {e1, e2, …, e t}∈B 1×1×v , where q is the number of mural categories.
[0131] Preferably, in step 3, the quantitative indicator used to measure the difference between the predicted result and the true value is an indicator difference function, which is defined as:
[0132]
[0133] in, represents the pattern outline, color spectrum and color level information label vector of the mural predicted by the model, Q = [Q1, Q2, ..., Q L ] represents the pattern outline, color spectrum and color level information label vector of the real mural.
[0134] Through the above embodiments, the present invention combines the image and the color spectrum, which has both the pattern outline or color level characteristics of the image and the color spectrum characteristics of the substance.
[0135] The color spectrum has high resolution, which reduces the phenomenon of "different objects with the same spectrum" and "same spectrum with different objects";
[0136] With multiple bands, the selectivity of the color spectrum becomes flexible and diverse when dealing with analyses of different applications.
[0137] Embodiment 2: The historical relics restoration and display device provided by the embodiment of the present invention comprises:
[0138] The high color spectrum data acquisition module area is used to collect full-spectrum data of murals through high-resolution and high-color spectrum technology, and to restore the mural outline and color spectrum information through image recognition, enhancement algorithm and spectrum fusion technology.
[0139] Image processing module area, used for processing high-resolution images and high-color spectrum images.
[0140] The recognition model area is used to perform high color spectrum analysis based on the processed image and recognize the ground objects by using the image and high color spectrum information.
[0141] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0142] Since the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present invention, their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0143] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and module areas is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional units and module areas as needed, that is, the internal structure of the device can be divided into different functional units or module areas to complete all or part of the functions described above. The functional units and module areas in the embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and module areas are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and module areas in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment.
[0144] An embodiment of the present invention further provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above-mentioned method embodiments when executing the computer program.
[0145] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0146] An embodiment of the present invention further provides an information data processing terminal, which, when executed on an electronic device, provides a user input interface to implement the steps in the above method embodiments. The information data processing terminal is not limited to mobile phones, computers, and switches.
[0147] An embodiment of the present invention further provides a server, which is used to provide a user input interface to implement the steps in the above method embodiments when executed on an electronic device.
[0148] An embodiment of the present invention provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0149] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.
[0150] Simulation experiment:
[0151] like Figure 3 Graph information display (image cube) effect diagram;
[0152] like Figure 4 Regular photos in original color areas and post-destructed color areas with similar colors, such as Figure 5 The effect diagram of the processing result based on the atlas information of the present invention in the original color area and the later damaged color area with similar colors;
[0153] like Figure 6 For conventional photos without multi-spectral data fusion, image enhancement and edge extraction,
[0154] like Figure 7 This is the spectrum enhancement effect diagram after multi-spectral data fusion, image enhancement and edge extraction;
[0155] like Figure 8 Display (partial) effect diagrams of color spectrum information of different bands;
[0156] like Fig. 9 This is an image of a damaged item acquired by a regular HD camera.
[0157] like Fig.10This is a pseudo-color effect diagram for enhanced display of atlas information, such as Fig.11 This is a color display effect diagram based on the atlas information (test).
[0158] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered within the protection scope of the present invention.
Claims
1. A method for restoring and displaying historical relics, It is characterized in that The method includes: S1, high color spectrum data acquisition: Use high resolution and high color spectrum technology to collect full spectrum data of the murals, and restore the mural outline and color spectrum information through image recognition, enhancement algorithm and spectrum fusion technology; S2, processing of high-resolution images and high-color spectrum images; S3, based on the processed image, high color spectrum analysis is performed to identify the ground objects using the image plus high color spectrum information.
2. The method for restoring and displaying historical relics according to claim 1, It is characterized in that In step S1, high color spectrum data acquisition includes: indoor painted area image acquisition, restoration and modeling; Collection, restoration and modeling of painted images of interior halls.
3. The method for restoring and displaying historical relics according to claim 1, It is characterized in that The processing of painting high-resolution images and high-color spectrum images in step S2 includes: S201, feature extraction, target recognition and enhanced image analysis and processing of high-definition images; S202, the high color spectrum image is subjected to pattern, spectrum, color spectrum information recognition and fusion technology to obtain the pattern outline, color spectrum segment and color level information of the mural.
4. The method for restoring and displaying historical relics according to claim 3, It is characterized in that The pattern outline, color spectrum and color level information of the mural obtained in step S202 include: Step 1: Generate an array of multidimensional pattern cubes using a multidimensional pattern profile or color level recognition method, and randomly divide the multidimensional pattern cube array into a recognition array, a fusion array, and a test array; Step 2: the recognition array is used to update the pattern outline, color spectrum and color level information parameters of the mural for multiple recognitions, while the fusion array is used to fuse the performance of the recognition arrays and select the array with the best recognition; Step three, select a test array to verify the effectiveness of the recognition array, and obtain the results of distinguishing the pattern outline, color spectrum and color level information of the mural.
5. The method for restoring and displaying historical relics according to claim 4, It is characterized in that The step 1 specifically includes: Select the center pixel z from the original spot i The m×m neighboring pixels generate an array of multidimensional spot cubes {z 1 ,z 2 ,...,z n }∈O m×m×e ; If the target pixel is at the edge of the image, the missing adjacent pixel values are set to zero; In the multidimensional pattern contour or color level recognition method, m is the patch size, the patch size is set to 216, and e is the number of color bands; The multidimensional pattern cube set is randomly divided into the recognition array U s , fusion array U l and the test array U c , the corresponding image label vector is divided into and Only the pattern outline or color level information around the target pixel is used.
6. The method for restoring and displaying historical relics according to claim 4, It is characterized in that The multi-dimensional pattern contour or color level recognition method described in step one is divided into three objects: color spectrum segment, pattern contour and color level information, which are respectively used to capture the color spectrum feature map, pattern contour feature map and color level information features, and the three acquired features are fused for differentiated display; wherein, the color spectrum segment includes the color spectrum area and the color spectrum focus area; the pattern contour includes the pattern contour area and the pattern contour focus area; the color level information includes the color level information area and the color level focus area.
7. The method for restoring and displaying historical relics according to claim 4, It is characterized in that The following models are used to identify and fuse arrays in step 2: (1) Multidimensional recognition network: Multidimensional recognition network is a common element in neural recognition models based on 3D image cubes; g ×m g ×e g Size of t g Feature map, a multi-dimensional recognition network layer, contains a size of γ g+1 ×γ g+1 ×h g+1 K g+1 channels, generating a size of m g+1 ×m g+1 ×e g+1 t g+1 Output feature map; the i-th output of the (g+1)-th multidimensional recognition network layer is calculated as: in, is the j-th input feature map of the (g+1) layer, is the output of the discriminator of layer g; D(·) and Var(·) represent the expected and variance functions of the input respectively; V i g+1 and They represent the weights and biases of the (g+1)-layer multidimensional recognition network, * is the 3D recognition operation, and L(·) is the processing function that introduces the network non-smoothing processor; (2) Concentrated connection: The concentrated area is the basic unit in pattern recognition. The output of the lth concentrated area is calculated as: With l =V l [With 0 ,With 1 ,...,With l-1 ]; Among them, V l is a region that includes the recognition layer, processing layer, and discriminator layer. 0 ,z 1 ,...,z l-1 The more connections there are, the more image flows there are in the concentrated network. A concentrated network with S layers has S(S+1) / 2 connections, while a traditional recognition network with the same number of layers has only S direct connections. (3) Attention strategy: Pattern contour attention mapping U∈O c×c It is directly from the initial input F∈O c×m×m Calculated, where m×m is the size of the input area, c represents the number of input channels; replace F with F T Perform matrix multiplication to obtain the channel attention mapping U∈O c ×c , connect the processing layer as follows: Among them, z ji represents the influence of the ith channel on the jth channel; F T The result of matrix multiplication with F can be restored to O c×m×m ; The reconstructed result is weighted by the parameter of the pattern outline size γ, and the input A is added to obtain the final color spectrum attention map D∈O c×m×m : Among them, γ is initialized to grayscale and learned step by step; the final graph D contains the weighted sum of all channel features, which is used to describe a dependency relationship and enhance the discriminability of features; Color spectrum or color level attention area: Given an input feature F∈O c×m×m , two recognition layers are used to generate new color spectrum feature maps I and color level features J, where {I,J}∈O c×m×m ; I and J can be restored to O c×t , where t = m × m is the number of pixels; matrix multiplication is performed between I and J, additional processing layers are added, and the color spectrum or color level attention feature map P∈O is calculated. t ×t : Among them, P ji Represents the influence of the i-th pixel to the j-th pixel; the closer the feature representations of two pixels are, the stronger the correlation between the pixels is; The initial input feature F is sent to the recognition layer at the same time to obtain a new feature map M∈O c×m×m Can be restored to O c×t ; in M and P T The matrix multiplication operation is performed between them, and the result can be restored to O c×m×m : Among them, the initial value of χ is 1, and more weights are gradually learned and assigned; all positions and original features are added with certain weights to obtain the final feature D∈O c×m×m , so the context information on the color spectrum or color level value is modeled as D; (4) Restorable multidimensional recognition: Restorable multidimensional recognition dynamically adjusts the restoration area of the receptive field according to the actual situation of the image. The input features of size C×H×W are passed through a multidimensional recognition network of size m×v×r to generate difference features of size 3N×C×H×W, where N=m×v×r is the size of the sampling grid; it has 3N values along the channel dimension, and the values represent the restorable values of the multidimensional sampling grid; the learned difference features are used for the restoration of the sampling grid of the multidimensional recognition network to generate a multidimensional sampling grid; the multidimensional sampling grid is used to generate the output features; Multidimensionality is expressed by the following formula: Among them, Δm t Represents the difference corresponding to the t-th value in the m×v×r identification sampling grid, using BIM to generate accurate values; the BIM method formula is: (5) Processing function: The processing function used in multidimensional recognition is the fusion function. The formula of the fusion function is: Where z represents the input; the fusion function has an upper bound of unbounded and a lower bound of [≈-0.31,∞]; the differential coefficient of the fusion function is defined as: in, (6) Selection of optimal weights. During the recognition process, the model with the highest accuracy on the fusion array is selected as the output. If the accuracy on the fusion array is consistent, the model with the smallest loss on the fusion array is selected as the output. The best model is saved for each recognition. If the model recognized next time is better, the last saved model is replaced, otherwise it is not replaced.
8. The method for restoring and displaying historical relics according to claim 4, It is characterized in that In step 3, based on the processed image, high color spectrum analysis is performed, and the use of image high color spectrum information to identify ground objects includes: The dataset F consists of N labeled pixels {b 1 ,b 2 ,…,b t }∈Β 1×1×m Composition, where m is the high color spectrum band, and the corresponding ground feature category label set is I = {e 1 ,e 2 ,…,e t }∈B 1×1×v , where q is the number of mural categories.
9. The method for restoring and displaying historical relics according to claim 8, It is characterized in that In step 3, the quantitative indicator used to measure the difference between the predicted result and the true value is the indicator difference function, which is defined as: in, represents the pattern outline, color spectrum and color level information label vector of the mural predicted by the model, Q = [Q 1 ,Q 2 ,...,Q L ] represents the pattern outline, color spectrum and color level information label vector of the real mural.
10. A restoration and display device for historical relics, It is characterized in that The method for restoring and displaying historical relics according to any one of claims 1 to 9 is implemented, and the device comprises: The high color spectrum data acquisition module area is used to collect full-spectrum data of murals through high-resolution and high-color spectrum technology, and to restore the mural outline and color spectrum information through image recognition, enhancement algorithm and spectrum fusion technology. Image processing module area, used for processing high-resolution images and high-color spectrum images. The recognition model area is used to perform high color spectrum analysis based on the processed image and recognize the ground objects by using the image and high color spectrum information.
