Hyperspectral-based method and device for extracting hidden information of a smoked mural

Abstract

The application discloses a method for extracting hidden information of smoked murals based on hyperspectrum, comprising the following steps: S1, obtaining a hyperspectrum image of a mural; S2, performing independent component analysis on the hyperspectrum image to obtain an independent component image, and screening a main independent component image from the independent components; S3, performing open operation and close operation on the main independent component image using at least two different scale structure elements, superimposing the obtained open operation features, close operation features and original waveband information of the main independent component image, generating an extended morphological profile, performing principal component analysis on the extended morphological profile to obtain a principal component image; and S4, performing equalization stretching on the principal component image according to a histogram of the principal component image to obtain the hidden information. The application further provides an extraction device. The application can improve the extraction effect of hidden information of severely smoked murals.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mural information extraction. More particularly, the present application relates to a method and device for extracting hidden information of smoked murals based on hyperspectral imaging. BACKGROUND

[0002] Ancient murals in China are mainly distributed in temples. During religious ceremonies in temples, burning substances produced by burning incense, candles and oil lamps often adhere to the surface of the murals, forming smoke pollution, greatly obscuring the content of the murals and seriously affecting their visual effects. In this case, revealing the patterns covered by smoke will help understand the artistic techniques and creative intentions of the murals and provide new clues for the archaeological and historical research of the murals.

[0003] With the development of imaging technology, more and more non-destructive analysis techniques, including X-ray, multispectral and hyperspectral imaging technology, are applied in the protection of cultural relics. Among them, hyperspectral imaging technology has attracted much attention because it can capture hundreds of images within a continuous narrow spectral band (bandwidth <10 nm), covering a range usually including visible light (VIS, 400 - 750 nm) and near infrared (NIR, 750 - 2500 nm) regions. Currently, this technology is widely used in on-site analysis, information extraction, pigment identification, handwriting extraction and virtual restoration of artworks, etc.

[0004] Although preliminary studies on the application of hyperspectral imaging technology in artwork analysis have made significant progress, there are still great challenges in extracting hidden information from heavily smoked murals. The research on extracting hidden information from smoked murals is relatively limited. Existing studies have proposed using minimum noise separation transform to reduce the smoke interference in the mural background, enhancing the mural patterns through spectral feature analysis and image difference, and extracting pattern information through density segmentation. However, this method usually requires selecting a region of interest on the mural to separate the pattern and smoke. This method is obviously not suitable for heavily smoked murals, as it is almost impossible to identify any pattern information by naked eye in this case. Therefore, there is an urgent need to design a technical solution that can overcome the above-mentioned defects. SUMMARY

[0005] An object of the present application is to provide a method for extracting hidden information of smoked murals based on hyperspectral imaging, which can improve the extraction effect of hidden information of heavily smoked murals.

[0006] In order to achieve the objects and other advantages of the present application, one aspect of the present application provides a hyperspectral-based hidden information extraction method for smoked murals, comprising: S1, obtaining a hyperspectral image of a mural; S2, performing independent component analysis on the hyperspectral image to obtain independent component images, and screening a main independent component image from the independent components; S3, performing opening operation and closing operation on the main independent component image using at least two structural elements of different scales, superimposing the obtained opening operation features, closing operation features and original band information of the main independent component image to generate an extended morphological profile, and performing principal component analysis on the extended morphological profile to obtain a principal component image; and S4, performing equalization stretching on the principal component image according to a histogram of the principal component image to obtain the hidden information.

[0007] Further, in the S1, the hyperspectral image is further subjected to reflectivity correction using dark current data.

[0008] Further, in the S1, the hyperspectral data is further subjected to band clipping on bands with relatively large noise at the front and rear ends of the spectral range.

[0009] Further, in the S1, the hyperspectral image is further subjected to minimum noise fraction transform.

[0010] Further, in the S3, the main independent component image is divided into multiple regions, the smoked level of each region is calculated, the resolution of each region is changed according to the smoked level, then the main independent component image with changed resolution is subjected to opening operation and closing operation using at least two structural elements of different scales, the obtained opening operation features, closing operation features and original band information of the main independent component image are superimposed to generate an extended morphological profile, and principal component analysis is performed on the extended morphological profile to obtain a principal component image.

[0011] Further, in the S3, the structural elements are circular, and the main independent component image is subjected to opening operation and closing operation using structural elements of different radii.

[0012] Further, the S4 comprises: arranging a feature matrix of the extended morphological profile into a two-dimensional array, determining multiple principal components, performing PCA transformation on the two-dimensional array according to the multiple principal components to obtain a reduced-dimension feature array and the principal component image.

[0013] According to another aspect of the present application, there is further provided a hyperspectral-based hidden information extraction device for smoked murals, comprising: an acquisition module configured to acquire a hyperspectral image of a mural; a screening module configured to perform independent component analysis on the hyperspectral image to obtain independent component images, and screen a main independent component image from the independent components; a generation module configured to perform opening operation and closing operation on the main independent component image using at least two different scale structure elements, superimpose the obtained opening operation features, closing operation features and original band information of the main independent component image, generate an extended morphological profile, and perform principal component analysis on the extended morphological profile to obtain a principal component image; and a stretching module configured to perform equalization stretching on the principal component image according to a histogram of the principal component image to obtain the hidden information.

[0014] According to another aspect of the present application, there is further provided a hyperspectral-based hidden information extraction device for smoked murals, comprising a processor and a memory, the memory being configured to store program instructions, and the processor being configured to invoke the program instructions to perform the hyperspectral-based hidden information extraction method for smoked murals.

[0015] According to another aspect of the present application, there is further provided a computer readable storage medium storing a computer program, the computer program being configured to be executed by a processor to implement the hyperspectral-based hidden information extraction method for smoked murals.

[0016] The present application at least has the following beneficial effects:

[0017] The present application constructs a new hidden information extraction method, which not only fuses spectral information from different principal components, but also extracts multi-scale spatial features through opening / closing operation to further reveal local structure information (such as brightness change and edge features) of the image, so as to realize effective fusion of spatial-spectral features. The present application verifies the availability of the model through real temple smoked mural data, the model improves the effect of hidden information extraction for heavily smoked murals, makes up for the deficiency of the current extraction method, and has important academic research value and application prospect.

[0018] Other advantages, objects, and features of the present application will be apparent to those skilled in the art from the following description, and will be understood to be within the scope of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Flowchart for an embodiment of the present application;

[0020] Figure 2 Original heavily smoked image for an embodiment of the present application;

[0021] Figure 3An image of principal independent components for an embodiment of the present application;

[0022] Figure 4 An image containing implicit information for another embodiment of the present application.

[0023] Figure 5 An image of principal components for an embodiment of the present application;

[0024] Figure 6 An image containing implicit information for another embodiment of the present application. DETAILED DESCRIPTION

[0025] The present application will be further described below in conjunction with the accompanying drawings, so that those skilled in the art can implement the present application according to the description in the specification.

[0026] It should be understood that the terms such as "have", "contain", and "include" used in the embodiments of the present application do not exclude the presence or addition of one or more other elements or combinations thereof. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative positional relationship, movement condition, etc. between components in a certain posture, and if the certain posture changes, the directional indications also change accordingly. When an element is referred to as "fixed to" or "disposed on" another element, it can be directly on another element or can have a middle element. When an element is referred to as "connected to" another element, it can be directly connected to another element or indirectly connected to another element through a middle element. The descriptions of "first", "second", etc. in the embodiments of the present application are only for the purpose of description and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include at least one of the features.

[0027] It should be noted that the technical solutions of the various embodiments of the present application can be combined with each other, but must be based on the fact that a person skilled in the art can implement it, and when the combination of technical solutions contradicts each other or cannot be implemented, it should be considered that the combination of technical solutions does not exist and is not within the protection scope claimed by the present application.

[0028] As shown in Figure 1 , the embodiments of the present application provide a hyperspectral-based smoke mural (see Figure 2 ) implicit information extraction method, which includes:

[0029] S1: Obtain a hyperspectral image of the mural;

[0030] Exemplarily, a device specially applicable to hyperspectral imaging is selected, such as an imaging spectrometer with high spectral resolution and spatial resolution; before collection, the device needs to be calibrated to ensure that the wavelength range of each waveband is accurate, the light intensity detection sensitivity is stable, and the performance of the image sensor is in the best state; the calibration process includes using a standard reference plate with known spectral characteristics to correct the response of the device at different wavebands, recording and adjusting the device parameters, so that the device can accurately obtain the reflection or radiation information of the target mural at multiple continuous spectral wavebands.

[0031] Exemplarily, considering that the mural may have complex textures, different light reflection characteristics, and occlusions, the hyperspectral image of the mural can be collected from multiple directions and angles. For example, around the center of the mural, take pictures at certain angles in the horizontal direction and at different height positions in the vertical direction, respectively, to obtain multiple sets of hyperspectral image data. In this way, the various regions of the mural can be fully covered, and as much complete spectral information as possible can be collected to provide a richer data basis for subsequent processing and accurate extraction of hidden information.

[0032] S2: Independent component analysis is performed on the hyperspectral image to obtain independent component images, and main independent component images are selected from the independent components, see Figure 3 ;

[0033] In this step, the independent component analysis is based on the assumption that the data is linearly mixed by a group of independent signal sources and can be decomposed by the statistical independence of the sources; in the hyperspectral image, the spectral characteristics of different substances, the reflection characteristics of different regions of the mural, and other information can be regarded as these independent signal sources, which are mixed together to form the complex hyperspectral image data collected by us; a suitable ICA algorithm is selected, such as the FastICA algorithm, which has the advantages of fast convergence speed and relatively high computational efficiency, and can decompose the hyperspectral image to obtain independent component images. The independent component image generation process is as follows:

[0034] The collected and sorted hyperspectral image data is input into the selected ICA algorithm, which will decompose the hyperspectral image according to the statistical characteristics of the image data, such as the correlation between wavebands, the covariance structure of the data, etc. In this process, through complex mathematical operations, various information mixed together in the original hyperspectral image is separated according to its independence, and each independent component image represents a relatively independent information source, for example, a certain independent component image may mainly reflect the spectral characteristics of the pigments in the mural, and another independent component image highlights the spectral changes corresponding to the smoke marks, etc.

[0035] In order to select the main independent component images from a large number of independent component images, it is necessary to set reasonable screening criteria; screening can be based on the variance contribution rate of the image. The higher the variance contribution rate, the greater the proportion of the information contained in the independent component image in the total amount of hyperspectral image information, and the more it can reflect the main characteristics of the image; specifically calculate the variance contribution rate of each independent component image, set a threshold (such as the variance contribution rate is greater than 20%), and select the independent component images that meet the threshold conditions as the main independent component images; in addition, it is also possible to combine professional field knowledge. For example, for smoked murals, focus on those independent component images that may be related to the spectral characteristics of the smoked material and the original pattern of the mural, and judge and screen based on the shape of the spectral curve, peaks in specific bands and other characteristics to ensure that the selected main independent component images are of great value for the subsequent extraction of implicit information.

[0036] S3: performing opening and closing operations on the main independent component image using at least two structuring elements of different scales, superimposing the obtained opening operation features and closing operation features with the original band information of the main independent component image to generate an extended morphological profile, and performing principal component analysis on the extended morphological profile to obtain a principal component image;

[0037] In this step, appropriate geometric shapes are selected as structuring elements. Common shapes include circles, rectangles, and ovals. For smoky mural images, circular structuring elements are usually a good choice, considering that their patterns and smoke marks often have irregular shapes and obvious isotropic characteristics. However, other shapes can also be used for supplementary analysis based on the texture and pattern characteristics of the specific mural. To meet the needs of different scales, at least two structuring elements with different radii are set, such as 1, 2, and 3 (the specific values ​​can be adjusted based on factors such as image resolution and the desired scale of spatial features to be extracted). By changing the radius of the structuring element, multi-scale analysis can be achieved. Different scales can capture image spatial structural features from fine to relatively macroscopic.

[0038] The opening operation is first carried out, and then the expansion operation is carried out. For each main independent component image, the opening operation is carried out in turn with a selected structural element of different scales; in the erosion operation, the structural element slides on the image, and if the pixels in the region covered by the structural element meet the erosion rule (for example, for a binary image, if the center pixel in the region covered by the structural element is not completely the same as the pixel corresponding to the structural element, the center pixel is set to a background pixel), the target object (such as the pattern part in the mural, the plaque formed by smoking, etc.) in the image will have its boundary inwardly contracted, and some isolated small bright noise or small protruding part will be removed; then the expansion operation is carried out, which will make the target object contracted after the erosion operation outwardly expand again to recover some, but the isolated small part removed before will not reappear, and through such a process, the target structural features brighter than the surrounding region are extracted, and the opening operation features under different scales are obtained.

[0039] The closing operation is opposite to the opening operation, that is, the expansion operation is first carried out, and then the erosion operation is carried out; first, the expansion operation is carried out, and in the sliding process of the structural element on the image, the dark region (such as the gap in the mural pattern, the hole in the smoking trace, etc.) in the image is expanded outwardly, some space cracks are filled, and the originally discontinuous dark region is connected; then the erosion operation is carried out, which makes the expanded region appropriately contract back to a reasonable size, smooths the image edge, and thus extracts the structural features darker than the surrounding region, and the closing operation features under different scales are obtained.

[0040] The opening operation and closing operation features obtained at different scales are superimposed and integrated with the original band information of the corresponding main independent component image. This superimposing process combines the spatial structure features excavated at different scales with the original spectral band information, so that the generated extended morphological profile (EMP) contains both the original spectral feature information of the image and the multi-scale spatial morphological features, which can more comprehensively depict various features of the smoked mural in the hyperspectral image and provide a rich feature basis for further analysis. For example, the original band information can reflect the spectral characteristics of the mural pigments, while the opening and closing operation features can reflect the spatial distribution of the patterns, the degree of smoke coverage, and the spatial structure information such as the shape, etc., which together constitute the EMP. For example, S2 obtains the main independent component images IC1 and IC2. For IC1, multi-scale opening and closing operations are defined, and the obtained features can include OpeningR=1, OpeningR=2, and ClosingR=1, ClosingR=2. Similarly, the profile of IC2 will also generate corresponding structure features. Subsequently, by superimposing the multi-scale profiles of each principal component, a complete extended morphological profile (EMP) is formed. At this time, the EMP contains multi-scale features from each principal component. If IC1 and IC2 each contain four features, then the EMP will contain a total of eight features. Therefore, for each band, the final EMP feature number is 1+2+2+1+2+2=10, i.e., it contains the original band data and four morphological features at different scales.

[0041] The generated extended morphological profile is subjected to principal component analysis. PCA will recombine and reduce the dimensions of the features in the EMP according to the variance of the data. By calculating the covariance matrix of the EMP data, mathematical operations such as eigenvalue decomposition or singular value decomposition are performed to find the directions that best represent the data changes as the principal components. A suitable number of principal components (such as determining the number of principal components according to the cumulative variance contribution rate reaching a certain percentage, such as more than 80%) are selected to extract the most representative features, which maximizes the information while reducing the feature dimension. After PCA transformation, the high-dimensional extended morphological profile data is converted into low-dimensional principal component images. Each principal component image reflects the key features of the smoked mural from different angles, such as some principal component images that highlight the texture features of the mural patterns, and some that emphasize the spatial relationship between the smoke marks and the patterns, etc.

[0042] S4: performing equalization stretching on the principal component image according to the histogram of the principal component image to obtain the hidden information, see Figure 4 ;

[0043] In this step, firstly, the histogram of the principal component image is analyzed. The histogram shows the distribution of the number of pixels of each gray level in the principal component image. By counting the number of pixels corresponding to different gray values, the gray distribution characteristics of the principal component image are understood. For example, it is checked whether the gray values are concentrated in a certain narrow interval, resulting in low image contrast and difficulty in distinguishing details, or whether the number of pixels in a certain gray value range is extremely small, indicating that the corresponding feature information is not obvious in the image. These histogram information provides a basis for subsequent equalization stretching operation;

[0044] Based on the principle of histogram equalization, the principal component image is equalized and stretched. This operation aims to redistribute the gray values of the image pixels, widen the gray range of the image, and enhance the contrast of the image, so that the details in the image are more clear and visible. The specific implementation is to first perform integral operation on the histogram to generate a cumulative histogram. The cumulative histogram records the cumulative number of pixels from the darkest gray value to a certain specific gray value. Then, a gray mapping function is constructed according to the cumulative histogram. This function establishes the corresponding relationship between the original gray values in the input image and the new gray values. By mapping the original gray values to the new gray values through this function, the gray values of the image are re-adjusted;

[0045] After equalization and stretching, the principal component image, whose hidden information originally obscured by smoke, is difficult to perceive due to low contrast or narrow gray range, is highlighted. These hidden information may include the original pattern details obscured by smoke in the mural, the fading of pigments, the penetration of smoke, and the like.

[0046] The present embodiment constructs a new hidden information extraction method. This method not only integrates spectral information from different principal components, but also extracts multi-scale spatial features through open / close operation to further reveal the local structure information of the image (such as brightness change and edge feature), thereby realizing effective fusion of spatial-spectral features and improving the effect of hidden information extraction of heavily smoked murals, making up for the shortcomings of current extraction methods, and having important academic research value and application prospect.

[0047] In another embodiment, in the S1, the hyperspectral image is also subjected to reflectance correction using dark current data.

[0048] Specifically, the hyperspectral image is subjected to reflectance correction, the purpose of which is to eliminate interference and obtain data of true reflectance. In addition, in some cases, the acquisition of the hyperspectral image also introduces noise. Therefore, the mural image must be subjected to certain noise removal to obtain a better final output effect. The correction formula is:

[0049]

[0050] In the formula, R is the reflectance, Rraw is the collected hyperspectral data, R dark is the dark current data, R white is the standard reflectance plate data, the reflectivity of the standard reflectance plate is 99%.

[0051] In another embodiment, in the S1, further comprising cropping the wavebands in the hyperspectral data that have relatively large noise in the front and rear sections of the spectral range; in the front section of the spectral range, there may be more noise in the collected data due to factors such as instability of the sensitivity of the detector in the initial waveband, insufficient precision of the initial calibration of the light energy, etc.; and in the rear section of the spectral range, there may also be relatively large noise due to factors such as light propagation loss and limitations of the response of the detector to long-wavelength light; for example, the standard deviation of the data of the first 50 wavebands and the last 50 wavebands is obviously higher than that of the middle wavebands, so it can be determined that these wavebands are the cropping objects. After cropping the wavebands with relatively large noise, the quality of the hyperspectral data is significantly improved; after cropping, the data volume is reduced to a certain extent, reducing unnecessary data operation in the subsequent processing process, improving the overall data processing efficiency, and more accurately reflecting the real spectral characteristics and spatial structure information of the mural; this is very beneficial to subsequent operations such as independent component analysis and extended morphological profile generation, and can avoid deviations in the analysis results or interference with the accurate extraction of the hidden information due to the existence of noise wavebands.

[0052] In another embodiment, in the S1, further comprising performing minimum noise separation transformation on the hyperspectral image; specifically, analyzing the statistical characteristics of the hyperspectral image data, calculating the covariance matrix and the noise covariance matrix, performing eigenvalue decomposition on the calculated noise covariance matrix to obtain corresponding eigenvalues and eigenvectors, arranging these eigenvalues in descending order, and the eigenvalues reflect the variance contribution size of the components represented by the corresponding eigenvectors in the image, that is, the importance degree; according to the sorted eigenvalues and eigenvectors, a minimum noise separation transformation matrix is constructed, and the original hyperspectral image data is multiplied by the transformation matrix to realize minimum noise separation transformation of the image; so that the components in the transformed new components mainly contain effective signal information and have relatively less noise, and the farther the components are, the greater the proportion of noise is, so as to achieve the effect of separating noise and highlighting effective information.

[0053] In another embodiment, in the S3, the main independent component image is segmented into multiple regions, the smoke grade of each region is calculated, the resolution of each region is changed according to the smoke grade, then open operation and close operation are performed on the main independent component image with at least two different scale structure elements, the obtained open operation features and close operation features are superimposed with the original band information of the main independent component image to generate an extended morphological profile, principal component analysis is performed on the extended morphological profile to obtain a principal component image, see Figure 5 ; the histogram according to Figure 5 is used to perform equalization stretching on the principal component image to obtain the hidden information, see Figure 6 ;

[0054] Compared with the foregoing embodiments, the main independent component image is segmented into multiple regions, the smoke grade of each region is calculated, and the resolution of each region is changed according to the smoke grade, which is more complex, but the obtained hidden information is significantly richer (compare Figure 4 and Figure 6 );

[0055] For example, the gray level histogram or spectral feature value distribution of the main independent component image is analyzed to find a suitable threshold to distinguish different regions; for example, there may be obvious gray level difference between the smoke region and the relatively non-smoke region in the smoke wall painting under certain spectral bands, and one or more thresholds are determined by calculating the mean, variance and other statistical quantities of different pixels in these key bands to divide the image pixels into different categories, thereby forming different regions.

[0056] For example, a large number of labeled smoke wall painting image samples (annotating different region categories, such as heavy smoke region, light smoke region, non-smoke region, etc.) are collected, and a U-Net deep learning model can be trained using these samples; the main independent component image is input into the trained model, and the model can automatically output the segmentation result of the image, output the segmentation mask of different regions, and accurately divide each region.

[0057] For example, the average gradient G is used to objectively describe the definition of the image, the larger the value of G, the more details and clearer the image, which proves that the smoke grade is low, and vice versa, the image is more blurred, which proves that the smoke grade is high; that is,

[0058]

[0059] In the formula, m and n are the width and height of the image, respectively; and are the gradients of the image in the i and j directions, respectively.

[0060] With the above average gradient G, combined with experimental analysis or reference to existing research results, the specific standard of smoke grade classification is formulated; for example, the smoke grade can be divided into three grades of light, moderate and heavy;

[0061] Exemplarily, a mapping relationship between smoke grade and resolution is established: for the area of heavy smoke grade, since the information contained therein may become blurred and complex due to smoke, higher resolution is needed to restore details as much as possible, and higher resolution enhancement method can be used, such as resampling technology to enhance the resolution to 2-3 times of the original resolution, and interpolation algorithm such as bilinear interpolation can be used for resampling to ensure the quality and detail distinguishability of the image after resolution enhancement; for the area of moderate smoke grade, the resolution is appropriately enhanced, for example, the resolution is enhanced to about 1.5 times of the original resolution; for the area of light smoke grade, since the features are relatively clear, in order to reduce the data volume and calculation amount, the resolution can be reduced to about 0.5-0.8 times of the original resolution;

[0062] Exemplarily, the extracted opening operation features, closing operation features and original waveband information are superimposed and integrated according to certain rules to form an extended morphological profile; for example, different features can be represented in the form of vectors, each feature corresponds to a dimension in the vector, and then the vectors are spliced to construct an extended morphological profile containing rich image feature information, which not only covers the changes of the image after morphological processing, but also retains the original spectral information.

[0063] For the areas in the image with relatively simple, clear and little changed features, lower resolution is used for processing, which can reduce the data volume contained in these areas, avoid excessive calculation of these relatively simple information in the subsequent processing process, save storage resources and calculation time, and improve the overall data processing efficiency; when high resolution is allocated to the key areas that need more detailed analysis (such as parts with more hidden information and more blurred smoke), although the data volume and processing complexity of these areas increase relatively, since only part of the image is involved, the overall calculation burden will not be too heavy, and at the same time, detailed analysis of the key areas can be carried out quickly and effectively; moreover, if high resolution is used uniformly for the whole image, unnecessary noise may be introduced or the originally clear features may become indistinguishable in the complex high resolution data when processing some simple areas; on the contrary, if low resolution is used uniformly, important details of the key areas may be lost; by allocating different resolutions to different areas, the embodiment can maintain the clarity of the information in each area without losing key details, avoid information confusion caused by improper resolution selection, and make the finally extracted hidden information more accurate and complete; comparison Figure 4 andFigure 6 It is also known that, Figure 4 With uniform resolution, noise is larger, and key information is not prominent, Figure 6 Then the pattern is more obvious.

[0064] In another embodiment, in the S3, the structural elements are circular, and the open and close operations are performed on the main independent component image using structural elements of different radii;

[0065] In this step, for the processing of the smoke wall painting image, the circular structural element has unique advantages. The smoke marks and wall painting patterns often exhibit irregular shapes and relatively uniform changes in all directions, with certain isotropic characteristics. The circular structural element can better adapt to this characteristic. When performing morphological operations, the processing effect of the target object (such as the wall painting pattern, smoke stain, etc.) in the image is relatively consistent in all directions, and will not overemphasize or suppress the characteristics in certain specific directions due to the directionality of the structural element shape. Compared with structural elements such as rectangles that have obvious directionality, the circular structural element can more comprehensively and evenly detect and extract the spatial structure information of the image, whether it is a circular or elliptical pattern outline or an irregularly shaped but isotropic smoke area. The circular structural element can interact with them in a relatively natural and accurate manner, thereby laying a good foundation for subsequent accurate extraction of hidden information.

[0066] Different radii of circular structural elements are set to realize multi-scale morphological operations. Circular structural elements with smaller radii can capture fine and small spatial structure details in the image. For example, there may be some subtle texture changes, small particle distribution of pigments, or small traces produced by early smoke erosion in the wall painting. When performing open and close operations, small-radius circular structural elements can accurately extract and strengthen these subtle features. As the radius gradually increases, the range of the area covered by the circular structural element becomes wider, and at this time, relatively more macro spatial structure characteristics can be reflected, such as the distribution regularity of large-area wall painting patterns, the overall morphology of large smoke areas, and the spatial relationship between them, etc.

[0067] When performing the opening operation on the main independent component image, circular structure elements with different radii are processed in sequence. For a circular structuring element with a radius of , as it slides across the image, the opening operation first performs an erosion process. During the erosion stage, if the center pixel in the area covered by the circular structuring element is not exactly the same as the pixel at the corresponding position of the structuring element (for example, in a binary image, according to the set pixel matching rule), the center pixel is considered a background pixel, causing the boundary of the target object in the image to shrink inward. This process removes some isolated small bright noise points or small protrusions on the edge of the target object, making the target object shape more regular and smooth. The dilation operation then causes the target object to expand outward to a certain extent after the erosion, but the isolated small parts that were previously eroded will not reappear. For example, for some tiny bright spot noise caused by paint peeling or slight smoke in a mural, the opening operation of a small-radius circular structuring element can effectively remove it while preserving the integrity of the main pattern. For larger pattern areas, the opening operation of a large-radius circular structuring element can highlight the overall outline, making it more clearly distinguishable from the surrounding background. The opening operation features at different radii are obtained, which reflect the spatial structural characteristics of the target object in the image at different scales.

[0068] The order of the closing operation is opposite to that of the opening operation, dilation first and then erosion; when the circular structuring element is dilated, it slides on the image, causing the dark areas in the image (such as gaps in mural patterns, holes in smoke traces, etc.) to expand outward, filling some small spatial cracks and connecting the originally discontinuous dark areas. An erosion operation is then performed to shrink the expanded area back to a reasonable size, thereby smoothing the image edges and extracting structural features that are darker than the surrounding areas. For example, when processing irregular holes caused by smoke in smoky murals or small dark areas within the patterns, the closing operation of a small-radius circular structuring element can appropriately fill and regularize these dark areas, making their spatial connection with the surrounding areas clearer. The closing operation of a large-radius circular structuring element can perform overall morphological adjustment and feature extraction on a larger range of dark areas (such as dark blocks covered by large areas of smoke), obtaining closing operation features at different radii. These features complement the opening operation features and together constitute multi-scale spatial structural information, which helps to more comprehensively characterize the characteristics of the smoky mural image.

[0069] The features obtained by the opening operation and the closing operation of the different radius circular structural elements are superimposed with the original band information of the main independent component image to generate an extended morphological profile. The features extracted by the multi-scale circular structural element operation inject rich spatial structure information into the extended morphological profile. The original band information mainly reflects the spectral characteristics of the mural, while the features obtained by the multi-scale operation of the circular structural elements are a deep analysis of the image from the spatial dimension, such as from the microscopic texture details to the macroscopic pattern distribution and the spatial morphology of the smoking coverage. The combination of them enables the extended morphological profile to comprehensively describe the features of the smoking mural in multiple dimensions (spectrum and space), provides a more powerful feature basis for subsequent principal component analysis and final extraction of hidden information, and improves the accuracy and integrity of the entire method for extracting hidden information of the smoking mural.

[0070] In another embodiment, the S4 comprises: arranging the feature matrix of the extended morphological profile into a two-dimensional array, determining a plurality of principal components, performing PCA transformation on the two-dimensional array according to the plurality of principal components, and obtaining a dimension-reduced feature array and the principal component image;

[0071] In this step, the principal component analysis (PCA) algorithm generally requires that the input data is in the form of a two-dimensional array, which facilitates the algorithm to accurately mine the principal component information in the data based on matrix operations and perform subsequent dimension reduction operations. Therefore, it is necessary to rearrange the feature matrix of the extended morphological profile to convert it into a two-dimensional array structure that meets the input requirements of the PCA algorithm.

[0072] Each principal component corresponds to a certain variance. The greater the variance, the more significant the change in the data direction represented by the principal component, and the more information it contains that can explain the overall data. When determining a plurality of principal components, the variance contribution rate is usually used for screening. The variance contribution rate refers to the proportion of the variance of a principal component in the total sum of the variances of all principal components, which directly reflects the contribution of the principal component to the overall data information. According to actual needs and data characteristics, a suitable threshold is set, such as setting the variance contribution rate to be greater than 80% (the specific value can be adjusted flexibly according to the requirement for information retention degree and the complexity of the data itself, etc.), and the principal components that meet the threshold condition are selected as the principal components to be retained. In this way, among the numerous possible principal components, those that play a key role in data variation and best represent the main features of the extended morphological profile can be focused on, and too many relatively unimportant principal components can be avoided, thereby realizing effective dimension reduction of the data and extraction of key information.

[0073] The PCA transformation is essentially a linear transformation, which constructs a transformation matrix based on the feature vectors corresponding to the plurality of principal components determined in advance, and then projects the extended morphological profile data in the form of a two-dimensional array onto a new coordinate axis system determined by the principal components; assuming that the feature matrix of the extended morphological profile has a dimension of m*n*p, and assuming that k principal components are determined, the corresponding feature vector matrix is V (V is a p*k matrix, where p is the number of columns of the original two-dimensional array, that is, the number of features of each pixel, and k is the number of principal components), for the two-dimensional array form of the extended morphological profile data X (the size is mn*p), through matrix multiplication operation Y=XV (Y is the data matrix obtained after PCA transformation, the size is changed to mn*k), realize the projection of the original high-dimensional data into the low-dimensional space composed of k principal components, this process is the core operation of PCA transformation; in this transformation process, the dimension of the data is reduced from p to k, realizing the purpose of dimension reduction; the principal component image is obtained according to the feature array after dimension reduction.

[0074] The embodiment of the present application also provides a hyperspectral-based hidden information extraction device for a smoked mural, comprising: an acquisition module configured to acquire a hyperspectral image of the mural; a screening module configured to perform independent component analysis on the hyperspectral image to obtain independent component images, and screen out main independent component images from the independent components; a generation module configured to perform opening operation and closing operation on the main independent component images using at least two different scale structure elements, superimpose the obtained opening operation features, closing operation features and original band information of the main independent component images, generate an extended morphological profile, and perform principal component analysis on the extended morphological profile to obtain principal component images; and a stretching module configured to perform equalization stretching on the principal component images according to a histogram of the principal component images to obtain the hidden information.

[0075] The embodiment of the present application utilizes a computer program to construct the acquisition module, the screening module, the generation module and the stretching module. The generation module not only fuses spectral information from different principal components, but also extracts multi-scale spatial features through opening / closing operation to further reveal local structure information (such as brightness change and edge features) of the image, so as to realize effective fusion of spatial-spectral features, improve the hidden information extraction effect of a severely smoked mural, make up for the shortcomings of the current extraction method, and has important academic research value and application prospect.

[0076] The embodiment of the present application also provides a device for extracting implicit information of smoked murals based on hyperspectrum, which comprises a processor and a memory, the memory is used for storing program instructions, and the processor is used for calling the program instructions to execute the method for extracting implicit information of smoked murals based on hyperspectrum; the device of the embodiment can be a mobile phone, a notebook computer, a tablet computer, a vehicle-mounted terminal, a drone, etc., which is internally provided with a memory and a processor to execute the method for extracting implicit information of smoked murals based on hyperspectrum of the above embodiment.

[0077] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the method for extracting implicit information of smoked murals based on hyperspectrum; in the embodiment, the computer readable medium can be a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, etc., which can store a computer program to execute the method for extracting implicit information of smoked murals based on hyperspectrum of the above embodiment.

[0078] Although the embodiments of the present application have been disclosed as above, they are not limited to the application listed in the specification and the embodiments, and can be fully applied to various fields suitable for the present application, and other modifications can be easily realized by those skilled in the art, and therefore the present application is not limited to specific details and the figures shown and described herein, without departing from the general concept defined by the claims and the equivalent scope.

Claims

1. A method for extracting implicit information from smoked murals based on hyperspectral, characterized by: include: S1: Acquire hyperspectral images of murals; S2: performing independent component analysis on the hyperspectral image to obtain independent component images, and screening out main independent component images from the independent components; S3: Segmenting the main independent component image into multiple regions, calculating the smoke level of each region, changing the resolution of each region according to the smoke level, then performing opening and closing operations on the main independent component image after the resolution change using structuring elements of at least two different scales, superimposing the obtained opening and closing operation features with the original band information of the main independent component image to generate an extended morphological profile, and performing principal component analysis on the extended morphological profile to obtain a principal component image; S4: performing equalization stretching on the principal component image according to the histogram of the principal component image to obtain the implicit information.

2. The method for extracting implicit information from smoked murals based on hyperspectral analysis as claimed in claim 1, wherein: In the S1, the method further includes performing reflectance correction on the hyperspectral image using dark current data.

3. The method for extracting implicit information from smoked murals based on hyperspectral analysis as claimed in claim 1, wherein: In the S1, the step further includes cropping the bands with larger noise in the front and back sections of the spectral range of the hyperspectral image.

4. The method for extracting implicit information from smoked murals based on hyperspectral analysis as claimed in claim 1, wherein: In the S1, it also includes performing minimum noise separation transformation on the hyperspectral image.

5. The method for extracting implicit information from smoked murals based on hyperspectral analysis as claimed in claim 1, wherein: In S3, the structural element is a circle, and the structural elements with different radii are used to perform opening and closing operations on the main independent component images.

6. The method for extracting implicit information from smoked murals based on hyperspectral analysis as claimed in claim 1, wherein: The S4 includes: The feature matrix of the extended morphological section is arranged into a two-dimensional array, a plurality of principal components are determined, and the two-dimensional array is subjected to PCA transformation according to the plurality of principal components to obtain a feature array after dimensionality reduction and the principal component image.

7. The device for extracting implicit information from smoked murals based on hyperspectral is characterized by: include: An acquisition module, used to acquire hyperspectral images of murals; a screening module, configured to perform independent component analysis on the hyperspectral image to obtain independent component images, and screen out main independent component images from the independent components; a generation module, configured to segment the main independent component image into a plurality of regions, calculate a smoke level for each region, change the resolution of each region according to the smoke level, then perform opening and closing operations on the main independent component image after the resolution change using structuring elements of at least two different scales, superimpose the obtained opening and closing operation features with the original band information of the main independent component image to generate an extended morphological profile, and perform principal component analysis on the extended morphological profile to obtain a principal component image; The stretching module is used to perform equalization stretching on the principal component image according to the histogram of the principal component image to obtain the implicit information.

8. The device for extracting implicit information from smoked murals based on hyperspectral is characterized by: The method comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions to execute the method for extracting implicit information from smoked murals based on hyperspectral according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for extracting implicit information from smoked murals based on hyperspectral is implemented as described in any one of claims 1 to 6.

Images