Hyperspectrum-based smoked mural implicit information extraction method and device

Abstract

The invention discloses a method for extracting hidden information of a smoked mural based on a hyperspectrum. The method comprises the following steps: S1, acquiring a hyperspectral image of the mural; s2, performing independent component analysis on the hyperspectral image to obtain independent component images, and screening out main independent component images from the independent components; and S3, carrying out opening operation and closing operation on the main independent component image by using at least two structural elements with different scales, superposing the obtained opening operation characteristics and closing operation characteristics with the original wave band information of the main independent component image to generate an expanded morphological section, carrying out principal component analysis on the expanded morphological section, and obtaining the main component of the main independent component image. Obtaining a principal component image; and S4, according to the histogram of the principal component image, performing equalization stretching on the principal component image to obtain the implicit information. The invention further provides an extraction device. According to the method, the extraction effect of the implicit information of the severe smoked mural can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mural information extraction. More specifically, the present invention relates to a method and device for extracting implicit information of smoked murals based on hyperspectral imaging. Background Art

[0002] Ancient murals in China are mainly distributed in temples. During religious ceremonies in temples, the combustion substances generated by burning incense, candles, and oil lamps often adhere to the surface of the murals, forming smoked pollution, which greatly obscures the mural content and seriously affects its visual effect. In this case, revealing the patterns covered by smoke will help to understand the artistic techniques and creative intentions of the murals and provide new clues for the archaeological and historical research of the murals.

[0003] With the development of imaging technology, more and more non-destructive analysis techniques, including X-ray, multispectral, and hyperspectral imaging technologies, have been applied to cultural relics protection. Among them, hyperspectral imaging technology has attracted much attention because it can capture hundreds of images within a continuous narrow spectral band (bandwidth < 10nm), and the coverage range usually includes the visible light (VIS, 400 - 750nm) and near-infrared (NIR, 750 - 2500 nm) regions. Currently, this technology is widely used in fields such as on-site analysis, information extraction, pigment identification, handwriting extraction, and virtual restoration of artworks.

[0004] Although significant progress has been made in the preliminary research on applying hyperspectral imaging technology to artwork analysis, there are still great challenges in extracting implicit information from heavily smoked murals. The research on extracting implicit information from smoked murals is relatively limited. Existing research has proposed using minimum noise separation transform to reduce the smoked interference in the mural background, enhancing mural patterns through spectral feature analysis and image difference, and extracting pattern information through density segmentation. However, this method usually requires selecting regions of interest on the murals to separate the patterns and the smoke. This method is obviously not applicable to heavily smoked murals because it is almost impossible to identify any pattern information by the naked eye in this case. Therefore, there is an urgent need to design a technical solution that can overcome the above defects. Summary of the Invention

[0005] An object of the present invention is to provide a method for extracting implicit information of smoked murals based on hyperspectral imaging, which can improve the extraction effect of implicit information from heavily smoked murals.

[0006] To achieve these objects and other advantages of the present invention, according to one aspect of the present invention, there is provided a method for extracting implicit information of smoked murals based on hyperspectral, including: S1: acquiring a hyperspectral image of the mural; S2: performing independent component analysis on the hyperspectral image to obtain an independent component image, and screening out the main independent component image from the independent components; S3: performing opening and closing operations on the main independent component image using at least two structural elements of different scales, superimposing the obtained opening operation features and closing operation features on the original band information of the main independent component image to generate an extended morphological profile, and performing principal component analysis on the extended morphological profile to obtain a principal component image; S4: performing equalization stretching on the principal component image according to the histogram of the principal component image to obtain the implicit information.

[0007] Further, in the S1, it further includes performing reflectance correction on the hyperspectral image using dark current data.

[0008] Further, in the S1, it further includes cropping the bands with larger noise in the front and rear sections of the spectral range in the hyperspectral data.

[0009] Further, in the S1, it further includes performing minimum noise separation transformation on the hyperspectral image.

[0010] Further, in the S3, the main independent component image is segmented into multiple regions, the smoked level of each region is calculated, the resolution of each region is changed according to the smoked level, and then opening and closing operations are performed on the main independent component image with the changed resolution using at least two structural elements of different scales, the obtained opening operation features and closing operation features are superimposed on the original band information of the main independent component image to generate an extended morphological profile, and principal component analysis is performed on the extended morphological profile to obtain a principal component image.

[0011] Further, in the S3, the structural element is circular, and the opening and closing operations are performed on the main independent component image using the structural elements with different radii.

[0012] Further, the S4 includes: arranging the feature matrix of the extended morphological profile into a two-dimensional array, determining a plurality of principal components, performing PCA transformation on the two-dimensional array according to the plurality of principal components to obtain a dimensionality-reduced feature array and the principal component image.

[0013] According to another aspect of the present invention, there is also provided a device for extracting implicit information of smoked murals based on hyperspectral, including: an acquisition module for acquiring hyperspectral images of murals; a screening module for performing independent component analysis on the hyperspectral images to obtain independent component images and screening out the main independent component images from the independent components; a generation module for performing opening and closing operations on the main independent component images using at least two different scales of structural elements, superimposing the obtained opening operation features, closing operation features and the original band information of the main independent component images to generate an extended morphological profile, and performing principal component analysis on the extended morphological profile to obtain a principal component image; a stretching module for performing equalization stretching on the principal component image according to the histogram of the principal component image to obtain the implicit information.

[0014] According to yet another aspect of the present invention, there is also provided a device for extracting implicit information of smoked murals based on hyperspectral, including a processor and a memory. The memory is used for storing program instructions, and the processor is used for calling the program instructions to execute the method for extracting implicit information of smoked murals based on hyperspectral as described above.

[0015] According to yet another aspect of the present invention, there is also provided a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the method for extracting implicit information of smoked murals based on hyperspectral as described above is implemented.

[0016] The present invention has at least the following beneficial effects: The present invention constructs a new method for extracting implicit information. This method not only integrates spectral information from different principal components, but also further reveals the local structural information of the image (such as brightness changes and edge features) through multi-scale spatial features extracted by opening / closing operations, thereby realizing the effective fusion of spatial-spectral features. The present invention verifies the usability of the model through real temple smoked mural data. The model improves the effect of extracting implicit information of severely smoked murals, makes up for the deficiencies of current extraction methods, and has important academic research value and application prospects.

[0017] Other advantages, objectives and features of the present invention will be partially reflected by the following description, and partially will also be understood by those skilled in the art through the research and practice of the present invention. Description of the Drawings

[0018] Figure 1 It is a flowchart of an embodiment of the present application; Figure 2 It is an original severely smoked image of an embodiment of the present application; Figure 3 It is a main independent component image of an embodiment of the present application; Figure 4 An image containing implicit information according to an embodiment of the present application; Figure 5 The principal component image according to an embodiment of the present application; Figure 6 An image containing implicit information according to another embodiment of the present application. Detailed implementation manners

[0019] The present invention will be further described in detail below with reference to the accompanying drawings, so that those skilled in the art can implement it according to the text of the specification.

[0020] It should be understood that terms such as "having", "including", and "comprising" used in the embodiments of the present application do not exclude the presence or addition of one or more other elements or combinations thereof. All directional indications (such as up, down, left, right, front, back...) in the embodiments of the present application are only used to explain the relative positional relationship and movement conditions between components in a specific posture. If the specific posture changes, the directional indication will also change accordingly. When an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or there may be an intermediate element at the same time. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or indirectly connected to the other element through an intermediate element. The descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one of such features.

[0021] It should be noted that the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that those skilled in the art can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present application.

[0022] As Figure 1 shown, the embodiments of the present application provide a method for extracting implicit information of a smoked mural based on hyperspectral, including: Figure 2 ). S1: Obtain the hyperspectral image of the mural; Exemplarily, a device specifically applicable to hyperspectral imaging is selected, such as an imaging spectrometer with high-precision spectral resolution and spatial resolution; before data acquisition, the device needs to be calibrated to ensure that the wavelength range of each band is accurate, the light intensity detection sensitivity is stable, and the performance of the image sensor is in the best state; the calibration process includes using a standard reference plate with known spectral characteristics to correct the response of the device at different bands, recording and adjusting the device parameters, so that it can accurately obtain the reflection or radiation information of the target mural on multiple consecutive spectral bands.

[0023] Exemplarily, considering that the mural may have complex textures, different light reflection characteristics, and occlusions, etc., hyperspectral image acquisition of the mural can be carried out from multiple directions and angles. For example, around the center of the mural, at regular intervals in the horizontal direction and at different height positions in the vertical direction, shooting is carried out respectively to obtain multiple sets of hyperspectral image data. This can comprehensively cover all areas of the mural and collect as complete spectral information as possible, providing a richer data basis for subsequent processing and accurate extraction of implicit information.

[0024] S2: Perform independent component analysis on the hyperspectral image to obtain independent component images, and screen out the main independent component images from the independent components, see Figure 3 ; In this step, independent component analysis is based on the assumption that the data is linearly mixed by a group of independent signal sources and can be decomposed through the statistical independence of the sources; in the hyperspectral image, information such as the spectral characteristics of different substances and the reflection characteristics of different regions of the mural can be regarded as these independent signal sources, and they are mixed together to form the complex hyperspectral image data we collected; select a suitable ICA algorithm, such as the FastICA algorithm, which has the advantages of fast convergence speed and relatively high computational efficiency, and can decompose the hyperspectral image to obtain independent component images. The process of generating independent component images: Input the collected and sorted hyperspectral image data into the selected ICA algorithm, and the algorithm will decompose the hyperspectral image according to the statistical characteristics of the image data, such as the correlation between bands and the covariance structure of the data. In this process, through complex mathematical operations, various information mixed together in the original hyperspectral image is separated according to its independence, and each independent component image represents a relatively independent information source. For example, perhaps a certain independent component image mainly reflects the spectral characteristics of the pigments in the mural, and another independent component image highlights the spectral changes corresponding to the soot marks, etc.

[0025] To select the main independent component images from a multitude of independent component images, reasonable selection criteria need to be set. Selection can be based on the variance contribution rate of the images. The higher the variance contribution rate, the greater the proportion of the information contained in the independent component image in the total amount of information in the entire hyperspectral image, and the more it can reflect the main features of the image. Specifically, calculate the variance contribution rate of each independent component image, set a threshold (such as the variance contribution rate being greater than 20%), and select the independent component images that meet the threshold conditions as the main independent component images. Additionally, professional domain knowledge can also be combined. For example, for smoked murals, focus on the independent component images that may be related to the spectral characteristics of the smoked substances and the original patterns of the murals, and make judgments and selections from features such as the shape of the spectral curve and the peaks at specific bands to ensure that the selected main independent component images are of great value for subsequent extraction of implicit information.

[0026] S3: Perform opening and closing operations on the main independent component image using at least two different scales of structural elements, superimpose the obtained opening operation features and closing operation features on the original band information of the main independent component image to generate an extended morphological profile, and perform principal component analysis on the extended morphological profile to obtain a principal component image; In this step, select a suitable geometric shape as the structural element. Common ones include circles, rectangles, ellipses, etc. For smoked mural images, considering that their patterns, smoked traces, etc. often have irregular shapes and obvious isotropic characteristics, a circular structural element is usually a good choice, but other shapes can also be selected for supplementary analysis according to the texture and pattern characteristics of the specific mural. For different scale requirements, set at least two structural elements with different radii, such as radii of 1, 2, 3, etc. (the specific values can be adjusted according to factors such as image resolution and the scale of spatial features expected to be extracted), and achieve multi-scale analysis by changing the radius size of the structural element. Different scales can capture image spatial structure features from fine to relatively macroscopic.

[0027] The opening operation first performs an erosion operation and then a dilation operation. For each main independent component image, the opening operation is performed sequentially with structuring elements of different selected scales; during the erosion operation, the structuring element slides on the image. If the pixels within the area covered by the structuring element conform to the erosion rule (for example, for a binary image, if the central pixel within the area covered by the structuring element is not exactly the same as the pixel at the corresponding position of the structuring element, the central pixel is set to the background pixel), the boundaries of the target objects in the image (such as the pattern part in the mural, the patches formed by soot, etc.) will shrink inward, removing some isolated small bright noises or small protruding parts; subsequently, the dilation operation is performed. The dilation operation will cause the target objects that have shrunk after erosion to expand outward appropriately again, but the previously removed isolated small parts will not reappear. Through such a process, the target structure features that are brighter than the surrounding area are extracted, and the opening operation features at different scales are obtained.

[0028] The order of the closing operation is opposite to that of the opening operation, which is dilation first and then erosion; first, the dilation operation is performed. During the process of the structuring element sliding on the image, the dark areas in the image (such as the gaps in the mural pattern, the holes in the soot marks, etc.) expand outward, filling some spatial fissures and connecting the originally discontinuous dark areas; then, the erosion operation is performed to make the expanded area shrink back to a reasonable size appropriately, smoothing the image edges, thereby extracting the structural features that are darker than the surrounding area, and similarly obtaining the closing operation features at different scales.

[0029] The opening operation features, closing operation features obtained through the above-mentioned opening and closing operations at different scales are superimposed and integrated with the original band information of the corresponding main independent component images. This superimposing process combines the spatial structure features mined at different scales with the original spectral band information, enabling the generated Extended Morphological Profile (EMP) to contain both the original spectral feature information of the image and multi-scale spatial morphological features, and being able to more comprehensively depict various features of the smoked mural in the hyperspectral image, providing a rich feature basis for subsequent further analysis. For example, the original band information can reflect the spectral characteristics of the mural pigments, while the opening and closing operation features can reflect spatial structure information such as the spatial distribution of the patterns, the degree and shape of the smoked coverage, and they together constitute the EMP. For example, for the main independent component images IC1 and IC2 obtained from S2, for IC1, multi-scale opening and closing operations are defined, and the obtained features may include OpeningR=1, OpeningR=2 and ClosingR=1, ClosingR=2. Similarly, the profile of IC2 will also generate corresponding structural features. Subsequently, by superimposing the multi-scale profiles of each principal component, a complete Extended Morphological Profile (EMP) is formed. At this time, the EMP contains multi-scale features from each principal component. If IC1 and IC2 each contain four features, then the EMP will contain a total of eight features. Therefore, for each band, the final number of EMP features is 1+2+2+1+2+2 = 10, that is, it contains the original band data and morphological features at four different scales.

[0030] Perform principal component analysis on the generated Extended Morphological Profile. PCA will recombine and reduce the dimensions of the features in the EMP according to the variance of the data; by calculating the covariance matrix of the EMP data and performing mathematical operations such as eigenvalue decomposition or singular value decomposition, find the directions with larger variances and most representative of the data changes as the principal components, and select an appropriate number of principal components (such as determining the number of principal components according to the cumulative variance contribution rate reaching a certain proportion, such as more than 80%) to extract the most representative features among them, reducing the feature dimensions while maximizing the retention of information; after the PCA transformation, the high-dimensional Extended Morphological Profile data is converted into low-dimensional principal component images, and each principal component image reflects the key features of the smoked mural from different angles. For example, some principal component images may prominently display the texture features of the mural patterns, while some may focus more on reflecting the spatial relationship between the smoked traces and the patterns.

[0031] S4: According to the histogram of the principal component image, perform equalization stretching on the principal component image to obtain the implicit information, see Figure 4 ; In this step, first, the histogram of the principal component image is analyzed. The histogram shows the distribution quantity of pixels at each gray level in the principal component image. By counting the number of pixels corresponding to different gray values, the gray distribution characteristics of the principal component image are understood. For example, check whether there is a situation where the gray values are concentrated in a relatively narrow interval, resulting in a low image contrast and difficult-to-distinguish details, or whether the number of pixels in certain gray value ranges is extremely small, indicating that the corresponding feature information is not obvious in the image. These histogram information provide a basis for the subsequent equalization stretching operation. Based on the principle of histogram equalization, perform equalization stretching on the principal component image. This operation aims to redistribute the gray values of the image pixels, broaden the gray range of the image, enhance the image contrast, and make the details in the image more clearly visible. The specific implementation is to first perform an integral operation on the histogram to generate a cumulative histogram, which records the cumulative number of pixels from the darkest gray value to a specific gray value. Then, according to the cumulative histogram, construct a gray mapping function, which establishes the correspondence between the original gray value in the input image and the new gray value. Through this function, map the original gray value to the new gray value to achieve the re-adjustment of the image gray value. For the principal component image after equalization stretching, the hidden information that was originally covered by smoke and difficult to detect due to low contrast or narrow gray range is highlighted. These hidden information may include the original pattern details blurred by smoke in the mural, the fading of pigments, the hierarchical structure of smoke penetration, etc. In this embodiment, a new method for extracting hidden information is constructed. This method not only integrates the spectral information from different principal components but also further reveals the local structure information of the image (such as brightness changes and edge features) through the multi-scale spatial features extracted by opening / closing operations, thereby realizing the effective fusion of spatial-spectral features, improving the effect of extracting hidden information from heavily smoked murals, making up for the deficiencies of the current extraction methods, and having important academic research value and application prospects.

[0032] In another embodiment, in the S1, it further includes using the dark current data to perform reflectance correction on the hyperspectral image. Specifically, the purpose of performing reflectance correction on the hyperspectral image is to eliminate interference and obtain data of the true reflectance. In addition, in some cases, noise will also be introduced during the acquisition of the hyperspectral image. Therefore, the mural image must undergo a certain amount of noise removal to obtain a better final output effect. The correction formula is: In the formula, R is the reflectance, R raw is the collected hyperspectral data, R dark is the dark current data, R whiteIt is the data of the standard reflector, and the reflectivity of the standard reflector is 99%.

[0033] In another embodiment, in the step S1, it further includes cropping the bands with relatively large noise in the front and back sections of the spectral range in the hyperspectral data; in the front section of the spectral range, due to factors such as unstable sensitivity of the detector in the starting band and inaccurate initial calibration of light energy, there may be more noise in the collected data; while in the back section of the spectral range, with the propagation loss of light and some limitations of the detector's response to long-wavelength light, it is also easy to have a situation where the noise is relatively large; for example, if the standard deviation of the data in the first 50 bands and the last 50 bands is significantly higher than that of the middle bands, then it can be determined that these bands are used as the cropping objects. After cropping the bands with relatively large noise, the quality of the hyperspectral data has been significantly improved; after cropping, the data volume has been streamlined to a certain extent, reducing the unnecessary data computation amount in the subsequent processing process, improving the overall data processing efficiency, and more accurately reflecting the true spectral characteristics and spatial structure information of the mural; this is very beneficial for subsequent operations such as independent component analysis and generating extended morphological profiles, and can avoid the deviation of the analysis results or interference with the accurate extraction of hidden information due to the existence of noise bands.

[0034] In another embodiment, in the step S1, it further includes performing minimum noise separation transformation on the hyperspectral image; specifically, analyzing the statistical characteristics of the hyperspectral image data, calculating its covariance matrix and noise covariance matrix, performing eigenvalue decomposition on the calculated noise covariance matrix to obtain the corresponding eigenvalues and eigenvectors, arranging these eigenvalues in descending order, and the eigenvalues reflect the variance contribution size of the components represented by the corresponding eigenvectors in the image, that is, their importance degree; according to the sorted eigenvalues and eigenvectors, constructing a minimum noise separation transformation matrix, and multiplying the original hyperspectral image data by this transformation matrix to achieve the minimum noise separation transformation of the image; so that in the new components after transformation, the components ranked in the front mainly contain effective signal information and relatively less noise, while the components ranked later have a larger proportion of noise, thus achieving the effect of separating noise and highlighting effective information.

[0035] In another embodiment, in the step S3, the main independent component image is segmented into multiple regions, the smoking level of each region is calculated, the resolution of each region is changed according to the smoking level, and then the main independent component image with the changed resolution is subjected to opening and closing operations using at least two different scales of structural elements, the obtained opening operation features and closing operation features are superimposed on the original band information of the main independent component image to generate an extended morphological profile, and the extended morphological profile is subjected to principal component analysis to obtain a principal component image, see Figure 5 ; according to Figure 5Histogram of, perform equalization stretching on the principal component image to obtain the implicit information, see Figure 6 ; Compared with the previous embodiment, in this embodiment, the main independent component image is segmented into multiple regions, the smoking level of each region is calculated, and the resolution of each region is changed according to the smoking level. Although it is more complex, the obtained implicit information is significantly richer (compare Figure 4 and Figure 6 ); Exemplarily, analyze the gray histogram or spectral eigenvalue distribution of the main independent component image to find a suitable threshold to distinguish different regions; for example, for a smoked mural, there may be obvious gray differences between the smoked area and the relatively unsmoked area in certain spectral bands. By statistically calculating statistics such as the gray mean and variance of different pixels in these key bands, one or more thresholds are determined to divide the image pixels into different categories, thereby forming different regions; Exemplarily, collect a large number of labeled smoked mural image samples (label different region categories, such as heavily smoked area, lightly smoked area, unsmoked area, etc.), and a U-Net deep learning model can be trained using these samples; input the main independent component image into the trained model, and the model can automatically output the segmentation result of the image, output the segmentation masks of different regions, and accurately divide each region; Exemplarily, use the average gradient G to objectively describe the clarity of the image. The larger the value of G, the more details the image has and the lower the smoking level is proved by the clarity. On the contrary, the more blurred the image is, the higher the smoking level is; that is: In the formula, m and n are the width and height of the image respectively; and are the gradients of the image in the i and j directions respectively; Using the above average gradient G, combined with experimental analysis or referring to existing relevant research results, formulate specific criteria for dividing the smoking level; for example, the smoking level can be divided into three levels: light, medium, and heavy; Exemplarily, a mapping relationship between the smoking level and the resolution is established: For areas with a heavy smoking level, since the information they contain may become blurred and complex due to smoking, a higher resolution is required to restore details as much as possible. A higher-magnification resolution improvement method can be adopted. For example, the resolution can be increased to 2-3 times the original resolution through resampling techniques. Interpolation algorithms such as bilinear interpolation can be used for resampling to ensure the quality and detail distinguishability of the image after the resolution is increased; for areas with a medium smoking level, the resolution is appropriately increased, for example, to about 1.5 times the original resolution. For areas with a light smoking level, since their features are relatively clear, to reduce the data volume and computational complexity, the resolution can be reduced to about 0.5-0.8 times the original resolution; Exemplarily, the extracted opening operation features, closing operation features, and the original band information are superimposed and integrated according to certain rules to form an extended morphological profile. For example, different features can be represented in the form of vectors, with each feature corresponding to a dimension in the vector, and then these vectors are concatenated to construct an extended morphological profile containing rich image feature information, which not only covers the changes in the image after morphological processing but also retains the original spectral information; In this embodiment, for areas in the image with relatively simple, clear, and little-changing features, a lower resolution is used for processing. This can reduce the data volume contained in these areas and avoid excessive calculation of these relatively simple information in the subsequent processing, thereby saving storage resources and computational time and improving the overall data processing efficiency. When high resolution is assigned to key areas that require more refined analysis (such as parts with severe smoking, where there may be more hidden and blurred information), although the data volume and processing complexity of these areas increase relatively, since they are only part of the image, the overall computational burden will not be too heavy, and at the same time, it can ensure that the detailed analysis of key areas can be carried out quickly and effectively. Moreover, if a high resolution is uniformly used for the entire image, when processing some simple areas, unnecessary noise may be introduced due to over-refinement, or the originally clear features may become difficult to distinguish in the complex high-resolution data. On the contrary, if a low resolution is uniformly used, important details in key areas will be lost. By assigning different resolutions to different areas in this embodiment, it is possible to maintain the clarity of information in each area without losing key details, avoid information confusion caused by inappropriate resolution selection, and make the finally extracted implicit information more accurate and complete; Comparison Figure 4 and Figure 6 It can also be seen that Figure 4 When using a unified resolution, the noise is relatively large and the key information is not prominent. Figure 6 Then the pattern is more obvious.

[0036] In another embodiment, in the step S3, the structural element is circular, and the opening operation and the closing operation are performed on the main independent component image by using the structural elements with different radii. In this step, for the processing of the smoked mural image, the circular structural element has unique advantages. The smoked traces and mural patterns often present irregular shapes and relatively uniform changes in all directions, having certain isotropic characteristics. The circular structural element can better adapt to this property. When performing morphological operations, its processing effect on the target objects in the image (such as mural patterns, smoked patches, etc.) is relatively consistent in all directions, and it will not overemphasize or suppress the features in certain specific directions due to the directionality of the structural element shape. Compared with structural elements with obvious directionality such as rectangles, the circular structural element can detect and extract the spatial structure information of the image more comprehensively and evenly. Whether it is the circular or elliptical pattern outline, or the isotropic smoked area with an irregular shape, the circular structural element can interact with it in a more natural and accurate way, thus laying a good foundation for accurately extracting the hidden information subsequently.

[0037] The multi-scale morphological operation is realized by setting circular structural elements with different radii. The circular structural element with a smaller radius can capture the finer and more minute spatial structure details in the image. For example, there may be some subtle texture changes, the distribution of tiny pigment particles, or the fine traces generated by early smoked erosion in the mural. When performing the opening operation and the closing operation, the circular structural element with a small radius can accurately extract and enhance these subtle features. As the radius gradually increases, the area covered by the circular structural element becomes wider, and at this time, it can reflect relatively more macroscopic spatial structure features, such as the distribution law of the larger area mural patterns, the overall shape of the larger smoked areas, and the spatial relationship between them. When performing the opening operation on the main independent component image, it is processed sequentially with circular structuring elements of different radii. For a circular structuring element with a radius of, when it slides on the image, the opening operation first performs the erosion process; during the erosion stage, if the central pixel within the area covered by the circular structuring element is not exactly the same as the pixel at the corresponding position of the structuring element (for example, in a binary image, according to the set pixel matching rule), the central pixel is regarded as a background pixel, thus causing the boundary of the target object in the image to contract inward; this process will remove some isolated small bright noise points or the small protruding parts on the edge of the target object, making the shape of the target object more regular and smooth; then the dilation operation is performed, and the dilation will cause the target object that has shrunk after erosion to expand outward to a certain extent and recover, but the isolated small parts that were eroded before will not reappear; for example, for some tiny bright spot noises in murals caused by paint peeling or slight soot, the opening operation with a small-radius circular structuring element can effectively remove them while retaining the integrity of the main pattern; for a larger pattern area, the opening operation with a large-radius circular structuring element can highlight its overall contour, making its distinction from the surrounding background more obvious, obtaining the opening operation features at different radii, and these features reflect the spatial structure characteristics of the target object in the image at different scales; The order of the closing operation is opposite to that of the opening operation, first dilating and then eroding; when the circular structuring element performs the dilation operation, during the sliding process of the circular structuring element on the image, it will cause the dark areas in the image (such as the gaps in the mural pattern, the holes in the soot marks, etc.) to expand outward, filling some small spatial fissures and connecting the originally discontinuous dark areas. Then the erosion operation is performed to appropriately contract the expanded area back to a reasonable size, thereby smoothing the image edge and extracting the structural features that are darker than the surrounding areas; for example, when dealing with some irregular holes caused by soot or the small dark areas inside the pattern in a soot-covered mural, the closing operation with a small-radius circular structuring element can moderately fill and regularize these dark areas, making their connection relationship with the surrounding areas clearer in terms of spatial structure; while the closing operation with a large-radius circular structuring element can perform overall morphological adjustment and feature extraction on a larger range of dark areas (such as the dark color blocks covered by a large area of soot), obtaining the closing operation features at different radii, and these features complement the opening operation features and jointly constitute the multi-scale spatial structure information, which helps to more comprehensively depict the features of the soot-covered mural image; The features obtained by performing opening and closing operations on circular structural elements with different radii are superimposed on the original band information of the main independent component image to generate an extended morphological profile; the features extracted by these multi-scale circular structural element operations inject rich spatial structure information into the extended morphological profile; the original band information mainly reflects the spectral characteristics of the mural, while the features obtained through multi-scale operations of circular structural elements deeply analyze the image from the spatial dimension, such as from microscopic texture details to macroscopic pattern distributions and the spatial form of smoke coverage, etc.; their combination enables the extended morphological profile to comprehensively describe the features of the smoked mural in multiple dimensions (spectral and spatial), providing a more powerful feature basis for subsequent principal component analysis and ultimately extracting implicit information, and improving the accuracy and integrity of the entire method for extracting implicit information from the smoked mural.

[0038] In another embodiment, S4 includes: arranging the feature matrix of the extended morphological profile into a two-dimensional array, determining multiple principal components, and performing PCA transformation on the two-dimensional array according to the multiple principal components to obtain a feature array with reduced dimensions and the principal component image; In this step, the principal component analysis (PCA) algorithm usually requires the input data to be in the form of a two-dimensional array, which facilitates the algorithm to accurately mine the principal component information in the data based on matrix operations and perform subsequent dimensionality reduction operations; therefore, it is necessary to rearrange the feature matrix of the extended morphological profile to convert it into a two-dimensional array structure that meets the input requirements of the PCA algorithm.

[0039] Each principal component corresponds to a certain variance. The larger the variance, the more significant the change in the data direction represented by the principal component, and the more information that is explanatory for the overall data. When determining multiple principal components, they are usually selected based on the variance contribution rate. The variance contribution rate refers to the proportion of the variance of a certain principal component in the total variance of all principal components, which intuitively reflects the contribution degree of the principal component to the entire data information; a suitable threshold is set according to actual requirements and data characteristics, such as setting the variance contribution rate to be greater than 80% (the specific value can be flexibly adjusted according to factors such as the required degree of information retention and the complexity of the data itself), and the principal components that meet the threshold condition are selected as the principal components to be retained; in this way, among the numerous possible principal components, those that play a key role in data changes and can best represent the main features in the extended morphological profile can be focused on, avoiding retaining too many relatively unimportant principal components, thereby achieving effective dimensionality reduction of the data and extraction of key information.

[0040] PCA transformation is essentially a linear transformation. It constructs a transformation matrix based on the eigenvectors corresponding to multiple principal components determined previously, and then projects the extended morphological profile data in the form of a two-dimensional array onto a new coordinate axis system determined by these principal components. Suppose the dimension of the feature matrix of the extended morphological profile is m×n×p, and k principal components are determined. The corresponding eigenvector matrix is V (V is a p×k matrix, where p is the number of columns of the original two-dimensional array, that is, the number of features of each pixel, and k is the number of principal components). For the extended morphological profile data X in the form of a two-dimensional array (with a size of mn×p), through matrix multiplication operation Y = XV (Y is the data matrix obtained after PCA transformation, and its size becomes mn×k), the original high-dimensional data is projected into a low-dimensional space composed of k principal components. This process is the core operation of PCA transformation. During this transformation process, the dimension of the data is reduced from the original p dimension to k dimension, achieving the purpose of dimensionality reduction. The principal component image is obtained based on the feature array after dimensionality reduction.

[0041] The embodiment of this application also provides a device for extracting implicit information of smoked murals based on hyperspectral, including: an acquisition module for acquiring the hyperspectral image of the mural; a screening module for performing independent component analysis on the hyperspectral image to obtain independent component images, and screening out the main independent component images from the independent components; a generation module for performing opening and closing operations on the main independent component images using at least two different scales of structural elements, superimposing the obtained opening operation features, closing operation features and the original band information of the main independent component images to generate an extended morphological profile, performing principal component analysis on the extended morphological profile to obtain a principal component image; a stretching module for performing equalization stretching on the principal component image according to the histogram of the principal component image to obtain the implicit information.

[0042] This embodiment uses a computer program to construct an acquisition module, a screening module, a generation module, and a stretching module. The generation module not only integrates the spectral information from different principal components, but also extracts multi-scale spatial features through opening / closing operations, further revealing the local structural information of the image (such as brightness changes and edge features), thereby realizing the effective fusion of spatial-spectral features, improving the effect of extracting implicit information of heavily smoked murals, making up for the deficiencies of current extraction methods, and having important academic research value and application prospects.

[0043] Embodiments of the present application further provide a device for extracting implicit information of smoked murals based on hyperspectral, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the program instructions to execute the method for extracting implicit information of smoked murals based on hyperspectral; The device of this embodiment can be a mobile phone, a laptop computer, a tablet computer, a vehicle-mounted terminal, a drone, etc., with a memory and a processor internally provided to execute the method for extracting implicit information of smoked murals based on hyperspectral in the above embodiment.

[0044] Embodiments of the present application further provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method for extracting implicit information of smoked murals based on hyperspectral; In this embodiment, the computer-readable medium can be a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, etc., which can store a computer program to execute the method for extracting implicit information of smoked murals based on hyperspectral in the above embodiment.

[0045] Although the embodiments of the present invention have been disclosed as above, they are not limited to only the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the field, additional modifications can be easily made. Therefore, without departing from the general concept defined by the claims and the equivalent scope, the present invention is not limited to the specific details and the illustrated and described examples here.

Claims

1. A method for extracting implicit information from smoked murals based on hyperspectral, characterized in that: include: S1: Acquire the hyperspectral image of the mural; S2: performing independent component analysis on the hyperspectral image to obtain independent component images, and screening out main independent component images from the independent components; S3: using at least two structural elements of different scales to perform opening and closing operations on the main independent component image, superimposing the obtained opening operation features and closing operation features with the original band information of the main independent component image to generate an extended morphological profile, and performing principal component analysis on the extended morphological profile to obtain a principal component image; S4: performing equalization stretching on the main component image according to the histogram of the main component image to obtain the implicit information.

2. The method for extracting implicit information from smoked murals based on hyperspectral according to claim 1, characterized in that: In the S1, it also includes performing reflectance correction on the hyperspectral image using dark current data.

3. The method for extracting implicit information from smoked murals based on hyperspectral according to claim 1, characterized in that: In the S1, it also includes cutting the bands with larger noise in the front and back parts of the spectral range of the hyperspectral data.

4. The method for extracting implicit information from smoked murals based on hyperspectral according to claim 1, characterized in that: In the S1, it also includes performing a minimum noise separation transformation on the hyperspectral image.

5. The method for extracting implicit information from smoked murals based on hyperspectral according to claim 1, characterized in that: In S3, the main independent component image is divided into multiple regions, the smoke level of each region is calculated, the resolution of each region is changed according to the smoke level, and then at least two structural elements of different scales are used to perform opening and closing operations on the main independent component image with changed resolution, and the obtained opening operation features and closing operation features are superimposed with the original band information of the main independent component image to generate an extended morphological profile, and the extended morphological profile is subjected to principal component analysis to obtain a principal component image.

6. The method for extracting implicit information from smoked murals based on hyperspectral according to claim 5, characterized in that: In S3, the structural element is a circle, and the structural elements with different radii are used to perform opening and closing operations on the main independent component images.

7. The method for extracting implicit information from smoked murals based on hyperspectral according to claim 5, characterized in that: The S4 includes: The feature matrix of the extended morphological section is arranged into a two-dimensional array, a plurality of principal components are determined, and the two-dimensional array is subjected to PCA transformation according to the plurality of principal components to obtain a feature array and the principal component image after dimensionality reduction.

8. A device for extracting hidden information from smoked murals based on hyperspectral, characterized in that: include: An acquisition module, used to acquire a hyperspectral image of the mural; A screening module is used to perform independent component analysis on the hyperspectral image to obtain independent component images, and screen out main independent component images from the independent components; A generating module, configured to perform an opening operation and a closing operation on the main independent component image using at least two structural elements of different scales, superimpose the obtained opening operation features and closing operation features with the original band information of the main independent component image to generate an extended morphological profile, and perform principal component analysis on the extended morphological profile to obtain a principal component image; The stretching module is used to perform equalization stretching on the main component image according to the histogram of the main component image to obtain the implicit information.

9. The device for extracting hidden information from smoked murals based on hyperspectrum as claimed in claim 7, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions to execute the method for extracting implicit information from smoked murals based on hyperspectrum as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for extracting implicit information from smoked murals based on hyperspectrum according to any one of claims 1 to 7 is implemented.

Images