Ultrahigh-resolution tangka mural image retrieval method and system, computer and storage medium

Abstract

The application discloses a kind of super high resolution Thangka mural image retrieval method, system and computer and storage medium, applied to image retrieval field, for computer cannot directly process super high resolution image, processing efficiency is low, and feature matching is prone to error Problem;The application first uses Gaussian smoothing processing to the image to be searched to reduce the noise of image;Then again the color feature analysis screening is carried out to the image after Gaussian smoothing processing, and the image to be searched with single image color feature and seriously eroded is screened out;Then the image in original library is down-sampled to carry out dimension reduction operation, and the image with resolution reducing layer by layer is obtained, and is stored in the image library of corresponding layer, so that machine can process entire image;Then the size of the image to be searched is used as sliding window size, to carry out multi-scale feature matching with each layer original image library;So as to obtain matching result.

Description

Technical Field

[0001] This invention belongs to the field of image retrieval, and specifically relates to an ultra-high resolution image retrieval technology. Background Technology

[0002] With the rapid development of multimedia and computer technology, various types of information data, mainly images, videos, and audio, are being generated explosively. Faced with such a large amount of data, people naturally hope to quickly obtain the information they want through retrieval. When faced with image information, people hope to find the image information they want by searching for images. In this case, image retrieval technology is needed.

[0003] Current image retrieval applications are mostly focused on searching for specific products or objects in daily life, including content-based and text-based searches. For deeper analysis of image details, deep learning-based image retrieval can be employed. This involves using Convolutional Neural Networks (CNNs) and other advanced technologies to extract and process images before using them for subsequent retrieval.

[0004] In everyday situations, the methods described above can effectively address people's needs. However, when faced with ultra-high resolution large images, such as Thangka murals, people may have images of one or several parts of a mural and want to retrieve the original whole image from these parts. Generally, hardware devices cannot directly process the entire large image, and the cost and efficiency are not ideal. Furthermore, the resolution and image size of the partial images that people have are not uniform or standardized, making it very difficult to perform searches on ultra-high resolution images. Summary of the Invention

[0005] This invention proposes an ultra-high resolution Thangka mural image retrieval method, which overcomes the problems of computers being unable to directly process ultra-high resolution images, low processing efficiency, and easy errors in feature matching. The method of this invention can realize the retrieval of ultra-high resolution images such as Thangka murals.

[0006] One of the technical solutions adopted in this invention is: a method for retrieving ultra-high resolution Thangka mural images, comprising:

[0007] S1. Denoise the image to be retrieved;

[0008] S2. Perform color feature analysis and screening on the denoised image to be retrieved;

[0009] S3. Generate multi-scale images from the original mural images using the Gaussian pyramid, and store the images of each size into the image library of the corresponding layer;

[0010] S4. Perform multi-scale feature matching between the image to be retrieved after step S2 and the images in each layer of the image library obtained in step S3.

[0011] Step S4 above specifically includes the following sub-steps:

[0012] S41. Use the size of the image to be retrieved as the sliding window size;

[0013] S42. Calculate the color features of the original image and the image to be retrieved in the current sliding window, respectively;

[0014] S43. By calculating the color feature distance between the original image of the current sliding window and the image to be retrieved, a matching judgment is made. If a match is found, step S44 is executed; otherwise, the sliding window is moved and the process returns to step S42.

[0015] S44. Perform feature matching using the SIFT algorithm on the original image and the image to be retrieved in the current sliding window.

[0016] Step S42 above specifically includes the following sub-steps:

[0017] S421. Divide the current image into several blocks;

[0018] S422. Use the average pixel color of each block as the representative color, and convert the RGB color space of the current image to the YCbCr color space.

[0019] S423. Perform DCT transformation on the current image converted to YCbCr color space, from luminance, blue chrominance and red chrominance, to obtain luminance DCT matrix DCTY, blue chrominance DCT matrix DCTCb and red chrominance DCT matrix DCTCr.

[0020] S424. Perform a zigzag scan on the luminance DCT matrix DCTY, the blue chromaticity DCT matrix DCTCb, and the red chromaticity DCT matrix DCTCr to complete the grouping, and use the DCT matrix grouping as the color features of the current image.

[0021] Step S44 above specifically includes the following sub-steps:

[0022] S441. Generate multi-scale images of the original image and the image to be retrieved in the current sliding window using a Gaussian pyramid.

[0023] S442. Perform Gaussian difference on the respective multi-scale images and extract the extreme points;

[0024] S443. By calculating the principal curvature, pixels with asymmetrical local curvature are filtered out to obtain their respective key points;

[0025] S444. Calculate the histogram of descriptor gradient directions based on the Gaussian image at the scale of the key points to generate key point descriptors;

[0026] S445. After obtaining the keypoint descriptors at the given scale, create a FLANN matcher and use knnMatch for feature matching. After matching, obtain the returned Dmatch tuples. Use Lowe's algorithm to filter the matching items. Save the filtered feature items to the good_matches array at the corresponding scale. If the number of matching items in good_matches at any scale is greater than the set threshold, the two images are considered similar. Return the window image and the original image number.

[0027] The second technical solution adopted in this invention is: an ultra-high resolution Thangka mural image retrieval system, comprising: a noise reduction module, a color feature filtering module, a first matching module, and a second matching module;

[0028] The denoising module is used to denoise the image to be retrieved;

[0029] The color feature filtering module is used to perform color feature analysis and filtering on the image to be retrieved after noise reduction processing.

[0030] The first matching module takes the image to be retrieved as output by the color feature filtering module and the original mural image as input, and outputs the color feature matching result.

[0031] The second matching module takes the image to be retrieved and the original mural image as inputs for color feature matching, and outputs the image matching results.

[0032] The first matching module includes: a first dimensionality reduction sampling unit, a representative color extraction unit, a DCT discrete cosine transform unit, a zigzag scanning unit, and a CLD feature distance calculation unit;

[0033] The input of the first dimensionality reduction sampling unit is the original mural image, and the output is the Gaussian image pyramid of the original mural image;

[0034] The input to the representative color extraction unit is the image to be retrieved and the image of each layer of the Gaussian image pyramid, and the output is the original image of the current sliding window and the representative color of the image to be retrieved.

[0035] The input of the DCT discrete cosine transform unit is the representative color of the original image and the image to be retrieved in the current sliding window, and the output is the brightness DCT matrix DCTY, the blue chromaticity DCT matrix DCTCb, and the red chromaticity DCT matrix DCTCr of the original image and the image to be retrieved in the current sliding window, respectively.

[0036] The input of the Z-shaped scanning unit is the brightness DCT matrix DCTY, blue chromaticity DCT matrix DCTCb, and red chromaticity DCT matrix DCTCr of the original image and the image to be retrieved in the current sliding window. The output is a group of DCT matrices of the original image and the image to be retrieved in the current sliding window. The group of DCT matrices of the original image and the image to be retrieved in the current sliding window is used as their respective color features.

[0037] The input to the CLD feature distance calculation unit is the color features of the original image of the current sliding window and the image to be retrieved, and the output is the first matching result. The first matching feature result is either that the color features of the original image of the current sliding window and the image to be retrieved match, or that the color features of the original image of the current sliding window and the image to be retrieved do not match.

[0038] The second matching module includes: a second dimensionality reduction sampling unit, a first Gaussian difference unit, a first key point filtering unit, a first key point descriptor generation unit, a third dimensionality reduction sampling unit, a second Gaussian difference unit, a second key point filtering unit, a second key point descriptor generation unit, and a FLANN matcher;

[0039] The second dimensionality reduction sampling unit takes as input the original image of the current sliding window and the original image of the current sliding window whose color features are matched with those of the image to be retrieved, and outputs as the Gaussian image pyramid of the original image of the current sliding window; the first Gaussian difference unit takes as input the Gaussian image pyramid of the original image of the current sliding window, and outputs as the extreme points of each layer of the Gaussian image pyramid of the original image of the current sliding window; the first keypoint filtering unit takes as input the extreme points of each layer of the Gaussian image pyramid of the original image of the current sliding window, and outputs as the keypoints of each layer of the Gaussian image pyramid of the original image of the current sliding window; the first keypoint descriptor generation unit takes as input the Gaussian image at the scale of each layer of the Gaussian image pyramid of the original image of the current sliding window, and outputs as the keypoint descriptors of each layer of the Gaussian image pyramid of the original image of the current sliding window.

[0040] The input of the third dimensionality reduction sampling unit is the original image of the current sliding window and the image to be retrieved whose color features are matched, and the output is the Gaussian image pyramid of the image to be retrieved; the input of the second Gaussian difference unit is the Gaussian image pyramid of the image to be retrieved, and the output is the extreme points of each layer of the Gaussian image pyramid of the image to be retrieved; the input of the second key point filtering unit is the extreme points of each layer of the Gaussian image pyramid of the image to be retrieved, and the output is the key points of each layer of the Gaussian image pyramid of the image to be retrieved; the input of the second key point descriptor generation unit is the Gaussian image at the scale of the key points of each layer of the Gaussian image pyramid of the image to be retrieved, and the output is the key point descriptor of each layer of the Gaussian image pyramid of the image to be retrieved.

[0041] The input to the FLANN matcher is the key point descriptors of each layer of the Gaussian image pyramid of the original image of the current sliding window and the key point descriptors of each layer of the Gaussian image pyramid of the image to be retrieved. The output is the second matching result. The second matching result is either that the original image of the current sliding window is similar to the image to be retrieved or that the original image of the current sliding window is not similar to the image to be retrieved.

[0042] The third technical solution adopted in this invention is: a computer device, including a processor and a memory for storing processor-executable programs, wherein when the processor of the computer executes the program stored in the memory, it realizes the above-mentioned ultra-high resolution Thangka mural image retrieval task.

[0043] The fourth technical solution adopted in this invention is: a storage medium storing a program, which, when executed by a processor, performs the aforementioned ultra-high resolution Thangka mural image retrieval task.

[0044] The beneficial effects of this invention are as follows: First, the image to be retrieved is processed using Gaussian smoothing to reduce image noise; then, the images in the original database are downsampled for dimensionality reduction, resulting in images with progressively lower resolution, which are stored in the corresponding layer's image database so that the machine can process the entire image; next, the size of the image to be retrieved is used as the sliding window size to perform multi-scale feature matching with each layer of the original image database; thus obtaining the matching result; this invention includes the following advantages:

[0045] 1. During the data screening stage, color analysis is used to remove image blocks with single colors and high repetition probability in the image library, and noise reduction processing is performed on the images to improve the efficiency and accuracy of subsequent retrieval work.

[0046] 2. In the original image processing stage, a multi-scale image is generated through Gaussian pyramids, enabling the computer to process the entire image. At the same time, the image to be retrieved is used as a sliding window to perform multi-scale feature matching. This takes into account feature changes at different scales and increases the robustness of matching.

[0047] 3. Regarding the feature matching stage, given that image erosion and contamination features may dominate, this invention incorporates color feature analysis constraints to reduce the false matching rate of erosion and contamination features, thereby improving the overall image retrieval accuracy. Attached Figure Description

[0048] Figure 1 This is a flowchart of the ultra-high resolution Thangka mural image retrieval method provided in Embodiment 1 of the present invention;

[0049] Figure 2 This is a schematic diagram of the multi-scale feature matching process in Embodiment 1 of the present invention;

[0050] Figure 3 The original image (a) and the images to be retrieved (b)-(e);

[0051] Figure 4 A line graph for pixel analysis in the HSV color space;

[0052] Figure 5 This is a schematic diagram of a zigzag scan.

[0053] Figure 6 To retrieve the image returned after a successful match.

[0054] Figure 7 This is a schematic diagram of the ultra-high resolution Thangka mural image retrieval system provided in Embodiment 2 of the present invention. Detailed Implementation

[0055] To facilitate understanding of the technical content of this invention by those skilled in the art, the following description, in conjunction with the accompanying drawings, further illustrates the invention.

[0056] Example 1

[0057] To achieve the retrieval of ultra-high resolution images such as Thangka murals, this invention proposes an ultra-high resolution Thangka mural image retrieval method. The main features are as follows:

[0058] First, the images to be retrieved are preprocessed using Gaussian smoothing to reduce noise. Considering the extremely high resolution of the original Thangka mural images, large areas of the image are of a single color, with indistinct features and potentially containing multiple identical repeating image blocks, the image data to be retrieved is filtered. Through color feature analysis, image data with indistinct features are removed.

[0059] Secondly, a Gaussian image pyramid is used to downsample the images in the original database for dimensionality reduction, resulting in images with progressively lower resolution. This allows the machine to process the entire image and store it in the corresponding image database. Images that have undergone data filtering are then selected, and a multi-scale sliding window algorithm is employed. Since the images to be retrieved are relatively small, their dimensions are directly used as the size of the sliding window, which is then input to perform multi-scale feature matching with images at various scales in the image database.

[0060] In the feature matching stage, the SIFT algorithm is mainly used for feature matching. However, due to the long-term erosion and damage of Thangka murals, some image blocks may suffer from large areas of damage. The SIFT algorithm may focus on extracting features from these damaged parts, which can easily lead to matching errors. The result may be two images with similar damage and cracks but different content. Therefore, color feature analysis and matching are added to the SIFT algorithm, and the Color Layout Descriptor (CLD) is used to improve the feature matching accuracy more accurately and efficiently.

[0061] Through the improvements made to the above methods, the problems of computers being unable to directly process ultra-high resolution images, low processing efficiency, and easy errors in feature matching have been solved, and the retrieval of ultra-high resolution images such as Thangka murals has been completed.

[0062] like Figure 1 and Figure 2 As shown, the ultra-high resolution Thangka mural image retrieval method provided in this embodiment specifically includes the following steps:

[0063] S1. Denoise the image to be retrieved.

[0064] Due to such Figure 3 The image to be retrieved may have been captured by various devices under different environmental conditions. First, Gaussian smoothing is applied to remove details and noise, with the weights of each pixel as follows:

[0065]

[0066] in, This is a constant value, representing the variance in Gaussian formula. In practice, this method is typically calculated using a specified Gaussian kernel size, and is referred to as the fuzzy radius or scale.

[0067] Next, take each pixel as the center, take out a window of the same size as the Gaussian kernel matrix, multiply each pixel in the window by the corresponding Gaussian kernel element, and sum the results to get the smoothed pixel value. Then divide the smoothed pixel value by the sum of all elements of the Gaussian kernel to ensure normalization.

[0068] S2. Perform color feature analysis and screening on the denoised image to be retrieved.

[0069] This step involves color feature analysis and filtering of the image after Gaussian smoothing.

[0070] For color feature analysis, the main focus is on whether the colors are uniform and whether the image has been severely eroded by wind and sand. If the colors are uniform or severely eroded, the image is not representative, as multiple similar images may exist in the original Thangka mural image. First, the image is converted to the HSV (hue, saturation, value) color space. The three channels are separated, and the number of colors with different color values ​​in each channel is counted. The results are then plotted as follows: Figure 4 The line graph shown is illustrated. For the tone channel, let the total number of pixels be M. Find the range Q where the number of color values ​​is relatively large and clustered, and then find the maximum value within this range, denoted as . Starting from the maximum value, find the beginning and end points of the range in both directions. , The color range is calculated. Total number of pixels Calculate the proportion of the total number of pixels in each range to the total number of pixels. Next, calculate the standard deviation of image saturation using the following formula:

[0071]

[0072] This represents the average saturation value at each point in the S channel.

[0073] In the HSV color space, hue is measured by angle, ranging from 0° to 360°. Saturation (S) represents how close a color is to a spectral color, ranging from 0 to 1; it represents the ratio between the purity of the selected color and its maximum purity. From the above formula, if a certain hue range is obtained... If the standard deviation of saturation (std) calculated from saturation is less than 25, then the image is considered to have a single color feature and is therefore excluded from the search. Examples of single-color search images include... Figure 3 As shown in (b).

[0074] For judging eroded images, since they mainly exhibit yellowish or earthy yellow colors due to wind and sand erosion, the analysis primarily focuses on the number of colors in the H hue channel of the HSV. A dataset of severely eroded images is set up and input sequentially. The color range of the H hue channel is detected, and the common range obtained from each image is taken, outputting a range of 0–14°. For the image to be retrieved, the color situation of its H channel is analyzed similarly, and the proportion D of pixels within the 0–14° range can be calculated using the following formula:

[0075]

[0076] If the percentage of pixels in the image within this range If so, the image is determined to be severely eroded and is then filtered out. Severely eroded images include... Figure 3 As shown in (d).

[0077] S3. Generate multi-scale images from the original mural images using the Gaussian pyramid, and store each size image into the corresponding layer's image library.

[0078] For example Figure 3 (a) shows the original Thangka mural image, which undergoes dimensionality reduction sampling, with the resolution decreasing layer by layer. First, a Gaussian kernel convolution operation (Gaussian filtering) is performed on the entire image, followed by downsampling. Specifically, all even-numbered rows and columns in the pixel matrix of that layer are deleted. Assuming the original image... The image obtained after downsampling has M × N pixels. It has M / 2 × N / 2 pixels, only one-quarter the size of the original image. By iterating the above steps over the input original image, an image pyramid with progressively decreasing resolution is obtained. In this method, we set the number of pyramid layers to 5, perform the above operations on all original mural images, and store them in the image library of the corresponding layer. Each original mural image has only one image code, and the codes of the mural images stored in each layer's image library correspond to it.

[0079] S4. Perform multi-scale feature matching between the image to be retrieved after step S2 and the images in each layer of the image library obtained in step S3.

[0080] This step primarily employs the SIFT feature matching algorithm constrained by the Color Layout Descriptor (CLD) for multi-scale feature matching. Specifically, it includes the following sub-steps:

[0081] S41. Use the size of the image to be retrieved as the sliding window size.

[0082] First, several images to be retrieved are input. Since the resolutions of the images to be retrieved vary, feature matching is performed between each image and the original image database at each layer, i.e., multi-scale feature matching. In this embodiment, the resolutions of the images to be retrieved are generally 700*800, 500*900, and 900*600. After processing with the Gaussian pyramid in step S3, the resolution of the original mural image is 8200*6200. Since the size of the image to be retrieved is relatively small, its size can be directly used as the size of the sliding window w. The step size of the sliding window w is set to step_size = 80. The sliding window starts from the upper left corner of the original mural image and performs feature matching according to steps S42-S44.

[0083] S42. Calculate the color features of the original image w and the image t to be retrieved in the current sliding window. This step includes the following sub-steps:

[0084] S421. Divide the current image into several blocks.

[0085] In step S41, the window w of the original image and the image t to be retrieved are captured. Both are first analyzed using the Color Layout Descriptor (CLD). Image segmentation is then performed, dividing the image (in the RGB color space) into 64 regions. The size of each region is M / 8 × N / 8, where M and N represent the width and height of the input image.

[0086] S422. Use the average pixel color of each block as the representative color, and convert the RGB color space of the current image to the YCbCr color space.

[0087] Next, a representative color is selected. In this process, YCbCr (brightness (Y), blue chromaticity (Cb), red chromaticity (Cr)) is used in the CLD color space to calculate the pixel color of the area block, and the average pixel color value of each area block is taken as the representative color.

[0088] S423. Perform DCT transformation on the current image converted to YCbCr color space, from luminance, blue chrominance and red chrominance, to obtain the luminance DCT matrix DCTY, the blue chrominance DCT matrix DCTCb and the red chrominance DCT matrix DCTCr.

[0089] After obtaining the image matrix after color selection, a Discrete Cosine Transform (DCT) is performed on the image. In this stage, luminance (Y), blue chroma (Cb), and red chroma (Cr) are transformed using an 8×8 DCT, resulting in three DCT matrices with 64 coefficients each: the luminance DCT matrix DCTY, the blue chroma DCT matrix DCTCb, and the red chroma DCT matrix DCTCr. The DCT transformation formula is as follows:

[0090]

[0091] ,

[0092]

[0093] in, This represents the color value for each region after image segmentation. Here are the matrix coefficients after DCT transformation, M is the width of the initial image, and N is the height of the initial image. and The two compensation coefficients can make the DCT transformation matrix an orthogonal matrix.

[0094] S424. Perform a zigzag scan on the luminance DCT matrix DCTY, the blue chromaticity DCT matrix DCTCb, and the red chromaticity DCT matrix DCTCr to complete the grouping, and use the DCT matrix grouping as the color features of the current image.

[0095] After DCT transformation, the resulting luminance DCT matrix DCTY, blue chroma DCT matrix DCTCb, and red chroma DCT matrix DCTCr—three matrices containing 64 DCT coefficients—are subjected to a zigzag scan, following the pattern shown below. Figure 5 The scanning is performed in a zigzag pattern as shown. The purpose of this scanning is to group the low-frequency coefficients of the 8×8 matrix.

[0096] S43. By calculating the color feature distance between the original image of the current sliding window and the image to be retrieved, a matching judgment is made. If a match is found, step S44 is executed; otherwise, the sliding window is moved and the process returns to step S42.

[0097] The processed window w and the CLD features of image t are matched by calculating the distance between the two CLD features. Assume the two CLD feature values ​​are:

[0098] {DY, DCb, DCr} and {DY', DCb', DCr'}

[0099] The formula for calculating the distance between CLD features is:

[0100]

[0101] in, , and This represents the i-th coefficient of the Y, Cb, and Cr color components. , and These represent the weighted values ​​of the coefficients in the i-th group.

[0102] Since the low-frequency part contains more information, the weighting value is reduced according to the zigzag scanning order.

[0103] Here, if the distance D between CLD features is set to ≥ 900 (a threshold is set), then the color features of the two images are considered to be mismatched.

[0104] Once the original image w of the sliding window is similar to the CLD features of the image t to be retrieved, feature matching using the SIFT algorithm is then performed.

[0105] S44. Perform feature matching of the original image and the image to be retrieved in the current sliding window using the SIFT algorithm, and output the matching image results.

[0106] This process includes the following sub-steps:

[0107] S441. Generate multi-scale images of the original image and the image to be retrieved in the current sliding window using a Gaussian pyramid.

[0108] This step processes the original image and the image to be retrieved in the current sliding window separately using a Gaussian pyramid (as described above), generating their respective multi-scale images. The scale in the Gaussian kernel function is... .

[0109] S442. Perform Gaussian difference on each of the multi-scale images and extract the extreme points.

[0110] Subtracting adjacent layers from each group of Gaussian pyramids yields the Difference of Gaussian (DoG) pyramid, where the extreme points are the feature points of the image. For each pixel, it is compared with its surrounding neighboring points, and repeating the above operation yields a series of extreme points.

[0111] S443. By calculating the principal curvature, pixels with asymmetrical local curvature are filtered out to obtain their respective key points.

[0112] For these extreme points, low-contrast points are discarded, and pixels with asymmetric local curvature in the DoG (Domain of Gaussian) are removed. A poorly defined Gaussian difference operator exhibits larger principal curvature at the edges and smaller principal curvature in the direction perpendicular to the edges. The principal curvature is calculated using a 2×2 Hessian matrix H, as shown in the following formula:

[0113]

[0114]

[0115] Among them, D xx D yy D xy Let represent the second-order partial derivatives of the Gaussian difference image, where the principal curvature of D is proportional to the eigenvalues ​​of H. For the larger eigenvalues ​​of H, If H is a smaller eigenvalue, then If < 0, discard the pixel.

[0116] S444. Calculate the histogram of descriptor gradient directions based on the Gaussian image at the scale of the keypoint, and generate keypoint descriptors.

[0117] After obtaining the keypoints, the image region required for the descriptor is determined through calculation. The histogram of the descriptor gradient direction is generated by calculating the Gaussian image at the scale of the keypoints. The formula for calculating the radius of the image region is as follows:

[0118]

[0119] in, The scale is represented by d = 4, indicating a division into 4×4 sub-blocks. Histogram analysis is performed on each sub-block in 8 directions to obtain the gradient magnitude in each direction, resulting in a total of 128-dimensional descriptive vectors. A descriptor is created for each keypoint, using a set of vectors to describe it, ensuring it remains unchanged despite variations in lighting, viewing angle, etc.

[0120] S445. After obtaining the keypoint descriptors at the given scale, create a FLANN matcher and use knnMatch for feature matching. After matching, obtain the returned Dmatch tuples. Use Lowe's algorithm to filter the matching items. Save the filtered feature items to the good_matches array at the corresponding scale. If the number of matching items in good_matches at any scale is greater than the set threshold, the two images are considered similar. Return the window image and the original image number.

[0121] In this step, after obtaining the keypoint descriptors at the specified scale, a FLANN matcher is created, and knnMatch is used for feature matching. After matching, the returned Dmatch tuples are obtained. Lowe's algorithm is used to filter the matches, and the filtered features are saved in the good_matches array. A threshold of 300 is set; if the number of matches in good_matches is greater than 300, the two images are considered similar. The window image and the original image number are returned. The final returned window image is shown below. Figure 6 As shown in (a) and (b) in the figure.

[0122] The system returns the original images retrieved from each window, sorted by similarity, and provides the top 5 images with the highest similarity. The similarity here is the number of matches in `good_matches`; the higher the number of matches, the higher the similarity. Essentially, it sorts the returned window images from highest to lowest similarity and provides the top 5 most similar images.

[0123] Example 2

[0124] like Figure 7 As shown, this embodiment provides an ultra-high resolution Thangka mural image retrieval system, including: a noise reduction module, a color feature filtering module, a first matching module, and a second matching module.

[0125] The denoising module is used to denoise the images to be retrieved.

[0126] The color feature filtering module is used to analyze and filter the color features of the image to be retrieved after denoising.

[0127] The first matching module takes the image to be retrieved (output from the color feature filtering module) and the original mural image as input, and outputs the color feature matching result. The first matching module includes: a first dimensionality reduction sampling unit, a representative color extraction unit, a DCT discrete cosine transform unit, a zigzag scanning unit, and a CLD feature distance calculation unit.

[0128] The input of the first dimensionality reduction sampling unit is the original mural image, and the output is the Gaussian image pyramid of the original mural image;

[0129] The input to the color extraction unit is the image to be retrieved and the image of each layer of the Gaussian image pyramid, and the output is the original image of the current sliding window and the representative color of the image to be retrieved.

[0130] The input to the DCT discrete cosine transform unit is the representative color of the original image and the image to be retrieved in the current sliding window. The output is the brightness DCT matrix DCTY, the blue chromaticity DCT matrix DCTCb, and the red chromaticity DCT matrix DCTCr of the original image and the image to be retrieved in the current sliding window, respectively.

[0131] The input to the Z-shaped scanning unit is the brightness DCT matrix DCTY, blue chromaticity DCT matrix DCTCb, and red chromaticity DCT matrix DCTCr of the original image and the image to be retrieved in the current sliding window. The output is a group of DCT matrices of the original image and the image to be retrieved in the current sliding window. The group of DCT matrices of the original image and the image to be retrieved in the current sliding window is used as their respective color features.

[0132] The input to the CLD feature distance calculation unit is the color features of the original image of the current sliding window and the image to be retrieved, and the output is the first matching result. The first matching feature result is either that the color features of the original image of the current sliding window and the image to be retrieved match, or that the color features of the original image of the current sliding window and the image to be retrieved do not match.

[0133] The second matching module takes the image to be retrieved (based on color feature matching) and the original mural image as input, and outputs the image matching result. The second matching module includes: a second dimensionality reduction sampling unit, a first Gaussian difference unit, a first keypoint filtering unit, a first keypoint descriptor generation unit, a third dimensionality reduction sampling unit, a second Gaussian difference unit, a second keypoint filtering unit, a second keypoint descriptor generation unit, and a FLANN matcher.

[0134] The second dimensionality reduction sampling unit takes as input the original image of the current sliding window and the original image of the current sliding window whose color features are matched with those of the image to be retrieved, and outputs a Gaussian image pyramid of the original image of the current sliding window; the first Gaussian difference unit takes as input the Gaussian image pyramid of the original image of the current sliding window, and outputs the extreme points of each layer of the Gaussian image pyramid of the original image of the current sliding window; the first keypoint filtering unit takes as input the extreme points of each layer of the Gaussian image pyramid of the original image of the current sliding window, and outputs the keypoints of each layer of the Gaussian image pyramid of the original image of the current sliding window; the first keypoint descriptor generation unit takes as input the Gaussian image at the scale of the keypoints of each layer of the Gaussian image pyramid of the original image of the current sliding window, and outputs the keypoint descriptors of each layer of the Gaussian image pyramid of the original image of the current sliding window.

[0135] The input to the third dimensionality reduction sampling unit is the original image of the current sliding window and the image to be retrieved whose color features are matched. The output is the Gaussian image pyramid of the image to be retrieved. The input to the second Gaussian difference unit is the Gaussian image pyramid of the image to be retrieved. The output is the extreme points of each layer of the Gaussian image pyramid of the image to be retrieved. The input to the second keypoint filtering unit is the extreme points of each layer of the Gaussian image pyramid of the image to be retrieved. The output is the keypoints of each layer of the Gaussian image pyramid of the image to be retrieved. The input to the second keypoint descriptor generation unit is the Gaussian image at the scale of the keypoints of each layer of the Gaussian image pyramid of the image to be retrieved. The output is the keypoint descriptor of each layer of the Gaussian image pyramid of the image to be retrieved.

[0136] The FLANN matcher takes as input the keypoint descriptors of each layer of the Gaussian pyramid of the original image of the current sliding window and the keypoint descriptors of each layer of the Gaussian pyramid of the image to be retrieved, and outputs the second matching result; the second matching result is either that the original image of the current sliding window is similar to the image to be retrieved or that the original image of the current sliding window is not similar to the image to be retrieved.

[0137] The specific implementation of each module in this embodiment can be found in Embodiment 1 above, and will not be repeated here. It should be noted that the system provided in this embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0138] Example 3

[0139] This embodiment provides a computer device, including a processor and a memory for storing processor-executable programs. When the processor of the computer executes the program stored in the memory, it realizes the ultra-high resolution Thangka mural image retrieval task of Embodiment 1 above.

[0140] Example 4

[0141] This embodiment provides a storage medium storing a program, which, when executed by a processor, performs the ultra-high resolution Thangka mural image retrieval task described in Embodiment 1 above.

[0142] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A method for retrieving ultra-high resolution Thangka mural images, characterized in that, include: S1. Denoise the image to be retrieved; S2. Perform color feature analysis and filtering on the denoised image to be retrieved; S3. Generate multi-scale images from the original mural images using the Gaussian pyramid, and store the images of each size into the image library of the corresponding layer; S4. Perform multi-scale feature matching between the image to be retrieved after step S2 and the images in the image databases obtained in step S3; step S4 specifically includes the following sub-steps: S41. Use the size of the image to be retrieved as the sliding window size; S42. Calculate the color features of the original image and the image to be retrieved in the current sliding window, respectively; step S42 specifically includes the following sub-steps: S421. Divide the current image into several blocks; S422. Use the average pixel color of each block as the representative color, and convert the RGB color space of the current image to the YCbCr color space. S423. Perform DCT transformation on the current image converted to YCbCr color space, from luminance, blue chrominance and red chrominance, to obtain luminance DCT matrix DCTY, blue chrominance DCT matrix DCTCb and red chrominance DCT matrix DCTCr. S424. Perform a zigzag scan on the luminance DCT matrix DCTY, the blue chromaticity DCT matrix DCTCb, and the red chromaticity DCT matrix DCTCr to complete the grouping, and use the DCT matrix grouping as the color features of the current image. S43. By calculating the color feature distance between the original image of the current sliding window and the image to be retrieved, a matching judgment is made. If a match is found, step S44 is executed; otherwise, the sliding window is moved and the process returns to step S42. S44. Perform feature matching of the original image and the image to be retrieved using the SIFT algorithm in the current sliding window, and output the matched image results; Step S44 specifically includes the following sub-steps: S441. Generate multi-scale images of the original image and the image to be retrieved in the current sliding window using a Gaussian pyramid. S442. Perform Gaussian difference on the respective multi-scale images and extract the extreme points; S443. By calculating the principal curvature, pixels with asymmetrical local curvature are filtered out to obtain their respective key points; S444. Calculate the histogram of descriptor gradient directions based on the Gaussian image at the scale of the key points to generate key point descriptors; S445. After obtaining the keypoint descriptors at the given scale, create a FLANN matcher and use knnMatch for feature matching. After matching, obtain the returned Dmatch tuples. Use Lowe's algorithm to filter the matching items. Save the filtered feature items to the good_matches array at the corresponding scale. If the number of matching items in good_matches at any scale is greater than the set threshold, the two images are considered similar. Return the sliding window image and the original image number.

2. The ultra-high resolution Thangka mural image retrieval method according to claim 1, characterized in that, Step S2 involves color feature analysis and screening, including filtering out images with single color features and images that are severely eroded. The process of filtering out images to be retrieved that have a single color feature is as follows: The image to be retrieved is converted to the HSV color space, and the hue, saturation, and lightness channels are separated. The number of colors with different color values ​​in each channel is counted, and a line graph is plotted. For the hue channel, let the total number of pixels be M, find the range Q where the number of color values ​​is large and clustered, and find the maximum value point within this range, denoted as . Starting from the maximum point, find the starting point of the range in both directions. End point The color range is calculated. Total number of pixels Calculate the proportion of the total number of pixels in each range to the total number of pixels. Next, calculate the standard deviation of image saturation using the following formula: ； in, This represents the average saturation value at each point in the saturation channel. If a certain color range is obtained If the standard deviation of saturation (std) calculated from saturation is < 25, then the image to be retrieved is judged to have a single color feature, and the image to be retrieved is filtered out. The process of filtering out severely eroded images is as follows: Analyze the number of colors in the hue channels of the HSV color space of the image to be retrieved; based on the known image dataset with severe erosion, detect the color range of the hue channels of each known image with severe erosion, and obtain the common range for judging severe erosion. Calculate the percentage of the color in the tone channel of the image to be retrieved that falls within the common range; if the percentage is greater than... If the image to be retrieved is found to be severely eroded, it will be removed from the search results.

3. A high-resolution Thangka mural image retrieval system, characterized in that, The method for retrieving ultra-high resolution Thangka mural images according to claim 1 or 2 includes: a denoising module, a color feature filtering module, a first matching module, and a second matching module; The denoising module is used to denoise the image to be retrieved; The color feature filtering module is used to perform color feature analysis and filtering on the image to be retrieved after noise reduction processing. The first matching module takes the image to be retrieved as output by the color feature filtering module and the original mural image as input, and outputs the color feature matching result. The second matching module takes the image to be retrieved and the original mural image as inputs for color feature matching, and outputs the image matching results.

4. The ultra-high resolution Thangka mural image retrieval system according to claim 3, characterized in that, The first matching module includes: a first dimensionality reduction sampling unit, a representative color extraction unit, a DCT discrete cosine transform unit, a zigzag scanning unit, and a CLD feature distance calculation unit; The input of the first dimensionality reduction sampling unit is the original mural image, and the output is the Gaussian image pyramid of the original mural image; The input to the representative color extraction unit is the image to be retrieved and the image of each layer of the Gaussian image pyramid, and the output is the original image of the current sliding window and the representative color of the image to be retrieved. The input of the DCT discrete cosine transform unit is the representative color of the original image and the image to be retrieved in the current sliding window, and the output is the brightness DCT matrix DCTY, the blue chromaticity DCT matrix DCTCb, and the red chromaticity DCT matrix DCTCr of the original image and the image to be retrieved in the current sliding window, respectively. The input of the Z-shaped scanning unit is the brightness DCT matrix DCTY, blue chromaticity DCT matrix DCTCb, and red chromaticity DCT matrix DCTCr of the original image and the image to be retrieved in the current sliding window. The output is a group of DCT matrices of the original image and the image to be retrieved in the current sliding window. The group of DCT matrices of the original image and the image to be retrieved in the current sliding window is used as their respective color features. The input to the CLD feature distance calculation unit is the color features of the original image of the current sliding window and the image to be retrieved, and the output is the first matching feature result. The first matching feature result is either that the color features of the original image of the current sliding window and the image to be retrieved match, or that the color features of the original image of the current sliding window and the image to be retrieved do not match.

5. The ultra-high resolution Thangka mural image retrieval system according to claim 3, characterized in that, The second matching module includes: a second dimensionality reduction sampling unit, a first Gaussian difference unit, a first key point filtering unit, a first key point descriptor generation unit, a third dimensionality reduction sampling unit, a second Gaussian difference unit, a second key point filtering unit, a second key point descriptor generation unit, and a FLANN matcher; The second dimensionality reduction sampling unit takes as input the original image of the current sliding window and the original image of the current sliding window whose color features are matched with those of the image to be retrieved, and outputs as the Gaussian image pyramid of the original image of the current sliding window; the first Gaussian difference unit takes as input the Gaussian image pyramid of the original image of the current sliding window, and outputs as the extreme points of each layer of the Gaussian image pyramid of the original image of the current sliding window; the first keypoint filtering unit takes as input the extreme points of each layer of the Gaussian image pyramid of the original image of the current sliding window, and outputs as the keypoints of each layer of the Gaussian image pyramid of the original image of the current sliding window; the first keypoint descriptor generation unit takes as input the Gaussian image at the scale of each layer of the Gaussian image pyramid of the original image of the current sliding window, and outputs as the keypoint descriptors of each layer of the Gaussian image pyramid of the original image of the current sliding window. The input of the third dimensionality reduction sampling unit is the original image of the current sliding window and the image to be retrieved whose color features are matched, and the output is the Gaussian image pyramid of the image to be retrieved; the input of the second Gaussian difference unit is the Gaussian image pyramid of the image to be retrieved, and the output is the extreme points of each layer of the Gaussian image pyramid of the image to be retrieved; the input of the second key point filtering unit is the extreme points of each layer of the Gaussian image pyramid of the image to be retrieved, and the output is the key points of each layer of the Gaussian image pyramid of the image to be retrieved; the input of the second key point descriptor generation unit is the Gaussian image at the scale of the key points of each layer of the Gaussian image pyramid of the image to be retrieved, and the output is the key point descriptor of each layer of the Gaussian image pyramid of the image to be retrieved. The input to the FLANN matcher is the key point descriptors of each layer of the Gaussian image pyramid of the original image of the current sliding window and the key point descriptors of each layer of the Gaussian image pyramid of the image to be retrieved. The output is the second matching result. The second matching result is either that the original image of the current sliding window is similar to the image to be retrieved or that the original image of the current sliding window is not similar to the image to be retrieved.

6. A computer device comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor of the computer executes the program stored in the memory, it implements the ultra-high resolution Thangka mural image retrieval method as described in claim 1 or 2.

7. A storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the ultra-high resolution Thangka mural image retrieval method as described in claim 1 or 2.

Images