Artware image similarity retrieval system and method based on deep learning

Abstract

The application discloses a handicraft image similarity retrieval system and method based on deep learning, and belongs to the field of handicraft image data processing.The specific implementation mode of the application is as follows: performing handicraft image anomaly inspection, performing threshold segmentation processing based on the detection result, deciding whether to take handicraft image progressive repair based on the processing result, implementing handicraft image progressive repair on the handicraft image, performing handicraft image feature dimension re-labeling after the implementation of the handicraft image progressive repair is completed, directly performing handicraft image feature dimension re-labeling when the handicraft image is not subjected to handicraft image progressive repair, performing handicraft image similarity distance calculation based on the result of the handicraft image feature dimension re-labeling after the handicraft image feature dimension re-labeling is completed, and performing handicraft image similarity retrieval based on the calculation result, so that the problem of low accuracy of handicraft image similarity retrieval in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of handicraft image data processing technology, and in particular to a handicraft image similarity retrieval system and method based on deep learning. Background Technology

[0002] Against the backdrop of global digital transformation and the accelerated development of digital culture, the image data of various ceramics, jade, and woven handicrafts generated in fields such as digital collections, online trading platforms for handicrafts, and the development of cultural and creative products are experiencing explosive growth. This massive and disorganized collection of handicraft images makes standardized management and accurate traceability difficult, failing to meet practical business needs such as online screening of handicrafts and screening of similar cultural and creative products. Therefore, conducting handicraft image similarity retrieval is of significant practical engineering necessity. The existing image similarity retrieval process mainly involves the following steps: First, preprocessing operations such as image size normalization, background removal, and noise reduction are performed using algorithms such as median filtering and Gaussian filtering. Image features such as contours, textures, and color distribution are extracted based on deep learning models such as convolutional neural networks. Next, feature discrimination is enhanced using optimization methods such as triplet loss. Image feature quantization is completed through hash encoding and high-dimensional feature vector mapping. Then, a feature index library is constructed based on vector retrieval algorithms such as FAISS and Locality Sensitive Hashing. Finally, the feature distance between the image to be retrieved and sample images in the database is calculated to rank the similarity, and the search result with the highest matching degree is output.

[0003] Existing image similarity retrieval methods continuously pursue multi-scale and multi-granular information fusion at the feature representation level to take into account the overall structural semantics of the image. For example, a multi-scale similar image matching retrieval method disclosed in Chinese invention patent application CN119322862B includes: preprocessing the image to be retrieved; extracting multi-scale features from the preprocessed image to obtain a multi-scale feature vector; measuring the similarity between the multi-scale feature vector of the image to be retrieved and the multi-scale feature vector of the matching image; and using the matching image with the highest similarity as the retrieval result.

[0004] Existing image similarity retrieval methods effectively improve the overall and local matching capabilities of regular images through multi-scale feature fusion. However, when it is necessary to perform similarity retrieval on a massive number of images of handicrafts such as ceramics, jade, and textiles that have faded or repaired traces—for example, when users perform similarity retrieval on uploaded images of partially repaired jade ornaments, faded textiles, etc., on an online art trading platform—these methods face challenges due to issues such as local edge anomalies, color distortion, and texture discontinuities in the handicraft images. For instance, the color saturation of painted patterns may decrease, leading to color distortion in the pattern area, or plaster-filled areas may appear different from the surrounding glaze. Texture, etc., and existing technologies usually use image processing algorithms such as median filtering and Gaussian filtering for image size normalization, background removal and other preprocessing. This may cause abnormal edges of the missing area, color distribution of the faded area, and abnormal texture of the repair material to be retained as valid image content. As a result, when extracting the image features of handicrafts based on preset image feature extraction models such as ResNet and VGG, there may be false features in the extracted handicraft image features. For example, missing edges may be misjudged as carved patterns, faded color distribution may be mistaken for inherent color distribution, and repair marks may be mistaken for components of the original texture.

[0005] Furthermore, when performing cross-database similarity ranking based on the features of craft images containing pseudo-features, existing technologies typically employ distance metrics such as Euclidean distance and cosine similarity for similarity calculation and ranking. Since these distance metrics involve each component of all craft image feature dimensions with equal weight or globally fixed participation in the distance calculation, such as the shape outline, decorative texture, and color distribution, pseudo-features may dominate the entire distance calculation result. This can obscure the true impact of the original craft image features on the similarity determination, ultimately leading to low accuracy in craft image similarity retrieval. For example, two Tang Sancai pieces that are originally from the same source and have the same patterns may differ in accuracy after similarity ranking because one piece has severely faded paint while the other has vibrant colors. The Euclidean distance between their global feature vectors will be dominated by the overall color bias caused by the fading, resulting in low accuracy in similarity retrieval. Summary of the Invention

[0006] To address the low accuracy of image similarity retrieval for handicrafts in existing technologies, this invention provides a deep learning-based image similarity retrieval system and method for handicrafts. The technical solution is as follows: On one hand, a deep learning-based craft image similarity retrieval system is provided, including: a craft image degradation monitoring module, a craft image feature weight acquisition module, and a craft image similarity retrieval module. The craft image degradation monitoring module is used to perform anomaly detection on craft images by quantifying the distribution of degradation areas. Based on the detection results, it performs threshold segmentation processing to distinguish different levels of degradation severity and determine whether to implement progressive restoration of the craft images. The craft image feature weight acquisition module is used to recalibrate the craft image feature dimensions after progressive restoration is implemented; otherwise, it directly performs recalibration. The craft image similarity retrieval module is used to calculate the craft image similarity distance based on the recalibration results after feature dimension recalibration, and then performs craft image similarity retrieval based on the calculation results.

[0007] On the other hand, a deep learning-based method for retrieving the similarity of handicraft images is provided, including the following steps: performing anomaly detection on handicraft images by quantifying the distribution of degraded regions; performing threshold segmentation processing based on the detection results to distinguish different levels of degradation severity and determine repairable and unrepairable handicraft images; and deciding whether to implement progressive restoration of handicraft images based on the processing results. When implementing progressive restoration of handicraft images, the feature dimensions of the handicraft images are recalibrated after the progressive restoration is completed; if progressive restoration is not implemented, the feature dimensions of the handicraft images are directly recalibrated. After the feature dimensions of the handicraft images are recalibrated, the similarity distance of the handicraft images is calculated based on the result of the recalibration, and the similarity retrieval of handicraft images is performed based on the calculation results.

[0008] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: 1. The deep learning-based similarity retrieval system and method for craft images provided by this invention performs anomaly detection on craft images, threshold segmentation based on the detection results, and decides whether to implement progressive image restoration based on the processing results. This helps to proactively identify and separate degraded pseudo-features such as incomplete edges, fading, color casts, and restoration traces in craft images, and performs semantic consistency restoration or preserves mask information, reducing feature contamination at the source. When implementing progressive image restoration, the feature dimensions of the craft image are recalibrated after the restoration is completed. If progressive image restoration is not implemented, the craft image features are directly analyzed. Dimension recalibration helps to further suppress residual degraded pseudo-features at the feature level, while also suppressing pseudo-feature dimensions that are susceptible to degradation interference. After the recalibration of the feature dimensions of the craft images is completed, the similarity distance of the craft images is calculated based on the results of the recalibration, which is used for weighted Euclidean distance calculation. Based on the calculation results, craft image similarity retrieval is performed. This helps to dynamically integrate the adaptive weight vector of the craft images into the craft image similarity distance, so that the similarity calculation focuses on features that are not sensitive to degradation and have strong craft discrimination power. At the same time, it effectively reduces the interference of degraded pseudo-features on the craft image retrieval results, effectively solving the problem of low accuracy of craft image similarity retrieval in the existing technology.

[0009] 2. This invention obtains a craft image fusion vector by directly concatenating the high-dimensional feature vector of the craft image with the mask feature vector at the end of the vectors. This helps to overcome the shortcomings of existing technologies that only use a single feature vector or simple feature splicing and ignore degradation priors, providing a basis for generating adaptive feature weights. The craft image fusion vector is input into a preset dimension importance prediction network to obtain an adaptive weight vector for the craft image. This helps to dynamically calculate the importance weight of each feature dimension for each craft image to be queried, based on its degradation state. This overcomes the limitations of existing technologies that use globally fixed weights, which cannot adapt to the differentiated effects of degradation factors such as defects, fading, and repair traces in different images. The adaptive weights of this invention can assign high weights to stable craft feature dimensions and low weights to easily contaminated pseudo-feature dimensions according to the degradation type and degree of the current image, realizing personalized recalibration of feature dimensions. Craft image similarity retrieval based on the craft image adaptive weight vector helps to ensure that the similarity ranking results closely match the essential similarity of craft images, thereby achieving highly robust and accurate similarity retrieval of degraded craft images under multiple acquisition conditions.

[0010] 3. When there are obvious periodic textures around the damaged or faded areas of an artifact image, such as lotus scrolls, cloud and thunder patterns, or locally symmetrical structures, like paired patterns on the belly of porcelain, the periodicity of the texture may be disordered or the symmetry may be disrupted, leading to inconsistencies between the restoration result and the original craft style. Therefore, an alternative scheme for progressive restoration of artifact images is needed. This involves performing a two-dimensional Fourier transform on the annular neighborhood image with a preset mask width around the binary mask of the artifact image to obtain the amplitude spectrum. This helps overcome the shortcomings of existing technologies that rely on large amounts of labeled data to train generative adversarial networks and are prone to periodic disorder. Frequency domain analysis obtains objective texture priors from the source, providing geometric constraints for subsequent restoration. If the absolute value of the difference between the orientation angles of at least two peak frequency points is greater than a preset angle difference threshold, and the proportion of fundamental frequency energy is greater than a preset energy threshold, then periodic texture craft image restoration is performed based on the sub-block to be filled. Otherwise, the symmetry of the craft image is judged, which helps to adaptively select the optimal restoration strategy based on the structural attributes of the craft image itself. This avoids the style inconsistency problem caused by blindly restoring all craft images with a single method, and improves the consistency between the restoration result and the original craft style. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A schematic diagram of the structure of a deep learning-based craft image similarity retrieval system provided in an embodiment of this application; Figure 2 A flowchart outlining the overall process of a deep learning-based craft image similarity retrieval system provided in this application embodiment; Figure 3 A flowchart of a deep learning-based craft image similarity retrieval method provided in an embodiment of this application; Figure 4 A logic diagram of progressive restoration of handicraft images in a deep learning-based handicraft image similarity retrieval system provided in this application embodiment; Figure 5 A comparison chart of precision-recall curves for four distance metrics in the deep learning-based craft image similarity retrieval system provided in the embodiments of this application; Figure 6 A comparison chart of model performance under different distance metrics for the deep learning-based craft image similarity retrieval system provided in the embodiments of this application. Detailed Implementation

[0013] Embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present disclosure are shown in the drawings, it should be understood that embodiments of the present disclosure may be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure.

[0014] It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure. In the description of the embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "this embodiment" should be understood as "at least one embodiment".

[0015] It should also be understood that before using the deep learning-based craft image similarity retrieval system and method provided in this application, a craft image feature retrieval database needs to be pre-constructed. This database includes core retrieval parameters verified by experts in the field of cultural relic digitization, such as preset incompleteness probability thresholds, preset fading probability thresholds, preset restoration probability thresholds, preset area first-level restoration thresholds, and preset area second-level restoration thresholds. Simultaneously, the database also includes standardized pre-processed historical data of craft images, covering original craft images under different acquisition environments, different shooting lighting conditions, and different types of degradation (incompleteness, fading, restoration marks), corresponding craft image binary masks, and similarity positive and negative sample pairs annotated by experts, as well as model iteration optimization data, such as multi-scale feature extraction networks. The database includes fine-tuned weights, attention parameters for the dimensionality importance prediction network, and generator parameters for the conditional generative adversarial network. It employs a hierarchical hybrid storage architecture, utilizing a relational database to store structured parameters and image metadata, a vector database to store high-dimensional feature vectors of the craft images and their corresponding adaptive weight vectors, and a distributed file system to store original craft images and intermediate restoration results. System maintenance personnel can periodically verify and dynamically adjust the core parameters and model weights stored in the database based on user search feedback, newly added craft data, and cross-database comparison evaluations. This ensures the database can continuously adapt to the needs of craft image similarity retrieval in various business scenarios, such as retrieving multiple types of degraded craft images and quickly filtering similar cultural and creative products.

[0016] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0017] Example 1, refer to Figure 1 This application provides a schematic diagram of the structure of a deep learning-based craft image similarity retrieval system. The system includes: a craft image degradation monitoring module, a craft image feature weight acquisition module, and a craft image similarity retrieval module. The craft image degradation monitoring module is used to perform anomaly detection during the craft image similarity retrieval process, quantifying the distribution of degraded areas such as missing edges, fading, color casts, and repair traces in the craft image. Based on the detection results, it performs threshold segmentation to distinguish different levels of degradation severity and determine repairable and irreparable craft images. Based on the processing results, it decides whether to adopt progressive craft image restoration. Progressive craft image restoration is used to perform semantic consistency restoration on the craft image to be restored, while retaining the mask information of irreparable areas for subsequent feature suppression. The craft image degradation monitoring module helps reduce secondary false features caused by blind restoration, and retains mask information for irreparable areas, reducing the pollution of subsequent processes by false features from the source and improving the tolerance for degraded craft images.

[0018] The craft image feature weight acquisition module is used when performing progressive restoration on craft images. After the progressive restoration is completed, the module recalibrates the craft image feature dimensions to suppress residual degradation pseudo-features and enhance the discriminative power of the craft image. When no progressive restoration is performed on the craft image, the recalibration of the craft image feature dimensions is performed directly. Through the craft image feature weight acquisition module, it helps to further suppress residual degradation pseudo-features after preprocessing at the feature level. By fusing depth features with degradation masks, high weights are assigned to craft features that are not sensitive to degradation, such as the shape outline, pattern period, and material texture, while low weights are assigned to feature dimensions that are susceptible to damage, fading, and restoration contamination. This achieves personalized recalibration of feature dimensions and enhances the discriminative power of craft image features.

[0019] The craft image similarity retrieval module is used to calculate the similarity distance of craft images based on the recalibration results of the craft image feature dimension recalibration, and to perform craft image similarity retrieval based on the calculation results to retrieve craft images with high similarity. Through the craft image similarity retrieval module, the similarity measurement can be focused on the real craft image features, while effectively suppressing the contribution of false features and improving the homo-source recall rate and the ability to suppress hetero-source false detection.

[0020] As described above, the craft image degradation monitoring module, craft image feature weight acquisition module, and craft image similarity retrieval module help to construct an end-to-end degradation-robust retrieval link from craft image preprocessing, feature recalibration to distance measurement, jointly achieving high-precision and high-robustness craft image similarity retrieval. Specifically, the restored craft image and mask information output by the craft image degradation monitoring module provide clean input and degradation priors for the craft image feature weight acquisition module, making weight generation more accurate. The craft image adaptive weight vector output by the craft image feature weight acquisition module directly serves the weighted distance calculation of the craft image similarity retrieval module, enabling the retrieval process to dynamically adjust the contribution of feature dimensions according to the degradation state of each image. The three modules work together to systematically solve the technical problem of low accuracy in craft image retrieval in the existing technology.

[0021] Reference Figure 2 The flowchart of the deep learning-based craft image similarity retrieval system provided in this application embodiment shows that: Anomaly detection is performed on the craft image, and threshold segmentation is performed based on the obtained probability response map. The area value and mask confidence score of the craft image are obtained. It is determined whether the area value of the craft image is less than a preset first-level area repair threshold and greater than a preset second-level area repair threshold, and whether the corresponding mask confidence score is greater than a preset confidence threshold. If not, the feature dimension of the craft image is recalibrated; otherwise, progressive repair of the craft image is triggered, and it is determined whether the repair evaluation confidence is greater than a preset quality threshold. If not, an unreliable repair prompt is sent; otherwise, the feature dimension of the craft image is recalibrated, and an adaptive weight vector of the craft image is obtained. Craft image similarity retrieval is performed based on the adaptive weight vector of the craft image.

[0022] As a preferred technical means, the anomaly detection of handicraft images involves the following process: The handicraft image to be retrieved is input into a preset lightweight semantic segmentation network, which outputs a three-channel probability response map. Each channel corresponds to the pixel-level classification probability of the damaged area, faded area, and repair trace area, respectively. The output value for each pixel is a probability score between 0 and 1, representing the confidence level that the pixel belongs to the corresponding degradation category. The three channels include a damage probability map channel, a faded probability map channel, and a repair trace probability map channel. The probability response maps of the three channels include a damage probability map, a faded probability map, and a repair probability map. The damage probability map corresponds to the pixel-level classification probability of the damaged area in the handicraft image; the faded probability map corresponds to the pixel-level classification probability of the faded area in the handicraft image; and the repair probability map corresponds to the pixel-level classification probability of the damaged area in the handicraft image. The pixel-level classification probability of the repair trace area is calculated. Threshold segmentation is performed on the probability response maps of the three channels. The specific process is as follows: pixels with a pixel-level classification probability greater than the corresponding preset probability threshold in each probability response map are set to 1, and those with a lower probability are set to 0, resulting in a binary mask for the craft image, including a defect mask, a fade mask, and a repair mask. The preset probability thresholds include a preset defect probability threshold, a preset fade probability threshold, and a preset repair probability threshold. The preset defect probability threshold is represented by the average pixel-level classification probability in the defect probability map of the historical time period, the preset fade probability threshold is represented by the average pixel-level classification probability in the fade probability map of the historical time period, and the preset repair probability threshold is represented by the average pixel-level classification probability in the repair probability map of the historical time period.

[0023] The incompleteness mask represents the distribution of pixel loss or edge abnormalities in an image of a craft item due to damage or breakage. It is represented as a binary matrix of the same size as the original image, where a pixel with a value of 1 indicates a missing area and a value of 0 indicates a non-missing area. The fading mask represents the distribution of color distortion areas in an image of a craft item caused by pigment aging or color saturation decay. It is also represented as a binary matrix of the same size as the original image, where a pixel with a value of 1 indicates a faded area and a value of 0 indicates a non-faded area. The repair mask represents areas of repair caused by plaster filling, bonding, etc., in an image of a craft item. The distribution of texture or reflective abnormalities caused by human intervention is represented by a binary matrix of the same size as the original image. Pixels with a value of 1 in the matrix indicate that they belong to the repair trace area, while those with a value of 0 indicate that they are not repair trace areas. Mask confidence scores are obtained for each craft image based on the probability response map. The mask confidence scores include confidence scores for incomplete areas, faded areas, and repaired areas. Each mask confidence score is obtained by taking the arithmetic mean of the probability values ​​of all pixels marked as 1 in the corresponding probability response map, which is used to characterize the overall reliability of the detection of craft images with this type of degraded area.

[0024] The process of determining whether repair processing is needed based on the binary mask of the craft image is as follows: First, obtain the connected regions to be repaired in the craft image based on connectivity criteria such as 4-connectivity and 8-connectivity. The craft image to be repaired represents the craft image with a binary mask. Second, obtain the area value of the craft image based on the connected regions to be repaired. Third, use the total number of pixels in the connected regions formed by all pixels with a value of 1 in each craft image binary mask as the craft image area value to measure the scale of abnormal regions in the craft image, including the area value of missing parts and the area value of faded parts. The repair trace probability map is only used for subsequent feature suppression decisions and does not participate in the repair trigger judgment because repair traces are non-original craft defects, and repairing them would introduce unrealistic synthetic textures. Finally, determine whether the corresponding craft image area value is less than a preset area-level repair threshold and greater than a preset area-level repair threshold. The system determines whether the pre-set area repair threshold is greater than a preset confidence threshold. If so, it indicates that the abnormal area of ​​the artwork is small and the detection reliability is high, meeting the conditions for repair. In this case, progressive repair of the artwork image is triggered. The preset area secondary repair threshold is represented by the average area value of the artwork image over a historical time period, and the preset area primary repair threshold is set in advance by a preset person. The preset confidence threshold is represented by the average mask confidence score over a historical time period. Conversely, if the pre-set area is smaller than a preset area, it indicates that the area of ​​the connected region to be repaired is large and lacks sufficient normal texture reference. Forced repair is likely to produce false textures or low mask confidence scores, resulting in unreliable detection results. In this case, the original mask information needs to be retained for subsequent feature extraction stages to guide feature suppression. Alternatively, it indicates that the area of ​​the connected region to be repaired is small, and the impact of the degraded region on the overall features is negligible. In this case, no repair is needed, and the feature dimension recalibration of the artwork image is performed.

[0025] It should be noted that the preset lightweight semantic segmentation network used in this embodiment is specifically U-Net (U-shaped Network), but SegNet (Segmentation Network) and PSPNet (Pyramid Scene Parsing Network) can also be used. The training process of a lightweight semantic segmentation network with an encoder-decoder structure, such as the Pyramid Scene Parsing Network, is as follows: First, a training dataset is constructed by collecting a large number of original images of handicrafts covering different degradation types (damage, fading, and repair marks) from historical time periods. These images are then manually annotated at the pixel level to generate a three-channel label image of the same size as the original handicraft image, with each channel corresponding to a mask for damage, fading, or repair marks. Next, data augmentation is performed on the training samples, including random rotation, horizontal flipping, and random scaling (e.g., scaling by 0.8 to 1.2 times) to improve the network's generalization ability. Then, a weighted cross-entropy loss function is used as the optimization objective, setting different weights for different degradation categories to alleviate class imbalance. The Adam optimizer is used for end-to-end training. Finally, after each training epoch, the average intersection-union ratio (OCR) is calculated on the validation set, and the model parameters with the highest values ​​on the validation set are saved as the final trained lightweight semantic segmentation network.

[0026] As described above, anomaly detection in handicraft images helps to accurately identify the spatial distribution and confidence level of degraded areas such as defects, fading, and repair marks. This reduces the risk of secondary false features caused by blindly repairing or directly discarding degraded information, minimizes the source pollution of degraded false features on deep feature extraction and similarity measurement, and enhances the ability of subsequent handicraft image feature dimension recalibration and handicraft image similarity retrieval to distinguish the essential attributes of the craft. It also realizes a highly robust retrieval mechanism that actively separates degraded noise from real craft features from the preprocessing stage, providing reliable quality assurance for handicraft image similarity retrieval under multiple acquisition conditions.

[0027] As a preferred technical means, the specific process of progressive restoration of handicraft images is as follows: The handicraft image to be restored, the corresponding binary mask of the handicraft image, and the boundary indicator map of the connected regions to be restored are input into a generative adversarial network (GAN) under preset conditions. The network outputs the restored handicraft image and the corresponding restoration confidence map. Boundary pixels of each connected region in the corresponding binary mask of the handicraft image are extracted based on morphological gradients or the Sobel operator. The obtained binary map is used as the boundary indicator map of the connected regions to be restored, where boundary pixels are marked as 1 and internal pixels are marked as 0, guiding the generator to progressively restore from the region boundary inwards. The restored handicraft image represents an image where pixels within the binary mask region of the handicraft image have been regenerated or color-corrected, while pixels outside the binary mask region remain unchanged. The restoration confidence map represents the image compared to the input... A two-dimensional matrix of the same size as the image of the craft to be repaired is generated, where the value of each pixel is between 0 and 1, representing the confidence level of the corresponding pixel, used to characterize the reliability of the repair result for that pixel. The average confidence level of all repaired regions in the repair confidence map, i.e., the binary mask region of the original craft image, is used as the repair judgment confidence level. It is determined whether the repair judgment confidence level is greater than a preset quality threshold. If it is, it indicates that the repair result of the craft image is reliable, and the feature dimension of the craft image is recalibrated. The preset quality threshold is represented by the average value of the repair judgment confidence level over a historical time period. Otherwise, it indicates that there may be unreliable synthetic artifacts in the repair result of the craft image, and an unreliable repair prompt is sent. The original unrepaired craft image and its corresponding binary mask are uploaded to the image data processing center.

[0028] It should be noted that the pre-defined conditional generative adversarial network used in this embodiment is specifically Pix2PixHD (Pix2Pix High-Definition, high-resolution image-to-image conversion network), but EdgeConnect (Edge-Connect, edge connectivity network) and LaMa (Large Mask) can also be used. The specific training process of image restoration networks (such as Inpainting and Large Mask Restoration Networks) with pre-defined conditions for generative adversarial networks (GANs) is as follows: First, a training dataset containing a large number of paired samples is constructed. Each pair of samples consists of a complete, non-degraded image of a craft item and an image of a craft item to be restored. The image of the craft item to be restored and its binary mask are used as input to the generator, and the complete, non-degraded image of the craft item is used as the supervision label. Next, the generator adopts a shared encoder and dual-head decoder architecture. One decoder outputs the restored image, and the other parallel decoder outputs a restoration confidence map of the same size. The generator is conditional on the image of the craft item to be restored and its binary mask, and simultaneously outputs the restored image of the craft item and the corresponding restoration confidence map. The discriminator simultaneously receives the output image of the generator and the complete, non-degraded image of the craft item and determines whether the input image is a real restoration result or a fake restoration result. Then, the loss functions of the generator and the discriminator are alternately optimized. The generator's loss includes adversarial loss, perceptual loss, style loss, and confidence loss. Finally, after training convergence, the network parameters of the generator are saved for progressive restoration of craft item images.

[0029] As described above, progressive restoration of handicraft images helps to accurately locate areas in handicraft images that have problems such as fading, repair marks, and abnormal local edges. This ensures that the restoration process conforms to the original texture, color, and shape characteristics of the handicraft, reduces the probability of synthetic artifacts during the restoration process, minimizes the generation of false features due to improper restoration, and improves the reliability and authenticity of the restored handicraft image results. At the same time, it enables accurate verification of the restoration effect, reduces the interference of unreliable restoration results with the subsequent similarity retrieval process, and ensures that the restored handicraft image can truly restore its original craft features, providing solid support for the accuracy of subsequent handicraft image similarity retrieval.

[0030] As a preferred technical means, the specific process of recalibrating the feature dimension of the craft image is as follows: First, the craft image is input into a preset multi-scale feature extraction network, which outputs a high-dimensional feature vector of the craft image. This high-dimensional feature vector is specifically a one-dimensional vector that encodes craft information such as the shape outline, pattern structure, and color distribution within the craft image. Second, the binary mask of the craft image is downsampled and encoded into a mask feature vector. Specifically, based on a bilinear interpolation algorithm, the size of each binary mask in the craft image's binary mask is scaled to a preset size, such as 7×7 pixels, to obtain the scaled mask matrix. In the first step, the preset size is set in advance by preset personnel; the scaled mask matrix is ​​flattened row by row into one-dimensional vectors, thus obtaining three one-dimensional vectors of equal length; then the three one-dimensional vectors of equal length are concatenated end to end in the channel dimension to obtain the mask feature vector; the high-dimensional feature vector of the craft image is directly concatenated end to end with the mask feature vector to obtain the craft image fusion vector; in the third step, the craft image fusion vector is input into a preset dimension importance prediction network to obtain the craft image adaptive weight vector, with each weight value between 0 and 1; in the fourth step, craft image similarity retrieval is performed based on the craft image adaptive weight vector.

[0031] It should be noted that the preset multi-scale feature extraction network used in this embodiment is specifically ResNet50 (Residual Network 50, 50 layers), but ResNet101, VGG16, InceptionV3, and other backbone networks can also be used. The specific training process of the preset multi-scale feature extraction network is as follows: First, pre-training is performed on the ImageNet large-scale image classification dataset to enable the network to have general image feature extraction capabilities; then, fine-tuning is performed on a craft image dataset labeled with craft categories, such as kiln, vessel type, and pattern type, using the cross-entropy loss function and Adam optimizer, training for a preset number of training epochs, and saving the model parameters with the highest accuracy on the validation set as the final feature extraction network. The preset number of training epochs is set in advance by a preset team.

[0032] It should also be noted that the preset dimension importance prediction network used in this embodiment is specifically SENet (Squeeze-and-Excitation Network), but CBAM (Convolutional Block Attention Module) or ECA-Net (Efficient Channel Attention Network) can also be used. The specific training process of the preset dimension importance prediction network is as follows: First, the high-dimensional feature vector of the craft image is concatenated with the corresponding mask feature vector and used as the sample input. The triple loss is used as the optimization objective, where positive sample pairs come from different views of the same craft under slight degradation, and negative sample pairs come from feature vectors of different crafts. Next, the Adam optimizer is used to train for a preset number of training rounds. In each round, the average inverse ranking of the weighted distance on the validation set is calculated, and the model weight with the highest MRR is saved, so that the network learns to allocate high weights to stable craft feature dimensions and low weights to dimensions that are prone to degradation and contamination.

[0033] As described above, recalibrating the feature dimensions of craft images helps to dynamically adjust the contribution weight of each feature dimension based on the degradation mask information of each craft image. This allows essential features of craft images that are not easily affected by degradation, such as the shape outline, pattern cycle, and material texture, to receive high weights, while pseudo-feature dimensions that are easily contaminated, such as incomplete edges, faded colors, and repaired textures, are given low weights. This reduces the risk of residual degraded pseudo-features dominating similarity measurements and achieves personalized recalibration at the feature level.

[0034] As a preferred technical means, the specific process of craft image similarity retrieval is as follows: Step 1, read the high-dimensional feature vector of each craft image from the preset craft image feature database. The preset craft image feature database is set in advance by preset personnel and stored in the craft image feature retrieval database. The preset craft image feature database adopts a hierarchical hybrid storage architecture. The vector database part specifically stores the high-dimensional feature vector of each craft image and its corresponding adaptive weight vector, which are pre-generated by the same preset multi-scale feature extraction network. At the same time, it associates the metadata stored in the relational database. The reading operation is realized through key-value pair query or vector index scanning, directly obtaining the high-dimensional feature vector of the target image as a candidate reference for subsequent weighted distance calculation. There is no need to repeatedly perform feature extraction during retrieval, thereby ensuring the real-time performance and computational efficiency of batch similarity retrieval. Step 2, obtain the craft image similarity distance between the craft image and each craft image in the preset craft image feature database based on the weighted Euclidean distance. The specific calculation formula is as follows: ; Where d j Let represent the similarity distance between the craft images, j represent the index of each craft image in the preset craft image feature database, j=1, 2, 3, ..., N, and N represent the total number of craft images in the preset craft image feature database, w i Let q be the i-th component of the adaptive weight vector, i = 1, 2, 3, ..., C, where i represents the index of each dimension in the high-dimensional feature vector of the craft image, C represents the total number of dimensions in the high-dimensional feature vector of the craft image, and q is the index of each dimension in the high-dimensional feature vector of the craft image. i p represents the value of the high-dimensional feature vector of the image of the craft to be queried in the i-th dimension. j,i This represents the value of the high-dimensional feature vector of the j-th craft image in the i-th dimension.

[0035] Step 3: Sort all craft images by similarity distance from smallest to largest. The smaller the similarity distance, the higher the similarity. The weighted Euclidean distance amplifies differences in high-weighted features, such as shape outlines and pattern repetition cycles, which are less susceptible to degradation, while suppressing differences in low-weighted features, such as abnormal edges caused by damage or overall color cast due to pigment decay, which are easily contaminated by defects, fading, or repair marks. This allows the similarity distance to more accurately reflect the similarity between two craft images. Select the craft images with the smallest similarity distance from a preset number of craft image feature databases as the craft image retrieval results. The preset number is set in advance by a preset team. Step 4: Obtain the craft image retrieval result list, which includes a unique identifier for each craft image in each retrieval result, such as image ID, collection number, and corresponding craft image similarity distance.

[0036] As described above, similarity retrieval of craft images helps to dynamically incorporate adaptive weights into weighted Euclidean distance calculations, allowing the retrieval process to focus on stable craft image features such as shape outlines and pattern cycles. Simultaneously, it effectively suppresses the contribution of false features such as incomplete edges and faded colors, reducing the false positive rate of genuine artifacts from the same source and the false positive rate of counterfeit artifacts from different sources due to image degradation. It also reduces retrieval bias caused by degradation-related false features dominating the ranking results under equal-weighted distance, achieving highly robust and accurate similarity retrieval of degraded craft images.

[0037] Reference Figure 3As shown in the flowchart of the deep learning-based craft image similarity retrieval method provided in this application embodiment: A1, Craft image degradation monitoring: Craft image anomaly detection is performed to quantify the distribution of degradation areas. Based on the detection results, threshold segmentation processing is performed to distinguish different degradation severity levels and determine repairable and unrepairable craft images. Based on the processing results, a decision is made on whether to adopt progressive restoration of the craft image. By monitoring the degradation of craft images, it is helpful to quantify the spatial distribution and confidence of degradation areas such as defects, fading, and repair traces with high precision, providing interpretable quantitative basis for restoration decisions, while reducing ineffective restoration.

[0038] A2, Acquisition of Feature Weights for Craft Images: When performing progressive restoration on craft images, the feature dimensions of the craft images are recalibrated after the progressive restoration is completed. If progressive restoration is not performed on the craft images, the feature dimensions of the craft images are recalibrated directly. By acquiring the feature weights of the craft images, it is helpful to effectively overcome the shortcomings of the global fixed weighting method in adapting to the degradation differences of a single craft image.

[0039] A3, Craft Image Similarity Retrieval: After the recalibration of the feature dimensions of the craft images is completed, the similarity distance of the craft images is calculated based on the results of the recalibration, and the craft image similarity retrieval is performed based on the calculation results. Through craft image similarity retrieval, the similarity ranking results are made more dependent on the essential attributes of the craft rather than degenerate noise, and the stability and anti-interference ability of the retrieval results under different acquisition conditions are enhanced.

[0040] As described above, by monitoring the degradation of handicraft images, obtaining the feature weights of handicraft images, and retrieving the similarity of handicraft images, it is helpful to suppress degradation interference step by step from three levels: preprocessing, feature transformation, and distance measurement. This progressive approach forms a closed-loop iterative optimization, enhancing the system's adaptability and retrieval accuracy in scenarios with multiple degradation types.

[0041] Example 2, refer to Figure 4The logic diagram of the progressive restoration of craft images in the deep learning-based craft image similarity retrieval system provided in this application embodiment shows that: progressive restoration of craft images is performed, and it is determined whether there is at least two peak frequency points whose absolute value of the difference in direction angle is greater than a preset angle difference threshold, and whether the fundamental frequency energy ratio is greater than a preset energy threshold. If so, periodic texture craft image restoration is performed; otherwise, craft image symmetry discrimination is performed, and it is determined whether any autocorrelation symmetry measure is greater than a preset symmetry threshold. If so, symmetrical structure craft image restoration is performed; otherwise, conventional progressive restoration of craft images is adopted. After restoration is completed, the restoration judgment confidence is re-acquired, and it is determined whether the restoration judgment confidence is greater than a preset quality threshold. If so, the craft image feature dimension is recalibrated; otherwise, an unreliable restoration prompt is sent.

[0042] As an alternative to the progressive restoration of the craft image in Example 1, when there are obvious periodic textures around the damaged or faded areas of the craft image, such as lotus scroll patterns, cloud and thunder patterns, or locally symmetrical structures, such as paired patterns on the belly of porcelain, the texture period may be disordered or the symmetry may be destroyed, resulting in the restoration result being inconsistent with the original craft style. Therefore, it is necessary to perform progressive restoration of the craft image. The specific process is as follows: Based on the Fourier spectrum analysis method, a two-dimensional Fourier transform is performed on the annular neighborhood image of the craft image with a preset mask width outside the binary mask, to obtain a complex frequency spectrum with the same size as the binary mask of the craft image. The amplitude spectrum is obtained by taking the modulus of the complex spectrum matrix. The preset mask width is pre-set by a pre-defined team. Non-maximum suppression (NMS) is applied to the amplitude spectrum to obtain local maxima. NMS means that for each pixel in the amplitude spectrum, it is compared with a preset number of neighbors within a preset neighborhood (e.g., 8 neighbors in a 3×3 neighborhood). If the amplitude value of the pixel is greater than the amplitude values ​​of all neighboring pixels, the frequency point is determined as a local maximum; otherwise, it is set to zero. The local maximum values ​​are then sorted in descending order, and the top preset number of frequency points are selected as peak frequencies. Each peak frequency point contains two components: horizontal frequency and vertical frequency. For example, a peak frequency point with coordinates (12, 5) indicates a horizontal frequency of 12 cycles per image width and a vertical frequency of 5 cycles per image height. The preset frequency count is determined in advance by the user. The horizontal frequency component of the current peak frequency point is summed with a preset minimum value. Then, the vertical frequency component of the current peak frequency point is divided by the summation result. The result of this division is input into the arctangent function to obtain a radian value. This radian value is then converted into a direction angle. The preset minimum value is also determined in advance by the user. The method is set to avoid invalid division by zero in the denominator; if the absolute value of the difference between the orientation angles of at least two peak frequency points is greater than the preset angle difference threshold, and the proportion of fundamental frequency energy is greater than the preset energy threshold, then it is determined that the current craft image to be repaired has periodic texture, and then the periodic texture craft image repair is performed based on the sub-block to be filled. The preset angle difference threshold is represented by the average value of the absolute value of the difference between the orientation angles of every two peak frequency points in the historical time period, and the preset energy threshold is represented by the average value of the proportion of fundamental frequency energy in the historical time period; otherwise, it means that no periodic texture is detected, and then the symmetry of the craft image is judged.

[0043] The specific process for determining the symmetry of the craft image is as follows: The normalized autocorrelation symmetry measure of the annular neighborhood image about the candidate symmetry axes is obtained to quantify the degree of mirror symmetry of the craft image structure along different axes. Candidate symmetry axes include the vertical axis, horizontal axis, 45-degree diagonal axis, and 135-degree diagonal axis. The specific method for obtaining the autocorrelation symmetry measure is as follows: For each candidate symmetry axis, the annular neighborhood image is mirrored along the candidate symmetry axis. A normalized cross-correlation matrix is ​​obtained between the original annular neighborhood image and the flipped annular neighborhood image based on the normalized cross-correlation algorithm. The size of this normalized cross-correlation matrix is ​​the same as that of the original annular neighborhood image. The value at each position in the normalized cross-correlation matrix represents the local region of the corresponding pixel, such as the correlation of a 5×5 window centered on that pixel. The average value of all pixels within the central region of the normalized cross-correlation matrix is ​​taken as the current candidate pair. The autocorrelation symmetry of the axis is measured; the central region represents a square region centered on the center of the normalized cross-correlation matrix with a side length equal to the preset ratio of the normalized cross-correlation matrix; the candidate symmetry axis with the largest autocorrelation symmetry is marked as the main symmetry axis; the binary mask region of the craft image, including the incomplete mask region and the faded mask region, is divided into several sub-blocks to be filled that match the preset period unit size. The specific process is as follows: the reciprocal of the frequency value corresponding to the fundamental frequency in the peak frequency point is taken as the main period length; the direction angle of the fundamental frequency in the peak frequency point is taken as the main direction; the fundamental frequency represents the frequency point with the largest amplitude value in the peak frequency point; starting from the boundary of the binary mask region of the craft image, a grid is generated along the main direction and its perpendicular direction with a step size of the main period length. Each grid unit divided by the grid within the binary mask region of the craft image is taken as a sub-block to be filled.

[0044] If any autocorrelation symmetry metric is greater than a preset symmetry threshold, the current image to be repaired is determined to be a locally symmetrical structure, and symmetrical structure image repair is then performed based on the sub-blocks to be filled. Otherwise, the progressive image repair method in Example 1 is adopted. The preset symmetry threshold is represented by the average value of the autocorrelation symmetry metric over a historical time period. The process of obtaining the fundamental frequency energy ratio is as follows: the frequency point with the largest amplitude value in the peak frequency is taken as the fundamental frequency; the sum of the squares of all amplitude values ​​in a circular neighborhood centered on the fundamental frequency with a radius of a preset amplitude size is taken as the fundamental frequency energy; the sum of the squares of all amplitude values ​​in the entire amplitude spectrum is taken as the total energy; the result of dividing the fundamental frequency energy by the total energy is taken as the fundamental frequency energy ratio.

[0045] Specifically, the process of periodic textured craft image restoration is as follows: Within the annular neighborhood image, image blocks at corresponding positions are moved along the main direction by one main period length and along the vertical direction by a preset integer number of period lengths, and these position image blocks are marked as candidate source blocks. Based on the candidate source blocks, the craft image is restored in a targeted manner, specifically as follows: For the incomplete mask region of the binary mask area of ​​the craft image, based on each sub-block to be filled, the similarity score between each candidate source block and the surrounding preset width neighborhood blocks is obtained based on the normalized cross-correlation algorithm. The candidate source block with the highest similarity score is selected as the filling sub-block. The preset width neighborhood is set in advance by a preset person. The period offset represents the distance moved along the main direction by one main period length. The filling sub-block is directly filled into the position of the sub-block to be filled according to the period position. Then, the gradient field of the block boundary and the surrounding pixels is seamlessly fused based on the Poisson image editing algorithm to eliminate the stitching seam. For the faded mask region of the binary mask area of ​​the craft image, the craft image is restored by fading. The periodic textured craft image restoration is repeated until all sub-blocks to be filled are processed.

[0046] Specifically, the process for restoring faded images of handicrafts is as follows: The cumulative distribution function of the RGB three channels is statistically analyzed within the corresponding annular neighborhood image; that is, for each channel, the cumulative frequency of pixel values ​​from 0 to 255 is statistically analyzed. The quantiles of the RGB values ​​of each pixel in the handicraft image to be restored within the cumulative distribution of the faded mask area are obtained. Then, based on a histogram specification algorithm, these quantiles are mapped to the pixel values ​​of the corresponding quantiles in a preset reference histogram, forming new color values. The new color values ​​are then migrated one by one according to their pixel positions to the corresponding pixels in the faded area of ​​the handicraft image to be restored, completing the color replacement. The preset reference histogram represents the cumulative distribution function of the RGB three channels obtained statistically from the annular neighborhood image along the main direction or from undamaged symmetrical areas, serving as the target distribution for color restoration in the faded area. The migrated new color values ​​are smoothed using a guided filtering algorithm to preserve the high-frequency details contained in the texture.

[0047] Specifically, the process for image restoration of symmetrical handicrafts is as follows: For the incomplete mask region of the binary mask area of ​​the handicraft image, the principal axis of symmetry is obtained; based on the image registration algorithm of thin-plate spline interpolation and the principal axis of symmetry, the binary mask region of the handicraft image is mapped to the undamaged symmetrical region through mirroring or rotation transformation; the undamaged symmetrical region is filled into the sub-block to be filled using thin-plate spline interpolation to obtain the filled sub-block, and then the gradient domain fusion of the filled sub-block and the surrounding area of ​​the sub-block to be filled is performed based on the Poisson image editing algorithm; the undamaged symmetrical region is represented by the axis of symmetry as a reference in the handicraft image. The same geometric location on the other side of the binary mask, but not covered by the binary mask of the craft image; for the faded mask area of ​​the binary mask area of ​​the craft image, perform craft image fading repair; after the progressive repair of the craft image is completed, reacquire the repaired craft image and the corresponding repair judgment confidence, and determine whether the repair judgment confidence is greater than the preset quality threshold. If so, the feature dimension of the craft image is recalibrated; otherwise, send a repair unreliable prompt and upload the original unrepaired craft image and its corresponding craft image binary mask to the image data processing center.

[0048] As described above, progressive restoration of handicraft images helps to automatically estimate the principal period and principal direction for degraded areas with periodic textures or symmetrical structures. This reduces the reliance on labeled training data and generative adversarial network restoration models, minimizes the problem of inconsistent styles or periodic errors in restoration results due to a lack of large-scale samples, and improves the geometric consistency and visual naturalness of the restoration results with the original handicraft image patterns and symmetrical structures. This enables a lightweight and style-fidelity restoration mechanism for rare patterns or isolated defects.

[0049] refer to Figure 5 The precision-recall curves of the deep learning-based craft image similarity retrieval system provided in this application are compared under four distance metrics. Figure 5It can be seen that the precision-recall curves of different preset dimensionality importance prediction networks show significant differences under weighted Euclidean distance based on SENet weights, weighted Euclidean distance based on CBAM weights, weighted Euclidean distance based on ECA-Net weights, and conventional Euclidean distance. The custom CNN model has the lowest performance across all distance methods, with an mAP (mean Average Precision) of only 0.495. Precision decreases rapidly with increasing recall, and the overall curve area is the smallest. The VGG16, ResNet50, and ResNet101 models perform significantly better. The ResNet50 model has the highest mAP at 0.927, and its precision-recall curve is closer to the upper right corner, maintaining the highest precision at the same recall level, with a more gradual downward trend. The ResNet101 model (mAP = 0.909) and the VGG16 model (mAP = 0.892) follow. Meanwhile, compared to... The conventional Euclidean distance and three weighted Euclidean distance methods based on attention mechanism weights all improved the precision-recall curves of ResNet50, ResNet101, and VGG16 models. Among them, the weighted Euclidean distance based on ECA-Net and SENet weights showed a more significant performance improvement for the ResNet50 model, with a more gradual decline in the curve. In contrast, the precision of each model decreased faster in the high recall range under the conventional Euclidean distance method. This indicates that the weighted Euclidean distance method adopted in this application can effectively improve the overall performance of precision and recall in the craft image similarity retrieval task and suppress spurious feature interference.

[0050] refer to Figure 6 The performance comparison chart of the deep learning-based craft image similarity retrieval system provided in this application under different distance metrics is shown in the following embodiments. Figure 6It can be seen that, under the four core retrieval evaluation metrics of F1-Score@4, mAP, Precision@4, and Recall@4, the performance of the three preset dimension importance prediction networks, ResNet50, ResNet101, and VGG16, is better than the custom CNN model. Among them, the ResNet50 model performs best in all metrics, with its F1-Score@4, mAP, Precision@4, and Recall@4 metrics all at a high level. At the same time, under the same preset dimension importance prediction network, the weighted Euclidean distance method based on the three attention weights of SENet, CBAM, and ECA-Net has better overall performance than the conventional Euclidean distance comparison method. In particular, the ResNet50 model reaches the peak of all metrics under the weighted Euclidean distance of ECA-Net weights and SENet weights. This result shows that the weighted Euclidean distance metric method proposed in this application can effectively improve the comprehensive performance of the craft image similarity retrieval task, suppress false feature interference, and improve the retrieval precision, recall, and overall matching quality.

[0051] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the above functions can be divided into different functional modules to complete all or part of the functions described above.

[0052] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0053] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A deep learning-based image similarity retrieval system for handicrafts, characterized in that, include: The module includes a craft image degradation monitoring module, a craft image feature weight acquisition module, and a craft image similarity retrieval module. The craft image degradation monitoring module is used to perform anomaly detection on craft images by quantifying the distribution of degradation areas, determine threshold segmentation processing for repairable and non-repairable craft images based on the detection results, and decide whether to adopt progressive restoration of the craft images based on the processing results; including: The image of the craft to be retrieved is input into a preset lightweight semantic segmentation network, which outputs a three-channel probability response map. Each channel corresponds to the pixel-level classification probability of the damaged area, the faded area, and the repair trace area, respectively. Threshold segmentation is performed on the probability response maps to obtain a binary mask of the craft image; The mask confidence score corresponding to the binary mask of each craft image is obtained based on the probability response map; Determine whether repair processing is needed based on a binary mask of the artwork image; Based on connectivity criteria, the connected regions to be repaired in the image of the craft item to be repaired are obtained; Obtain the area value of the artwork image based on the connected regions to be repaired; The total number of pixels in the connected region formed by all pixels with a value of 1 in the binary mask of each craft image is taken as the area value of the craft image, including the area value of defects and the area value of fading. Determine whether the area value of the corresponding craft image is less than the preset first-level area repair threshold and greater than the preset second-level area repair threshold, and whether the corresponding mask confidence score is greater than the preset confidence threshold. If so, trigger progressive repair of the craft image; otherwise, retain the original mask information. The craft image feature weight acquisition module is used to perform feature dimension recalibration on the craft image after progressive restoration is completed when progressive restoration is performed on the craft image; otherwise, feature dimension recalibration is performed directly when progressive restoration is not performed on the craft image; including: Periodic texture detection is performed based on Fourier spectrum analysis. If periodic texture is detected, the corresponding image block is moved along the main direction by one main period length and along the vertical direction by a preset integer number of period lengths within the annular neighborhood image. This position image block is marked as a candidate source block. The craft image is then repaired based on the candidate source blocks. If no periodic texture is detected, the symmetry of the craft image is determined, and the normalized cross-correlation matrix of the annular neighborhood image about the candidate symmetry axis is obtained. The average value of all pixels in the central region of the normalized cross-correlation matrix is ​​taken as the autocorrelation symmetry measure of the current candidate symmetry axis. It is determined whether the autocorrelation symmetry measure is greater than the preset symmetry threshold. If so, the undamaged symmetry region is filled into the sub-block to be filled by thin plate spline interpolation to obtain the filled sub-block. The symmetric structure craft image restoration is performed based on the sub-block to be filled. Otherwise, the restoration is based on the conditional generative adversarial network. The image of the craft is input into a preset multi-scale feature extraction network, which outputs a high-dimensional feature vector of the craft image; the high-dimensional feature vector encodes the craft information such as the shape outline, pattern structure, and color distribution in the craft image. The binary mask of the craft image is downsampled and encoded into a mask feature vector. The high-dimensional feature vector is concatenated with the mask feature vector and input into a preset dimension importance prediction network to obtain the adaptive weight vector of the craft image. The craft image similarity retrieval module is used to calculate the similarity distance of craft images based on the result of recalibration of the feature dimensions of craft images, and to perform craft image similarity retrieval based on the calculation result.

2. The deep learning-based craft image similarity retrieval system as described in claim 1, characterized in that, The specific process for detecting anomalies in the artwork images is as follows: The probability response diagram includes the incompleteness probability diagram, the fading probability diagram, and the repair probability diagram; Threshold segmentation is performed on the probability response maps of the three channels respectively. The specific process is as follows: In each probability response image, pixels with a classification probability greater than the corresponding preset probability threshold are set to 1, and those with a lower probability are set to 0, thus obtaining a binary mask for the craft image. The preset probability thresholds include a preset defect probability threshold, a preset fading probability threshold, and a preset repair probability threshold; The mask confidence score includes confidence in the defective area, confidence in the faded area, and confidence in the repaired area.

3. The deep learning-based craft image similarity retrieval system as described in claim 1, characterized in that, The image of the craft to be repaired refers to the craft image with a binary mask.

4. The deep learning-based craft image similarity retrieval system as described in claim 1, characterized in that, The specific process of progressive restoration of the artwork image is as follows: The image of the artwork to be repaired, the corresponding binary mask of the artwork image, and the boundary indicator map of the connected region to be repaired are input into preset conditions to generate an adversarial network, which outputs the repaired artwork image and the corresponding repair confidence map. Extract the boundary pixels of each connected region in the corresponding binary mask of the craft image, and use the resulting binary image as the boundary indicator map of the connected region to be repaired; The average of the confidence scores of all repaired regions in the repair confidence map, i.e., the binary mask regions of the original craft image, is used as the repair evaluation confidence score. Determine whether the confidence level of the repair assessment is greater than the preset quality threshold. If so, recalibrate the feature dimensions of the artwork image. Conversely, if the repair is unreliable, a prompt will be sent, and the original unrepaired image of the craft and its corresponding binary mask will be uploaded to the image data processing center.

5. The deep learning-based craft image similarity retrieval system as described in claim 4, characterized in that, The specific process for recalibrating the feature dimensions of the craft image is as follows: The image of the handicraft is input into a preset multi-scale feature extraction network, and the high-dimensional feature vector of the handicraft image is output. The process of downsampling and encoding the binary mask of the craft image into a mask feature vector is as follows: The size of each binary mask in the binary mask of the craft image is scaled to a preset size based on the bilinear interpolation algorithm; The scaled mask matrix is ​​flattened row by row into a one-dimensional vector, resulting in three one-dimensional vectors of equal length. Then, the three one-dimensional vectors of equal length are concatenated end to end along the channel dimension to obtain the mask feature vector; The high-dimensional feature vector of the craft image is directly concatenated with the mask feature vector at the end of the vector to obtain the craft image fusion vector. The fused vector of the craft image is input into a preset dimension importance prediction network to obtain the adaptive weight vector of the craft image. Similarity retrieval of handicraft images based on adaptive weight vectors of handicraft images.

6. The deep learning-based craft image similarity retrieval system as described in claim 5, characterized in that, The specific process of image similarity retrieval for the handicrafts is as follows: Read the high-dimensional feature vector of each craft image from the preset craft image feature database; The similarity distance between the craft images and each craft image in the preset craft image feature database is obtained based on the weighted Euclidean distance. Sort all craft images by similarity distance from smallest to largest; The craft images with the smallest similarity distance among the first preset number of craft images are selected as the craft image retrieval results; Retrieve a list of handicraft image search results, containing a unique identifier for each handicraft image in each search result.

7. The deep learning-based craft image similarity retrieval system as described in claim 1, characterized in that, The progressive restoration of the artwork image also includes: Based on the Fourier spectrum analysis method, a two-dimensional Fourier transform is performed on the annular neighborhood image of the craft image with a preset mask width around the binary mask. A complex spectrum matrix with the same size as the binary mask of the craft image is obtained. The amplitude spectrum is obtained by taking the modulus of the complex spectrum matrix. Non-maximum suppression is applied to the amplitude spectrum to obtain local maxima points. Then, the local maxima points are sorted in descending order of amplitude values, and the first preset number of frequency points are selected as peak frequency points. The horizontal frequency component of the current peak frequency point is summed with the preset minimum value. Then, the vertical frequency component of the current peak frequency point is divided with the summation result. The result of the division is input into the arctangent function to obtain the radian value. The radian value is then converted into a direction angle. The non-maximum suppression means that for each pixel in the amplitude spectrum, it is compared with a preset number of neighbors in a preset neighborhood. If the amplitude value of the pixel is greater than the amplitude values ​​of all neighboring pixels, then the frequency point is determined as a local maximum point; otherwise, it is set to zero. If the absolute value of the difference in the directional angles of at least two peak frequency points is greater than the preset angle difference threshold, and the proportion of fundamental frequency energy is greater than the preset energy threshold, perform periodic texture craft image repair based on the sub-block to be filled. Conversely, the symmetry of the artwork images is determined. The specific process for determining the symmetry of the craft image is as follows: Obtain the normalized autocorrelation symmetry measure of the annular neighborhood image about the candidate symmetry axis; The specific method for obtaining the autocorrelation symmetry measure is as follows: For each candidate axis of symmetry, the annular neighborhood image is mirrored and flipped along the candidate axis of symmetry. The normalized cross-correlation matrix between the original annular neighborhood image and the flipped annular neighborhood image is obtained based on the normalized cross-correlation algorithm. The value of each position in the normalized cross-correlation matrix represents the local region of the corresponding pixel. The average value of all pixels within the central region of the normalized cross-correlation matrix is ​​taken as the autocorrelation symmetry measure of the current candidate symmetry axis. The candidate symmetry axis with the largest autocorrelation symmetry measure is marked as the principal symmetry axis; The binary mask region of the craft image is divided into several sub-blocks to be filled, each matching the size of a preset periodic unit. The specific process is as follows: The reciprocal of the frequency value corresponding to the fundamental frequency at the peak frequency point is taken as the length of the main period. The direction angle of the fundamental frequency at the peak frequency point is taken as the main direction; The fundamental frequency represents the frequency point with the largest amplitude value among the peak frequency points; Starting from the boundary of the binary mask region of the craft image, a grid is generated along the main direction and its perpendicular direction with the main period length as the step size. Each grid cell divided by the grid within the binary mask region of the craft image is a sub-block to be filled. Determine if any autocorrelation symmetry metric is greater than a preset symmetry threshold. If so, perform symmetric structure craft image restoration based on the sub-block to be filled; otherwise, adopt conventional progressive craft image restoration. The process for obtaining the fundamental frequency energy percentage is as follows: The frequency point with the largest amplitude value in the peak frequency is taken as the fundamental frequency; The sum of the squares of all amplitude values ​​within a circular neighborhood centered on the fundamental frequency and with a radius of a preset amplitude size is taken as the fundamental frequency energy. The total energy is the sum of the squares of all amplitude values ​​in the entire amplitude spectrum. The result of dividing the fundamental frequency energy by the total energy is taken as the proportion of fundamental frequency energy.

8. The deep learning-based craft image similarity retrieval system as described in claim 7, characterized in that, The specific process for restoring the periodic textured craft image is as follows: Within the annular neighborhood image, the corresponding image block is moved along the main direction by one main cycle length and along the vertical direction by a preset integer number of cycle lengths, and its position image block is marked as a candidate source block. The handicraft image is repaired based on candidate source blocks, and the specific process is as follows: For the incomplete mask region of the binary mask region of the craft image, based on each sub-block to be filled, the similarity score between each candidate source block and the surrounding neighborhood block of the sub-block to be filled is obtained based on the normalized cross-correlation algorithm, and the candidate source block with the highest similarity score is selected as the filling sub-block. The filling sub-blocks are directly filled into the positions of the sub-blocks to be filled according to the periodic positions. The gradient fields of the block boundaries and surrounding pixels are seamlessly integrated based on the Poisson image editing algorithm to eliminate the stitching seams. For the faded mask area of ​​the binary mask region of the craft image, perform fading repair on the craft image; Repeat the periodic texture craft image restoration until all sub-blocks to be filled have been processed.

9. The deep learning-based craft image similarity retrieval system as described in claim 8, characterized in that, The specific process for restoring the faded image of the handicraft is as follows: The cumulative distribution function of the three RGB channels is calculated within the corresponding annular neighborhood image; The quantile of the RGB values ​​of each pixel in the image of the craft to be repaired is obtained in the cumulative distribution of the faded mask area. Based on the histogram specification algorithm, the quantile is mapped to the pixel value of the corresponding quantile of the preset reference histogram to form a new color value. The new color values ​​are migrated one by one to the corresponding pixels in the faded area of ​​the image of the craft to be repaired, according to their pixel positions, to complete the color replacement; The new color values ​​after migration are smoothed using a guided filtering algorithm; The specific process for restoring the image of the symmetrical structured craft is as follows: For the incomplete mask region of the binary mask region of the craft image, obtain the principal axis of symmetry; Based on the thin plate spline interpolation image registration algorithm and principal axis of symmetry, the binary mask region of the craft image is mapped to the undamaged symmetrical region through mirror or rotation transformation; The undamaged symmetrical region is filled into the sub-block to be filled using thin-plate spline interpolation to obtain the filled sub-block. The filled sub-block is then fused with the surrounding region of the sub-block to be filled using the Poisson image editing algorithm in the gradient domain. For the faded mask area of ​​the binary mask region of the craft image, perform fading repair on the craft image; After the progressive restoration of the craft image is completed, the restored craft image and the corresponding restoration judgment confidence are reacquired. It is then determined whether the restoration judgment confidence is greater than the preset quality threshold. If so, the feature dimensions of the craft image are recalibrated. Otherwise, an unreliable restoration prompt is sent, and the original unrestored craft image and its corresponding craft image binary mask are uploaded to the image data processing center.

10. A deep learning-based method for retrieving craft images based on similarity, characterized in that, Includes the following steps: Anomaly detection of handicraft images is performed by quantifying the distribution of degraded regions. Based on the detection results, threshold segmentation processing is determined to classify repairable and non-repairable handicraft images. Based on the processing results, a decision is made on whether to implement progressive restoration of the handicraft images. This includes: The image of the craft to be retrieved is input into a preset lightweight semantic segmentation network, which outputs a three-channel probability response map. Each channel corresponds to the pixel-level classification probability of the damaged area, the faded area, and the repair trace area, respectively. Threshold segmentation is performed on the probability response maps to obtain a binary mask of the craft image; The mask confidence score corresponding to the binary mask of each craft image is obtained based on the probability response map; Determine whether repair processing is needed based on a binary mask of the artwork image; Based on connectivity criteria, the connected regions to be repaired in the image of the craft item to be repaired are obtained; Obtain the area value of the artwork image based on the connected regions to be repaired; The total number of pixels in the connected region formed by all pixels with a value of 1 in the binary mask of each craft image is taken as the area value of the craft image, including the area value of defects and the area value of fading. Determine whether the area value of the corresponding craft image is less than the preset first-level area repair threshold and greater than the preset second-level area repair threshold, and whether the corresponding mask confidence score is greater than the preset confidence threshold. If so, trigger progressive repair of the craft image; otherwise, retain the original mask information. When performing progressive image restoration on an artwork image, the feature dimensions of the artwork image are recalibrated after the progressive restoration is completed. If progressive image restoration is not performed, the feature dimensions of the artwork image are recalibrated directly. This includes: Periodic texture detection is performed based on Fourier spectrum analysis. If periodic texture is detected, the corresponding image block is moved along the main direction by one main period length and along the vertical direction by a preset integer number of period lengths within the annular neighborhood image. This position image block is marked as a candidate source block. The craft image is then repaired based on the candidate source blocks. If no periodic texture is detected, the symmetry of the craft image is determined, and the normalized cross-correlation matrix of the annular neighborhood image about the candidate symmetry axis is obtained. The average value of all pixels in the central region of the normalized cross-correlation matrix is ​​taken as the autocorrelation symmetry measure of the current candidate symmetry axis. It is determined whether the autocorrelation symmetry measure is greater than the preset symmetry threshold. If so, the undamaged symmetry region is filled into the sub-block to be filled by thin plate spline interpolation to obtain the filled sub-block. The symmetric structure craft image restoration is performed based on the sub-block to be filled. Otherwise, the restoration is based on the conditional generative adversarial network. The image of the craft is input into a preset multi-scale feature extraction network, which outputs a high-dimensional feature vector of the craft image; the high-dimensional feature vector encodes the craft information such as the shape outline, pattern structure, and color distribution in the craft image. The binary mask of the craft image is downsampled and encoded into a mask feature vector. The high-dimensional feature vector is concatenated with the mask feature vector and input into a preset dimension importance prediction network to obtain the adaptive weight vector of the craft image. The similarity distance between handicraft images is calculated based on the result of recalibration of the feature dimensions of the handicraft images, and the similarity retrieval of handicraft images is performed based on the calculation result.

Images