A printed image retrieval method and system

Abstract

The application discloses a printed image retrieval method and system, and belongs to the technical field of image processing. In order to solve the technical problems of difficult to balance macro layout and micro details, color and structure coupling and blurred boundary of dense patterns in printed image retrieval, gradient energy field is extracted from a query image, and significant area screening is performed, so that global image blocks and a plurality of local image blocks are obtained. A feature extraction model is used to obtain global features and a plurality of local features respectively, and fusion features are generated through pooling, weighted splicing and normalization. Finally, similarity matching is performed with a feature library to obtain a retrieval result. The application realizes effective decoupling of color and structure, significantly improves the retrieval accuracy of images with no boundary dense patterns, cross-scale and non-rigid deformation without additional detection models, and reduces the algorithmic cost of industrial deployment.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for retrieving printed images. Background Technology

[0002] Efficient and accurate similar image retrieval from massive image databases is one of the core tasks in the field of computer vision. Traditional image retrieval methods often rely on single global feature extraction, such as the global average pooling layer features of convolutional neural networks. However, when the retrieval target focuses on fine-grained images such as floral patterns, printed fabrics, and complex CAD designs, existing technologies face the following insurmountable technical bottlenecks.

[0003] It's difficult to balance macro-level layout with micro-level details: Floral patterns have complex, multi-layered structures, including both "micro-level details" such as the outlines of individual flower petals and the direction of branches and leaves, and "macro-level layout" such as two-dimensional and four-dimensional continuous patterns. Existing global pooling networks tend to "blur" dense pattern features, losing edge details; while purely extracting local feature points easily ignores the overall layout and rhythm of the pattern.

[0004] Structural features are heavily coupled with color / style features: In industrial design, the definition of "similar patterns" usually focuses on geometric topology, such as "isomorphic but different colors" design. However, the feature space of existing pre-trained models (such as models trained on ImageNet) often has color, texture and structure highly entangled, making the retrieval system easily interfered with by the "background color" or "dominant color" that occupies the largest pixel area, thus failing to accurately match the "skeleton structure" of the pattern.

[0005] The patterns are densely stacked and lack clear boundaries: Unlike conventional object detection tasks, branches in floral patterns are often intertwined, and flowers and leaves overlap and obscure each other, without clear semantic boundaries (bounding boxes). Directly introducing traditional object detection or instance segmentation models to assist in extracting local features will not only incur huge computational costs, but also lead to chaotic feature extraction due to the large amount of overlap and failure of detection boxes.

[0006] Significant non-rigid topological deformation and scale differences exist: In real-world retrieval scenarios, image library data comes from diverse sources, including planar CAD design drawings, real-life photos of wrinkled fabrics, and webpage thumbnails at different scaling ratios. Image cropping, scaling, and physical deformation of fabrics require feature descriptors to possess extremely strong scale invariance and spatial deformation robustness. Summary of the Invention

[0007] The purpose of this invention is to address the technical problems in print image retrieval, such as the difficulty in balancing macroscopic layout and microscopic details, color and structure coupling, and blurred boundaries of dense patterns. This invention proposes a print image retrieval method and system that extracts local salient regions based on the native gradient energy field and performs weighted fusion with global features to achieve accurate capture of microscopic details and macroscopic layout of complex patterns. This enhances the system's robustness to color interference, non-rigid topological deformation, and scale differences.

[0008] To achieve the above objectives, the present invention adopts the following technical solution.

[0009] A method for retrieving printed images includes the following steps: Gradient extraction and energy smoothing are performed on the query image to obtain the gradient energy field; Based on the gradient energy field, salient regions are filtered in the query image to extract global image blocks and multiple local image blocks; Feature extraction is performed on the global image patch and the multiple local image patches to obtain global features and multiple local features; Pooling is performed on the multiple local features to obtain a comprehensive local feature, and the comprehensive local feature is fused with the global feature to generate a fused feature; The similarity between the fused features and image features in the preset feature library is calculated to obtain the retrieval results of the query image.

[0010] Furthermore, gradient extraction and energy smoothing are performed on the query image to obtain the gradient energy field, including: The query image is converted to grayscale and the gradient magnitudes in the horizontal and vertical directions are calculated to obtain a comprehensive gradient magnitude map. Energy smoothing and normalization mapping are performed on the comprehensive gradient magnitude map to generate a gradient energy field.

[0011] Furthermore, energy smoothing and normalization mapping are performed on the comprehensive gradient magnitude map to generate a gradient energy field, including: The Gaussian kernel is used to smooth and filter the comprehensive gradient magnitude map to eliminate noise on the fabric surface; The filtered gradient data is normalized by performing maximum and minimum value normalization, and the energy is mapped to a specified integer interval to obtain the gradient energy field.

[0012] Furthermore, based on the gradient energy field, salient regions are filtered in the query image to extract global image patches and multiple local image patches, including: A sliding window scan is performed on the gradient energy field, and the average gradient energy of each pixel within the sliding window is calculated as the significance score. Based on the saliency score, each sliding window is deduplicated to determine the coordinates of multiple salient regions; Multiple local image blocks are cropped from the query image according to the coordinates of the salient region, and the complete query image is used as a global image block.

[0013] Furthermore, based on the saliency score, each sliding window is deduplicated to determine the coordinates of multiple salient regions, including: Sort all sliding windows in descending order of significance score; Based on a preset crossover-union ratio threshold, non-maximum suppression is applied to the arranged sliding windows to remove windows with excessive overlap. A preset number of windows that are ranked at the top are selected as the coordinates of the significant region.

[0014] Further, feature extraction is performed on the global image patch and the plurality of local image patches to obtain global features and multiple local features, including: Input the global image patch and multiple local image patches into the preset convolutional neural network backbone; Remove the fully connected classification layer of the backbone network and extract the output feature tensor to obtain global features and multiple local features.

[0015] Furthermore, pooling is performed on the multiple local features to obtain a comprehensive local feature, and the comprehensive local feature is weighted and concatenated with the global feature and normalized to generate a fused feature, including: Max pooling is performed on multiple local features in the region dimension to extract the strongest features of each channel and generate comprehensive local features. Local and global weight coefficients are assigned to the integrated local and global features, respectively. The weighted integrated local features and the weighted global features are concatenated into vectors and their magnitudes are normalized to generate fused features.

[0016] Furthermore, the weighted integrated local features and the weighted global features are concatenated as vectors and their magnitudes are normalized to generate fused features, including: Dynamically adjust local and global weight coefficients based on the specific preferences of the search database; Element-wise multiplication is performed on the feature vectors according to each weight coefficient, and the processed feature vectors are then concatenated. The concatenated high-dimensional vector is normalized using the L2 norm so that it is mapped onto the unit hypersphere.

[0017] Further, the fused features are matched with image features in a preset feature library to obtain retrieval results for the query image, including: The cosine similarity score is obtained by calculating the vector dot product between the fused feature and each feature vector in the preset feature library. The search results are sorted in descending order based on similarity scores, and the top few matching images are returned as the search results for the query image.

[0018] A print image retrieval system, comprising: The energy field construction module is used to extract gradients and smooth energy in the query image to obtain the gradient energy field. The region extraction module is used to perform salient region filtering in the query image based on the gradient energy field, and extract global image blocks and multiple local image blocks; The feature extraction module is used to extract features from the global image patch and the multiple local image patches to obtain global features and multiple local features; The feature fusion module is used to pool the multiple local features to obtain a comprehensive local feature, and then fuse the comprehensive local feature with the global feature to generate a fused feature; The image retrieval module is used to calculate the similarity between the fused features and image features in a preset feature library to obtain the retrieval results of the query image.

[0019] The present invention has achieved the following beneficial effects.

[0020] 1. This invention utilizes gradient extraction and energy smoothing to construct a gradient energy field, stripping color information from the physical layer, achieving a natural decoupling of color and structure, and solving the problem of retrieving isomorphic heterochromatic patterns; at the same time, it smooths out fabric texture noise through large-size Gaussian filtering, ensuring that the algorithm can firmly grasp the core geometric topological skeleton of the pattern design.

[0021] 2. This invention employs a sliding window based on gradient energy field and a non-maximum suppression algorithm, which can accurately locate the core local patch with the highest information entropy without prior training of the detection model. It effectively solves the feature extraction bottleneck of dense patterns without clear boundaries, such as intertwined branches and overlapping petals, and completes the localization in milliseconds, significantly reducing the computational cost and the difficulty of industrial deployment.

[0022] 3. This invention designs a global and local dual-branch feature architecture. It preserves micro-strokes through local feature extraction and channel-level max pooling, and combines them with global features that preserve macro-typesetting information for dynamic weighted fusion. The introduction of weight factors achieves a dynamic balance between micro-comparison and macro-examination, thereby improving the model's ability to identify complex design schemes.

[0023] 3. This invention endows feature descriptors with good spatial translation invariance and scale robustness by performing channel-level max pooling aggregation on multiple local features. It can effectively cope with non-rigid topological deformation caused by fabric wrinkles and scale differences caused by changes in shooting distance and thumbnail cropping, thus ensuring the stability of cross-modal retrieval. Attached Figure Description

[0024] Figure 1 This is a flowchart of the print image retrieval method in the embodiment; Figure 2 This is a flowchart illustrating the feature extraction process of the print image retrieval method in the embodiment. Figure 3 This is a block diagram of the print image retrieval system in the embodiment. Detailed Implementation

[0025] To make the various technical features, advantages, or effects of the present invention more apparent and understandable, detailed descriptions are provided below through embodiments.

[0026] This invention provides a method for retrieving printed images, such as... Figure 1 and Figure 2 As shown, it includes the following steps: Step S1: Perform gradient extraction and energy smoothing on the query image to obtain the gradient energy field.

[0027] Specifically, this step aims to extract the structural features of the image and filter out interference from the fabric itself, thereby obtaining an energy distribution map that can characterize the core pattern of the print.

[0028] In an optional embodiment of the present invention, step S1 may include: Step S11: Convert the query image to grayscale and calculate the gradient magnitude in the horizontal and vertical directions to obtain a comprehensive gradient magnitude map.

[0029] Specifically, the query image can be a color image, such as a real-life fabric photograph in RGB format or a computer-aided design (CAD) drawing. The input color image is converted to a single-channel grayscale image; a Sobel operator with a kernel size of 3×3 is used to calculate the gradient magnitudes of the grayscale image in the horizontal and vertical directions, respectively. and Subsequently, according to the formula Calculate the overall gradient magnitude map.

[0030] Step S12: Perform energy smoothing and normalization mapping on the comprehensive gradient magnitude map to generate a gradient energy field.

[0031] In an optional embodiment of the present invention, step S12 may include: Step S121: Use a Gaussian kernel to smooth and filter the comprehensive gradient magnitude map to eliminate noise on the fabric surface.

[0032] Specifically, in order to eliminate minor defects or noise interference on the fabric surface and retain the core pattern structure, the gradient amplitude map is subjected to Gaussian smoothing filtering, wherein the preferred Gaussian kernel size is 31×31.

[0033] Step S122: Perform maximum and minimum value normalization on the filtered gradient data, and map the energy to a specified integer interval to obtain the gradient energy field.

[0034] Specifically, the filtered gradient energy is mapped to an integer range of 0 to 255 using Min-Max normalization to form the final gradient energy field.

[0035] Step S2: Based on the gradient energy field, perform salient region filtering in the query image to extract global image blocks and multiple local image blocks.

[0036] In an optional embodiment of the present invention, step S2 may include: Step S21: Perform a sliding window scan on the gradient energy field and calculate the average gradient energy of each pixel within the sliding window as a significance score.

[0037] Specifically, the standard size of the local window is set to 224×224, and the sliding step size is set to half the window size, i.e., 112. Dense sliding scans are performed on the gradient energy field. For each sliding window, the average gradient energy of all pixels within the window is calculated as the saliency score of that local region. The higher the score, the denser the lines, edges, or floral structures in that region.

[0038] Step S22: Based on the saliency score, deduplication is performed on each sliding window to determine the coordinates of multiple salient regions.

[0039] In an optional embodiment of the present invention, step S22 may include: Step S221: Sort all sliding windows in descending order according to their significance scores.

[0040] Specifically, initialize an empty target list and sort all windows from highest to lowest salience score.

[0041] Step S222: Non-Maximum Suppression (NMS) is applied to the arranged sliding windows based on a preset crossover-union ratio threshold to remove windows with excessive overlap.

[0042] Specifically, after traversing the sorted windows, if the intersection-union ratio (IoU) between the current window and an existing window in the target list is greater than a preset overlap threshold, the window is removed; otherwise, it is added to the target list. In this embodiment, the preferred threshold is 0.3.

[0043] Step S223: Extract a preset number of windows that are ranked first as the coordinates of the significant region.

[0044] Specifically, the N regions with the highest scores in the target list are extracted. In this embodiment, N is preferably equal to 4.

[0045] Step S23: Crop multiple local image patches (LocalCrops) onto the query image according to the coordinates of the significant regions, and use the complete query image as a global image patch (Global Crop).

[0046] Specifically, N local image patches are cropped from the original RGB format query image using coordinate mapping. If the full image size is insufficient for cropping, the original image is used directly. Meanwhile, the uncropped, complete original image is retained as a global image patch.

[0047] Step S3: Extract features from the global image block and the multiple local image blocks to obtain global features and multiple local features.

[0048] In an optional embodiment of the present invention, step S3 may include: Step S31: Input the global image block and multiple local image blocks into the preset convolutional neural network backbone network.

[0049] Specifically, a pre-trained ConvNeXt V2 model is selected as the backbone network, for example, the convnextv2_base model is selected.

[0050] Step S32: Remove the fully connected classification layer of the backbone network and extract the output feature tensor to obtain global features and multiple local features.

[0051] Specifically, the last classification header is removed by setting the classification category parameter num_classes = 0. The preprocessed N local image patches and 1 global image patch are input into the model, and the corresponding feature tensor is output, which includes both global and local features.

[0052] Step S4: Pool the multiple local features to obtain a comprehensive local feature, and then fuse the comprehensive local feature with the global feature to generate a fused feature.

[0053] In an optional embodiment of the present invention, step S4 may include: Step S41: Perform max pooling on multiple local features in the region dimension to extract the strongest features of each channel and generate comprehensive local features.

[0054] Specifically, for a set of N local feature vectors, a max pooling operation is performed on the 0th dimension, i.e., the region dimension. This operation can extract the channel features with the strongest activation from different local regions, thereby generating a single comprehensive local feature vector with spatial translation invariance.

[0055] Step S42: Assign local weight coefficients and global weight coefficients to the integrated local features and global features, respectively.

[0056] Specifically, local feature weight coefficients are introduced. With global feature weight coefficients .

[0057] Step S43: The weighted integrated local features and the weighted global features are concatenated into vectors and their magnitudes are normalized to generate fused features.

[0058] In an optional embodiment of the present invention, step S43 may include: Step S431: Dynamically adjust the local weight coefficient and the global weight coefficient according to the details of the search database.

[0059] Specifically, the system default settings It equals 1.0. It equals 1.0. In practical applications, it can be adjusted according to the macro or detail-oriented tendencies of different image libraries.

[0060] Step S432: Perform element-wise multiplication on the feature vectors according to each weight coefficient, and then concatenate the processed feature vectors.

[0061] Step S433: Normalize the spliced ​​high-dimensional vector using the second norm so that it is mapped onto the unit hypersphere.

[0062] Specifically, the final fusion feature descriptor lies on the unit hypersphere, and the calculation formula is as follows: in, Represents the fusion feature descriptor; Represents the weight coefficients of local features; Indicates the first Feature vectors of local regions; Indicates the number of local regions; Represents the global feature weight coefficients; Represents the global feature vector; symbol This represents a vector concatenation operation; the denominator represents the calculation of the L2 norm of the concatenated vector.

[0063] Step S5: Calculate the similarity between the fused features and the image features in the preset feature library to obtain the retrieval result of the query image.

[0064] Specifically, this method includes an offline phase and an online retrieval phase. In the offline phase, the above steps are performed on all images in the database to construct a high-dimensional feature vector database.

[0065] In an optional embodiment of the present invention, step S5 may include: Step S51: Calculate the vector point product of the fused feature and each feature vector in the preset feature library to obtain the cosine similarity score.

[0066] Specifically, during the online retrieval phase, the query image is transformed into a fused feature descriptor, which is then multiplied by the vector dot product of the descriptor and each feature vector in the database.

[0067] Step S52: Sort the search results in descending order based on similarity scores, and return the top few matching images as the search results for the query image.

[0068] Specifically, the similarity scores are sorted from highest to lowest, and the top-ranked results are returned. Image results.

[0069] This invention also provides a print image retrieval system for performing the above-described method, such as... Figure 3 As shown, it includes: The energy field construction module is used to extract gradients and smooth energy in the query image to obtain the gradient energy field. The region extraction module is used to perform salient region filtering in the query image based on the gradient energy field, and extract global image blocks and multiple local image blocks; The feature extraction module is used to extract features from the global image patch and the multiple local image patches to obtain global features and multiple local features; The feature fusion module is used to pool the multiple local features to obtain a comprehensive local feature, and then fuse the comprehensive local feature with the global feature to generate a fused feature; The image retrieval module is used to calculate the similarity between the fused features and image features in a preset feature library to obtain the retrieval results of the query image.

[0070] Method performance testing: To verify the effectiveness and practical performance of the method of the present invention, this experiment conducted a performance test on similar image retrieval in a large-scale database containing hundreds of thousands of samples.

[0071] 1. Experimental Dataset and Evaluation Metrics: Dataset size: The test database covers hundreds of thousands of industrial-grade textile patterns, complex CAD design originals, and real-life images of fabrics in various scenarios. The samples have rich texture features and scale differences. Evaluation metrics: Recall@10 (top 10 hit rate) and mAP@10 (mean precision) are used as the recognized quantitative evaluation standards in the retrieval field to comprehensively measure the accuracy and ranking performance of retrieval results.

[0072] 2. Compare with the baseline model: To objectively demonstrate the technical advantages of this invention, the following three representative image retrieval schemes were selected as comparative baselines in the experiment: ResNet-50: It adopts the traditional image retrieval scheme based on global average pooling (GAP) and only extracts the global low-frequency information of the image; ConvNeXt V2: Only the ConvNeXt V2 backbone network is used to extract global features, without introducing the local salient region extraction and multi-scale weighted fusion strategy described in this invention; DINOv2: Features are extracted using the DINOv2 large visual model based on self-supervised pre-training. This model has strong feature representation capabilities in instance-level retrieval tasks.

[0073] 3. Results Analysis: The specific test data comparison is shown in Table 1 below.

[0074] Table 1 Comparison of Test Data As can be seen from the test results in Table 1, the method of the present invention is significantly better than the above comparative model in all evaluation indicators, which proves the superior performance of the method of the present invention in solving the multi-scale feature fusion and structural detail capture of printed patterns.

[0075] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention should be covered within the protection scope of the present invention, which is defined by the claims.

Claims

1. A printed image retrieval method characterized by, Includes the following steps: Gradient extraction and energy smoothing are performed on the query image to obtain the gradient energy field; Based on the gradient energy field, salient regions are filtered in the query image to extract global image blocks and multiple local image blocks; Feature extraction is performed on the global image patch and the multiple local image patches to obtain global features and multiple local features; Pooling is performed on the multiple local features to obtain a comprehensive local feature, and the comprehensive local feature is fused with the global feature to generate a fused feature; The similarity between the fused features and image features in the preset feature library is calculated to obtain the retrieval results of the query image.

2. The method of claim 1, wherein, Gradient extraction and energy smoothing are performed on the query image to obtain the gradient energy field, including: The query image is converted to grayscale and the gradient magnitudes in the horizontal and vertical directions are calculated to obtain a comprehensive gradient magnitude map. Energy smoothing and normalization mapping are performed on the comprehensive gradient magnitude map to generate a gradient energy field.

3. The method as described in claim 2, characterized in that, The comprehensive gradient magnitude map is subjected to energy smoothing and normalization mapping to generate a gradient energy field, including: The Gaussian kernel is used to smooth and filter the comprehensive gradient magnitude map to eliminate noise on the fabric surface; The filtered gradient data is normalized by performing maximum and minimum value normalization, and the energy is mapped to a specified integer interval to obtain the gradient energy field.

4. The method as described in claim 1, characterized in that, Based on the gradient energy field, salient regions are filtered in the query image to extract global image patches and multiple local image patches, including: A sliding window scan is performed on the gradient energy field, and the average gradient energy of each pixel within the sliding window is calculated as the significance score. Based on the saliency score, each sliding window is deduplicated to determine the coordinates of multiple salient regions; Multiple local image blocks are cropped from the query image according to the coordinates of the salient region, and the complete query image is used as a global image block.

5. The method as described in claim 4, characterized in that, Based on the saliency score, each sliding window is deduplicated to determine the coordinates of multiple salient regions, including: Sort all sliding windows in descending order of significance score; Based on a preset crossover-union ratio threshold, non-maximum suppression is applied to the arranged sliding windows to remove windows with excessive overlap. A preset number of windows that are ranked at the top are selected as the coordinates of the significant region.

6. The method as described in claim 1, characterized in that, Feature extraction is performed on the global image patch and the plurality of local image patches to obtain global features and multiple local features, including: Input the global image patch and multiple local image patches into the preset convolutional neural network backbone; Remove the fully connected classification layer of the backbone network and extract the output feature tensor to obtain global features and multiple local features.

7. The method as described in claim 1, characterized in that, Pooling the multiple local features yields a composite local feature, which is then fused with the global feature to generate a fused feature, including: Max pooling is performed on multiple local features in the region dimension to extract the strongest features of each channel and generate comprehensive local features. Local and global weight coefficients are assigned to the integrated local and global features, respectively. The weighted integrated local features and the weighted global features are concatenated into vectors and their magnitudes are normalized to generate fused features.

8. The method as described in claim 7, characterized in that, The weighted local features and weighted global features are concatenated into vectors and their magnitudes are normalized to generate fused features, including: Dynamically adjust local and global weight coefficients based on the specific preferences of the search database; Element-wise multiplication is performed on the feature vectors according to each weight coefficient, and the processed feature vectors are then concatenated. The concatenated high-dimensional vector is normalized using the L2 norm so that it is mapped onto the unit hypersphere.

9. The method as described in claim 1, characterized in that, The fused features are matched with image features in a preset feature library based on similarity to obtain the retrieval results for the query image, including: The cosine similarity score is obtained by calculating the vector dot product between the fused feature and each feature vector in the preset feature library. The search results are sorted in descending order based on similarity scores, and the top few matching images are returned as the search results for the query image.

10. A print image retrieval system, characterized in that, include: The energy field construction module is used to extract gradients and smooth energy in the query image to obtain the gradient energy field. The region extraction module is used to perform salient region filtering in the query image based on the gradient energy field, and extract global image blocks and multiple local image blocks; The feature extraction module is used to extract features from the global image patch and the multiple local image patches to obtain global features and multiple local features; The feature fusion module is used to pool the multiple local features to obtain a comprehensive local feature, and then fuse the comprehensive local feature with the global feature to generate a fused feature; The image retrieval module is used to calculate the similarity between the fused features and image features in a preset feature library to obtain the retrieval results of the query image.

Images