National fabric pattern image retrieval method and system based on deep learning and storage medium
Through a deep learning-based method, the feature extraction and hashing coding of ethnic fabric pattern images is used to use VIT network and deep hash modules to solve the problem of low accuracy in ethnic fabric pattern images in the prior art, and the pixel level is achieved Retrieval promotes efficient identification and interpretation of cultural content.
Patent Information
- Application Number
- CN202411108093.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-26
AI Technical Summary
The existing national fabric pattern image retrieval methods are difficult to achieve high-accurate pixel-level retrieval, especially in the interpretation and inheritance of rich cultural content in the era of cultural big data.
Using a deep learning-based method, image feature extraction and hashing coding is performed using a vision transformer (VIT) network and a depth hash module to calculate the mixing distance of the image, and achieve pixel-level pixel-level Ethnic fabric pattern image search.
It improves the accuracy of national fabric pattern image retrieval, provides an effective digital image processing method, is suitable for application promotion, and significantly enhances the ability to identify and interpret cultural content.
Smart Images

Figure CN119025698A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method for retrieving images of ethnic fabric patterns, and more specifically, to a method, system and storage medium for retrieving images of ethnic fabric patterns based on deep learning. Background Art
[0002] As an important part of Chinese traditional culture, ethnic costumes contain rich cultural symbols and cultural connotations. With the advent of the era of cultural big data, ethnic fabric patterns have also formed rich digital resources. Integrating technology and culture to explore the cultural content contained in ethnic fabric patterns is a widely mentioned topic. Using image retrieval methods to extract and interpret the content of ethnic fabric patterns is a scientific and technological method that can achieve cultural identification, cultural interpretation, and cultural inheritance. Summary of the invention
[0003] The present invention provides a method, system and storage medium for retrieval of ethnic fabric pattern images based on deep learning, so as to realize retrieval of ethnic fabric pattern images with retrieval accuracy meeting the requirements.
[0004] To this end, the present invention adopts the following technical solutions:
[0005] On the one hand, the present invention provides a method for retrieving ethnic fabric pattern images based on deep learning, comprising:
[0006] Collect pictures containing ethnic fabric patterns to form an ethnic fabric pattern dataset;
[0007] Annotate various types of ethnic fabric patterns appearing in the images in the dataset;
[0008] Generate a segmentation map corresponding to each labeled image;
[0009] Preprocess each image;
[0010] A retrieval model based on deep learning is constructed; and the retrieval model is trained using the preprocessed data set; the retrieval model includes a visual transformer VIT network module and a deep hash module, the VIT network performs a block operation on each input image, and performs feature extraction, fuses the acquired feature maps of different scales, and gradually restores the image to the input resolution size; and the fused features are input into the deep hash module, the deep hash module hashes the image, and calculates and compares the mixed distance of the image hash code, and determines the first n images similar to the input image, that is, the retrieval result image of the ethnic fabric pattern;
[0011] Retrieve ethnic fabric pattern images based on the trained retrieval model to achieve pixel-level retrieval of ethnic fabric pattern images.
[0012] Furthermore, each image is preprocessed, including:
[0013] Randomly adjust the brightness and contrast of the image;
[0014] The Gaussian blur method is used to reduce the details of the image, thereby achieving image smoothing and blurring;
[0015] Data augmentation is performed on the input images using random horizontal flips, random vertical flips, and randomly applied translation, scaling, and rotation transformations.
[0016] Furthermore, the VIT network divides each input image into blocks and performs feature extraction, including:
[0017] Assume that the VIT encoder is composed of L stacked transformer modules, and the transformer input T j is the hidden state h j-1 , which is the previous transformer module T j-1 The output of, where j = 1, 2, ... L, h j-1 ∈R N+1,de , the input of the first transformer block T1 is the final projected embedding; the last transformer block T L is the hidden state h L , the input and output of the jth transformer block are represented by h j-1 ,h j Represents; the input of the normalization layer is h j-1 , the output is l j-1 ∈R N +1,de , is the first layer of the transformer block; the output of the normalization layer comes from the input of the self-attention layer, which has three parameter projections, Q, K, and V, using the learnable weight matrix W Q ,W K ,W v , perform projection;
[0018] Input an image of size h*w*c, divide the image into small blocks of p*p*c, then a total of n image blocks can be obtained [n=(h*w) / (p*p)], and add a learnable category block, then a total of (n+1) blocks are to be processed;
[0019] Perform a flatten operation on the image blocks, that is, convert each image block (p*p*c) into a one-dimensional vector of size 1*(p*p*c), and then concatenate n one-dimensional vectors to form a two-dimensional vector of size n*(p*p*c);
[0020] Then use the fully connected layer to reduce the dimension of the two-dimensional vector to obtain the n*d two-dimensional feature a;
[0021] Add position encoding to the input features; position encoding is used to mark the relative position of each image block;
[0022] The preprocessed features are sent to the transformer encoder to obtain the interactive feature f; the most important thing in the encoder is the multi-head attention layer. For the input feature a(n*d), it is divided into m heads to obtain m different features (a1, a2...am); self-attention calculation is performed on the m features to obtain m weighted features (b1,..., b12), which are then concatenated together to obtain z(n*d), and through the nonlinear change w, the interactive feature f(n*d) of the same size as the input feature is finally obtained.
[0023] Furthermore, the deep hash module includes:
[0024] Train a transformer using image pairs and corresponding similarity labels;
[0025] The loss function is designed to learn similar class binary image representations, and then the transformer output is quantized to generate binary codes for new images.
[0026] Furthermore, the hybrid distance of the image hash code is calculated, including:
[0027] If the images are similar, the image distance is minimized; if the images are dissimilar, the image distance is maximized.
[0028] Furthermore, a loss function is designed to learn to maintain similar class binary image representations, and then the transformer output is quantized to generate binary codes for new images, including:
[0029] Let Ω be the RGB space, the goal is to learn from Ω to k-bit binary code, and pull the binary codes of similar images together and push the binary codes of dissimilar images apart; for a pair of images I1, I2∈Ω, the corresponding binary network outputs b1, b2∈{+1, -1} k , if the images are similar, define y = 0, otherwise y = 1;
[0030] The loss for the image is:
[0031]
[0032] Where D hrepresents the distance between two binary vectors, m>0 is the boundary threshold parameter; ‖·‖1 is the L1 norm of the vector, |·| is the absolute value operation of the element, and α is a hyperparameter that controls the strength of regularization.
[0033] In another aspect, the present invention further provides a national fabric pattern image retrieval system based on deep learning, comprising:
[0034] A data collection module collects pictures containing ethnic fabric patterns to form an ethnic fabric pattern data set;
[0035] A labeling module for labeling various types of ethnic fabric patterns appearing in the images in the data set;
[0036] Segmentation module, generates segmentation maps corresponding to each labeled image;
[0037] Preprocessing module, preprocessing each image;
[0038] A model building module is used to build a retrieval model based on deep learning; and the retrieval model is trained using the preprocessed data set; the retrieval model includes a visual transformer VIT network module and a deep hash module, the VIT network performs a block operation on each input image, and performs feature extraction, fuses the acquired feature maps of different scales, and gradually restores the image to the input resolution size; and the fused features are input into the deep hash module, the deep hash module hashes the image, and calculates and compares the mixed distance of the image hash code, and determines the first n images similar to the input image, that is, the retrieval result image of the ethnic fabric pattern;
[0039] The image retrieval module retrieves ethnic fabric pattern images based on the trained retrieval model, realizing pixel-level retrieval of ethnic fabric pattern images.
[0040] On the other hand, the present invention also provides a computer-readable storage medium, which stores a computer instruction set. When the computer instruction set is executed by a processor, the above-mentioned ethnic fabric pattern image retrieval method based on deep learning is implemented.
[0041] The beneficial effects of the present invention are as follows: the ethnic fabric pattern retrieval method of the present invention first marks the ethnic fabric patterns, marks the various patterns in the image respectively, and then generates a corresponding retrieval graph for training the model according to the marked image. After the image is preprocessed, the vision transformer (Vision Transformer, referred to as VIT, i.e., using the transformer (Transformer) architecture to process the vision (Vision) task) network extracts features from each image, and the deep hash module performs image distance optimization and hash coding. After training, a retrieval model with a retrieval accuracy rate that meets the requirements is obtained, which provides an effective digital image processing method for the fields of ethnic pattern image retrieval, has significant beneficial effects, and is suitable for application and promotion. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0043] Figure 1 This is a flowchart of an ethnic fabric pattern image retrieval in an embodiment of the present invention;
[0044] Figure 2 A retrieval module diagram in an embodiment of the present invention;
[0045] Figure 3 Schematic diagram of a VIT network model in an embodiment of the present invention. DETAILED DESCRIPTION
[0046] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0047] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0048] like Figure 1 As shown, a flowchart of the ethnic fabric pattern image retrieval of the present invention is given. First, the ethnic fabric pattern image retrieval method of the present invention is described in a modular form. It consists of an image preprocessing module, a VIT module, a deep hash module, and a hybrid distance calculation mode. The image preprocessing performs image enhancement processing on the input ethnic fabric pattern image as the input of the VIT network module; the VIT network module first performs a patch operation on each input image. During the whole process, the extracted features are input into the deep hash module. The deep hash module hashes the image, calculates and compares the image distance, outputs the optimal result, and finally obtains the retrieval result diagram of the ethnic fabric pattern.
[0049] The ethnic pattern image retrieval method based on deep learning of the present invention realizes the pixel-level ethnic fabric pattern image retrieval, which is specifically achieved through the following steps:
[0050] a). Collect pictures of ethnic costumes. Use cameras, mobile phones and other devices to take pictures of various ethnic fabric patterns, collect pictures from the Internet, and form a dataset of ethnic fabric patterns.
[0051] b) Image annotation: on the ethnic fabric pattern dataset obtained in step a), circle various types of ethnic fabric patterns appearing in the image and add corresponding type labels to the circled areas;
[0052] c). Use image retrieval-related toolkits to generate retrieval graphs corresponding to each image in the labeled ethnic fabric pattern dataset for subsequent model training;
[0053] d). Image preprocessing: In order to enrich the background of the picture and increase the diversity of the scene, each picture is subjected to random brightness and contrast adjustment, Gaussian blur, color gamut change and other operations;
[0054] In specific implementation, Figure 2As shown, a flowchart of the image preprocessing mechanism in the present invention is given, which is specifically implemented by the following steps:
[0055] Assume that the pixel value of the original image is I(x,y), where (x,y) is the position of the pixel. Brightness adjustment is achieved by weighted addition of the brightness value of each pixel. Specifically, the new brightness value I'(x,y) of each pixel can be calculated by the following formula:
[0056] I′(x,y)=I(x,y)+ΔB(1)
[0057] Where ΔB is a randomly generated brightness increment, which can be positive or negative. The size of this increment is limited by the range set by the user.
[0058] Contrast adjustment is achieved by scaling the brightness difference of each pixel. Specifically, the new brightness value I'(x,y) of each pixel can be calculated by the following formula:
[0059] I′(x,y)=α*(I(x,y)-μ)+μ(2)
[0060] Where α is a randomly generated contrast multiplier and μ is the average brightness of the image. The value of α is limited by the contrast range set by the user.
[0061] In summary, this method randomly generates brightness increment ΔB and contrast multiplier α, and then applies them to each pixel of the image to adjust the brightness and contrast. This can produce diverse images and help the model better generalize to input data under different brightness and contrast conditions.
[0062] Then the Gaussian blur method is also used in the preprocessing process, which reduces the details of the image by applying a Gaussian filter to achieve image smoothing and blurring. The following is the specific implementation process of this method:
[0063] Also assume that the pixel value of the original image is I(x,y), where (x,y) is the position of the pixel. The kernel of the Gaussian filter is a two-dimensional Gaussian function, usually represented by a matrix. The size of the Gaussian kernel is determined by the size of the filter, usually an odd number, such as 3x3, 5x5, 7x7, etc. The center value of the kernel is the largest, and then gradually decreases to weight the values of the surrounding pixels when filtering. The one-dimensional form of the Gaussian kernel is as follows:
[0064]
[0065] Where G(x) is a Gaussian function, σ 2 is the standard deviation and x is the offset from the center of the kernel.
[0066] The two-dimensional form of the Gaussian kernel is the superposition of the one-dimensional kernel in two directions, as shown below:
[0067] G(x,y)=G(x)*G(y)(4)
[0068] From the Gaussian convolution function G(x,y), we can see that the scale factor σ determines the range of the convolution kernel. When the value of σ is smaller, about 2% to 6% of the image size, the local details of the image are more prominent, but the color is easily distorted, and the grayscale dynamic range will be compressed more. When the value of σ is larger, about 30% to 60% of the image size, when calculating the pixel point, the surrounding pixel points have a larger influence on the calculation point, that is, the convolution kernel has a greater effect, the color can be maintained, the overall image processing effect is better, and the local details are not easy to stand out.
[0069] For each image pixel (x, y), the Gaussian kernel is convolved with the local area of the image to obtain a new pixel value I'(x, y). The convolution operation can be expressed as follows:
[0070] I′(x,y)=∑I(x+i,y+j)*G(i,j)(5)
[0071] Where Σ represents the summation and (i,j) is the coordinate of the Gaussian kernel.
[0072] In the boundary regions of the image, since there are not enough pixels to perform a complete convolution, different boundary handling strategies are usually used, such as padding or truncation, to ensure that the output image has the same size as the input image.
[0073] In summary, the Gaussian blur method smoothes the image, reduces details, and blurs the image by applying a Gaussian filter kernel. The size and standard deviation of the filter kernel determine the degree of blurring. The larger the standard deviation, the more obvious the blurring. This operation is often used to remove noise from an image or reduce image details for preprocessing of specific tasks.
[0074] In the final preprocessing process, other methods such as random horizontal flipping, random vertical flipping, random translation, scaling and rotation transformations were used to enhance the input images.
[0075] e). Feature extraction of transformer network. The input layer of the network extracts the features of the image through the convolution kernel. The extracted features form a feature map. As the depth of the network increases, the size of the feature map gradually decreases and the depth gradually deepens.
[0076] Assume that the VIT encoder is composed of L stacked transformer modules, and the module drops the given hidden state input into the same dimension output under the self-attention mechanism. j is the hidden state hj-1 , which is the previous transformer module T j-1 The output of, where j = 1, 2, ... L, h j-1 ∈R N+1,de , the input of the first transformer block T1 is the final projected embedding. The last transformer block T L is the hidden state h L , the input and output of the jth transformer block are represented by h j-1 ,h j express.
[0077] The input of the normalization layer is h j-1 , the output is l j-1 ∈R N+1,de , is the first layer of the transformer block. The output of the normalization layer comes from the input of the self-attention layer. The self-attention layer has three parameter projections, Q, K, and V, which use the learnable weight matrix W respectively. Q ,W K ,W V , for projection.
[0078] The images are input into the network in the order of data preprocessing. The network consists of the basic transformer encoder module. Each image is divided into 3*3 blocks, and the representation from input to block is as follows:
[0079] I∈R m,m,c →I∈R i k,k,c ;
[0080] where i=1,2,...,N,m 2 =Nk 2 ,c=3;
[0081] I i is flattened into a vector V i ∈R 1, , where d = c·k 2 , linearly project each flattened vector, and the generated embedding is:
[0082] PE i =V i ×W PE ;
[0083] Where V i ∈R 1,d , is the flattened vector corresponding to the i-th block, and the parameter matrix W PE ∈R d,de , PE i ∈R1,de is the projection embedding corresponding to the i-th block, de is the embedding dimension of the projection; the projection embedding of the entire input image is PE∈R N,de , introduce class token embeddingCT∈R 1,de , and initialize it to 0, concatenate the projected embedding to get the expanded embedding (EE∈R) in the first dimension N+1,de ) is EE = [CT, PE];
[0084] Position embedding (PoE∈R N+1,de ) is also initialized to 0 and added to the extended embedding as a learnable parameter;
[0085] The final projection embedding is:
[0086] FE = dropout (EE + PoE, 0.1);
[0087] The dropout factor is dropout = 0.1, FE∈R N+1,de It is the projection embedding that will eventually be used as the input of the VIT encoder.
[0088] like Figure 3 As shown, a schematic diagram of the VIT network of the present invention is given, which is implemented by the following steps:
[0089] Input an image of size h*w*c, divide the image into p*p*c small blocks, then a total of n image blocks can be obtained [n=(h*w) / (p*p)], and add a learnable category block, then there are a total of (n+1) blocks to be processed. This category block will be used to interact with all the image blocks, and finally learn the features used for classification from the category block; flatten the image blocks, that is, convert each image block (p*p*c) into a one-dimensional vector of size 1*(p*p*c), and then concatenate the n one-dimensional vectors to form a two-dimensional vector of n*(p*p*c). Then use the fully connected layer to reduce the dimension of the two-dimensional vector to obtain the n*d two-dimensional feature a (d is set manually, 1024 or 2048 is acceptable). So far, the linear projection of flattened patches has been completed. Add position encoding to the input features. Position encoding is used to indicate the relative position of each image block. The preprocessed features are sent to the transformer encoder to obtain the interactive feature f. The most important thing in the encoder is the multi-head attention layer. For the input feature a(n*d), it is divided into m heads to obtain m different features (a1, a2...am). For example, if there are 12 heads, then for a certain input feature a(n*d), 12 different features (a1, a2...a12) will be divided, with a size of n*(d / 12). Then, self-attention calculation is performed on these 12 features to obtain 12 weighted features (b1,...,b12), which are then concatenated together to obtain z(n*d). Through nonlinear changes w, the interaction feature f(n*d) of the same size as the input feature is finally obtained.
[0090] f). Calculation of mixed distance: if the images are similar, the image distance is minimized; if the images are dissimilar, the image distance is maximized;
[0091] g). Deep hash binarization image. The feature maps of different scales obtained in steps e) and f) are fused at this stage to gradually restore the image to the input resolution size; and the image is binarized to generate a hash code for the corresponding image;
[0092] In the specific implementation, in step f) and step g), the module used is a deep hashing module. First, the transformer is trained using image pairs and corresponding similarity labels, and a loss function is designed to learn to maintain similar class binary image representations. Then, the transformer output is quantized to generate a binary code for the new image.
[0093] Let Ω be the RGB space, the goal is to learn from Ω to k-bit binary code, and pull the binary codes of similar images together and push the binary codes of dissimilar images apart. For a pair of images I1, I2∈Ω, the corresponding binary network outputs b1, b2∈{+1, -1} k , if the images are similar, define y = 0 otherwise y = 1. The loss of the image is defined as:
[0094]
[0095] Where D h represents the distance between two binary vectors, and m>0 is the boundary threshold parameter. Our goal is to minimize the overall loss function. When optimizing formula (10), due to memory problems and in order to make the actual network output closer to the required discrete values (-1 and +1), it is necessary to add regularization to this formula to replace the binary constraint.
[0096] Then the formula after adding regularization is:
[0097]
[0098] Where ‖·‖1 is the L1 norm of the vector, |·| is the absolute value operation of the element, and α is a hyperparameter that controls the strength of the regularization.
[0099] The hybrid distance that is finally used to optimize the hash code combines the advantages of both methods and uses the hash code to finally determine the top n images that are similar to the input image.
[0100] h). Model building and tuning. Steps e), f) and g) are the process of continuous iterative tuning. When building the model, the loss value and the number of iterations of the model are set. The model is built using the image data in the training set, and the overall parameters are continuously adjusted using the image data in the validation set until the loss value and the number of iterations of the model reach the set threshold.
[0101] In step h), the loss value and number of iterations of the model are set when the model is established. When the model loss value is greater than the set loss threshold or does not reach the set number of iterations threshold, the parameters in the formula are adjusted to make the model converge.
[0102] i). Accuracy calculation: After the model training is completed, the accuracy of each category in the test set is calculated, and finally the final model accuracy is obtained by calculating the average accuracy.
[0103] In the above embodiment, the ethnic fabric patterns are first annotated, and the various patterns in the image are annotated separately, and then the corresponding retrieval graph is generated according to the annotated image for training the model. After the image is preprocessed, the Vision Transformer network extracts features from each image, and the deep hash module performs image distance optimization and hash coding. After training, a retrieval model with a retrieval accuracy that meets the requirements is obtained, which provides an effective digital image processing method for ethnic pattern image retrieval and other fields, with significant beneficial effects and suitable for application promotion.
[0104] Corresponding to the ethnic fabric pattern image retrieval method based on deep learning in the above embodiment, the present invention also provides an ethnic fabric pattern image retrieval system based on deep learning, comprising:
[0105] A data collection module collects pictures containing ethnic fabric patterns to form an ethnic fabric pattern data set;
[0106] A labeling module for labeling various types of ethnic fabric patterns appearing in the images in the data set;
[0107] Segmentation module, generates segmentation maps corresponding to each labeled image;
[0108] Preprocessing module, preprocessing each image;
[0109] A model building module is used to build a retrieval model based on deep learning; and the retrieval model is trained using the preprocessed data set; the retrieval model includes a visual transformer VIT network module and a deep hash module, the VIT network performs a block operation on each input image, and performs feature extraction, fuses the acquired feature maps of different scales, and gradually restores the image to the input resolution size; and the fused features are input into the deep hash module, the deep hash module hashes the image, and calculates and compares the mixed distance of the image hash code, and determines the first n images similar to the input image, that is, the retrieval result image of the ethnic fabric pattern;
[0110] The image retrieval module retrieves ethnic fabric pattern images based on the trained retrieval model, realizing pixel-level retrieval of ethnic fabric pattern images.
[0111] For a national fabric pattern image retrieval system based on deep learning in an embodiment of the present invention, since it corresponds to a national fabric pattern image retrieval method based on deep learning in the above embodiment, the description is relatively simple. For relevant similarities, please refer to the description of a national fabric pattern image retrieval method based on deep learning in the above embodiment, which will not be described in detail here.
[0112] An embodiment of the present invention further discloses a computer-readable storage medium, which stores a computer instruction set. When the computer instruction set is executed by a processor, a method for retrieving ethnic fabric pattern images based on deep learning as provided in any of the above embodiments is implemented.
[0113] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0114] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0115] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0116] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A deep learning-based ethnic fabric pattern image retrieval method, It is characterized in that include: Collect pictures containing ethnic fabric patterns to form an ethnic fabric pattern dataset; Annotate various types of ethnic fabric patterns appearing in the images in the dataset; Generate a segmentation map corresponding to each labeled image; Preprocess each image; A retrieval model based on deep learning is constructed; and the retrieval model is trained using the preprocessed data set; the retrieval model includes a visual transformer VIT network module and a deep hash module, the VIT network performs a block operation on each input image, and performs feature extraction, fuses the acquired feature maps of different scales, and gradually restores the image to the input resolution size; and the fused features are input into the deep hash module, the deep hash module hashes the image, and calculates and compares the mixed distance of the image hash code, and determines the first n images similar to the input image, that is, the retrieval result image of the ethnic fabric pattern; Retrieve ethnic fabric pattern images based on the trained retrieval model to achieve pixel-level retrieval of ethnic fabric pattern images.
2. According to the method for retrieving ethnic fabric patterns based on deep learning in claim 1, It is characterized in that Preprocess each image, including: Randomly adjust the brightness and contrast of the image; The Gaussian blur method is used to reduce the details of the image, thereby achieving image smoothing and blurring; Data augmentation is performed on the input images using random horizontal flips, random vertical flips, and randomly applied translation, scaling, and rotation transformations.
3. According to the deep learning-based ethnic fabric pattern image retrieval method of claim 2, It is characterized in that The VIT network divides each input image into blocks and performs feature extraction, including: Assume that the VIT encoder is composed of L stacked transformer modules, and the transformer input T j is the hidden state h j-1 , which is the previous transformer module T j-1 The output of, where j = 1, 2, ... L, h j-1 ∈R N+1,de , the first transformer block T 1 The input of is the final projected embedding; the last transformer block T L is the hidden state h L , the input and output of the jth transformer block are represented by h j-1 ,h j Represents; the input of the normalization layer is h j-1 , the output is l j-1 ∈R N +1,de , is the first layer of the transformer block; the output of the normalization layer comes from the input of the self-attention layer, which has three parameter projections, Q, K, and V, using the learnable weight matrix W Q ,W K ,W V , for projection; Input an image of size h*w*c, divide the image into small blocks of p*p*c, then a total of n image blocks can be obtained [n=(h*w) / (p*p)], and add a learnable category block, then a total of (n+1) blocks are to be processed; Perform a flatten operation on the image blocks, that is, convert each image block (p*p*c) into a one-dimensional vector of size 1*(p*p*c), and then concatenate n one-dimensional vectors to form a two-dimensional vector of size n*(p*p*c); Then use the fully connected layer to reduce the dimension of the two-dimensional vector to obtain the n*d two-dimensional feature a; Add position encoding to the input features; position encoding is used to mark the relative position of each image block; The preprocessed features are sent to the transformer encoder to obtain the interactive feature f; the most important thing in the encoder is the multi-head attention layer. For the input feature a(n*d), it is divided into m heads to obtain m different features (a1, a2...am); self-attention calculation is performed on the m features to obtain m weighted features (b1,..., b12), which are then concatenated together to obtain z(n*d), and through the nonlinear change w, the interactive feature f(n*d) of the same size as the input feature is finally obtained.
4. The ethnic fabric pattern image retrieval method based on deep learning according to claim 1, It is characterized in that The deep hash module includes: Train a transformer using image pairs and corresponding similarity labels; The loss function is designed to learn similar class binary image representations, and then the transformer output is quantized to generate binary codes for new images.
5. According to the method for retrieving ethnic fabric patterns based on deep learning in claim 4, It is characterized in that Calculates the hybrid distance of the image hash code, including: If the images are similar, the image distance is minimized; if the images are dissimilar, the image distance is maximized.
6. The ethnic fabric pattern image retrieval method based on deep learning according to claim 4, It is characterized in that The loss function is designed to learn to keep similar class binary image representations, and then the transformer output is quantized to generate binary codes for new images, including: Let Ω be the RGB space, the goal is to learn from Ω to k-bit binary codes, and pull the binary codes of similar images together and push the binary codes of dissimilar images apart; for a pair of images I 1 , I 2 ∈Ω, the corresponding binary network output b 1 ,b 2 ∈{+1,-1} k , if the images are similar, define y = 0, otherwise y = 1; The loss for the image is: Where D h Represents the distance between two binary vectors, m>0 is the boundary threshold parameter; ‖·‖ 1 is the L1 norm of the vector, |·| is the absolute value operation of the element, and α is a hyperparameter that controls the strength of the regularization.
7. A deep learning-based ethnic fabric pattern image retrieval system, It is characterized in that include: A data collection module collects pictures containing ethnic fabric patterns to form an ethnic fabric pattern data set; A labeling module for labeling various types of ethnic fabric patterns appearing in the images in the data set; Segmentation module, generates segmentation maps corresponding to each labeled image; Preprocessing module, preprocessing each image; A model building module is used to build a retrieval model based on deep learning; and the retrieval model is trained using the preprocessed data set; the retrieval model includes a visual transformer VIT network module and a deep hash module, the VIT network performs a block operation on each input image, and performs feature extraction, fuses the acquired feature maps of different scales, and gradually restores the image to the input resolution size; and the fused features are input into the deep hash module, the deep hash module hashes the image, and calculates and compares the mixed distance of the image hash code, and determines the first n images similar to the input image, that is, the retrieval result image of the ethnic fabric pattern; The image retrieval module retrieves ethnic fabric pattern images based on the trained retrieval model, realizing pixel-level retrieval of ethnic fabric pattern images.
8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer instruction set, and when the computer instruction set is executed by the processor, a method for retrieving ethnic fabric pattern images based on deep learning as described in any one of claims 1 to 6 is implemented.
