Prior enhancement-based multi-modal vamp material image retrieval method and device

Abstract

The application provides a priori enhancement-based multi-modal vamp material image retrieval method and device, relates to the technical field of image retrieval, and through priori enhancement processing containing periodic peak detection and low-rank reconstruction, introduces the physical priori information of the vamp material, explicitly corrects the position dislocation and shooting variation of the image, and the multi-modal fusion features extracted by the pre-tuned multi-modal pre-training model can improve the discrimination accuracy of the features, realize high-robustness and fine-grained retrieval of industrial vamp materials, reduce the research and development cost of manual material selection, improve the research and development efficiency, provide a quick reference of similar vamp materials for a footwear designer, assist the footwear designer in finding a matched existing design in a large amount of materials, and reduce the time cost of repeated research.

Description

Technical Field

[0001] This invention relates to the field of image retrieval technology, and in particular to a method and apparatus for retrieving images of multimodal shoe upper materials based on prior enhancement. Background Technology

[0002] With the rapid development of computer vision technology, image retrieval technology has evolved from traditional manual feature extraction methods such as Scale-Invariant Feature Transform (SIFT) and Histogram of Oriented Gradients (HOG) to feature representation based on deep convolutional neural networks, and in recent years, multimodal pre-trained models based on large-scale image-text contrastive learning. In the fashion industry, such as textiles and footwear, fine-grained image retrieval is of great significance for design assistance, material traceability, and supply chain management.

[0003] However, when performing similarity searches on images of shoe upper materials in the footwear industry, general multimodal pre-trained models are mainly based on natural or consumer-grade images. Industrial-grade shoe upper materials, however, exhibit significant local texture repetition, low-rank statistical characteristics, and extremely subtle physical material differences. Therefore, the ability of general multimodal pre-trained models to represent and distinguish the material details of shoe upper materials is limited. Furthermore, due to factors such as shooting environment and positional misalignment, using existing methods for image retrieval reduces the robustness of the retrieval results. Summary of the Invention

[0004] This invention provides a method and apparatus for retrieving multimodal shoe upper material images based on prior enhancement, in order to address the deficiencies in related technologies.

[0005] This invention provides a multimodal shoe upper material image retrieval method based on prior enhancement, comprising: Obtain the original image of the shoe upper material to be retrieved; The original image of the shoe upper material is subjected to prior enhancement processing to obtain a standardized enhanced image; The standardized enhanced image is input into a pre-tuned multimodal pre-trained model to extract multimodal fusion features; Based on the multimodal fusion features, one or more target shoe upper material images that match the original shoe upper material image are retrieved from the shoe upper material image library; The prior enhancement process includes: An overlapping block strategy is used to split the original image of the shoe upper material into multiple block images; A two-dimensional fast Fourier transform is performed on each image block to extract the amplitude spectrum to detect periodic spikes. The periodic spikes are then rotated, aligned, and normalized to obtain the processing result. The processing result is subjected to singular value decomposition, and a preset number of singular values ​​are retained for low-rank reconstruction to obtain a reconstructed block image. The reconstructed image blocks are weighted and stitched together to synthesize the standardized enhanced image.

[0006] According to the present invention, a multimodal shoe upper material image retrieval method based on prior enhancement is provided, wherein each candidate image of shoe upper material in the shoe upper material image library corresponds to a candidate feature vector with different attribute dimensions; The step of retrieving one or more target shoe upper material images that match the original shoe upper material image from the shoe upper material image library based on the multimodal fusion features includes: The multimodal fusion features are mapped to multiple independent subspaces for feature decoupling, resulting in query feature vectors under different attribute dimensions; Determine a specified number of candidate images of shoe upper materials with high similarity scores between the query feature vector and the candidate feature vector under different attribute dimensions; The similarity scores corresponding to each of the candidate images of the upper material are weighted and fused to obtain the overall similarity score corresponding to each candidate image of the upper material. Based on the overall similarity score corresponding to each of the candidate images of the upper material, the target upper material image is retrieved from each of the candidate images of the upper material.

[0007] According to the present invention, a multimodal shoe upper material image retrieval method based on prior enhancement is provided, wherein different attribute dimensions correspond to different approximate nearest neighbor search index objects; The step of determining a specified number of candidate images of shoe upper materials with high similarity scores between the query feature vector and the candidate feature vector under different attribute dimensions includes: For any attribute dimension, based on the query feature vector and the candidate feature vector under that attribute dimension, an approximate nearest neighbor search is performed in the approximate nearest neighbor search index object corresponding to that attribute dimension to obtain the candidate image of the shoe upper material under that attribute dimension.

[0008] According to the present invention, a multimodal shoe upper material image retrieval method based on prior enhancement is provided, wherein the multimodal fused features are mapped to multiple independent subspaces for feature decoupling to obtain query feature vectors under different attribute dimensions, including: For any independent subspace, based on the projection head corresponding to the independent subspace, the multimodal fusion feature is mapped to the independent subspace for feature decoupling, and the query feature vector under the attribute dimension corresponding to the independent subspace is obtained. Each of the projection heads is trained based on prior enhanced shoe upper material image samples and corresponding labeled data under different attribute dimensions.

[0009] According to the prior enhancement-based multimodal shoe upper material image retrieval method provided by the present invention, the step of weightedly fusing the similarity scores corresponding to each candidate shoe upper material image to obtain the overall similarity score corresponding to each candidate shoe upper material image includes: Receive similarity weights corresponding to different attribute dimensions input by the user; Based on the aforementioned similarity weights, the similarity scores corresponding to each candidate image of the upper material are weighted and fused to obtain the overall similarity score corresponding to each candidate image of the upper material.

[0010] According to the present invention, a multimodal shoe upper material image retrieval method based on prior enhancement is provided, wherein the step of performing rotation alignment and amplitude normalization based on the periodic peaks to obtain the processing result includes: Based on the position information of the periodic peak, calculate the angle between the periodic peak and the center of the spectrum, and perform rotation alignment based on the angle; The periodic spikes are frequency-domain filtered to obtain the filtered spectrum, and the filtered spectrum is subjected to inverse two-dimensional fast Fourier transform to obtain a spatial image. The pixel values ​​of each pixel in the spatial domain image are normalized to obtain the processing result.

[0011] According to the present invention, a multimodal shoe upper material image retrieval method based on prior enhancement is provided, wherein the training steps of the multimodal pre-trained model include: Multiple prior enhanced upper material image samples and multimodal text labels corresponding to each prior enhanced upper material image sample are obtained, and positive image-text matching pairs and negative image-text matching pairs are constructed based on each prior enhanced upper material image sample and each multimodal text label; Based on the positive and negative image-text matching pairs, the multimodal pre-trained model is subjected to comparative learning.

[0012] The present invention also provides a multimodal shoe upper material image retrieval device based on prior enhancement, comprising: The image acquisition module is used to acquire the original image of the shoe upper material to be retrieved; The prior enhancement module is used to perform prior enhancement processing on the original image of the shoe upper material to obtain a standardized enhanced image; The feature extraction module is used to input the standardized enhanced image into a pre-tuned multimodal pre-trained model to extract multimodal fusion features; The image retrieval module is used to retrieve one or more target shoe upper material images that match the original image of the shoe upper material from the shoe upper material image library based on the multimodal fusion features. The prior enhancement process includes: An overlapping block strategy is used to split the original image of the shoe upper material into multiple block images; A two-dimensional fast Fourier transform is performed on each image block to extract the amplitude spectrum to detect periodic spikes. The periodic spikes are then rotated, aligned, and normalized to obtain the processing result. The processing result is subjected to singular value decomposition, and a preset number of singular values ​​are retained for low-rank reconstruction to obtain a reconstructed block image. The reconstructed block images are weighted and stitched together to synthesize the standardized enhanced image.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the prior-enhanced multimodal shoe upper material image retrieval method as described above.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the prior-enhanced multimodal shoe upper material image retrieval method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the prior-enhanced multimodal shoe upper material image retrieval method as described above.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The multimodal image retrieval method and apparatus for shoe upper materials based on prior enhancement provided by this invention introduces physical prior information of shoe upper materials through prior enhancement processing including periodic peak detection and low-rank reconstruction. This explicitly corrects image positional misalignment and shooting variations. Combined with multimodal fusion features extracted by a pre-tuned multimodal pre-trained model, the feature discrimination accuracy can be improved, achieving highly robust and fine-grained retrieval of industrial-grade shoe upper materials. This reduces the R&D cost of manual material selection, improves R&D efficiency, provides shoe designers with quick references for similar shoe upper materials, and assists shoe designers in finding matching existing designs in massive amounts of materials, reducing the time cost of repetitive research. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the multimodal shoe upper material image retrieval method based on prior enhancement provided by the present invention.

[0019] Figure 2 This is a schematic diagram of the structure of the multimodal shoe upper material image retrieval device based on prior enhancement provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] Consumer-grade images refer to the WIT (WebImageText) dataset with 400 million image-text pairs used by multimodal pre-trained models and the ImageNet-1K dataset used by ResNet. This type of data mainly consists of general natural images, covering various categories of items, but there are almost no specific samples of industrial shoe upper materials; industrial-grade shoe upper material images, on the other hand, are all close-ups of shoe upper materials in the footwear industry, focusing on the texture and structure of specific materials such as leather, mesh, and weave.

[0023] Consumer-grade image annotation focuses on item category, with multimodal pre-trained models learning global shape and semantic information. Industrial-grade shoe upper material images, however, prioritize material / structural properties, requiring differentiation of subtle physical differences such as texture density, embossing depth, and weaving cycle. Furthermore, consumer-grade image retrieval only needs to identify the item, allowing for some error; whereas industrial-grade shoe upper material image retrieval requires determining substitutability, where even minute texture differences can render the material unusable, demanding extremely high fine-grained precision.

[0024] Therefore, directly applying general multimodal pre-trained models to feature extraction from shoe upper material images limits the ability of these models to represent and distinguish the material details of the shoe upper. Furthermore, due to factors such as shooting environment and positional misalignment, using existing methods for image retrieval reduces the robustness of the retrieval results.

[0025] Based on this, this embodiment of the invention provides a multimodal shoe upper material image retrieval method based on prior enhancement.

[0026] like Figure 1 As shown in the embodiment of the present invention, the multimodal shoe upper material image retrieval method based on prior enhancement includes: S1, Obtain the original image of the shoe upper material to be retrieved; S2, perform prior enhancement processing on the original image of the shoe upper material to obtain a standardized enhanced image; S3, input the standardized enhanced image into a pre-tuned multimodal pre-trained model to extract multimodal fusion features; S4. Based on the multimodal fusion features, retrieve one or more target shoe upper material images that match the original shoe upper material image from the shoe upper material image library; The prior enhancement process includes: An overlapping block strategy is used to split the original image of the shoe upper material into multiple block images; A two-dimensional fast Fourier transform is performed on each image block to extract the amplitude spectrum to detect periodic spikes. The periodic spikes are then rotated, aligned, and normalized to obtain the processing result. The processing result is subjected to singular value decomposition, and a preset number of singular values ​​are retained for low-rank reconstruction to obtain a reconstructed block image. The reconstructed image blocks are weighted and stitched together to synthesize the standardized enhanced image.

[0027] Specifically, the prior-enhanced multimodal shoe upper material image retrieval method provided in this embodiment of the invention is executed by a prior-enhanced multimodal shoe upper material image retrieval device. This device can be configured in a shoe upper material retrieval server, an intelligent design assistance system, or an electronic terminal with image processing capabilities, without specific limitations here.

[0028] First, step S1 is executed. The original image of the shoe upper material refers to image materials of shoe upper materials acquired under non-natural lighting conditions in the shoe manufacturing industry, possessing significant industrial characteristics. This includes, but is not limited to, close-ups of natural leather, synthetic materials, woven mesh, and shoe uppers with embossed or knitted patterns. The original images of shoe upper materials exhibit significant local texture repetition, global structural periodicity, subtle differences in material details, and low-rank statistical characteristics, serving as the foundational data for subsequent fine-grained retrieval. The original image of the shoe upper material to be retrieved can be obtained by calling a preset image acquisition interface, local storage path, or cloud database.

[0029] Since the original image of the shoe upper material was taken under non-natural lighting conditions, it is greatly affected by illumination, angle, and misalignment, resulting in significant distributional differences compared to general natural images or consumer-grade images. To improve the robustness of the features extracted by the multimodal pre-trained model, step S2 is then performed to perform prior enhancement processing on the original image of the shoe upper material, obtaining a standardized enhanced image.

[0030] The core of step S2 lies in preprocessing using the inherent physical and geometric laws of the shoe upper material. Among them, prior enhancement processing refers to explicitly modeling the original shoe upper material image using mathematical means and utilizing the frequency domain peaks and low-rank laws in the original image to standardize and correct the image, thereby eliminating interference from environmental factors, without the need for model learning.

[0031] In the specific implementation of the prior enhancement process, an overlapping block strategy is first adopted to divide the original image of the shoe upper material into multiple block images. The overlapping block strategy refers to maintaining a certain proportion of overlap between adjacent blocks according to a preset step size and block size. For example, for a 512×512 pixel original image of the shoe upper material, a block size of 128×128 and an overlap step size of 64 pixels can be used. This allows for a smooth transition of the overlapping parts in subsequent compositing processes, avoiding obvious seam breaks or boundary artifacts.

[0032] After obtaining each image block, a two-dimensional fast fourier transform (2D-FFT) is performed on each block to convert the image information from the spatial domain to the frequency domain. The amplitude spectrum is then extracted from this result to detect periodic spikes. The amplitude spectrum is a spectrum formed by the amplitude values ​​at each coordinate position in the frequency domain. Periodic spikes refer to discrete peak points in the amplitude spectrum where the amplitude values ​​are highly concentrated due to the repetitive geometric structure of the shoe upper material, such as weave, mesh, or embossing. The distribution pattern of each periodic spike in the frequency domain directly maps to the spacing and arrangement direction of the shoe upper texture in physical space, and is a key physical basis for achieving image standardization.

[0033] Based on periodic spikes, the segmented image can be rotated, aligned, and its amplitude normalized to obtain the processed result. Rotation alignment automatically corrects the tilt angle of the segmented image during capture based on the offset of the periodic spikes. Amplitude normalization suppresses low-frequency components representing background illumination and high-frequency components representing noise, retaining only the mid-frequency components reflecting texture structure. Through rotation alignment and amplitude normalization, the distribution variations caused by misaligned capture positions or uneven illumination can be explicitly mitigated at the frequency domain level, significantly improving the robustness of feature representations extracted by multimodal pre-trained models.

[0034] Subsequently, Singular Value Decomposition (SVD) is performed on the processed results. Leveraging the low-rank prior of the image mathematically, singular values ​​representing the main structure are extracted to remove unstructured interference from the segmented image. In implementation, a preset number of singular values ​​can be retained for low-rank reconstruction to obtain the reconstructed segmented image. This preset number is typically between 10 and 20, retaining the first 10 to 20 largest singular values, which carry the main low-frequency structural information of the segmented image. By reconstructing the segmented image using the preset number of singular values, high-frequency noise, creases, or subtle occlusion interference can be effectively filtered out.

[0035] This step utilizes low-rank reconstruction to remove high-frequency noise such as illumination, wrinkles, and occlusion, achieving standardized enhancement of the segmented image.

[0036] Finally, the reconstructed image blocks are weighted and stitched together to synthesize a standardized enhanced image. During the stitching process, pixels in overlapping areas are assigned different weight coefficients based on their distance from the block center and then weighted and averaged to achieve seamless fusion. The standardized enhanced image is a high-quality input after frequency domain correction and low-rank denoising, and its distribution more closely resembles the standard template for industrial material selection. Through the above series of prior enhancement methods, it is possible to proactively respond to changes in the industrial environment without relying on massive amounts of data for training, thus building a highly consistent input foundation for subsequent deep feature extraction.

[0037] Next, step S3 is executed. In the feature extraction stage, the standardized enhanced image is input into the pre-fine-tuned multimodal pre-trained model to extract multimodal fusion features. The multimodal pre-trained model refers to a pre-trained model trained on large-scale image and text pairings, such as Contrastive Language-Image Pre-training (CLIP) or Bootstrapping Language-Image Pre-training (BLIP), which possesses powerful visual-language joint understanding capabilities. The fine-tuning process of the multimodal pre-trained model can utilize efficient parameter fine-tuning techniques such as Low-Rank Adaptation (LoRA) to perform domain transfer on a shoe upper-specific dataset containing text labels such as material names, process descriptions, and color codes.

[0038] Multimodal fusion features refer to high-dimensional vectors output by multimodal pre-trained models that achieve spatial alignment between visual pixel information and textual semantic information on the shoe upper. Multimodal pre-trained models combine multimodal semantic information, enabling the extracted features to go beyond surface-level visual pixels and capture fine-grained attributes with industrial significance, such as weave density and embossing depth.

[0039] Finally, step S4 is executed to retrieve one or more target shoe upper material images that match the original shoe upper material image from the shoe upper material image library using multimodal fusion features. Here, a distance metric can be calculated between the multimodal fusion features and the candidate fusion features of each candidate shoe upper material image in the shoe upper material image library. This distance metric can be a similarity score or other distance measurement parameter. The images are sorted from high to low based on similarity scores, and the top target number of candidate shoe upper material images are output as the target shoe upper material images for retrieval feedback. This target number can be set as needed, for example, it can be set to one or more.

[0040] The prior-enhanced multimodal shoe upper material image retrieval method provided in this invention introduces physical prior information about shoe upper materials through prior enhancement processing including periodic peak detection and low-rank reconstruction. This explicitly corrects image positional misalignment and shooting variations. Combined with multimodal fusion features extracted from a pre-tuned multimodal pre-trained model, the method can improve feature discrimination accuracy, achieve highly robust and fine-grained retrieval of industrial-grade shoe upper materials, reduce the R&D cost of manual material selection, improve R&D efficiency, provide shoe designers with quick references for similar shoe upper materials, and assist shoe designers in finding matching existing designs in massive amounts of materials, reducing the time cost of repetitive research.

[0041] Because existing search ranking methods mostly use a single global cosine distance, it is difficult to meet the personalized needs of industrial material selection, which involves multiple dimensions such as visual appearance, material texture, design style and structural pattern. As a result, the search results lack interpretability and interactivity.

[0042] Based on this, and on the basis of the above embodiments, each candidate image of a shoe upper material in the shoe upper material library corresponds to a candidate feature vector with different attribute dimensions. The step of retrieving one or more target shoe upper material images that match the original shoe upper material image from the shoe upper material image library based on the multimodal fusion features includes: The multimodal fusion features are mapped to multiple independent subspaces for feature decoupling, resulting in query feature vectors under different attribute dimensions; Determine a specified number of candidate images of shoe upper materials with high similarity scores between the query feature vector and the candidate feature vector under different attribute dimensions; The similarity scores corresponding to each of the candidate images of the upper material are weighted and fused to obtain the overall similarity score corresponding to each candidate image of the upper material. Based on the overall similarity score corresponding to each of the candidate images of the upper material, the target upper material image is retrieved from each of the candidate images of the upper material.

[0043] Specifically, the shoe upper material image library is not a simple image stack, but a preprocessed, large-scale structured database. Each candidate image of a shoe upper material has pre-extracted and stored candidate feature vectors corresponding to different attribute dimensions. These candidate feature vectors are mathematical representations generated offline for each candidate image, using the same feature extraction and decoupling methods as the original shoe upper material images. These representations characterize the visual appearance, material feel, design style, and structural layout of the shoe upper material. Understandably, the different attribute dimensions can include four dimensions: overall visual appearance, material texture details, design style, and structural repetition and symmetry.

[0044] In the process of image matching using multimodal fusion features, the multimodal fusion features can first be mapped to multiple independent subspaces for feature decoupling, resulting in query feature vectors under different attribute dimensions. Here, feature decoupling can utilize a lightweight projection head to split the mixed image and text information, which was originally coupled in a single high-dimensional vector, into different attribute dimensions that do not interfere with each other, according to physical attributes or semantic logic.

[0045] Each independent subspace specifically includes a visual subspace, a material subspace, a style subspace, and a structural subspace. In the visual subspace, the query feature vector corresponding to the overall visual appearance reflects the overall color distribution and outline of the original image of the shoe upper material. In the material subspace, the query feature vector corresponding to the material texture details captures the local fiber texture details and weave tightness of the original image of the shoe upper material. In the style subspace, the query feature vector corresponding to the design style reflects high-level semantics such as design sense and fashion trends. In the structural subspace, the query feature vector corresponding to the structural repetition pattern and symmetry quantifies the periodic repetition pattern and symmetry of the texture in the original image of the shoe upper material. This feature decoupling process allows for the extraction of multi-layered attribute representations from a single feature, enabling subsequent fine-grained differentiated retrieval.

[0046] After obtaining the query feature vectors under different attribute dimensions, it is also necessary to determine a specified number of candidate images of shoe upper materials with high similarity scores between the query feature vectors and candidate feature vectors under different attribute dimensions. In the specific implementation process, for each attribute dimension, the similarity score between the query feature vector and the candidate feature vectors of the corresponding attribute dimension in the shoe upper material image library can be calculated, and a specified number of candidate images of shoe upper materials with high similarity scores can be selected as candidate images of shoe upper materials.

[0047] Subsequently, the similarity scores for each candidate image of the upper material are weighted and fused to obtain the overall similarity score for each candidate image. Weighted fusion refers to the weighted fusion of similarity scores across four attribute dimensions: overall visual appearance, material texture details, design style, and structural repetition and symmetry. Weighted fusion methods can include linear weighting and attention weighting, which are not specifically limited here. This weighted fusion mechanism allows the discrete similarity scores from different attribute dimensions to be integrated into an overall similarity score.

[0048] Finally, using the overall similarity score corresponding to each candidate image of the upper material, the target upper material image is retrieved from each candidate image. Here, the candidate images of the upper material can be reordered according to the overall similarity score from high to low, and a number of target upper material images with high overall similarity scores can be selected from each candidate image. This number can be less than or equal to a specified number.

[0049] In this embodiment of the invention, feature decoupling is achieved by mapping multimodal fusion features to multiple independent subspaces, ensuring that the retrieval results have high reference value in each attribute dimension. This avoids the mutual cancellation of dimensional information caused by a single global distance, which can solve the technical problem of significant differences in multidimensional attributes in industrial material selection scenarios and improve image retrieval accuracy.

[0050] Based on the above embodiments, different attribute dimensions correspond to different approximate nearest neighbor search index objects; The step of determining a specified number of candidate images of shoe upper materials with high similarity scores between the query feature vector and the candidate feature vector under different attribute dimensions includes: For any attribute dimension, based on the query feature vector and the candidate feature vector under that attribute dimension, an approximate nearest neighbor search is performed in the approximate nearest neighbor search index object corresponding to that attribute dimension to obtain the candidate image of the shoe upper material under that attribute dimension.

[0051] Specifically, in determining a specified number of candidate images of shoe upper materials with high similarity scores between query feature vectors and candidate feature vectors under different attribute dimensions, an efficient retrieval architecture can be constructed for the four decoupled attribute dimensions to achieve real-time response in a large-scale image library. Each dimension corresponds to a different Approximate Nearest Neighbor (ANN) search index object. This ANN search index object can be a dedicated data structure built based on the Facebook AI Similarity Search (FAISS) framework. It organizes massive candidate feature vectors into a fast-searchable index file, i.e., an inverted file (IVF), through algorithms such as clustering, quantization, or spatial partitioning. Since the distribution patterns and vector dimension characteristics of the four attribute dimensions—overall visual appearance, material texture details, design style, and structural repetition and symmetry—are different, by establishing an independent ANN search index object for each attribute dimension, search parameters can be optimized in a targeted manner to ensure that the retrieval accuracy and speed of each dimension reach the optimal balance.

[0052] In the specific implementation, for any attribute dimension, an approximate nearest neighbor search is performed on the corresponding approximate nearest neighbor search index object to obtain candidate images of shoe upper materials under that attribute dimension. During the offline database construction phase, candidate feature vectors for the attribute dimensions of all candidate shoe upper material images in the shoe upper material image library are pre-extracted, and approximate nearest neighbor search index objects and IVF indexes are constructed respectively. During the online retrieval phase, for any query feature vector of any attribute dimension extracted from the original shoe upper material image, only the top specified number of candidate feature vectors with high similarity scores need to be found in its corresponding approximate nearest neighbor search index object, and their corresponding shoe upper material candidate images are used as shoe upper material candidate images, without exhaustively comparing the entire shoe upper material image library. By introducing the approximate nearest neighbor search index object, the retrieval time for image libraries of tens of thousands or even hundreds of thousands can be compressed from seconds to milliseconds, greatly improving the search efficiency in industrial material selection scenarios and meeting the performance requirements of designers for real-time filtering of massive amounts of materials.

[0053] Furthermore, this attribute-based indexing strategy enables a deep fusion of retrieval efficiency and multi-dimensional semantic expression. Each near nearest neighbor search index object can independently store and quickly provide feedback on candidate images of shoe upper materials and their similarity scores under that attribute dimension, providing a standardized input source for subsequent weighted fusion. This independent index design also brings a significant advantage: it allows users to dynamically adjust the weights of dimensions such as visual and material without rebuilding the index, and the ranking results can be updated in real time simply through weighted calculations in the backend logic.

[0054] In this embodiment of the invention, by constructing independent approximate nearest neighbor search index objects for different attribute dimensions and performing targeted approximate nearest neighbor searches, not only is extremely high retrieval real-time performance guaranteed when processing large-scale industrial image data, but also, through index-level decoupling, multi-dimensional fine-grained attribute matching is supported, providing an efficient, accurate and responsive intelligent auxiliary means for shoe manufacturing R&D.

[0055] Based on the above embodiments, the step of mapping the multimodal fusion features to multiple independent subspaces for feature decoupling to obtain query feature vectors under different attribute dimensions includes: For any independent subspace, based on the projection head corresponding to the independent subspace, the multimodal fusion feature is mapped to the independent subspace for feature decoupling, and the query feature vector under the attribute dimension corresponding to the independent subspace is obtained. Each of the projection heads is trained based on prior enhanced shoe upper material image samples and corresponding labeled data under different attribute dimensions.

[0056] Specifically, the core of feature decoupling lies in using lightweight neural structures to accurately split highly coupled multimodal fusion features. To achieve this, projection heads corresponding to each independent subspace are introduced, and each projection head can be linked in parallel with the multimodal pre-trained model.

[0057] For any independent subspace, the multimodal fusion features are mapped to that subspace using the corresponding projection head to decouple the features, resulting in a query feature vector for the attribute dimension of that subspace. Each projection head for an independent subspace can be a Multi-Layer Perceptron (MLP) or a lightweight fully connected network layer. Its role is to act as a non-linear transformer, filtering query feature vectors related to a specific attribute dimension from the original multimodal fusion features. For example, the projection head corresponding to material texture details, through trained weights, can filter out interference such as color and amplify feature signals reflecting fiber intersections or texture direction as query feature vectors.

[0058] Each projection head can process data in parallel and can be synchronously trained using contrastive learning by combining prior enhanced image samples of shoe upper materials with corresponding labeled data across different attribute dimensions. The prior enhanced image samples refer to industrial-grade training images that have undergone prior enhancement processing such as frequency domain alignment and low-rank reconstruction, significantly highlighting the essential physical characteristics of the shoe upper material. The labeled data includes precise quantitative evaluations of the samples by professional footwear designers or process experts across four attribute dimensions: overall visual appearance, material texture details, design style, and structural repetition and symmetry.

[0059] In the specific implementation, the multimodal fusion features output from the pre-tuned multimodal pre-trained model are simultaneously input into four parallel projection heads. Each projection head receives the same input, namely the multimodal fusion features, but through its unique internal weight configuration, it projects the multimodal fusion features into a shorter, more information-concentrated subspace vector. The projection heads enable high-purity feature extraction, effectively avoiding semantic occlusion of features with different attribute dimensions in a single vector. For example, it avoids the problem of large differences in material texture being masked by extremely similar colors, thus providing the most expressive query benchmark for subsequent fine-grained attribute searches.

[0060] Furthermore, since the training process of each projection head incorporates the physical priors of the shoe upper material, the resulting query feature vector has extremely high industrial interpretability, laying a technical foundation for subsequently outputting search reasons and enhancing users' trust in the search results.

[0061] In this embodiment of the invention, by using a dedicated projection head trained based on prior enhanced samples and multi-dimensional labeled data for feature mapping, efficient decoupling of multimodal fusion features is achieved. This enables the retrieval process to delve into the fine-grained physical properties of shoe upper materials, which not only improves the expression accuracy of features in specific business dimensions, but also gives the retrieval system flexible adaptability to complex industrial scenarios. It is a core link in realizing the transformation of industrial-grade image search to intelligence and refinement.

[0062] Based on the above embodiments, the step of weighted fusion of the similarity scores corresponding to each candidate image of the upper material to obtain the overall similarity score corresponding to each candidate image of the upper material includes: Receive similarity weights corresponding to different attribute dimensions input by the user; Based on the aforementioned similarity weights, the similarity scores corresponding to each candidate image of the upper material are weighted and fused to obtain the overall similarity score corresponding to each candidate image of the upper material.

[0063] Specifically, when weighting and fusing the similarity scores for each candidate image of the shoe upper material, the core lies in deeply integrating the calculated objective physical metric, i.e., the similarity score, with the user's subjective judgment logic in actual industrial scenarios. Similarity weight refers to the relative importance coefficients assigned by users to four attribute dimensions: overall visual appearance, material texture details, design style, and structural repetition and symmetry. It is usually represented as a set of normalized values. Its role is to act as a control variable, directly intervening in the retrieval system's standards, enabling the retrieval results to shift from a single algorithmic optimal solution to a business-optimal solution that meets specific business needs. For example, in the early stages of R&D, designers may rely more on design style weights to find inspiration; while in the production prototyping stage, process engineers will increase the weight of material texture details to ensure the matching of physical properties.

[0064] In the specific implementation, the executing entity can display interactive sliders or numerical input boxes to the user through a graphical user interface (GUI), allowing the user to dynamically adjust the similarity weights of the four attribute dimensions according to the current image search focus, and then receive the similarity weights corresponding to different attribute dimensions input by the user. After receiving each similarity weight, a weight verification can be performed to ensure that the sum of the similarity weights of the four attribute dimensions for each candidate image of shoe upper material equals 1. Subsequently, using each similarity weight, the similarity scores corresponding to each candidate image of shoe upper material are weighted and fused to obtain the overall similarity score for each candidate image of shoe upper material. Here, the weighted fusion adopts a linear combination model, multiplying the similarity scores calculated in each independent subspace by their corresponding similarity weights and then summing them.

[0065] In this step, by introducing similarity weights from user input, the interpretability and interactivity of the retrieval process can be achieved. Through the collaboration of human feedback and machine computation, the satisfaction of retrieval results in complex and ever-changing industrial application scenarios can be significantly improved.

[0066] Subsequently, all candidate images of shoe upper materials are globally re-ranked based on the calculated overall similarity score. This weighted fusion process not only effectively eliminates potential dimensional differences between different attribute dimensions, but also leverages weights to ensure that images excelling in specific attribute dimensions of interest to the user stand out, avoiding the loss of fine-grained features caused by smoothing and averaging different attribute information in traditional retrieval methods.

[0067] In this embodiment of the invention, prior enhancement processing ensures the standardization of the original data. A multimodal pre-trained model is used to extract deep features, and combined with independent subspace decoupling and user-defined weighted fusion techniques, a complete industrial retrieval closed loop is constructed, from the physical layer, representation layer to the decision layer. This transforms traditional black-box search into multi-dimensional transparent retrieval, not only solving the feature extraction problem caused by repetitive textures in shoe upper materials, but also perfectly adapting to the differentiated needs of various stages of design and production in the footwear industry through a weight adjustment mechanism. This greatly improves the efficiency of enterprises in accurately selecting materials from massive material databases, demonstrating significant engineering application value and socio-economic benefits. By introducing a user subjective preference feedback mechanism, the retrieval results can dynamically adapt to the personalized needs of different business scenarios such as design, prototyping, and procurement. This solves the technical challenges of highly subjective and multi-dimensionally different needs in industrial material selection scenarios, significantly improving the interactive flexibility and interpretability of image retrieval while ensuring retrieval accuracy.

[0068] Based on the above embodiments, the step of performing rotation alignment and amplitude normalization according to the periodic peak to obtain the processing result includes: Based on the position information of the periodic peak, calculate the angle between the periodic peak and the center of the spectrum, and perform rotation alignment based on the angle; The periodic spikes are frequency-domain filtered to obtain the filtered spectrum, and the filtered spectrum is subjected to inverse two-dimensional fast Fourier transform to obtain a spatial image. The pixel values ​​of each pixel in the spatial domain image are normalized to obtain the processing result.

[0069] Specifically, the core objective of rotation alignment and amplitude normalization is to eliminate interference caused by shooting angle deviations and complex lighting environments on the texture features of the shoe upper during industrial imaging. Rotation alignment and amplitude normalization refer to transforming the originally acquired uncontrolled block images into feature-consistent processed results through geometric space correction and energy intensity normalization, respectively.

[0070] In the specific implementation process, firstly, based on the position information of the periodic peak, the angle between the periodic peak and the center of the spectrum is calculated, and the amplitude of this angle is used as the rotation target to rotate and align the block images. This not only eliminates geometric interference in the block images and unifies the main texture direction of the block images, avoiding texture feature misalignment caused by shooting angle and tilt, but also makes subsequent periodic peak detection more accurate and significantly improves the robustness of the multimodal pre-trained model to geometric variations such as tilt and flip. It also makes the same type of shoe upper material with different rotation angles closer in the independent subspace corresponding to the overall visual appearance and material texture details, greatly reducing the false negative rate of similarity retrieval. This provides standardized block images with unified direction for subsequent SVD low-rank decomposition and block weighted stitching, avoiding artifacts in illumination correction and texture reconstruction caused by chaotic direction.

[0071] Subsequently, the periodic spikes are frequency-domain filtered to obtain the filtered spectrum, and then an Inverse Two-Dimensional Fast Fourier Transform (i2D-FFT) is performed on the filtered spectrum to obtain the spatial domain image. In this process, by designing specific bandpass filters or spike masks, only the periodic spike components carrying the core texture structure are allowed to pass through, while low-frequency interference representing uneven illumination and high-frequency components representing random noise are suppressed. By restoring the filtered spectrum back to the spatial domain using i2D-FFT, a grayscale or color image, i.e., the spatial domain image, is obtained, which removes environmental interference and retains only pure structural information. This step achieves preliminary feature purification, eliminating visual redundancy unrelated to similarity retrieval through physical filtering, and significantly enhancing the expression of material structure.

[0072] Finally, the pixel values ​​of each pixel in the spatial domain image are normalized, that is, the pixel values ​​of each pixel in the spatial domain image are mapped to the range [0,1] or [0,255] to obtain the processing result. This normalization operation not only balances the brightness differences caused by shooting with different image acquisition devices, but also makes the subtle material differences more significant in mathematical representation by stretching the texture contrast.

[0073] In this embodiment of the invention, rotation alignment is achieved by accurately quantifying the position information of periodic peaks, and amplitude correction is completed by combining frequency domain filtering and spatial domain normalization. This can elevate traditional pixel-level enhancement to feature specification at the level of physical laws. It can not only effectively solve the technical pain points of inconsistent shooting angles and huge differences in lighting conditions that are common in the image retrieval of industrial shoe upper materials, but also greatly alleviate the learning pressure of neural networks when processing fine-grained textures by explicitly introducing physical priors. This provides high-quality, consistent underlying data support for the high-precision extraction of subsequent multimodal features.

[0074] Based on the above embodiments, the training steps of the multimodal pre-trained model include: Multiple prior enhanced upper material image samples and multimodal text labels corresponding to each prior enhanced upper material image sample are obtained, and positive image-text matching pairs and negative image-text matching pairs are constructed based on each prior enhanced upper material image sample and each multimodal text label; Based on the positive and negative image-text matching pairs, the multimodal pre-trained model is subjected to comparative learning.

[0075] Specifically, in the training process of the multimodal pre-trained model, a high-quality vertical domain training dataset can be constructed first. This dataset includes multiple prior augmented shoe upper material image samples and corresponding multimodal text labels for each sample. The prior augmented shoe upper material image samples are the sets of images obtained after performing prior augmentation processing on the shoe upper material image samples using techniques such as two-dimensional fast Fourier transform and singular value decomposition before model learning. Each prior augmented shoe upper material image sample eliminates interference from lighting, angle, and environmental noise, ensuring that the model learns the purest material texture patterns. The multimodal text labels are structured descriptions provided by material experts for each shoe upper material image sample, including multiple modalities such as material name, weaving process, surface treatment method, and style attributes. By obtaining the prior augmented shoe upper material image samples and their corresponding multimodal text labels, a learning benchmark with image-text comparison is provided for the multimodal pre-trained model. This step, by introducing prior enhanced image samples of shoe upper materials, can significantly reduce the convergence difficulty of multimodal pre-trained models during training in vertical domains and avoid the risk of overfitting caused by background noise.

[0076] Subsequently, positive image-text matching pairs are constructed using prior enhanced shoe upper material image samples and corresponding multimodal text labels, and negative image-text matching pairs are constructed using the multimodal text labels corresponding to other prior enhanced shoe upper material image samples.

[0077] Positive and negative image-text matching pairs are input into a multimodal pre-trained model. The visual encoder of the multimodal pre-trained model processes prior enhanced shoe upper material image samples, while the text encoder processes text labels. This maps the input information from different modalities to the same high-dimensional semantic space, obtaining and outputting the joint feature of each matching pair. This joint feature is a cross-modal vector that simultaneously represents visual texture and textual semantics, serving as the mathematical core for subsequently measuring image-text matching accuracy. This step, by inputting positive and negative image-text matching pairs into the multimodal pre-trained model, achieves deep integration of visual features and industrial semantics. The features generated by the multimodal pre-trained model are no longer isolated pixel stacks but possess the ability to understand material properties, providing semantic support for fine-grained retrieval.

[0078] Finally, contrastive learning is performed on the multimodal pre-trained model using the joint features of positive and negative image-text matching pairs. The core logic of this contrastive learning is to narrow the distance between positive image-text matching pairs in the feature space while widening the distance between negative image-text matching pairs. In implementation, triplet loss or information noise-contrastive estimation (InfoNCE) loss can be calculated, and efficient fine-tuning techniques such as low-rank adaptation (LoRA) can be used to dynamically update only the key attention layer of the multimodal pre-trained model. This significantly enhances the feature discrimination power of the multimodal pre-trained model, enabling it to keenly capture subtle texture and manufacturing differences between different shoe upper materials, thus achieving extremely high accuracy in actual retrieval.

[0079] In this embodiment of the invention, an intelligent retrieval system for complex texture scenarios in the footwear industry is constructed by combining standardized prior enhancement, multimodal feature decoupling, multi-dimensional index object construction, and contrastive learning fine-tuning. This system not only explicitly addresses the instability caused by the changing shooting environment using physical priors, but also integrates deep expert knowledge into feature representations through a multimodal pre-trained model, giving users the flexibility to intervene in the retrieval logic through weight adjustment. Furthermore, it elevates traditional single image matching to the level of multi-dimensional semantic alignment, significantly improving the accuracy and efficiency of shoe upper material retrieval. It also adapts to diversified business needs such as design, production, and procurement, providing a solid technical guarantee for the digital and intelligent transformation of the footwear industry.

[0080] like Figure 2 As shown, based on the above embodiments, this embodiment of the invention provides a multimodal shoe upper material image retrieval device based on prior enhancement, comprising: Image acquisition module 21 is used to acquire the original image of the shoe upper material to be retrieved; Prior enhancement module 22 is used to perform prior enhancement processing on the original image of the shoe upper material to obtain a standardized enhanced image; Feature extraction module 23 is used to input the standardized enhanced image into a pre-tuned multimodal pre-trained model to extract multimodal fusion features; Image retrieval module 24 is used to retrieve one or more target shoe upper material images that match the original image of the shoe upper material in the shoe upper material image library based on the multimodal fusion features; The prior enhancement process includes: An overlapping block strategy is used to split the original image of the shoe upper material into multiple block images; A two-dimensional fast Fourier transform is performed on each image block to extract the amplitude spectrum to detect periodic spikes. The periodic spikes are then rotated, aligned, and normalized to obtain the processing result. The processing result is subjected to singular value decomposition, and a preset number of singular values ​​are retained for low-rank reconstruction to obtain a reconstructed block image. The reconstructed block images are weighted and stitched together to synthesize the standardized enhanced image.

[0081] Specifically, the functions of each module in the multimodal shoe upper material image retrieval device based on prior enhancement provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0082] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the prior-enhanced multimodal shoe upper material image retrieval method provided in the above embodiments.

[0083] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the prior-enhanced multimodal shoe upper material image retrieval method provided in the above embodiments.

[0085] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the prior-enhanced multimodal shoe upper material image retrieval method provided in the above embodiments. This computer-readable storage medium can be either a non-transitory computer-readable storage medium or a transient computer-readable storage medium, and is not specifically limited herein.

[0086] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal shoe upper material image retrieval method based on prior enhancement, characterized in that, include: Obtain the original image of the shoe upper material to be retrieved; The original image of the shoe upper material is subjected to prior enhancement processing to obtain a standardized enhanced image; The standardized enhanced image is input into a pre-tuned multimodal pre-trained model to extract multimodal fusion features; Based on the multimodal fusion features, one or more target shoe upper material images that match the original shoe upper material image are retrieved from the shoe upper material image library; The prior enhancement process includes: An overlapping block strategy is used to split the original image of the shoe upper material into multiple block images; A two-dimensional fast Fourier transform is performed on each image block to extract the amplitude spectrum to detect periodic spikes. The periodic spikes are then rotated, aligned, and normalized to obtain the processing result. The processing result is subjected to singular value decomposition, and a preset number of singular values ​​are retained for low-rank reconstruction to obtain a reconstructed block image. The reconstructed image blocks are weighted and stitched together to synthesize the standardized enhanced image.

2. The multimodal shoe upper material image retrieval method based on prior enhancement according to claim 1, characterized in that, Each candidate image of a shoe upper material in the shoe upper material library corresponds to a candidate feature vector with different attribute dimensions; The step of retrieving one or more target shoe upper material images that match the original shoe upper material image from the shoe upper material image library based on the multimodal fusion features includes: The multimodal fusion features are mapped to multiple independent subspaces for feature decoupling, resulting in query feature vectors under different attribute dimensions; Determine a specified number of candidate images of shoe upper materials with high similarity scores between the query feature vector and the candidate feature vector under different attribute dimensions; The similarity scores corresponding to each of the candidate images of the upper material are weighted and fused to obtain the overall similarity score corresponding to each candidate image of the upper material. Based on the overall similarity score corresponding to each of the candidate images of the upper material, the target upper material image is retrieved from each of the candidate images of the upper material.

3. The multimodal shoe upper material image retrieval method based on prior enhancement according to claim 2, characterized in that, Different attribute dimensions correspond to different approximate nearest neighbor search index objects; The step of determining a specified number of candidate images of shoe upper materials with high similarity scores between the query feature vector and the candidate feature vector under different attribute dimensions includes: For any attribute dimension, based on the query feature vector and the candidate feature vector under that attribute dimension, an approximate nearest neighbor search is performed in the approximate nearest neighbor search index object corresponding to that attribute dimension to obtain the candidate image of the shoe upper material under that attribute dimension.

4. The multimodal shoe upper material image retrieval method based on prior enhancement according to claim 2, characterized in that, The step of mapping the multimodal fused features to multiple independent subspaces for feature decoupling to obtain query feature vectors under different attribute dimensions includes: For any independent subspace, based on the projection head corresponding to the independent subspace, the multimodal fusion feature is mapped to the independent subspace for feature decoupling, and the query feature vector under the attribute dimension corresponding to the independent subspace is obtained. Each of the projection heads is trained based on prior enhanced shoe upper material image samples and corresponding labeled data under different attribute dimensions.

5. The multimodal shoe upper material image retrieval method based on prior enhancement according to claim 2, characterized in that, The step of weightedly fusing the similarity scores corresponding to each of the candidate images for shoe upper materials to obtain the overall similarity score for each candidate image for shoe upper materials includes: Receive similarity weights corresponding to different attribute dimensions input by the user; Based on the aforementioned similarity weights, the similarity scores corresponding to each candidate image of the upper material are weighted and fused to obtain the overall similarity score corresponding to each candidate image of the upper material.

6. The multimodal shoe upper material image retrieval method based on prior enhancement according to any one of claims 1-5, characterized in that, The process of rotating and aligning the periodic spikes and normalizing their amplitudes to obtain the processing result includes: Based on the position information of the periodic peak, calculate the angle between the periodic peak and the center of the spectrum, and perform rotation alignment based on the angle; The periodic spikes are frequency domain filtered to obtain the filtered spectrum, and the filtered spectrum is subjected to inverse two-dimensional fast Fourier transform to obtain a spatial domain image. The pixel values ​​of each pixel in the spatial domain image are normalized to obtain the processing result.

7. The multimodal shoe upper material image retrieval method based on prior enhancement according to any one of claims 1-5, characterized in that, The training steps of the multimodal pre-trained model include: Multiple prior enhanced upper material image samples and multimodal text labels corresponding to each prior enhanced upper material image sample are obtained, and positive image-text matching pairs and negative image-text matching pairs are constructed based on each prior enhanced upper material image sample and each multimodal text label; Based on the positive and negative image-text matching pairs, the multimodal pre-trained model is subjected to comparative learning.

8. A multimodal shoe upper material image retrieval device based on prior enhancement, characterized in that, include: The image acquisition module is used to acquire the original image of the shoe upper material to be retrieved; The prior enhancement module is used to perform prior enhancement processing on the original image of the shoe upper material to obtain a standardized enhanced image; The feature extraction module is used to input the standardized enhanced image into a pre-tuned multimodal pre-trained model to extract multimodal fusion features; The image retrieval module is used to retrieve one or more target shoe upper material images that match the original image of the shoe upper material from the shoe upper material image library based on the multimodal fusion features. The prior enhancement process includes: An overlapping block strategy is used to split the original image of the shoe upper material into multiple block images; A two-dimensional fast Fourier transform is performed on each image block to extract the amplitude spectrum to detect periodic spikes. The periodic spikes are then rotated, aligned, and normalized to obtain the processing result. The processing result is subjected to singular value decomposition, and a preset number of singular values ​​are retained for low-rank reconstruction to obtain a reconstructed block image. The reconstructed block images are weighted and stitched together to synthesize the standardized enhanced image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multimodal shoe upper material image retrieval method based on prior enhancement as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multimodal shoe upper material image retrieval method based on prior enhancement as described in any one of claims 1-7.

Images