Image retrieval method and device based on multi-dimensional feature fusion and storage medium

Abstract

The invention provides an image retrieval method and device based on multi-dimensional feature fusion and a storage medium, and relates to the technical field of image retrieval. The method comprises the following steps: importing an image to be retrieved, and performing multi-dimensional feature extraction processing on the image to be retrieved to obtain a plurality of dimensional features; performing feature fusion on the multiple dimension features to obtain an image feature representation vector; and performing image retrieval based on the similarity between the image feature representation vector and an image feature vector in a preset database, and outputting an image with relatively high similarity as a retrieval result. According to the method, the essential features of the image are captured from different angles through multi-dimensional feature extraction, the limitation of a single feature is avoided, the features are more comprehensive and richer, a foundation is laid for retrieval, and the method can be suitable for various image retrieval scenes and has a good effect on retrieval of various images.

Description

Technical Field

[0001] This invention relates to the field of image retrieval technology, specifically to an image retrieval method, device, and storage medium based on multi-dimensional feature fusion. Background Technology

[0002] In today's information-saturated world, images, as a crucial carrier of information, are growing exponentially in number. A key challenge is how to quickly and accurately retrieve the images users need from this massive amount of image data. Image retrieval technology has wide-ranging applications in numerous fields, including security monitoring, e-commerce, medical diagnosis, and digital libraries.

[0003] Traditional image retrieval methods are mostly based on single-dimensional features, such as visual features like color, texture, and shape. However, this single-feature retrieval approach has significant limitations. On the one hand, a single feature cannot comprehensively and accurately describe the essential content of an image, failing to capture the rich semantic and structural information within it. On the other hand, in complex real-world application scenarios, changes in factors such as the image's shooting angle, lighting conditions, and background interference can significantly impact the effectiveness of a single feature, leading to lower accuracy and robustness of the retrieval results.

[0004] In practical applications, users' retrieval needs are often diverse and complex. Single-dimensional features and simple feature fusion methods are insufficient to meet users' requirements for the accuracy and relevance of retrieval results. Therefore, there is an urgent need for an image retrieval method that can comprehensively utilize multi-dimensional image features and effectively fuse these features to improve image retrieval performance and meet the needs of real-world application scenarios. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an image retrieval method, device and storage medium based on multi-dimensional feature fusion, which addresses the shortcomings of the prior art.

[0006] The technical solution of this invention to solve the above-mentioned technical problems is as follows: An image retrieval method based on multi-dimensional feature fusion, comprising the following steps: S1. Import the image to be retrieved and perform multi-dimensional feature extraction processing on the image to be retrieved to obtain multi-dimensional features; S2. Perform feature fusion on multiple dimensions to obtain an image feature representation vector; S3. Based on the similarity between the image feature representation vector and the image feature vector in the preset database, perform image retrieval and output the images with higher similarity as the retrieval results.

[0007] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: By performing multi-dimensional feature extraction on the image, the essential features of the image can be captured from different angles, avoiding the limitations of a single feature extraction method, making the extracted features more comprehensive and richer, and laying a solid foundation for subsequent image retrieval; This method clearly divides the image retrieval process into three main steps: feature extraction, feature fusion, and retrieval matching. Each step has a clear objective and function, forming a complete technical process that ensures the orderly progress of the retrieval work. This retrieval framework based on multi-dimensional feature fusion has strong versatility and can be applied to various types of image retrieval scenarios, playing a good role in various types of image retrieval.

[0008] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: An image retrieval device based on multi-dimensional feature fusion, comprising: The multidimensional feature extraction module is used to import the image to be retrieved and perform multidimensional feature extraction processing on the image to be retrieved to obtain multidimensional features; The feature fusion module is used to fuse features from multiple dimensions to obtain an image feature representation vector. The image retrieval module is used to perform image retrieval based on the similarity between the image feature representation vector and the image feature vector in the preset database, and outputs images with high similarity as retrieval results.

[0009] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: an image retrieval device based on multi-dimensional feature fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image retrieval method based on multi-dimensional feature fusion as described above.

[0010] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the image retrieval method based on multi-dimensional feature fusion as described above.

[0011] The beneficial effects of this invention are: by performing multi-dimensional feature extraction on images, the essential features of images can be captured from different angles, avoiding the limitations of a single feature extraction method, making the extracted features more comprehensive and richer, and laying a solid foundation for subsequent image retrieval. This method clearly divides the image retrieval process into three main steps: feature extraction, feature fusion, and retrieval matching. Each step has a clear objective and function, forming a complete technical process that ensures the orderly progress of the retrieval work. It is applicable to various types of image retrieval scenarios and can play a good role in retrieving various types of images. Attached Figure Description

[0012] Figure 1 A flowchart of an image retrieval method based on multi-dimensional feature fusion provided in an embodiment of the present invention; Figure 2 This is a block diagram of an image retrieval device based on multi-dimensional feature fusion provided in an embodiment of the present invention. Detailed Implementation

[0013] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0014] like Figure 1 As shown, Figure 1 As shown, this embodiment of the invention provides an image retrieval method based on multi-dimensional feature fusion, including the following steps: S1. Import the image to be retrieved and perform multi-dimensional feature extraction processing on the image to be retrieved to obtain multi-dimensional features; S2. Perform feature fusion on multiple dimensions to obtain an image feature representation vector; S3. Based on the similarity between the image feature representation vector and the image feature vector in the preset database, perform image retrieval and output the images with higher similarity as the retrieval results.

[0015] In this embodiment, by performing multi-dimensional feature extraction on the image, the essential features of the image can be captured from different angles, avoiding the limitations of a single feature extraction method, making the extracted features more comprehensive and richer, and laying a solid foundation for subsequent image retrieval. This method clearly divides the image retrieval process into three main steps: feature extraction, feature fusion, and retrieval matching. Each step has a clear objective and function, forming a complete technical process that ensures the orderly progress of the retrieval work. It is applicable to various types of image retrieval scenarios and can play a good role in retrieving various types of images.

[0016] Preferably, in step S1, the multi-dimensional feature extraction process for the image to be retrieved includes: The multi-dimensional feature extraction process includes a visual feature extraction process; the visual feature extraction process includes: The image to be retrieved is resized based on a preset size standard, and the pixel values ​​in the resized image are normalized. The normalized image is then enhanced to obtain the preprocessed image. The preprocessed image is input into a pre-trained EfficientNet-B7 model, and the output of the global average pooling layer in the EfficientNet-B7 model is used as the global visual feature vector. The global visual feature vector is then reduced to 256 dimensions. Keypoints in the preprocessed image are extracted using the ORB feature detection tool, and a 256-dimensional local feature descriptor is generated for each keypoint. Each local feature descriptor is quantized into a visual word, and a visual vocabulary is constructed based on all visual words. A fixed-length BoW vector is generated based on the frequency of occurrence of each visual word in the visual vocabulary. The reduced global visual feature vector and the BoW vector are normalized using a normalization method. The normalized global visual feature vector and the BoW vector are then fused to obtain the visual fusion feature.

[0017] In this embodiment, the global visual feature vector can grasp the macroscopic visual information of the image as a whole, while the local feature descriptor can capture the features of the details in the image. The two complement each other, making the extracted visual features more complete and accurate, and better describing the visual content of the image. By reducing the dimensionality of the global visual feature vector to 256 dimensions, the integrity of the feature information is maintained, the dimensionality of the feature vector is greatly reduced, the efficiency of feature storage and processing is improved, and subsequent feature fusion and retrieval calculations are made more efficient. Preprocessing operations such as resizing, pixel value normalization, and image enhancement performed on the images to be retrieved can improve image quality, eliminate noise and interference that may occur during image acquisition and transmission, and make subsequent feature extraction more accurate.

[0018] Preferably, in step S2, the multi-dimensional feature extraction process for the image to be retrieved further includes: The multi-dimensional feature extraction process includes a semantic feature extraction process; the semantic feature extraction process includes: Text data is extracted from the relevant text description of the image to be retrieved, the text data is cleaned, and the cleaned text data is segmented into words, and each segmented word is tagged with part of speech. Specifically, text data is extracted from the relevant text description of the image to be retrieved. These text data may include the title, description, tags, user comments, etc. of the image. The extracted text data is cleaned to remove noise information such as irrelevant symbols, stop words (such as "of", "and", "a", "the", etc.), and HTML tags. The cleaned text data is tokenized. For English text, natural language processing tools (such as NLTK or SpaCy) are used for word segmentation; for Chinese text, a tokenization tool (such as Jieba) is used for phrase segmentation. Each token is tagged with its词性 information to identify noun, verb, adjective and other词性 information. This step can be achieved through natural language processing tools (such as NLTK's pos_tag or SpaCy's pos attribute).

[0019] Based on the pre-trained ALIGN image-text joint embedding model, the image to be retrieved and the tokens with词性 tags are mapped into the same semantic space, and the semantic feature vector of the image to be retrieved is extracted; Specifically, the image to be retrieved is input into the image encoder part of the ALIGN model to extract the global semantic feature vector of the image. The text description composed of tokens with词性 tags is input into the text encoder part of the ALIGN model to extract the semantic feature vector of the text. The ALIGN model makes the image feature vector and the text feature vector similar in the same semantic space through contrastive learning, so as to extract the semantic feature vector of the image to be retrieved.

[0020] A knowledge graph is constructed using an external knowledge base, and the semantic feature vector in the image to be retrieved is associated with the nodes in the knowledge graph. The knowledge graph is encoded using a GNN (Graph Neural Network) to aggregate the feature vectors of the nodes and the relationship information of the edges, generating a semantic feature enhanced vector.

[0021] Specifically, object detection algorithms (such as YOLO, Faster R-CNN) or image classification models (such as ResNet, EfficientNet) are used to extract the category labels of objects, scenes, and concepts from the image to be retrieved. The extracted category labels are matched with the entity nodes in the knowledge graph to establish an association relationship. For example, if "cat" is detected in the image, the corresponding "cat" entity node is found in the knowledge graph. The neighbor nodes and their relationship information related to the objects, scenes, and concepts in the image are extracted. For example, for the "cat" node, its neighbor nodes "mammal", "pet", etc. and their relationships "belong to", "is", etc. are extracted.

[0022] It should be noted that there seems to be an error in the original text where "词性" is used without proper definition. It's assumed this is a placeholder for a correct term like "word type" or "part of speech". The translation is adjusted accordingly while keeping the original structure and tags intact.Initialize the feature vector for each node in the knowledge graph. Pre-trained word embedding models (such as Word2Vec, GloVe, and BERT) can be used to convert the node's name or description into a vector. For each edge, extract the semantic information of the relationship. Pre-trained language models (such as BERT) can be used to convert relationship descriptions (such as "belongs to" or "is") into vectors. Using the message passing mechanism of a GNN, aggregate the feature vectors of neighboring nodes and the relationship information of the edges to update the feature vector of each node. For example, for the "cat" node, its updated feature vector can be calculated by aggregating the feature vectors and relationship vectors of its neighboring nodes "mammals," "pets," etc. After propagation through multiple layers of GNNs, the feature vector of each node will contain richer semantic information. Fuse the enhanced semantic features of the knowledge graph with the semantic features extracted by the image-text joint embedding model. Weighted summation, concatenation, or nonlinear transformations can be used to fuse semantic features from different sources to generate the final semantic feature vector.

[0023] In this embodiment, the pre-trained ALIGN model maps images and text into the same semantic space, making the semantic features of images and text comparable and consistent, thus enabling semantic understanding and cross-modal retrieval of images. By constructing a knowledge graph using an external knowledge base and encoding the knowledge graph using a GNN, the semantic features in the image can be associated and fused with the rich knowledge in the knowledge graph, which enhances the expressive power of the image's semantic features and makes the semantic information of the image richer and more in-depth. Fine-grained processing of text data, such as cleaning, word segmentation, and part-of-speech tagging, can remove noise information from the text, extract valuable semantic units, and clarify their parts of speech and semantic roles, providing high-quality text data for subsequent semantic feature extraction and knowledge graph construction.

[0024] Preferably, the multi-dimensional feature extraction process for the image to be retrieved in step S2 further includes: The multi-dimensional feature extraction process includes a structural feature extraction process; the structural feature extraction process includes: The bounding box region and its coordinates of each object are extracted from the image to be retrieved based on the Mask R-CNN object detection algorithm. Specifically, the input image to be retrieved is used to extract feature maps using ResNet-50 of Mask R-CNN as the backbone network. The RPN (Region Proposal Network) generates candidate regions, and the bounding box and bounding box coordinates (x1, y1, x2, y2) of each object are obtained through classification, regression and mask prediction.

[0025] Spatial relationship detection is performed on the coordinates of each extracted bounding box to obtain the spatial relationship between objects. The spatial relationship between objects is represented by edges and the object structure relationship graph is constructed by representing objects by nodes. Specifically, spatial relationships include: Adjacency relationship: The distance between the bounding boxes of two objects in the horizontal or vertical direction is less than a preset threshold (e.g., 10 pixels). Containment relationship: The bounding box of one object completely contains another object (such as "the cup on the coffee table"). Top / bottom / left / right relationship: The relative offset of the center position is calculated using the bounding box coordinates (e.g., if the average y-coordinate of object A is greater than that of object B, it is determined to be "top"). Next, construct the structural relationship diagram: Node representation: Each detected object (e.g., "sofa", "TV"); Edges represent the spatial relationships between objects (e.g., "the sofa is to the left of the TV" is represented as a directed edge).

[0026] The bounding box regions of the detected objects are input into a pre-trained ResNet model. The global feature vectors of the bounding box regions are extracted as the initial features of the nodes, and the embedding vectors of the edges are based on predefined relation types, including adjacent relations or containment relations. Specifically, initial node feature extraction: The bounding box region of each object is cropped and then fed into the pre-trained ResNet-101 model; Take the output of the last convolutional layer (e.g., 2048-dimensional features), reduce its dimensionality to 512 dimensions through global average pooling, and use it as the initial feature for the node. Edge relation vector generation: A predefined set of relation types R = {adjacent, contain, above, below, left, right}; For each relation type, the relation words are converted into 128-dimensional embedding vectors using a Word2Vec pre-trained model (e.g., "contains" → [0.2, -0.5, ..., 0.1]).

[0027] Based on the message passing mechanism of GNN, the feature vector of each node is updated by fusing the initial features of neighboring nodes and the relationship vector of the edges. Specifically, the feature update formula for node i in the first layer is: , in, Let i be the set of neighbors of node i. The normalization coefficient is... For learnable weight matrix, For activation functions; After propagation through 3-5 layers of GNN, the features of each node will be fused with the structural information of its k-th order neighbors (k being the number of layers).

[0028] The feature vectors of all updated nodes are aggregated using the average pooling method to generate the global structural feature vector of the image to be retrieved. Specifically, the average pooling method is used to take the mean of all updated node features; The global structural features are normalized to a unit vector using the L2 norm, resulting in a global structural feature vector that integrates the spatial layout information of all objects in the image.

[0029] In this embodiment, by extracting the bounding box coordinates of objects and performing spatial relationship detection, the spatial positional relationships between objects in the image can be explicitly represented, such as adjacent, contained, up and down, left and right, etc., which makes up for the deficiency of insufficient representation of spatial structure information in traditional image retrieval methods. Updating node features based on the message passing mechanism of GNN can make full use of the features of neighboring nodes and the relationship information of edges, so that the features of each node can dynamically integrate the structural information of surrounding nodes, thereby better reflecting the overall structural relationship of objects in the image.

[0030] By aggregating the updated node features using the average pooling method, a global structural feature vector is generated. This vector comprehensively considers the spatial layout and structural relationships of all objects in the image, providing a comprehensive structural feature representation for the image and helping to improve the accuracy of image retrieval.

[0031] Preferably, in step S2, feature fusion is performed on multiple dimensional features to obtain an image feature representation vector, including: The importance of each dimension of features is evaluated, and the weight of each dimension of features is determined based on the evaluation results. Each dimension of features is then multiplied by its corresponding weight to obtain a fused feature vector, which is expressed as follows: , in, To fuse feature vectors, The weights of the visual fusion features, For visual fusion features, To enhance the weights of the semantic feature vector, For semantic feature enhancement vectors, The weights are the global structural feature vectors. This represents the global structural feature vector; The fused feature vector is normalized using the L2 norm to obtain the image feature representation vector, which is expressed as: .

[0032] In this embodiment, by evaluating the importance of features in each dimension and adaptively determining the weight of each feature based on the evaluation results, the importance of different features in image retrieval can be fully considered, so that important features receive more attention during the fusion process, thereby improving the effect of feature fusion. By using a weighted fusion approach to combine features from different dimensions, the unique information of each feature can be preserved while the organic combination of features can be achieved through the adjustment of weights, so that the fused feature vector can better represent the comprehensive features of the image. Normalizing the fused feature vector using the L2 norm can eliminate the influence of the feature vector length on similarity calculation, making the feature vector more fair and accurate in similarity calculation and improving the reliability of retrieval results.

[0033] Preferably, in step S3, image retrieval is performed based on the similarity between the image feature representation vector and the image feature vector in a preset database, and images with high similarity are output as retrieval results, including: The similarity between an image feature vector and the image feature vectors of multiple images in a pre-defined database is calculated using cosine similarity. The similarity is expressed as: , in, For image feature vectors in the preset database, This is the image feature vector of the i-th image in the pre-defined database; Based on the calculated similarity, the images in the preset database are sorted from high to low, and images with similarity greater than the set threshold are selected from the sorting table as the search results.

[0034] In this embodiment, cosine similarity is used to calculate the similarity between the image feature representation vector and the image feature vector in the preset database. This method has low computational complexity and can quickly complete the similarity calculation of a large number of images, meeting the needs of real-time image retrieval. Based on the calculated similarity, the images in the preset database are sorted from high to low, which can put the images most similar to the image to be searched at the top, making it easier for users to quickly find the image they need, improving search efficiency and user experience; By setting a threshold, images with similarity greater than the set threshold are selected from the sorting table as search results. Users can flexibly adjust the threshold according to actual needs, thereby controlling the quantity and quality of search results and meeting the search requirements in different scenarios.

[0035] Example 2: Figure 2As shown, this embodiment of the invention also provides an image retrieval device based on multi-dimensional feature fusion, comprising: The multidimensional feature extraction module is used to import the image to be retrieved and perform multidimensional feature extraction processing on the image to be retrieved to obtain multidimensional features; The feature fusion module is used to fuse features from multiple dimensions to obtain an image feature representation vector. The image retrieval module is used to perform image retrieval based on the similarity between the image feature representation vector and the image feature vector in the preset database, and outputs images with high similarity as retrieval results.

[0036] Preferably, in step S1, the multi-dimensional feature extraction process for the image to be retrieved includes: The multi-dimensional feature extraction process includes a visual feature extraction process; the visual feature extraction process includes: The image to be retrieved is resized based on a preset size standard, and the pixel values ​​in the resized image are normalized. The normalized image is then enhanced to obtain the preprocessed image. The preprocessed image is input into a pre-trained EfficientNet-B7 model, and the output of the global average pooling layer in the EfficientNet-B7 model is used as the global visual feature vector. The global visual feature vector is then reduced to 256 dimensions. Keypoints in the preprocessed image are extracted using the ORB feature detection tool, and a 256-dimensional local feature descriptor is generated for each keypoint. Each local feature descriptor is quantized into a visual word, and a visual vocabulary is constructed based on all visual words. A fixed-length BoW vector is generated based on the frequency of occurrence of each visual word in the visual vocabulary. The reduced global visual feature vector and the BoW vector are normalized using a normalization method. The normalized global visual feature vector and the BoW vector are then fused to obtain the visual fusion feature.

[0037] Example 3: This embodiment of the invention also provides an image retrieval device based on multi-dimensional feature fusion, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image retrieval method based on multi-dimensional feature fusion as described above.

[0038] Example 4: This embodiment of the invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the image retrieval method based on multi-dimensional feature fusion as described above.

[0039] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0040] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0041] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0042] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0043] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0044] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0045] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image retrieval method based on multi-dimensional feature fusion, characterized in that, Includes the following steps: Import the image to be retrieved, and perform multi-dimensional feature extraction processing on the image to be retrieved to obtain multi-dimensional features; Feature fusion is performed on features from multiple dimensions to obtain an image feature representation vector; Image retrieval is performed based on the similarity between the image feature representation vector and the image feature vector in the preset database, and the images with higher similarity are output as the retrieval results.

2. The image retrieval method according to claim 1, characterized in that, Multi-dimensional feature extraction processing is performed on the image to be retrieved, including: The multi-dimensional feature extraction process includes a visual feature extraction process; the visual feature extraction process includes: The image to be retrieved is resized based on a preset size standard, and the pixel values ​​in the resized image are normalized. The normalized image is then enhanced to obtain the preprocessed image. The preprocessed image is input into a pre-trained EfficientNet-B7 model, and the output of the global average pooling layer in the EfficientNet-B7 model is used as the global visual feature vector. The global visual feature vector is then reduced to 256 dimensions. Keypoints in the preprocessed image are extracted using the ORB feature detection tool, and a 256-dimensional local feature descriptor is generated for each keypoint. Each local feature descriptor is quantized into a visual word, and a visual vocabulary is constructed based on all visual words. A fixed-length BoW vector is generated based on the frequency of occurrence of each visual word in the visual vocabulary. The reduced global visual feature vector and the BoW vector are normalized using a normalization method. The normalized global visual feature vector and the BoW vector are then fused to obtain the visual fusion feature.

3. The image retrieval method according to claim 2, characterized in that, The multi-dimensional feature extraction process for the image to be retrieved also includes: The multi-dimensional feature extraction process includes a semantic feature extraction process; the semantic feature extraction process includes: Text data is extracted from the relevant text description of the image to be retrieved, the text data is cleaned, and the cleaned text data is segmented into words, and each segmented word is tagged with part of speech. The pre-trained ALIGN image-text joint embedding model maps the image to be retrieved and the word segmentation with part-of-speech tagging to the same semantic space, and extracts the semantic feature vector of the image to be retrieved. A knowledge graph is constructed using an external knowledge base. The semantic feature vectors in the image to be retrieved are associated with the nodes in the knowledge graph. A GNN graph neural network is used to encode the knowledge graph to aggregate the feature vectors of the nodes and the relationship information of the edges, thereby generating semantic feature enhancement vectors.

4. The image retrieval method according to claim 3, characterized in that, The multi-dimensional feature extraction process for the image to be retrieved also includes: The multi-dimensional feature extraction process includes a structural feature extraction process; the structural feature extraction process includes: The bounding box region and its coordinates of each object are extracted from the image to be retrieved based on the Mask R-CNN object detection algorithm. Spatial relationship detection is performed on the coordinates of each extracted bounding box to obtain the spatial relationship between objects. The spatial relationship between objects is represented by edges and the object structure relationship graph is constructed by representing objects by nodes. The bounding box regions of the detected objects are input into a pre-trained ResNet model. The global feature vectors of the bounding box regions are extracted as the initial features of the nodes, and the embedding vectors of the edges are based on predefined relation types, including adjacent relations or containment relations. Based on the message passing mechanism of GNN, the feature vector of each node is updated by fusing the initial features of neighboring nodes and the relationship vector of the edges. The feature vectors of all updated nodes are aggregated using the average pooling method to generate the global structural feature vector of the image to be retrieved.

5. The image retrieval method according to claim 4, characterized in that, Feature fusion is performed on multiple dimensions of features to obtain an image feature representation vector, including: The importance of each dimension of features is evaluated, and the weight of each dimension of features is determined based on the evaluation results. Each dimension of features is then multiplied by its corresponding weight to obtain a fused feature vector, which is expressed as follows: ， in, To fuse feature vectors, The weights of the visual fusion features, For visual fusion features, To enhance the weights of the semantic feature vector, For semantic feature enhancement vectors, The weights of the global structural feature vectors. This represents the global structural feature vector; The fused feature vector is normalized using the L2 norm to obtain the image feature representation vector, which is expressed as: 。 6. The image retrieval method according to claim 5, characterized in that, Image retrieval is performed based on the similarity between the image feature representation vector and the image feature vector in a pre-defined database. Images with high similarity are output as retrieval results, including: The similarity between an image feature vector and the image feature vectors of multiple images in a pre-defined database is calculated using cosine similarity. The similarity is expressed as: ， in, For image feature vectors in the preset database, This is the image feature vector of the i-th image in the pre-defined database; Based on the calculated similarity, the images in the preset database are sorted from high to low, and images with similarity greater than the set threshold are selected from the sorting table as the search results.

7. An image retrieval device based on multi-dimensional feature fusion, characterized in that, include: The multidimensional feature extraction module is used to import the image to be retrieved and perform multidimensional feature extraction processing on the image to be retrieved to obtain multidimensional features; The feature fusion module is used to fuse features from multiple dimensions to obtain an image feature representation vector. The image retrieval module is used to perform image retrieval based on the similarity between the image feature representation vector and the image feature vector in the preset database, and outputs images with high similarity as retrieval results.

8. The image retrieval device according to claim 7, characterized in that, Multi-dimensional feature extraction processing is performed on the image to be retrieved, including: The multi-dimensional feature extraction process includes a visual feature extraction process; the visual feature extraction process includes: The image to be retrieved is resized based on a preset size standard, and the pixel values ​​in the resized image are normalized. The normalized image is then enhanced to obtain the preprocessed image. The preprocessed image is input into a pre-trained EfficientNet-B7 model, and the output of the global average pooling layer in the EfficientNet-B7 model is used as the global visual feature vector. The global visual feature vector is then reduced to 256 dimensions. Keypoints in the preprocessed image are extracted using the ORB feature detection tool, and a 256-dimensional local feature descriptor is generated for each keypoint. Each local feature descriptor is quantized into a visual word, and a visual vocabulary is constructed based on all visual words. A fixed-length BoW vector is generated based on the frequency of occurrence of each visual word in the visual vocabulary. The reduced global visual feature vector and the BoW vector are normalized using a normalization method. The normalized global visual feature vector and the BoW vector are then fused to obtain the visual fusion feature.

9. An image retrieval device based on multi-dimensional feature fusion, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image retrieval method based on multi-dimensional feature fusion as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the image retrieval method based on multi-dimensional feature fusion as described in any one of claims 1 to 6.

Images