Image retrieval method based on fusion of multi-scale features and spatial attention mechanism

Abstract

The present invention discloses an image retrieval method based on the fusion of multi-scale features and spatial attention mechanism, and relates to the field of image retrieval technology. By preprocessing and extracting features of demand text data, user needs are accurately captured. Multi-scale feature compression extraction is performed using a graph neural network to capture details and global information in the image, thereby improving the expressiveness and accuracy of image features. Text and image features are mapped to the same embedding space through a multimodal alignment model to achieve efficient alignment of cross-modal retrieval. Image matching indexes are obtained by calculating similarity, and the time decay and confidence of images are analyzed to improve the accuracy and timeliness of retrieval results. At the same time, the retrieval results are graded and sorted to improve user experience. The validity of the retrieval results is judged by the user feedback index. When the retrieval results do not meet user needs, the images are retrieved again to improve the accuracy of image retrieval.

Description

Technical Field

[0001] The present invention relates to the field of image retrieval technology, and in particular to an image retrieval method based on the fusion of multi-scale features and spatial attention mechanism. Background Art

[0002] With the popularization of the Internet and smart devices, the amount of global image and video data has exploded. Social media platforms, e-commerce websites, medical imaging systems, smart city monitoring, etc. have all generated a large amount of image data. Processing and effectively retrieving information from them has become an urgent problem to be solved.

[0003] For example, the Chinese invention patent with the announcement number CN113868449B discloses an image retrieval method based on the fusion of multi-scale features and spatial attention mechanism, which is specifically as follows: Step 1, extracting the features of the image as local features α through a convolutional neural network; Step 2, transmitting the local features α to the global average pooling layer output as global features β; Step 3, processing the local features α using the attention mechanism module of the convolution module to obtain local fusion features γ; Step 4, fusing γ and β to obtain the final features λ; Step 5, extracting the final features λ of the image to be queried and the database image according to the method of steps 1-4, and using the Manhattan distance formula to calculate the distance between the final features λ of each image in the image to be queried and the database image, and then sorting the obtained distances from small to large to obtain matching results. The present invention can reasonably allocate computing resources according to the content in the image when computing power is limited, so that the prominent parts in the image get more attention.

[0004] However, existing image retrieval methods have difficulty in accurately aligning and matching the features of image and text modalities, resulting in the inability to fully express the complex information of the image. At the same time, they ignore the impact of image temporal correlation and confidence on image retrieval, resulting in inaccurate image retrieval. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides an image retrieval method based on the fusion of multi-scale features and spatial attention mechanism, which solves the problems of the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: an image retrieval method based on the fusion of multi-scale features and spatial attention mechanism, comprising the following steps: S1. obtaining user's demand text data for image retrieval, preprocessing and feature extraction of the demand text data, obtaining a retrieval text feature vector, performing multi-scale feature compression extraction on the image to be retrieved through a graph neural network, and obtaining a feature vector of the image to be retrieved; S2. mapping the retrieval text feature vector and the image to be retrieved feature vector to the same embedding space through a preset multimodal alignment model, performing vector feature extraction on the retrieval text feature vector and the image to be retrieved feature vector; S1. Perform feature matching to obtain the matching index of the image to be retrieved; S2. Screen the image to be retrieved based on the matching index of the image to be retrieved, obtain the retrieved image, analyze the time decay and confidence of the retrieved image, and obtain the comprehensive retrieval index of the retrieved image; S3. Classify and sort the comprehensive retrieval index of the retrieved image, display it to the user, and obtain the user feedback index based on the user feedback information; S4. Based on the user feedback index, determine whether the retrieved image meets the user's demand for image retrieval. If not, obtain the user's modified text data for image retrieval, and re-retrieve the image according to the method of steps S1-S4.

[0007] Furthermore, the specific process of preprocessing and feature extraction of the demand text data to obtain the retrieval text feature vector is as follows: preprocessing the demand text data includes text cleaning, word segmentation, stem extraction and word form restoration; feature extraction of the demand text data is performed through a pre-trained word embedding model to obtain the retrieval text feature vector.

[0008] Furthermore, the specific process of performing multi-scale feature compression and extraction on the image to be retrieved through the graph neural network and obtaining the feature vector of the image to be retrieved is as follows: load the image to be retrieved and adjust it to a standard size, use the convolutional neural network to extract the preliminary feature map of the image to be retrieved, and convert the preliminary feature map into a graph structure; use the graph neural network to perform multi-scale feature extraction and compression on the graph structure, and integrate the spatial attention mechanism through the graph convolution layer to perform weighted adjustment on the feature map, extract global features, and obtain the feature vector of the image to be retrieved.

[0009] Furthermore, the specific process of vector feature matching between the retrieval text feature vector and the image feature vector to be retrieved is as follows: map the retrieval text feature vector and the image feature vector to be retrieved to the same embedding space for feature alignment, calculate the cosine value of the angle between the retrieval text feature vector and the image feature vector to be retrieved, quantify the similarity, and obtain the vector cosine similarity; calculate the Euclidean distance between the retrieval text feature vector and the image feature vector to be retrieved, and obtain the vector Euclidean distance.

[0010] Furthermore, the specific process of obtaining the matching index of the image to be retrieved is as follows: based on the vector cosine similarity and the vector Euclidean distance, a comprehensive operation is performed with the set vector cosine similarity threshold and the vector Euclidean distance threshold respectively to obtain the matching index of the image to be retrieved.

[0011] Furthermore, the images to be retrieved are screened based on their matching indexes, and the specific process of obtaining the retrieved images is as follows: the matching index of the images to be retrieved is compared with a set threshold of the matching index of the images to be retrieved; when the matching index of the images to be retrieved is greater than the threshold of the matching index of the images to be retrieved, the images to be retrieved are marked and the retrieved images are obtained.

[0012] Furthermore, the time decay and confidence of the retrieval image are analyzed, and the specific process of obtaining the comprehensive retrieval index of the retrieval image is as follows: a decay function is selected to simulate the influence of time on the time correlation of the retrieval image, and the time decay index of the retrieval image is obtained; the confidence index of the retrieval image is obtained through the confidence output of the convolutional neural network; based on the matching index of the image to be retrieved, the time decay index of the retrieval image and the confidence index of the retrieval image are comprehensively processed with the matching index of the image to be retrieved to obtain the comprehensive retrieval index of the retrieval image.

[0013] Furthermore, the specific process of grading and sorting the comprehensive retrieval index of the retrieved image is as follows: based on the comprehensive retrieval index of the retrieved image, compare it with the set retrieval index grade value; when the comprehensive retrieval index of the retrieved image is greater than or equal to the first retrieval grade value, the image to be retrieved is divided into a high-level sequence; when the comprehensive retrieval index of the retrieved image is less than the first retrieval grade value and greater than or equal to the second retrieval grade value, the image to be retrieved is divided into a medium-level sequence; when the comprehensive retrieval index of the retrieved image is less than the second retrieval grade value, the image to be retrieved is divided into a low-level sequence; and the images in different grade sequences are sorted from large to small according to the comprehensive retrieval index of the retrieved image.

[0014] Furthermore, the specific process of judging whether the retrieved image meets the user's demand for image retrieval is as follows: based on the user feedback index, it is compared with the set standard value of the user feedback index. If the user feedback index is greater than or equal to the standard value of the user feedback index, it means that the retrieved image meets the user's demand for image retrieval.

[0015] The present invention has the following beneficial effects:

[0016] (1) This image retrieval method based on the fusion of multi-scale features and spatial attention mechanism can accurately capture user needs by preprocessing and extracting features from the demand text data, ensuring that the retrieval process is clear in purpose. Using graph neural networks for multi-scale feature compression extraction can capture details and global information in the image and improve the expressiveness and accuracy of image features. Through the multimodal alignment model, text and image features can be mapped to the same embedding space to achieve efficient alignment for cross-modal retrieval. In the process of vector feature matching, the image matching index is obtained by calculating the similarity to ensure the relevance and accuracy of the retrieval results.

[0017] (2) This image retrieval method based on the fusion of multi-scale features and spatial attention mechanism can classify and sort the retrieval results, so that users can quickly identify the most relevant images and improve the user experience. Collect user feedback information and obtain user feedback index, which can dynamically adjust the retrieval model and results and continuously optimize system performance. The validity of the retrieval results is judged by the user feedback index to ensure that the system can continuously learn and adapt to user needs. When the retrieval results do not meet user needs, the text data is re-acquired and modified, and the retrieval is performed again to ensure the flexibility and adaptability of the system.

[0018] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of the image retrieval method based on the fusion of multi-scale features and spatial attention mechanism of the present invention.

[0020] Figure 2 The present invention is a flowchart for obtaining a comprehensive retrieval index for a retrieval image. DETAILED DESCRIPTION

[0021] The embodiment of the present application solves the problems of difficulty in aligning features between text and image, poor matching accuracy and difficulty in processing complex information, and the impact of image time correlation and confidence on image retrieval in image retrieval through an image retrieval method based on the fusion of multi-scale features and spatial attention mechanism, thereby improving the overall performance of image retrieval and user experience.

[0022] The overall idea of ​​the problem in the embodiment of this application is as follows:

[0023] Obtain the user's demand text data for image retrieval, preprocess and extract features of the demand text data, obtain the retrieval text feature vector, perform multi-scale feature compression extraction on the image to be retrieved through the graph neural network, and obtain the feature vector of the image to be retrieved.

[0024] The retrieved text feature vector and the to-be-retrieved image feature vector are mapped to the same embedding space by a preset multimodal alignment model, and vector feature matching is performed on the retrieved text feature vector and the to-be-retrieved image feature vector to obtain the to-be-retrieved image matching index.

[0025] The images to be retrieved are screened based on the matching index of the images to be retrieved, the retrieved images are obtained, the time decay and confidence of the retrieved images are analyzed, and the comprehensive retrieval index of the retrieved images is obtained.

[0026] The comprehensive retrieval index of the retrieved images is graded and sorted, and displayed to the user, and the user feedback index is obtained based on the user feedback information.

[0027] Based on the user feedback index, it is determined whether the retrieved image meets the user's requirements for image retrieval. If not, the user's modified text data for image retrieval is obtained and the image retrieval is performed again.

[0028] See also Figure 1 , an embodiment of the present invention provides a technical solution: an image retrieval method based on the fusion of multi-scale features and spatial attention mechanism, comprising the following steps: S1. obtaining user demand text data for image retrieval, preprocessing and feature extraction of the demand text data, obtaining a retrieval text feature vector, performing multi-scale feature compression extraction on the image to be retrieved through a graph neural network, and obtaining a feature vector of the image to be retrieved; S2. mapping the retrieval text feature vector and the feature vector of the image to be retrieved to the same embedding space through a preset multimodal alignment model, performing vector feature matching on the retrieval text feature vector and the feature vector of the image to be retrieved, and obtaining a matching index of the image to be retrieved; S3. screening the image to be retrieved based on the matching index of the image to be retrieved, obtaining a retrieval image, analyzing the time decay and confidence of the retrieval image, and obtaining a comprehensive retrieval index of the retrieval image; S4. classifying and sorting the comprehensive retrieval index of the retrieval image, displaying it to the user, and obtaining a user feedback index based on the user feedback information; S5. judging whether the retrieval image meets the user's demand for image retrieval based on the user feedback index, if not, obtaining the user's modified text data for image retrieval, and re-performing image retrieval according to the method of steps S1-S4.

[0029] In this embodiment, the demand text data represents the text information input by the user to describe his image retrieval requirements. The graph neural network is a neural network model for processing graph structure data, which can capture the relationship between nodes and edges in the graph and is widely used in the fields of social network analysis and recommendation systems. Multi-scale features are image features extracted at different scales, which can more comprehensively describe the image content. Multimodal alignment model is a model that can map features of different modalities (such as text and images) to the same embedding space to achieve cross-modal data alignment and matching. Embedding space is a high-dimensional vector space used to represent the numerical features of data. After being mapped to the same embedding space, different types of data can be compared and matched by similarity metrics. Vector feature matching refers to calculating the similarity or distance between two vectors in the embedding space. Common methods include cosine similarity, Euclidean distance, etc., which are used to measure the similarity of data. Time decay is a property of an image that may no longer be relevant or important over time. For example, an older news picture may no longer be relevant to the current event. Confidence represents a measure of the reliability and accuracy of the retrieval results, usually based on the model's confidence level in predicting a specific image. The user feedback index is an indicator calculated based on user feedback information, which is used to evaluate and adjust the effectiveness of system retrieval results.

[0030] Specifically, the specific process of preprocessing and feature extraction of demand text data to obtain the retrieval text feature vector is as follows: preprocessing of demand text data includes text cleaning, word segmentation, stem extraction and word form restoration; feature extraction of demand text data is performed through a pre-trained word embedding model to obtain the retrieval text feature vector.

[0031] In this embodiment, text cleaning refers to the removal of irrelevant characters and noise data in the text, such as punctuation marks, HTML tags, special symbols, etc. The purpose of this step is to simplify the text so that subsequent processing is more accurate and efficient. Word segmentation is the process of dividing a continuous text string into meaningful words or phrases. Different languages ​​have different word segmentation methods. For example, for English, words can be segmented according to spaces and punctuation marks; for Chinese, a special word segmentation algorithm is required. Feature extraction is the process of converting preprocessed text data into a numerical representation. Through a pre-trained word embedding model, each word or entire sentence can be mapped to a high-dimensional vector space to generate a vector representing the text features.

[0032] Specifically, the specific process of performing multi-scale feature compression and extraction on the image to be retrieved through the graph neural network and obtaining the feature vector of the image to be retrieved is as follows: load the image to be retrieved and adjust it to a standard size, use the convolutional neural network to extract the preliminary feature map of the image to be retrieved, and convert the preliminary feature map into a graph structure; use the graph neural network to perform multi-scale feature extraction and compression on the graph structure, and integrate the spatial attention mechanism through the graph convolution layer to perform weighted adjustment on the feature map, extract global features, and obtain the feature vector of the image to be retrieved.

[0033] In this embodiment, the image to be retrieved is loaded from a file or database into memory, and then the image is adjusted to a predefined standard size to ensure the consistency and efficiency of subsequent processing. The standard size is a fixed width and height. The adjusted image is processed using a pre-trained convolutional neural network to extract a preliminary feature map. The convolutional neural network captures edges, textures and other low-level features in the image through a series of convolutional layers, pooling layers and activation functions. The feature map extracted by the convolutional neural network is converted into a graph structure, in which each feature point is used as a node of the graph, and the connection relationship between the nodes can be determined based on spatial proximity or feature similarity. The graph structure representation can better capture the local and global relationships of image features. The graph structure is used to extract and compress multi-scale features of the graph structure. Through an iterative message passing mechanism, the information of neighboring nodes is fused to capture the local and global features of the nodes in the graph. The spatial attention mechanism is integrated in the graph convolution layer to dynamically adjust the weight of each node feature. The spatial attention mechanism can assign different weights according to the correlation between the node feature and its neighbor feature, thereby highlighting important features and suppressing unimportant features. After being processed by the graph neural network and the spatial attention mechanism, the extracted features contain the multi-scale information and spatial relationships of the image. These features can be further globally pooled to generate a feature vector representing the entire image.

[0034] Specifically, the specific process of vector feature matching between the retrieval text feature vector and the image feature vector to be retrieved is as follows: map the retrieval text feature vector and the image feature vector to be retrieved to the same embedding space for feature alignment, calculate the cosine value of the angle between the retrieval text feature vector and the image feature vector to be retrieved, quantify the similarity, and obtain the vector cosine similarity; calculate the Euclidean distance between the retrieval text feature vector and the image feature vector to be retrieved, and obtain the vector Euclidean distance.

[0035] In this implementation scheme, the calculation formula of cosine similarity and Euclidean distance is used, the retrieval text feature vector is Te, the retrieval image feature vector is Tc, the dimension is n, and the dimensions are numbered 1, 2, 3...i...n in sequence; the calculation formulas for calculating vector cosine similarity and vector Euclidean distance are as follows: , where represents the vector cosine similarity, represents the dot product of vectors, Represents the Euclidean norm of a vector; , where represents the i-th component of the vector Te and, represents the i-th component of the vector Tc and, Represents the Euclidean distance of a vector. Gets the cosine similarity of a vector and vector Euclidean distance .

[0036] Specifically, the specific process of obtaining the matching index of the image to be retrieved is as follows: based on the vector cosine similarity and the vector Euclidean distance, a comprehensive operation is performed with the set vector cosine similarity threshold and the vector Euclidean distance threshold respectively to obtain the matching index of the image to be retrieved.

[0037] In this implementation, the calculation formula of the matching index of the image to be retrieved is as follows:

[0038] , where represents the matching index of the image to be retrieved, which is used to measure the matching degree between the image and the feature vector of the retrieved text. bY represents the vector cosine similarity threshold. Indicates the allowed deviation value of vector cosine similarity, represents the vector cosine similarity, Indicates the allowed deviation value of the vector Euclidean distance, DG indicates the vector Euclidean distance threshold, Represents the Euclidean distance between vectors. The allowed deviation value indicates how different two vectors can be and still be accepted as similar.

[0039] Specifically, the images to be retrieved are screened based on their matching indexes, and the specific process of obtaining the retrieved images is as follows: the matching index of the images to be retrieved is compared with a set threshold of the matching index of the images to be retrieved; when the matching index of the images to be retrieved is greater than the threshold of the matching index of the images to be retrieved, the images to be retrieved are marked and the retrieved images are obtained.

[0040] In this implementation, the matching index calculated for each image to be retrieved is compared with the threshold of the matching index of the image to be retrieved. By setting a reasonable threshold, images with low relevance to the retrieval text can be effectively filtered out, and only those with high relevance can be retained. This helps to improve the accuracy of the retrieval results and ensure that the results obtained by the user are more in line with their needs.

[0041] See also Figure 2Specifically, the time decay and confidence of the retrieval image are analyzed, and the specific process of obtaining the comprehensive retrieval index of the retrieval image is as follows: a decay function is selected to simulate the influence of time on the time correlation of the retrieval image, and the time decay index of the retrieval image is obtained; the confidence index of the retrieval image is obtained through the confidence output of the convolutional neural network; based on the matching index of the image to be retrieved, the time decay index of the retrieval image and the confidence index of the retrieval image are comprehensively processed with the matching index of the image to be retrieved to obtain the comprehensive retrieval index of the retrieval image.

[0042] In this implementation, the calculation formula for the time decay index of the retrieval image is as follows: , where represents the temporal decay index of the retrieved image, represents the decay factor, e represents the natural constant, and t represents the time since the image was last updated. The output confidence value of the CNN model is extracted as the confidence index of the retrieved image. , the calculation formula of the comprehensive retrieval index of the retrieved image is as follows:

[0043] , where represents the comprehensive retrieval index of the retrieved image, Indicates the matching index of the image to be retrieved, and 0< <1, represents the temporal decay index of the retrieved image, Represents the confidence index of the retrieved image.

[0044] Specifically, the specific process of grading and sorting the comprehensive retrieval index of the retrieval image is as follows: based on the comprehensive retrieval index of the retrieval image, compare it with the set retrieval index level value; when the comprehensive retrieval index of the retrieval image is greater than or equal to the first retrieval level value, divide the image to be retrieved into a high-level sequence; when the comprehensive retrieval index of the retrieval image is less than the first retrieval level value and greater than or equal to the second retrieval level value, divide the image to be retrieved into a medium-level sequence; when the comprehensive retrieval index of the retrieval image is less than the second retrieval level value, divide the image to be retrieved into a low-level sequence; sort the images in different level sequences from large to small according to the comprehensive retrieval index of the retrieval image.

[0045] In this implementation, the searched images are divided into different hierarchical sequences according to their comprehensive search indexes, so as to facilitate subsequent display and sorting. The hierarchical division and sorting help to display the search results according to relevance and quality, ensuring that users see the most relevant images first. Sorting by comprehensive search index can effectively sort according to the comprehensive evaluation results of the images, avoiding the deviation that may be caused by a single indicator.

[0046] Specifically, the specific process of judging whether the retrieved image meets the user's demand for image retrieval is as follows: based on the user feedback index, it is compared with the set standard value of the user feedback index. If the user feedback index is greater than or equal to the standard value of the user feedback index, it means that the retrieved image meets the user's demand for image retrieval.

[0047] In this implementation, by comparing the user feedback index with the standard value, it is possible to accurately determine whether the search results meet the user's expectations. The standard value of the user feedback index can be adjusted according to actual usage and user needs, and the quality and relevance of the searched images can be timely determined and adjusted, and optimized according to user feedback, reflecting the ability of the search system to adapt to changes in user needs.

[0048] In summary, this application has at least the following effects:

[0049] The image retrieval method based on the fusion of multi-scale features and spatial attention mechanism can accurately capture user needs by preprocessing and extracting features of the required text data, ensuring that the goal of the retrieval process is clear. The use of graph neural networks for multi-scale feature compression extraction can capture the details and global information in the image and improve the expressiveness and accuracy of image features. The multimodal alignment model can map text and image features to the same embedding space to achieve efficient alignment of cross-modal retrieval. In the process of vector feature matching, the image matching index is obtained by calculating the similarity to ensure the relevance and accuracy of the retrieval results. The retrieval results are graded and sorted so that users can quickly identify the most relevant images and improve the user experience. Collecting user feedback information and obtaining the user feedback index can dynamically adjust the retrieval model and results and continuously optimize system performance. The user feedback index is used to judge the effectiveness of the retrieval results to ensure that the system can continuously learn and adapt to user needs. When the retrieval results do not meet user needs, the modified text data is retrieved and the retrieval is performed again to ensure the flexibility and adaptability of the system.

[0050] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0051] The present invention is described with reference to flowcharts and / or block diagrams of systems, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0052] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0053] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0054] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0055] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An image retrieval method based on the fusion of multi-scale features and spatial attention mechanism, characterized in that: The following steps are involved: S1. Obtain the user's demand text data for image retrieval, preprocess and extract features of the demand text data, obtain the retrieval text feature vector, perform multi-scale feature compression extraction on the image to be retrieved through the graph neural network, and obtain the feature vector of the image to be retrieved; S2. Mapping the retrieval text feature vector and the image feature vector to be retrieved to the same embedding space through a preset multimodal alignment model, performing vector feature matching on the retrieval text feature vector and the image feature vector to be retrieved, and obtaining a matching index of the image to be retrieved; S3. Screening the images to be retrieved based on the matching index of the images to be retrieved, obtaining the retrieved images, analyzing the time decay and confidence of the retrieved images, and obtaining the comprehensive retrieval index of the retrieved images; S4. grading and ranking the comprehensive retrieval index of the retrieved image, displaying it to the user, and obtaining the user feedback index according to the user feedback information; S5. Based on the user feedback index, determine whether the retrieved image meets the user's needs for image retrieval. If not, obtain the user's modified text data for image retrieval and re-perform image retrieval according to the method of steps S1-S4; The specific process of performing multi-scale feature compression extraction on the image to be retrieved through the graph neural network and obtaining the feature vector of the image to be retrieved is as follows: Load the image to be retrieved and resize it to a standard size, use a convolutional neural network to extract the preliminary feature map of the image to be retrieved, and convert the preliminary feature map into a graph structure; Use graph neural networks to extract and compress multi-scale features of graph structures, integrate spatial attention mechanisms through graph convolutional layers, perform weighted adjustments on feature maps, extract global features, and obtain feature vectors of images to be retrieved; The specific process of analyzing the time decay and confidence of the retrieved image to obtain the comprehensive retrieval index of the retrieved image is as follows: Select a decay function to simulate the influence of time on the temporal correlation of the retrieved image, and obtain the temporal decay index of the retrieved image; Obtain the confidence index of the retrieved image through the confidence output by the convolutional neural network; Based on the matching index of the image to be retrieved, the time decay index of the retrieval image and the confidence index of the retrieval image are comprehensively processed with the matching index of the image to be retrieved to obtain a comprehensive retrieval index of the retrieval image.

2. The image retrieval method based on fusion of multi-scale features and spatial attention mechanism according to claim 1, characterized in that: The specific process of preprocessing and feature extraction of the required text data and obtaining the retrieval text feature vector is as follows: Preprocessing of demand text data includes text cleaning, word segmentation, stem extraction and word form restoration; The feature extraction of the demand text data is performed through the pre-trained word embedding model to obtain the retrieval text feature vector.

3. The image retrieval method based on fusion of multi-scale features and spatial attention mechanism according to claim 2, characterized in that: The specific process of performing vector feature matching on the retrieved text feature vector and the image feature vector to be retrieved is as follows: Map the feature vector of the retrieved text and the feature vector of the image to be retrieved to the same embedding space for feature alignment, calculate the cosine value of the angle between the feature vector of the retrieved text and the feature vector of the image to be retrieved, quantify the similarity, and obtain the vector cosine similarity; The Euclidean distance between the feature vector of the retrieved text and the feature vector of the image to be retrieved is calculated to obtain the vector Euclidean distance.

4. The image retrieval method based on fusion of multi-scale features and spatial attention mechanism according to claim 3 is characterized in that: The specific process of obtaining the matching index of the image to be retrieved is as follows: Based on the vector cosine similarity and the vector Euclidean distance, a comprehensive operation is performed with the set vector cosine similarity threshold and the vector Euclidean distance threshold to obtain the matching index of the image to be retrieved.

5. The image retrieval method based on fusion of multi-scale features and spatial attention mechanism according to claim 4, characterized in that: The specific process of screening the images to be retrieved based on the matching index of the images to be retrieved and obtaining the retrieved images is as follows: The matching index of the image to be retrieved is compared with the set threshold of the matching index of the image to be retrieved. When the matching index of the image to be retrieved is greater than the threshold of the matching index of the image to be retrieved, the image to be retrieved is marked and the retrieved image is obtained.

6. The image retrieval method based on fusion of multi-scale features and spatial attention mechanism according to claim 5, characterized in that: The specific process of grading and sorting the comprehensive retrieval index of the retrieved image is as follows: The comprehensive retrieval index based on the retrieval image is compared with the set retrieval index level value; When the comprehensive retrieval index of the retrieval image is greater than or equal to the first retrieval level value, the image to be retrieved is divided into a high-level sequence; When the comprehensive retrieval index of the retrieval image is less than the first retrieval level value and greater than or equal to the second retrieval level value, the image to be retrieved is divided into a medium-level sequence; When the comprehensive retrieval index of the retrieval image is less than the second retrieval level value, the image to be retrieved is divided into a low-level sequence; Sort the images in different level sequences from large to small according to the comprehensive retrieval index of the retrieval image.

7. The image retrieval method based on fusion of multi-scale features and spatial attention mechanism according to claim 6, characterized in that: The specific process of judging whether the retrieved image meets the user's requirements for image retrieval is as follows: Based on the user feedback index, it is compared with the set standard value of the user feedback index. If the user feedback index is greater than or equal to the standard value of the user feedback index, it means that the retrieved image meets the user's demand for image retrieval.

Images