Cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction

Abstract

The application relates to the field of information retrieval, and discloses a cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction. The method acquires a retrieval query containing a reference image and modified text and a candidate image library and carries out multimodal standard preprocessing; positive and negative alignment double-branch decoupling is used to extract field-related features and cross-domain shared field-independent features, to acquire positive and negative global and image-text local features; multi-view local semantics are extracted for the local features, a concept anchor point is obtained by weighted sharing of a concept prototype matrix, and cross-attention is used to reconstruct instance-level local concept features; the positive and negative global and local concept features are spliced to obtain joint retrieval embedding, and the inner product similarity is calculated to recall matching target images. The application alleviates field deviation through global double-branching, improves the utilization efficiency of low-resource samples by using shared concept reconstruction, and effectively improves the cross-domain retrieval accuracy and robustness.

Description

Technical Field

[0001] This invention relates to the field of information retrieval technology, and in particular to a cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction. Background Technology

[0002] Image retrieval aims to retrieve target images from an image library that are similar to a query image provided by a user. However, in practical applications, users often do not want results that are completely similar to the reference image, but rather want to make specific modifications while retaining some content of the reference image. Traditional image retrieval only uses single-modal information of the image for matching, which cannot express such complex retrieval needs that include modification intentions.

[0003] Combined image retrieval is a multimodal retrieval technology that combines visual and textual information. Its goal is to retrieve the corresponding target image from a candidate image library based on the retrieval intent jointly described by a reference image and modified text. The reference image provides the basic visual content of the retrieval target, while the modified text describes the changes in the target relative to the reference image. Combined image retrieval can utilize information from both image and text modalities to express more flexible and complex user needs, offering stronger retrieval capabilities compared to traditional single-modal image retrieval.

[0004] However, most existing combined image retrieval methods are based on the assumption that there is sufficient training data and that the training and test sets originate from the same domain. In practical applications, directly acquiring large-scale manually labeled samples from new domains is usually costly, while existing domains typically have accumulated rich training data. Therefore, how to leverage existing domain knowledge and a small amount of new domain reference data to improve the performance of combined image retrieval in new domains has become an important issue in cross-domain, low-resource combined image retrieval research. Studying this issue can help reduce the cost of relying on manual annotation and improve the deployment efficiency of combined image retrieval systems in new domains.

[0005] Cross-domain low-resource image retrieval faces challenges in two aspects: difficulty in transferring knowledge from the source domain and difficulty in utilizing knowledge from the target domain. On the one hand, there are significant differences in image content, text descriptions, and the way combined intent is expressed across different domains. This leads to the retrieval representations learned by the model easily containing a large amount of domain-related information, resulting in domain bias and affecting the transfer of knowledge from the source domain to the target domain. On the other hand, although different domains have their own unique semantic features, there are still shareable and transferable conceptual units. Insufficient utilization of the potential associations and alignments of these conceptual units reduces the utilization rate of low-resource reference annotations in the target domain, thereby limiting the model's generalization ability to samples from the target domain. In addition, annotation bias and noise in both the source and target domain annotations can also affect the model training process, reducing the model's stability and robustness.

[0006] Based on the above description, significant semantic distribution differences exist between different domains in cross-domain low-resource scenarios, necessitating further optimization of the cross-domain knowledge transfer process to improve the transferability of source domain knowledge to the target domain. Insufficient utilization of shared semantic knowledge between different domains requires further optimization of the cross-domain semantic modeling process to improve the efficiency of utilizing limited labeled information in the target domain. Labeling bias and noise in the training data affect the model learning process, requiring further optimization of the retrieval representation learning process to enhance the stability and robustness of the retrieval model. These findings have significant research and application value for improving the performance of combined image retrieval in cross-domain low-resource scenarios. Summary of the Invention

[0007] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides a cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction.

[0008] A cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction includes: Obtain a search query and a candidate image library, wherein the search query includes reference images and modified text; Data preprocessing is performed on the reference image, the modified text, and the candidate images in the candidate image library to generate a normalized multimodal input; The normalized multimodal input is subjected to feature extraction and fusion using positive alignment branches and negative alignment branches to obtain positive global features, negative global features, and local features corresponding to the retrieval query, as well as positive global features, negative global features, and local features corresponding to the candidate image. The positive alignment branch is used to extract domain-related features, and the negative alignment branch is used to extract domain-independent features shared across domains. From the query local features and the candidate local features, extract the query local semantic perspective and the candidate local semantic perspective from multiple perspectives, respectively; The probability distribution of query concepts is predicted using the local semantic perspective of the query, and the probability distribution of candidate concepts is predicted using the local semantic perspective of the candidate. The shared concept prototype matrix is ​​weighted and aggregated using the probability distribution of the query concept to obtain the query concept anchor point; the shared concept prototype matrix is ​​weighted and aggregated using the probability distribution of the candidate concept to obtain the candidate concept anchor point. The query local features and candidate local features are cross-attention calculated using the query concept anchor and candidate concept anchor respectively, the corresponding instance-level semantic expressions are extracted, and the query local concept features and candidate local concept features are aggregated. The positive query global features, the negative query global features, and the query local concept features are concatenated to obtain the query joint retrieval embedding; and the positive candidate global features, the negative candidate global features, and the candidate local concept features are concatenated to obtain the candidate joint retrieval embedding. Calculate the similarity between the query joint retrieval embedding and each of the candidate joint retrieval embeddings; select the candidate image with the highest similarity from the candidate image library as the target image based on the similarity.

[0009] Furthermore, the step of preprocessing the reference image, the modified text, and the candidate images in the candidate image library to generate a standardized multimodal input includes: performing character removal and spell correction processing on the modified text, and replacing the corrected spellings with standard word forms; if a single search query contains multiple modified texts, merging the multiple modified texts into a single modified text using a preset connection symbol; and uniformly scaling the reference image and candidate images to a preset standard size and converting them into standard color format images.

[0010] Furthermore, a pre-trained visual language model is used to extract the positive image query global features of the reference image, the positive text query global features of the modified text, and the positive image candidate global features of the candidate image. The positive image candidate global features are projected onto the text semantic space to obtain the positive text candidate global features; The weight prediction network is used to calculate modal weights and perform weighted fusion on the global features of the positive image query and the global features of the positive text query to obtain the global features of the positive query; the weight prediction network is used to calculate modal weights and perform weighted fusion on the global features of the candidate global features of the positive image and the global features of the candidate global features of the positive text to obtain the global features of the candidate global features. In the same batch of samples, construct positive and negative sample pairs for query and candidate, calculate the similarity between the global features of the positive query and the global features of the positive candidate, and use batch contrastive learning to increase the similarity of positive sample pairs and decrease the similarity of negative sample pairs, thereby co-optimizing the global features of the positive query and the global features of the positive candidate.

[0011] Furthermore, after obtaining the positive query global features corresponding to the retrieval query and the positive candidate global features corresponding to the candidate images, the method further includes an optimization step based on the positive probability distribution: A positive gradient scaling coefficient is introduced for both the forward query global feature and the forward candidate global feature. During backpropagation, the degree of gradient optimization is adjusted using the positive gradient scaling coefficient. The domain discriminator is used to predict the positive probability distribution of the query data source corresponding to the positive global query feature and the positive probability distribution of the candidate data source corresponding to the positive candidate global feature; The determination of the positive probability distribution is optimized using cross-entropy loss to enhance the expressive power of the domain-related features.

[0012] Furthermore, the negative alignment branch adopts the same structure as the positive alignment branch, and the specific steps include: The negative image query global features of the reference image, the negative text query global features of the modified text, and the negative image candidate global features of the candidate image are extracted using a pre-trained visual language model. The negative image candidate global features are projected onto the text semantic space to obtain negative text candidate global features; The modal weights of the global features of the negative image query and the global features of the negative text query are calculated and weighted using a weight prediction network; the modal weights of the candidate global features of the negative image and the candidate global features of the negative text are calculated and weighted using a weight prediction network. Batch contrastive learning is used to increase the similarity of positive sample pairs and decrease the similarity of negative sample pairs. The negative probability distribution of the data source for the negative query global features and the negative candidate global features is predicted using a domain discriminator.

[0013] Furthermore, the process of obtaining the query local features and candidate local features includes: Obtain the positive image query local features of the reference image, the positive image candidate local features of the candidate image, and the positive text query local features of the modified text extracted by the positive alignment branch; and the negative image query local features of the reference image, the negative image candidate local features of the candidate image, and the negative text query local features of the modified text extracted by the negative alignment branch. The positive image query local features of the reference image and the positive text query local features of the modified text are concatenated along the sequence dimension to obtain a positive concatenation feature. The negative image query local features of the reference image and the negative text query local features of the modified text are concatenated along the sequence dimension to obtain a negative concatenation feature. The positive concatenation feature and the negative concatenation feature are then concatenated and projected along the feature dimension to generate the query local feature composed of a query local label sequence. The positive and negative local image candidate features of the candidate image are spliced ​​and projected along the feature dimension to generate the candidate local features composed of a sequence of candidate local labels.

[0014] Further, the step of extracting multi-perspective query local semantic view and candidate local semantic view from the query local features and candidate local features respectively includes: Based on the query local label sequence in the query local features, predict the contribution of each local label to each local semantic perspective to form a contribution matrix; use the contribution matrix to perform weighted aggregation on the query local features to generate a query local semantic perspective; Based on the candidate local label sequence in the candidate local features, a candidate local semantic perspective is generated using the same weighted aggregation process as the query. The query local semantic perspective is input into a multilayer perceptron and normalized to predict the concept probability distribution of each query local semantic perspective; the same processing flow is used to predict the concept probability distribution of each candidate local semantic perspective based on the candidate local semantic perspective. The shared concept prototype matrix is ​​weighted and aggregated using the probability distribution of the query concept to obtain query concept anchor points; the shared concept prototype matrix is ​​weighted and aggregated using the probability distribution of candidate concepts to obtain candidate concept anchor points; including: The query local semantic perspective is input into a multilayer perceptron and normalized to predict the concept probability distribution of each local semantic perspective. The shared concept prototype matrix in the network model is weighted and aggregated using the concept probability distribution to generate query concept anchors corresponding to the retrieval query. Using the same processing flow, the corresponding concept probability distribution is predicted based on the candidate local semantic perspective, and the shared concept prototype matrix is ​​weighted and aggregated to obtain candidate concept anchor points corresponding to the candidate images.

[0015] Further, the step of using the query concept anchor and candidate concept anchor to perform cross-attention calculation on the query local features and candidate local features respectively, extracting the corresponding instance-level semantic expressions, and aggregating them to obtain query local concept features and candidate local concept features includes: Using the query concept anchor as the query and the query local features as keys and values, cross-attention calculation is performed to obtain the attention weight of the query concept anchor to each local label in the query local features. Based on the attention weight, each local label in the query local features is weighted and summed to obtain the instance-level semantic expression corresponding to the query. After projection and average pooling, the query local concept features are obtained. The same process is used to obtain the candidate local concept features corresponding to the candidates. In the same batch of samples, the query local concept features and the candidate local concept features that match the retrieval query are constructed into positive sample pairs, and the query local concept features and the remaining candidate local concept features in the same batch of samples are constructed into negative sample pairs. The similarity between the query local concept features and the candidate local concept features is calculated. Batch contrastive learning is used to improve the similarity of the positive sample pairs and reduce the similarity of the negative sample pairs to collaboratively optimize the query local concept features and the candidate local concept features.

[0016] Furthermore, after obtaining the query joint retrieval embedding and the candidate joint retrieval embedding, the process also includes an alignment optimization step using a random neighborhood distribution: Within the same batch of samples, a first similarity probability distribution is constructed between the query joint retrieval embedding and each of the candidate joint retrieval embeddings, and a second similarity probability distribution is constructed between the candidate joint retrieval embeddings that match the retrieval query and each of the candidate joint retrieval embeddings; Calculate the KL divergence between the first similarity probability distribution and the second similarity probability distribution. Optimize the query joint retrieval embedding and the candidate joint retrieval embedding by aligning the KL divergence distribution with the matching distribution between the actual query and the candidates, with the goal of minimizing the KL divergence.

[0017] Further, the step of calculating the similarity between the query joint retrieval embedding and each of the candidate joint retrieval embeddings; and selecting the candidate image with the highest similarity from the candidate image library as the target image based on the similarity, includes: Calculate the inner product similarity between the query joint retrieval embedding and all candidate joint retrieval embeddings; Arrange all the calculated inner product similarities in descending order; The candidate image corresponding to the maximum inner product similarity is selected as the predicted target image that matches the retrieval query.

[0018] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: 1. Through an innovative positive and negative dual-branch adversarial learning architecture, the model decouples feature extraction from cross-domain data. The positive branch preserves the differences in features across different domains, while the negative branch uses a gradient reversal mechanism to suppress domain-specific information, thereby extracting cross-domain shared semantics. This effectively solves the problem of difficulty in transferring features from the source domain to the target domain due to feature mixing.

[0019] 2. When faced with the challenge of scarce labeled data in the target domain, the model constructs a shared concept space, abstracting and aggregating fine-grained local features of images and text into concept prototypes, and then remapping them to instance-level features. This process activates potential transferable semantic relationships between different domains, enhancing the model's generalization ability under low-resource conditions.

[0020] 3. This invention employs a random neighborhood distribution soft alignment strategy. By optimizing the consistency of similarity distribution among sample features within a batch, it utilizes a soft alignment strategy in addition to the hard contrast loss. This design significantly reduces the interference of noise annotations on model training, ensuring the stability and accuracy of cross-domain retrieval results.

[0021] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 The flowchart shows a cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction. Figure 2 This is a schematic diagram illustrating the decoupling of the global representation part through explicit domain supervision; Figure 3 This is a schematic diagram illustrating the reconstruction of local representations guided by implicit concepts. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0025] This invention proposes an optimization method for cross-domain low-resource combined image retrieval tasks. It models retrieval representations through two approaches: global decoupling and local reconstruction. Based on joint embedding, it optimizes representation alignment, thereby achieving effective transfer of source domain knowledge to the target domain and improving the utilization rate of limited annotations in the target domain. Ultimately, this improves the model's retrieval performance in the target domain.

[0026] The overall process is as follows Figure 1 As shown, the method comprises three parts: a global decoupling method, a local reconstruction method, and a joint representation optimization method. The global decoupling method uses two parallel processing branches, positive and negative, to decouple the domain-related and domain-independent features of the data, and improves the model's ability to understand cross-domain preferences by guiding explicit domain labels. The local reconstruction method extracts multi-perspective semantic focuses from fine-grained local features, and improves the model's ability to utilize low-resource annotations by learning implicit concepts. The joint representation optimization method combines the relevant representations extracted by the global decoupling method and the local reconstruction method to perform alignment prediction, thereby improving the model's robustness to potential noise annotations.

[0027] like Figure 2 The diagram illustrates the architecture of this invention, which decouples the global representation part through explicit domain supervision. This architecture consists of structurally symmetric positive and negative global branches. The collaborative adversarial learning between the two branches achieves effective decoupling of domain-relevant features from domain-independent features.

[0028] like Figure 3 As shown, this invention illustrates the architecture for reconstructing local representations through implicit concepts. This architecture aims to mine fine-grained shared semantics using local features extracted from positive and negative branches. The specific implementation details of the technical solution of this invention will be described below.

[0029] Global decoupling: 1. Data preprocessing: Correct spelling errors and standardize multimodal input formats.

[0030] Specifically, the input modified text may contain unrecognizable characters or spelling errors, requiring filtering and removal of abnormal characters. A text correction model or a predefined mapping dictionary should be used to standardize the modified text, replacing misspelled words with their corresponding standard forms to correct spelling errors. If a single query contains multiple modified texts, they need to be merged into a single modified text using predefined connection symbols. Input reference or candidate images may vary in width, height, and format due to different sources; they need to be uniformly scaled to a standard size that the model can handle and use RGB-related formats instead of grayscale images.

[0031] 2. Forward modality feature extraction: Extracting reference images using a pre-trained visual language model. Global features of forward image query Local features in forward image query Similarly, extract and modify text. Global features of positive text queries Local features of positive text queries and candidate images Positive image candidate global features With positive image candidate local features The global features of a modality are a single vector, the local features of an image consist of a sequence of visual block embeddings, and the local features of text consist of a sequence of text tag embeddings.

[0032] 3. Optimization of forward global feature extraction: Query global features from the forward image. Global features of positive text queries Dynamic adaptive fusion is performed to obtain global features for positive queries. ; Candidate global features for positive images Projecting onto the text semantic space yields positive text candidate global features. Candidate positive global features are calculated using the same process as extracting global query features. ;Utilize batch comparison learning to align global features for forward queries and candidate positive global features .

[0033] 3.1 Dynamic adaptive fusion includes two processes: modality weight prediction and weighted fusion; using positive query global features. Taking computation as an example, first query the global features of the forward image. Global features of positive text queries The images are concatenated along the feature dimension, and a weighted prediction network is used to calculate the positive image modality weights. and positive text modal weights Then, the global features of the query are obtained by weighting the modal global features based on the modal weights. The calculation method is expressed as follows: (1) in, The term "feature concatenation" represents the feature concatenation operation, and MLP stands for Multilayer Perceptron. For candidates that only have image features and lack text features, a linear layer is used. Candidate global features of the candidate image Projecting onto the text semantic space yields the positive global semantic features of the candidate text. Candidate positive global features The global features are obtained using a similar calculation method to those for forward query global features, and the forward query global features and candidate forward global features share the same weight prediction network.

[0034] 3.2 Batch Comparative Learning: Positive and negative sample pairs are constructed using samples from the same batch to collaboratively optimize the global features of queries and candidates. Within a batch... The nth sample, the th The global features of the positive query for each sample are: The candidate positive global features corresponding to its target As positive samples, the remaining positive global features in the batch are negative samples. The similarity between the query and the candidates is calculated, and the similarity of positive samples is continuously increased while the similarity of negative samples is decreased.

[0035] (2) in, superscript This indicates the transpose operation. For the first The positive query global features of the sample and the first sample Inner product similarity of candidate positive global features for each sample This represents the natural exponential function. Represents the natural logarithm function. for Similarity normalized within batches, For matching loss between batch queries and candidate positive global features, This is the preset temperature coefficient.

[0036] 4. Optimization of positive probability distribution determination: global features for positive queries. With candidate positive global features Introducing gradient scaling factor ,according to Predict the positive query probability distribution of the data domain from which the query data originates. , Similarly, calculate the positive candidate probability distribution. ; Optimize the positive query probability distribution using negative cross-entropy and positive candidate probability distribution The determination.

[0037] 4.1 A gradient scaling layer and a neighborhood discriminator are used to predict the probability distribution of the data domain to which the global features belong. The gradient scaling layer keeps the input features unchanged during forward propagation and adjusts the degree of gradient optimization using gradient scaling coefficients during backpropagation. The forward alignment branch uses a positive gradient scaling coefficient. ,in Use forward query to view global features. For example, its query data source positive probability distribution The calculation method is expressed as follows: (3) in, This represents the global features of a positive query after gradient scaling. This indicates finding the partial derivative with respect to x. It is a multilayer sensing network. This indicates that the vector is normalized using the softmax activation function. Candidate positive global features are calculated using a similar method. Corresponding positive probability distribution of candidate data sources The query and candidate positive global features share the same domain discriminator.

[0038] 4.2 Cross-entropy loss is used to optimize the neighborhood probability distribution; within a batch... The nth sample, the th The samples are from the field. The candidate domain distributions corresponding to the query and the target are as follows: and Continuously improve the alignment between the predicted domain distribution and the actual domain distribution: (4) in, This indicates an index retrieval operation. Represents the natural logarithm function. The loss is calculated for in-batch queries and candidate positive domain classification.

[0039] 5. Negative modal feature extraction, negative global feature extraction optimization, and negative probability distribution determination optimization: The process from steps 2 to 4 in the global structure is denoted as the positive alignment branch. Another parallel negative alignment branch with independent parameters is constructed, and the process from steps 2 to 4 is repeated to obtain the corresponding image and text features. ), negative query global features Candidate negative global features Query the negative probability distribution of the data source Negative probability distribution of candidate data sources The difference between positive alignment branches and negative alignment branches lies in the fact that in step 4, the positive alignment branch uses... As the gradient scaling factor, the negative alignment branch adopts ,in .

[0040] 5.1 Extracting Reference Images Using Pre-trained Visual Language Models Negative image query global features Local features in negative image queries Similarly, extract and modify text. Global features of negative text queries Local features of negative text queries and candidate images Negative image candidate global features With negative image candidate local features The global features of a modality are a single vector, the local features of an image consist of a sequence of visual block embeddings, and the local features of text consist of a sequence of text tag embeddings.

[0041] 5.2 Negative Global Feature Extraction: Extracting global features from the negative image Global features of negative text queries Dynamic adaptive fusion is performed to obtain negative query global features. ; Candidate global features for negative images Projecting onto the text semantic space yields negative text candidate global features. Candidate negative global features are calculated using the same process as extracting global query features. Dynamic adaptive fusion includes two processes: modality weight prediction and weighted fusion; it uses negative queries to obtain global features. Taking computation as an example, first query the global features of the negative image. Global features of negative text queries The images are concatenated along the feature dimension, and a weighted prediction network is used to calculate the negative image modality weights. and negative text modal weights Then, based on the modal weights, the global modal features are weighted to obtain the negative query global features, which are calculated as follows: (5) in, The term "feature concatenation" represents the feature concatenation operation, and MLP stands for Multilayer Perceptron. For candidates that only have image features and lack text features, a linear layer is used. Negative image candidate global features Projecting onto the text semantic space yields negative global semantic features of the candidate text. Candidate negative global features The negative query global features are obtained using a similar calculation method to the negative query global features, and the negative query global features and the candidate negative global features share the same weight prediction network.

[0042] 5.3 Negative Global Feature Optimization: Aligning Negative Query Global Features Using Batch Comparison Learning and candidate negative global features By constructing positive and negative sample pairs from samples in the same batch, the global features of the query and candidates are collaboratively optimized; within the batch, there are... The nth sample, the th The global feature of negative queries for each sample is The candidate negative global features corresponding to its target As positive samples, the remaining candidate negative global features within the batch are considered negative samples. The similarity between the query and the candidates is calculated, and the similarity of positive samples is continuously increased while the similarity of negative samples is decreased.

[0043] (6) in, superscript This indicates the transpose operation. For the first The negative query global features of the nth sample and the nth sample Inner product similarity of candidate negative global features for each sample This represents the natural exponential function. Represents the natural logarithm function. for Similarity normalized within batches, For matching loss between batch queries and candidate negative global features, This is the preset temperature coefficient.

[0044] 5.4 Negative Probability Distribution Determination: Global Features for Negative Queries With candidate negative global features Introducing gradient scaling factor ,according to The negative probability distribution of the query data source from the data domain from which the query data originates. , Similarly, calculate the negative probability distribution of candidate data sources. Specifically, a gradient scaling layer and a neighborhood discriminator are used to predict the probability distribution of the data domain to which the global features belong. The gradient scaling layer keeps the input features unchanged during forward propagation and adjusts the degree of gradient optimization using gradient scaling coefficients during backpropagation. The global features are queried in the negative direction. For example, its query data source has a negative probability distribution. The calculation method is expressed as follows: (7) in, This represents the negative query global feature after gradient scaling. This indicates finding the partial derivative with respect to x. It is a multilayer sensing network. This indicates that the vector is normalized using the softmax activation function. Candidate negative global features are calculated using a similar method. Corresponding negative probability distribution of candidate data sources The query and candidate negative global features share the same domain discriminator.

[0045] 5.5 Negative Probability Distribution Optimization: Optimizing the Negative Probability Distribution of the Query Data Source Using Negative Cross-Entropy Negative probability distribution of candidate data sources The determination. A total of [number] batches. The nth sample, the th The samples are from the field. The candidate domain distributions corresponding to the query and the target are as follows: and Continuously improve the alignment between the predicted domain distribution and the actual domain distribution: (8) in, This indicates an index retrieval operation. Represents the natural logarithm function. The loss is calculated for in-batch queries and candidate negative domain classification.

[0046] Local reconstruction: 6. Local Feature Initialization: The local features obtained in step 2 (positive alignment branch) and step 5 (negative alignment branch) are concatenated and fused to obtain the query local features. and candidate local features Specifically, this involves utilizing positive and negative local features of the reference image and modified text. Construct query local features Utilizing the positive and negative local features of candidate images ( Construct candidate local features .

[0047] (9) in, This indicates a concatenation operation along the sequence dimension. This indicates a concatenation operation along the feature dimension. This indicates a projection operation.

[0048] If the concatenation symbols appear simultaneously, such as [a,b;c,d], it is equivalent to concatenating ab and cd along the sequence dimension and then concatenating them along the feature dimension.

[0049] Since the local features of an image consist of a sequence of visual patch embeddings, and the local features of text consist of a sequence of text marker embeddings, the query local features generated after concatenation and projection are... Essentially, it is a sequence of query local labels composed of a series of local labels. Similarly, candidate local features... In essence, they are all candidate local label sequences composed of a series of local labels.

[0050] 7. Concept Anchor Representation Extraction: Aggregating the obtained local features into a query local semantic perspective. Candidate local semantic perspective The resulting local semantic perspective is projected onto the concept distribution space and used to weighted aggregate the shared concept prototype matrix. Obtain the anchor representation of the query concept. and candidate concept anchor representation .

[0051] In the aforementioned partial reconstruction process, the shared concept prototype matrix (denoted as...) The matrix is ​​a built-in learnable parameter matrix in the network model. This matrix is ​​not a static, fixed-value constant matrix, but rather a tensor that is autonomously learned and updated by the model through backpropagation during training. Its purpose is to construct a cross-domain semantic memory library, and the shared concept prototype matrix... Include Each shared concept prototype corresponds to an embedding vector, and each vector represents a specific, cross-domain-shared potential semantic concept.

[0052] To clarify the physical logic of each feature variable in the network model of this invention, the dimensional distribution of the relevant variables is further defined here: Let the uniform feature dimension size of the network be... The global features extracted by the model (such as the forward query global features, forward candidate global features, etc.) are all of shape [shape missing]. A one-dimensional vector; the neighborhood probability distribution obtained from global feature prediction has a shape of... A one-dimensional vector, where This refers to the number of data domains involved.

[0053] For local features, the local feature corresponding to the modified text is the shape. Two-dimensional tensors (where (where the length of the local feature sequence of the text is given); the local features corresponding to the reference image or candidate image have a shape of... Two-dimensional tensors (where (where is the length of the local feature sequence of the image). In the concept extraction stage, the shape of the local semantic perspective is... (in (The number of local semantic perspectives); the shared concept prototype matrix The shape is (in (For the number of shared concept prototypes). The concept probability distribution is predicted through the local semantic perspective, and the shared concept prototype matrix is ​​then analyzed. After weighted aggregation, the shape of the resulting concept anchor representation is also as follows: Finally, after extracting the corresponding instance-level semantic representation using cross-attention and performing mean pooling, the output local concept features are re-aggregated into a shape of... A one-dimensional vector is used to ensure dimensional alignment with the global features in subsequent stitching processes.

[0054] 7.1 Based on local features, predict the contribution of each local label in the sequence to different local semantic perspectives, and aggregate local features using weighted aggregation of contribution to generate local semantic perspectives. This allows for querying local features. For example, the contribution of each local label is calculated based on the query local features (i.e., the query local label sequence) to generate a query local semantic perspective. : (10) in, This is the contribution matrix of each local label to each semantic perspective. This means that the vector is normalized using the softmax activation function. Indicates the length of the local label sequence. This indicates the number of local semantic viewpoints. Following the same process, candidate local features are used... Extracting candidate local semantic perspectives .

[0055] 7.2 Concept distribution is predicted using a local semantic perspective, and the shared concept prototype matrix is ​​weighted and aggregated based on the concept distribution to generate a concept query representation. This is done from a query local semantic perspective. For example, calculate the anchor representation of the query concept. : (11) in, The probability distribution of concepts from various semantic perspectives. Represents a multilayer sensing network. This means that the vector is normalized using the softmax activation function. Indicates the number of concept prototypes. Following the same process, based on candidate local semantic perspectives... Extracting candidate concept anchor representations .

[0056] 8. Local concept feature extraction optimization: Represent the query concept anchor point Candidate concept anchor representation and query local features Candidate local features Perform cross-attention calculations to extract instance-level semantic representations corresponding to concepts and aggregate them to obtain query local concept features. and candidate local concept features Using batch comparison learning to align local concept features of queries and candidate local concept features .

[0057] 8.1 Cross-attention calculation is performed using concept anchor representations and local features to extract instance-level semantic expressions corresponding to concepts, and these are aggregated to obtain local concept features. Taking a query sample as an example, based on the concept anchor representation... and local features Calculate query local concept features The calculation method is as follows: (12) in, Each concept anchor point represents the level of attention paid to each local marker. Indicates a multilayer sensing network, superscript This indicates the transpose operation. This means that the vector is normalized using the softmax activation function. This means pooling multiple local concept features into a single local concept feature. This represents the projection operation. Candidate local concept features are extracted following the same process. .

[0058] 8.2 Batch Comparative Learning: Positive and negative sample pairs are constructed using samples from the same batch to collaboratively optimize local concept features for queries and candidates. Within a batch... The nth sample, the th The query local concept features of each sample are The candidate local concept features corresponding to its target As positive samples, the remaining candidate local concept features within the batch are negative samples. The similarity between the query and the candidates is calculated, and the similarity of the positive samples is continuously increased while the similarity of the negative samples is decreased.

[0059] (13) in, For the first The query local concept features of the first sample and the first The inner product similarity of candidate local concept features for each sample, with superscript... This indicates the transpose operation. for Similarity normalized within batches, This represents the natural exponential function. The loss is for matching intra-batch queries and candidate local concept features. Represents the natural logarithm function. This is the preset temperature coefficient.

[0060] Joint characterization optimization: 9. Joint retrieval embedding merging and random neighborhood alignment optimization: The query joint retrieval embedding is obtained by concatenating positive global features, negative global features, and local concept features. Candidate joint retrieval embedding Optimization is achieved by using soft alignment based on random neighborhood distribution.

[0061] 9.1 Positive global features, negative global features, and local concept features are concatenated to generate a joint retrieval embedding. Query the joint retrieval embedding. Candidate joint retrieval embedding ,in This indicates a concatenation operation along the feature dimension.

[0062] 9.2 Optimizing Joint Retrieval Embeddings Using Soft Alignment Based on Random Neighborhood Distribution. First, a similarity probability distribution is constructed between the query joint retrieval embedding and candidate joint retrieval embeddings. Then, a similarity probability distribution is constructed between the target joint retrieval embedding and candidate joint retrieval embeddings. Next, the difference between the query similarity probability distribution and the target similarity probability distribution is calculated. This process continuously optimizes the joint retrieval embedding to improve the matching degree of the similarity probability distributions between the query and target within the same sample. (Batch total...) The nth sample, the th The query joint retrieval embedding and candidate joint retrieval embedding for each sample are respectively and : (14) superscript This indicates the transpose operation. This represents the natural exponential function. Represents the natural logarithm function. For the first The query joint retrieval embedding of the first sample and the first sample Inner product similarity of candidate joint retrieval embeddings for each sample. For the first The candidate joint retrieval embedding corresponding to the target of the sample and the first sample Inner product similarity of candidate joint retrieval embeddings for each sample. For the first The query similarity distribution of the nth sample and the nth sample The negative KL divergence of the target similarity distribution of each sample They are respectively Similarity normalized within batches, The alignment loss is calculated for the query and target similarity distribution within the batch. This is the preset temperature coefficient.

[0063] 10. Target Prediction: The similarity between the calculated query joint retrieval embedding and all candidate joint retrieval embeddings is calculated, and the candidate with the highest similarity is selected as the predicted target matching the given query. Specifically, during the inference phase, the model predicts the target matching the query based on the similarity between the query and candidate joint retrieval embeddings. Given the reference image and modified text contained in the query, the query's joint retrieval embedding is encoded. , and the pre-encoded results of all candidate images Calculate the inner product similarity one by one ,according to Sort in descending order and select the largest. The corresponding candidate image is used as the prediction target.

[0064] Example 1 This embodiment demonstrates the effectiveness of the present invention compared to existing combined image retrieval methods and their general cross-domain learning variants: Retrieval tasks are performed on the FashionIQ, Shoes, and CIRR datasets, each containing a training set and a test set, with no overlap between the two. The task requires the model to learn based on the complete training annotations of the source domain dataset and the low-resource training annotations of the target domain dataset, and to evaluate model performance on the test set of the target domain dataset. Taking FashionIQ as the source domain and Shoes as the target domain as an example, the model's training set is obtained by combining the entire FashionIQ training set and a subset obtained by low-resource sampling of the Shoes training set, and the model's test set is the Shoes test set. This embodiment uses R10 and R50 as evaluation metrics on FashionIQ and Shoes, and R5 and Rs1 as evaluation metrics on CIRR. The metric Rk represents the proportion of the target image in the top k retrieval results of the predicted recall, and the metric Rsk represents the Rk metric on the test subset. Table 1 presents the experimental results of the method of this invention and other research methods on FashionIQ, Shoes, and CIRR. DWC, DQUCIR, ENCODER, and OFFSET are the baseline models for combined image retrieval compared with each other. w / align and w / adapt represent the variants obtained by adding the corresponding general cross-domain learning methods to the aforementioned baseline models for combined image retrieval. RaD represents the present invention. All index values ​​in the table are percentages. Bold text indicates the best value in the same column, and underlined text indicates the second best value in the same column.

[0065] Table 1 Experimental Results of Example 1

[0066] Compared with the baseline model, the cross-domain low-resource combined image retrieval method of the present invention achieves the best performance in all the above indicators. More specifically, (1) the indicators of the present invention are better than the baseline combined image retrieval method, indicating that the present invention can handle combined image retrieval tasks well; (2) the variants obtained by combining the baseline method with the general cross-domain learning method are worse than the original baseline method in most indicators, indicating that the general cross-domain learning method cannot be well adapted to combined image retrieval tasks; (3) the indicators of the present invention are better than these variants, indicating that the present invention can effectively cope with the cross-domain low-resource scenario setting of combined image retrieval tasks. This reflects the superiority of the present invention.

[0067] Example 2 This embodiment demonstrates the effectiveness of each component of the method of the present invention: retrieval tasks are performed on the FashionIQ, Shoes, and CIRR datasets. Each dataset contains a training set and a test set, and the two sets do not overlap. This task requires the model to learn based on the complete training annotations of the source domain dataset and the low-resource training annotations of the target domain dataset, and to evaluate the model performance on the test set of the target domain dataset. Taking FashionIQ as the source domain and Shoes as the target domain as an example, the model's training set is obtained by combining the entire training set of FashionIQ and a subset obtained by low-resource sampling of the Shoes training set, and the model's test set is the Shoes test set. This embodiment uses R10 and R50 as evaluation metrics on FashionIQ and Shoes, and R5 and Rs1 as evaluation metrics on CIRR. The metric Rk represents the proportion of the target image in the top k retrieval results of the predicted recall, and the metric Rsk represents the Rk metric on the test subset. Table 2 presents the experimental results of the method of this invention and different ablation variants on FashionIQ, Shoes, and CIRR. w / o PAB and w / o NAB represent the positive alignment branch and negative alignment branch in the global ablation decoupling module, respectively. w / o CQA and w / o CIE represent the step of extracting concept anchors from local features and the step of instantiating local concept features based on concept anchors in the local ablation reconstruction module, respectively. w / o SNA represents the random neighborhood alignment loss term in the ablation training objective. All index values ​​in the table are percentages. Bold text indicates the best value in the same column, and underlined text indicates the second best value in the same column.

[0068] Table 2 Experimental Results of Example 2

[0069] Compared with the ablation variant, the cross-domain low-resource combined image retrieval method of the present invention achieves the best performance in all the above indicators. More specifically, (1) the variants with / o PAB and with / o NAB are inferior to the complete method of the present invention, indicating that the domain-related features extracted by the positive branch and the domain-independent features extracted by the negative branch of the global decoupling method are both important and can improve the model's understanding of cross-domain samples. (2) the variants with / o CQA and with / o CIE are inferior to the complete method of the present invention, indicating that the two-stage processing of the local reconstruction method, which first abstracts and integrates discrete local features into several shared concept anchors and then re-examines and refines local features according to the shared concept anchors, is indispensable. This can improve the model's utilization of information from low-resource samples. (3) the variant with / o SNA is inferior to the complete method of the present invention, indicating that the random neighborhood alignment of the joint representation helps to improve the robustness of the model to potential noise annotations in the training samples. This reflects the necessity of each component of the method of the present invention.

[0070] The beneficial effects of this invention are as follows: 1. Instead of simply aligning retrieval features across different domains, this invention introduces positive and negative alignment branches. A complementary adversarial learning mechanism between these two branches decouples domain-related and domain-independent semantics. Specifically, the positive alignment branch enhances domain-related information representation through domain discriminative learning, enabling the model to fully preserve feature differences across domains. The negative alignment branch uses gradient inversion to suppress domain-specific information and learns semantic representations with cross-domain shared properties. By collaboratively optimizing and decoupling domain-related and domain-independent representations through positive and negative branches, this invention effectively alleviates the domain shift problem caused by feature confounding, improves the ability to transfer knowledge from the source domain to the target domain, and provides a more stable global semantic foundation for subsequent cross-domain retrieval representation learning.

[0071] 2. A shared concept space is constructed, and local fine-grained semantics are explicitly modeled through concept prototype learning and instance-level semantic recovery mechanisms. Specifically, this invention predicts concept distribution from a local semantic perspective and extracts shared semantic structures with potential connections between different domains through shared concept prototypes; further, it combines a cross-attention mechanism to recover the instance-level expressions corresponding to concepts, achieving local concept reorganization. This method can fully exploit transferable fine-grained semantic knowledge between different domains, improve the utilization efficiency of limited labeled samples in the target domain, and enhance the model's expressive and generalization capabilities for combined semantic changes.

[0072] 3. Optimizing the unified retrieval representation using a soft alignment strategy based on random neighborhood distribution. Specifically, this invention constructs neighborhood probability distributions between query samples and candidate samples, as well as between target samples and candidate samples, and optimizes the spatial structure of the retrieval representation using distribution consistency constraints. This method not only maintains the stability of local neighborhood relationships but also reduces the impact of noise annotations and outliers on model training, improving the robustness and consistency of the retrieval representation, thereby further enhancing retrieval performance in cross-domain low-resource scenarios.

[0073] In summary, this invention effectively improves the performance of the combined image retrieval system in cross-domain low-resource scenarios, enhances the model's ability to transfer knowledge from the source domain and utilize knowledge from the target domain, and strengthens the model's robustness to noise annotation.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction, characterized in that, include: Obtain a search query and a candidate image library, wherein the search query includes reference images and modified text; Data preprocessing is performed on the reference image, the modified text, and the candidate images in the candidate image library to generate a normalized multimodal input; The normalized multimodal input is subjected to feature extraction and fusion using positive alignment branches and negative alignment branches to obtain positive global features, negative global features, and local features corresponding to the retrieval query, as well as positive global features, negative global features, and local features corresponding to the candidate image. The positive alignment branch is used to extract domain-related features, and the negative alignment branch is used to extract domain-independent features shared across domains. From the query local features and the candidate local features, extract the query local semantic perspective and the candidate local semantic perspective from multiple perspectives, respectively. The probability distribution of query concepts is predicted using the local semantic perspective of the query, and the probability distribution of candidate concepts is predicted using the local semantic perspective of the candidate. The shared concept prototype matrix is ​​weighted and aggregated using the probability distribution of the query concept to obtain the query concept anchor point; the shared concept prototype matrix is ​​weighted and aggregated using the probability distribution of the candidate concept to obtain the candidate concept anchor point. The query local features and candidate local features are cross-attention calculated using the query concept anchor and candidate concept anchor respectively, the corresponding instance-level semantic expressions are extracted, and the query local concept features and candidate local concept features are aggregated. The positive query global features, the negative query global features, and the query local concept features are concatenated to obtain the query joint retrieval embedding; and the positive candidate global features, the negative candidate global features, and the candidate local concept features are concatenated to obtain the candidate joint retrieval embedding. Calculate the similarity between the query joint retrieval embedding and each of the candidate joint retrieval embeddings; select the candidate image with the highest similarity from the candidate image library as the target image based on the similarity.

2. The cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction according to claim 1, characterized in that, The step of preprocessing the reference image, the modified text, and the candidate images in the candidate image library to generate a standardized multimodal input includes: performing character removal and spell correction on the modified text, and replacing the corrected spellings with standard word forms; if a single search query contains multiple modified texts, merging the multiple modified texts into a single modified text using a preset connection symbol; and uniformly scaling the reference image and candidate images to a preset standard size and converting them to standard color format images.

3. The cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction according to claim 1, characterized in that, The positive image query global features of the reference image, the positive text query global features of the modified text, and the positive image candidate global features of the candidate image are extracted using a pre-trained visual language model. The positive image candidate global features are projected onto the text semantic space to obtain the positive text candidate global features; The positive query global features are obtained by performing modal weight calculation and weighted fusion on the positive image query global features and the positive text query global features using a weight prediction network; the positive image candidate global features and the positive text candidate global features are obtained by performing modal weight calculation and weighted fusion on the weight prediction network. In the same batch of samples, construct positive and negative sample pairs for query and candidate, calculate the similarity between the global features of the positive query and the global features of the positive candidate, and use batch contrastive learning to increase the similarity of positive sample pairs and decrease the similarity of negative sample pairs, thereby co-optimizing the global features of the positive query and the global features of the positive candidate.

4. The cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction according to claim 3, characterized in that, After obtaining the positive query global features corresponding to the retrieval query and the positive candidate global features corresponding to the candidate images, the method further includes an optimization step based on the positive probability distribution: A positive gradient scaling coefficient is introduced for both the forward query global feature and the forward candidate global feature. During backpropagation, the degree of gradient optimization is adjusted using the positive gradient scaling coefficient. The domain discriminator is used to predict the positive probability distribution of the query data source corresponding to the positive global query feature and the positive probability distribution of the candidate data source corresponding to the positive candidate global feature, respectively. The determination of the positive probability distribution is optimized using cross-entropy loss to enhance the expressive power of the domain-related features.

5. The cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction according to claim 3, characterized in that, The negative alignment branch adopts the same structure as the positive alignment branch, and the specific steps include: The negative image query global features of the reference image, the negative text query global features of the modified text, and the negative image candidate global features of the candidate image are extracted using a pre-trained visual language model. The negative image candidate global features are projected onto the text semantic space to obtain negative text candidate global features; The modal weights of the global features of the negative image query and the global features of the negative text query are calculated and weighted using a weight prediction network; the modal weights of the candidate global features of the negative image and the candidate global features of the negative text are calculated and weighted using a weight prediction network. Batch contrastive learning is used to increase the similarity of positive sample pairs and decrease the similarity of negative sample pairs. The negative probability distribution of the data source for the negative query global features and the negative candidate global features is predicted using a domain discriminator.

6. The cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction according to claim 1, characterized in that, The process of obtaining the query local features and candidate local features includes: Obtain the positive image query local features of the reference image, the positive image candidate local features of the candidate image, and the positive text query local features of the modified text extracted by the positive alignment branch; and the negative image query local features of the reference image, the negative image candidate local features of the candidate image, and the negative text query local features of the modified text extracted by the negative alignment branch. The positive image query local features of the reference image and the positive text query local features of the modified text are concatenated along the sequence dimension to obtain a positive concatenation feature. The negative image query local features of the reference image and the negative text query local features of the modified text are concatenated along the sequence dimension to obtain a negative concatenation feature. The positive concatenation feature and the negative concatenation feature are then concatenated and projected along the feature dimension to generate the query local feature composed of a query local label sequence. The positive and negative local image candidate features of the candidate image are spliced ​​and projected along the feature dimension to generate the candidate local features composed of a sequence of candidate local labels.

7. The cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction according to claim 6, characterized in that, The step of extracting multi-perspective query local semantic perspectives and candidate local semantic perspectives from the query local features and candidate local features respectively includes: Based on the query local label sequence in the query local features, predict the contribution of each local label to each local semantic perspective to form a contribution matrix; use the contribution matrix to perform weighted aggregation on the query local features to generate a query local semantic perspective; Based on the candidate local label sequence in the candidate local features, a candidate local semantic perspective is generated using the same weighted aggregation process as the query. The query local semantic perspective is input into a multilayer perceptron and normalized to predict the concept probability distribution of each query local semantic perspective; the same processing flow is used to predict the concept probability distribution of each candidate local semantic perspective based on the candidate local semantic perspective. The shared concept prototype matrix is ​​weighted and aggregated using the probability distribution of the query concept to obtain query concept anchor points; the shared concept prototype matrix is ​​weighted and aggregated using the probability distribution of candidate concepts to obtain candidate concept anchor points; including: The query local semantic perspective is input into a multilayer perceptron and normalized to predict the concept probability distribution of each local semantic perspective. The shared concept prototype matrix in the network model is weighted and aggregated using the concept probability distribution to generate query concept anchors corresponding to the retrieval query. Using the same processing flow, the corresponding concept probability distribution is predicted based on the candidate local semantic perspective, and the shared concept prototype matrix is ​​weighted and aggregated to obtain candidate concept anchor points corresponding to the candidate images.

8. The cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction according to claim 7, characterized in that, The process involves performing cross-attention calculations on the query local features and candidate local features using the query concept anchors and candidate concept anchors respectively, extracting the corresponding instance-level semantic expressions, and aggregating them to obtain query local concept features and candidate local concept features, including: Using the query concept anchor as the query and the query local features as keys and values, cross-attention calculation is performed to obtain the attention weight of the query concept anchor to each local label in the query local features. Based on the attention weight, each local label in the query local features is weighted and summed to obtain the instance-level semantic expression corresponding to the query. After projection and average pooling, the query local concept features are obtained. The same process is used to obtain the candidate local concept features corresponding to the candidates. In the same batch of samples, the query local concept features and the candidate local concept features that match the retrieval query are constructed into positive sample pairs, and the query local concept features and the remaining candidate local concept features in the same batch of samples are constructed into negative sample pairs. The similarity between the query local concept features and the candidate local concept features is calculated. Batch contrastive learning is used to improve the similarity of the positive sample pairs and reduce the similarity of the negative sample pairs to collaboratively optimize the query local concept features and the candidate local concept features.

9. The cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction according to claim 1, characterized in that, After obtaining the query joint retrieval embedding and the candidate joint retrieval embedding, the process also includes an alignment optimization step using a random neighborhood distribution: Within the same batch of samples, a first similarity probability distribution is constructed between the query joint retrieval embedding and each of the candidate joint retrieval embeddings, and a second similarity probability distribution is constructed between the candidate joint retrieval embeddings that match the retrieval query and each of the candidate joint retrieval embeddings; Calculate the KL divergence between the first similarity probability distribution and the second similarity probability distribution. Optimize the query joint retrieval embedding and the candidate joint retrieval embedding by aligning the KL divergence distribution with the matching distribution between the actual query and the candidates, with the goal of minimizing the KL divergence.

10. The cross-domain low-resource combined image retrieval method based on global decoupling and local reconstruction according to claim 1, characterized in that, The similarity between the query joint retrieval embedding and each of the candidate joint retrieval embeddings is calculated. Selecting the candidate image with the highest similarity from the candidate image library as the target image based on the similarity score includes: Calculate the inner product similarity between the query joint retrieval embedding and all candidate joint retrieval embeddings; Arrange all the calculated inner product similarities in descending order; The candidate image corresponding to the maximum inner product similarity is selected as the predicted target image that matches the retrieval query.

Images