Cross-domain cross-modal pedestrian search method based on cache correction test time reordering

Abstract

The application discloses a cross-domain cross-modal pedestrian search method based on cache correction test time reordering. The method comprises the following steps: constructing a reward cache and a penalty cache for each cluster center; obtaining a reward matrix based on the feature vector of each sentence of the target domain and the feature vector of each sentence of the target domain stored in the corresponding reward cache of each cluster center; obtaining a penalty matrix based on the feature vector of each sentence of the target domain and the feature vector of each sentence of the target domain stored in the corresponding penalty cache of each cluster center; respectively performing adaptive correction on the reward matrix and the penalty matrix to obtain a corrected reward matrix and a corrected penalty matrix; obtaining a target correction matrix based on the corrected reward matrix and the corrected penalty matrix; and performing image sorting based on the target correction matrix and the target correlation matrix. The application solves the technical problems of poor cross-domain style adaptability and low cross-domain search accuracy in natural language cross-modal pedestrian search.

Description

Technical Field

[0001] This invention relates to the field of online reordering technology in the testing phase of natural language cross-modal pedestrian search tasks, and more specifically, to a cross-domain cross-modal pedestrian search method based on cache correction and test-time reordering. Background Technology

[0002] Natural language cross-modal person search aims to retrieve images of target individuals from image databases using text descriptions. It offers more flexible and effective querying capabilities in complex environments such as low light, occlusion, and poor viewing angles, and has broad application prospects in fields like intelligent video analysis and security monitoring. However, most existing representative methods primarily focus on addressing modal heterogeneity, neglecting the challenges of cross-domain issues in real-world applications. Regarding single-modal cross-domain problems, some methods attempt to fine-tune the target domain by generating pseudo-labels (such as pseudo-person identities), but such methods are difficult to apply in natural language cross-modal person search because it's challenging to generate sufficient high-quality pseudo-sentence descriptions (sentences are flexible but difficult to generate). Furthermore, the simultaneous existence of cross-modal and cross-domain problems makes it difficult for a single cross-domain adaptation strategy to effectively address the different domain style variations in two modalities simultaneously. Summary of the Invention

[0003] This invention provides a cross-domain, cross-modal pedestrian search method based on cache correction and test-time reordering, in order to at least solve the technical problems of poor cross-domain style adaptability and low cross-domain search accuracy in natural language cross-modal pedestrian search.

[0004] According to one aspect of the present invention, a cross-domain, cross-modal pedestrian search method based on cache correction test-time reordering is provided. The method may include: acquiring a plurality of images and a plurality of statements in a target domain, wherein each image is a pedestrian image and each statement is a statement describing a pedestrian image; processing the plurality of images and the plurality of statements to obtain feature vectors for each image and feature vectors for each statement; clustering the feature vectors of each image using a target algorithm to obtain target clusters; determining the features of each cluster center based on the feature vectors of all images within each cluster; for each cluster center, obtaining a relation matrix based on the feature vectors of each statement and the features of each cluster center; determining the cache region of the cluster center to which the feature vector of each statement belongs based on each element in the relation matrix, wherein the cache region includes a reward cache and a penalty cache; and determining the cache region based on the feature vectors of each statement in the target domain. The reward matrix is ​​obtained by combining the feature vectors of each statement in the target domain stored in the reward cache corresponding to each cluster center. The penalty matrix is ​​then obtained based on the feature vectors of each statement in the target domain and the penalty matrix stored in the penalty cache corresponding to each cluster center. Adaptive adjustments are made to the reward and penalty matrices to obtain the adjusted reward and penalty matrices. A target adjustment matrix is ​​then obtained based on the adjusted reward and penalty matrices. The relevance matrices corresponding to several images and statements in the target domain are obtained. A target relevance matrix is ​​obtained based on the relevance matrices and the target adjustment matrix. Finally, the images in the target domain are sorted based on the target relevance matrix to obtain a sequence of adjusted images corresponding to each statement in the target domain.

[0005] Optionally, the step of processing several images and several sentences to obtain the feature vector of each image and the feature vector of each sentence includes: using an image encoder in the source domain to extract features from several images to obtain the feature vector of each image; and using a text encoder in the source domain to extract features from several sentences to obtain the feature vector of each sentence.

[0006] Optionally, the step of clustering the feature vectors of each image using a target algorithm to obtain a target number of category clusters includes: clustering the feature vectors of each image using the k-means algorithm to obtain a target number of category clusters.

[0007] Optionally, determining the features of each cluster center based on the feature vectors of all images within each category cluster includes: performing average pooling calculation on the feature vectors of all images within each category cluster to obtain the features of each cluster center.

[0008] Optionally, obtaining the reward matrix based on the feature vector of each statement in the target domain and the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center includes: determining a first weight for each statement based on the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center; obtaining a reward value for each statement in the target domain based on the first weight of each statement, the feature vector of each statement in the target domain, and the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center; and obtaining the reward matrix based on the reward value of each statement in the target domain.

[0009] Optionally, obtaining the penalty matrix based on the feature vector of each statement in the target domain and the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center includes: determining a second weight for each statement based on the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center; obtaining a penalty value for each statement in the target domain based on the second weight of each statement, the feature vector of each statement in the target domain, and the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center; and obtaining the penalty matrix based on the penalty value of each statement in the target domain.

[0010] Optionally, obtaining the target correction matrix based on the modified reward matrix and the modified penalty matrix includes: determining the difference between the modified reward matrix and the modified penalty matrix as the target correction matrix.

[0011] Optionally, obtaining the target correlation matrix based on the correlation matrix and the target correction matrix includes: determining the target correlation matrix by summing the correlation matrix and the target correction matrix.

[0012] The beneficial effects of this invention are:

[0013] This invention proposes a cross-domain, cross-modal pedestrian search method based on cache correction and test-time reordering. First, pedestrian images are clustered, and appropriate sentences are selected as unimodal relays based on each cluster center to construct reward and penalty caches, which store sentences that help alleviate cross-domain difficulties. Then, guided by the sentences stored in the reward and penalty caches, the corresponding reward and penalty values ​​are calculated, and the positions of the original image-sentence pairs that need correction are precisely located, thus completing the fine-grained correction of the original relevance. Finally, based on the corrected image-sentence relevance, the original search results are reordered during testing. This method does not require reference to source domain data, does not increase additional computation or time overhead, and can be quickly and easily applied to several representative advanced methods in the field of natural language cross-modal pedestrian search, all of which achieve significant improvements in cross-domain search accuracy. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0015] Figure 1 This is a flowchart of a cross-domain, cross-modal pedestrian search method based on cache correction and test-time reordering according to an embodiment of the present invention;

[0016] Figure 2 This is a structural block diagram of a cross-domain, cross-modal pedestrian search method based on cache correction and test-time reordering according to an embodiment of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] Example 1

[0020] According to embodiments of the present invention, a cross-domain, cross-modal pedestrian search method based on cache correction test reordering is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system containing at least one set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0021] Figure 1 This is a flowchart of a cross-domain, cross-modal pedestrian search method based on cache correction and test-time reordering according to an embodiment of the present invention, as shown below. Figure 1As shown, the method may include the following steps:

[0022] Step S101: Obtain several images and several statements from the target domain, wherein each image is a pedestrian image and each statement is a statement describing the pedestrian image.

[0023] In the technical solution provided by step S101 of the present invention, the target domain is obtained. One image and A statement.

[0024] Step S102: Process several images and several sentences to obtain the feature vector of each image and the feature vector of each sentence.

[0025] In the technical solution provided by step S102 of the present invention, Figure 2 This is a structural block diagram of a cross-domain, cross-modal pedestrian search method based on cache correction and test-time reordering according to an embodiment of the present invention, as shown below. Figure 2 As shown, for the target domain One image and Each statement is processed to obtain the feature vector of each image. and the feature vector of each statement ,in, Representing feature dimension, For the feature space.

[0026] Step S103: The target algorithm is used to cluster the feature vectors of each image to obtain the target number of category clusters.

[0027] In the technical solution provided by step S103 of the present invention, such as Figure 2 As shown, the target algorithm is used to cluster the feature vectors of each image to obtain... Each category is a cluster.

[0028] Step S104: Determine the features of each cluster center based on the feature vectors of all images within each category cluster.

[0029] In the technical solution provided by step S104 of the present invention, such as Figure 2 As shown, the feature vectors of all images within each category cluster are processed to determine the features of each cluster center. .

[0030] Step S105: For each cluster center, obtain the relation matrix based on the feature vector of each statement and the features of each cluster center.

[0031] In the technical solution provided by step S105 of the present invention, for each cluster center, for each statement's feature vector... Calculate its characteristics relative to the cluster centers. The cosine similarity between them yields the relation matrix. :

[0032]

[0033] in, This is a relation matrix.

[0034] Step S106: Based on each element in the relation matrix, determine the cache region of the cluster center to which the feature vector of each statement belongs, wherein the cache region includes a reward cache and a penalty cache.

[0035] In the technical solution provided by step S106 of the present invention, such as Figure 2 As shown, in order to clarify the feature vector of each statement The distribution characteristics between the features of all cluster centers are calculated. entropy :

[0036]

[0037] Through a Hyperparameters within the range Determine the feature vector of the statement The direction of influence, that is, when solving the problem of domain style differences, the feature vector of the statement. Should it be filled in the reward cache or the penalty cache, depending on entropy? Less than the threshold This means that the feature vector of the statement The impact of cross-domain issues is relatively small, so the reward cache is filled; instead, the entropy value... Greater than or equal to the threshold The feature vector of the statement This should be used to fill the penalty cache. To save cache space, the maximum capacity of the reward cache and penalty cache for each cluster center is set to [value missing]. and If the number of feature vectors of statements in the cache exceeds the maximum capacity or The feature vectors of statements with smaller entropy values ​​will be retained, while the feature vectors of all other statements that exceed the maximum capacity will be discarded.

[0038] Step S107: Based on the feature vector of each statement in the target domain and the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center, a reward matrix is ​​obtained.

[0039] In the technical solution provided by step S107 of the present invention, such as Figure 2As shown, the feature vectors of each statement in the target domain and the feature vectors of each statement in the target domain stored in the reward cache corresponding to each cluster center are processed to obtain the reward matrix.

[0040] Step S108: Based on the feature vector of each statement in the target domain and the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center, a penalty matrix is ​​obtained.

[0041] In the technical solution provided by step S108 of the present invention, such as Figure 2 As shown, the penalty matrix is ​​obtained by combining the feature vector of each statement in the target domain with the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center.

[0042] Step S109: Adaptively correct the reward matrix and the penalty matrix respectively to obtain the corrected reward matrix and the corrected penalty matrix.

[0043] In the technical solution provided by step S109 of the present invention, such as Figure 2 As shown, the reward matrix and penalty matrix are adaptively corrected using the following formula to obtain the corrected reward matrix and penalty matrix:

[0044]

[0045] Among them, when for At that time, the reward matrix is ​​adaptively adjusted to obtain the adjusted reward matrix. ,when for At this time, the penalty matrix is ​​adaptively corrected to obtain the corrected penalty matrix. , As the first parameter, This is the second parameter.

[0046] Step S110: Based on the corrected reward matrix and the corrected penalty matrix, obtain the target correction matrix.

[0047] In the technical solution provided by step S110 of the present invention, such as Figure 2 As shown, based on the corrected reward matrix and the corrected penalty matrix, the expression for the target correction matrix is ​​obtained as follows:

[0048]

[0049] in, Correct the target matrix. The corrected reward matrix, This is the corrected penalty matrix.

[0050] Step S111: Obtain the correlation matrix corresponding to several images and several statements in the target domain.

[0051] In the technical solution provided by step S111 of the present invention, such as Figure 2 As shown, the relevance matrix of several images and several statements in the target domain is obtained. .

[0052] Step S112: Based on the correlation matrix and the target correction matrix, obtain the target correlation matrix.

[0053] In the technical solution provided by step S112 of the present invention, such as Figure 2 As shown, by adding the correlation matrix and the target correction matrix, the expression for the target correlation matrix is ​​obtained as follows:

[0054]

[0055] in, This is the target relevance matrix.

[0056] Step S113: Sort several images in the target domain based on the target relevance matrix to obtain a sequence of corrected images corresponding to each statement in the target domain.

[0057] In the technical solution provided by step S113 of the present invention, several images in the target domain are sorted according to the target correlation matrix to obtain a sequence of corrected images corresponding to each statement in the target domain.

[0058] The method described in this embodiment will be further described below.

[0059] As an optional embodiment, step S102, which involves processing several images and several sentences to obtain feature vectors for each image and each sentence, includes: using an image encoder in the source domain to extract features from several images to obtain feature vectors for each image; and using a text encoder in the source domain to extract features from several sentences to obtain feature vectors for each sentence.

[0060] As an optional embodiment, step S103, which involves using a target algorithm to cluster the feature vectors of each image to obtain a target number of category clusters, includes: using the k-means algorithm to cluster the feature vectors of each image to obtain a target number of category clusters.

[0061] In this embodiment, the k-means algorithm is used to cluster the feature vectors of each image to obtain the target number of category clusters.

[0062] As an optional embodiment, step S104, which involves determining the features of each cluster center based on the feature vectors of all images within each category cluster, includes: performing average pooling calculation on the feature vectors of all images within each category cluster to obtain the features of each cluster center.

[0063] In this embodiment, average pooling is performed on the feature vectors of all images within each category cluster to obtain the features of each cluster center.

[0064] As an optional embodiment, step S107, obtaining the reward matrix based on the feature vector of each statement in the target domain and the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center, includes: determining a first weight for each statement based on the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center; obtaining a reward value for each statement in the target domain based on the first weight of each statement, the feature vector of each statement in the target domain, and the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center; and obtaining the reward matrix based on the reward value of each statement in the target domain.

[0065] In this embodiment, based on the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center, the entropy of the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center is determined. Based on the entropy of the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center, the expression for determining the first weight of each statement is as follows:

[0066]

[0067] in, The entropy of the feature vector of any statement in the target domain stored in the reward cache corresponding to each cluster center. The sum of the entropies of the feature vectors of all statements in the target domain stored in the reward cache corresponding to each cluster center. The first weight for each statement.

[0068] Based on the first weight of each statement, the feature vector of each statement in the target domain, and the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center, the expression for the reward value of each statement in the target domain is as follows:

[0069]

[0070] in, For each statement in the target domain, (This is the feature vector) For each cluster center, the reward cache stores the feature vector of each statement in the target domain. The reward value for each statement in the target domain.

[0071] The reward value of each statement in the target domain is stacked according to its reward position to obtain a reward matrix.

[0072] As an optional embodiment, step S108, obtaining the penalty matrix based on the feature vector of each statement in the target domain and the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center, includes: determining a second weight for each statement based on the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center; obtaining a penalty value for each statement in the target domain based on the second weight of each statement, the feature vector of each statement in the target domain, and the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center; and obtaining the penalty matrix based on the penalty value of each statement in the target domain.

[0073] In this embodiment, based on the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center, the entropy of the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center is determined. Based on the entropy of the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center, the expression for determining the second weight of each statement is as follows:

[0074]

[0075] in, The second weight for each statement, The entropy of the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center. The sum of the entropies of the feature vectors of all statements in the target domain stored in the penalty cache corresponding to each cluster center.

[0076] Based on the second weight of each statement, the feature vector of each statement in the target domain, and the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center, the expression for the penalty value of each statement in the target domain is as follows:

[0077]

[0078] in, For each statement in the target domain, (This is the feature vector) For each cluster center, the feature vector of each statement in the target domain stored in the penalty cache is used. The penalty value for each statement in the target domain.

[0079] The penalty values ​​of each statement in the target domain are stacked according to their penalty positions to obtain a penalty matrix.

[0080] As an optional embodiment, step S110, obtaining the target correction matrix based on the modified reward matrix and the modified penalty matrix, includes: determining the difference between the modified reward matrix and the modified penalty matrix as the target correction matrix.

[0081] In this embodiment, the difference between each value at each position in the corrected reward matrix and each value at each corresponding position in the corrected penalty matrix is ​​determined as each value at each position in the target corrected matrix.

[0082] As an optional embodiment, step S112, obtaining the target correlation matrix based on the correlation matrix and the target correction matrix, includes: determining the target correlation matrix as the sum of the correlation matrix and the target correction matrix.

[0083] In this embodiment, the difference between each value at each position in the correlation matrix and each value at each corresponding position in the target correction matrix is ​​determined as each value at each position in the target correlation matrix.

[0084] In this embodiment of the invention, several images and several statements of a target domain are acquired, wherein each image is a pedestrian image and each statement is a statement describing a pedestrian image; the images and statements are processed to obtain feature vectors for each image and each statement; a target algorithm is used to cluster the feature vectors of each image to obtain target clusters; based on the feature vectors of all images within each cluster, the features of each cluster center are determined; for each cluster center, a relation matrix is ​​obtained based on the feature vectors of each statement and the features of each cluster center; based on each element in the relation matrix, a cache region for the cluster center to which the feature vector of each statement belongs is determined, wherein the cache region includes a reward cache and a penalty cache; based on the feature vectors of each statement in the target domain and the feature vectors of each statement in the target domain stored in the reward cache corresponding to each cluster center, a reward matrix is ​​obtained; Based on the feature vectors of each statement in the target domain and the feature vectors of each statement in the target domain stored in the penalty cache corresponding to each cluster center, a penalty matrix is ​​obtained. The reward matrix and penalty matrix are adaptively corrected to obtain corrected reward and penalty matrices. Based on the corrected reward and penalty matrices, a target correction matrix is ​​obtained. The relevance matrices corresponding to several images and statements in the target domain are obtained. Based on the relevance matrices and the target correction matrix, a target relevance matrix is ​​obtained. Based on the target relevance matrix, several images in the target domain are sorted to obtain a sequence of corrected images corresponding to each statement in the target domain. This approach solves the technical problems of poor cross-domain style adaptability and low cross-domain search accuracy in natural language cross-modal person search, achieving good cross-domain style adaptability and improved cross-domain search accuracy.

[0085] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0086] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0088] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a first processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0090] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A cross-domain, cross-modal pedestrian search method based on cache correction and test-time reordering, characterized in that, include: Obtain several images and several statements from the target domain, where each image is a pedestrian image and each statement is a statement describing the pedestrian image; Process several images and several sentences to obtain the feature vector of each image and the feature vector of each sentence; The target algorithm is used to cluster the feature vectors of each image to obtain a target number of category clusters; Based on the feature vectors of all images within each category cluster, the features of each cluster center are determined; For each cluster center, a relation matrix is ​​obtained based on the feature vector of each statement and the features of each cluster center; Based on each element in the relation matrix, determine the cache region of the cluster center to which the feature vector of each statement belongs, where the cache region includes reward cache and penalty cache; Based on the feature vector of each statement in the target domain and the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center, a reward matrix is ​​obtained; The reward matrix is ​​obtained by combining the feature vector of each statement in the target domain with the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center, including: Based on the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center, determine the first weight of each statement; Based on the first weight of each statement, the feature vector of each statement in the target domain, and the feature vector of each statement in the target domain stored in the reward cache corresponding to each cluster center, the reward value of each statement in the target domain is obtained. A reward matrix is ​​obtained based on the reward value of each statement in the target domain; Based on the feature vector of each statement in the target domain and the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center, a penalty matrix is ​​obtained; The penalty matrix is ​​obtained by combining the feature vector of each statement in the target domain with the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center, including: Based on the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center, the second weight of each statement is determined. Based on the second weight of each statement, the feature vector of each statement in the target domain, and the feature vector of each statement in the target domain stored in the penalty cache corresponding to each cluster center, the penalty value of each statement in the target domain is obtained. Based on the penalty value of each statement in the target domain, a penalty matrix is ​​obtained; The reward matrix and penalty matrix are adaptively corrected to obtain the corrected reward matrix and penalty matrix respectively; The corrected reward matrix and corrected penalty matrix are obtained by adaptively correcting the reward matrix and penalty matrix using the following formula: Where X is V R At that time, the reward matrix is ​​adaptively corrected to obtain the corrected reward matrix A(V). R When X is V P At this time, the penalty matrix is ​​adaptively corrected to obtain the corrected penalty matrix A(V). P ), As the first parameter, This is the second parameter; Based on the corrected reward matrix and the corrected penalty matrix, the target correction matrix is ​​obtained; The process of obtaining the target correction matrix based on the corrected reward matrix and the corrected penalty matrix includes: The difference between the corrected reward matrix and the corrected penalty matrix is ​​determined as the target correction matrix; Obtain the relevance matrix corresponding to several images and several statements in the target domain; Based on the correlation matrix and the target correction matrix, the target correlation matrix is ​​obtained; The process of obtaining the target correlation matrix based on the correlation matrix and the target correction matrix includes: The sum of the correlation matrix and the target correction matrix is ​​determined as the target correlation matrix; Based on the target relevance matrix, several images in the target domain are sorted to obtain a sequence of corrected images corresponding to each statement in the target domain.

2. The method according to claim 1, characterized in that, The process of processing several images and several statements to obtain the feature vector of each image and the feature vector of each statement includes: The source domain image encoder is used to extract features from several images to obtain the feature vector of each image; The source domain text encoder is used to extract features from several sentences, resulting in a feature vector for each sentence.

3. The method according to claim 1, characterized in that, The target algorithm is used to cluster the feature vectors of each image to obtain target category clusters, including: The k-means algorithm is used to cluster the feature vectors of each image to obtain the target number of category clusters.

4. The method according to claim 3, characterized in that, The process of determining the features of each cluster center based on the feature vectors of all images within each category cluster includes: Average pooling is performed on the feature vectors of all images within each category cluster to obtain the features of each cluster center.

5. A computer system, characterized in that... include: One or more processors, a computer-readable storage medium for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of claim 1.

6. A computer-readable storage medium storing computer program instructions thereon, characterized in that... When executed by a processor, the computer program instructions are used to implement the method of claim 1.

Images