Cross-domain and cross-modal pedestrian search method based on reordering during cache correction test

Abstract

The invention discloses a cross-domain and cross-modal pedestrian search method based on reordering during cache correction testing. The method comprises the following steps: constructing a reward cache and a punishment cache for each clustering center; obtaining a reward matrix based on the feature vector of each statement of the target domain and the feature vector of each statement of the target domain stored in the reward cache corresponding to each clustering center; based on the feature vector of each statement of the target domain and the feature vector of each statement of the target domain stored in the corresponding punishment cache of each clustering center, obtaining a punishment matrix; performing self-adaptive correction on the reward matrix and the penalty matrix to obtain a corrected reward matrix and a corrected penalty matrix; obtaining a target correction matrix based on the corrected reward matrix and the corrected penalty matrix; and based on the target correction matrix, obtaining a target correlation matrix to sort the images. The technical problems of poor cross-domain style adaptability and low cross-domain search accuracy in natural language cross-modal pedestrian search are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of online reordering in the test phase of a natural language cross-modal pedestrian search task, and in particular to a cross-domain and cross-modal pedestrian search method based on cache correction test-time reordering. Background Art

[0002] Natural language cross-modal pedestrian search aims to search for images corresponding to target persons from image libraries through text descriptions. It can provide more flexible and effective queries in complex environments such as low light, occlusion, and poor viewing angles, and has broad application prospects in intelligent video analysis, security monitoring and other fields. Most of the existing representative methods focus on solving the problem of modal heterogeneity and ignore the cross-domain difficulties in real applications. For single-modal cross-domain problems, some methods attempt to fine-tune the target domain by generating pseudo labels (such as pseudo pedestrian identities), but such methods are difficult to apply in natural language cross-modal pedestrian search because it is difficult to generate sufficient and high-quality pseudo-sentence descriptions (sentences are more flexible and more difficult to generate). In addition, the simultaneous existence of cross-modal and cross-domain problems makes it difficult for a single cross-domain adaptation strategy to effectively cope with the style changes in different domains of two modal data at the same time. Summary of the invention

[0003] The embodiment of the present invention provides a cross-domain and cross-modal pedestrian search method based on cache correction test-time reordering, so as to at least solve the technical problems of poor cross-domain style adaptability and low cross-domain search accuracy in natural language cross-modal pedestrian search.

[0004] According to one aspect of an embodiment of the present invention, a cross-domain and cross-modal pedestrian search method based on cache correction test-time reordering is provided. The method may include: obtaining a number of images and a number of sentences in the target domain, wherein each image is a pedestrian image and each sentence is a sentence describing the pedestrian image; processing the number of images and the number of sentences to obtain a feature vector for each image and a feature vector for each sentence; clustering the feature vector of each image using a target algorithm to obtain target category clusters; determining the features of each cluster center based on the feature vectors of all images in each category cluster; for each cluster center, obtaining a relationship matrix based on the feature vector of each sentence and the features of each cluster center; determining the cache area of ​​the cluster center to which the feature vector of each sentence belongs based on each element in the relationship matrix, wherein the cache area includes a reward cache and a penalty cache; based on the feature vector of each sentence in the target domain; The reward matrix is ​​obtained by combining the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center; the penalty matrix is ​​obtained based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center; the reward matrix and the penalty matrix are adaptively modified respectively to obtain a modified reward matrix and a modified penalty matrix; a target correction matrix is ​​obtained based on the modified reward matrix and the modified penalty matrix; a correlation matrix corresponding to several images and several sentences in the target domain is obtained; a target correlation matrix is ​​obtained based on the correlation matrix and the target correction matrix; several images in the target domain are sorted based on the target correlation matrix to obtain a sequence of each modified image corresponding to each sentence in the target domain.

[0005] Optionally, the processing of the plurality of images and the plurality of sentences to obtain a feature vector for each image and a feature vector for each sentence includes: performing feature extraction on the plurality of images using an image encoder of a source domain to obtain a feature vector for each image; and performing feature extraction on the plurality of sentences using a text encoder of the source domain to obtain a feature vector for each sentence.

[0006] Optionally, clustering the feature vector of each image using a target algorithm to obtain a target number of category clusters includes: clustering the feature vector of each image using a k-means algorithm to obtain a target number of category clusters.

[0007] Optionally, determining the features of each cluster center based on the feature vectors of all images in each category cluster includes: performing average pooling calculation on the feature vectors of all images in each category cluster to obtain the features of each cluster center.

[0008] Optionally, the reward matrix is ​​obtained based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center, including: determining the first weight of each sentence based on the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center; obtaining the reward value of each sentence in the target domain based on the first weight of each sentence, the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center; obtaining the reward matrix based on the reward value of each sentence in the target domain.

[0009] Optionally, the penalty matrix is ​​obtained based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center, including: determining the second weight of each sentence based on the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center; obtaining the penalty value of each sentence in the target domain based on the second weight of each sentence, the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center; and obtaining the penalty matrix based on the penalty value of each sentence in the target domain.

[0010] Optionally, obtaining the target correction matrix based on the corrected reward matrix and the corrected penalty matrix includes: determining the difference between the corrected reward matrix and the corrected penalty matrix as the target correction matrix.

[0011] Optionally, obtaining the target correlation matrix based on the correlation matrix and the target correction matrix includes: determining the sum of the correlation matrix and the target correction matrix as the target correlation matrix.

[0012] Beneficial effects of the present invention: The present invention proposes a cross-domain and cross-modal pedestrian search method based on cache correction and re-ranking during testing. First, pedestrian images are clustered, and appropriate sentences are selected as single-modal relays based on each cluster center to construct reward and penalty caches, which are used to store sentences that help alleviate cross-domain difficulties respectively; then, under the guidance of the sentences stored in the reward and penalty caches, the corresponding reward values ​​and penalty values ​​are calculated respectively, and the positions of the original image-sentence pairs that need to be corrected are accurately located to complete the refined correction of the original correlation; finally, based on the corrected image-sentence correlation, the original retrieval results are re-ranked during testing without referring to the source domain data and without adding additional calculation and time overhead. It can be quickly and easily applied to multiple representative advanced methods in the field of natural language cross-modal pedestrian search, and all of them can achieve significant improvement in cross-domain search accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of a cross-domain and cross-modal pedestrian search method based on cache correction test-time reordering according to an embodiment of the present invention; Figure 2 It is a structural block diagram of a cross-domain and cross-modal pedestrian search method based on cache correction test-time reordering according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0016] Example 1 According to an embodiment of the present invention, a cross-domain and cross-modal pedestrian search method based on reordering during cache correction testing is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0017] Figure 1 is a flow chart of a cross-domain and cross-modal pedestrian search method based on cache correction test-time reordering according to an embodiment of the present invention, such as Figure 1 As shown, the method may include the following steps: Step S101, obtaining a plurality of images and a plurality of sentences in a target domain, wherein each image is a pedestrian image, and each sentence is a sentence describing a pedestrian image.

[0018] In the technical solution provided in step S101 of the present invention, the target domain is obtained. images and statements.

[0019] Step S102, processing a plurality of images and a plurality of sentences to obtain a feature vector of each image and a feature vector of each sentence.

[0020] In the technical solution provided in the above step S102 of the present invention, Figure 2 is a structural block diagram of a cross-domain and cross-modal pedestrian search method based on cache correction test-time reordering according to an embodiment of the present invention, such as Figure 2 As shown in images and statements to obtain the feature vector of each image and the feature vector for each sentence ,in, represents the feature dimension, is the feature space.

[0021] Step S103: clustering the feature vector of each image using a target algorithm to obtain target category clusters.

[0022] In the technical solution provided in step S103 of the present invention, if Figure 2 As shown, the target algorithm is used to cluster the feature vectors of each image, and the Category clusters.

[0023] Step S104: determining the features of each cluster center based on the feature vectors of all images in each category cluster.

[0024] In the technical solution provided in step S104 of the present invention, if Figure 2 As shown, the feature vectors of all images in each category cluster are processed to determine the features of each cluster center. .

[0025] Step S105 , for each cluster center, a relationship matrix is ​​obtained based on the feature vector of each sentence and the features of each cluster center.

[0026] In the technical solution provided in step S105 of the present invention, for each cluster center, for each sentence feature vector , calculate its characteristics with the cluster center The cosine similarity between them gives the relationship matrix :

[0027] in, is the relationship matrix.

[0028] Step S106, based on each element in the relationship matrix, determining the cache area of ​​the cluster center to which the feature vector of each sentence belongs, wherein the cache area includes a reward cache and a penalty cache.

[0029] In the technical solution provided in step S106 of the present invention, if Figure 2 As shown, in order to clarify the feature vector of each sentence And the distribution characteristics between the characteristics of all cluster centers, calculate Entropy :

[0030] Through a Hyperparameters in range To determine the feature vector of the sentence The direction of influence, that is, when solving the problem of domain style differences, the feature vector of the sentence Should be filled in the reward cache or the penalty cache, if entropy Less than threshold , which means that the feature vector of the sentence The impact caused by cross-domain is small, filling into the reward cache, on the contrary, the entropy value Greater than or equal to threshold The feature vector of the sentence Should be used to fill the penalty cache. In order to save cache space, the maximum capacity of the reward cache and penalty cache of each cluster center is set to and , if the number of feature vectors for statements in the cache exceeds the maximum capacity or , the feature vectors of sentences with smaller entropy values ​​will be retained, and the feature vectors of each sentence that exceeds the maximum capacity will be discarded.

[0031] Step S107, obtaining a reward matrix based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center.

[0032] In the technical solution provided in step S107 of the present invention, if Figure 2 As shown, the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center are processed to obtain a reward matrix.

[0033] Step S108 , obtaining a penalty matrix based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center.

[0034] In the technical solution provided in step S108 of the present invention, if Figure 2 As shown, the penalty matrix is ​​obtained for the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center.

[0035] Step S109, adaptively modify the reward matrix and the penalty matrix to obtain a modified reward matrix and a modified penalty matrix.

[0036] In the technical solution provided in step S109 of the present invention, if Figure 2 As shown, the reward matrix and penalty matrix are adaptively modified by the following formula to obtain the modified reward matrix and the modified penalty matrix:

[0037] Among them, when for When , the reward matrix is ​​adaptively modified to obtain the modified reward matrix ,when for When , the penalty matrix is ​​adaptively modified to obtain the modified penalty matrix , is the first parameter, is the second parameter.

[0038] Step S110, obtaining a target correction matrix based on the corrected reward matrix and the corrected penalty matrix.

[0039] In the technical solution provided in step S110 of the present invention, if Figure 2 As shown, based on the corrected reward matrix and the corrected penalty matrix, the expression of the target correction matrix is:

[0040] in, is the target correction matrix, is the modified reward matrix, is the modified penalty matrix.

[0041] Step S111, obtaining a correlation matrix corresponding to a plurality of images and a plurality of sentences in the target domain.

[0042] In the technical solution provided in step S111 of the present invention, if Figure 2As shown, the correlation matrix corresponding to several images and several sentences in the target domain is obtained. .

[0043] Step S112, obtaining a target correlation matrix based on the correlation matrix and the target correction matrix.

[0044] In the technical solution provided in step S112 of the present invention, if Figure 2 As shown, the correlation matrix and the target correction matrix are added to obtain the expression of the target correlation matrix:

[0045] in, is the target correlation matrix.

[0046] Step S113, sorting a plurality of images of the target domain based on the target relevance matrix to obtain a sequence of each corrected image corresponding to each sentence of the target domain.

[0047] In the technical solution provided in the above step S113 of the present invention, a plurality of images in the target domain are sorted according to the target correlation matrix to obtain a sequence of each corrected image corresponding to each sentence in the target domain.

[0048] The above method of this embodiment is further introduced below.

[0049] As an optional implementation method, step S102, the processing of a plurality of images and a plurality of sentences to obtain a feature vector for each image and a feature vector for each sentence includes: using an image encoder of a source domain to perform feature extraction on a plurality of images to obtain a feature vector for each image; using a text encoder of a source domain to perform feature extraction on a plurality of sentences to obtain a feature vector for each sentence.

[0050] As an optional implementation manner, step S103, clustering the feature vector of each image using a target algorithm to obtain target category clusters, includes: clustering the feature vector of each image using a k-means algorithm to obtain target category clusters.

[0051] In this embodiment, the k-means algorithm is used to cluster the feature vector of each image to obtain target category clusters.

[0052] As an optional implementation manner, step S104, determining the features of each cluster center based on the feature vectors of all images in each category cluster, includes: performing average pooling calculation on the feature vectors of all images in each category cluster to obtain the features of each cluster center.

[0053] In this embodiment, average pooling calculation is performed on the feature vectors of all images in each category cluster to obtain the features of each cluster center.

[0054] As an optional implementation method, step S107, the reward matrix is ​​obtained based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center, including: determining the first weight of each sentence based on the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center; obtaining the reward value of each sentence in the target domain based on the first weight of each sentence, the feature vector of each sentence in the target domain, and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center; obtaining the reward matrix based on the reward value of each sentence in the target domain.

[0055] In this embodiment, based on the feature vector of each sentence of the target domain stored in the reward cache corresponding to each cluster center, the entropy of the feature vector of each sentence of the target domain stored in the reward cache corresponding to each cluster center is determined. Based on the entropy of the feature vector of each sentence of the target domain stored in the reward cache corresponding to each cluster center, the expression for determining the first weight of each sentence is:

[0056] in, is the entropy of the feature vector of any sentence in the target domain stored in the reward cache corresponding to each cluster center, The sum of the entropy of the feature vectors of all sentences in the target domain stored in the reward cache corresponding to each cluster center, The first weight for each statement.

[0057] Based on the first weight of each sentence, the feature vector of each sentence in the target domain, and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center, the expression of the reward value of each sentence in the target domain is obtained as follows:

[0058] in, is the feature vector of each sentence in the target domain, For each cluster center, the feature vector of each sentence in the target domain stored in the reward cache is is the reward value for each sentence in the target domain.

[0059] The reward value of each sentence in the target domain is stacked according to its reward position to obtain the reward matrix.

[0060] As an optional implementation method, step S108, the penalty matrix is ​​obtained based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center, including: determining the second weight of each sentence based on the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center; obtaining the penalty value of each sentence in the target domain based on the second weight of each sentence, the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center; obtaining the penalty matrix based on the penalty value of each sentence in the target domain.

[0061] In this embodiment, based on the feature vector of each sentence of the target domain stored in the penalty cache corresponding to each cluster center, the entropy of the feature vector of each sentence of the target domain stored in the penalty cache corresponding to each cluster center is determined. Based on the entropy of the feature vector of each sentence of the target domain stored in the penalty cache corresponding to each cluster center, the expression for determining the second weight of each sentence is:

[0062] in, is the second weight of each statement, For each cluster center, the entropy of the feature vector of each sentence in the target domain stored in the penalty cache is calculated. For each cluster center, the sum of the entropies of the feature vectors of all sentences in the target domain stored in the penalty cache is calculated.

[0063] Based on the second weight of each sentence, the feature vector of each sentence in the target domain, and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center, the expression of the penalty value of each sentence in the target domain is obtained as follows:

[0064] in, is the feature vector of each sentence in the target domain, For each cluster center pair, the feature vector of each sentence in the target domain stored in the penalty cache is is the penalty value for each sentence in the target domain.

[0065] The penalty value of each sentence in the target domain is stacked according to its penalty position to obtain the penalty matrix.

[0066] As an optional implementation manner, step S110, obtaining the target correction matrix based on the corrected reward matrix and the corrected penalty matrix, includes: determining the difference between the corrected reward matrix and the corrected penalty matrix as the target correction matrix.

[0067] In this embodiment, the difference between each value at each position in the modified reward matrix and each value at each position corresponding to the modified penalty matrix is ​​determined as each value at each position of the target modified matrix.

[0068] As an optional implementation manner, step S112, obtaining the target correlation matrix based on the correlation matrix and the target correction matrix, includes: determining the sum of the correlation matrix and the target correction matrix as the target correlation matrix.

[0069] In this embodiment, the difference between each value at each position in the correlation matrix and each value at each position corresponding to the target correction matrix is ​​determined as each value at each position of the target correlation matrix.

[0070] In an embodiment of the present invention, a plurality of images and a plurality of sentences of a target domain are obtained, wherein each image is a pedestrian image, and each sentence is a sentence describing a pedestrian image; the plurality of images and the plurality of sentences are processed to obtain a feature vector of each image and a feature vector of each sentence; the feature vector of each image is clustered using a target algorithm to obtain target category clusters; based on the feature vectors of all images in each category cluster, the features of each cluster center are determined; for each cluster center, based on the feature vector of each sentence and the features of each cluster center, a relationship matrix is ​​obtained; based on each element in the relationship matrix, a cache area of ​​the cluster center to which the feature vector of each sentence belongs is determined, wherein the cache area includes a reward cache and a penalty cache; based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center, a reward matrix is ​​obtained; Based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center, a penalty matrix is ​​obtained; the reward matrix and the penalty matrix are adaptively corrected respectively to obtain a corrected reward matrix and a corrected penalty matrix; based on the corrected reward matrix and the corrected penalty matrix, a target correction matrix is ​​obtained; a correlation matrix corresponding to several images and several sentences in the target domain is obtained; based on the correlation matrix and the target correction matrix, a target correlation matrix is ​​obtained; based on the target correlation matrix, several images in the target domain are sorted to obtain a sequence of each corrected image corresponding to each sentence in the target domain, which solves the technical problems of poor cross-domain style adaptability and low cross-domain search accuracy in natural language cross-modal pedestrian search, and achieves the technical effect of good cross-domain style adaptability and improved cross-domain search accuracy in natural language cross-modal pedestrian search.

[0071] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0072] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0073] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0074] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0075] In addition, each functional unit in each embodiment of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0076] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A cross-domain and cross-modal pedestrian search method based on cache-corrected test-time reordering, characterized in that: include: Obtain a number of images and a number of sentences in the target domain, wherein each image is a pedestrian image, and each sentence is a sentence describing the pedestrian image; Processing a number of images and a number of sentences to obtain a feature vector of each image and a feature vector of each sentence; The target algorithm is used to cluster the feature vector of each image to obtain the target category clusters; Based on the feature vectors of all images in each category cluster, the features of each cluster center are determined; For each cluster center, a relationship matrix is ​​obtained based on the feature vector of each sentence and the features of each cluster center; Based on each element in the relationship matrix, determine the cache area of ​​the cluster center to which the feature vector of each sentence belongs, wherein the cache area includes a reward cache and a penalty cache; A reward matrix is ​​obtained based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center; A penalty matrix is ​​obtained based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center; Adaptively modify the reward matrix and the penalty matrix respectively to obtain a modified reward matrix and a modified penalty matrix; Based on the corrected reward matrix and the corrected penalty matrix, a target correction matrix is ​​obtained; Obtain the correlation matrix corresponding to several images and several sentences in the target domain; Based on the correlation matrix and the target correction matrix, a target correlation matrix is ​​obtained; Several images of the target domain are sorted based on the target relevance matrix to obtain a sequence of each corrected image corresponding to each sentence in the target domain.

2. The method according to claim 1, characterized in that The processing of the plurality of images and the plurality of sentences to obtain a feature vector of each image and a feature vector of each sentence includes: Use the image encoder in the source domain to extract features from several images and obtain the feature vector of each image; The text encoder of the source domain is used to extract features from several sentences to obtain the feature vector of each sentence.

3. The method according to claim 1, characterized in that The target algorithm is used to cluster the feature vectors of each image to obtain target category clusters, including: The k-means algorithm is used to cluster the feature vector of each image to obtain the target category clusters.

4. The method according to claim 3, characterized in that The feature vectors of all images in each category cluster are used to determine the features of each cluster center, including: The feature vectors of all images in each category cluster are averaged and pooled to obtain the features of each cluster center.

5. The method according to claim 4, characterized in that The reward matrix is ​​obtained based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center, including: Determine a first weight of each sentence based on a feature vector of each sentence of the target domain stored in the reward cache corresponding to each cluster center; Based on the first weight of each sentence, the feature vector of each sentence in the target domain, and the feature vector of each sentence in the target domain stored in the reward cache corresponding to each cluster center, a reward value of each sentence in the target domain is obtained; Based on the reward value of each sentence in the target domain, a reward matrix is ​​obtained.

6. The method according to claim 5, characterized in that The penalty matrix is ​​obtained based on the feature vector of each sentence in the target domain and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center, including: Determine a second weight of each sentence based on a feature vector of each sentence of the target domain stored in the penalty cache corresponding to each cluster center; Based on the second weight of each sentence, the feature vector of each sentence in the target domain, and the feature vector of each sentence in the target domain stored in the penalty cache corresponding to each cluster center, a penalty value of each sentence in the target domain is obtained; Based on the penalty value of each sentence in the target domain, a penalty matrix is ​​obtained.

7. The method according to claim 6, characterized in that The target correction matrix is ​​obtained based on the corrected reward matrix and the corrected penalty matrix, including: The difference between the modified reward matrix and the modified penalty matrix is ​​determined as the target correction matrix.

8. The method according to claim 7, characterized in that The target correlation matrix is ​​obtained based on the correlation matrix and the target correction matrix, including: The sum of the correlation matrix and the target correction matrix is ​​determined as the target correlation matrix.

9. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

10. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

Images