Training Method for Weakly Supervised Cross-Modal Target Retrieval Model Based on Pseudo-Label Optimization

Abstract

This invention relates to the fields of computer vision and pattern recognition, and more particularly to a weakly supervised cross-modal target retrieval model training method based on pseudo-label optimization. By constructing a multi-view hypergraph and performing reliability-guided relation consistency inference based on the hypergraph, this invention effectively fuses sample relationships from multiple views, making the information from multiple views complementary. It can uncover the consistency in multi-view sample relationships, reduce the mutual influence of noise from multi-view samples, and generate final pseudo-labels for learning the joint feature embedding space of visual text, thus improving the quality of the pseudo-labels and achieving better target retrieval performance.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and pattern recognition, and in particular to a method for training a weakly supervised cross-modal target retrieval model based on pseudo-label optimization. Background Technology

[0002] Cross-modal object retrieval aims to associate images of targets such as people and vehicles with corresponding query text descriptions, and has wide applications in smart cities and smart security. However, obtaining large amounts of labeled data is not easy. Therefore, the weakly supervised cross-modal object retrieval task has been proposed, which only includes text-image pairs without identity annotations. This task faces challenges such as complex intramodal variations and cross-modal matching ambiguities.

[0003] Existing text-based unsupervised pedestrian retrieval methods typically employ self-training and clustering to generate pseudo-labels. However, due to the significant noise present in image and text data, existing cross-modal retrieval models perform poorly in cross-modal retrieval tasks. Furthermore, because clustering-based pseudo-labels can only utilize sample relationships from a limited view, their reliability is poor, further degrading the performance of the trained cross-modal retrieval model. Therefore, obtaining a cross-modal retrieval model with better performance is a crucial problem that urgently needs to be solved. Summary of the Invention

[0004] This invention provides a weakly supervised cross-modal target retrieval model training method based on pseudo-label optimization, which solves the problem of low pseudo-label quality in the prior art and enables the model trained based on pseudo-label optimization to have better target retrieval performance.

[0005] This invention provides a method for supervising the training of a cross-modal target retrieval model based on a weak target retrieval model optimized by pseudo-labels, comprising the following steps:

[0006] Visual embedding features are extracted from target domain sample images; text embedding features are extracted from target domain text images.

[0007] A multi-view hypergraph is constructed based on visual and text embedding features using clustering methods.

[0008] Reliability-guided relational consistency inference is performed based on the multi-view hypergraph to obtain reliable pseudo-labels;

[0009] The target retrieval model is obtained by training based on reliable pseudo-labels.

[0010] The method for supervising the training of a cross-modal target retrieval model based on a weak target retrieval model optimized by pseudo-labels, provided by the present invention, includes the following steps: Constructing a multi-view hypergraph using clustering methods based on visual embedding features and text embedding features:

[0011] Based on visual embedding features, a visual clustering relationship graph is obtained using clustering methods.

[0012] Based on the text embedding features, a text clustering relationship graph is obtained using a clustering method;

[0013] The fused features are obtained by fusing visual embedding features and text embedding features, and a joint visual-text clustering relationship graph is obtained by using the fused features and a clustering method.

[0014] A multi-view hypergraph is obtained based on the visual clustering relationship graph, the text clustering relationship graph, and the visual-text joint clustering relationship graph.

[0015] The present invention provides a method for supervising the training of a cross-modal target retrieval model based on a weak target retrieval model optimized by pseudo-labels.

[0016] Based on the visual clustering relationship graph, the text clustering relationship graph, and the joint visual-text clustering relationship graph, a multi-view hypergraph is obtained, specifically as follows:

[0017] Based on the visual clustering relationship graph, the text clustering relationship graph, and the visual-text joint clustering relationship graph, determine the hyperedges of the multi-view hypergraph and the association matrix of the multi-view hypergraph;

[0018] The weight of the hyperedge is determined based on the node similarity corresponding to the hyperedge in the multi-view hypergraph.

[0019] The adjacency matrix of a multi-view hypergraph is determined based on the association matrix and the weights of the hyperedges.

[0020] The method for supervising cross-modal target retrieval model training based on pseudo-label optimization provided by the present invention includes the following steps: Reliability-guided relation consistency inference based on a multi-view hypergraph to obtain reliable pseudo-labels.

[0021] Based on the visual clustering relationship diagram, the text clustering relationship diagram, and the visual-text joint clustering relationship diagram, determine the initial clusters and initial pseudo-labels;

[0022] Based on the initial clustering and initial pseudo-labels, calculate the intra-cluster tightness and determine the candidate set with the highest intra-cluster tightness to obtain reliable instances and reliable pseudo-labels of the candidate set;

[0023] Label propagation is performed based on reliable instances and reliable pseudo-labels in the candidate set to obtain pseudo-labels for all data instances;

[0024] Unreliable instances are removed based on classification uncertainty to obtain reliable pseudo-labels.

[0025] According to the method for supervising the training of a weak target retrieval model based on pseudo-label optimization provided by the present invention, the target retrieval model is trained based on reliable pseudo-labels to obtain the target retrieval model. Specifically, the target retrieval model is obtained by training based on reliable pseudo-labels and training the intra-modal and cross-modal matching objective function between visual or text embeddings and ID prototypes.

[0026] The present invention also provides a supervised cross-modal target retrieval model training device based on pseudo-label optimization weak target retrieval model, comprising:

[0027] The feature extraction module is used to extract visual embedding features from target domain sample images and text embedding features from target domain text images.

[0028] The multi-view hypergraph module is used to construct multi-view hypergraphs based on visual embedding features and text embedding features using clustering methods.

[0029] The reliable pseudo-label reasoning module is used to perform reliability-guided relation consistency reasoning based on the multi-view hypergraph to obtain reliable pseudo-labels.

[0030] The training module is used to train a target retrieval model based on reliable pseudo-labels.

[0031] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method for training a cross-modal target retrieval model supervised by a weak target retrieval model based on pseudo-label optimization as described above.

[0032] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for training a cross-modal target retrieval model supervised by a weak target retrieval model based on pseudo-label optimization as described above.

[0033] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method for training a cross-modal target retrieval model supervised by a weak target retrieval model based on pseudo-label optimization as described above.

[0034] The present invention provides a weakly supervised cross-modal target retrieval model training method based on pseudo-label optimization. By constructing a multi-view hypergraph and performing reliability-guided relation consistency inference based on the multi-view hypergraph, it can effectively fuse sample relationships from multiple views, making the information of multiple views complementary, mining the consistency in multi-view sample relationships, reducing the mutual influence of multi-view sample noise, and generating the final pseudo-label for visual-text joint feature embedding space learning, thereby improving the quality of pseudo-labels and achieving better target retrieval performance. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0036] Figure 1 This is one of the flowcharts illustrating the supervised training method for cross-modal target retrieval models based on pseudo-label optimization provided by the present invention.

[0037] Figure 2 This is the second flowchart of the method for supervising the training of a cross-modal target retrieval model based on pseudo-label optimization provided by the present invention.

[0038] Figure 3 This is a schematic diagram of the structure of the training device for a cross-modal target retrieval model supervised by a weak target retrieval model based on pseudo-label optimization provided by the present invention;

[0039] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0041] The following is combined Figures 1-4 This invention describes a method for supervising the training of a cross-modal target retrieval model based on a pseudo-label-optimized weak target retrieval model.

[0042] Figure 1 This is one of the flowcharts illustrating the supervised cross-modal target retrieval model training method based on pseudo-label optimization according to an embodiment of the present invention. Figure 1 As shown, steps 110-140 are included, specifically:

[0043] Step 110: Extract visual embedding features from the target domain sample images; extract text embedding features from the target domain text images.

[0044] Step 120: Construct a multi-view hypergraph using clustering methods based on visual embedding features and text embedding features;

[0045] Step 130: Perform reliability-guided relational consistency reasoning based on the multi-view hypergraph to obtain reliable pseudo-labels;

[0046] Step 140: Train the target retrieval model based on reliable pseudo-labels.

[0047] Figure 2 This is the second flowchart illustrating the supervised cross-modal target retrieval model training method based on pseudo-label optimization of the present invention. The specific implementation of each step of the supervised cross-modal target retrieval model training method based on pseudo-label optimization of the present invention is as follows:

[0048] In step 110 of this embodiment of the invention, a text encoder and a visual encoder can be used to extract embedded features from images and videos. Specifically, let f θ and f φ These are the text encoder and the visual encoder, respectively, with θ and φ being their parameters. Given a training dataset, where I i It is the i-th image, T i Let f represent the i-th text description, and N be the number of text-image pairs. θ and f φ Extract visual embedding features F respectively v and text embedding features F t It is understood that any text encoder and visual encoder can be used in the embodiments of the present invention to achieve the technical effects of the present invention.

[0049] In step 120 of this embodiment of the invention, a multi-view hypergraph G is constructed using a clustering method based on visual embedding features and text embedding features. Specifically, multiple relationship graphs are first constructed. Where g j ={V,E j} is the j-th relation graph, where the image and text data X are nodes of relation graph g. Let N be the set of vertices in a relational graph, containing N nodes. Let L be the set of edges in the graph, containing L edges, and M be the number of edges in the graph. (The last part, "e", appears to be a typo and can be left as is.) k The instances corresponding to the included nodes belong to the same ID. Then, multiple relation graphs g are merged to obtain a multi-relation hypergraph G = (V, E, W), where V is the vertex set of the multi-relation hypergraph. G is the set of hyperedges in a multi-relation hypergraph, and W is the edge weight of each hyperedge in the multi-relation hypergraph. Thus, the view hypergraph G can be used to mine complementary information in different views and to represent complex sample relationships.

[0050] In step 130 of this embodiment, reliable pseudo-labels are obtained by performing reliability-guided relation consistency reasoning based on the multi-view hypergraph. Specifically, a subset of reliable instances and their corresponding reliable pseudo-labels are determined, and pseudo-labels for all data instances are obtained using a label propagation method on the multi-view hypergraph G based on the reliable instances and their corresponding reliable pseudo-labels. It is understood that, compared to existing semi-supervised label propagation methods based on partially known real labels, the label propagation method of this embodiment is based on utilizing noisy pseudo-labels from multiple single views, achieving high reliability of pseudo-labels without requiring additional data annotation work.

[0051] Therefore, the weakly supervised cross-modal target retrieval model training method based on pseudo-label optimization provided in this embodiment of the invention can effectively fuse sample relationships from multiple views by constructing a multi-view hypergraph and performing reliability-guided relation consistency inference based on the multi-view hypergraph. This enables the information from multiple views to be complementary, uncovers the consistency in multi-view sample relationships, reduces the mutual influence of multi-view sample noise, and generates the final pseudo-label for learning the joint feature embedding space of visual text, thereby improving the quality of the pseudo-label and achieving better target retrieval performance.

[0052] In this embodiment of the invention, step 12 includes steps 121-124, specifically:

[0053] Step 121: Based on the visual embedding features, use clustering methods to obtain a visual clustering relationship graph;

[0054] Step 122: Based on the text embedding features, use clustering methods to obtain a text clustering relationship graph;

[0055] Step 123: Perform feature fusion based on visual embedding features and text embedding features to obtain fused features, and use clustering methods based on the fused features to obtain a joint visual-text clustering relationship graph;

[0056] Step 124: Obtain the multi-view hypergraph based on the visual clustering relationship graph, the text clustering relationship graph, and the visual-text joint clustering relationship graph.

[0057] Specifically, in step 121 of this embodiment of the invention, the image feature similarity matrix is ​​first calculated based on the visual embedding features. in Then in S v Run the DBSCAN clustering algorithm and generate pseudo-labels y. v According to y v Determine the connections between nodes to obtain a visual clustering relationship graph g. v .

[0058] In step 122 of this embodiment of the invention, the text feature similarity matrix is ​​first calculated based on the text embedding features. in Then in S t Run the DBSCAN clustering algorithm and generate pseudo-labels y. t According to y t Determine the connections between nodes to obtain a visual clustering relationship graph g. t .

[0059] In step 123 of this embodiment of the invention, the visual embedding F is first performed. v and text embedding F t By concatenating the components, the visual-text fusion feature F is obtained. vt Then, the similarity matrix of fusion features is calculated based on the visual-text fusion features. in Then in S vt Run the DBSCAN clustering algorithm and generate pseudo-labels y. vt According to y vt Determine the connections between nodes to obtain a visual clustering relationship graph g. vt .

[0060] It is understood that steps 121-123 of the embodiments of the present invention can employ other clustering algorithms, such as the OPTICS clustering algorithm and the DENCLUE clustering algorithm, all of which can achieve the technical effects of the present invention.

[0061] In this embodiment of the invention, step 124 includes the following steps:

[0062] Based on the visual clustering relationship graph, the text clustering relationship graph, and the visual-text joint clustering relationship graph, determine the hyperedges of the multi-view hypergraph and the association matrix of the multi-view hypergraph;

[0063] The weight of the hyperedge is determined based on the node similarity corresponding to the hyperedge in the multi-view hypergraph.

[0064] The adjacency matrix of a multi-view hypergraph is determined based on the association matrix and the weights of the hyperedges.

[0065] In step 124 of this embodiment of the invention, the clustering relationship graph g obtained in steps 121-123 is... v g t g vt edge The merging process yields a multi-view hypergraph G, which can be represented by an incidence matrix. express:

[0066]

[0067] The weight of each hyperedge e is:

[0068]

[0069] Where S(i,j) represents the similarity between nodes i and j in hyperedge e, and σ is the median of the similarity between all vertex pairs.

[0070] The adjacency matrix of the multiview hypergraph G is:

[0071]

[0072] in A is the diagonal matrix corresponding to the degree of the hypergraph. Using A, label propagation can be performed on the graph, and pseudo-labels can be optimized.

[0073] The embodiments of the present invention obtain a multi-view hypergraph containing complementary information from multiple angles through the above steps, which enables the pseudo-labels to be optimized based on other relationships across viewpoints, making the obtained pseudo-labels more reliable.

[0074] In this embodiment of the invention, the reliable pseudo-label obtained by performing reliability-guided relation consistency reasoning based on a multi-view hypergraph includes the following steps:

[0075] Based on the visual clustering relationship diagram, the text clustering relationship diagram, and the visual-text joint clustering relationship diagram, determine the initial clusters and initial pseudo-labels;

[0076] Based on the initial clustering and initial pseudo-labels, calculate the intra-cluster tightness and determine the candidate set with the highest intra-cluster tightness to obtain reliable instances and reliable pseudo-labels of the candidate set;

[0077] Label propagation is performed based on reliable instances and reliable pseudo-labels in the candidate set to obtain pseudo-labels for all data instances;

[0078] Unreliable instances are removed based on classification uncertainty to obtain reliable pseudo-labels.

[0079] After constructing the multi-relation hypergraph G, reliability-guided relation consistency reasoning will be deployed on it to optimize the noisy labels of single views, resulting in the final multi-view structure. Figure 1 To avoid false labels, label propagation is an effective method for smoothing label prediction. The process followed in this embodiment of the invention performs label propagation on a multi-relationship hypergraph G.

[0080] In this embodiment of the invention, for the i-th cluster, we take each instance as a reference and select its nearest K-1 instances to construct several candidate sets s. t Based on the initial clustering and initial pseudo-labels, the intra-cluster tightness is calculated, and its expression is:

[0081]

[0082] Where M is the number of relational graphs. Let be the number of instances in the candidate set that belong to the same cluster as the reference instance in the j-th relationship graph, and K be the number of instances in the candidate set. Based on intra-cluster tightness, determine the candidate set with the highest intra-cluster tightness, and obtain the k reliable instances of the i-th cluster. and reliable fake labels

[0083] The embodiments of the present invention capture the relationship between the reference instance and its nearest neighbor instances in the candidate set from the perspective of the M view through the intra-class compactness C, reflecting the consistency of the relationship between multiple views.

[0084] After obtaining a reliable instance and its corresponding pseudo-label in each cluster, this embodiment of the invention uses an iterative method to smooth the prediction through label propagation, the expression of which is:

[0085]

[0086] in, It is based on the initial noise label y joint The generated initial prediction matrix has a preset coefficient, a2 (0.9 in this embodiment), and D as the degree matrix. Iterative updates of the prediction... It converges to the final prediction.

[0087] In this embodiment of the invention, unreliable instances are removed based on classification uncertainty to obtain reliable pseudo-labels. Specifically, the entropy of each sample's prediction is calculated as the classification uncertainty. Instances with sharp prediction distributions are considered reliable, while instances with uniform predictions are considered unreliable. For each instance X... i We obtain its classification uncertainty as follows:

[0088]

[0089] Where Nc is the number of clusters. Let represent the classification score of the i-th instance, which is assigned to the k-th class. Then, we select the top r% of instances with the highest entropy as unreliable instances, where r decreases to zero with each iteration. After removing these unreliable instances, we base our classification on... Obtain the final pseudo-label

[0090] The embodiments of the present invention characterize the reliability of multi-view samples through intra-class compactness, and remove unreliable instances through classification uncertainty, thereby achieving adaptive fusion of complementary information from multiple views and overcoming the noise influence introduced by multiple views.

[0091] In this embodiment of the invention, a target retrieval model is obtained by training based on reliable pseudo-labels. Specifically, the model is trained using both reliable pseudo-labels and intra-modal and cross-modal matching objective functions between visual or text embeddings and ID prototypes. This embodiment of the invention, by introducing a dual contrastive loss, can learn a discriminative visual-text joint feature space, thereby improving the performance of the trained model.

[0092] In summary, the weakly supervised cross-modal target retrieval model training method based on pseudo-label optimization provided in this invention can effectively fuse sample relationships from multiple views by constructing a multi-view hypergraph and performing reliability-guided relation consistency inference based on the multi-view hypergraph. This enables the information from multiple views to be complementary, uncovers the consistency in multi-view sample relationships, reduces the mutual influence of noise in multi-view samples, and generates the final pseudo-labels for learning the visual-text joint feature embedding space. This improves the quality of pseudo-labels, performs well in handling noisy data and weakly supervised settings, and achieves better target retrieval performance.

[0093] The following describes the training device for a weakly supervised cross-modal target retrieval model based on pseudo-label optimization provided by the present invention. The training device for a weakly supervised cross-modal target retrieval model based on pseudo-label optimization described below can be referred to in correspondence with the training method for a weakly supervised cross-modal target retrieval model based on pseudo-label optimization described above.

[0094] Figure 3 This is a schematic diagram of the structure of the supervised cross-modal target retrieval model training device based on pseudo-label optimization provided by the present invention, as shown in the figure. Figure 3 As shown, it includes:

[0095] The feature extraction module 310 is used to extract visual embedding features from target domain sample images and to extract text embedding features from target domain text images.

[0096] The multi-view hypergraph module 320 is used to construct a multi-view hypergraph based on visual embedding features and text embedding features using a clustering method;

[0097] The reliable pseudo-label reasoning module 330 is used to perform reliability-guided relation consistency reasoning based on the multi-view hypergraph to obtain reliable pseudo-labels.

[0098] Training module 340 is used to train a target retrieval model based on reliable pseudo-labels.

[0099] As can be seen, the weakly supervised cross-modal target retrieval model training device based on pseudo-label optimization provided in this embodiment of the invention can construct a multi-view hypergraph and perform reliability-guided relation consistency inference based on the multi-view hypergraph. It can effectively fuse sample relations from multiple views, making the information of multiple views complementary, mining the consistency in multi-view sample relations, reducing the mutual influence of multi-view sample noise, generating the final pseudo-label for visual text joint feature embedding space learning, improving the quality of pseudo-labels, performing well in processing noisy data and weakly supervised settings, and obtaining better target retrieval performance.

[0100] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a method for supervising the training of a cross-modal target retrieval model based on a pseudo-label optimization weak target retrieval model, including the following steps:

[0101] Visual embedding features are extracted from target domain sample images; text embedding features are extracted from target domain text images.

[0102] A multi-view hypergraph is constructed based on visual and text embedding features using clustering methods.

[0103] Reliability-guided relational consistency inference is performed based on the multi-view hypergraph to obtain reliable pseudo-labels;

[0104] The target retrieval model is obtained by training based on reliable pseudo-labels.

[0105] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the training method for a cross-modal target retrieval model supervised by a weak target retrieval model based on pseudo-label optimization provided by the above methods, including the following steps:

[0107] Visual embedding features are extracted from target domain sample images; text embedding features are extracted from target domain text images.

[0108] A multi-view hypergraph is constructed based on visual and text embedding features using clustering methods.

[0109] Reliability-guided relational consistency inference is performed based on the multi-view hypergraph to obtain reliable pseudo-labels;

[0110] The target retrieval model is obtained by training based on reliable pseudo-labels.

[0111] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for supervising the training of a cross-modal target retrieval model based on a pseudo-label optimization weak target retrieval model provided by the above methods, including the following steps:

[0112] Visual embedding features are extracted from target domain sample images; text embedding features are extracted from target domain text images.

[0113] A multi-view hypergraph is constructed based on visual and text embedding features using clustering methods.

[0114] Reliability-guided relational consistency inference is performed based on the multi-view hypergraph to obtain reliable pseudo-labels;

[0115] The target retrieval model is obtained by training based on reliable pseudo-labels.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A supervised training method for a cross-modal target retrieval model based on a weak target retrieval model optimized by pseudo-labels, characterized in that, Includes the following steps: Visual embedding features are extracted from target domain sample images; text embedding features are extracted from target domain text images. A multi-view hypergraph is constructed based on visual and text embedding features using clustering methods. Reliability-guided relational consistency inference is performed based on the multi-view hypergraph to obtain reliable pseudo-labels; The target retrieval model is trained based on reliable pseudo-labels; the construction of a multi-view hypermap using clustering methods based on visual and text embedding features includes the following steps: Based on visual embedding features, a visual clustering relationship graph is obtained using clustering methods. Based on the text embedding features, a text clustering relationship graph is obtained using a clustering method; The fused features are obtained by fusing visual embedding features and text embedding features, and a joint visual-text clustering relationship graph is obtained by using the fused features and a clustering method. Based on the visual clustering relationship graph, the text clustering relationship graph, and the visual-text joint clustering relationship graph, a multi-view hypergraph is obtained; Based on the visual clustering relationship graph, the text clustering relationship graph, and the joint visual-text clustering relationship graph, a multi-view hypergraph is obtained, specifically as follows: Based on the visual clustering relationship graph, the text clustering relationship graph, and the visual-text joint clustering relationship graph, determine the hyperedges of the multi-view hypergraph and the association matrix of the multi-view hypergraph; The weight of the hyperedge is determined based on the node similarity corresponding to the hyperedge in the multi-view hypergraph. Based on the association matrix and the weights of the hyperedges in the view hypergraph, the adjacency matrix of the multi-view hypergraph is determined; the reliable pseudo-labels are obtained by performing reliability-guided relation consistency inference based on the multi-view hypergraph, including the following steps: Based on the visual clustering relationship diagram, the text clustering relationship diagram, and the visual-text joint clustering relationship diagram, determine the initial clusters and initial pseudo-labels; Based on the initial clustering and initial pseudo-labels, calculate the intra-cluster tightness and determine the candidate set with the highest intra-cluster tightness to obtain reliable instances and reliable pseudo-labels of the candidate set; Label propagation is performed based on reliable instances and reliable pseudo-labels in the candidate set to obtain pseudo-labels for all data instances; Unreliable instances are removed based on classification uncertainty to obtain reliable pseudo-labels.

2. The method for supervising cross-modal target retrieval model training based on pseudo-label optimization for weak target retrieval models according to claim 1, characterized in that, The target retrieval model is obtained by training based on reliable pseudo-labels. Specifically, the target retrieval model is obtained by training based on reliable pseudo-labels and training the intra-modal and cross-modal matching objective function between visual or text embeddings and ID prototypes.

3. A supervised cross-modal target retrieval model training device based on pseudo-label optimization weak target retrieval model, employing the supervised cross-modal target retrieval model training method based on pseudo-label optimization as described in claim 1, characterized in that... include: The feature extraction module is used to extract visual embedding features from sample images of the target domain; Extract text embedding features from the target domain text image; The multi-view hypergraph module is used to construct multi-view hypergraphs based on visual embedding features and text embedding features using clustering methods. The reliable pseudo-label reasoning module is used to perform reliability-guided relation consistency reasoning based on the multi-view hypergraph to obtain reliable pseudo-labels. The training module is used to train a target retrieval model based on reliable pseudo-labels.

4. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for supervising the training of a cross-modal target retrieval model based on a pseudo-label optimized weak target retrieval model as described in any one of claims 1 to 2.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for supervising the training of a cross-modal target retrieval model based on a weak target retrieval model optimized by pseudo-labels as described in any one of claims 1 to 2.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for supervising the training of a cross-modal target retrieval model based on a weak target retrieval model optimized by pseudo-labels as described in any one of claims 1 to 2.

Images