Self-supervised target positioning method, system, device and medium based on joint graph segmentation

Abstract

The present invention discloses a self-supervised target positioning method, system, device and medium based on joint graph segmentation. The method includes: obtaining unlabeled images and obtaining image pairs using geometric transformation; constructing a self-supervised target positioning model; the self-supervised target positioning model includes a backbone network, a joint graph segmentation module, a convolutional network and a Softmax classifier; inputting the image pairs into the backbone network to extract semantic feature pairs; inputting the semantic feature pairs into the joint graph segmentation module to obtain mask pairs that highlight shared foreground targets; inputting the mask pairs and the semantic feature pairs into the convolutional network, training the convolutional network on the image pairs, and then performing target positioning through the Softmax classifier. The present invention uses a twin network as the backbone network and, by introducing a joint graph segmentation module, solves the problem of inconsistent activation areas under different geometric transformations of the same image and enhances the common concurrent areas of the object; at the same time, the model is trained in a self-supervised learning manner to improve the performance and accuracy of target positioning.

Description

Technical Field

[0001] The present invention belongs to the technical field of image target positioning, and in particular relates to a self-supervised target positioning method, system, device and medium based on joint graph segmentation. Background Art

[0002] In recent years, with breakthroughs in computer hardware and deep learning, the field of artificial intelligence has achieved rapid development in an increasing number of fields, leading a wave of technological innovation in all aspects of society. Computer vision, as the most extensive field of artificial intelligence research, encompasses many specific areas such as image classification, object localization, object detection, image segmentation, image generation, and optical character recognition (OCR). It has broad applications in areas such as face recognition, autonomous driving, and image retrieval. Object localization, a major branch of computer vision, not only determines the category of an object in an image but also marks its location within the image, identifying the object with a bounding box. However, since fully supervised object localization methods require extensive manual annotation, current research directions have gradually shifted to weakly supervised object localization, which requires less supervised information, and even self-supervised object localization, which requires no supervised information.

[0003] Self-supervised learning is a key technique in unsupervised tasks. By setting up different proxy tasks, the network itself learns meaningful image feature representations. In existing technologies, some studies attempt to learn information about different patches in an image by solving jigsaw puzzles. For example, Larsson et al. established a mapping from objects to colors based on image colorization, learning object features by predicting missing parts of the image. RotNet proposed a simple rotation transformation to learn object features, achieving significant results. Other studies, on the other hand, use contrastive learning to construct image pairs using different image enhancements. These studies leverage the network's invariance to various data enhancements to learn image features. For example, MoCo further improves classification performance by using a memory bank and relaxing large-batch training. However, its activation regions are inconsistent for different geometric transformations of the same image, resulting in poor performance stability and low accuracy. Summary of the Invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a self-supervised target positioning method, system, device and medium based on joint graph segmentation. The present invention introduces a joint graph segmentation module to generate a mask that highlights the shared foreground, and then combines it with the semantic features obtained by the backbone network to solve the problem of inconsistent activation areas under different geometric transformations of the same image, thereby improving the performance and accuracy of target positioning.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The first object of the present invention is to provide a self-supervised target localization method based on joint graph segmentation, comprising the following steps:

[0007] Get unlabeled images and use geometric transformation to obtain image pairs;

[0008] Constructing a self-supervised target localization model; the self-supervised target localization model includes a backbone network, a joint graph segmentation module, a convolutional network, and a Softmax classifier; the backbone network is a twin network with shared weights, including a first backbone network and a second backbone network; the parameters of the joint graph segmentation model are learnable;

[0009] Input the image pairs into the backbone network of the self-supervised object localization model to extract semantic feature pairs;

[0010] The semantic feature pairs are fed into the joint graph segmentation module of the self-supervised object localization model to obtain mask pairs that highlight the shared foreground objects.

[0011] The mask pair and semantic feature pair are input into the convolutional network together, the convolutional network is trained on the image pair, and then the target is located through the Softmax classifier.

[0012] As a preferred technical solution, the unlabeled image refers to an image without image-level labels and pixel-level mask labels; the geometric transformation includes scaling, rotation, flipping, and symmetry; the image pair obtained after the unlabeled image I is geometrically transformed is represented as (I1, I2); the image pair is a random combination of the unlabeled image and the image after the geometric transformation;

[0013] The backbone network is a pre-trained VGG16 classification network.

[0014] As a preferred technical solution, the extraction of semantic feature pairs is specifically as follows:

[0015] The image pair is input into the backbone network for semantic feature extraction to obtain the semantic feature pair Where w is the width of the semantic feature, h is the height of the semantic feature map, and c is the dimension of the semantic feature;

[0016] The first backbone network extracts semantic features from the image pair I1:

[0017] F1=Backbone1(I1)

[0018] in, Represents the semantic features of I1 in the image pair, and Backbone1 is the first backbone network;

[0019] The second backbone network extracts semantic features from I2 in the image pair:

[0020] F2=Backbone2(I2)

[0021] in, Represents the semantic features of I2 in the image pair, and Backbone2 is the second backbone network.

[0022] As a preferred technical solution, the mask pair for highlighting the shared foreground object is obtained as follows:

[0023] The semantic feature pairs are input into the joint graph segmentation module. Treat the semantic feature pair as a graph G(V,E), where V represents the nodes in the graph G and E represents the edges of the nodes in the graph G;

[0024] It is known that there are 2wh nodes in graph G Then the adjacency matrix of graph G is Degree matrix of graph G (i, j = 0 ~ 2wh);

[0025] Construct the initial loss function of the joint graph segmentation module:

[0026]

[0027] Where RatioCut is the spectral clustering function, k represents the foreground object O and background in the semantic feature V i ,…,V k represents a subset of nodes V in graph G;

[0028] Introducing subgraph indicators The calculation formula is:

[0029]

[0030] Since the degree matrix D is a diagonal matrix, combined with the Laplace matrix properties, the subgraph index is embedded in the initial loss function L m* , we get the target loss function of the joint graph segmentation module:

[0031] L m =h T (DA)h

[0032] Among them, h T is the transpose of the subgraph index;

[0033] The joint graph segmentation module is optimized using the target loss function to obtain mask pairs that highlight shared foreground objects.

[0034] As a preferred technical solution, the subgraph index is embedded into the initial loss function L by combining the Laplace matrix properties.m* , the process of obtaining the target loss function of the joint graph segmentation module is:

[0035] Bring the adjacency matrix into the initialization loss function and transform it:

[0036]

[0037] in, is the foreground target in the semantic feature, is the background in the semantic feature, A m,n Represents an element in the adjacency matrix of graph G;

[0038] According to the properties of the Lass matrix, the subgraph index is embedded in the initialization function to obtain the target loss function of the joint graph segmentation module:

[0039]

[0040] Among them, h T is the transpose of the subplot indices.

[0041] As a preferred technical solution, the convolutional network includes a convolutional layer and a CAAM layer;

[0042] The mask and semantic feature pairs The convolutional network is trained on the image pair (I1, I2) and the feature map pair is obtained by global average pooling in the last CAAM layer of the convolutional network. Use equivariant regularization to construct the loss function of the convolutional network:

[0043] L ER =||P1-P2||

[0044] Input the feature map pair into the Softmax classifier to obtain the category label pair, use the cross entropy classification function to construct the classification loss function, and use the gradient descent method for optimization;

[0045] The classification loss function is:

[0046] L cls =CrossEntroy(N(T i (I)),y i )

[0047] Among them, N() is the parameter of Softmax classifier, T i (I) is an unlabeled image or an image after geometric transformation, y i is the category label corresponding to one of the images in the image pair;

[0048] The total loss function of the self-supervised target positioning model is obtained as:

[0049] L total =L cls +L ER +L n

[0050] The second object of the present invention is to provide a self-supervised target localization system based on joint graph segmentation, the system comprising a data acquisition module, a model construction module, a feature extraction module, a mask generation module and a target localization module;

[0051] The data acquisition module is used to acquire unlabeled images and obtain image pairs using geometric transformation;

[0052] The model construction module is used to construct a self-supervised target positioning model; the self-supervised target positioning model includes a backbone network, a joint graph segmentation module, a convolutional network and a Softmax classifier; the backbone network is a twin network with shared weights, including a first backbone network and a second backbone network; the parameters of the joint graph segmentation model are learnable;

[0053] The feature extraction module is used to input the image pair into the backbone network of the self-supervised target localization model to extract the semantic feature pair;

[0054] The mask generation module is used to input the semantic feature pairs into the joint graph segmentation module of the self-supervised object localization model to obtain mask pairs that highlight the shared foreground objects;

[0055] The target localization module is used to input the mask pair and the semantic feature pair into the convolutional network, train the convolutional network on the image pair, and then perform target localization through the Softmax classifier.

[0056] A third object of the present invention is to provide an electronic device, characterized in that the electronic device comprises:

[0057] at least one processor; and,

[0058] a memory communicatively connected to the at least one processor; wherein,

[0059] The memory stores computer program instructions that can be executed by the at least one processor, the computer program

[0060] The instructions are executed by the at least one processor to enable the at least one processor to perform the self-supervised target localization method based on joint graph segmentation as described above.

[0061] The fourth object of the present invention is to provide a computer-readable storage medium storing a program, characterized in that when the program is executed by a processor, the above-mentioned self-supervised target positioning method based on joint graph segmentation is implemented.

[0062] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0063] The self-supervised target positioning method based on joint graph segmentation proposed in this invention uses the twin classification network as the backbone network, encodes image pairs with different transformations, and introduces a joint graph segmentation module to solve the problem of inconsistent activation areas under different geometric transformations of the same image, enhances the common concurrent areas of the target object, and trains the network model in a self-supervised learning manner to achieve target positioning, effectively improving the positioning performance and providing a good solution for practical application scenarios such as autonomous driving and security identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0065] Figure 1 4 is a flow chart of a self-supervised target localization method based on joint graph segmentation according to an embodiment of the present invention;

[0066] Figure 2 A schematic diagram of activation areas of a backbone network of an existing method according to an embodiment of the present invention for different geometric transformations of the same image;

[0067] Figure 3 4 is a block diagram of a self-supervised target positioning system based on joint graph segmentation in an embodiment of the present invention;

[0068] Figure 4 2 is a structural diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0070] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0071] See also Figure 1 In one embodiment of the present application, a self-supervised target localization method based on joint graph segmentation is provided, comprising the following steps:

[0072] S1. Obtain unlabeled images and use geometric transformation to obtain image pairs;

[0073] The unlabeled image I in this embodiment refers to an image without image-level labels and pixel-level mask labels; the image is geometrically transformed by scaling, rotation, flipping, symmetry, etc. to obtain a transformed image, and then the unlabeled image and the transformed image are combined to obtain an image pair (I1, I2), where I1 is the unlabeled image and I2 is the image after the unlabeled image has been geometrically transformed.

[0074] S2. Construct a self-supervised target localization model, including a backbone network, a joint graph segmentation module, a convolutional network, and a Softmax classifier. The backbone network is a twin network with shared weights, and any existing pre-trained classification network can be used, including the first backbone network and the second backbone network. The parameters of the joint graph segmentation model are learnable.

[0075] The backbone network in this embodiment is a trained VGG16 classification network.

[0076] S3. Input the image pair into the backbone network of the self-supervised target localization model to extract semantic feature pairs, specifically:

[0077] The image pair is input into the backbone network for semantic feature extraction to obtain the semantic feature pair Where w is the width of the semantic feature, h is the height of the semantic feature map, and c is the dimension of the semantic feature;

[0078] The first backbone network extracts semantic features from I1 in the image pair:

[0079] F1=Backbone1(I1)

[0080] in, Represents the semantic features of I1 in the image pair, and Backbone1 is the first backbone network;

[0081] The second backbone network extracts semantic features from I2 in the image pair:

[0082] F2=Backbone2(I2)

[0083] in, Represents the semantic features of I2 in the image pair, and Backbone2 is the second backbone network.

[0084] S4. Input the semantic feature pair into the joint graph segmentation module of the self-supervised object localization model to obtain a mask pair that highlights the shared foreground object, specifically:

[0085] The semantic feature pairs are input into the joint graph segmentation module. Treat the semantic feature pair as a graph G(V,E), where V represents the nodes in the graph G and E represents the edges of the nodes in the graph G;

[0086] It is known that there are 2wh nodes in graph G Then the adjacency matrix of graph G is Degree matrix of graph G (i, j = 0 ~ 2wh);

[0087] In order to increase the distance between different subgraphs and reduce the distance between the same subgraphs, the initial loss function of the joint graph segmentation module is constructed:

[0088]

[0089] Where RatioCut is the spectral clustering function, k represents the foreground object O and background in the semantic feature V i ,…,V k represents a subset of nodes V in the graph G; in this implementation, k=2.

[0090] In order to solve the optimal graph cut problem and avoid the occurrence of a single sample node as a subset, the subgraph indicator is introduced The calculation formula is:

[0091]

[0092] Since the degree matrix D is a diagonal matrix, combined with the properties of the Laplace matrix, the subgraph index is embedded in the initial loss function L m* , we get the target loss function of the joint graph segmentation module:

[0093] L m =h T (DA)h

[0094] Among them, h T is the transpose of the subgraph index;

[0095] The joint graph segmentation module is optimized using the target loss function to obtain mask pairs that highlight shared foreground objects.

[0096] In this embodiment, the process of using the Laplace matrix property to embed the subgraph index to obtain the target loss function of the joint graph segmentation module is as follows:

[0097] Bring the adjacency matrix into the initialization loss function and transform it:

[0098]

[0099] in, is the foreground target in the semantic feature, is the background in the semantic feature, A m,n Represents an element in the adjacency matrix of graph G;

[0100] According to the properties of the Lass matrix, the subgraph index is embedded in the initialization function to obtain the target loss function of the joint graph segmentation module:

[0101]

[0102] S5. Input the mask pair and semantic feature pair into the convolutional network together, train the convolutional network on the image pair, and then use the Softmax classifier to locate the target.

[0103] like Figure 1 As shown in , the convolutional network includes convolutional layers and CAAM layers; Figure 2 As shown in the figure, existing methods have inconsistent activation areas for different geometric transformations of the same image. This paper takes this as a starting point and introduces a joint graph segmentation module to highlight the mask of shared foreground objects. Combined with the semantic features obtained by the backbone network, the self-supervised object localization model ensures that the activation area remains the same for different geometric transformations of the same image. Specifically:

[0104] The mask and semantic feature pairs The convolutional network is trained on the image pair (I1, I2) and the feature map pair is obtained by global average pooling in the last layer of the convolutional network. And use equivariant regularization to construct the loss function of the convolutional network to ensure that the activation areas of the image pairs are consistent:

[0105] L ER =||P1-P2||

[0106] Input the feature map pair into the Softmax classifier to obtain the category label pair, use the cross entropy classification function to construct the classification loss function, and use the gradient descent method for optimization;

[0107] The classification loss function is:

[0108] L cls =CrossEntroy(N(T i (I)),y i )

[0109] Among them, N() is the parameter of Softmax classifier, T i (I) is an unlabeled image or an image after geometric transformation, y i is the category label corresponding to one of the images in the image pair;

[0110] The total loss function of the self-supervised target positioning model is obtained as:

[0111] L total =L cls +L ER +L m

[0112] To demonstrate the effectiveness and advancement of our method, we compared it to the existing PSY method on the CUB-200-2011 dataset, achieving a GT-Known Loc performance of 88.83%, a 5.3% improvement over the PSY method. On the Stanford Car and Airplane datasets, our method achieved significant performance, reaching 97.73% and 96.72%, respectively. When our method employed the more powerful InceptionV3 network as its backbone, it further improved GT-Known Loc performance on the Stanford Car and Airplane datasets, reaching 98.62% and 97.94%, respectively. Notably, our proposed self-supervised object localization method based on joint graph segmentation outperformed the state-of-the-art weakly supervised method, TS-CAM, on the CUB-200-2011 dataset, achieving 88.83% compared to TS-CAM's 87.70%. Therefore, our proposed self-supervised object localization method based on joint graph segmentation can effectively improve both the performance and accuracy of object localization, demonstrating significant progress.

[0113] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0114] Based on the same concept as the self-supervised target localization method based on joint graph segmentation in the above-mentioned embodiment, the present invention also provides a self-supervised target localization system based on joint graph segmentation, which can be used to execute the above-mentioned self-supervised target localization method based on joint graph segmentation. For ease of explanation, the structural diagram of the embodiment of the self-supervised target localization system based on joint graph segmentation only shows the parts related to the embodiment of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation of the device, and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0115] See also Figure 3 , in another embodiment of the present application, a self-supervised target localization system based on joint graph segmentation is provided, the system comprising a data acquisition module, a model construction module, a feature extraction module, a mask generation module and a target localization module;

[0116] The data acquisition module is used to obtain unlabeled images and obtain image pairs using geometric transformation;

[0117] The model building module is used to build a self-supervised target localization model, including a backbone network, a joint graph segmentation module, a convolutional network, and a Softmax classifier. The backbone network is a twin network with shared weights, including a first backbone network and a second backbone network. The parameters of the joint graph segmentation model are learnable.

[0118] The feature extraction module is used to input the image pairs into the backbone network of the self-supervised object localization model to extract semantic feature pairs;

[0119] The mask generation module is used to input semantic feature pairs into the joint graph segmentation module of the self-supervised object localization model to obtain mask pairs that highlight the shared foreground objects;

[0120] The target localization module is used to input mask pairs and semantic feature pairs into the convolutional network, train the convolutional network on image pairs, and then perform target localization through the Softmax classifier.

[0121] It should be noted that the self-supervised target positioning system based on joint graph segmentation of the present invention corresponds one-to-one to the self-supervised target positioning method based on joint graph segmentation of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the self-supervised target positioning method based on joint graph segmentation are applicable to the embodiment of the self-supervised target positioning system based on joint graph segmentation. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.

[0122] In addition, in the implementation of the self-supervised target positioning system based on joint graph segmentation in the above embodiment, the logical division of each program module is only an example. In actual application, the above functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the self-supervised target positioning system based on joint graph segmentation is divided into different program modules to complete all or part of the functions described above.

[0123] See also Figure 4 In one embodiment, an electronic device for implementing a self-supervised target positioning method based on joint graph segmentation is provided. The electronic device may include a first processor, a first memory and a bus, and may also include a computer program stored in the first memory and executable on the first processor, such as a self-supervised target positioning program.

[0124] The first memory includes at least one type of readable storage medium, including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the first memory may be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the first memory may also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Furthermore, the first memory may include both an internal storage unit of the electronic device and an external storage device. The first memory can be used not only to store application software and various types of data installed in the electronic device, such as the code of the self-supervised target positioning program, but also to temporarily store data that has been output or is to be output.

[0125] In some embodiments, the first processor may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The first processor is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing programs or modules stored in the first memory (such as a self-supervised target positioning program, etc.), as well as calling data stored in the first memory, to perform various functions of the electronic device and process data.

[0126] Figure 4 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 4 The structure shown does not limit the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0127] The self-supervisory target positioning program stored in the first memory of the electronic device is a combination of multiple instructions, which, when executed in the first processor, can achieve:

[0128] Get unlabeled images and use geometric transformation to obtain image pairs;

[0129] Construct a self-supervised target localization model, including a backbone network, a joint graph segmentation module, a convolutional network, and a Softmax classifier. The backbone network is a twin network with shared weights, consisting of a first backbone network and a second backbone network. The parameters of the joint graph segmentation model are learnable.

[0130] Input the image pairs into the backbone network of the self-supervised object localization model to extract semantic feature pairs;

[0131] The semantic feature pairs are fed into the joint graph segmentation module of the self-supervised object localization model to obtain mask pairs that highlight the shared foreground objects.

[0132] The mask pair and semantic feature pair are input into the convolutional network together, the convolutional network is trained on the image pair, and then the target is located through the Softmax classifier.

[0133] Furthermore, if the module / unit integrated into the electronic device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0134] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-described methods. In particular, any reference to memory, storage, database, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory.

[0135] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A self-supervised target localization method based on joint graph segmentation, characterized by: The steps include: Obtain an unlabeled image and use geometric transformation to obtain an image pair; the unlabeled image refers to the obtained image without image-level labels and pixel-level mask labels; the geometric transformation includes scaling, rotation, flipping, and symmetry; the image pair obtained after the unlabeled image I is geometrically transformed is represented as (I1, I2); the image pair is a random combination of the unlabeled image and the image after geometric transformation; Construct a self-supervised target localization model; the self-supervised target localization model includes a backbone network, a joint graph segmentation module, a convolutional network, and a Softmax classifier; the backbone network is a twin network with shared weights, including a first backbone network and a second backbone network; the parameters of the joint graph segmentation module are learnable; the backbone network is a pre-trained VGG16 classification network; Input the image pairs into the backbone network of the self-supervised object localization model to extract semantic feature pairs; The extraction of semantic feature pairs is specifically as follows: The image pair is input into the backbone network for semantic feature extraction to obtain the semantic feature pair Where w is the width of the semantic feature, h is the height of the semantic feature map, and c is the dimension of the semantic feature; The first backbone network extracts semantic features from the image pair I1: F1=Backbone1(I1) in, Represents the semantic features of I1 in the image pair, and Backbone1 is the first backbone network; The second backbone network extracts semantic features from I2 in the image pair: F2=Backbone2(I2) in, Represents the semantic features of I2 in the image pair, and Backbone2 is the second backbone network; The semantic feature pairs are fed into the joint graph segmentation module of the self-supervised object localization model to obtain mask pairs that highlight the shared foreground objects. The mask pair obtained to highlight the shared foreground target is specifically: The semantic feature pairs are input into the joint graph segmentation module. Treat the semantic feature pair as a graph G(V,E), where V represents the nodes in the graph G and E represents the edges of the nodes in the graph G; It is known that there are 2wh nodes in graph G Then the adjacency matrix of graph G is Degree matrix of graph G Construct the initial loss function of the joint graph segmentation module: Where RatioCut is the spectral clustering function, k represents the foreground object O and background in the semantic feature V i ,…,V k represents a subset of nodes V in graph G; Introducing subgraph indicators The calculation formula is: Since the degree matrix D is a diagonal matrix, combined with the Laplace matrix properties, the subgraph index is embedded in the initial loss function L m* , we get the target loss function of the joint graph segmentation module: L m =h T (D-A)h Among them, h T is the transpose of the subgraph index; The joint graph segmentation module is optimized using the target loss function to obtain mask pairs that highlight shared foreground objects. The mask pair and semantic feature pair are input into the convolutional network together, the convolutional network is trained on the image pair, and then the target is located through the Softmax classifier.

2. The self-supervised target localization method based on joint graph segmentation according to claim 1, characterized in that The subgraph index is embedded into the initial loss function L by combining the Laplace matrix properties. m* , the process of obtaining the target loss function of the joint graph segmentation module is: Bring the adjacency matrix into the initialization loss function and transform it: in, is the foreground target in the semantic feature, is the background in the semantic feature, A m,n Represents an element in the adjacency matrix of graph G; According to the properties of the Lass matrix, the subgraph indicators are embedded in the initialization function to obtain the target loss function of the joint graph segmentation module.

3. The self-supervised target localization method based on joint graph segmentation according to claim 2, characterized in that The convolutional network includes a convolutional layer and a CAAM layer; The mask and semantic feature pairs The convolutional network is trained on the image pair (I1, I2) and the feature map pair is obtained by global average pooling in the last CAAM layer of the convolutional network. Use equivariant regularization to construct the loss function of the convolutional network: L ER =||P1-P2|| Input the feature map pair into the Softmax classifier to obtain the category label pair, use the cross entropy classification function to construct the classification loss function, and use the gradient descent method for optimization; The classification loss function is: L cls =CrossEntroy(N(T i (I)),y i ) Among them, N() is the parameter of Softmax classifier, T i (I) is an unlabeled image or an image after geometric transformation, y i is the category label corresponding to one of the images in the image pair; The total loss function of the self-supervised target positioning model is obtained as: L total =L cls +L ER +L m 。 4. A self-supervised target localization system based on joint graph segmentation, characterized by: The system includes a data acquisition module, a model building module, a feature extraction module, a mask generation module and a target positioning module; The data acquisition module is used to acquire unlabeled images and obtain image pairs using geometric transformation; the unlabeled images refer to images without image-level labels and pixel-level mask labels; the geometric transformation includes scaling, rotation, flipping, and symmetry; the image pair obtained after the unlabeled image I is geometrically transformed is represented as (I1, I2); the image pair is a random combination of the unlabeled image and the image after the geometric transformation; The model construction module is used to construct a self-supervised target positioning model; the self-supervised target positioning model includes a backbone network, a joint graph segmentation module, a convolutional network and a Softmax classifier; the backbone network is a twin network with shared weights, including a first backbone network and a second backbone network; the parameters of the joint graph segmentation module are learnable; the backbone network is a pre-trained VGG16 classification network; The feature extraction module is used to input the image pair into the backbone network of the self-supervised target localization model to extract semantic feature pairs; the extraction of semantic feature pairs is specifically: The image pair is input into the backbone network for semantic feature extraction to obtain the semantic feature pair Where w is the width of the semantic feature, h is the height of the semantic feature map, and c is the dimension of the semantic feature; The first backbone network extracts semantic features from the image pair I1: F1=Backbone1(I1) in, Represents the semantic features of I1 in the image pair, and Backbone1 is the first backbone network; The second backbone network extracts semantic features from I2 in the image pair: F2=Backbone2(I2) in, Represents the semantic features of I2 in the image pair, and Backbone2 is the second backbone network; The mask generation module is used to input the semantic feature pair into the joint graph segmentation module of the self-supervised object localization model to obtain a mask pair that highlights the shared foreground object; the mask pair that highlights the shared foreground object is specifically: The semantic feature pairs are input into the joint graph segmentation module. Treat the semantic feature pair as a graph G(V,E), where V represents the nodes in the graph G and E represents the edges of the nodes in the graph G; It is known that there are 2wh nodes in graph G Then the adjacency matrix of graph G is Degree matrix of graph G Construct the initial loss function of the joint graph segmentation module: Where RatioCut is the spectral clustering function, k represents the foreground object O and background in the semantic feature V i ,…,V k represents a subset of nodes V in graph G; Introducing subgraph indicators The calculation formula is: Since the degree matrix D is a diagonal matrix, combined with the Laplace matrix properties, the subgraph index is embedded in the initial loss function L m* , we get the target loss function of the joint graph segmentation module: L m =h T (D-A)h Among them, h T is the transpose of the subgraph index; The joint graph segmentation module is optimized using the target loss function to obtain mask pairs that highlight shared foreground objects. The target positioning module is used to input the mask pair and the semantic feature pair into the convolutional network, train the convolutional network on the image pair, and then perform target positioning through the Softmax classifier.

5. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, the computer program The instruction is executed by the at least one processor so that the at least one processor can execute the self-supervised target localization method based on joint graph segmentation as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the self-supervised target positioning method based on joint graph segmentation according to any one of claims 1 to 3 is implemented.

Images