A human re-identification method and system based on uncertainty joint attention

Abstract

The application discloses a person re-identification method and system based on uncertainty joint attention, and is applied to the technical field of human body identification. The method comprises the following steps: S1, global features, spatial features and key point features of an input image are extracted; S2, joint learning is performed on the key point features and the spatial features to obtain joint features; and S3, a relationship feature of the joint features and context information is constructed to obtain context relationship features. The application designs a context relationship network, so that all local features of the same individual contain context relationship features, and the problem that when local features of different individuals have similar attributes, the local features between individuals are confused to cause misrecognition is solved; low-score pixel regions are discarded, and are combined with corresponding local spatial features, the reliability of the local features is enhanced, and more significant local features can be obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human recognition, and more particularly to a human re-identification method and system based on uncertainty joint attention. BACKGROUND

[0002] Human re-identification mainly refers to obtaining a target pedestrian from a pedestrian image data set captured by multiple cameras in different scenes, and is a popular and important research topic in the field of computer vision in recent years. It has great application prospects in the field of public safety, and is often used for finding missing persons or suspects, cross-camera tracking, multi-camera scene detection and the like. However, due to the influence of imaging devices, environmental noise and human body posture and the like, the human re-identification task often has the problem that some noticeable local features are suppressed, the same person can show a large 'intra-class' difference, and at the same time, due to the complexity and largeness of pedestrian data, different human bodies can show a small 'inter-class' difference. Thus, a person'mutates' into another person or two images are the same person but cannot be accurately identified, so for researchers, human re-identification is still a very challenging research problem. The existing human re-identification methods mostly do not consider the relationship between local features, which will increase the misidentification problem of people with similar local features.

[0003] In the prior art, many methods focus on extracting global features of human images by using CNN to obtain representation information, but they ignore the key information of specific parts of the body. Two people have a very small inter-class difference in global features, but the inter-class difference in head region features is very obvious, which undoubtedly hinders the expansion of the inter-class difference of the overall features. In order to obtain more significant local features, some methods extract local features from the body and combine them with global features to form more effective feature descriptors. However, this also raises new problems. Since the human body images collected by some camera detectors often have misplacement or occlusion, some regions of the image have no effective feature information, so the extracted local feature information cannot accurately represent the human body features, and even has a negative impact on the description of the overall feature information. Therefore, how to provide a human re-identification method capable of accurately and effectively extracting local features is a problem that those skilled in the art need to solve. SUMMARY

[0004] Therefore, the present application provides a human re-identification method and system based on uncertainty joint attention. Through posture activation and spatial guidance multi-attention branch, the model focuses on the effective and visible local area, and the two branches jointly work to obtain more effective and significant local features. By adding a context relationship network, a strong connection between local features and context relationship features is constructed to obtain distinctive local features and relationship features.

[0005] To achieve the above object, the present application provides the following technical solutions:

[0006] A person re-identification method based on uncertainty joint attention, comprising the following steps:

[0007] S1, extracting global features, spatial features and key point features of an input image;

[0008] S2, jointly learning the key point features and the spatial features to obtain joint features;

[0009] S3, constructing relationship features of the joint features and context information to obtain context relationship features.

[0010] Optionally, S2 is specifically:

[0011] S21, performing element-level multiplication operation on the global features and the key point features to obtain key point local semantics;

[0012] S22, mapping the key point feature heat map into an attention heat map to obtain an attention score;

[0013] S23, performing feature erasing according to the attention score to obtain posture local features;

[0014] S24, taking the global features as a center to add topological relationships to each spatial feature to obtain spatial local features;

[0015] S25, jointly representing the spatial local features and the posture local features to obtain joint features.

[0016] Optionally, the calculation process of S21 is specifically:

[0017]

[0018] In the formula, is the key point local semantics, k is the serial number of the kth key point region, is the local feature of the cth channel of the key point feature, g is the global maximum value pooling, M global is the global feature map, M kp is the key point feature map.

[0019] Optionally, S22 is specifically: according to the corresponding relationship between the key point coordinates and the original image, the key point feature heat map is mapped into an attention heat map by using a Gaussian method and normalization processing:

[0020]

[0021] In the formula, F scoreFor attention score, Norm is normalization, Gaussian(·) is Gaussian method, L i is the center position of the key point region, is the standard deviation

[0022] Optionally, S23 is specifically: comparing the attention scores of single pixel positions in the region pixel p(i,j), and counting the sum of attention scores in the influence range c i of p(i,j):

[0023]

[0024] In the formula, as ij is the region attention score with the region pixel p(i,j) as the center and the influence range c i with a radius, according to the attention score, the feature information in the pixel and the influence range c i with a score less than a preset threshold is erased.

[0025] Optionally, S24 is specifically: constructing a topological relationship feature based on global features

[0026]

[0027] In the formula, G 0 is the global feature after maximum pooling, η is the influence weight of the global feature on the topological relationship feature, and S i is a local feature based on spatial division.

[0028] The topological relationship feature is compensated, connected and G 0 , and 1×1 fusion convolution is performed to obtain a spatial local feature F sg :

[0029]

[0030] In the formula, C fase is 1×1 fusion convolution, and T represents the connection of two features.

[0031] Optionally, S25 is specifically:

[0032] A i =Joint{F p ,F sg}

[0033] In the formula, A i is a joint feature, F p is a pose local feature, and Joint is a joint representation.

[0034] Optionally, the context relationship feature in S3 includes a first-order context relationship feature and a second-order context relationship feature, wherein the calculation of the first-order context relationship feature is specifically as follows:

[0035] The first-order context feature is calculated as follows:

[0036]

[0037] In the formula, is the first-order context feature, γ is a closeness hyperparameter, and represents the closeness degree of the joint feature A i and the global feature G 0 .

[0038] The first-order context feature and the global feature G 0 after maximum pooling are compensated as follows:

[0039]

[0040]

[0041] In the formula, is the first-order context feature after compensation, G 1 is the global feature after compensation, R c is a compensation subnetwork, f R is a first-order context relationship layer, μ is a compensation degree hyperparameter, is the first-order context relationship feature.

[0042] Optionally, the calculation of the second-order context relationship feature in S3 is specifically as follows:

[0043] The context relationship of {A i , G 1} and is constructed as follows:

[0044]

[0045] R hr = F hr {F ag (T ag ), F cg (T cg )}

[0046]

[0047] In the formula, T ag is the connection between the joint feature and the global feature after compensation, T cg is the connection between the first-order context feature after compensation and the global feature after compensation, and Fag is a joint and global relation module, F cg is a context and global relation module, F hr is a joint feature and first-order context relation feature module, R hr is a high-order relation feature, F c is a fully connected layer, and δ is a compensation parameter, is a second-order context relation feature, and F is the final context relation feature.

[0048] A person re-identification system based on uncertainty joint attention, applying any one of the person re-identification methods based on uncertainty joint attention described above, comprising a feature extraction module, a joint attention module, and a relation module connected in sequence;

[0049] The feature extraction module is configured to extract global features, spatial features, and key point features of an input image.

[0050] The joint attention module is configured to jointly learn the key point features and the spatial features to obtain joint features.

[0051] The relation module is configured to construct relation features between the joint features and context information to obtain context relation features.

[0052] Compared with the prior art, the person re-identification method and system based on uncertainty joint attention provided by the present disclosure have the following beneficial effects: a context relation network is designed to focus on constructing the relation between local features and context information features, so that all local features of the same individual contain context relation features, and the problem of misrecognition caused by confusing local features between individuals when the local features of different individuals have similar attributes is solved; a joint attention module guided by uncertainty is proposed to focus on the effectiveness of single pixels and local spatial features, discard the pixel regions with low scores based on the attention scores of single pixels, and combine the discarded pixel regions with corresponding local spatial features to enhance the credibility of the local features and obtain more significant local features. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only embodiments of the present disclosure, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0054] Figure 1 is a method flowchart of the present disclosure;

[0055] Figure 2 The erasing schematic diagram is a feature of the present application, wherein 2.1 is a schematic diagram before erasing, and 2.2 is a schematic diagram after erasing;

[0056] Figure 3 The flowchart is a combined feature of the present application;

[0057] Figure 4 The flowchart is a relationship feature of the present application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0059] The embodiments of the present application disclose a person re-identification method based on uncertainty combined attention, as shown in the following steps: Figure 1

[0060] S1, extracting global features, spatial features and key point features of an input image;

[0061] S2, jointly learning the key point features and the spatial features to obtain combined features;

[0062] S3, constructing relationship features of the combined features and context information to obtain context relationship features.

[0063] Further, S2 specifically includes the following steps:

[0064] S21, performing element-level multiplication operation on the global features and the key point features to obtain key point local semantics;

[0065] S22, mapping the key point feature heat map into an attention heat map to obtain an attention score;

[0066] S23, performing feature erasing according to the attention score to obtain posture local features;

[0067] S24, taking the global features as a center, adding topological relationships to each spatial feature to obtain spatial local features;

[0068] S25, jointly representing the spatial local features and the posture local features to obtain combined features.

[0069] ​Specifically, in some pedestrian datasets, there is a problem of camera detection misalignment, that is, the image will have more irrelevant background or missing part of the body features. In order to weaken the influence caused by these adverse factors, more effective and certain feature information is extracted, and the key point feature is divided into two granularities, that is, local feature information F P 、F S :

[0070] F P ={f p1 ,f p2 ,…,f pn}∈R N×1

[0071] F S ={f s1 ,f s2 ,…,f sm}∈R M×1

[0072] In the formula, f pn represents the feature information of the nth part, f pm represents the feature information of the mth region, M and N respectively represent the number of local features activated by the posture and guided by the space, and I represents the number of images.

[0073] Further, the calculation process of S21 is specifically:

[0074]

[0075] In the formula, s is the local semantics of the key point, k is the serial number of the kth key point region, is the local feature of the cth channel of the key point feature, g is the global maximum value pooling, M global is the global feature map, M kp is the key point feature map.

[0076] Further, S22 is specifically: according to the corresponding relationship between the key point coordinates and the original image, the key point feature heat map is mapped into the attention heat map by using the Gaussian method and normalization processing:

[0077] F score =Norm(Gaussian(L i ,ζ))

[0078] In the formula, F score is the attention score, Norm is the normalization processing, Gaussian is the Gaussian method, the expectation L i of the Gaussian model is the center position of the key point region, and ζ is the standard deviation.

[0079] Specifically, the standard deviation ζ can be 16.

[0080] Further, S23 specifically compares the attention scores of individual pixel positions within the region pixel p(i,j) and counts the sum of the attention scores within the influence range c i of p(i,j):

[0081]

[0082] where as ij is the region attention score centered on the region pixel p(i,j) with an influence range c i of radius, according to the attention score, as Figure 2 indicated, the feature information of the pixels with scores less than a preset threshold and the feature information within the influence range c i of the pixels are erased, as shown in Figure 2 .1, and as shown in Figure 2 .2. Because there is an inherent necessary connection between the local features of the human body, the attention scores of the region pixels with a radius c i will show relative consistency. Thus, after obtaining the local features of the key points activated by the posture, the attention scores of individual pixel positions are compared.

[0083] Further, as shown in Figure 3 , S24 specifically constructs a topological relationship feature based on global features

[0084]

[0085] where G 0 is the global feature after maximum pooling, η is the influence weight of the global feature on the topological relationship feature, and S i is the local feature based on spatial division.

[0086] The topological relationship feature is compensated, connected and G 0 , and a 1×1 fusion convolution is performed to obtain the spatial local feature F sg :

[0087]

[0088] where C fase is a 1×1 fusion convolution, and T represents the connection of two features.

[0089] The feature erasure eliminates The representation of the minimum attention score low area feature information is low, but this will cause the problem of too little feature information and uncertain representation. Therefore, we introduce the joint attention mechanism, add the spatially guided local feature, and ensure the certainty of the joint attention mechanism in the way of increasing the granularity of feature extraction. Considering the problem of excessive dependence on key point positioning accuracy, the posture activated local feature is supplemented and reconstructed, and the spatially guided local feature is combined to ensure more certain and effective feature representation. Among them, the spatially guided way is different from the simple division based on the geometric way. We increase the topological relationship for each local feature based on the global feature as the center, and extract two granularity feature information including the original spatial local feature and the local feature with topological relationship.

[0090] Optionally, S25 is specifically:

[0091] A i =Joint{F p ,F sg}

[0092] In the formula, A i is the joint feature, F p is the posture local feature, Joint is the joint representation, specifically, Joint includes a 1x1 convolution, batch normalization and Relu operation, and finally the joint feature A i will be used as input to construct the context relationship feature.

[0093] Further, because of the complexity and diversity of the data set, the local features of different individual images may show small inter-class differences, and sometimes there may be misidentification problems. Based on this, it is necessary to construct a strong feature relationship network to strengthen the connection between local features of the same individual and weaken the similarity between local features of different individuals. The global feature and the adjacent feature are used to guide the construction of the relationship network, so that each part-level feature can contain the context affinity relationship, but not lose the original key feature information of each part. The detailed process is shown in Figure 4 The context relationship feature in S3 includes first-order context relationship feature and second-order context relationship feature, and the calculation of the first-order context relationship feature is specifically:

[0094] The first-order context feature is calculated as:

[0095]

[0096] In the formula, is the first-order context feature, γ is the affinity hyperparameter, representing the affinity degree of the joint feature A i and the global feature G 0 ;

[0097] The first-order context feature and the global feature after max-pooling into G 0 Row compensation:

[0098]

[0099]

[0100] wherein, is the first-order context feature after compensation, G 1 is the global feature after compensation, R c is the compensation sub-network, f R is the first-order context relationship layer, μ is the compensation degree hyperparameter, is the first-order context relationship feature.

[0101] Further, the calculation of the second-order context relationship feature in S3 is specifically:

[0102] The context relationship of {A i , G 1} and is constructed:

[0103]

[0104] R hr = F hr {F ag (T ag ), F cg (T cg )}

[0105]

[0106] wherein, T ag is the connection between the joint feature and the global feature after compensation, T cg is the connection between the first-order context feature after compensation and the global feature after compensation, F ag is the joint and global relationship module, F cg is the context and global relationship module, F hr is the joint feature and first-order context relationship feature module, F ag , f cg , F hr all contain multiple convolution, batch normalization and Relu operation, R hr is the high-order relationship feature, F c is the fully connected layer, δ is the compensation parameter, is the second-order context relationship feature, and is taken as the final context relationship feature.

[0107] Specifically, the value range of the compensation parameter can be 0-1.

[0108] Further, the re-identification system CUGJ disclosed by the present application is a multi-task network, and the real label of a human image is used for pedestrian representation learning, and the main loss function of CUGJ is as follows:

[0109]

[0110] In the formula, The loss function corresponding to each task is as follows: ε i represents the weight of the task loss, x and Q respectively represent the input training image and the weight thereof, W is the weight on the input x, T is the total number of loss tasks, but the losses used by different tasks are not the same.

[0111] Specifically, the present application uses a batch hard triplet loss and a cross-entropy loss, which are respectively represented by and The cross-entropy loss can be expressed as:

[0112]

[0113] N and y n respectively represent the number of images in a batch and the real label corresponding to the image, is the predicted label of the network to the feature.

[0114]

[0115] In the formula, L is the number of identification labels, is the classification result of the feature q i and the label L, and finally all will pass through a full connection layer. The batch hard triplet loss in the model is as follows:

[0116]

[0117] In the formula, N L is the number of labels in a batch, N M is the number of images corresponding to each label in a batch

[0118] (N=N L N M ). Alpha is a parameter for controlling the distance between positive and negative pairs in the feature space. respectively represent the reference image, the positive image and the negative image, and i and j correspond to the index of the label and the image. Among them, t is not equal to l.

[0119] Further, experiments are performed on the data sets Market-1501, DukeMTMC-reID and Cuhk-03, and the performance is compared with the existing most advanced human re-identification method to verify the performance of the method in the application. Specifically:

[0120] The Market1501 data set is collected by 6 outdoor cameras of different resolutions of Tsinghua University, including 5 high-resolution cameras and 1 low-resolution camera. The data set contains 32668 images, and 1501 different identities are extracted by a DPM detector. The data set has a total of 12936 images of 751 identities as a training set, 3368 query images, and 15913 images of 750 identities as a test set. The DukeMTMC-ReID data set is captured by 8 outdoor cameras of Duke University, including 36,411 images of 1404 identities. The training set contains 16522 images of 702 identities, and the other 17661 images containing 702 identities are used as a test data set. In this data set, the width and height of the image and the aspect ratio vary greatly, and the data set is collected by manual annotation. The CUHK-03 data set contains 14096 pictures of 1467 identities taken by 6 cameras of the Chinese University of Hong Kong, wherein the same identity person only appears in two disjoint camera views, with an average of 4.8 images per view, and the data set contains two versions of manual labeling and automatic detection.

[0121] In the embodiment, the mean average precision (mAP) and cumulative matching characteristic (CMC) are selected as evaluation indexes. The mAP measures the good and bad of the model on all classes, and the CMC represents the set of the kth hit probability of the search result. In order to make a more direct display, when comparing between models, only the CMC accuracy of the selected level is compared, that is, Rank-k.

[0122] For the Market1501 data set, the application is compared with 15 existing most advanced methods, including handcrafted methods and deep learning based methods. The handcrafted methods include Bow(+HS) and MFFM, and the deep learning methods include some strong competitive local feature representation based methods and attention mechanism based methods in recent years. The existing methods and experimental results are shown in Table 1:

[0123] Table 1

[0124]

[0125] It can be seen that the CUGJ method proposed in the application has strong competitiveness, the mAP of the model can reach 87.87%, and the Rank-1 can reach 95.10%. Compared with the latest strong competitive person re-identification method CDNet, the mAP is increased by nearly 1.87% points, and compared with the classic PCB+RPP method, the Rank-1 is increased by nearly 1.3%, and the mAP is increased by nearly 6.27%. Overall, the method can achieve good results under the existing evaluation standard, and the front Rank-1 / 5 / 10 precision has strong competitiveness.

[0126] The horizontal comparison is carried out on the DukeMTMC-ReID data set. Compared with the Market-1501 data set, the images in the DukeMTMC-ReID data set show more complex and variable scene views. In short, it is more challenging. The results are shown in Table 2:

[0127] Table 2

[0128]

[0129] It can be seen that on this data set, the Rank-1 / mAP of the CUGJ method proposed in the application reaches 90.39% / 79.56%, which greatly surpasses some advanced methods. Compared with the method MHN with a hybrid high-order attention mechanism, the rank-1 is increased by 1.29%, and the mAP is increased by nearly 2.36%, which is undoubtedly a great breakthrough. Compared with the comparative method, the application is dedicated to extracting joint attention, and further enhances the relationship between features, improves the sensitivity of the network to feature information and the integrity of the features. As can be seen from the table, the performance of the method greatly surpasses some advanced methods.

[0130] The Cuhk03 data set is different from the previous two data sets in that Cuhk03 specifically distinguishes between manual labeling and machine labeling, tests the manual labeling data and machine labeling data respectively, and compares the results with the methods in the past two years. The comparison results are shown in Table 3:

[0131] Table 3

[0132]

[0133]

[0134] It can be seen that the present application also has good effect on the Cuhk03 dataset, wherein the Rank-1 and mAP of the artificial annotation part reach 79.64% and 77.22% respectively, and the Rank-1 and mAP of the machine annotation part reach 74.84% and 72.37% respectively. Compared with the best MHN method, the Rank-1 is improved by about 2.44%, and the mAP is improved by about 4.82%. It is worth noting that the present application not only focuses on the relationship between features, but also extracts joint features to make up for the deficiency of the local feature of the posture excessively relying on the key point extraction accuracy, so as to improve the ability of the network to learn feature information.

[0135] With Figure 1 Corresponding to the method, the present application also provides a human re-identification system based on uncertainty joint attention, which is used for the implementation of any one of the human re-identification methods based on uncertainty joint attention.

[0136] The feature extraction module is used for extracting global features, spatial features and key point features of the input image.

[0137] The joint attention module is used for joint learning of the key point features and the spatial features to obtain joint features.

[0138] The relationship module is used for constructing relationship features of the joint features and the context information to obtain context relationship features.

[0139] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0140] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for human re-identification based on uncertainty joint attention, characterized in that, The method comprises the following steps: S1, extracting global features, spatial features and key point features of an input image; S2, jointly learning the key point features and the spatial features to obtain joint features; S3, constructing relationship features between the joint features and context information to obtain context relationship features; S2 specifically comprises: S21, performing element-level multiplication operation on the global features and the key point features to obtain key point local semantics; S22, mapping the key point feature heat map into an attention heat map to obtain an attention score; S23, performing feature erasing according to the attention score to obtain pose local features; S24, adding topological relationships to each spatial feature with the global features as the center to obtain spatial local features; S25, jointly representing the spatial local features and the pose local features to obtain joint features; S24 specifically is: constructing topological relationship features based on global features : In the formula, is the global feature after maximum pooling, is the influence weight of the global feature on the topological relationship feature, is the local feature based on spatial division; On topological relationship characteristics Compensation, connection And , 1x1 fusion convolution is performed to obtain spatial local features : wherein is a fused convolution of 1x1, denotes concatenation of two features; The context relationship features in S3 comprise first-order context relationship features and second-order context relationship features, wherein the calculation of the first-order context relationship features is specifically as follows: The first-order context features are calculated as follows: wherein, is a first-order context feature, is a proximity hyperparameter representing a joint feature with a global feature of proximity; for a first-order context feature and global features after max pooling compensation: In the formula, is a first-order context feature after compensation, is a global feature after compensation, is a compensation subnetwork, is a first-order context relationship layer, is a compensation degree hyperparameter, is a first-order context relationship feature; The calculation of the second-order context relationship features in S3 is specifically as follows: Construction and Contextual relationships: wherein, is the connection between the joint feature and the compensated global feature, is the connection between the compensated first-order context feature and the compensated global feature, is the joint and global relation module, is the context and global relation module, is the joint feature and the first-order context relation feature module, is the high-order relation feature, is the fully connected layer, is the compensation parameter, is the second-order context relation feature, and is the final context relation feature.

2. The human re-identification method based on uncertainty joint attention according to claim 1, characterized in that, The calculation process of S21 is specifically as follows: wherein, is a local semantic of a key point, is a sequence number of a key point region, is a local feature of a channel of a key point feature, is a global max pooling, is a global feature map, is a key point feature map.

3. The human re-identification method based on uncertainty joint attention according to claim 2, characterized in that, S22 is specifically as follows: according to the correspondence between the key point coordinates and the original image, the key point feature heat map is mapped into an attention heat map by using a Gaussian method and normalization processing: In the formula, is an attention score, is a normalization process, Gaussian (·) is a Gaussian method, is a center position of a key point region, is a standard deviation.

4. The human re-identification method based on uncertainty joint attention according to claim 3, characterized in that, S23 is specifically: comparing the attention scores of individual pixel positions within the region of pixels counting the sum of the attention scores within the influence range ​ In the formula, As the regional pixel Center, the influence range The radius of the regional attention score, according to the attention score, the feature information in the pixel and its influence range whose score is less than the preset threshold is erased. ​ 5. The human re-identification method based on uncertainty joint attention according to claim 1, characterized in that, S25 is specifically as follows: In the formula, is a joint feature, is a pose local feature, is a joint representation. 6.A human re-identification system based on uncertainty joint attention, characterized in that, The application of the human re-identification system method based on uncertainty joint attention according to any one of claims 1-5 comprises a feature extraction module, a joint attention module and a relationship module connected in sequence; The feature extraction module is used to extract global features, spatial features and key point features of an input image; The joint attention module is used to jointly learn the spatial features and the key point features to obtain joint features; The relationship module is used to construct relationship features between the joint features and context information to obtain context relationship features.

Images