Hierarchical supervised person re-identification method and system combining global and local information

Abstract

The present invention provides a method and system for hierarchical supervision of pedestrian re-identification that combines global and local information, including: cutting the input image into multiple small image blocks as the input of the branch network; introducing the network block feature information of some branch networks into the corresponding residual convolution blocks in the ResNet50 branch network after passing through the convolution layer, and the convolution layer expands the 2D tensor into a 4D tensor; channel cascading the 4D tensor feature information and the corresponding residual convolution block; passing the features of the network branches converted by the convolution layer and the corresponding ResNet50 features through a bilinear pooling layer and feature fusion interaction, and constraining them through a loss function. The method and system proposed by the present invention perform a unidirectional aggregation operation to promote CNN to learn global feature information, while combining bilinear pooling to perform hierarchical supervision, and have a significant performance improvement on pedestrian re-identification datasets spanning long time periods.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a hierarchical supervised pedestrian re-identification method and system combining global and local information. Background Art

[0002] Person re-identification is a typical task in computer vision, which aims to associate the same person under different camera perspectives. With the widespread deployment of surveillance systems in public places in recent years, person re-identification technology has received increasing attention in recent years. Person re-identification relies on the powerful feature learning ability of CNN to obtain many advanced and significant performances. However, in actual scenarios, pedestrians' clothes will change, which is a problem ignored by current pedestrian re-identification algorithms. Therefore, the existing pedestrian re-identification algorithms target short-term scenarios where pedestrians' appearance and clothing do not change much.

[0003] Person re-identification is a challenging task due to factors such as posture changes, viewpoint changes, illumination intensity changes, background interference, etc. For scenes spanning long time periods, appearance changes can be regarded as one of the main factors affecting the performance of person re-identification systems.

[0004] At present, pedestrian re-identification is still mainly aimed at studying short-term scenarios, ignoring scenarios spanning long time periods. However, in real applications, pedestrian re-identification algorithms spanning long time periods are a major method. For scenarios spanning long time periods, not only the local details of pedestrians should be considered, but also the global information in the image, and the two types of information need to complement each other. Existing pedestrian re-identification (re-ID) work mainly focuses on short-term situations, where pedestrians' clothing will basically not change much. A robust pedestrian re-identification system should take into account these changes in pedestrian clothing. Based on the powerful representation ability of deep convolutional neural networks (CNNs), short-term pedestrian re-identification has achieved many state-of-the-art performances (Top-1 and mAP). However, due to the lack of global feature information, existing short-term pedestrian re-identification methods cannot be extended to pedestrian re-identification scenarios spanning long time periods (long-term). Summary of the invention

[0005] In order to solve the technical problems that the pedestrian re-identification (re-ID) work in the prior art is mainly concentrated in short-term situations, the pedestrian's clothing will basically not change much, and due to the lack of global feature information, the existing short-term pedestrian re-identification method cannot be extended to the pedestrian re-identification scenario across a long period of time (long-term), the present invention proposes a hierarchical supervised pedestrian re-identification method and system that combines global and local information to solve the above technical problems.

[0006] According to a first aspect of the present invention, a hierarchical supervised person re-identification method combining global and local information is proposed, comprising:

[0007] S1: Cut the input image into multiple small image patches and use the small image patches as the input of the Transformers branch network;

[0008] S2: The feature information of some Transformers branch networks is introduced into the corresponding residual convolution blocks in the ResNet50 branch network through the expand layer. The expand layer expands the 2D tensor into a 4D tensor.

[0009] S3: Channel concatenate the expanded 4D tensor feature information and the corresponding residual convolution block;

[0010] S4: The features converted by the expand layer of the Transformers branch and the corresponding ResNet50 features are passed through a bilinear pooling layer for feature fusion interaction and constrained by a loss function.

[0011] In some specific embodiments, the Transformers branch network includes 12 network blocks of the same structure, the network blocks are numbered 0-11, and the input image is cut into N image blocks. Among them, the values ​​of P and S are both 16. Represents the floor function, H and W represent the height and width. Transformers can be used to capture global feature dependencies in an image.

[0012] In some specific embodiments, feature information of network blocks numbered 1, 3, 5, and 8 are respectively introduced into corresponding residual convolution blocks in ResNet50.

[0013] In some specific embodiments, S3 is specifically: channel cascading the expanded 4D tensor feature information and the corresponding residual convolution block, the number of channels is increased to twice the original number, in order to enable the features to be normally fed into the next residual convolution block, the channel is restored by 1*1 convolution, and then passed through the BN layer and the ReLU activation function, and its expression is as follows: Among them, R represents the ReLU activation function, B represents the BN layer, and C 1*1 represents a 1*1 convolutional layer, Represents the features of the Transformers branch after transformation through the expand layer, Represents the features of ResNet50, i corresponds to the sequence number of the branch network, when i=0, the corresponding Transformers branch sequence number is 1, which is conv1 of ResNet50.

[0014] In some specific embodiments, the specific expression of S4 is as follows:

[0015] Among them, (x, y) represents the value of the feature at the position (x, y), and the vec operation converts the matrix into a vector. The outer product of two vectors, S represents a constant.

[0016] In some specific embodiments, the loss function includes a triplet loss function and a label smoothing regularized cross entropy loss function, wherein the formula of the triplet loss function is: Where P represents the number of different pedestrian identities, K0 represents the selection of K0 different images for each pedestrian, and for each training image x a , in and x a Select an image x with the largest similarity distance among the images of the same identity p As a positive sample, in and x a Select an image x with the smallest similarity distance among images of different identities n As negative samples, is the threshold set, d(·,·) is defined as the Euclidean distance, batch represents the parameter size of batch processing; the formula of the cross entropy loss function of label smoothing regularization is Where ε∈[0,1], K is the number of pedestrian categories, and p(k) represents the probability of predicting belonging to category k.

[0017] According to a second aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, any of the above methods is implemented.

[0018] According to a third aspect of the present invention, a hierarchical supervised person re-identification system combining global and local information is proposed, the system comprising:

[0019] Image cutting module: configured to cut the input image into multiple image patches and use the image patches as input to the Transformers branch network;

[0020] Tensor expansion module: It is configured to introduce the feature information of some Transformers branch networks into the corresponding residual convolution blocks in the ResNet50 branch network through the expand layer. The expand layer expands the 2D tensor into a 4D tensor.

[0021] Channel cascade module: configured to perform channel cascade on the expanded 4D tensor feature information and the corresponding residual convolution block;

[0022] Feature fusion module: It is configured to perform feature fusion interaction on the features converted by the Transformers branch through the expand layer and the corresponding ResNet50 features through the bilinear pooling layer, and constrain them through the loss function.

[0023] In some specific embodiments, the Transformers branch network includes 12 network blocks of the same structure, the network blocks are numbered 0-11, and the input image is cut into N image blocks. Among them, the values ​​of P and S are both 16. stands for floor function, H and W represent height and width.

[0024] In some specific embodiments, feature information of network blocks numbered 1, 3, 5, and 8 are respectively introduced into corresponding residual convolution blocks in ResNet50.

[0025] In some specific embodiments, the expanded 4D tensor feature information and the corresponding residual convolution block are channel-concatenated, and the number of channels is increased to twice the original number. In order to allow the features to be normally fed into the next residual convolution block, the channel is restored by 1*1 convolution, and then passes through the BN layer and the ReLU activation function. The expression is as follows: Among them, R represents the ReLU activation function, B represents the BN layer, and C 1*1 represents a 1*1 convolutional layer, Represents the features of the Transformers branch after transformation through the expand layer, Represents the features of ResNet50, i corresponds to the sequence number of the branch network, when i=0, the corresponding Transformers branch sequence number is 1, which is conv1 of ResNet50.

[0026] In some specific embodiments, Among them, (x, y) represents the value of the feature at the position (x, y), and the vec operation converts the matrix into a vector. The outer product of two vectors, S represents a constant.

[0027] In some specific embodiments, the loss function includes a triplet loss function and a label smoothing regularized cross entropy loss function, wherein the formula of the triplet loss function is: Where P represents the number of different pedestrian identities, K0 represents the selection of K0 different images for each pedestrian, and for each training image x a , in and x a Select an image x with the largest similarity distance among the images of the same identity pAs a positive sample, in and x a Select an image x with the smallest similarity distance among images of different identities n As negative samples, is the threshold set, d(·,·) is defined as the Euclidean distance, batch represents the parameter size of batch processing; the formula of the cross entropy loss function of label smoothing regularization is Where ε∈[0,1], K is the number of pedestrian categories, and p(k) represents the probability of predicting belonging to category k.

[0028] The present invention proposes a hierarchical supervised person re-identification method and system that combines global and local information, fully utilizes the advantages of CNN and Transformers, and proposes a parallel CNN-Transformers Network (CT-Net) algorithm to learn global information and local salient information. The CT-Net proposed in the present invention performs a unidirectional aggregation operation to promote CNN to learn global feature information. At the same time, in order to make full use of global feature information and local information, CT-Net combines bilinear pooling to perform hierarchical supervision. In order to prove the effectiveness of the algorithm of the present invention, we conducted experiments on a pedestrian re-identification dataset spanning a long period of time. The experimental performance has been significantly improved and can reach the most advanced performance at present. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and together with the description are used to explain the principles of the present invention. Other embodiments and many expected advantages of the embodiments will be readily appreciated as they become better understood by reference to the following detailed description. Other features, objects and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings:

[0030] Figure 1 is a flowchart of a hierarchical supervised person re-identification method combining global and local information according to an embodiment of the present application;

[0031] Figure 2 This is a framework diagram of a hierarchical supervised person re-identification algorithm combining global and local information in a specific embodiment of the present application;

[0032] Figure 3 is a framework diagram of a hierarchical supervised person re-identification system combining global and local information according to an embodiment of the present application;

[0033] Figure 4 It is a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0034] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0035] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0036] According to an embodiment of the present application, a hierarchical supervised person re-identification method combining global and local information is provided. Figure 1 FIG. 4 is a flowchart of a hierarchical supervised person re-identification method combining global and local information according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0037] S101: Cut the input image into multiple small image blocks, and use the small image blocks as the input of the Transformers branch network. The Transformers branch network includes 12 network blocks with the same structure. The sequence numbers of the network blocks are 0-11. The input image is cut into N small image blocks. Among them, the values ​​of P and S are both 16. stands for floor function, H and W represent height and width.

[0038] S102: The feature information of the network blocks of some Transformers branch networks is introduced into the corresponding residual convolution blocks in the ResNet50 branch network after passing through the expand layer. The expand layer expands the 2D tensor into a 4D tensor. Specifically, the feature information of the network blocks numbered 1, 3, 5, and 8 are introduced into the corresponding residual convolution blocks in ResNet50 respectively.

[0039] S103: Perform channel concatenation on the expanded 4D tensor feature information and the corresponding residual convolution block.

[0040] In a specific embodiment, the expanded 4D tensor feature information and the corresponding residual convolution block are channel-concatenated, and the number of channels is increased to twice the original number. In order to allow the features to be normally fed into the next residual convolution block, a 1*1 convolution is used to restore the channel, and then the BN layer and the ReLU activation function are passed through. The expression is as follows: Among them, R represents the ReLU activation function, B represents the BN layer, and C 1*1 represents a 1*1 convolutional layer, Represents the features of the Transformers branch after transformation through the expand layer, Represents the features of ResNet50, i corresponds to the sequence number of the branch network, when i=0, the corresponding Transformers branch sequence number is 1, which is conv1 of ResNet50.

[0041] S104: The features converted by the expand layer of the Transformers branch and the corresponding ResNet50 features are passed through a bilinear pooling layer for feature fusion interaction, and are constrained by a loss function.

[0042] The above method proposes a hierarchical supervised person re-identification algorithm CT-Net that combines global features and local information, combining the advantages of CNN and Transformers to obtain local detail information and global information. Transformers are mainly used to obtain global feature dependencies in images, and CNN is mainly used to focus on local detail information. The global information of Transformers is unidirectionally aggregated to CNN during the training process to prevent CNN from overfitting on local detail information and ignoring the learning of global information. At the same time, bilinear pooling is also introduced in CT-Net to make full use of global information and local information to perform multi-level constraints on CT-Net.

[0043] In a specific embodiment, the present invention proposes a hierarchical supervised person re-identification algorithm CT-Net that combines global features and local information. Figure 2 FIG. 4 shows a framework diagram of a hierarchical supervised person re-identification algorithm combining global and local information according to a specific embodiment of the present application. Figure 2 As shown, the specific steps of the algorithm include:

[0044] Step S1: The Transformers used in this application is ViT. The entire network has 12 network blocks with the same structure, numbered 0-11. The input image (C×H×W) is cut into N image blocks, and the expression of N is as follows The values ​​of P and S are both 16. Represents the floor function. The N divided small blocks serve as the input of the Transformers branch in CT-Net.

[0045] Step S2: In order to prevent the CNN branch in CT-Net from over-learning local information and ignoring feature information, this application introduces the feature information of the network blocks numbered 1, 3, 5, and 8 of the Transformers branch into the corresponding residual convolution blocks in ResNet50. The global feature information of the Transformers branch is 2D, so it is necessary to first expand the 2D tensor to a 4D tensor through the expand layer, and expand the width, height, and number of channels to the same size as the corresponding residual convolution block.

[0046] Step S3: The expanded 4D tensor feature information and the corresponding residual convolution block are channel-concatenated, and the number of channels is doubled. In order to allow the features to be sent to the next residual convolution block normally, a 1*1 convolution is used to restore the channel to the same as the original one, and then pass through the BN layer and the ReLU activation function. The expression is as follows: R represents the ReLU activation function, B represents the BN layer, C 1*1 represents a 1*1 convolutional layer, Represents the features of the Transformers branch after transformation through the expand layer, Represents the features of ResNet50. When i=0, the corresponding Transformers branch number is 1, which is conv1 of ResNet50, and so on.

[0047] Step S4: The features converted by the Transformers branch through the expand layer and the corresponding ResNet50 features are passed through the bilinear pooling layer for feature fusion interaction, and then constrained by the loss function. and the characteristics of ResNet50 The matrix is ​​converted into a vector through the vec operation, and the feature F is obtained by the outer product of the two vectors. TC , after the bilinear pooling layer, the fused feature F is obtained TC ′, and perform unit operation on the feature to obtain the feature The specific formula is as follows: Where (x, y) represents the value of the feature at position (x, y), and the vec operation converts the matrix into a vector. The outer product of two vectors, S represents a constant.

[0048] Step S5: The loss functions involved in the entire graph are the triplet loss function and the label smoothing regularized cross entropy loss function. The triplet loss function and the label smoothing regularized cross entropy loss function are shown as follows: The formula of the triplet loss function is Where P represents the number of different pedestrian identities, K0 represents the selection of K0 different images for each pedestrian, and for each training image x a , in and x a Select an image x with the largest similarity distance among the images of the same identity p As a positive sample, in and x a Select an image x with the smallest similarity distance among images of different identities n As negative samples, is the threshold set, d(·,·) is defined as the Euclidean distance, batch represents the parameter size of batch processing; the formula of the cross entropy loss function of label smoothing regularization is Where ε∈[0,1], K is the number of pedestrian categories, and p(k) represents the probability of predicting belonging to category k.

[0049] The present invention proposes a hierarchical supervised person re-identification algorithm that combines global features and local information, which is mainly used to address the problem of pedestrian re-identification in real scenes. The advantages of CNN and Transformers are combined to obtain local detail information and global information. Transformers are mainly used to obtain global feature dependencies in images, and CNN is mainly used to focus on local detail information. The global information of Transformers is unidirectionally aggregated to CNN during the training process to prevent CNN from overfitting on local detail information and ignoring the learning of global information. At the same time, bilinear pooling is also introduced in CT-Net to make full use of global information and local information to perform multi-level constrained CT-Net. The present invention conducts experiments on pedestrian re-identification datasets spanning long time periods, and uses the Top-1 indicator of pedestrian re-identification for verification, which can achieve 60.2% Top-1, which is far higher than the best performance of existing algorithms.

[0050] Continue to refer Figure 3 , Figure 3A framework diagram of a hierarchical supervised pedestrian re-identification system combining global and local information according to an embodiment of the present application is shown. The system specifically includes a coarse image cutting module 301, a tensor expansion module 302, a channel cascade module 303 and a feature fusion module 304. Among them, the image cutting module 301 is configured to cut the input image into multiple image blocks, and use the image blocks as the input of the Transformers branch network; the tensor expansion module 302 is configured to introduce the feature information of the network block of part of the Transformers branch network into the corresponding residual convolution block in the ResNet50 branch network after passing through the expand layer, and the expand layer expands the 2D tensor into a 4D tensor; the channel cascade module 303 is configured to perform channel cascade on the expanded 4D tensor feature information and the corresponding residual convolution block; the feature fusion module 304 is configured to pass the features of the Transformers branch converted by the expand layer and the corresponding ResNet50 features through a bilinear pooling layer, perform feature fusion interaction, and constrain through a loss function.

[0051] Reference below Figure 4 , which shows a schematic diagram of the structure of a computer system 400 suitable for implementing an electronic device of an embodiment of the present application. Figure 4 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0052] like Figure 4 As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the system 400 are also stored. The CPU 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0053] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.

[0054] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

[0055] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0056] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0057] The modules involved in the embodiments of the present application may be implemented by software or by hardware.

[0058] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device: cuts the input image into multiple small image blocks, and uses the small image blocks as the input of the Transformers branch network; introduces the feature information of the network block of some Transformers branch networks into the corresponding residual convolution blocks in the ResNet50 branch network after passing through the expand layer, and the expand layer expands the 2D tensor into a 4D tensor; performs channel cascading on the expanded 4D tensor feature information and the corresponding residual convolution block; passes the features of the Transformers branch converted by the expand layer and the corresponding ResNet50 features through the bilinear pooling layer, performs feature fusion interaction, and constrains them through the loss function.

[0059] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A hierarchical supervised person re-identification method combining global and local information, It is characterized in that include: S1: Cut the input image into multiple small image patches, and use the small image patches as input to the Transformers branch network; S2: introducing the feature information of some network blocks of the Transformers branch network into the corresponding residual convolution blocks in the ResNet50 branch network through an expand layer, wherein the expand layer expands the 2D tensor into a 4D tensor; S3: performing channel cascading on the expanded 4D tensor feature information and the corresponding residual convolution block; S4: The features converted by the expand layer of the Transformers branch and the corresponding ResNet50 features are passed through a bilinear pooling layer for feature fusion interaction and constrained by a loss function.

2. The pedestrian re-identification method according to claim 1, It is characterized in that The Transformers branch network includes 12 network blocks of the same structure, the network blocks are numbered 0-11, and the input image is cut into N small image blocks. stands for floor function, H and W represent height and width.

3. The pedestrian re-identification method according to claim 2, It is characterized in that The feature information of the network blocks numbered 1, 3, 5, and 8 are respectively introduced into the corresponding residual convolution blocks in ResNet50.

4. The pedestrian re-identification method according to claim 1, It is characterized in that Specifically, S3 is as follows: the expanded 4D tensor feature information and the corresponding residual convolution block are channel-concatenated, and the number of channels is doubled. In order to allow the features to be normally fed into the next residual convolution block, the channels are restored by 1*1 convolution, and then passed through the BN layer and the ReLU activation function. The expression is as follows: Among them, R represents the ReLU activation function, B represents the BN layer, and C 1*1 represents a 1*1 convolutional layer, Represents the features of the Transformers branch after transformation through the expand layer, Represents the features of ResNet50, i corresponds to the sequence number of the branch network, when i=0, the corresponding Transformers branch sequence number is 1, which is conv1 of ResNet50.

5. The pedestrian re-identification method according to claim 4, It is characterized in that The specific expression of S4 is as follows: Where (x, y) represents the value of the feature at position (x, y), and the vec operation converts the matrix into a vector. The outer product of two vectors, S represents a constant.

6. The pedestrian re-identification method according to claim 1, It is characterized in that The loss function includes a triplet loss function and a label smoothing regularized cross entropy loss function, wherein the formula of the triplet loss function is: Where P represents the number of different pedestrian identities, K 0 Select K for each pedestrian 0 different images, for each training image x a , in and x a Select an image x with the largest similarity distance among the images of the same identity a As a positive sample, in and x a Select an image x with the smallest similarity distance among images of different identities n As negative samples, is the threshold value set, d(·,·) is defined as the Euclidean distance, batch represents the parameter size of batch processing; the formula of the cross entropy loss function of the label smoothing regularization is Where ε∈[0,1], K is the number of pedestrian categories, and p(k) represents the probability of predicting belonging to category k.

7. A computer-readable storage medium having one or more computer programs stored thereon, It is characterized in that When the one or more computer programs are executed by a computer processor, the method according to any one of claims 1 to 6 is implemented.

8. A hierarchical supervised person re-identification system combining global and local information, It is characterized in that The system comprises: Image cutting module: configured to cut the input image into multiple image patches, and use the image patches as input to the Transformers branch network; Tensor expansion module: configured to introduce the feature information of some network blocks of the Transformers branch network into the corresponding residual convolution block in the ResNet50 branch network through the expand layer, and the expand layer expands the 2D tensor into a 4D tensor; Channel cascade module: configured to perform channel cascade on the expanded 4D tensor feature information and the corresponding residual convolution block; Feature fusion module: It is configured to perform feature fusion interaction on the features converted by the Transformers branch through the expand layer and the corresponding ResNet50 features through the bilinear pooling layer, and constrain them through the loss function.

9. The pedestrian re-identification system according to claim 8, It is characterized in that The Transformers branch network includes 12 network blocks of the same structure, the network blocks are numbered 0-11, and the input image is cut into N small image blocks. in, stands for floor function, H and W represent height and width.

10. The pedestrian re-identification system according to claim 9, It is characterized in that The feature information of the network blocks numbered 1, 3, 5, and 8 are respectively introduced into the corresponding residual convolution blocks in ResNet50.

11. The pedestrian re-identification system according to claim 8, It is characterized in that The expanded 4D tensor feature information and the corresponding residual convolution block are channel-concatenated, and the number of channels is doubled. In order to allow the features to be sent to the next residual convolution block normally, 1*1 convolution is used to restore the channel, and then pass through the BN layer and ReLU activation function. The expression is as follows: Among them, R represents the ReLU activation function, B represents the BN layer, and C 1*1 Represents a 1*1 convolutional layer Represents the features of the Transformers branch after transformation through the expand layer, Represents the features of ResNet50, i corresponds to the sequence number of the branch network, when i=0, the corresponding Transformers branch sequence number is 1, which is conv1 of ResNet50.

12. The pedestrian re-identification system according to claim 11, It is characterized in that Where (x, y) represents the value of the feature at position (x, y), and the vec operation converts the matrix into a vector. The outer product of two vectors, S represents a constant.

13. The pedestrian re-identification system according to claim 8, It is characterized in that The loss function includes a triplet loss function and a label smoothing regularized cross entropy loss function, wherein the formula of the triplet loss function is: Where P represents the number of different pedestrian identities, K 0 Select K for each pedestrian 0 different images, for each training image x a , in and x a Select an image x with the largest similarity distance among the images of the same identity a As a positive sample, in and x a Select an image x with the smallest similarity distance among images of different identities n As negative samples, is the threshold value set, d(·,·) is defined as the Euclidean distance, batch represents the parameter size of batch processing; the formula of the cross entropy loss function of the label smoothing regularization is Where ε∈[0,1], K is the number of pedestrian categories, and p(k) represents the probability of predicting belonging to category k.

Images