Person re-identification method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a person re-identification method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring a current image frame to be identified; using a feature extraction network to extract features of a person to be identified of the current image frame; using a feature detection algorithm to perform detection in the current image frame to obtain a current bounding box; acquiring an appearance feature library corresponding to a previous image frame, and using appearance features in the current bounding box to update the appearance feature library corresponding to the previous image frame, so as to obtain a current appearance feature library; determining an appearance cost matrix on the basis of the current bounding box and the current appearance feature library; using a Kalman filter to predict the position of a target box of the current image frame; determining a motion cost matrix on the basis of the current bounding box and the target box; performing trajectory matching on the basis of the appearance cost matrix and the motion cost matrix; and implementing person re-identification on the basis of a trajectory matching result.

Description

Pedestrian re-identification method, device, electronic device and storage medium

[0001] This disclosure is based on and claims the priority of Chinese patent application with application number 2023111690705 and application date September 12, 2023. The entire content of the Chinese patent application is hereby incorporated into this application by reference. Technical Field

[0002] The present disclosure relates to the field of image processing technology, and in particular to a pedestrian re-identification method, device, electronic device, and storage medium. Background Art

[0003] With the development of deep learning technology, the accuracy of multi-target tracking algorithms has continued to improve. To improve tracking accuracy, some multi-target tracking algorithms, such as the DeepSort algorithm, have incorporated person re-identification features for correlation. However, these algorithms update the person re-identification features of the trajectory based on a feature library, which does not effectively utilize the person re-identification features.

[0004] Summary of the Invention

[0005] In view of this, embodiments of the present disclosure provide a pedestrian re-identification method, device, electronic device, and storage medium to solve the problem of low pedestrian re-identification accuracy in the prior art.

[0006] According to a first aspect of the present disclosure, a method for person re-identification is provided, comprising:

[0007] Get the current image frame to be recognized;

[0008] Use the feature extraction network to extract the features of the pedestrian to be identified in the current image frame;

[0009] Use feature detection algorithm to detect and obtain the current detection frame from the current image frame;

[0010] Obtain the appearance feature library corresponding to the previous image frame, and use the appearance features in the current detection frame to update the appearance feature library corresponding to the previous image frame to obtain the current appearance feature library;

[0011] Determine the appearance cost matrix based on the current detection box and the current appearance feature library;

[0012] Use the Kalman filter to predict the position of the target box in the current frame;

[0013] Determine the motion cost matrix based on the current detection frame and the target frame;

[0014] Trajectory matching based on appearance cost matrix and motion cost matrix;

[0015] Pedestrian re-identification is achieved based on trajectory matching results.

[0016] According to a second aspect of the present disclosure, a pedestrian re-identification apparatus is provided, comprising:

[0017] an acquisition module, configured to acquire a current image frame to be recognized;

[0018] An extraction module is configured to extract features of a pedestrian to be identified in a current image frame using a feature extraction network;

[0019] A detection module is configured to detect a current detection frame from a current image frame using a feature detection algorithm;

[0020] An updating module is configured to obtain an appearance feature library corresponding to a previous image frame, and update the appearance feature library corresponding to the previous image frame using the appearance features in the current detection frame to obtain a current appearance feature library;

[0021] a determination module configured to determine an appearance cost matrix based on a current detection frame and a current appearance feature library;

[0022] A prediction module is configured to predict the position of the target box in the current frame using a Kalman filter;

[0023] The determination module is further configured to determine a motion cost matrix based on the current detection frame and the target frame;

[0024] a matching module configured to perform trajectory matching based on an appearance cost matrix and a motion cost matrix;

[0025] The recognition module is configured to realize pedestrian re-identification based on the trajectory matching result.

[0026] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0027] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0028] Compared with the prior art, the embodiments of the present disclosure have the following beneficial effects: the embodiments of the present disclosure improve the calculation accuracy of the appearance cost matrix by updating the appearance feature library by using the appearance features in the current detection frame, and calculating the appearance cost matrix based on the updated appearance feature library and the current detection frame; at the same time, the Kalman filter is used to predict the target frame of the current image frame, the motion cost matrix is ​​determined based on the current detection frame and the target frame, and the appearance cost matrix and the motion cost matrix are used for trajectory matching to realize pedestrian re-identification, and the accuracy of pedestrian re-identification is improved through dual-stage target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0030] FIG1 is a schematic diagram of an application scenario of an embodiment of the present disclosure.

[0031] FIG2 is a flow chart of a pedestrian re-identification method provided in an embodiment of the present disclosure.

[0032] FIG3 is a flow chart of a method for updating an appearance feature library corresponding to a previous image frame using appearance features in a current detection frame to obtain a current appearance feature library, provided by an embodiment of the present disclosure.

[0033] FIG4 is a flowchart of a method for determining an appearance cost matrix based on a current detection frame and a current appearance feature library, provided by an embodiment of the present disclosure.

[0034] FIG5 is a flow chart of a method for determining a motion cost matrix based on a current detection frame and a target frame provided by an embodiment of the present disclosure.

[0035] FIG6 is a flowchart of a method for trajectory matching based on an appearance cost matrix and a motion cost matrix provided by an embodiment of the present disclosure.

[0036] FIG7 is a flow chart of a pedestrian re-identification method provided in an embodiment of the present disclosure.

[0037] FIG8 is a schematic diagram of a pedestrian re-identification device provided by an embodiment of the present disclosure.

[0038] FIG9 is a schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present disclosure with unnecessary detail.

[0040] A method and apparatus for pedestrian re-identification according to an embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0041] FIG1 is a schematic diagram of an application scenario of an embodiment of the present disclosure, which may include terminal devices 1 , 2 , and 3 , a server 4 , and a network 5 .

[0042] The terminal devices 1, 2 and 3 can be hardware or software. When the terminal devices 1, 2 and 3 are hardware, they can be various electronic devices with display screens that support communication with the server 4, including but not limited to smart phones, tablet computers, laptop computers and desktop computers; when the terminal devices 1, 2 and 3 are software, they can be installed in the electronic devices described above. The terminal devices 1, 2 and 3 can be implemented as multiple software or software modules, or as a single software or software module, and the embodiments of the present disclosure are not limited to this. Furthermore, various applications can be installed on the terminal devices 1, 2 and 3, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0043] Server 4 can be a server that provides various services, for example, a backend server that receives requests sent by terminal devices that establish communication connections with it. The backend server can receive and analyze the requests sent by the terminal devices, and generate processing results. Server 4 can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center, all of which are not limited in the present embodiment.

[0044] It should be noted that the server 4 can be either hardware or software. When the server 4 is hardware, it can be various electronic devices that provide various services to the terminal devices 1, 2, and 3. When the server 4 is software, it can be multiple software programs or software modules that provide various services to the terminal devices 1, 2, and 3, or it can be a single software program or software module that provides various services to the terminal devices 1, 2, and 3. This is not limited in the present embodiment.

[0045] The network 5 can be a wired network connected by coaxial cable, twisted pair and optical fiber, or it can be a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), infrared, etc. The embodiments of the present disclosure are not limited to this.

[0046] The user can establish a communication connection with the server 4 via the network 5 through the terminal devices 1, 2 and 3 to receive or send information, etc. Specifically, the user can obtain the video to be identified or the current image frame to be identified through the terminal device, and transmit the video to be identified or the current image frame to be identified to the server for identification. The server can be configured with functional modules such as a feature extraction network and a Kalman filter, and can also store an appearance feature library. The server can obtain the video to be identified or the current image frame to be identified from the terminal device or the server's own database, and after processing it based on the feature extraction network, the Kalman filter, the appearance feature library and other functional modules, realize pedestrian re-identification.

[0047] It should be noted that the specific types, quantities and combinations of the terminal devices 1, 2 and 3, the server 4 and the network 5 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present disclosure do not limit this.

[0048] As mentioned above, with the advancement of deep learning technology, the accuracy of multi-target tracking algorithms has continued to improve. To improve tracking accuracy, some multi-target tracking algorithms, such as the DeepSort algorithm, have incorporated person re-identification features for correlation. However, these algorithms update the person re-identification features within the trajectory based on a feature library, which doesn't effectively utilize the person re-identification features.

[0049] In view of this, an embodiment of the present disclosure provides a pedestrian re-identification method, which improves the calculation accuracy of the appearance cost matrix by using the appearance features in the current detection frame to update the appearance feature library, and calculating the appearance cost matrix based on the updated appearance feature library and the current detection frame; at the same time, a Kalman filter is used to predict the target frame of the current image frame, and a motion cost matrix is ​​determined based on the current detection frame and the target frame. The appearance cost matrix and the motion cost matrix are used for trajectory matching, thereby realizing pedestrian re-identification, and improving the accuracy of pedestrian re-identification through dual-stage target tracking.

[0050] FIG2 is a flow chart of a pedestrian re-identification method provided by an embodiment of the present disclosure. The pedestrian re-identification method of FIG2 can be executed by the terminal device or server of FIG1. ​​As shown in FIG2, the pedestrian re-identification method includes the following steps:

[0051] In step S201 , a current image frame to be recognized is obtained.

[0052] In step S202, a feature extraction network is used to extract features of the pedestrian to be identified in the current image frame.

[0053] In step S203, a feature detection algorithm is used to detect and obtain a current detection frame from the current image frame.

[0054] In step S204, the appearance feature library corresponding to the previous image frame is obtained, and the appearance feature library corresponding to the previous image frame is updated using the appearance features in the current detection frame to obtain the current appearance feature library.

[0055] In step S205 , an appearance cost matrix is ​​determined based on the current detection frame and the current appearance feature library.

[0056] In step S206, a Kalman filter is used to predict a target frame of the current image frame.

[0057] In step S207 , a motion cost matrix is ​​determined based on the current detection frame and the target frame.

[0058] In step S208 , trajectory matching is performed based on the appearance cost matrix and the motion cost matrix.

[0059] In step S209 , pedestrian re-identification is performed based on the trajectory matching result.

[0060] In the disclosed embodiment, the pedestrian re-identification method can be executed by a server or by a terminal device with certain computing capabilities. For ease of description, the following description takes the target recognition method executed by a server as an example.

[0061] In the embodiment of the present disclosure, the server may first obtain a current image frame to be identified, wherein the current image frame to be identified may be an image frame in a video, and the image includes a pedestrian to be identified.

[0062] In the disclosed embodiment, the server may use a feature extraction network to extract features of the person to be identified in the current image frame. Specifically, the features may include appearance features and position features. Feature extraction networks include convolutional neural networks (CNNs), recurrent neural networks (RNNs), and the like.

[0063] In the disclosed embodiment, the server may use a feature detection algorithm to detect a current detection frame from the current image frame. Specifically, based on the features extracted by the feature extraction network, the feature detection algorithm may be used to detect all detection frames in the current image frame that meet a preset threshold. These detection frames are referred to as observation values.

[0064] In the embodiment of the present disclosure, the appearance feature library corresponding to the previous image frame stored in the server can be obtained, and the appearance feature library corresponding to the previous image frame can be updated using the appearance features in the current detection frame to obtain the current appearance feature library. Specifically, the server can store an appearance feature library. Unlike the related art in which the features in the feature library remain unchanged or are updated at a lower frequency, the appearance feature library in the embodiment of the present disclosure can be updated once after each frame of the image is detected to improve the matching accuracy of the appearance features, avoid situations where the appearance features cannot be accurately matched when the pedestrian is blurred or occluded in the image frame, or when pedestrian re-identification is performed across scenes, thereby improving the matching accuracy of the appearance features and thus improving the accuracy of pedestrian re-identification. Furthermore, the server can determine the appearance cost matrix based on the current detection frame and the current appearance feature library.

[0065] In the disclosed embodiment, the server can also use a Kalman filter to predict the target frame of the current image frame and determine a motion cost matrix based on the current detection frame and the target frame. Subsequently, trajectory matching is performed based on the appearance cost matrix and the motion cost matrix, and pedestrian re-identification is performed based on the trajectory matching results.

[0066] According to the technical solution provided by the embodiments of the present disclosure, the appearance feature library is updated by using the appearance features in the current detection frame, and the appearance cost matrix is ​​calculated based on the updated appearance feature library and the current detection frame, thereby improving the calculation accuracy of the appearance cost matrix. At the same time, a Kalman filter is used to predict the target frame of the current image frame, and a motion cost matrix is ​​determined based on the current detection frame and the target frame. The appearance cost matrix and the motion cost matrix are used for trajectory matching, thereby realizing pedestrian re-identification. The accuracy of pedestrian re-identification is improved through dual-stage target tracking.

[0067] FIG3 is a flow chart of a method for updating the appearance feature library corresponding to the previous image frame using the appearance features in the current detection frame to obtain the current appearance feature library provided by an embodiment of the present disclosure. As shown in FIG3 , the method includes the following steps:

[0068] In step S301, the appearance features in the appearance feature library corresponding to the previous image frame and the appearance features in the current detection frame are obtained.

[0069] In step S302, a sliding average algorithm is used to calculate the appearance features of the current appearance feature library based on the appearance features in the appearance feature library corresponding to the previous image frame and the appearance features in the current detection frame.

[0070] In step S303, a current appearance feature library is constructed based on the appearance features of the current appearance feature library.

[0071] In the disclosed embodiment, when using the appearance features in the current detection frame to update the appearance feature library corresponding to the previous image frame to obtain the current appearance feature library, the appearance features in the appearance feature library corresponding to the previous image frame and the appearance features in the current detection frame can be first obtained. Then, a sliding average algorithm is used to calculate the appearance features of the current appearance feature library based on the appearance features in the appearance feature library corresponding to the previous image frame and the appearance features in the current detection frame. Finally, the current appearance feature library is constructed based on the appearance features of the current appearance feature library.

[0072] In the embodiment of the present disclosure, the appearance features of the current appearance feature library can be calculated using the following formula:

[0073] Among them, β is the sliding average parameter, is the appearance feature of the current appearance feature library, P t-1 is the appearance feature in the appearance feature library corresponding to the previous image frame, P t is the appearance feature of the current detection frame.

[0074] In the embodiment of the present disclosure, the sliding average parameter can be calculated using the following formula:

[0075] β=ε+(1-ε)(1-T h +T l );

[0076] Among them, ε is a hyperparameter, and its value range is greater than or equal to 0 and less than or equal to 1, T h To detect the upper confidence limit, T l is the lower limit of detection confidence. Further, in a preferred embodiment of the present disclosure, ε can be taken as 0.6. Furthermore, T h and T l The value of can be determined according to actual needs and is not restricted here.

[0077] By adopting the technical solution of the embodiment of the present disclosure, the appearance feature library is updated when pedestrian re-identification is performed on each image frame through the sliding average algorithm, thereby improving the matching accuracy of the appearance features and further improving the accuracy of pedestrian re-identification.

[0078] FIG4 is a flow chart of a method for determining an appearance cost matrix based on a current detection frame and a current appearance feature library provided by an embodiment of the present disclosure. As shown in FIG4 , the method includes the following steps:

[0079] In step S401, the appearance features in the current detection frame are obtained.

[0080] In step S402, the similarity between the appearance features in the current detection frame and the appearance features in the current appearance feature library is calculated to obtain an appearance cost matrix.

[0081] In the disclosed embodiment, the server can first obtain the appearance features of the current detection frame, and then calculate the similarity between the appearance features of the current detection frame and the appearance features in the current appearance feature library to obtain an appearance cost matrix. The appearance cost matrix is ​​used to represent the appearance similarity of each detection frame with the historical trajectory.

[0082] FIG5 is a flow chart of a method for determining a motion cost matrix based on a current detection frame and a target frame according to an embodiment of the present disclosure. As shown in FIG5 , the method includes the following steps:

[0083] In step S501, the positions of the current detection frame and the target frame are obtained.

[0084] In step S502, the position similarity between the current detection frame and the target frame is calculated to obtain a motion cost matrix.

[0085] In the disclosed embodiment, the server can first obtain the positions of the current detection frame and the target frame, and then calculate the positional similarity between the current detection frame and the target frame to obtain a motion cost matrix. The motion cost matrix is ​​used to represent the positional similarity of each detection frame with the historical trajectory, such as coordinate similarity.

[0086] FIG6 is a flow chart of a method for performing trajectory matching based on an appearance cost matrix and a motion cost matrix according to an embodiment of the present disclosure. As shown in FIG6 , the method includes the following steps:

[0087] In step S601 , in response to determining that the appearance cost matrix is ​​greater than or equal to a preset appearance similarity threshold and the motion cost matrix is ​​greater than or equal to a preset position similarity threshold, it is determined that the trajectory of the pedestrian to be identified in the current image frame matches the existing trajectory.

[0088] In step S602 , in response to determining that the appearance cost matrix is ​​greater than or equal to a preset appearance similarity threshold and the motion cost matrix is ​​greater than or equal to a preset position similarity threshold, the confidence levels of the appearance cost matrix and the motion cost matrix are determined.

[0089] In step S603 , in response to the confidence of the appearance cost matrix and the motion cost matrix being greater than or equal to a preset confidence threshold, the trajectory of the pedestrian to be identified in the current image frame is determined to be a newly created trajectory.

[0090] In step S604 , in response to the confidence of the appearance cost matrix and the motion cost matrix being less than a preset confidence threshold, it is determined that the current trajectory matching fails.

[0091] In the disclosed embodiment, trajectory matching can be performed based on the appearance cost matrix and the motion cost matrix. Specifically, an appearance similarity threshold and a position similarity threshold can be preset. The appearance cost matrix is ​​compared with the preset appearance similarity threshold, and the motion cost matrix is ​​compared with the preset position similarity threshold. When the appearance cost matrix is ​​greater than or equal to the preset appearance similarity threshold, and the motion cost matrix is ​​greater than or equal to the preset position similarity threshold, it can be determined that the trajectory of the pedestrian to be identified in the current image frame matches the existing trajectory.

[0092] For example, if the appearance similarity between detection frame 1 and historical trajectory 1 in the appearance cost matrix is ​​greater than or equal to the preset appearance similarity threshold, and the position similarity between detection frame 1 and historical trajectory 1 is greater than or equal to the preset position similarity threshold, then it can be determined that detection frame 1 matches historical trajectory 1, that is, the pedestrian in detection frame 1 is the pedestrian corresponding to historical trajectory 1.

[0093] In the disclosed embodiment, when the appearance cost matrix is ​​greater than or equal to a preset appearance similarity threshold, and the motion cost matrix is ​​greater than or equal to a preset position similarity threshold, the confidence levels of the appearance cost matrix and the motion cost matrix can be further determined. When the confidence levels of the appearance cost matrix and the motion cost matrix are greater than or equal to the preset confidence threshold, the trajectory of the person to be identified in the current image frame can be determined to be a newly created trajectory.

[0094] For example, if the appearance similarity between the detection frame 2 and the historical trajectory 2 in the appearance cost matrix is ​​less than the preset appearance similarity threshold, and the position similarity between the detection frame 2 and the historical trajectory 2 is also less than the preset position similarity threshold, and further, the confidence of the detection frame 2 in the appearance cost matrix and the detection frame 2 in the motion cost matrix is ​​greater than or equal to the preset confidence threshold, then it can be considered that a new pedestrian appears in the detection frame 2, and the trajectory of the pedestrian to be identified in the current image frame is then set as a new trajectory.

[0095] In the embodiment of the present disclosure, when the confidence of the appearance cost matrix and the motion cost matrix is ​​less than a preset confidence threshold, it can be determined that the current trajectory matching has failed.

[0096] Figure 7 is a schematic flow chart of the person re-identification method provided by an embodiment of the present disclosure. As shown in Figure 7, the method first performs feature extraction. Specifically, a feature extraction network is used to extract features from the input image. The better the feature extraction network, the greater the impact of the features extracted. Feature detection is then performed. Based on the extracted features, a detection algorithm is used to detect all detection frames in the current frame that meet a threshold. These detection frames are referred to as observations. Better object detection algorithms can often more accurately detect objects in the current frame. Therefore, simply updating the detection algorithm can often improve tracking performance. In real-world scenarios, this stage is the speed bottleneck of the entire tracking algorithm. Next, an adaptive appearance feature update step is performed. The adaptive appearance feature module updates the appearance features stored in the historical trajectory. An appearance cost matrix is ​​then calculated. This is achieved by extracting the person re-identification features of the current detection frame and then calculating the similarity between these features and the appearance features stored in the historical trajectory. Simultaneously, a Kalman filter is used to predict the position of the target frame in the current frame, which is then used to calculate a motion cost matrix. This motion cost matrix is ​​calculated based on the coordinates of the current detection frame and the frame predicted by the Kalman filter. This matrix reflects the degree of match between the predicted and detected frames. Finally, the Hungarian algorithm is used to match the appearance cost matrix and the motion cost matrix. If a trajectory is not matched for a long time, the current trajectory is disconnected. If a detection box is not matched to any trajectory but has a high detection confidence, a new trajectory is created.

[0097] Using the technical solutions of the disclosed embodiments, a two-stage target tracking algorithm based on adaptive appearance feature updates is proposed based on existing multi-target tracking algorithms. This algorithm significantly improves the accuracy of multi-target tracking algorithms. Furthermore, through analysis of the appearance feature update algorithm, an adaptive appearance feature update algorithm is proposed. This algorithm dynamically updates the appearance features of the historical trajectory for each detection frame in each frame, thus avoiding the impact of blurred or occluded detection frames on the historical trajectory features.

[0098] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0099] The following are embodiments of the apparatus disclosed herein, which can be used to implement the method embodiments disclosed herein. For details not disclosed in the apparatus embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0100] FIG8 is a schematic diagram of a pedestrian re-identification device provided by an embodiment of the present disclosure. As shown in FIG8 , the pedestrian re-identification device includes:

[0101] The acquisition module 801 is configured to acquire a current image frame to be recognized.

[0102] The extraction module 802 is configured to use a feature extraction network to extract features of the pedestrian to be identified in the current image frame.

[0103] The detection module 803 is configured to detect and obtain a current detection frame from the current image frame using a feature detection algorithm;

[0104] An updating module 804 is configured to obtain an appearance feature library corresponding to a previous image frame, and update the appearance feature library corresponding to the previous image frame using the appearance features in the current detection frame to obtain a current appearance feature library;

[0105] A determination module 805 is configured to determine an appearance cost matrix based on the current detection frame and the current appearance feature library;

[0106] The prediction module 806 is configured to predict the position of the target frame in the current frame using a Kalman filter;

[0107] The determination module is further configured to determine a motion cost matrix based on the current detection frame and the target frame;

[0108] a matching module 807 configured to perform trajectory matching based on the appearance cost matrix and the motion cost matrix;

[0109] The recognition module 808 is configured to implement pedestrian re-identification based on the trajectory matching result.

[0110] According to the technical solution provided by the embodiments of the present disclosure, the appearance feature library is updated by using the appearance features in the current detection frame, and the appearance cost matrix is ​​calculated based on the updated appearance feature library and the current detection frame, thereby improving the calculation accuracy of the appearance cost matrix. At the same time, a Kalman filter is used to predict the target frame of the current image frame, and a motion cost matrix is ​​determined based on the current detection frame and the target frame. The appearance cost matrix and the motion cost matrix are used for trajectory matching, thereby realizing pedestrian re-identification. The accuracy of pedestrian re-identification is improved through dual-stage target tracking.

[0111] In an embodiment of the present disclosure, the appearance features in the current detection frame are used to update the appearance feature library corresponding to the previous image frame to obtain the current appearance feature library, including: obtaining the appearance features in the appearance feature library corresponding to the previous image frame and the appearance features in the current detection frame; using a sliding average algorithm, the appearance features of the current appearance feature library are calculated based on the appearance features in the appearance feature library corresponding to the previous image frame and the appearance features in the current detection frame; and constructing the current appearance feature library based on the appearance features of the current appearance feature library.

[0112] In the embodiment of the present disclosure, the appearance features of the current appearance feature library are calculated using the following formula: Among them, β is the sliding average parameter, is the appearance feature of the current appearance feature library, Pt-1 is the appearance feature in the appearance feature library corresponding to the previous image frame, P t is the appearance feature of the current detection frame.

[0113] In the embodiment of the present disclosure, the sliding average parameter is calculated using the following formula: β = ε + (1-ε) (1-T h +T l ); where ε is a hyperparameter, and its value range is greater than or equal to 0 and less than or equal to 1, T h To detect the upper confidence limit, T l is the lower confidence limit of detection.

[0114] In an embodiment of the present disclosure, an appearance cost matrix is ​​determined based on a current detection frame and a current appearance feature library, including: obtaining appearance features in the current detection frame; calculating the similarity between the appearance features in the current detection frame and the appearance features in the current appearance feature library to obtain an appearance cost matrix.

[0115] In the disclosed embodiment, a motion cost matrix is ​​determined based on the current detection frame and the target frame, including: obtaining the positions of the current detection frame and the target frame; and calculating the position similarity between the current detection frame and the target frame to obtain the motion cost matrix.

[0116] In an embodiment of the present disclosure, trajectory matching is performed based on the appearance cost matrix and the motion cost matrix, including: in response to determining that the appearance cost matrix is ​​greater than or equal to a preset appearance similarity threshold, and the motion cost matrix is ​​greater than or equal to a preset position similarity threshold, determining that the trajectory of the pedestrian to be identified in the current image frame matches the existing trajectory; in response to determining that the appearance cost matrix is ​​greater than or equal to the preset appearance similarity threshold, and the motion cost matrix is ​​greater than or equal to the preset position similarity threshold, determining the confidence of the appearance cost matrix and the motion cost matrix; in response to the confidence of the appearance cost matrix and the motion cost matrix being greater than or equal to the preset confidence threshold, determining that the trajectory of the pedestrian to be identified in the current image frame is a newly created trajectory; in response to the confidence of the appearance cost matrix and the motion cost matrix being less than the preset confidence threshold, determining that the trajectory matching has failed.

[0117] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0118] Figure 9 is a schematic diagram of an electronic device provided in an embodiment of the present disclosure. As shown in Figure 9, the electronic device 9 of this embodiment includes: a processor 901, a memory 902, and a computer program 903 stored in the memory 902 and executable by the processor 901. When the processor 901 executes the computer program 903, the steps of the aforementioned method embodiments are implemented. Alternatively, when the processor 901 executes the computer program 903, the functions of the modules / units in the aforementioned device embodiments are implemented.

[0119] Electronic device 9 may be a desktop computer, laptop, PDA, cloud server, or other electronic device. Electronic device 9 may include, but is not limited to, a processor 901 and a memory 902. Those skilled in the art will appreciate that FIG9 is merely an example of electronic device 9 and does not limit the electronic device 9. The electronic device 9 may include more or fewer components than shown, or different components.

[0120] The processor 901 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0121] The memory 902 can be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device 9. The memory 902 can also be an external storage device of the electronic device 9, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the electronic device 9. The memory 902 can also include both an internal storage unit of the electronic device 9 and an external storage device. The memory 902 is used to store computer programs and other programs and data required by the electronic device.

[0122] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or software functional units.

[0123] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0124] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure, and should all be included in the scope of protection of the present disclosure.

Claims

1. A pedestrian re-identification method, characterized in that: include: Get the current image frame to be recognized; Extracting features of the pedestrian to be identified in the current image frame using a feature extraction network; Using a feature detection algorithm to detect and obtain a current detection frame from the current image frame; Acquire an appearance feature library corresponding to a previous image frame, and use the appearance features in the current detection frame to update the appearance feature library corresponding to the previous image frame to obtain a current appearance feature library; Determine an appearance cost matrix based on the current detection frame and the current appearance feature library; Use the Kalman filter to predict the target frame of the current image frame; Determine a motion cost matrix based on the current detection frame and the target frame; Performing trajectory matching based on the appearance cost matrix and the motion cost matrix; Pedestrian re-identification is achieved based on the trajectory matching result.

2. The method according to claim 1, characterized in that The method of updating the appearance feature library corresponding to the previous image frame using the appearance feature in the current detection frame to obtain the current appearance feature library comprises: Acquire the appearance features in the appearance feature library corresponding to the previous image frame and the appearance features in the current detection frame; Using a sliding average algorithm, the appearance features of the current appearance feature library are calculated based on the appearance features in the appearance feature library corresponding to the previous image frame and the appearance features in the current detection frame; The current appearance feature library is constructed based on the appearance features of the current appearance feature library.

3. The method according to claim 2, characterized in that The appearance features of the current appearance feature library are calculated using the following formula: t new =βP t-1 +(1-β)P t ; Among them, β is the sliding average parameter, P t new is the appearance feature of the current appearance feature library, P t-1 is the appearance feature in the appearance feature library corresponding to the previous image frame, P t is the appearance feature of the current detection frame.

4. The method according to claim 3, characterized in that The sliding average parameter is calculated using the following formula: β = ε + (1-ε) (1-T h +T l ); Among them, ε is a hyperparameter, the value range is greater than or equal to 0 and less than or equal to 1, T h To detect the upper confidence limit, T l is the lower confidence limit of detection.

5. The method according to claim 1, characterized in that The determining of the appearance cost matrix based on the current detection frame and the current appearance feature library includes: Obtaining appearance features in the current detection frame; The similarity between the appearance features in the current detection frame and the appearance features in the current appearance feature library is calculated to obtain the appearance cost matrix.

6. The method according to claim 1, characterized in that The determining of a motion cost matrix based on the current detection frame and the target frame includes: Obtaining the positions of the current detection frame and the target frame; The position similarity between the current detection frame and the target frame is calculated to obtain the motion cost matrix.

7. The method according to claim 1, characterized in that The performing trajectory matching based on the appearance cost matrix and the motion cost matrix includes: In response to determining that the appearance cost matrix is ​​greater than or equal to a preset appearance similarity threshold, and the motion cost matrix is ​​greater than or equal to a preset position similarity threshold, determining that the trajectory of the pedestrian to be identified in the current image frame matches the existing trajectory; In response to determining that the appearance cost matrix is ​​greater than or equal to a preset appearance similarity threshold and the motion cost matrix is ​​greater than or equal to a preset position similarity threshold, determining confidences of the appearance cost matrix and the motion cost matrix; In response to the confidence of the appearance cost matrix and the motion cost matrix being greater than or equal to a preset confidence threshold, determining the trajectory of the pedestrian to be identified in the current image frame as a newly created trajectory; In response to the confidence of the appearance cost matrix and the motion cost matrix being less than a preset confidence threshold, it is determined that the current track matching fails.

8. A pedestrian re-identification device, characterized in that: include: An acquisition module, configured to acquire a current image frame to be recognized; An extraction module is configured to extract features of a pedestrian to be identified in the current image frame using a feature extraction network; A detection module is configured to detect and obtain a current detection frame from the current image frame using a feature detection algorithm; An updating module is configured to obtain an appearance feature library corresponding to a previous image frame, and use the appearance features in the current detection frame to update the appearance feature library corresponding to the previous image frame to obtain a current appearance feature library; A determination module, configured to determine an appearance cost matrix based on the current detection frame and the current appearance feature library; A prediction module, configured to predict the position of the target box in the current frame using a Kalman filter; The determination module is further configured to determine a motion cost matrix based on the current detection frame and the target frame; a matching module, configured to perform trajectory matching based on the appearance cost matrix and the motion cost matrix; The recognition module is configured to realize pedestrian re-recognition based on the trajectory matching result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to claim 1 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.

Images