Video object re-identification method and device and medium

Abstract

The invention relates to the technical field of computer vision, in particular to a video object re-identification method and device and a medium, and the method comprises the following steps: obtaining video data containing a track; constructing a video object re-identification model, wherein the video object re-identification model comprises a pyramid alignment module and an ID image matching module; frame features of each frame in the track are extracted through a feature extractor, and all frame features in the same track are aligned through a pyramid alignment module; an ID image matching module is adopted to fully utilize frame features, the matching degree of a query track and each ID is obtained, and the model is assisted in achieving more accurate video object re-recognition. According to the method, the staggered video frames are aligned through the pyramid alignment module, and the re-identification accuracy is improved on the premise of ensuring the calculation efficiency. In addition, through an ID image matching module, the model is guided to eliminate external condition interference, and tracks belonging to the same ID are retrieved and queried.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a video object re-identification method, device and medium. Background Art

[0002] The re-identification task aims to use a video or image captured by a camera as a query sample, and retrieve samples that belong to the same object (such as ships, pedestrians, etc.) as the query sample in the video or image library captured by different cameras. Deep learning-based re-identification models are widely used in crime investigation and intelligent monitoring due to their ability to efficiently extract valuable information from surveillance data. Compared with image-based re-identification tasks, the query and gallery data of video object re-identification tasks are both the motion trajectories of the objects. Trajectory data not only contains richer information, but also introduces some unique problems.

[0003] Key issues that affect the performance of video object re-identification models include video frame misalignment. Video frame misalignment refers to the phenomenon that there are obvious differences in the size, position and other features of the object body between different frames in the same trajectory. When building re-identification datasets, in order to reduce the interference of the background on the model, the object detection model is usually used to annotate the bounding box for the object. However, inaccurate bounding box annotation will lead to video frame misalignment. In addition, the rapid movement of the object will also cause the position of the object in adjacent video frames to be different. These two factors make the video frame misalignment phenomenon prevalent in re-identification datasets.

[0004] To solve the problem of video frame misalignment, existing research mainly focuses on two types of methods: 1. Adjacent frame alignment method. The adjacent frame alignment method realizes the alignment of two frames by designing an adjacent frame alignment module, and transmits this alignment frame by frame, thereby realizing the alignment of the entire video sequence frame. This is a relatively lightweight alignment method, but this type of method can only solve the alignment of video frames within a small range and has low computational efficiency.

[0005] 2. Partial alignment method. The partial alignment method uses external models or complex modules to divide the object body into several parts to achieve part-to-part alignment. Therefore, compared with the lightweight method of adjacent frame alignment, the computational cost of the partial alignment method is higher.

[0006] Furthermore, key issues that affect the performance of video object re-identification models include differences in external conditions, specifically the problem of increased intra-class differences between different trajectories of the same object due to differences in lighting, viewing angle, background and other conditions. When multiple external conditions change significantly, the appearance similarity of different trajectories of the same object is low, causing the model to be more inclined to retrieve samples that look similar but belong to different objects, thereby reducing the accuracy of re-identification. In industrial scenarios, the core goal of the re-identification task is to determine the identity of the query object rather than to find samples with similar appearance. Therefore, how to accurately identify samples with the correct ID under the interference of external conditions is the key to improving the performance of video re-identification models and meeting industrial needs.

[0007] The idea of ​​existing methods to solve this problem can be summarized as: processing external condition differences by disentangling subject features and external condition features. For example, the method to solve the perspective difference is essentially to disentangle and reorganize the features related to the perspective change and the features of the object itself. The method to solve background interference is to disentangle the background features from the main features of the object. The disentanglement process usually requires the use of auxiliary labels, such as perspective labels and foreground and background labels, which brings additional annotation costs.

[0008] In summary, there is an urgent need for a video object re-identification method, device and medium to solve the problems in the prior art. Summary of the invention

[0009] The present invention aims to provide a video object re-identification method, device and medium, and the specific technical solution is as follows: A video object re-identification method comprises the following steps: S100: Obtain video data containing the trajectory; S200: constructing a video object re-recognition model, wherein the video object re-recognition model includes a pyramid alignment module and an ID map matching module; S300: extracting frame features of each frame in the trajectory through a feature extractor, and aligning all frame features in the same trajectory using a pyramid alignment module; S400: An ID graph matching module is used to fully utilize frame features to obtain the ID matching degree between the query trajectory and each ID, and video object re-identification is achieved based on the ID matching degree.

[0010] Optionally, in S300, the process of using the pyramid alignment module to align all frame features is as follows: A feature extractor is used to extract frame features of each frame in the trajectory; the pyramid alignment module adopts a hierarchical pairwise alignment method to achieve alignment of all frames in the trajectory; a temporal transfer attention mechanism is introduced in the process of aligning the features of two frames to achieve alignment of the features of the two frames.

[0011] Optionally, in S400, the process of using the ID map matching module to process frame features is as follows: S401: Composition: For query and gallery data, different composition methods are used to construct query trajectory graph and ID graph respectively; S402: ID map pooling, pooling all ID maps to the same size as the query trajectory map; S403: Graph matching, calculating the ID matching degree between the query trajectory graph and the pooled ID graph; S404: ID matching guidance, using the ID matching as auxiliary information to help the model retrieve and query trajectories belonging to the same ID.

[0012] Optionally, in S401, the composition process includes: Different composition methods are used for query and gallery, and query trajectory graph and ID graph are constructed respectively; For query, the aligned frame features are used as graph nodes to construct a query trajectory graph, and each graph corresponds to a query trajectory; For the image library, the pooled trajectory features are used as graph nodes to construct an ID graph, where each graph corresponds to an ID; The adjacency matrix is ​​adaptively learned according to the similarity between nodes to complete the edge construction.

[0013] Optionally, in S402, all ID maps are pooled to the same size as the query trajectory map, and the process is as follows: A query-guided ID graph pooling strategy is adopted. Specifically, downsampling is performed based on the similarity between the nodes in the ID graph and the query trajectory graph, and nodes with high similarity to the ID graph are retained, so that the number of nodes in the ID graph is the same as that in the query trajectory graph. Perform graph pooling operations on all ID graphs to achieve pooling of all ID graphs.

[0014] Optionally, in S403, the graph matching process is as follows: The query trajectory graph and the pooled ID graph are matched node to node, and the ID matching degree between the query trajectory graph and the pooled ID graph is calculated based on the matched nodes.

[0015] Optionally, in S404, the process of ID matching guidance is as follows: The feature similarity between the pooled query trajectory and all gallery trajectories is calculated to obtain a similarity vector, and the similarity vector is corrected using the ID matching degree to obtain the final re-identification result.

[0016] In addition, the present invention also provides a computer device, including a memory and a processor; The memory is used to store a computer program executable on the processor; The processor is used to implement the steps of the above-mentioned video object re-identification method when executing the computer program.

[0017] In addition, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned video object re-identification method are implemented.

[0018] The application of the technical solution of the present invention has the following beneficial effects: The present invention provides a video object re-identification method, in which a video object re-identification model based on pyramid alignment and ID map matching, namely PAGM, is constructed. The model not only accurately aligns the misaligned video frames in the entire video range through the pyramid alignment module, but also has good parallelism. In addition, the present invention designs an ID map matching module, which constructs the query into a query trajectory map, and constructs all trajectories of the same ID in the gallery into an ID map. The ID matching degree obtained by matching the query trajectory map and the ID map can be used as auxiliary information to guide the model to eliminate interference from external conditions and retrieve trajectories belonging to the same ID as the query trajectory.

[0019] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a flow chart of the steps of the video object re-identification method in a preferred embodiment of the present invention.

[0022] Figure 2 4 is a model structure diagram of a video object re-identification model in a preferred embodiment of the present invention.

[0023] Figure 3 It is a comparison diagram of the traditional adjacent frame alignment module and the pyramid alignment module, where (a) is the traditional adjacent frame alignment module and (b) is the pyramid alignment module. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] See also Figure 1 , Figure 2 and Figure 3 , Figure 1 is a flowchart of the steps of the video object re-identification method in a preferred embodiment of the present invention; Figure 2 is a model structure diagram of a video object re-identification model in a preferred embodiment of the present invention; Figure 3 This is a comparison diagram of the traditional adjacent frame alignment module and the pyramid alignment module. It should be noted that Figure 3 Only the alignment process of the first frame is shown in the figure. The alignment process of each frame is similar, and each frame will get a result through the alignment operation.

[0026] like Figure 1 As shown, this embodiment provides a video object re-identification method, comprising the following steps: S100: Obtain video data containing the trajectory; S200: Construct a video object re-identification model, wherein the video object re-identification model includes a pyramid alignment module and an ID map matching module. The model structure is as follows: Figure 2 As shown; S300: Extract the features of each frame in the trajectory through the feature extractor, and use the pyramid alignment module to align the features of all frames in the same trajectory. The process is as follows: First, this embodiment uses the feature extractor to obtain the features of each frame in the trajectory. Then, this embodiment uses a hierarchical pairwise alignment method to achieve the alignment of all frames in the trajectory with the least alignment rounds. In the specific process of aligning the features of two frames, a temporal transfer attention mechanism is introduced to achieve the alignment of the features of the two frames. Figure 3 Specifically, for a The trajectory of the frame needs to be Wheel alignment. Indicates Round The input features of the frame, then The wheel alignment process is as follows: 1); 2); 3); Formula 1) describes the pairwise alignment process, where is called the reference frame, and Called an alignment frame, two frames together form an alignment group .function Achieved and alignment.

[0027] Formula 2) specifies the set of reference frame indices for each round, represents a set of non-negative integers. On the one hand, the index of the reference frame should be within the range of the trajectory, i.e. On the other hand, in each round of alignment, a pair of frames is grouped and the frame on the left is selected as the reference frame. After alignment, only the reference frame is kept for the next round. can be formalized as , Take all that satisfies A non-negative integer for the condition. For example, when ,and When, Formula 2) requires satisfy ,Right now .therefore . The index of the alignment group to be considered at each round.

[0028] Formula 3) specifies the index of the aligned frame in each round. For example, when hour, .because ,therefore, , the corresponding alignment frame is . Similarly, when hour, , the corresponding reference frame is , the alignment frame is It should be noted that In the first round and The result after alignment is In the first round and The result after alignment, so in the second round and The alignment of actually achieves the alignment of the first 4 frames. And so on. That is, after three rounds of alignment, the first frame and all frames in the trajectory sequence are aligned.

[0029] Furthermore, the alignment process of other frames is similar to that of frame 1. In this embodiment, a forward single-cycle arrangement is preferably used to ensure that each frame can be aligned once with other frames in the sequence. For example, when aligning frame 2, the initial arrangement of the sequence is ; In the alignment When the frame is , the initial arrangement of the sequence is In this embodiment, the sequence after the forward single cycle arrangement is reassigned with the number , to adapt to the above formula. It should be noted that the alignment process aims to enrich the features of each frame by integrating the information of other frames in the same trajectory. Therefore, alignment cannot completely replace the pooling operation of aggregating information. That is, the pyramid alignment module in this embodiment outputs a result for each frame feature, which represents the features of the aligned frame.

[0030] In a more specific embodiment, a temporal transfer attention module is used to achieve video frame alignment. The module is an improved self-attention mechanism, where Query is , both Key and Value contain and to be aligned This improved self-attention mechanism can align the features of the two frames.

[0031] This embodiment uses a pyramid alignment module to improve the problems existing in the traditional adjacent frame alignment method. Figure 3 As shown, in the traditional method, each frame has to go through at most Alignment times. Since information will gradually decay during the transmission process, in the case of a long sequence, the information collected by the model at the beginning of the sequence may be forgotten before reaching the end. Therefore, the traditional method is not suitable for long sequences such as motion trajectories. This embodiment adopts a hierarchical pairwise alignment method to reduce the number of alignments of all frames to , and each layer of alignment focuses on a smaller sequence fragment, effectively reducing the possibility of forgetting. In addition, the traditional adjacent frame alignment method only allows each frame to be aligned with the frame after the current frame, and does not achieve alignment within the entire video range; the pyramid alignment module in this embodiment ensures that each frame can align all frames in the entire sequence through a forward single-cycle arrangement design. Finally, the traditional adjacent frame alignment method needs to align all frames in sequence, and each alignment depends on the alignment result of the previous frame, resulting in poor parallelism and low computational efficiency; the pyramid alignment module in this embodiment allows the alignment operations within each layer to be performed in parallel, effectively improving the computational efficiency. In summary, the pyramid alignment module not only ensures that each frame is aligned once with other frames in the trajectory, but also avoids the problems of memory forgetting, limited alignment range and poor parallelism in the traditional adjacent frame alignment method.

[0032] S400: The ID graph matching module is used to fully utilize frame features to obtain the ID matching degree between the query trajectory and each ID, and the auxiliary model achieves more accurate video object re-identification.

[0033] In this embodiment, the process of using the ID map matching module to process frame features is as follows: S401: Mapping. For query and gallery data, different mapping methods are used to construct query trajectory graph and ID graph respectively. The process is as follows: For query, this embodiment uses the aligned frame features as graph nodes to obtain the node matrix of the query graph: , Indicates the number of nodes, Represents the node feature dimension. Then, this embodiment adaptively learns the adjacency matrix according to the similarity between nodes , to complete the edge construction. In the edge construction process, this embodiment first calculates the distance matrix between nodes ( represents the number of edges in the graph), the expression is as follows: 4); In formula 4), Representation Node The edge between The index in .

[0034] Furthermore, the distance matrix is mapped into an edge weight matrix This process involves the sigmoid function , scaling parameters , fully connected layer and batch normalization layers The application of the mapping expression is as follows: 5); In formula 5), ​​the edge weight matrix Mapping to adjacency matrix ,in, , Is a node The edge between In this embodiment, the query trajectory graph constructed is an undirected graph, each node in the graph represents a frame of the query trajectory, the number of nodes is equal to the length of the trajectory sequence, and the query trajectory graph reflects the characteristics of the query trajectory.

[0035] For the gallery, the method of this embodiment pools the aligned frame features to obtain trajectory features. For each ID, the method of this embodiment constructs an ID graph, in which each trajectory feature under the ID is used as a graph node. The edge construction method is the same as the query trajectory graph. In the ID graph constructed in this way in this embodiment, each node represents a trajectory feature, and the number of nodes is equal to the number of trajectories under the ID. The entire graph reflects the characteristics of the ID.

[0036] S402: ID map pooling, pooling all ID maps to the same size as the query trajectory map. Pooling all ID maps to the same size as the trajectory, the process is as follows: Based on the similarity between the nodes in the ID graph and the query trajectory graph, downsampling is performed, and nodes with high similarity to the ID graph are retained, so that the number of nodes in the ID graph is the same as that in the query trajectory graph. The above operation is performed on all ID graphs to achieve pooling of all ID graphs. Specifically, the pooling process is as follows: 6); In formula 6), Indicates the calculation of the features of the entire query trajectory graph. It means calculating the similarity between each node in the ID graph and the query trajectory graph, that is, the similarity between each trajectory corresponding to the ID and the query trajectory. Used to retain the ID graph that is most similar to the query graph nodes, and the rest are discarded. Equal to the number of nodes in the query trajectory graph.

[0037] S403: Graph matching, calculating the ID matching degree between the query trajectory graph and the pooled ID graph, the process is as follows: The query trajectory graph and the pooled ID graph are matched node to node, and the ID matching degree between the query trajectory graph and the pooled ID graph is calculated based on the matched nodes.

[0038] Specifically, this embodiment uses a cross-graph embedded-alignment layer (Cross-Graph Embedded-AlignmentLayer, CGEA for short) and a similarity prediction layer (Similarity Prediction layer, SP) for graph matching. The expression of graph matching is as follows: 7); 8); 9); Formula 7) and Formula 8) respectively represent the matching from gallery to query and from query to gallery in the CGEA layer. The input includes the query trajectory map And the ID map after pooling ,The role of the CGEA layer is to match the query graph and the ID graph node to node, ensuring that each node in the query graph can find the best matching node in the ID graph, facilitating the subsequent calculation of the graph matching degree.

[0039] In formula 9), the SP layer is used to calculate the similarity between the two graphs, that is, the ID matching degree between the query and the ID .

[0040] S404: ID matching guidance, using ID matching as auxiliary information to help the model retrieve and query tracks belonging to the same ID. In this embodiment, the feature similarity between the query track after pooling and all gallery tracks is calculated to obtain a similarity vector, and the similarity vector is corrected using the ID matching to obtain the final re-identification result. The process is as follows: Based on formula 9), we can calculate the query The match score between all IDs ,in Indicates the number of IDs in the gallery.

[0041] In addition, as shown in Formula 10), the model calculates the pooled features of the query trajectory And the pooled features of all tracks in the gallery The similarity between them generates a similarity vector ,in Indicates the number of trajectories in the gallery; 10); ID Match Score Guide Using Match Score To correct the similarity vector The specific process is as follows: First, Map to and The same dimensions, With ID matching The dimension mapping process is as follows: 11); 12); In formula 11), The correspondence between all trajectories and IDs is recorded. Represents search and The index of the relevant trajectory is stored in middle.

[0042] In formula 12), Representation query and The matching score between . Formula 12) is scaled by a factor right Scale and assign the scaled matching degree to the matching degree vector The index is those elements.

[0043] Further, by traversing all IDs, for the query Constructs a matching vector containing all ID matching information Through the above steps, the matching scores have been mapped to Same dimension, now we can use the matching score to correct the similarity vector . Added to This process introduces ID matching into the similarity vector, making the model more inclined to find the trajectory corresponding to the correct ID. .

[0044] In this embodiment, the total loss of the video object re-identification model is As shown in formula 13): 13) 14) 15) 16) In formula 14), represents the cross entropy loss, represents the number of samples, Representation sample The real label, Represents the model for the sample The predicted probability of .

[0045] In formula 15), represents the triplet loss, Represents the distance between two samples of the same category. Represents the distance between two samples of different categories, Represents the boundary value, which is used to ensure that the distance between positive samples is at least smaller than the distance between negative samples.

[0046] In formula 16), represents the graph matching loss, The labels representing the graph matches, Represents the query trajectory graph And the ID map after pooling From the same object, Indicates the ID matching degree.

[0047] In order to verify the advantages of the method of this embodiment, the experimental process is as follows: This embodiment conducts experiments on the video vessel re-identification dataset VVRD (Video Vessel Re-identification Dataset) and the video pedestrian re-identification dataset Mars. The detailed information of the datasets is shown in Table 1. Among them, the VVRD dataset is a video ship dataset obtained by collecting it from the Internet and undergoing a strict cleaning, trajectory division, and ship detection processing process. This dataset covers ship videos under a variety of different perspectives, lighting, and scenes, fully reflecting the diversity and complexity of ships in the real environment. In terms of data composition, the number of trajectories for each ID in the dataset is no less than 2, and the number of frames for each trajectory is evenly distributed between 10-20, thereby ensuring the richness and balance of the data. All datasets are divided into training sets and test sets in equal proportions.

[0048] Table 1 Statistics of video object re-identification dataset

[0049] The model of this embodiment is trained on a server with 4 Tesla V100 NVlink GPUs and 1 Intel Xeon Gold 6248 CPU. In the data loading stage, this embodiment randomly selects 32 IDs and samples 4 trajectories for each ID. For each trajectory, this embodiment adopts a restricted random sampling strategy, divides each trajectory into 8 segments, and randomly selects a frame from each segment. The size of all frames is adjusted to 256×128. In model construction, this embodiment uses ViTMultiScaleFlow as the backbone and modifies it to achieve the pyramid alignment function. During the training process, this embodiment uses the SGD optimizer for training. For different parts of the model, this embodiment sets different initial learning rates and dynamically adjusts the learning rate during the training process. The initial learning rate of the backbone is set to 0.01, and the initial learning rate of the ID graph matching module is set to 0.0035. All experiments are performed for a maximum of 40 epochs.

[0050] This embodiment uses two evaluation indicators to comprehensively evaluate the performance of the model. They are CMC (Cumulative Matching Characteristics) and mAP (mean Average Precision). CMC is an indicator commonly used to evaluate the performance of recognition systems. This indicator evaluates the performance of the model by checking whether each query sample is successfully matched to the corresponding sample in the library under different rankings. Among them, Rank-n represents the probability that there is a correct result in the first n samples of the retrieval result. mAP is an indicator widely used to evaluate the performance of re-identification. This indicator combines the precision-recall curves under different Intersection over Union (IoU) thresholds to calculate an average precision value.

[0051] Experimental results: This embodiment conducts simulation experiments on the video ship dataset VVRD and compares it with the current SOTA (State-of-the-Art) video re-identification model. The results are shown in Table 2.

[0052] PSTA: It proposes a pyramid spatiotemporal aggregation framework, which obtains trajectory-level features by gradually aggregating frame-level features, and supervises each layer of the training pyramid. Therefore, it can establish long-term temporal dependencies without losing useful local information, and is computationally efficient and has no feature redundancy.

[0053] SINet: In order to solve the problem that the frame-level attention regions of existing methods are local and highly similar, it proposes an attention region expansion method to make the attention region in the time series frame more and more extensive over time, thereby introducing more information. Then, frame-level feature integration is performed to obtain video-level features.

[0054] CAViT: In feature extraction, we abandon temporal interaction and adopt ViT. We also design a temporal transfer attention module for spatiotemporal interaction, so that each frame is aligned with the previous frame, thereby achieving alignment of all frames.

[0055] DCCT: It proposes a spatiotemporal complementary learning framework by combining the advantages of CNN and Transformer, and also proposes a self-distillation training strategy to transfer the video-level features obtained by complementary learning to each frame-level feature, so as to enhance the frame-level feature extraction capability.

[0056] TF-CLIP: For the first time, the CLIP model is combined with video pedestrian re-identification to achieve SOTA results.

[0057] In this simulation experiment, all models use the same configuration and are retrained on the VVRD dataset to ensure fairness. The experimental results show that the Rank1 accuracy of the method in this embodiment on the VVRD dataset reaches 92.83%, and the Rank5 accuracy reaches 100%, both of which achieve the best results, and the mAP index is also better than most existing methods.

[0058] Table 2 Comparison of PAGM and SOTA methods on VVRD dataset

[0059] Compared with other re-identification objects, ships have a greater difference in appearance under different viewing angles, which places higher requirements on the re-identification model's ability to handle differences in external conditions. The method in this embodiment introduces an ID graph matching module, which uses features unrelated to external conditions as the main reference for matching, thereby alleviating the interference caused by differences in external conditions. This module accurately calculates the ID matching degree for each query through graph pooling and graph matching technology, thereby providing strong guidance for the final trajectory retrieval. This design makes PAGM superior to other models in overall performance and achieves more accurate video ship re-identification.

[0060] In addition, this embodiment also conducts a comparative experiment on pedestrian objects: The performance of PAGM on pedestrian objects was verified on the Mars dataset and compared with the current SOTA video pedestrian re-identification method. As shown in Table 3, PAGM achieved a Rank 1 accuracy of 92.78% on the Mars dataset, which is better than all the compared SOTA methods.

[0061] Table 3 Comparison of PAGM and SOTA methods on Mars dataset

[0062] The Mars dataset was collected in outdoor scenes with complex backgrounds and partial occlusion of objects. Therefore, compared with other re-identification datasets, the inaccurate bounding box annotations in Mars are more prominent. However, the model of the method in this embodiment efficiently realizes the alignment of video frames by introducing a pyramid alignment module, thereby achieving better performance on the Mars dataset.

[0063] In addition, the performance of PAGM on the Mars dataset also fully demonstrates that the method of this embodiment has excellent generalization ability. Whether dealing with a ship dataset with significant changes in external conditions or a pedestrian dataset with inaccurate bounding box annotations, PAGM can show excellent performance.

[0064] In addition, this embodiment also provides a computer device, including a memory and a processor; The memory is used to store a computer program executable on the processor; The processor is used to implement the steps of the above-mentioned video object re-identification method when executing the computer program.

[0065] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the computer device.

[0066] The computer device may be a computing device such as a mobile phone, a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may include, but is not limited to, a processor and a memory. For example, the computer device may also include an input / output device, a network access device, a bus, etc.

[0067] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device.

[0068] The memory can be used to store the computer program and / or module, and the processor implements the computer program by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0069] Wherein, if the module / unit integrated in the computer device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0070] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned video object re-identification method are implemented.

[0071] The embodiment of the present invention provides a video object re-identification method, in which a video object re-identification model based on pyramid alignment and ID map matching, namely PAGM, is constructed. Through the pyramid alignment module, the model not only accurately aligns the misaligned video frames in the entire video range, but also has good parallelism. In addition, the present invention designs an ID map matching module, which constructs the query into a query trajectory map, and constructs all trajectories of the same ID in the gallery into an ID map. The ID matching degree obtained by matching the query trajectory map and the ID map can be used as auxiliary information to guide the model to eliminate interference from external conditions and retrieve trajectories belonging to the same ID as the query trajectory.

[0072] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video object re-identification method, characterized in that: The steps include: S100: Obtain video data containing the trajectory; S200: constructing a video object re-recognition model, wherein the video object re-recognition model includes a pyramid alignment module and an ID map matching module; S300: extracting frame features of each frame in the trajectory through a feature extractor, and aligning all frame features in the same trajectory using a pyramid alignment module; S400: using an ID graph matching module to fully utilize frame features, obtaining an ID matching degree between the query trajectory and each ID, and realizing video object re-identification based on the ID matching degree; In S300, the process of aligning all frame features using the pyramid alignment module is as follows: A feature extractor is used to obtain the frame features of each frame in the trajectory. The pyramid alignment module adopts a hierarchical pairwise alignment method to achieve the alignment of all frames in the trajectory. A temporal transfer attention mechanism is introduced in the process of aligning the features of two frames to achieve the alignment of the features of the two frames. In S400, the process of using the ID map matching module to process frame features is as follows: S401: Composition: For query and gallery data, different composition methods are used to construct query trajectory graph and ID graph respectively; S402: ID map pooling, pooling all ID maps to the same size as the query trajectory map; S403: Graph matching, calculating the ID matching degree between the query trajectory graph and the pooled ID graph; S404: ID matching guidance, using the ID matching as auxiliary information to help the model retrieve and query trajectories belonging to the same ID.

2. The video object re-identification method according to claim 1, characterized in that: In S401, the composition process includes: Different composition methods are used for query and gallery, and query trajectory graph and ID graph are constructed respectively; For query, the aligned frame features are used as graph nodes to construct a query trajectory graph, and each graph corresponds to a query trajectory; For the image library, the pooled trajectory features are used as graph nodes to construct an ID graph, where each graph corresponds to an ID; The adjacency matrix is ​​adaptively learned according to the similarity between nodes to complete the edge construction.

3. The video object re-identification method according to claim 1, characterized in that: In S402, all ID maps are pooled to the same size as the query trajectory map. The process is as follows: A query-guided ID graph pooling strategy is adopted. Specifically, downsampling is performed based on the similarity between the nodes in the ID graph and the query trajectory graph, and nodes with high similarity to the ID graph are retained, so that the number of nodes in the ID graph is the same as that in the query trajectory graph. Perform graph pooling operations on all ID graphs to achieve pooling of all ID graphs.

4. The video object re-identification method according to claim 1, characterized in that: In S403, the graph matching process is as follows: The query trajectory graph and the pooled ID graph are matched node to node, and the ID matching degree between the query trajectory graph and the pooled ID graph is calculated based on the matched nodes.

5. The video object re-identification method according to claim 1, characterized in that: In S404, the process of ID matching guidance is as follows: The feature similarity between the pooled query trajectory and all gallery trajectories is calculated to obtain a similarity vector, and the similarity vector is corrected using the ID matching degree to obtain the final re-identification result.

6. A computer device, characterized in that: including memory and processor; The memory is used to store a computer program executable on the processor; The processor is configured to implement the steps of the video object re-identification method according to any one of claims 1 to 5 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video object re-identification method according to any one of claims 1 to 5 are implemented.

Images