Method for identity re-identification based on extraction of golden monkey torso features
By constructing a hypergraph structure based on the trunk features of golden snub-nosed monkeys, and using convolutional neural networks and MMpose tools to extract global and local features of golden snub-nosed monkeys, the problems of misalignment of local features and pose diversity in golden snub-nosed monkey identification were solved, and higher recognition accuracy was achieved.
Patent Information
- Application Number
- CN202310158824.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-23
- Publication Date
- 2025-12-19
- Estimated Expiration
- 2043-02-23
AI Technical Summary
Existing technologies for identifying golden snub-nosed monkeys rely on high facial feature similarity and auxiliary information. Video-based re-identification methods suffer from problems such as local feature misalignment and pose diversity, making it difficult to accurately identify individual golden snub-nosed monkeys.
A convolutional neural network was used to extract global features of golden snub-nosed monkeys, and the MMpose tool was used to detect key points. Global and local hypergraphs were constructed, and node features were updated using hypergraph neural networks and feature fusion was performed through an attention mechanism, which solved the problems of local feature misalignment and pose diversity.
The accuracy of video re-identification of golden monkeys was improved by constructing a hypergraph structure that integrates temporal and spatial relationships, effectively fusing feature information and enhancing the accuracy of identification.
Smart Images
Figure CN116229511B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of target recognition, in particular to an identity re-identification method based on extraction of golden monkey torso features. BACKGROUND
[0002] The golden monkey is a national first-class protected animal. Individual identification and detection of the golden monkey not only play a protective role, but also help better understand their habits. At present, observation and research on the golden monkey are mainly based on naked eye observation. With the wide application of computer vision technology, deep learning technology has gradually solved the problems in computer vision. The main application is facial recognition technology, but the facial recognition technology has high requirements for data and has problems such as high similarity of facial features of different golden monkeys. In addition, deep learning has made progress in the identification of other animals such as zebras, tigers, penguins and elephants, but it needs to rely on auxiliary information of animals such as body shape description, appearance characteristics and labels. Through re-identification technology, the appearance features and gait time sequence features of the golden monkey are combined to achieve the purpose of identifying the golden monkey.
[0003] At present, the research on re-identification mainly targets pedestrians. According to the analysis of pedestrian re-identification, there are two types of data forms, namely image-based and video-based. However, in the image-based identification task, the information contained in the image is less, and only appearance feature information can be obtained, and the time sequence feature information cannot be extracted according to the time change. Compared with the image-based re-identification, the video-based re-identification is more accurate. The current video-based re-identification research extracts global and local features and then combines distance metric learning. The extraction of local features mostly adopts the method of average segmentation, or segments the input picture, or segments the global features, but there are problems of whether the individual parts are aligned and whether the specific parts are accurately segmented. SUMMARY
[0004] In order to overcome at least one deficiency in the prior art, the embodiments of the present application provide an identity re-identification method based on extraction of golden monkey torso features.
[0005] In a first aspect, a target feature extraction model construction method is provided, comprising:
[0006] Preprocessing the collected target video to obtain model construction data; the target video contains a target;
[0007] Training the target feature extraction model based on the model construction data to obtain the trained target feature extraction model; the target feature extraction model comprises a feature extraction module and a hypergraph module;
[0008] The feature extraction module is configured to extract global feature information of the target according to the model construction data, and to obtain key point coordinate information of the target, split the global feature based on the key point coordinate information, and extract local feature information of the target.
[0009] The hypergraph module is configured to obtain video-level features of the target based on the global feature information and the local feature information.
[0010] In one embodiment, the feature extraction module includes a convolutional neural network module and a key point coordinate information extraction module, the convolutional neural network module is configured to extract global feature information of the target, and the key point coordinate information extraction module is configured to extract key point coordinate information of the target by using an MMpose tool.
[0011] In one embodiment, the video-level features of the target are obtained based on the global feature information and the local feature information, including:
[0012] The global feature information is taken as a node, and two adjacent nodes are connected by a hyperedge to construct a global hypergraph.
[0013] The local feature information is taken as a node, and two adjacent nodes are connected by a hyperedge to construct a local hypergraph.
[0014] The graph features of the global hypergraph and the local hypergraph are extracted.
[0015] The graph features of the global hypergraph and the graph features of the local hypergraph are aggregated to obtain the video-level features of the target.
[0016] In one embodiment, the graph features of the global hypergraph and the local hypergraph are extracted, including:
[0017] For any one node of the global hypergraph or the local hypergraph, the initial feature of each hyperedge of the node is determined.
[0018] The importance of each hyperedge is determined according to the node feature of the node and the initial feature of each hyperedge.
[0019] The final feature of all hyperedges corresponding to the node is obtained according to the importance of each hyperedge and the initial feature of each hyperedge.
[0020] The final feature of all hyperedges corresponding to the node is connected to the node feature of the node by using a fully connected layer to obtain the final feature of the node.
[0021] The graph features of the global hypergraph or the graph features of the local hypergraph are obtained according to the importance of each node and the final feature of each node.
[0022] In one embodiment, the collected target video is preprocessed to obtain model construction data, including:
[0023] The target detection network YOLOv3 is used for target detection on the target video, and a plurality of target-containing video frames are obtained.
[0024] The plurality of target-containing video frames are subjected to data augmentation processing, and model construction data is obtained.
[0025] In one embodiment, the target feature extraction model is trained based on the model construction data, comprising:
[0026] The target feature extraction model is trained by a classification loss function and a triplet loss function.
[0027] In a second aspect, an identity re-identification method based on a golden monkey torso local feature extraction is provided, comprising:
[0028] The target video to be identified and the candidate video are preprocessed respectively, and a plurality of target video frames to be identified and a plurality of candidate video frames are obtained; the target to be identified is contained in the target video to be identified and the candidate video;
[0029] The plurality of target video frames to be identified and the plurality of candidate video frames are input into the target feature extraction model, and video-level features of the plurality of target video frames to be identified and video-level features of the plurality of candidate video frames are output respectively;
[0030] The identity of the target to be identified is re-identified according to the video-level features of the plurality of target video frames to be identified and the video-level features of the plurality of candidate video frames.
[0031] The target feature extraction model is obtained according to the target feature extraction model construction method.
[0032] In one embodiment, the identity of the target to be identified is re-identified according to the video-level features of the plurality of target video frames to be identified and the video-level features of the plurality of candidate video frames, comprising:
[0033] The similarity between the video-level feature of each target video frame to be identified and the video-level feature of each candidate video frame is calculated to form a similarity matrix.
[0034] The identity of the target to be identified is re-identified according to the similarity matrix.
[0035] In a third aspect, a target feature extraction model construction device is provided, comprising:
[0036] A first video preprocessing unit is configured to preprocess a target video collected to obtain model construction data; the target video contains a target.
[0037] The model training unit is configured to train the target feature extraction model based on the model construction data, and obtain a trained target feature extraction model; the target feature extraction model comprises a feature extraction module and a hypergraph module;
[0038] The feature extraction module is configured to extract global feature information of the target according to the model construction data, and obtain key point coordinate information of the target, split the global feature based on the key point coordinate information, and extract local feature information of the target;
[0039] The hypergraph module is configured to obtain video-level features of the target based on the global feature information and the local feature information.
[0040] In a fourth aspect, an identity re-identification device based on the local features of the golden monkey torso is provided, comprising:
[0041] The second video preprocessing unit is configured to preprocess the target video to be identified and the candidate video respectively, and obtain a plurality of target video frames to be identified and a plurality of candidate video frames; the target video to be identified and the candidate video both contain the target to be identified;
[0042] The feature extraction unit is configured to input the plurality of target video frames to be identified and the plurality of candidate video frames into the target feature extraction model, and output video-level features of the plurality of target video frames to be identified and video-level features of the plurality of candidate video frames respectively;
[0043] The identity re-identification unit is configured to perform identity re-identification on the target to be identified according to the video-level features of the plurality of target video frames to be identified and the video-level features of the plurality of candidate video frames.
[0044] The target feature extraction model is obtained according to the target feature extraction model construction method.
[0045] Compared with the prior art, the present application has the following beneficial effects:
[0046] (1) The present application uses a convolutional neural network to extract global features, and uses a part segmentation method to extract local features. First, key point detection is performed using the MMpose tool, and then the obtained key point coordinate information is used to position and split the global features to obtain local features. Compared with using an average hard split method, using a skeleton structure can better express the body structure information of the golden monkey, and can solve the problems of part misalignment and pose diversification.
[0047] (2) The present application uses the obtained feature information to construct a hypergraph structure of time and space relationship, updates the hypergraph node features using a hypergraph neural network method, and finally uses an attention mechanism for feature fusion, effectively improving the accuracy of video golden monkey re-identification. BRIEF DESCRIPTION OF DRAWINGS
[0048] The present application can be better understood with reference to the following description in conjunction with the accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and, together with the description, serve to explain the principles of the application. In the drawings:
[0049] Figure 1 A flow chart of a target feature extraction model construction method according to an embodiment of the present application is shown;
[0050] Figure 2 A captured golden monkey video frame is shown;
[0051] Figure 3 A golden monkey video frame after target detection is shown;
[0052] Figure 4 A preprocessed golden monkey video frame is shown;
[0053] Figure 5 A schematic diagram of a target feature extraction model according to an embodiment of the present application is shown;
[0054] Figure 6 A golden monkey key point coordinate information extraction result is shown;
[0055] Figure 7 A flow chart of an identity re-identification method based on golden monkey torso local feature extraction according to an embodiment of the present application is shown;
[0056] Figure 8 A structural block diagram of a target feature extraction model construction device according to an embodiment of the present application is shown;
[0057] Figure 9 A structural block diagram of an identity re-identification device based on golden monkey torso local feature extraction according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0058] In the following, exemplary embodiments of the present application will be described with reference to the accompanying drawings. In the specification, all features of the practical embodiments are not described in order to make the present application clear and brief. It should be appreciated that many embodiment-specific decisions can be made in the process of developing any such practical embodiments in order to achieve the specific goals of the developers, and these decisions can vary from embodiment to embodiment.
[0059] It should also be noted here that, in order to avoid obscuring the present application due to unnecessary details, only the device structures closely related to the scheme according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.
[0060] It should be understood that the present application is not limited to the described embodiments by virtue of the description with reference to the drawings. In this context, the embodiments can be combined with one another, features can be replaced or borrowed between different embodiments, or one or more features can be omitted in an embodiment, if appropriate.
[0061] The present application aims at the problem that local features cannot be aligned in the existing identity re-identification process of golden monkeys. By using related knowledge in the field of deep learning, local features are obtained by segmentation according to parts, and the video re-identification task of golden monkeys is realized by combining temporal information.
[0062] The identity re-identification method based on the local features of the torso of a golden monkey according to the present application first extracts features of a target to be identified by using a target feature extraction model, and then performs identity re-identification of the target to be identified based on the extracted features. The construction method of the target feature extraction model is described in detail below.
[0063] The embodiment of the present application provides a target feature extraction model construction method, Figure 1 The flowchart of the target feature extraction model construction method according to the embodiment of the present application is shown in FIG. 1. Figure 1 The method comprises the following steps:
[0064] Step S1: Preprocessing the collected target video to obtain model construction data; the target video contains a target; here, the target can be a golden monkey.
[0065] In this step, a high-resolution camera is used to collect the video of a golden monkey, and videos meeting the conditions are manually selected, such as good angle, clear individual, good light, etc. Then, the YOLOv3 target detection network can be used for target detection on the selected videos to obtain multiple video frames containing only golden monkeys, and the redundant background is removed to reduce the interference of background factors and the processed video frames are corrected. Finally, data augmentation processing is performed on the video frames containing the target to increase the data quantity. Here, the data augmentation processing can include rotation, flipping, color jittering, Gaussian noise, etc. Thus, the model construction data is obtained, which is multiple preprocessed video frames containing golden monkeys. Figure 2 The collected golden monkey video is shown in FIG. 2. Figure 3 The golden monkey video frame after target detection is shown in FIG. 3. Figure 4 The preprocessed golden monkey video frame is shown in FIG. 4.
[0066] Step S2: Training the target feature extraction model based on the model construction data to obtain the trained target feature extraction model.
[0067] Figure 5A schematic diagram of a target feature extraction model according to an embodiment of the present application is shown, the target feature extraction model comprising a feature extraction module and a hypergraph module; the feature extraction module is configured to extract global feature information of a target according to model construction data; and is further configured to obtain key point coordinate information of the target, split the global feature based on the key point coordinate information, and extract local feature information of the target; and the hypergraph module is configured to obtain video-level features of the target based on the global feature information and the local feature information.
[0068] Specifically, in this step, the target feature extraction model can be trained by using a classification loss function and a triplet loss function.
[0069] In this embodiment, the global feature is extracted by using a convolutional neural network, and the local feature is extracted by using a part segmentation method, first, the key point detection is performed by using an MMpose tool, and then the global feature is split based on the obtained key point coordinate information; compared with using an average hard split method, the skeleton structure can better express the body structure information of the golden monkey, and can solve the problems of part misalignment and pose diversification.
[0070] In one embodiment, the feature extraction module comprises a convolutional neural network module and a key point coordinate information extraction module, the convolutional neural network module is configured to extract global feature information of a target, and the key point coordinate information extraction module is configured to extract key point coordinate information of the target by using an MMpose tool. Figure 6 The key point coordinate information extraction result of the golden monkey is shown.
[0071] In one embodiment, the video-level features of the target are obtained based on the global feature information and the local feature information, comprising:
[0072] In step S21, the global feature information is taken as a node, two adjacent nodes are connected by a hyperedge, and a global hypergraph is constructed;
[0073] In step S22, the local feature information is taken as a node, two adjacent nodes are connected by a hyperedge, and a local hypergraph is constructed;
[0074] In step S23, the graph features of the global hypergraph and the local hypergraph are extracted;
[0075] In step S24, the graph features of the global hypergraph and the graph features of the local hypergraph are aggregated to obtain the video-level features of the target.
[0076] In this embodiment, when the temporal distance between two nodes meets a time threshold, these two nodes are considered neighboring nodes and are connected using a hyperedge. The length of the hyperedge is related to the temporal distance between the two nodes connected by the hyperedge. Here, mutual information minimization can be used to aggregate the graph features of the global hypergraph and the graph features of the local hypergraph, which can reduce information redundancy and obtain the video-level features of the target.
[0077] In one embodiment, extracting graph features from the global hypergraph and local hypergraph includes:
[0078] Step S231: For any node in the global hypergraph or local hypergraph, determine node v. i The initial characteristics of each hyperedge;
[0079] Here, let node v i There are a total of K superedges. For the k-th superedge e ik Super-edge e ik There are multiple nodes, and the superedge e will be... ik The node features of all nodes are summed to obtain the hyperedge e. ik The initial characteristics of the hyperedge.
[0080] Step S232, based on node v i The importance of each hyperedge is determined by analyzing the node features and the initial hyperedge features of each hyperedge.
[0081] Here, the computation node v i The similarity between the node features and the initial features of each hyperedge represents the importance of each hyperedge. This similarity can be calculated using the cosine similarity method.
[0082] Step S233: Based on the importance of each hyperedge and its initial characteristics, obtain node v. i The final features of all corresponding hyperedges;
[0083] Specifically, node v i The final feature g of all corresponding hyperedges i :
[0084]
[0085]
[0086] Among them, a ik For the superedge e ik Importance, σ ik For a ik The result of normalization, b ik For the superedge e ik The initial characteristics of the hyperedge.
[0087] Step S234, move node v i The final features of all corresponding hyperedges and node v i The node features are connected using a fully connected layer to obtain node v. i The final feature
[0088] Step S235: Based on the importance of each node and the final features of each node, obtain the graph features of the global hypergraph or the graph features of the local hypergraph;
[0089] Specifically, graph feature h:
[0090]
[0091] Where v is the set of all nodes in the hypergraph. For node v i The final feature α i For node v i Importance weight values.
[0092]
[0093] in, d i For node v i The importance of W d This is the weight matrix.
[0094] In this embodiment, the node features are either global feature information or local feature information.
[0095] This application also provides an identity re-identification method based on local feature extraction of the golden monkey's torso. Figure 7 A flowchart of an identity re-identification method based on local feature extraction of the golden monkey torso according to an embodiment of this application is shown. The method includes:
[0096] Step S71: Preprocess the target video and candidate video to be identified respectively to obtain multiple target video frames and multiple candidate video frames; both the target video and the candidate video contain the target to be identified.
[0097] Step S72: Input multiple target video frames to be identified and multiple candidate video frames into the target feature extraction model, and output the video-level features of multiple target video frames to be identified and the video-level features of multiple candidate video frames respectively; the target feature extraction model is obtained according to the target feature extraction model construction method of the above embodiment.
[0098] Here, the plurality of candidate video frames constitute a candidate image set, and the candidate image set represents a series of images to be searched and identified. After the plurality of target video frames to be identified are input into the target feature extraction model, a plurality of video-level features are output. After the plurality of candidate video frames are input into the target feature extraction model, a plurality of video-level features are also output.
[0099] In step S73, identity re-identification of the target to be identified is performed according to the video-level features of the plurality of target video frames to be identified and the video-level features of the plurality of candidate video frames.
[0100] In this step, identity re-identification of the target to be identified means that for each target video frame to be identified, the video frame with the highest similarity in the candidate image set is found. The higher the similarity, the greater the possibility that the individuals in the two video frames are the same golden monkey.
[0101] Specifically, identity re-identification of the target to be identified is performed according to the video-level features of the plurality of target video frames to be identified and the video-level features of the plurality of candidate video frames, comprising:
[0102] First, the similarity between the video-level features of each target video frame to be identified and the video-level features of each candidate video frame is calculated to form a similarity matrix. Then, identity re-identification of the target to be identified is performed according to the similarity matrix.
[0103] In this embodiment, for the video-level features of each target video frame to be identified, the similarity between the video-level features of the plurality of candidate video frames is calculated. The candidate video frame with the highest similarity is determined as the most similar candidate video frame of the target video frame to be identified. The most similar candidate video frame and the target to be identified in the target video frame to be identified belong to the same target, and the greater the similarity value, the greater the possibility that the target to be identified belongs to the same target.
[0104] Further, the accuracy of identity re-identification can also be determined according to the similarity matrix. The average precision mean mAP can be used as the accuracy, and the accuracy is used to determine the accuracy of the identity re-identification result. Here, the higher the accuracy, the more accurate the re-identification result.
[0105] Based on the same inventive concept as the target feature extraction model construction method of the embodiments of the present application, the present application also provides a target feature extraction model construction device, Figure 8 The structure block diagram of the target feature extraction model construction device according to the embodiments of the present application is shown, and the device comprises:
[0106] The first video preprocessing unit 81 is used for preprocessing the collected target video to obtain model construction data; the target video contains a target;
[0107] The model training unit 82 is configured to train the target feature extraction model based on the model construction data, to obtain a trained target feature extraction model; the target feature extraction model comprises a feature extraction module and a hypergraph module;
[0108] The feature extraction module is configured to extract global feature information of the target according to the model construction data, and acquire key point coordinate information of the target, and split the global feature based on the key point coordinate information to extract local feature information of the target;
[0109] The hypergraph module is configured to obtain video-level features of the target based on the global feature information and the local feature information.
[0110] In this embodiment, the specific implementation functions of each module are the same as those of the foregoing embodiments of the target feature extraction model construction method, and will not be described in detail, and can be referred to in the embodiments of the target feature extraction model construction device.
[0111] Based on the same inventive concept as the identity re-identification method based on the local feature extraction of the golden monkey torso provided in the embodiments of the present application, the embodiments of the present application also provide an identity re-identification device based on the local feature extraction of the golden monkey torso, Figure 9 A structural block diagram of the identity re-identification device based on the local feature extraction of the golden monkey torso according to the embodiments of the present application is shown, and the device comprises:
[0112] The second video preprocessing unit 91 is configured to pre-process the target video to be identified and the candidate video respectively to obtain a plurality of target video frames to be identified and a plurality of candidate video frames; the target video to be identified and the candidate video both contain a target to be identified;
[0113] The feature extraction unit 92 is configured to input the plurality of target video frames to be identified and the plurality of candidate video frames into the target feature extraction model, and output video-level features of the plurality of target video frames to be identified and video-level features of the plurality of candidate video frames respectively; the target feature extraction model is obtained according to the target feature extraction model construction method in the foregoing embodiments;
[0114] The identity re-identification unit 93 is configured to perform identity re-identification on the target to be identified according to the video-level features of the plurality of target video frames to be identified and the video-level features of the plurality of candidate video frames.
[0115] In summary, the present application has the following beneficial effects:
[0116] (1) The application uses a convolutional neural network for global feature extraction, and local features are extracted using part segmentation. First, key point detection is performed using the MMpose tool, and then the obtained key point coordinate information is used to position and segment the global features. Compared with using average hard segmentation, using the skeleton structure can better express the body structure information of the golden monkey, and can solve the problems of part misalignment and pose diversification.
[0117] (2) The application uses the obtained feature information to construct a hypergraph structure of time and space relationship, updates the hypergraph node features using a hypergraph neural network, and finally uses an attention mechanism for feature fusion, effectively improving the accuracy of video golden monkey re-identification.
[0118] In order to verify the effectiveness of the identity re-identification method based on the golden monkey torso local feature extraction of the application, an experiment is built for identity re-identification. The evaluation index used is the average precision mean (mAP). mAP is a more comprehensive indicator of the effectiveness of the ReID algorithm. It reflects the degree to which all correct pictures of the individual to be identified are placed in front of the result queue in the candidate gallery, not just the first hit. When the mAP index is high, it means that all pictures of the individual to be identified are placed in the front. When the individual to be identified is placed in the front, and no other individual photo is inserted in front of it, it is equivalent to all photos of the same individual being the closest. This is the best case.
[0119] Using the created golden monkey video re-identification dataset, the application is compared with other re-identification algorithms BiCnetTKS, GRL, AITL, MGH, AP3D and TCL. 250 epochs are run, and the results are calculated every 50 epochs. The best results in the experiment are selected for comparison. The experimental results are shown in Table 1. According to Table 1, the method of the application has the highest mAP, indicating that the re-identification result of the method of the application is better.
[0120] Table 1 Comparison with other video re-identification algorithms
[0121]
[0122] The above is only for various embodiments of the application, but the protection scope of the application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the application, which should be covered within the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.
Claims
1. A method for constructing a target feature extraction model, characterized in that, The method comprises the following steps: Preprocessing a collected target video to obtain model construction data; the target video contains a target; Training a target feature extraction model based on the model construction data to obtain a trained target feature extraction model; the target feature extraction model comprises a feature extraction module and a hypergraph module; The feature extraction module is configured to extract global feature information of the target according to the model construction data; Further configured to obtain key point coordinate information of the target, split the global feature based on the key point coordinate information, and extract local feature information of the target; The hypergraph module is configured to obtain video-level features of the target based on the global feature information and the local feature information; The step of obtaining video-level features of the target based on the global feature information and the local feature information comprises the following steps: Constructing a global hypergraph by taking the global feature information as nodes and connecting adjacent two nodes with a hyperedge; Constructing a local hypergraph by taking the local feature information as nodes and connecting adjacent two nodes with a hyperedge; Extracting graph features of the global hypergraph and the local hypergraph; Aggregating the graph features of the global hypergraph and the graph features of the local hypergraph to obtain the video-level features of the target; The step of extracting graph features of the global hypergraph and the local hypergraph comprises the following steps: For any one node of the global hypergraph or the local hypergraph, determining a hyperedge initial feature of each hyperedge of the node; Determining the importance of each hyperedge according to the node feature of the node and the hyperedge initial feature of each hyperedge; Obtaining the final feature of all hyperedges corresponding to the node according to the importance of each hyperedge and the hyperedge initial feature; Connecting the final feature of all hyperedges corresponding to the node and the node feature of the node by using a full connection layer to obtain the final feature of the node; Obtaining the graph feature of the global hypergraph or the graph feature of the local hypergraph according to the importance of each node and the final feature of each node.
2. The method of claim 1, wherein, The feature extraction module comprises a convolutional neural network module and a key point coordinate information extraction module; the convolutional neural network module is configured to extract the global feature information of the target; and the key point coordinate information extraction module is configured to extract the key point coordinate information of the target by using an MMpose tool.
3. The method of claim 1, wherein, The step of preprocessing the collected target video to obtain the model construction data comprises the following steps: Detecting the target by using a YOLOv3 target detection network for the target video to obtain a plurality of video frames containing the target; Performing data augmentation processing on the plurality of video frames containing the target to obtain the model construction data.
4. The method of claim 1, wherein, The step of training the target feature extraction model based on the model construction data comprises the following steps: Training the target feature extraction model by using a classification loss function and a triplet loss function.
5. A method for identity re-identification based on extraction of local features of a golden monkey torso, characterized in that, The method comprises the following steps: Preprocessing a collected target video and a candidate video to obtain a plurality of target video frames to be identified and a plurality of candidate video frames; The target video to be identified and the candidate video both contain a target to be identified; inputting the plurality of target video frames to be identified and the plurality of candidate video frames into a target feature extraction model, and respectively outputting video-level features of the plurality of target video frames to be identified and video-level features of the plurality of candidate video frames; performing identity re-identification on the target to be identified according to the video-level features of the plurality of target video frames to be identified and the video-level features of the plurality of candidate video frames; The target feature extraction model is obtained by the target feature extraction model construction method in any one of claims 1-4.
6. The method of claim 5, wherein, In the method, performing identity re-identification on the target to be identified according to the video-level features of the plurality of target video frames to be identified and the video-level features of the plurality of candidate video frames, comprising: calculating the similarity between the video-level features of each target video frame to be identified and the video-level features of each candidate video frame to form a similarity matrix; performing identity re-identification on the target to be identified according to the similarity matrix.
7. A target feature extraction model construction apparatus characterized by comprising: Comprising: a first video preprocessing unit configured to pre-process a target video collected to obtain model construction data, wherein the target video includes a target; a model training unit configured to train a target feature extraction model based on the model construction data to obtain a trained target feature extraction model, wherein the target feature extraction model includes a feature extraction module and a hypergraph module; the feature extraction module is configured to extract global feature information of the target based on the model construction data, and obtain key point coordinate information of the target, split the global feature based on the key point coordinate information, and extract local feature information of the target; the hypergraph module is configured to obtain video-level features of the target based on the global feature information and the local feature information; the method of obtaining the video-level features of the target based on the global feature information and the local feature information, comprising: constructing a global hypergraph by taking the global feature information as nodes and connecting adjacent two nodes with a hyperedge; constructing a local hypergraph by taking the local feature information as nodes and connecting adjacent two nodes with a hyperedge; extracting graph features of the global hypergraph and the local hypergraph; aggregating the graph features of the global hypergraph and the graph features of the local hypergraph to obtain the video-level features of the target; the method of extracting the graph features of the global hypergraph and the local hypergraph, comprising: for any one node of the global hypergraph or the local hypergraph, determining a hyperedge initial feature of each hyperedge of the node; determining the importance of each hyperedge according to the node feature of the node and the hyperedge initial feature of each hyperedge; obtaining the final feature of all hyperedges corresponding to the node according to the importance of each hyperedge and the hyperedge initial feature; connecting the final feature of all hyperedges corresponding to the node and the node feature of the node with a full connection layer to obtain the final feature of the node; obtaining the graph features of the global hypergraph or the graph features of the local hypergraph according to the importance of each node and the final feature of each node.
8. An identity re-identification device based on extraction of local features of a golden monkey torso, characterized in that, Comprising: A second video preprocessing unit is configured to preprocess the target video to be identified and the candidate video respectively to obtain a plurality of target video frames to be identified and a plurality of candidate video frames; The target video to be identified and the candidate video both contain a target to be identified; A feature extraction unit is configured to input the plurality of target video frames to be identified and the plurality of candidate video frames into a target feature extraction model, and output video-level features of the plurality of target video frames to be identified and video-level features of the plurality of candidate video frames respectively; An identity re-identification unit is configured to perform identity re-identification on the target to be identified according to the video-level features of the plurality of target video frames to be identified and the video-level features of the plurality of candidate video frames. The target feature extraction model is obtained by the method for constructing a target feature extraction model according to any one of claims 1-4.
Citation Information
Patent Citations
Multi-view human action recognition method based on hypergraph learning
US20240177525A1