An open-world-oriented black deer cross-domain identity recognition method and system

Abstract

The application provides a method and system for open-world black muntjac cross-domain identity recognition, and belongs to the field of animal individual identity recognition.The method comprises the following steps: constructing a feature system and a data set; constructing a cross-domain re-identification model, using an intermediate mode and a paired cross-camera mode to constrain and process the cross-domain problem; constructing a hierarchical retrieval black muntjac identity database, clustering the feature vectors of black muntjacs with the same identity, and sequentially retrieving the cluster center vectors and the intra-cluster feature vectors in the order of retrieval, and optimizing the sorting; constructing an extensible black muntjac open-world re-identification system, periodically collecting black muntjac pictures, obtaining feature vectors and identity IDs through identity recognition, discriminating the information amount of feature vectors with the same ID, adding the remaining pictures to the data set after removing the redundancy, and updating the model and the database. The method uses an artificial intelligence algorithm to realize the identity recognition of black muntjac individuals under the condition of visible light and infrared light cross-modal and cross-camera in an open-world scenario, and is beneficial to the protection of endangered animals, i.e., black muntjacs with nocturnal habits.

Description

Technical Field

[0001] This invention relates to the field of animal individual identification, and more particularly to a method and system for cross-domain identification of black muntjacs in an open world. Background Technology

[0002] Rare wild animals are scarce in the wild, making them highly vulnerable to extinction. Species extinction has a significant impact on the ecological balance of the environment. Therefore, protecting rare wild animals is particularly urgent and important at present.

[0003] The black muntjac is a rare wild animal belonging to the Cervidae family and the Reeves genus. It is a Class I protected animal in my country. It measures 100-110 cm in length, about 60 cm in shoulder height, and weighs 21-26 kg. Its winter coat is dark brown on the upper body; the summer coat has an increased brown component. The tail is relatively long, generally exceeding 20 cm, black on the back, and pure white on the belly and sides, making the white tail very conspicuous. There are tufts of bright brown, light brown, or pale yellow long hairs on the forehead and nape behind the eyes. The black muntjac is endemic to China, with no subspecies. Its distribution range is very small, mainly found in southern Anhui, western Zhejiang, Huaiyuan in eastern Jiangxi, and the Wuyi Mountains in northern Fujian. It inhabits evergreen broad-leaved forests, mixed evergreen and deciduous broad-leaved forests, and shrublands at altitudes of around 1000 meters.

[0004] Black muntjac identification can provide crucial information about individual species' behavior, migration, reproduction, and interactions, helping researchers understand their role and interactions within the ecosystem. Identifying and tracking individual black muntjacs allows for the assessment of habitat use, migration routes, and ecological needs, thereby evaluating and monitoring the health of the ecosystem. This information is critical for ecosystem restoration and management, aiding in the development of effective conservation measures, the restoration of damaged habitats, and the promotion of black muntjac conservation and ecological balance.

[0005] Traditional methods for protecting rare animals primarily rely on Radio Frequency Identification (RIFD) technology. The basic principle is to attach electronic tags to rare animals and track them via radio frequency. However, this technology typically suffers from two problems: firstly, rare animals have wide ranges and are difficult to capture; secondly, the placement of electronic tags poses risks to the animals' health. Furthermore, with the development of artificial intelligence technology, especially the rapid advancement of computer vision, techniques for identifying rare wild animals using image and algorithm models have been widely researched and applied in academia and industry, particularly for identifying Siberian tigers. However, the black muntjac, also a rare wild animal, lacks the highly distinctive fur texture features of the Siberian tiger, making existing identification methods unsuitable. Therefore, an AI-based method is urgently needed to solve the problem of identifying and tracking individual black muntjacs.

[0006] When the re-identification system is actually deployed, it operates in an open-world scenario. The open world presents a more complex and challenging environment, closely resembling real-world scenarios. It contains targets with unknown identities or those not present in the training set, and the data is noisy and contains labeling errors. This means a large number of previously unseen black muntjac individuals will appear, and our labeling of the data based on existing knowledge will inevitably lead to numerous errors. Therefore, a method is needed to update and correct the dataset and algorithm model in an open-world environment, enabling the system to function correctly in real-world conditions. Furthermore, because black muntjacs are more active at dawn and dusk, visible light is weak, necessitating the use of infrared cameras for photography. However, there is an inconsistency in data distribution between visible light and infrared images. Additionally, due to multiple camera angles and internal differences between cameras, the data distribution of images from different cameras is also inconsistent. These inconsistencies are collectively referred to as cross-domain problems, requiring a cross-domain algorithm model to address the inconsistencies in data distribution caused by visible light and infrared images, as well as across different cameras, thereby improving the identification and tracking of black muntjac individuals.

[0007] In conclusion, the protection of black muntjac urgently requires a cross-domain identity recognition method and system oriented towards the open world. Summary of the Invention

[0008] To address the current lack of identification methods for black muntjac, this invention first proposes a cross-domain identification method for black muntjac in open-world scenarios. This method utilizes artificial intelligence algorithms to re-identify individual black muntjacs in visible and infrared images from different cameras. This method is then extended to open-world scenarios, facilitating the practical application of the model and contributing to the protection of this endangered species.

[0009] The specific technical solution is as follows:

[0010] This invention first provides a method for cross-domain identity recognition of black muntjacs in an open world, which includes the following steps:

[0011] 1) Collect visible light and infrared images of black muntjac in the field using multiple cameras, and perform data cleaning and processing to construct a dataset;

[0012] 2) Construct a cross-domain re-identification model. Train the cross-domain re-identification model using the dataset constructed in step 1) to obtain the trained cross-domain re-identification model. The cross-domain re-identification model includes an intermediate modality generation module, a feature extraction module, and a domain fusion module. The intermediate modality generation module includes two intermediate modality generators. The inputs of the two intermediate modality generators are visible light images and infrared light images in the dataset, respectively. The two intermediate modality generators generate two intermediate modalities based on the input images. The feature extraction module takes the visible light modality, infrared light modality, and the two intermediate modalities as inputs to obtain the feature vectors corresponding to each modality. The domain fusion module performs feature space transformation on the feature vectors, mapping the feature spaces of different modalities to a cross-domain invariant feature space, and then normalizes it to serve as the identity feature vector output by the entire cross-domain re-identification model.

[0013] 3) Construct a black muntjac identity image library, which stores all black muntjac images and their identity feature vectors in the dataset; during the construction of the black muntjac identity image library, for black muntjacs with the same identity, the K-Means algorithm is used to obtain K feature clusters, the redundancy of the identity feature vectors within the cluster is removed, and the identity feature vector of the cluster center is used to represent the cluster.

[0014] 4) For the image of the black muntjac to be detected, input it into the trained cross-domain re-identification model to obtain the identity feature vector, put the identity feature vector into the black muntjac identity image library for retrieval, and obtain the corresponding black muntjac individual identity ID.

[0015] 5) Construct a scalable open-world re-identification system for black muntjacs; collect all black muntjac images captured by cameras in each time period, and after data cleaning and processing, obtain the corresponding identity feature vector through a cross-domain re-identification model, query the image library to obtain the corresponding identity ID, remove redundancy and label the images according to the identity ID, incorporate the labeled images into the dataset and image library, and update the cross-domain re-identification model according to the dataset.

[0016] The present invention also provides a cross-domain identity recognition system for black muntjacs that implements the method, comprising:

[0017] Dataset building module: Used to process visible light and infrared light images captured by multiple cameras and build datasets;

[0018] Cross-domain re-identification model: used to extract identity feature vectors from images in the dataset; the cross-domain re-identification model is trained on the dataset;

[0019] Black Muntjac Identity Image Database: This database stores all black muntjac images, identity feature vectors, and their identity IDs in the dataset; the black muntjac identity image database can retrieve the identity ID based on the identity feature vector of an input image;

[0020] Open World System: The open world system collects all images of black muntjac captured by cameras in each time period. After data cleaning and processing, the corresponding identity feature vector is obtained through the cross-domain re-identification model, and the corresponding identity ID is obtained by querying the image library. The images are deredundant and labeled according to the identity ID, and the labeled images are merged into the dataset and image library. The cross-domain re-identification model is updated according to the dataset.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1) This invention employs an intermediate modality generation module and cross-modal constraints, combined with domain fusion module and cross-camera constraints, to overcome the problem in the prior art where inconsistent data distribution due to cross-domain issues adversely affects the performance of identity recognition tasks. This enables the re-recognition model to have good recognition performance for images of different modalities captured by different cameras.

[0023] 2) This invention adopts a hierarchical image library retrieval technology and uses the K-Means algorithm to cluster feature vectors with the same identity. During retrieval, the feature vector of the cluster center is retrieved first, and then the identity feature vectors within the cluster are retrieved. Therefore, it overcomes the problem that the retrieval efficiency of the existing technology decreases rapidly as the size of the image library increases in an open world environment, thereby improving the image library retrieval efficiency.

[0024] 3) This invention adopts scalable open-world identity recognition technology. For newly acquired images, the information content is used to determine the identity, and images with rich information content are retained and added to the database and image library. Therefore, it can overcome the problems of excessively large datasets and long model update and training time caused by the increase of deployment time in existing open-world systems, thereby alleviating storage pressure, improving model update and training efficiency, and improving the robustness of open-world systems. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a cross-domain identity recognition system for black muntjacs in an open world, according to an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of a cross-domain re-identification model according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of the image library retrieval process according to an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of an open-world system according to an embodiment of the present invention. Detailed Implementation

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the embodiments described below are intended to facilitate the understanding of the present invention and do not limit it in any way.

[0030] like Figure 1 As shown, the cross-domain identity recognition system for black muntjacs in the open world according to an embodiment of the present invention includes a dataset construction module, a re-identification model, a graph library, and an open world system.

[0031] The dataset construction process of the dataset construction module is as follows: Visible light and infrared images of black muntjacs captured by multiple field cameras are collected. These images undergo data cleaning, filtering out non-black muntjac images and eliminating duplicate and similar images. Building upon the previous step, images where the muntjac's face faces at an angle exceeding 90 degrees to the normal direction of the photograph are filtered out. For the remaining images, rectangular bounding boxes are used to define the muntjac's face (excluding the ears) as location labels. Additionally, image annotations include the muntjac's identification ID, lighting type, shooting time, and camera number. The dataset is divided into training, validation, and test sets in a 6:2:2 ratio. The training set participates in gradient backpropagation and weight updates for the re-identification model. The validation set does not participate in model weight updates and is only used for evaluating model performance metrics during training. The test set does not participate in training and is not used for performance evaluation during training; it is only used for evaluating model metrics after training is complete.

[0032] The re-identification model in this embodiment is as follows: Figure 2 As shown. The input of a visible light image contains three channels (RGB), while the input of an infrared image contains only one channel. For each visible light image input, after passing through an intermediate modality generator, a three-channel A-mode image is obtained, which is between the visible light and infrared modalities but closer to the visible light modality. For each infrared image, after passing through another intermediate modality generator, a three-channel B-mode image is obtained, which is between the visible light and infrared modalities but closer to the infrared modality. These two intermediate modality generators are composed of CNNs, and their weights are not shared. Then, these four images—visible light, A-mode, B-mode, and infrared light—are input into a weight-shared feature extractor to obtain the corresponding feature vectors. The feature extractor uses a ResNet network pre-trained on ImageNet. Then, cross-modal constraints are applied between the feature vectors of two adjacent modalities to calculate the cross-modality loss (CML). cm This involves calculating the sum of triplet losses within and between modes. cm The calculation formula is as follows:

[0033]

[0034] in, Represents the sum of triplet losses between modes. Represents the sum of triplet losses within a mode. The calculation formula is as follows:

[0035]

[0036] in, The intermodal triplet loss representing the visible light mode and the A-mode. The inter-modal triplet loss represents the difference between modes A and B. This represents the intermodal triplet loss between the B-mode and the infrared mode. For example, the calculation formula is as follows:

[0037]

[0038] Among them, Q A The set representing the eigenvector indices of mode A, |Q A | represents the number of feature vectors of mode A, α is the distance parameter, and y i f represents the identity ID corresponding to the feature vector with number i. i The eigenvector with ID i is represented by d(f i ,f j ) represents the eigenvector f i and f j The cosine similarity distance between them is calculated using the following formula:

[0039]

[0040] Wherein, c(f) i ,f j ) represents the eigenvector f i and f j The cosine similarity between them is calculated using the following formula:

[0041]

[0042] The sum of triplet losses within a mode L m The calculation formula is as follows:

[0043]

[0044] in, Represents the triplet loss within the visible light mode. Represents the triplet loss within mode A. Represents the triplet loss within mode B. This represents the triplet loss within the infrared light mode. For example, the calculation formula is as follows:

[0045]

[0046] For the feature vectors of the four modalities, cross-camera loss (CCL) is calculated using cross-camera constraints. cc This involves calculating the sum of triplet losses within and between cameras. L cc The calculation formula is as follows:

[0047]

[0048] in, Represents the sum of the triplet losses between cameras. This represents the sum of the triplet losses within the camera. The calculation formula is as follows:

[0049]

[0050] Where k represents the camera number, P k The set of feature vector indices representing the camera with number k, N represents the total number of feature vectors, and β is the distance parameter. The calculation formula is as follows:

[0051]

[0052] Then, all feature vectors are input into the domain fusion unit, which consists of a multilayer perceptron. The domain fusion unit outputs the fused identity feature vector and calculates the identity loss (IDL). id This involves calculating the sum of the classification cross-entropy loss and the additive angular margin loss for all images. id The calculation formula is as follows:

[0053]

[0054] in, The classification cross-entropy loss represents the loss for all images. This represents the additive angular spacing loss for all images. The calculation formula is as follows:

[0055]

[0056] Where M represents the number of different IDs of black muntjacs in the current image library, p ic y represents the predicted probability of ID c obtained after processing the i-th identity feature vector through FC and Softmax. ic This indicates whether the ID of the i-th identity feature vector is c. If it is, set it to 1; otherwise, set it to 0. The calculation formula is as follows:

[0057]

[0058] Where s represents the amplification factor parameter, Representing the yth i The cosine value of each feature vector and its corresponding class feature vector, θ j The meaning is the same as above, where m represents the distance parameter.

[0059] In the training of the re-identification model, image augmentation is performed on each input image in the dataset. In each training epoch, the following random transformations are applied: random center cropping, random image jittering, random affine transformation, and random image erasure. Random image jittering involves changes to image attributes including brightness, contrast, saturation, and hue. Random image erasure involves randomly selecting a rectangular region in the image and filling it with random values. The Adam algorithm is used as the weight optimizer for the re-identification model. The weighted aggregation of the re-identification model's mAP and identity loss IDL on the validation set yields the fitness index F, calculated using the following formula:

[0060] F = mAP - 0.01L id

[0061] The optimal model weights are determined by F; the larger the F value, the better the model weights are considered.

[0062] The image library construction process in this embodiment is as follows: All images of black muntjacs and their identity feature vectors in the dataset are used as the initial image library. For all images of each black muntjac identity ID, the K-Means algorithm is used to cluster the identity feature vectors. The initial value of K is chosen to be 2^m (where m is the number of different cameras where the black muntjac of that identity appears). The final K value is searched in the vicinity to minimize the variance of elements within the cluster, and the K value is used as a penalty term. After the K-Means algorithm, each black muntjac identity ID obtains K compact clusters. A redundancy removal algorithm is used for each cluster to remove identity feature vectors with insufficient information. The cluster center feature vector is used as the representative of that cluster.

[0063] The identity feature vector output by the re-identification module is then used to retrieve the corresponding identity ID from the image library. The image library retrieval process is as follows: Figure 3As shown, the specific steps are as follows: A hierarchical retrieval method is used to optimize retrieval speed. First, cluster center feature vectors are retrieved, resulting in a ranking of input feature vectors based on their distances to cluster center feature vectors. Then, a second retrieval is performed on all feature vectors within a cluster represented by the top-ranked cluster center feature vectors, resulting in a ranking of the distances between feature vectors within the cluster. Both retrieval processes use k-reciprocal encoding for ranking optimization. Specifically, for each retrieval, the cosine similarity between the query object and the image library is used as the initial ranking. Then, the distance of the k-reciprocal neighbor metric is calculated and aggregated with the distance of the cosine similarity metric to obtain the final distance metric table and ranking table.

[0064] The entire process of black muntjac identity re-identification is as follows: The collected image is input into the re-identification model to obtain the corresponding identity feature vector. The identity feature vector is then put into the image library for retrieval to obtain the black muntjac identity corresponding to the image.

[0065] The open-world system update scheme in this embodiment is as follows: Figure 4 As shown, the specific steps are as follows: Images captured by the cameras are collected periodically. First, based on the geographical location of the cameras, a fully connected graph of camera geographic distribution is obtained. Each node represents a camera, each edge represents the shortest commute time between nodes, and each node maintains information about individual black muntjacs appearing in each time period (black muntjac ID, lighting type, and shooting time). For the images collected in each time period, the corresponding identity feature vector is obtained through a re-identification model, and the corresponding identity ID is obtained by querying the image database.

[0066] For individuals with the same identity ID, if they appear in locations other than those in the previous time period, and the shortest commute time between these locations exceeds the time interval between the images, then these individuals are considered to be different individuals with similar characteristics. Similarly, if multiple individuals with the same identity appear in the same image, these individuals are also considered to be different individuals with similar characteristics. Labeling these individuals can obtain more distinguishing information. New identity IDs are assigned to these individuals, and the corresponding images are dereduplicated and added to the dataset and image library.

[0067] For individuals with unknown identity IDs, i.e., whose extracted identity feature vectors are more than a certain threshold away from all feature vectors in the image library, they are identified as new individuals, assigned a new identity ID, and their corresponding images are added to the dataset and image library after redundancy removal.

[0068] For individuals with known identity IDs (i.e., the distance between the extracted identity feature vector and any feature vector in the image library is less than a threshold, and they do not fall under the above-mentioned cases requiring identification as new individuals), further review and judgment (e.g., manual review by experts) is needed to verify the accuracy of the retrieved ID, i.e., whether it truly belongs to the same individual. If further judgment determines that they are not the same individual, they are treated as individuals with unknown identity IDs. If they are indeed the same individual, the new data is compared with the existing dataset to remove redundancy, and then added to both the dataset and the image library.

[0069] Query the individual information of black muntjacs maintained by each node in the fully connected graph of camera geographic distribution, find all identity IDs that have never appeared simultaneously on different cameras, find the same individuals and merge the identity IDs, and incorporate them into the dataset and the image library.

[0070] After each time period, the weights of the re-identification model are updated using the updated dataset according to the above method.

[0071] like Figure 1 As shown, this embodiment also provides a cross-domain identity recognition method for black muntjacs in an open world, including the following steps:

[0072] (1) Visible and infrared images of black muntjac were collected in the field using multiple cameras, and the data was cleaned and processed to construct a dataset;

[0073] (2) Construct a cross-domain re-identification model, introduce intermediate modes to handle the cross-modal problem of visible light and infrared light, use paired cross-camera modal constraints to handle the cross-camera problem, and use metric learning to optimize the angular space distribution of the feature vectors extracted by the re-identification model.

[0074] (3) Construct a black muntjac identity image database that supports fast retrieval. For black muntjac features with the same identity, the K-Means algorithm is used to obtain K more compact feature clusters. Redundancy is removed from the features within each cluster, and the cluster center feature vector is used to represent the cluster. Layered query verification is performed during retrieval, and the ranking results are optimized.

[0075] (4) The identity feature vector of the black muntjac image to be detected is obtained through the re-identification model, and it is put into the image library for retrieval to obtain the corresponding black muntjac individual identity ID.

[0076] (5) Construct a scalable open-world re-identification system for black muntjac. Collect all black muntjac images captured by cameras in each time period, determine the information content of the identity feature vectors corresponding to all images with the same identity ID, remove redundancy, label the remaining images, incorporate them into the dataset and image library, and update the re-identification model.

[0077] Considering the uncertainty of the movement of the black muntjac and its unrestricted changes in posture in the natural environment, coupled with the fact that the color and luster of the black muntjac's fur change over time, and the lack of obvious feature points (distinguishing it from the stripes of the Siberian tiger, the stripes of the sika deer, etc.), and its susceptibility to injury in the natural environment, the torso is removed from the feature system architecture. The ears, compared to the head, exhibit more posture changes, and the bounding box containing the ears will contain more background clutter; therefore, the ears are removed from the feature system architecture. Step (1) specifically involves: acquiring visible light and infrared images of the black muntjac taken by multiple field cameras, cleaning these images, filtering out images of non-black muntjacs, and simplifying duplicate and similar images. Based on the previous step, images in which the black muntjac's face faces at an angle exceeding 90 degrees to the normal direction of the photograph are filtered out (because it is difficult for annotators to distinguish the identity of black muntjacs in such extreme postures, this method can reduce a large number of annotation errors). For the remaining images, use a rectangle to outline the face of the black muntjac excluding its ears as a location marker. In addition, add annotations to the images including the black muntjac's ID, lighting type (visible light or infrared light), shooting time, and camera number.

[0078] The re-identification model in step (2) includes an intermediate modality generation module, a feature extraction module, and a domain fusion module.

[0079] The intermediate modality generation module includes two intermediate modality generators: a visible-infrared modality generator and an infrared-visible modality generator. The input of a visible light image contains three RGB channels, while the input of an infrared image contains only one channel. For each visible light image input, after passing through the visible-infrared modality generator, a three-channel A-mode image is obtained, which is somewhere between the visible and infrared modalities but closer to the visible light modality. For each infrared image, after passing through the infrared-visible modality generator, a three-channel B-mode image is obtained, which is also somewhere between the visible and infrared modalities but closer to the infrared modality. Both intermediate modality generators are constructed using CNNs, and their weights are not shared.

[0080] The feature extraction module uses a ResNet network pre-trained on ImageNet. Four images—visible light mode, A mode, B mode, and infrared light mode—are input into a weight-shared feature extractor to obtain the corresponding feature vectors. Then, for adjacent modalities, cross-modality loss (CML) is calculated using cross-modality constraints; this is the sum of intra-modal and inter-modal triplet losses. For all four modalities, cross-camera loss (CCL) is calculated using cross-camera constraints; this is the sum of intra-camera and inter-camera triplet losses.

[0081] The domain fusion unit consists of a multilayer perceptron. It performs a feature space transformation on the original feature vector, mapping the feature spaces of different modalities to a cross-domain invariant feature space. After normalization, this transformed feature vector is used as the output feature vector of the entire re-identification model. The identity loss (IDL) is calculated, which is the sum of the classification cross-entropy loss and the additive angular margin loss for all images.

[0082] The re-identification model training process is as follows:

[0083] Data augmentation: During the model training phase, the following enhancement measures are taken on the input image: random center cropping, which involves randomly cropping the image within a certain size and aspect ratio range; random image jittering, which involves randomly changing brightness, contrast, saturation, and hue; random affine transformation, which involves randomly rotating or shearing the image; and random image erasure, which involves erasing pixels within a certain size and aspect ratio range of the image and setting them to random values.

[0084] Modal constraints and loss functions: Joint modal constraints are applied to the four modes: visible light, A-mode, B-mode, and infrared light. Triple loss is used as the metric loss function to minimize the metric loss between visible light and A-mode, A-mode and B-mode, and B-mode and infrared light, while also minimizing the metric loss within the same mode. Cross-camera constraints are applied to the same black muntjac individual appearing in different cameras, using triple loss as the metric loss function to minimize the metric loss within and between cameras for the same black muntjac species. For all feature vectors processed by the domain fusion module, the Additive Angular Margin Loss proposed in Arcface is used as the metric loss function. This treats the feature space as a hypersphere, reducing the arc distance between intra-class feature vectors and increasing the arc distance between classes, thus minimizing the classification cross-entropy loss and the metric loss.

[0085] The method for constructing the black muntjac identity image library in step (3) is as follows:

[0086] (3-a) Use all images of black muntjacs and their identity feature vectors in the dataset as the initial image library.

[0087] (3-b) Based on the initial image library, the K-Means algorithm is used to cluster the feature vectors of all images for each black muntjac identity. The initial value of K is 2m (m is the number of different cameras in which the black muntjac of this identity appears, and 2 is set according to the modality and pose changes). The optimal K value is searched near the initial value by minimizing the weighted sum of the variance within the cluster and the K value to obtain K compact clusters, and the feature vector of the cluster center is used as the representative of the cluster.

[0088] (3-c) For each cluster of black muntjacs with each identity, remove redundancy to obtain the sub-cluster with the largest information content, and replace the original cluster to reduce the size of the image library.

[0089] During retrieval, the cluster center feature vectors are retrieved first, resulting in a ranking of the distances between these vectors. Then, the intra-cluster feature vectors represented by the top-ranked cluster center feature vectors are retrieved, yielding another ranking of the intra-cluster feature vectors based on their distances. Both retrieval processes utilize k-reciprocal encoding for ranking optimization. Specifically, for each retrieval, the cosine similarity between the query object and the database is used as the initial ranking. The distances of the k-reciprocal neighbors are then calculated and combined with the cosine similarity distances to obtain the final distance metric table and ranking table.

[0090] In step (5), the entire re-identification system was extended for the open world environment. For cross-domain problems, in response to the feature and identity misalignment caused by them, based on the camera network with known geographical distribution, spatiotemporal information can be used to obtain new samples with similar features but different identities and different features but the same identity. With the help of a small amount of manual annotation, samples with different features but the same identity are merged, and the data set capacity is increased less, resulting in a more discriminative model update.

[0091] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for cross-domain identity recognition of black muntjacs in an open world, characterized in that, Includes the following steps: 1) Collect visible light and infrared images of black muntjac in the field using multiple cameras, and perform data cleaning and processing to construct a dataset; 2) Construct a cross-domain re-identification model. Train the cross-domain re-identification model using the dataset constructed in step 1) to obtain the trained cross-domain re-identification model. The cross-domain re-identification model includes an intermediate modality generation module, a feature extraction module, and a domain fusion module. The intermediate modality generation module includes two intermediate modality generators. The inputs of the two intermediate modality generators are visible light images and infrared light images in the dataset, respectively. The two intermediate modality generators generate two intermediate modalities based on the input images. The feature extraction module takes the visible light modality, infrared light modality, and the two intermediate modalities as inputs to obtain the feature vectors corresponding to each modality. The domain fusion module performs feature space transformation on the feature vectors, mapping the feature spaces of different modalities to a cross-domain invariant feature space, and then normalizes it to serve as the identity feature vector output by the entire cross-domain re-identification model. The intermediate mode generation module in step 2) includes a visible-infrared mode generator and an infrared-visible mode generator. The input of the visible light image contains three RGB channels, while the input of the infrared light image contains only one channel. For each input visible light image, after passing through the visible-infrared mode generator, an A-mode image with three channels is obtained, which is between the visible light mode and the infrared mode but closer to the visible light mode. For each infrared light image, after passing through the infrared-visible mode generator, a B-mode image with three channels is obtained, which is between the visible light mode and the infrared mode but closer to the infrared mode. The two intermediate modes are the A-mode and the B-mode. The two intermediate mode generators are composed of CNNs, and the weights of the two intermediate mode generators are not shared. 3) Construct a black muntjac identity image library, which stores all black muntjac images and their identity feature vectors in the dataset; during the construction of the black muntjac identity image library, for black muntjacs with the same identity, the K-Means algorithm is used to obtain K feature clusters, the redundancy of the identity feature vectors within the cluster is removed, and the identity feature vector of the cluster center is used to represent the cluster. 4) For the image of the black muntjac to be detected, input it into the trained cross-domain re-identification model to obtain the identity feature vector, put the identity feature vector into the black muntjac identity image library for retrieval, and obtain the corresponding black muntjac individual identity ID. 5) Construct a scalable open-world re-identification system for black muntjacs; collect all black muntjac images captured by cameras in each time period, and after data cleaning and processing, obtain the corresponding identity feature vector through a cross-domain re-identification model, query the image library to obtain the corresponding identity ID, remove redundancy and label the images according to the identity ID, merge the labeled images into the dataset and image library, and update the cross-domain re-identification model according to the dataset.

2. The method for cross-domain identity recognition of black muntjacs in an open world as described in claim 1, characterized in that, Step 1) is as follows: Visible and infrared images of black muntjac captured by multiple field cameras were collected. These images were then cleaned to filter out images of non-black muntjac species and to remove duplicate and similar images. Images of the black muntjac whose face faces at an angle greater than 90 degrees to the normal direction of the photo are filtered out. For the remaining images, the face of the black muntjac excluding its ears is outlined with a rectangle as a location marker. In addition, the images are labeled with information including the black muntjac's ID, lighting type, shooting time, and camera number.

3. The method for cross-domain identity recognition of black muntjacs in an open world as described in claim 1, characterized in that, The feature extraction module in step 2) uses a ResNet network pre-trained on ImageNet; the domain fusion unit is composed of a multilayer perceptron.

4. The method for cross-domain identity recognition of black muntjacs in an open world as described in claim 1, characterized in that, Step 2) involves training the cross-domain re-identification model using the dataset, specifically including the following steps: Image augmentation is performed on the images in the dataset, and the augmented images are input into the cross-domain re-identification model. The input visible light image and infrared light image are used as the visible light mode and infrared light mode, respectively. Based on the input visible light image and infrared light image, the intermediate mode generation module is used to obtain two intermediate modes, namely mode A and mode B. Feature vectors are extracted from all modalities using a feature extractor. Cross-modal constraints are applied between the feature vectors of two adjacent modalities to calculate cross-modal loss. Cross-camera constraints are applied to the feature vectors of all modalities to calculate cross-camera loss. The feature vectors of all modalities are input into a domain fusion unit, which outputs a domain-fused identity feature vector. Identity loss is calculated on the identity feature vector. The model weights are optimized based on the loss.

5. The method for cross-domain identity recognition of black muntjacs in an open world according to claim 4, characterized in that, The image augmentation includes the following random transformations: random center cropping, random image jittering, random affine transformation, and random image erasure; wherein the image attribute changes involved in random image jittering include brightness, contrast, saturation, and hue.

6. The method for cross-domain identity recognition of black muntjacs in an open world according to claim 4, characterized in that, The cross-modal loss is the sum of the additive angular interval loss within and between modes; the cross-camera loss is the additive angular interval loss within the same camera; the identity loss is the classification cross-entropy loss and additive angular interval loss for all images.

7. The method for cross-domain identity recognition of black muntjacs in an open world as described in claim 1, characterized in that, Step 4) specifically refers to: During retrieval, the cluster center features are retrieved first, resulting in a ranking of the distances between the cluster center features. Then, the intra-cluster feature vectors represented by the top-ranked cluster center features are retrieved, resulting in a ranking of the distances between the feature vectors. Both retrieval processes use k-nearest neighbor encoding for ranking optimization. Specifically, for each retrieval, the cosine similarity between the query object and the image library is used as the initial ranking. Then, the distance of the k-nearest neighbor metric is calculated and aggregated with the distance of the cosine similarity metric to obtain the final distance metric table and ranking table.

8. The method for cross-domain identity recognition of black muntjacs in an open world as described in claim 1, characterized in that, Step 5) specifically refers to: Images captured by the cameras are collected periodically; based on the geographical locations of the cameras, a fully connected graph of camera geographic distribution is obtained, where each node represents a camera, each edge represents the shortest commute time between nodes, and each node maintains information on individual black muntjacs appearing in each time period; for the images collected in each time period, the corresponding identity feature vector is obtained through a cross-domain re-identification model, and the corresponding identity ID is obtained by querying the image library; For individuals with the same identity ID, if they appear in locations other than those in the previous time period, and the shortest commute time between these locations exceeds the time interval between the images, then these individuals are considered to be different individuals with similar characteristics. Similarly, if multiple individuals with the same identity appear in the same image, these individuals are also considered to be different individuals with similar characteristics. Labeling these individuals can yield more discriminative information, assigning them new identity IDs, and adding the corresponding images to the dataset and image library after removing redundancy. For individuals with unknown identity IDs, i.e., the distance between the extracted identity feature vector and all feature vectors in the image library exceeds a certain threshold, they are identified as new individuals, assigned a new identity ID, and the corresponding image is added to the dataset and image library after redundancy removal. For individuals with known identity IDs, i.e., the distance between the extracted identity feature vector and any feature vector in the image library is less than a threshold, and they do not fall under the above-mentioned cases that require identification as new individuals, further review and judgment are made to determine whether they are indeed the same individual; if it is further determined that they are not the same individual, they are treated as individuals with unknown identity IDs; if they are indeed the same individual, the newly added image data is compared with the existing dataset to remove redundancy, and then added to the dataset and image library. Query the individual information of black muntjacs maintained by each node in the fully connected graph of camera geographic distribution, find all identity IDs that have never appeared simultaneously on different cameras, find the same individuals and merge the identity IDs, and incorporate them into the dataset and the image library; After each time period, the weights of the re-identification model are updated using the updated dataset according to the above method.

9. A cross-domain identity recognition system for black muntjacs implementing the method of claim 1, characterized in that, include: Dataset building module: Used to process visible light and infrared light images captured by multiple cameras and build datasets; Cross-domain re-identification model: used to extract identity feature vectors from images in the dataset; the cross-domain re-identification model is trained on the dataset; Black Muntjac Identity Image Library: This library stores all images of black muntjacs in the dataset, along with their identity feature vectors. The black muntjac identity database can retrieve identity IDs based on the identity feature vectors of input images; Open World System: Collect all images of black muntjac captured by cameras in each time period. After data cleaning and processing, obtain the corresponding identity feature vector through the cross-domain re-identification model, query the image library to obtain the corresponding identity ID, remove redundancy and label the images according to the identity ID, merge the labeled images into the dataset and image library, and update the cross-domain re-identification model according to the dataset.

Images