A method and system for re-identification of swimming crab based on pyramid feature fusion

Abstract

The invention discloses a method and system for re-identifying swimming crabs of three tubercles based on pyramid feature fusion, comprising: constructing a deep learning model, the deep learning model is used for decomposing a carapace image of swimming crabs of three tubercles into a pyramid block structure, and jointly training according to a combination of ID loss and triple loss to generate a swimming crab re-identification model for tracing the swimming crabs of three tubercles; collecting carapace images of swimming crabs of three tubercles, constructing a training set, training the deep learning model, and generating a swimming crab re-identification model, the swimming crab re-identification model is used for tracing the swimming crabs of three tubercles by identifying the carapace images of swimming crabs of three tubercles; the intelligent identification method of swimming crabs of three tubercles designed by the invention has high algorithm accuracy and strong robustness, and provides a new technical solution for intelligent tracing of swimming crabs of three tubercles.

Description

Technical Field

[0001] The present invention relates to the technical field of biological traceability, and in particular to a method and system for re-identifying a swimming crab (Panthera trituberculata) based on pyramid feature fusion. Background Art

[0002] The swimming crab is a large marine economic crab species in my country's coastal areas. It has fat and delicious meat and high nutritional and economic value. In order to achieve the sustainable development of the swimming crab farming industry and the safety of aquatic food, it is necessary to implement an information retrieval mechanism for swimming crabs from farming to sales to ensure the effective operation of aquatic product production and the health rights and interests of consumers. In this regard, a complete traceability system for swimming crabs needs to be established during the production and operation of aquatic products.

[0003] In terms of the traceability technology of swimming crabs, traditional traceability methods include biological traceability, chemical traceability, and information traceability. Biological traceability has the problem of slow speed, chemical traceability has the problem of not being able to be promoted on a large scale, and information traceability has the problem of artificial falsification. In order to solve the problems brought by traditional traceability methods, it is proposed to introduce artificial intelligence into the traceability solution of swimming crabs.

[0004] ReID (Re-identification) is increasingly used in biometrics. Re-identification technology can be roughly divided into methods based on feature description and methods based on distance metric learning. Traditional ReID focuses on pedestrian recognition. In terms of feature description, the underlying visual features are extracted to obtain the feature representation of the object. Feature selection techniques such as Fisher vector coding further improve the recognition rate performance of re-identification. In terms of distance measurement, a relative distance comparison algorithm RDC is proposed in the prior art. RDC uses the Adaboost algorithm to reduce the demand for labeled samples.

[0005] With the development of deep learning technology, more and more scholars are trying to combine re-identification and deep learning. Unlike traditional pedestrian re-identification methods, pedestrian re-identification based on deep learning is not just a separate process. It can not only learn each part during the recognition process, but also learn the connectivity of each part.

[0006] However, the above methods only compare the extracted features without further analyzing the features. When encountering a swimming crab with a high similarity, misjudgment may occur. In addition, these methods only consider the features of the swimming crab locally or globally. Therefore, the present invention designs a method and system for re-identifying the swimming crab based on pyramid feature fusion to solve the technical problems of inaccurate identification in the prior art. Summary of the invention

[0007] In order to solve the above problems, the purpose of the present invention is to propose a method and system for re-identification of swimming crabs based on pyramid feature fusion, which is used to improve the re-identification accuracy of individuals with incomplete carapaces and reduce the ReID misidentification rate.

[0008] In order to achieve the above technical objectives, the present application provides a method for re-identifying the swimming crab based on pyramid feature fusion, comprising:

[0009] A deep learning model is constructed. The deep learning model is used to decompose the carapace image of the swimming crab into a pyramid-like block structure, and is jointly trained based on a combination of ID loss and triplet loss to generate a swimming crab re-identification model for tracing the swimming crab.

[0010] The carapace image of the swimming crab was collected, a training set was constructed, the deep learning model was trained, and a swimming crab re-identification model was generated. The swimming crab re-identification model was used to trace the swimming crab by identifying the carapace image of the swimming crab.

[0011] Preferably, in the process of constructing the deep learning model, the deep learning model is composed of a backbone module, a pyramid module, a basic block module, and a loss module connected in sequence;

[0012] The backbone module is used to extract the original features of the carapace image of the swimming crab, where the original features include ridges, grooves, ridges and irregular spots;

[0013] The Pyramid module is used to decompose the carapace image of the swimming crab into a pyramid-like block structure based on the original features extracted by the backbone module through a pyramid-like block strategy;

[0014] The Basic Block module is used to enhance the identification effect of each local area feature of the pyramid-shaped block structure;

[0015] The Loss module is used to optimize the training of each enhanced local area feature through the combined loss training mode of ID loss and triplet loss, and perform feature fusion in the inference stage to generate a feature vector for identifying the three-tuberculate crab.

[0016] Preferably, in the process of constructing the training set, each image of the carapace of the swimming crab three tubercles is marked with a first ID number;

[0017] Based on each first ID number, a carapace image of the swimming crab collected in different scenes is generated, and the second ID number is added after the first ID number to generate an ID number of the carapace image of the swimming crab.

[0018] Preferably, in the process of generating the feature vector by the Loss module, the carapace image of the swimming crab with a pyramid-shaped block structure is used as an anchor point to obtain the first ID number of the anchor point;

[0019] Based on the same first ID number, a sample with the lowest feature similarity with the anchor point is obtained as a positive sample, wherein the Euclidean distance between the two feature vectors is obtained as the feature similarity;

[0020] Based on different first ID numbers, obtain the sample with the highest feature similarity to the anchor point as a negative sample;

[0021] Construct triplets based on anchor points, positive samples, and negative samples;

[0022] According to the triples, obtain the triple loss;

[0023] According to the ID number, get the ID loss.

[0024] Preferably, in the process of obtaining the triplet loss, the triplet loss is expressed as:

[0025]

[0026] Where f(·) represents the feature extraction operation, They are the feature vectors of the anchor point with the first ID number i, the positive sample with the first ID number i, and the negative sample with the first ID number k, respectively, and α is 0.2.

[0027] Preferably, in the process of decomposing the carapace image of the swimming crab into a pyramid-like block structure, the carapace image of the swimming crab is divided into several functional areas based on the body functional characteristics of the back of the swimming crab, and according to the connection position relationship between each functional area, a pyramid-like block structure that completely covers the global features and local features of the swimming crab is generated, wherein the global features include protrusions, grooves, ridges and irregular spots, and the local features include local texture and / or spot features.

[0028] Preferably, in the process of generating a pyramid block structure, the pyramid block structure consists of a first block map and a second block map, wherein the first block map includes all functional areas, the second block map includes at least one functional area, and the number of functional areas in the second block map is less than that in the first block map.

[0029] The invention discloses a swimming crab re-identification system based on pyramid feature fusion, comprising:

[0030] A data acquisition module, used for collecting carapace images of the swimming crab three tubercles;

[0031] The data recognition module is used to trace the origin of the swimming crab by constructing a re-identification model for the swimming crab, identifying the feature vector of the carapace image of the swimming crab, wherein the feature vector includes global features and local features, the global features include protrusions, grooves, ridges and irregular spots, and the local features include local texture and / or spot features.

[0032] The present invention discloses the following technical effects:

[0033] The present invention designs a pyramid block strategy based on the characteristics of the swimming crab. The strategy obtains the local and global features of the swimming crab through a block fusion strategy, thereby avoiding the influence of the global features on the final detection performance.

[0034] In order to improve the accuracy of re-identification of individuals with incomplete carapaces, the present invention adopts a weighted adaptive integration strategy to deeply fuse multiple block features to reduce the impact of the incomplete parts of the three-tuberculate crab on the global features;

[0035] The present invention introduces triplet loss on the basis of using classification loss, which can effectively enhance the differences between individual characteristics of swimming crabs. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 It is a display diagram of different scenes of the swimming crab three tubercles according to the embodiment of the present invention;

[0038] Figure 2 is a schematic diagram of the carapace of the swimming crab trituberculate according to an embodiment of the present invention;

[0039] Figure 3 This is a diagram showing the block division strategy of the swimming crab (Portunus trituberculatus) according to an embodiment of the present invention;

[0040] Figure 4 It is a schematic diagram of pyramid-type block division according to an embodiment of the present invention;

[0041] Figure 5 This is a diagram of the deep learning model architecture for re-identification of the swimming crab trituberculate according to an embodiment of the present invention;

[0042] Figure 6 is a schematic diagram of feature fusion according to an embodiment of the present invention;

[0043] Figure 7is a TripletLoss description diagram according to an embodiment of the present invention;

[0044] Figure 8 It is the distance between the feature vector of the swimming crab trituberculate in the query set described in the embodiment of the present invention and the feature vector of the swimming crab trituberculate in the gallery set. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0046] like Figure 1-8 As shown, the present invention provides a method for re-identifying the swimming crab trituberculate based on pyramid feature fusion, comprising:

[0047] A deep learning model is constructed. The deep learning model is used to decompose the carapace image of the swimming crab into a pyramid-like block structure, and is jointly trained based on a combination of ID loss and triplet loss to generate a swimming crab re-identification model for tracing the swimming crab.

[0048] The carapace image of the swimming crab was collected, a training set was constructed, the deep learning model was trained, and a swimming crab re-identification model was generated. The swimming crab re-identification model was used to trace the swimming crab by identifying the carapace image of the swimming crab.

[0049] Further preferably, in the process of constructing the deep learning model of the present invention, the deep learning model is composed of a backbone module, a pyramid module, a basic block module, and a loss module connected in sequence;

[0050] The backbone module is used to extract the original features of the carapace image of the swimming crab, where the original features include ridges, grooves, ridges and irregular spots;

[0051] The Pyramid module is used to decompose the carapace image of the swimming crab into a pyramid-like block structure based on the original features extracted by the backbone module through a pyramid-like block strategy;

[0052] The Basic Block module is used to enhance the identification effect of each local area feature of the pyramid-shaped block structure;

[0053] The Loss module is used to optimize the training of each enhanced local area feature through the combined loss training mode of ID loss and triplet loss, and perform feature fusion in the inference stage to generate a feature vector for identifying the three-tuberculate crab.

[0054] Further preferably, in the process of constructing the training set, in order to facilitate model training and feature recognition, the carapace images of the swimming crab three tubercles are numbered, and each carapace image of the swimming crab three tubercles is marked with a first ID number;

[0055] Based on each first ID number, a carapace image of the swimming crab collected in different scenes is generated, and the second ID number is added after the first ID number to generate an ID number of the carapace image of the swimming crab.

[0056] Further preferably, in the process of generating feature vectors through the Loss module, the present invention constructs a triplet for triplet loss training, uses the carapace image of the three-tuberculate swimming crab with a pyramid-shaped block structure as an anchor point, and obtains the first ID number of the anchor point;

[0057] Based on the same first ID number, a sample with the lowest feature similarity with the anchor point is obtained as a positive sample, wherein the feature similarity is used to represent the Euclidean distance between two feature vectors;

[0058] Based on different first ID numbers, obtain the sample with the highest feature similarity to the anchor point as a negative sample;

[0059] Construct triplets based on anchor points, positive samples, and negative samples;

[0060] According to the triples, obtain the triple loss;

[0061] According to the ID number, get the ID loss.

[0062] Further preferably, in the process of obtaining the triplet loss, the triplet loss is expressed as:

[0063]

[0064] Where f(·) represents the feature extraction operation, They are the feature vectors of the anchor point with the first ID number i, the positive sample with the first ID number i, and the negative sample with the first ID number k, respectively, and α is 0.2.

[0065] Further preferably, in the process of decomposing the carapace image of the swimming crab of three tubercles into a pyramid-like block structure, the present invention divides the carapace image of the swimming crab of three tubercles into several functional areas based on the body functional characteristics of the back of the swimming crab of three tubercles, and generates a pyramid-like block structure that completely covers the global features and local features of the swimming crab of three tubercles according to the connection position relationship between each functional area, wherein the global features include protrusions, grooves, ridges and irregular spots, and the local features include local texture and / or spot features.

[0066] Further preferably, in the process of generating a pyramid block structure, the pyramid block structure consists of a first block map and a second block map, wherein the first block map includes all functional areas, the second block map includes at least one functional area, and the number of functional areas in the second block map is less than that in the first block map.

[0067] The invention discloses a swimming crab re-identification system based on pyramid feature fusion, comprising:

[0068] A data acquisition module, used for collecting carapace images of the swimming crab three tubercles;

[0069] The data recognition module is used to trace the origin of the swimming crab by constructing a re-identification model for the swimming crab, identifying the feature vector of the carapace image of the swimming crab, wherein the feature vector includes global features and local features, the global features include protrusions, grooves, ridges and irregular spots, and the local features include local texture and / or spot features.

[0070] Example 1: The present invention designs a method for tracing the origin of the swimming crab and conducts an experiment, the specific process is as follows:

[0071] Preparation of experimental materials:

[0072] The 201 adult swimming crabs used in the present invention were from a crab farm in Ningbo. The experimental crabs had intact appendages, an average body weight of 318.90±38.07 g (mean±SD), a total carapace width of 16.73±.75 cm, and a body length of 8.65±0.41 cm. The above-mentioned swimming crabs were numbered and their carapaces were photographed using a mobile phone (Huawei P30 rear camera).

[0073] Overall experimental plan:

[0074] The experimental material is the cultured swimming crabs, with a total of 201 swimming crabs. The above swimming crabs are numbered in sequence from 1 to 201 as the ID of the swimming crabs, and 201 images of swimming crabs are used to form a sample library. In order to ensure that the model of the present invention can be applied to different scenes, the present invention sets up multiple environments for swimming crabs. The descriptions of different scenes are shown in Table 1. The scene is described in detail, where 25 represents the number of swimming crabs, and 0-11 represents different scenes. The final sample library is expanded to 2532 images of swimming crabs (number of swimming crabs * number of images collected for each swimming crab). Among them, 1848 images are selected as training sets and 684 images are selected as test sets.

[0075] Table 1

[0076] serial number describe serial number describe 25_0 Standard scenario 25_6 Swimming crab rotates counterclockwise 25_1 Using a noisy camera 25_7 Capture the lower left range 25_2 Swimming crab rotates clockwise 25_8 Capture the upper right range 25_3 Using a low-resolution camera 25_9 Dark Light 25_4 Use a higher resolution camera 25_10 Overexposure 25_5 Lower the viewing angle 25_11 Lower the view angle while rotating

[0077] The present invention photographs individual swimming crabs under different lighting conditions to simulate a variety of actual scenes. Figure 1 The carapace images of the swimming crab with ID 25 in the crab-back-201 dataset in different scenes are shown. 25_0 is the carapace image of the swimming crab in the standard scene.

[0078] In the present invention, the training set includes 154 types of swimming crabs with a total of 1848 pictures, and the test set includes 57 types of swimming crabs with a total of 684 pictures, wherein the test set is composed of a query set and a gallery set. The query set is composed of all objects to be queried, and the gallery set stores the pictures of swimming crabs currently collected. The feature vector of the object to be queried in the query set obtained by PR (Pyramid-based ReID) and the feature vector of all objects in the gallery set are used to calculate the Euclidean distance between the feature vector of the query object and the feature vector of all objects in the gallery set, and the ID of the swimming crab in the gallery set with the closest distance is used as the ID of the query object, so as to realize the re-identification of the swimming crabs.

[0079] In terms of model training:

[0080] The optimizer uses Adam with a weight decay of 0.0005, and each batch uses 32 images. The total number of training rounds is 60, and the initial learning rate is set to 0.0003. After 30 rounds of training, the learning rate becomes 0.00003. In terms of loss, ID loss and triple loss are combined, where ID loss and triple loss account for 0.3 and 0.7 respectively. The weights used by Backbone are pre-trained with ImageNet. The input length and width of the image are (384, 128), and the data enhancement scheme uses horizontal flipping.

[0081] In terms of model reasoning:

[0082] The trained model is used in the inference stage. An image of a swimming crab is input as the query object, and the feature vector identifying the query object is obtained through (PR). The feature distance between the feature vector and the feature vectors of all swimming crabs in the database is calculated, and the ID of the swimming crab closest to the query object is returned. This ID is the ID of the query object, and the re-identification of the swimming crab is realized.

[0083] The experiments designed by the present invention verify the effectiveness of the pyramid-type block feature fusion strategy in terms of the structure, versatility and robustness of the pyramid. In terms of robustness, the experiments designed by the present invention all occlude the query set. In terms of the pyramid structure and versatility, the present invention designs a pyramid model strategy comparison experiment and an individual feature specificity experiment of the swimming crab, using Resnet18, Resnet50, and Resnet101 as the backbone of feature extraction, and experimentally verifies the presence or absence of a pyramid structure, the size of the pyramid structure, and the influence of the pyramid structure on the feature distance. Finally, a comparative experiment with other re-identification algorithms is designed to verify the effectiveness of the pyramid block feature fusion strategy.

[0084] The experiment of the present invention uses two evaluation indicators to measure the performance, namely, mean Average Precision (mAP) and Cumulative Match Characteristics (CMC) Rank-k precision. The metric function calculates the distance between the query image and the gallery sample, combined with the real three-tuberculate crab entity input.

[0085] Swimming crab ReID algorithm:

[0086] The traditional ReID algorithm takes the whole image of the object as input, extracts the overall features of the object, and uses the features as the identification information of the object. Based on this information, the similarity between the query object and other objects in the database can be calculated to achieve object matching (object re-identification). This type of method overemphasizes the global features of the object when extracting features, and ignores some insignificant detail features. Therefore, when there are local texture differences in the object, this type of method cannot express the difference, which affects the accuracy of object feature extraction. Therefore, more and more studies have comprehensively considered global features and local features, and improved the performance of re-identification by strengthening the contribution of key local areas to object features. This method for local feature extraction requires the image to be divided into blocks to strengthen the role of local features. For example, in the field of pedestrian re-identification, the pedestrian image is divided into three blocks: head, upper body, and lower body, and local features are extracted separately. In terms of local area division, different block division schemes are adopted to improve the performance of re-identification, but this method has high requirements for the degree of alignment of the image.

[0087] For the swimming crab, the surface of its carapace has features such as bumps, grooves, ridges and irregular spots, which have distinct individual local differences. Figure 2 This is a picture of the carapace of the swimming crab. The back texture and spots have strong individual differences. Therefore, the re-identification algorithm that emphasizes local texture and spot features can effectively improve the re-identification accuracy of the swimming crab.

[0088] Pyramid-based ReID Algorithm (PRA):

[0089] Aiming at the local features of the carapace of the swimming crab, the present invention designs a ReID scheme for the swimming crab based on pyramid-type partitioning, which can effectively avoid the "feature separation" problem caused by local area division and can more smoothly integrate local and global features. In addition, the present invention uses a combination of ID loss and triplet loss to jointly train the model to distinguish the differences in individual features of the swimming crab.

[0090] Block strategy:

[0091] In order to enhance the local detail features in the carapace of the swimming crab, it is necessary to divide the carapace into local areas. Figure 3 The present invention divides the swimming crab into six blocks according to the body function characteristics of the back of the swimming crab, and divides it into six areas, numbered 1-6. This division strategy also needs to consider two aspects: a) how to weigh the contribution of the six regional characteristics to the overall characteristics. b) The six regional blocks are independent of each other, how to solve the "feature separation" problem of the six independent areas.

[0092] In this regard, the present invention designs a pyramid-type block fusion strategy based on the block strategy, which can effectively train the feature contribution of the six blocks and can also deeply fuse the features of the six blocks.

[0093] Pyramid-based ReID

[0094] The pyramid model can realize the fusion of local features based on the block strategy. Figure 2 The six-block results of the present invention on the swimming crab are shown in FIG. On this basis, the present invention combines the six blocks into 18 continuous and non-repetitive block groups. Figure 3 It is a combination of 18 block groups, among which the (1, 2, 4, 5) block group contains the continuous information of blocks 1, 2, 4, and 5, which is the fusion information of 4 local areas. The pyramid structure composed of 18 block groups can completely cover the global and local features of the swimming crab. This method can not only extract global features, but also comprehensively extract key local features, and can also fuse local features, integrate and enhance feature information, and improve the accuracy of the model.

[0095] Figure 5 The deep learning model architecture proposed in the present invention is mainly composed of a feature extraction backbone network, a pyramid block structure, and a basic convolution block structure. The following sections introduce each structure.

[0096] Backbone:

[0097] Backbone is a network for extracting features. Its main function is to extract the original features in the image for use by the subsequent Pyramid. The present invention uses this module for preliminary feature extraction of the image of the swimming crab. Backbone networks such as ResNet-50 and ResNet-101 can be selected as the basic network.

[0098] Pyramid:

[0099] In order to allow the network to extract the global and local features of the swimming crab, the present invention uses the Pyramid strategy to process the features obtained by the backbone. Figure 4 As shown in the Pyramid dotted box in the figure, Pyramid takes the features extracted by backbone as input, and divides the local area features according to the pyramid-like blocking strategy based on the original features extracted by backbone, forming 18 local area features that can characterize the three-spotted swimming crab, realizing the preliminary grouping and fusion of local area features.

[0100] Basic Block:

[0101] The Basic Block structure used in the present invention is as follows Figure 5 As shown, it includes pooling layer, convolution layer, BN, ReLU, FN and softmax layer. Basic Block is the processing of 18 local area features after preliminary grouping and fusion. After BasicBlock processing, the local area features can better characterize the image texture and morphology of the three-spotted swimming crab to enhance the identification effect of the local area features. In order to enhance the difference of local area features, the present invention adds triplet loss on the basis of classification loss, and uses a multi-loss training mode to optimize the training parameters. In the inference stage, the Features in Block0-17 are merged to obtain the feature vector used to identify the three-spotted swimming crab, as shown in Figure 6 shown.

[0102] Triplet Loss

[0103] When using Triplet Loss for parameter training, the present invention selects a swimming crab as an anchor (Anchor), selects a sample with the same ID as the anchor and the lowest feature similarity as a positive sample (Positive), and selects a sample with a different ID from the anchor and the highest feature similarity as a negative sample (Negative), thereby forming a triplet (anchor, positive sample and negative sample). Figure 7 The selection process of triplet tuples is shown.

[0104] definition For the i-th swimming crab with ID j, Represents a triplet, and the combination of an anchor point and a positive sample is called a positive sample pair, that is, The combination of an anchor point and a negative sample is called a negative sample pair, i.e. The purpose of Triplet Loss is to promote compactness within the same ID and separability between different IDs. Improving the similarity of positive sample pairs can promote compactness within the same ID, and reducing the similarity of negative sample pairs can promote separability between different IDs. Triplet Loss can be expressed as:

[0105]

[0106] Where f(·) represents the feature extraction operation, are the feature vectors of the anchor point with ID i, the positive sample with ID i, and the negative sample with ID k, respectively. α is a hyperparameter, and its value here is 0.2.

[0107] result:

[0108] Table 2 shows the effectiveness of the pyramid block fusion strategy under different backbones (Resnet18, Resmet50, Resnet101), where “√” indicates the use of a pyramid structure. The test indicators use mAP, rank1, and rank5. The higher the indicator, the better.

[0109] Table 3 tests the performance of the models with different block sizes. The test indicators are mAP, rank1, and rank5. The higher the indicator, the better. Figure 8 The performance of the pyramid in distinguishing the carapace features of the swimming crab under different backbones (Resnet18, Resmet50, Resnet101) is intuitively demonstrated, and the scattered points reflect the differences in the feature distances of different swimming crabs.

[0110] Table 4 shows the performance comparison of the algorithm of the present invention with those based on Vgg16, Resnet50, and PCB. PCB-2 indicates that PCB is divided into two horizontal blocks, and Ours uses Resnet50 as the backbone.

[0111] Table 2

[0112]

[0113]

[0114] Table 3

[0115]

[0116] Table 4

[0117]

[0118] Pyramid model strategy comparison:

[0119] Pyramid vs. non-pyramid strategy:

[0120] The purpose of the present invention is to verify the effectiveness of the pyramid structure and its fusion strategy. To this end, the present invention adopts three backbones (Resnet18, Resnet50, Resnet101) as the original feature extraction algorithm of the image, and divides the comparison algorithm into Resnet18, Resnet18-pyramid, Resnet18, Resnet50-pyramid, Resnet50, Resnet101-pyramid6 groups of experiments according to whether the pyramid strategy is adopted. In order to fully verify the performance of each algorithm, the present invention blocks the query set and only retains the right half of the original image. The experimental results of each algorithm are shown in Table 2. The algorithms Resnet18-pyramid, Resnet50-pyramid, and Resnet101-pyramid, which adopt the pyramid block fusion strategy, have significantly improved mAP, rangk1, and rangk5 compared to Resnet18, Resnet50, and Resnet101, which do not adopt the pyramid block fusion strategy.

[0121] Resnet18, Resnet50, and Resnet101 models mainly perform re-identification based on the global features of the swimming crab. When the query set is occluded, a large global feature deviation will occur, resulting in a decrease in mAP, rangk1, and rangk5. The pyramid block fusion strategy proposed in the present invention uses a method of extracting global and local features to avoid the impact of using only global features on the final detection performance. The present invention illustrates that the use of the pyramid block fusion strategy can improve the performance of the swimming crab re-identification, and using Resnet101 as the backbone has the best effect.

[0122] Comparison of block solutions:

[0123] In order to verify the optimal block scheme of the pyramid strategy, the present invention uses 2×3 and 2×4 as block schemes to compare the experimental effects of each pyramid block fusion strategy. The present invention uses 3 backbones (Resnet18, Resnet50, Resnet101) as the original feature extraction algorithm of the image, and the experimental results obtained by using 2×3 and 2×4 as block schemes are shown in Table 3. Among them, the model parameters of the pyramid number of 2×4 are larger than those of the pyramid number of 2×3. In terms of accuracy indicators mAP, rank1%, rank5%, the pyramid number of 2×3 also has an advantage. Due to the 2×4 block scheme, the features are divided more finely, resulting in the destruction of the continuity of local features, thereby affecting the accuracy of local features and increasing the burden of model training. Therefore, the optimal block scheme of the pyramid strategy is 2×3.

[0124] Comparison of the specific characteristics of individual swimming crabs:

[0125] In order to verify the universality of the pyramid fusion strategy, the present invention uses different backbone networks combined with the pyramid block fusion strategy to analyze the characteristic distances between the three-tuberculate swimming crabs and verify the individual characteristic differences of the three-tuberculate swimming crabs. To this end, the present invention calculates the characteristic distances of samples with the same ID and the characteristic distances of samples with different IDs for each three-tuberculate swimming crab. Figure 8 The characteristic distance distribution of the same ID and different IDs of the three-tuberculate swimming crabs under the resnet18, resnet50, resnet101 and pyramid block fusion strategy is shown. The horizontal axis represents the three-tuberculate swimming crabs, the vertical axis represents the distance between the three-tuberculate swimming crabs, the dark blue scatter points represent the characteristic distance of the same ID samples, and the light orange represents the characteristic distance of the different ID samples. The dark blue scatter points are mainly distributed at the bottom of the figure, indicating that the distance between the samples with the same ID is close, while the scatter points of the samples with different IDs are mainly concentrated in the middle and upper part of the figure, and the fluctuation is large, indicating that the distance between the samples with different IDs is large.

[0126] from Figure 8 It can be seen that after the feature vectors of the three-tuberculate swimming crab extracted by each backbone are fused by the pyramid block strategy, the feature distances between samples with the same ID and the feature distances between samples with different IDs are significantly different. This shows that the pyramid block fusion strategy can distinguish the feature differences between different three-tuberculate swimming crabs and reflect the individual feature specificity of the three-tuberculate swimming crabs.

[0127] Comparison and verification:

[0128] The present invention verifies the comparison between the pyramid block feature fusion strategy and other algorithms, and the comparison results are shown in Table 4. In the present invention, the query set is occluded, and the left half and the upper half are occluded respectively. The missing of some feature information of the swimming crab and the interference of uncertain factors are simulated by occlusion. When Vgg16 and Resnet50 are used as backbone, the global features extracted by the backbone are used for classification training in the training stage, and the global features of the swimming crab are extracted by the backbone in the inference stage as the identification information of each swimming crab, and the Euclidean distance between the global features is compared to determine the swimming crab ID. PCB adopts a horizontal two-block scheme for training and inference. The inference scheme is the same as the scheme using Vgg16 and Resnet50 as backbone. Ours is a scheme using Resnet50 as backbone and the pyramid block feature fusion strategy as the head. As can be seen from Table 5, the mAp values ​​of Vgg16 and Resnet50 solutions that only consider global features are much lower than those of PCB and Ours that adopt the block solution. Therefore, it is difficult to complete the re-identification of the three-tuberculate swimming crab for the solution that only considers global features in the case of missing information. The pyramid block feature fusion strategy adopted by the present invention has an mAP value of 9.6% higher than that of PCB when the upper half is occluded, a rank1 value of 26.3% higher, and a rank5 value of 7% higher. Therefore, the pyramid block feature fusion strategy proposed in the present invention has a certain anti-interference ability in the case of missing information.

[0129] The present invention designs a pyramid block feature fusion strategy for the swimming crab. This strategy groups and fuses the original features obtained through the backbone to obtain all local and global information of the swimming crab, avoiding the impact of using only global features on the final detection performance. In addition, the present invention uses a combination of classification loss and triplet loss, and the triplet loss can promote the compactness within the same ID and the separability between different IDs. Experimental results show that the present invention has better performance when using resnet50 and the partitioning scheme is 2×3, and can still maintain a high ReID accuracy rate when the carapace image is incomplete.

[0130] Swimming crab is an important aquaculture species in coastal areas of my country, with high nutritional value and economic value. In order to ensure the food safety of swimming crab, it is necessary to establish a perfect traceability mechanism. With the development of artificial intelligence and deep learning, the re-identification (ReID) technology can be used to extract the carapace texture features of swimming crab, and use the features as the "biological fingerprint" of swimming crab to achieve the traceability of swimming crab. In this regard, the present invention designs a re-identification model of swimming crab based on pyramid block feature fusion, which can extract global features and local detail features at the same time. On this basis, according to the importance of each local feature, a weight adaptive integration strategy is established, which can improve the re-identification accuracy of individuals with incomplete carapaces. In addition, in order to enhance the difference of the features of swimming crab, the present invention adopts a triple loss function for difficult sample mining, which can effectively reduce the ReID misrecognition rate of swimming crab. The present invention uses 211 swimming crabs sampled in a real breeding environment as a data set. When the left half of the carapace is blocked, the mAP and Rank-1 values ​​reach 93.2% and 93% respectively. When the upper half of the carapace is blocked, the mAP and Rank-1 values ​​reach 71.8% and 75.4% respectively, which fully verifies the feasibility and effectiveness of the algorithm.

[0131] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

Claims

1. A method for re-identifying swimming crabs based on pyramid feature fusion, characterized in that: include: Constructing a deep learning model, wherein the deep learning model is used to decompose the carapace image of the swimming crab into a pyramid-like block structure, and jointly train according to a combination of ID loss and triplet loss to generate a swimming crab re-identification model for tracing the swimming crab; Collecting the carapace image of the swimming crab, constructing a training set, training the deep learning model, and generating the swimming crab re-identification model, wherein the swimming crab re-identification model is used to trace the swimming crab by identifying the carapace image of the swimming crab; In the process of constructing a deep learning model, the deep learning model is composed of a backbone module, a pyramid module, a basic block module, and a loss module connected in sequence; The backbone module is used to extract the original features of the carapace image of the swimming crab trituberculate, wherein the original features include protrusions, grooves, ridges and irregular spots; The Pyramid module is used to decompose the carapace image of the swimming crab into the pyramid block structure according to the original features extracted by the backbone module through a pyramid block strategy; the Basic Block module is used to enhance the identification effect of each local area feature of the pyramid block structure; The Loss module is used to optimize the training of each enhanced local area feature through a combined loss training mode of ID loss and triplet loss, and to perform feature fusion in the inference stage to generate a feature vector for identifying the swimming crab three-tuberculates.

2. The method for re-identifying Swimming Crab trituberculate based on pyramid feature fusion according to claim 1, characterized in that: In the process of constructing the training set, each of the carapace images of the swimming crab trituberculate is marked with a first ID number; Based on each of the first ID numbers, a carapace image of the swimming crab collected in different scenes is generated, and a second ID number is added after the first ID number to generate an ID number of the carapace image of the swimming crab.

3. The method for re-identifying Swimming Crab trituberculate based on pyramid feature fusion according to claim 2, characterized in that: In the process of generating a feature vector through the Loss module, the carapace image of the swimming crab trituberculate having the pyramid-shaped block structure is used as an anchor point, and the first ID number of the anchor point is obtained; Based on the same first ID number, a sample with the lowest feature similarity with the anchor point is obtained as a positive sample, wherein the Euclidean distance between two feature vectors is obtained as the feature similarity; Based on different first ID numbers, obtaining a sample with the highest feature similarity to the anchor point as a negative sample; Constructing a triplet according to the anchor point, the positive sample, and the negative sample; According to the triplet, obtaining the triplet loss; According to the ID number, the ID loss is obtained.

4. The method for re-identifying Swimming Crab trituberculate based on pyramid feature fusion according to claim 3, characterized in that: In the process of obtaining the triplet loss, the triplet loss is expressed as: Where f(·) represents the feature extraction operation, They are the feature vectors of the anchor point with the first ID number i, the positive sample with the first ID number i, and the negative sample with the first ID number k, respectively, and α is 0.

2.

5. The method for re-identifying Swimming Crab trituberculate based on pyramid feature fusion according to claim 4, characterized in that: In the process of decomposing the carapace image of the swimming crab into a pyramid-like block structure, the carapace image of the swimming crab is divided into several functional areas based on the body functional characteristics of the back of the swimming crab, and according to the connection position relationship between each functional area, the pyramid-like block structure that completely covers the global features and local features of the swimming crab is generated, wherein the global features include protrusions, grooves, ridges and irregular spots, and the local features include local texture and / or spot features.

6. The method for re-identifying Swimming Crab trituberculate based on pyramid feature fusion according to claim 5, characterized in that: In the process of generating a pyramid block structure, the pyramid block structure is composed of a first block map and a second block map, wherein the first block map includes all the functional areas, the second block map includes at least one functional area, and the number of the functional areas in the second block map is less than that in the first block map.

7. A re-identification system for Portunus trituberculatus based on pyramid feature fusion, characterized in that: The system is used to implement a method for re-identifying the swimming crab trituberculate based on pyramid feature fusion as described in any one of claims 1 to 6, and the system comprises: A data acquisition module, used for collecting carapace images of the swimming crab three tubercles; A data recognition module is used to identify the feature vector of the carapace image of the swimming crab by constructing a re-identification model of the swimming crab, and trace the origin of the swimming crab, wherein the feature vector includes global features and local features, the global features include protrusions, grooves, ridges and irregular spots, and the local features include local texture and / or spot features.

Images