Ear brushing identity recognition method based on pseudo-label semi-supervised learning

Abstract

The application belongs to the field of identity recognition, and discloses an ear brushing identity recognition method based on pseudo-label semi-supervised learning, which comprises the following steps: 1, constructing a data set for training and labeling the data set to obtain a labeled training set, an unlabeled training set and a labeled verification set; S2, determining a convolutional neural network model to be used, and modifying the training process and the detection process of the convolutional neural network model according to an improved algorithm and an identification principle; S3, training the convolutional neural network model by using an improved MeanTeacher algorithm to obtain a trained neural network model; S4, calculating a human ear image to be identified by using the trained neural network model to obtain a feature vector; and S5, matching the feature vector obtained in S4 with a human ear feature vector that has been recorded in a database and needs to be used for identification by using a nearest neighbor search algorithm to realize end-to-end human ear identity recognition. The application significantly improves the detection performance of small targets and tail categories.

Description

Technical Field

[0001] This invention relates to the field of identity recognition, and more particularly to an ear-scanning identity recognition method based on pseudo-label semi-supervised learning. Background Technology

[0002] With the continuous advancement of deep learning technology, facial and ear biometrics have become a key component of modern security systems. Facial recognition technology, benefiting from the salient nature of its targets and abundant datasets, has been widely applied and is becoming increasingly mature in multiple fields. However, ear recognition, as an emerging field of biometric identification, demonstrates enormous application potential and theoretical value due to its unique physiological structure and ease of data acquisition.

[0003] Traditional ear recognition methods, such as the twelve-point measurement method, rely heavily on manual identification, which is not only inefficient but also susceptible to subjectivity. With the introduction of neural network algorithms in the latter half of the 20th century, ear recognition technology began to evolve towards automation and intelligence. In the 21st century, researchers have significantly improved the accuracy and robustness of recognition through techniques such as Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), and Support Vector Machines. While traditional machine learning methods have made significant progress in ear recognition, the rapid development of deep learning technology has led researchers to explore its application potential in ear recognition. This transformative technology has greatly boosted recognition accuracy and efficiency. The introduction of deep learning, especially convolutional neural networks (CNNs), has provided powerful feature extraction capabilities for automatic recognition, reducing reliance on manual feature engineering. Similarly, deep learning has been applied to ear recognition. By designing network architectures suitable for human ear characteristics, such as increasing network depth and employing multi-scale feature fusion, deep learning models have outperformed traditional methods on public datasets, demonstrating their superiority in ear recognition tasks.

[0004] Research on ear recognition technology started relatively late and on a small scale. Currently, many ear detection methods are actually inspired by facial detection techniques. The main challenges facing ear recognition include:

[0005] (1) For the model: The ear appears tiny in the original image, and a single ear is far smaller than the face in terms of both volume and surface area. Furthermore, mainstream object detection network architectures often include one or more image downsampling steps, which can adversely affect the performance and robustness of the ear detector.

[0006] (2) Regarding data: Relevant training data resources are relatively scarce, and a standardized ear database like the facial database has not yet been established. Deep learning relies heavily on a large number of training samples when training ear detection models. Therefore, when the number of available samples is limited, the model is prone to overfitting. Overfitting occurs when the model performs well on the training data but its predictions on the validation or test datasets differ significantly from expectations. This indicates that the model has overfitted the training data, resulting in a decrease in its generalization ability and making it difficult to apply to data outside the training data. Summary of the Invention

[0007] The purpose of this invention is to disclose an ear-scanning identity recognition method based on pseudo-label semi-supervised learning, thereby solving the technical problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] This invention provides a method for ear-scanning identification based on pseudo-label semi-supervised learning, including:

[0010] S1, construct the dataset for training and label the dataset to obtain a labeled training set, an unlabeled training set, and a labeled validation set;

[0011] S2, determine the convolutional neural network model to be used, and modify its training and detection processes according to the improved algorithm and recognition principle;

[0012] S3, the improved MeanTeacher algorithm is used to train the convolutional neural network model to obtain a trained neural network model;

[0013] S4. Use the trained neural network model to calculate the feature vector of the human ear image to be recognized.

[0014] S5 uses the nearest neighbor search algorithm to match the feature vectors obtained in S4 with the human ear feature vectors that have been entered into the database for identification, thereby achieving end-to-end human ear identification.

[0015] Preferably, S1 includes:

[0016] The dataset is constructed by searching publicly available datasets on the Internet and then fusing small, medium and large ear images in a preset ratio (9:4:1).

[0017] Through manual review, erroneous images, duplicate images, and images that are too difficult to identify in the constructed dataset are removed; erroneous images include images that contain ears, but the ears are not human ears;

[0018] The processed dataset is divided into three parts: a labeled training set, an unlabeled training set, and a labeled validation set; the number of images in the labeled training set is 1% of the total number of ear images in the dataset.

[0019] LabelImg software was used to annotate the labeled training set and the labeled validation set. The ear region in the image was outlined using a bounding box, and the outlined region was labeled with the word "ear".

[0020] Preferably, images that are too difficult to identify are those that do not contain an ear outline.

[0021] Preferably, the convolutional neural network model includes the Faster-RCNN model.

[0022] Preferably, S3 includes:

[0023] In the first iteration, the following calculations are performed:

[0024] Images are extracted from the labeled and unlabeled training sets at a ratio of 1:4 as the current training batch. Let b represent the number of labeled images, then the number of unlabeled images is 4b.

[0025] Weak enhancement processing is performed on the unlabeled image to obtain a weakly enhanced image;

[0026] The teacher model is used to predict weakly enhanced images to obtain pseudo-labels for the weakly enhanced images, thus obtaining pseudo-labeled images; low-confidence pseudo-labeled images are filtered out to generate a pseudo-labeled dataset; the pseudo-labeled dataset is stored in a cache library; low confidence refers to a confidence level of less than 0.85;

[0027] Starting from the second iteration, the following operations are performed:

[0028] Images are extracted from the labeled and unlabeled training sets at a ratio of 1:4 as the current training batch. Let b represent the number of labeled images, then the number of unlabeled images is 4b.

[0029] Weak enhancement processing is performed on the unlabeled image to obtain a weakly enhanced image;

[0030] The teacher model is used to predict weakly enhanced images to obtain pseudo-labels for the weakly enhanced images, thus obtaining pseudo-labeled images; low-confidence pseudo-labeled images are filtered out to generate a pseudo-labeled dataset; let A represent the pseudo-labeled dataset obtained in the current iteration, and store A in the cache; low confidence refers to a confidence level less than 0.85;

[0031] Multiple images are extracted from the cache to form a set B. Set B does not contain any images from set A, and the number of images in set B is the same as the number of images in set A.

[0032] The images in set A and set B are mixed using Mixup to obtain the first image set;

[0033] The images from set A and those extracted from set B are processed using Mosaic to obtain a second set of images; the number of images extracted from set B is the same as the number of images from set A.

[0034] The images in the first and second image sets are subjected to strong enhancement processing respectively to obtain the third image set;

[0035] Labeled supervised augmentation processing is performed on the labeled image to obtain the augmented image;

[0036] The augmented images and the third set of images are mixed and used as input data for the student model;

[0037] By monitoring loss L supervised Unsupervised loss L unsupervised Optimize the student model;

[0038] The teacher model is updated using an exponential moving average strategy.

[0039] Preferably, the weak enhancement process includes:

[0040] The first step is to load the image to be weakly enhanced;

[0041] The second step is to randomly scale the image from the first step;

[0042] The third step is to randomly flip the image obtained in the second step.

[0043] Preferably, the process of enhanced label supervision includes:

[0044] The first step is to load the image and its corresponding annotation information;

[0045] The second step is to randomly scale the image from the first step;

[0046] The third step is to randomly flip the image obtained in the second step;

[0047] The fourth step is to apply a random enhancement strategy based on color space to the image obtained in the third step;

[0048] The fifth step is to filter out target boxes that are too small and the images in which the target boxes are located. Target boxes that are too small are those in images with a size smaller than 80×80.

[0049] Preferably, the enhancement process includes:

[0050] The first step is to load the image to be weakly enhanced;

[0051] The second step is to randomly scale the image from the first step;

[0052] The third step is to randomly flip the image obtained in the second step;

[0053] The fourth step is to apply a random enhancement strategy based on color space to the image obtained in the third step;

[0054] Fifth, apply a geometric-based random augmentation strategy to the image obtained in the fourth step;

[0055] The sixth step is to apply a random enhancement strategy based on color space to the image obtained in the fifth step;

[0056] Step 7: Apply a geometric-based random augmentation strategy to the image obtained in step 6.

[0057] Step 8: Filter out target boxes that are too small and the images in which the target boxes are located. Target boxes that are too small are target boxes in images with a size of less than 80×80.

[0058] Preferably, for supervised learning, a smooth L1 loss function is used:

[0059]

[0060] L1(x,y) represents the smoothing loss function, where x and y represent the predicted and actual values ​​of the neural network model, respectively.

[0061] For unsupervised learning, use consistency regularization loss:

[0062]

[0063] N represents the number of samples in each batch; L consistency f(x) represents the consistency regularization loss function. i g(x) represents the output of the student model on the i-th image; i ) represents the output result of the teacher model's calculation on the i-th image;

[0064] In each training iteration, the parameters of the student model are updated by minimizing the total loss function, which is a weighted sum of the two losses, L. total Let L be the total loss function. supervised To monitor the loss, L unsupervised For unsupervised loss, the weight w u Used to balance the contributions of the two:

[0065] L total =L supervised +w u *L unsupervised .

[0066] Preferably, S2 includes:

[0067] The required convolutional neural network model is Faster R-CNN.

[0068] Modify the FPN and global average pooling layers in the neck part of the Faster-RCNN model to enable the Faster-RCNN model to output the feature vector of the ear in the human ear image to be identified, and then train the modified model.

[0069] The ear image to be identified is input into the trained model to obtain the feature vector of the ear in the ear image output by the trained model.

[0070] The global average pooling operation is calculated using the following formula:

[0071]

[0072] Among them, G c F represents the output value of the c-th channel after global average pooling; i,j,c Let G represent the pixel value at position (i,j) and channel c in the original feature map. After global average pooling, each channel c has its original feature map simplified to a single value G. c Ultimately, the global average of all channels will form a new feature vector G = (G1, G2, ..., G...). c The feature map has dimensions of 1×1×c. This compact feature vector can be directly connected to the classification layer for final class prediction. H and W represent the height and width of the feature map, respectively.

[0073] Beneficial effects:

[0074] This invention significantly improves the detection performance of small targets and tail categories by effectively utilizing unlabeled data to enhance model training. It employs a Faster R-CNN model combined with Feature Pyramid Network (FPN) and Global Average Pooling (GAP) techniques to achieve an end-to-end human ear identification process. The innovation lies in improving the Mean Teacher algorithm and introducing two data augmentation techniques: pixel-level mixing of pseudo-labels (Mixup) and image stitching of pseudo-labels (Mosaic). The former mitigates the negative impact of missed targets, while the latter enhances the model's ability to identify small targets by increasing the number of labels for small-scale targets. These techniques, combined with the improved algorithm, collectively improve the performance of deep convolutional neural networks on semi-supervised target detection tasks.

[0075] This invention selected Faster R-CNN as the optimal model by conducting all experiments on a high-performance server and comparing it with baselines of different models, and then made targeted improvements and optimizations to it. Although the model's generalization ability and robustness need further improvement, this research provides new ideas for human ear identification technology and has important guiding significance for algorithm improvement and application expansion. Furthermore, the algorithm framework proposed in this invention is not only applicable to human ear identification but can also be extended to other small object detection tasks, showing broad application prospects and practical value. Attached Figure Description

[0076] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a schematic diagram of the ear-scanning identity recognition method based on pseudo-label semi-supervised learning according to the present invention.

[0078] Figure 2 This is a schematic diagram of the MeanTeacher semi-supervised learning method.

[0079] Figure 3 This is a schematic diagram of the algorithm framework based on MeanTeacher.

[0080] Figure 4 This is a schematic diagram of the identity recognition algorithm process.

[0081] Figure 5 This is a partial illustration of the EarVN1.0 dataset.

[0082] Figure 6 This is a partial illustration of the USTB Ear dataset.

[0083] Figure 7 This is a partial illustration of the Kaggle Scattered Dataset.

[0084] Figure 8 For different data enhancement processes.

[0085] Figure 9 This is a schematic diagram of image aliasing in Mixup.

[0086] Figure 10 This is a schematic diagram of Mosaic splicing.

[0087] Figure 11a This is a schematic diagram of the total loss function.

[0088] Figure 11b This is a diagram illustrating the change in the learning rate.

[0089] Figure 11c This is a schematic diagram of the average precision of the training set.

[0090] Figure 11d This is a schematic diagram of the average accuracy of the test set. Detailed Implementation

[0091] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0092] To overcome the aforementioned challenges, increasing the amount of training data becomes a key solution. This study explores semi-supervised learning methods to alleviate the shortage of high-quality labeled data, thereby improving the performance and generalization ability of ear detection models. By employing semi-supervised learning, models can be trained on a large amount of unlabeled data, effectively enhancing the model's accuracy in recognizing new samples, even with limited labeled data.

[0093] like Figure 1 This invention provides a method for ear-scanning identity recognition based on pseudo-label semi-supervised learning, including:

[0094] S1, construct the dataset for training and label the dataset to obtain a labeled training set, an unlabeled training set, and a labeled validation set;

[0095] S2, determine the convolutional neural network model to be used, and modify its training and detection processes according to the improved algorithm and recognition principle;

[0096] S3, the improved MeanTeacher algorithm is used to train the convolutional neural network model to obtain a trained neural network model;

[0097] S4. Use the trained neural network model to calculate the feature vector of the human ear image to be recognized.

[0098] S5 uses the nearest neighbor search algorithm to match the feature vectors obtained in S4 with the human ear feature vectors that have been entered into the database for identification, thereby achieving end-to-end human ear identification.

[0099] Preferably, S1 includes:

[0100] The dataset is constructed by retrieving publicly available datasets from the internet and then merging small, medium, and large ear images in a preset ratio (e.g., 9:4:1).

[0101] For example: if the number of pixels horizontally or vertically is less than or equal to 512, it is a small size; if it is greater than 512 and less than or equal to 1280, it is a medium size.

[0102] All sizes greater than 1280 are considered large.

[0103] Through manual review, erroneous images, duplicate images, and images that are too difficult to identify in the constructed dataset are removed; erroneous images include images that contain ears, but the ears are not human ears;

[0104] The processed dataset is divided into three parts: a labeled training set, an unlabeled training set, and a labeled validation set; the number of images in the labeled training set is 1% of the total number of ear images in the dataset.

[0105] LabelImg software was used to annotate the labeled training set and the labeled validation set. The ear region in the image was outlined using a bounding box, and the outlined region was labeled with the word "ear".

[0106] The sources of each piece of data used are detailed below:

[0107] EarVN1.0: This portion constitutes the largest proportion of the dataset and was constructed from ear images of 164 Asian individuals collected by the Open University of Ho Chi Minh City, Vietnam in 2018. It includes a total of 28,412 color images, comprising 98 males and 66 females. Its unique value lies in the fact that the image acquisition phase utilized a camera system to record facial information of individuals under various lighting conditions, encompassing rich variations in pose, scale, and illumination. A partial illustration of the EarVN1.0 dataset is shown below. Figure 5 .

[0108] USTB Ear: This dataset, taken by a team from the University of Notre Dame, contains 2D and 3D ear images. After filtering out the 3D portions and removing duplicate images due to differences in orientation alone, a total of 308 color ear images from 77 faculty and students were compiled. A partial illustration of the USTTB Ear dataset is shown below. Figure 6 .

[0109] Kaggle Scattered Datasets: Many competition-related publicly available ear datasets exist on the Kaggle platform. While some datasets are used for ear detection or ear acupoint recognition, their actual source and content are difficult to verify due to the lack of detailed descriptions provided by some uploaders. After self-collection and organization, and removal of materials inconsistent with research needs, nearly six thousand color and black-and-white ear images have been compiled. Some illustrations of the Kaggle Scattered Datasets are shown below. Figure 7 .

[0110] The self-constructed dataset lacked annotation information, and even if it did, it wasn't ear-related annotation information, requiring data processing. Through manual review, erroneous images, duplicate images, and images with excessive difficulty in identification were removed from the self-constructed dataset.

[0111] The resulting comprehensive dataset contains 33,591 ear images, comprehensively covering small, medium, and large sizes, as well as different scenarios where the ear occupies a small or large area in the image. Here, "size" refers to the overall size of the image, and "target size" specifically refers to the relative area proportion of the ear in the image.

[0112] A script was written to divide the dataset into three parts: a labeled training set, an unlabeled training set, and a labeled validation set. This ensured a balanced distribution of image size and target size across categories, with 455, 30,270, and 2,866 images respectively. Thus, the labeled training samples comprised approximately 1% of the total dataset. Subsequently, the LabelImg software was used to annotate the data in the labeled training and validation sets.

[0113] Preferably, images that are too difficult to identify are those that do not contain an ear outline.

[0114] Preferably, the convolutional neural network model includes the Faster-RCNN model.

[0115] Preferably, S3 includes:

[0116] In the first iteration, the following calculations are performed:

[0117] Images are extracted from the labeled and unlabeled training sets at a ratio of 1:4 as the current training batch. Let b represent the number of labeled images, then the number of unlabeled images is 4b.

[0118] Weak enhancement processing is performed on the unlabeled image to obtain a weakly enhanced image;

[0119] The teacher model is used to predict weakly enhanced images to obtain pseudo-labels for the weakly enhanced images, thus obtaining pseudo-labeled images; low-confidence pseudo-labeled images are filtered out to generate a pseudo-labeled dataset; the pseudo-labeled dataset is stored in a cache library; low confidence refers to a confidence level of less than 0.85;

[0120] Starting from the second iteration, the following operations are performed:

[0121] Images are extracted from the labeled and unlabeled training sets at a ratio of 1:4 as the current training batch. Let b represent the number of labeled images, then the number of unlabeled images is 4b.

[0122] Weak enhancement processing is performed on the unlabeled image to obtain a weakly enhanced image;

[0123] The teacher model is used to predict weakly enhanced images to obtain pseudo-labels for the weakly enhanced images, thus obtaining pseudo-labeled images; low-confidence pseudo-labeled images are filtered out to generate a pseudo-labeled dataset; let A represent the pseudo-labeled dataset obtained in the current iteration, and store A in the cache; low confidence refers to a confidence level less than 0.85;

[0124] Multiple images are extracted from the cache to form a set B. Set B does not contain any images from set A, and the number of images in set B is the same as the number of images in set A.

[0125] The images in set A and set B are mixed using Mixup to obtain the first image set;

[0126] The images from set A and those extracted from set B are processed using Mosaic to obtain a second set of images; the number of images extracted from set B is the same as the number of images from set A.

[0127] The images in the first and second image sets are subjected to strong enhancement processing respectively to obtain the third image set;

[0128] Labeled supervised augmentation processing is performed on the labeled image to obtain the augmented image;

[0129] The augmented images and the third set of images are mixed and used as input data for the student model;

[0130] By monitoring loss L supervised Unsupervised loss L unsupervised Optimize the student model;

[0131] The teacher model is updated using an exponential moving average strategy.

[0132] Meanteacher is a well-known semi-supervised learning algorithm, such as... Figure 2 As shown, MeanTeacher consists of two models: a Teacher Model and a Student Model. These two models start with the same weights and have the same structure.

[0133] In each training iteration, the system randomly samples labeled and unlabeled images to form a batch of data. For each unlabeled image, two views are generated through weak augmentation and strong augmentation.

[0134] The teacher model predicts on weakly augmented unlabeled images, generating pseudo-labels. Low-confidence pseudo-labels are filtered out using a confidence threshold, and the remaining pseudo-labels are transferred to strongly augmented views to form a pseudo-labeled dataset.

[0135] The student model is updated by optimizing the supervised loss for labeled data and the unsupervised loss for pseudo-labeled data. The teacher model's weights are frozen, but as the student model changes, the teacher model must also improve. Once the student model surpasses the teacher model, the teacher model can update its parameters using an exponential moving average (EMA) based on the student model's weights. In other words, the knowledge learned by the student model can be transferred to the teacher model, and only then will the teacher model's parameters change.

[0136] The hybrid pseudo-label algorithm proposed in this invention is based on and improved upon MeanTeacher, and also adopts a teacher-student model architecture. This method aims to improve the quality and quantity of pseudo-labels by integrating multiple data augmentation techniques, thereby enhancing the model's performance in semi-supervised object detection tasks.

[0137] For the overall process, for example, if an iteration includes one labeled image and four unlabeled images, the algorithm also selects four additional pseudo-labeled images from the cache, performs pseudo-label mixup processing to obtain four mixed images; then, it randomly selects eight of these images (including enhanced versions of the original four unlabeled images) and uses pseudo-label masais techniques to generate a new mixed image. Finally, the five images generated by this series of operations replace the original four unlabeled images as the training input for the student model.

[0138] Through this system strategy, the proposed algorithm effectively improves the performance of various target detectors (including one-stage, two-stage, end-to-end, etc.) in a semi-supervised environment, especially for small target detection.

[0139] This invention enhances the model's generalization ability by increasing the amount of data. To ensure the proposed solution has high feasibility, a data augmentation strategy is employed to enrich the data sample. Considering space efficiency, data augmentation is applied only during actual runtime and does not require static storage. To further improve the reproducibility of the experiment, a randomization mechanism is introduced, specifically including the definition of two key modules: color_space and Geometric.

[0140] The color_space function encompasses a variety of color transformation operations, such as automatic contrast, equalization, sharpening, tone separation, sunlight effect, color adjustment, contrast adjustment, and brightness adjustment.

[0141] Geometric encompasses a series of geometric transformation operations, including rotation, X-axis shearing, Y-axis shearing, X-axis translation, and Y-axis translation.

[0142] In actual operation, depending on whether the data is labeled, this study designed different data augmentation processes for different data types, such as... Figure 8 As shown.

[0143] Preferably, the weak enhancement process includes:

[0144] The first step is to load the image to be weakly enhanced;

[0145] The second step is to randomly scale the image from the first step;

[0146] The third step is to randomly flip the image obtained in the second step.

[0147] Preferably, the process of enhanced label supervision includes:

[0148] The first step is to load the image and its corresponding annotation information;

[0149] The second step is to randomly scale the image from the first step;

[0150] The third step is to randomly flip the image obtained in the second step;

[0151] The fourth step is to apply a random enhancement strategy based on color space to the image obtained in the third step;

[0152] The fifth step is to filter out target boxes that are too small and the images in which the target boxes are located. Target boxes that are too small are those in images with a size smaller than 80×80.

[0153] Preferably, the enhancement process includes:

[0154] The first step is to load the image to be weakly enhanced;

[0155] The second step is to randomly scale the image from the first step;

[0156] The third step is to randomly flip the image obtained in the second step;

[0157] The fourth step is to apply a random enhancement strategy based on color space to the image obtained in the third step;

[0158] Fifth, apply a geometric-based random augmentation strategy to the image obtained in the fourth step;

[0159] The sixth step is to apply a random enhancement strategy based on color space to the image obtained in the fifth step;

[0160] Step 7: Apply a geometric-based random augmentation strategy to the image obtained in step 6.

[0161] Step 8: Filter out target boxes that are too small and the images in which the target boxes are located. Target boxes that are too small are target boxes in images with a size of less than 80×80.

[0162] The strong pipeline for student models handling unlabeled data is more complex than the pipeline for teacher models handling unlabeled data, as it includes a geometric transformation step. This design aims to provide student models with more challenging training samples within a semi-supervised learning framework, thereby enhancing their robustness to geometric changes.

[0163] In addition to the data augmentation process described above, this invention introduces two additional data augmentation methods.

[0164] The first type is pseudo-label Mixup data augmentation. In the field of computer vision, Mixup is a data augmentation method that operates on images at the pixel level, mainly used in image classification tasks. To reduce the negative impact of missed detections in pseudo-labels on model training and to enhance the gradient response of correct pseudo-labels, this paper applies Mixup data augmentation to the strong augmentation process of unlabeled data, naming it pseudo-label Mixup.

[0165] The specific workflow is as follows:

[0166] (1) Image aliasing: Select two images I with pseudo-annotations i and I j and their corresponding pseudo-label set B i and Bj Each set contains multiple bounding boxes and their category labels. Linear interpolation is applied to alias the images, generating a blended image I. mix ,like Figure 9 As shown, the specific formula is:

[0167] I mix =λ·I i +(1-λ)·I j

[0168] Where λ is a random weight sampled from the Beta distribution Beta(α,α), and α is a hyperparameter.

[0169] (2) Pseudo-tag merging: For I mix For each pixel location in the image, its corresponding pseudo-label is defined by I. i and I j The pseudo-labels at corresponding locations are linearly merged according to a weight of λ. Specifically, if both images have bounding boxes covering the location, the intersection of the two is taken; if only one image has a bounding box covering the location, that box is used directly; if neither image has a bounding box, the location is considered unlabeled. This results in a blended image I. mix pseudo-label set

[0170] The second type is pseudo-labeled Mosaic data augmentation. Mosaic data augmentation is also commonly used in image classification tasks, simulating larger image sizes by stitching together multiple images to increase the contextual information of the images. To improve the model's ability to detect small-sized targets, this paper applies the Mosaic data augmentation algorithm to the strong augmentation process of unlabeled data. By downsampling and stitching images, it increases the number of labels for small and medium-sized targets, which is named pseudo-labeled Mosaic. The specific workflow is as follows:

[0171] Image scaling: Select four images I1, I2, I3, and I4 with pseudo-labels, and their corresponding pseudo-label sets. Since the task only needs to determine the existence of objects without considering their specific locations, this paper employs max pooling to scale down the four images proportionally (e.g., by 1 / 2 or 1 / 4), resulting in scaled-down images I′1, I′2, I′3, and I′4. Because downsampling reduces the image size, objects that might be considered large in the original image may appear as small or medium-sized objects in the downsampled image. In this way, when these downsampled images are stitched together into a larger image, the new image will contain pseudo-labels for more small and medium-sized objects.

[0172] (2) Image stitching: The specific effect is as follows Figure 10The scaled-down images are stitched together in a 2x2 pattern to form a single composite image I. mosaic :

[0173]

[0174] (3) Pseudo-label transformation: For composite image I mosaic For each pixel location in the image, the corresponding pseudo-label is migrated from the pseudo-labels of the corresponding locations in the original four images. If a location is covered by bounding boxes in multiple smaller images, these boxes are merged, and the category label is updated to the category that appears most frequently in all covering boxes to reflect possible multi-category overlap.

[0175] Preferably, for supervised learning, a smooth L1 loss function is used:

[0176]

[0177] L1(x,y) represents the smoothing loss function, where x and y represent the predicted and actual values ​​of the neural network model, respectively.

[0178] For unsupervised learning, use consistency regularization loss:

[0179]

[0180] N represents the number of samples in each batch; L consistency f(x) represents the consistency regularization loss function. i g(x) represents the output of the student model on the i-th image; i ) represents the output result of the teacher model's calculation on the i-th image;

[0181] In each training iteration, the parameters of the student model are updated by minimizing the total loss function, which is a weighted sum of the two losses, L. total Let L be the total loss function. supervised To monitor the loss, L unsupervised For unsupervised loss, the weight w u Used to balance the contributions of the two:

[0182] L total =L supervised +w u *L unsupervised .

[0183] Preferably, S2 includes:

[0184] The required convolutional neural network model is Faster R-CNN.

[0185] Modify the FPN and global average pooling layers in the neck part of the Faster-RCNN model to enable the Faster-RCNN model to output the feature vector of the ear in the human ear image to be identified, and then train the modified model.

[0186] The ear image to be identified is input into the trained model to obtain the feature vector of the ear in the ear image output by the trained model.

[0187] The global average pooling operation is calculated using the following formula:

[0188]

[0189] Among them, G c F represents the output value of the c-th channel after global average pooling; i,j,c Let G represent the pixel value at position (i,j) and channel c in the original feature map. After global average pooling, each channel c has its original feature map simplified to a single value G. c Ultimately, the global average of all channels will form a new feature vector G = (G1, G2, ..., G...). c The feature map has dimensions of 1×1×c. This compact feature vector can be directly connected to the classification layer for final class prediction. H and W represent the height and width of the feature map, respectively.

[0190] Experimental verification and analysis:

[0191] The model evaluation metrics used are all commonly used in object detection tasks in deep learning, as follows:

[0192] Average Precision (AP). For a specific category, we need to obtain all predicted bounding boxes for that category and their corresponding confidence scores, as well as the corresponding ground truth bounding boxes. First, the predicted bounding boxes are sorted from highest to lowest confidence score. Then, AP is calculated using the following steps:

[0193] Precision: For the current confidence threshold, calculate the ratio of correctly detected samples (TruePositives, TP) to the total number of detections (TP + FalsePositives, FP). TP refers to the condition where the Intersection over Union (IoU) ratio between the predicted bounding box and any ground truth bounding box is greater than a predefined threshold, and the ground truth bounding box has not been matched by any other predicted bounding box with higher confidence. FalsePositives refers to predicted bounding boxes that fail to meet these conditions.

[0194] Recall: Calculated for the current confidence threshold, it is the ratio of correctly detected samples (TP) to the total number of actual positive samples (TP + False Negatives, FN). FN refers to the portion of all ground truth (GT) that has not yet been correctly matched by any predicted bounding box.

[0195] Precision-Recall Curve (PR Curve): As the confidence threshold gradually decreases, we obtain a series of values ​​for Precision and Recall. The curve that shows the change of Precision with Recall is the PR curve.

[0196] AP: For each Recall value on the PR curve, calculate the corresponding maximum Precision, and then average these maximum Precision values ​​to obtain AP. The calculation rule is expressed by the following formula:

[0197]

[0198] Where m is the number of equally spaced recall points (e.g., the commonly used 101 points), and P... interp (r i ) is in the recall rate r i The interpolation accuracy value at that point.

[0199] Mean Average Precision (mAP). Because IoU varies, mAP is the average of the average precision across multiple different rules. It measures the model's detection accuracy across different classes, and the average yields the overall detection performance. Its calculation formula is as follows:

[0200]

[0201] Where N is the total number of categories, AP c This represents the Average Precision of category c.

[0202] The intersection-over-union ratio (IoU) measures the degree of overlap between the predicted bounding box and the ground truth bounding box. The specific calculation formula is as follows:

[0203] Given a predicted bounding box PbB and a ground truth bounding box GtB, which are defined by their top-left corner coordinates... and the coordinates of the bottom right corner definition.

[0204] Intersection Area: The area of ​​the intersection of two bounding boxes, denoted as I(A), is calculated using the following formula:

[0205]

[0206] Union Area: The total area of ​​two bounding boxes, including the intersection and each bounding box's individual area, denoted as U(A), is calculated using the following formula:

[0207]

[0208] IoU: The ratio of the area of ​​intersection to the area of ​​union, i.e.:

[0209]

[0210] The range of IoU is [0,1]. IoU = 0 means that the two bounding boxes have no intersection, IoU = 1 means that the two bounding boxes completely overlap, and values ​​between these two are 0 to 1.

[0211] After repeated training and parameter adjustments, this paper finally determined the following hyperparameter settings for the experiments: AdamW was selected as the optimizer, with a base learning rate of 0.0002 and a regularization parameter for weight decay set to 0.0001. For the learning rate scheduling strategy, this paper designed three stages:

[0212] Linear warm-up phase: The initial learning rate is linearly increased from 0 to the base learning rate of 0.0002, and this process is completed in 5000 iterations.

[0213] Static hold phase: The learning rate remains constant at 0.0002 until the 35000th iteration.

[0214] Cosine Annealing Phase: From this point onward, the learning rate is adjusted using a cosine annealing (Cosine AnnealingLR) strategy. Over the next 40,000 iterations, this strategy gradually decays the learning rate from its current value to zero, following the decay pattern of the cosine function.

[0215] The cosine annealing learning rate is updated using the following formula:

[0216]

[0217] Where, η t η is the learning rate for the current training cycle (e.g., the t-th iteration); min and η max These are the minimum learning rate and the initial maximum learning rate, respectively; T max It is the total number of training cycles; T cur This is the current training cycle.

[0218] To constrain the magnitude of gradient updates, this paper applies gradient clipping, sets the maximum norm of gradient updates to 0.1, and calculates it using the L2 norm. Furthermore, the batch size is set to 8, and the planned number of iterations for the entire training process is 90,000. The data loader uses the custom loading process mentioned above.

[0219] Based on the data above, Faster R-CNN was ultimately selected as the final model. Modifications were made to the original baseline model, including the addition of GAP integration and output vectorization processing as described above, and end-to-end model training was performed.

[0220] As shown in the graphs in TensorBoard, the final loss function trended towards stabilization, reaching a peak average accuracy of 82.7% on the validation set at the 87,500th iteration (IoU used for mAP ranged from 0.5 to 0.95). On the labeled training set, the average accuracy reached a high of 98.4%, with a 15.7% difference between training and validation accuracy, indicating a potential risk of overfitting. While the final result wasn't entirely satisfactory, it provides a clear direction for future optimization: the model already possesses a certain feature extraction capability, but to ensure its effectiveness in practical applications, measures are needed to improve its generalization ability, such as reducing overfitting, performing hyperparameter tuning, and using more data for training.

[0221] Figure 11a This is a schematic diagram of the total loss function. Figure 11b This is a diagram illustrating the change in the learning rate. Figure 11c This is a schematic diagram of the average precision of the training set. Figure 11d This is a schematic diagram of the average accuracy of the test set.

[0222] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for ear-scanning identity recognition based on pseudo-label semi-supervised learning, characterized in that, include: S1, construct the dataset for training and label the dataset to obtain a labeled training set, an unlabeled training set, and a labeled validation set; S2, determine the convolutional neural network model to be used, and modify its training and detection processes according to the improved algorithm and recognition principle; S3, the improved MeanTeacher algorithm is used to train the convolutional neural network model to obtain a trained neural network model; S4. Use the trained neural network model to calculate the feature vector of the human ear image to be recognized. S5 uses the nearest neighbor search algorithm to match the feature vectors obtained in S4 with the human ear feature vectors that have been entered into the database for identification, thereby achieving end-to-end human ear identification.

2. The ear-scanning identity recognition method based on pseudo-label semi-supervised learning according to claim 1, characterized in that, S1 includes: The dataset is constructed by searching publicly available datasets on the Internet and then fusing small, medium and large ear images of three different sizes according to a preset ratio. Through manual review, erroneous images, duplicate images, and images that are too difficult to identify in the constructed dataset are removed; erroneous images include images that contain ears, but the ears are not human ears; The processed dataset is divided into three parts: a labeled training set, an unlabeled training set, and a labeled validation set; the number of images in the labeled training set is 1% of the total number of ear images in the dataset. LabelImg software was used to annotate the labeled training set and the labeled validation set. The ear region in the image was outlined using a bounding box, and the outlined region was labeled with the word "ear".

3. The ear-scanning identity recognition method based on pseudo-label semi-supervised learning according to claim 2, characterized in that, Images that are too difficult to identify are those that do not have an ear outline.

4. The ear-scanning identity recognition method based on pseudo-label semi-supervised learning according to claim 1, characterized in that, Convolutional neural network models include the Faster-RCNN model.

5. The ear-scanning identity recognition method based on pseudo-label semi-supervised learning according to claim 1, characterized in that, S3 include: In the first iteration, the following calculations are performed: Images are extracted from the labeled and unlabeled training sets at a ratio of 1:4 as the current training batch. Let b represent the number of labeled images, then the number of unlabeled images is 4b. Weak enhancement processing is performed on the unlabeled image to obtain a weakly enhanced image; The teacher model is used to predict weakly enhanced images to obtain pseudo-labels for the weakly enhanced images, thus obtaining pseudo-labeled images; low-confidence pseudo-labeled images are filtered out to generate a pseudo-labeled dataset; the pseudo-labeled dataset is stored in a cache library; low confidence refers to a confidence level of less than 0.85; Starting from the second iteration, the following operations are performed: Images are extracted from the labeled and unlabeled training sets at a ratio of 1:4 as the current training batch. Let b represent the number of labeled images, then the number of unlabeled images is 4b. Weak enhancement processing is performed on the unlabeled image to obtain a weakly enhanced image; The teacher model is used to predict weakly enhanced images to obtain pseudo-labels for the weakly enhanced images, thus obtaining pseudo-labeled images; low-confidence pseudo-labeled images are filtered out to generate a pseudo-labeled dataset; let A represent the pseudo-labeled dataset obtained in the current iteration, and store A in the cache; low confidence refers to a confidence level less than 0.85; Multiple images are extracted from the cache to form a set B. Set B does not contain any images from set A, and the number of images in set B is the same as the number of images in set A. The images in set A and set B are mixed using Mixup to obtain the first image set; The images from set A and those extracted from set B are processed using Mosaic to obtain a second set of images; the number of images extracted from set B is the same as the number of images from set A. The images in the first and second image sets are subjected to strong enhancement processing respectively to obtain the third image set; Labeled supervised augmentation processing is performed on the labeled image to obtain the augmented image; The augmented images and the third set of images are mixed and used as input data for the student model; By monitoring loss L supervised Unsupervised loss L unsupervised Optimize the student model; The teacher model is updated using an exponential moving average strategy.

6. The ear-scanning identity recognition method based on pseudo-label semi-supervised learning according to claim 5, characterized in that, The process of weak enhancement includes: The first step is to load the image to be weakly enhanced; The second step is to randomly scale the image from the first step; The third step is to randomly flip the image obtained in the second step.

7. The ear-scanning identity recognition method based on pseudo-label semi-supervised learning according to claim 5, characterized in that, The process of labeled supervision augmentation includes: The first step is to load the image and its corresponding annotation information; The second step is to randomly scale the image from the first step; The third step is to randomly flip the image obtained in the second step; The fourth step is to apply a random enhancement strategy based on color space to the image obtained in the third step; The fifth step is to filter out target boxes that are too small and the images in which the target boxes are located. Target boxes that are too small are those in images with a size of less than 80×80.

8. The ear-scanning identity recognition method based on pseudo-label semi-supervised learning according to claim 5, characterized in that, The process of strong enhancement includes: The first step is to load the image to be weakly enhanced; The second step is to randomly scale the image from the first step; The third step is to randomly flip the image obtained in the second step; The fourth step is to apply a random enhancement strategy based on color space to the image obtained in the third step; Fifth, apply a geometric-based random augmentation strategy to the image obtained in the fourth step; The sixth step is to apply a random enhancement strategy based on color space to the image obtained in the fifth step; Step 7: Apply a geometric-based random augmentation strategy to the image obtained in step 6. Step 8: Filter out target boxes that are too small and the images in which the target boxes are located. Target boxes that are too small are target boxes in images with a size of less than 80×80.

9. The ear-scanning identity recognition method based on pseudo-label semi-supervised learning according to claim 5, characterized in that, For supervised learning, a smooth L1 loss function is used: L1(x,y) represents the smoothing loss function, where x and y represent the predicted and actual values ​​of the neural network model, respectively. For unsupervised learning, use consistency regularization loss: N represents the number of samples in each batch; L consistency f(x) represents the consistency regularization loss function. i g(x) represents the output of the student model on the i-th image; i ) represents the output result of the teacher model's calculation on the i-th image; In each training iteration, the parameters of the student model are updated by minimizing the total loss function, which is a weighted sum of the two losses, L. total Let L be the total loss function. supervised To monitor the loss, L unsupervised For unsupervised loss, the weight w u Used to balance the contributions of the two: L total =L supervised +w u *L unsupervised 。 10. The ear-scanning identity recognition method based on pseudo-label semi-supervised learning according to claim 4, characterized in that, S2 include: The required convolutional neural network model is Faster R-CNN. Modify the FPN and global average pooling layers in the neck part of the Faster-RCNN model to enable the Faster-RCNN model to output the feature vector of the ear in the human ear image to be identified, and then train the modified model. The ear image to be identified is input into the trained model to obtain the feature vector of the ear in the ear image output by the trained model. The global average pooling operation is calculated using the following formula: Among them, G c F represents the output value of the c-th channel after global average pooling; i,j,c Let G represent the pixel value at position (i,j) and channel c in the original feature map. After global average pooling, each channel c has its original feature map simplified to a single value G. c Ultimately, the global average of all channels will form a new feature vector G = (G1, G2, ..., G...). c The feature map has dimensions of 1×1×c. This compact feature vector can be directly connected to the classification layer for final class prediction. H and W represent the height and width of the feature map, respectively.

Images