Human ear detection and identification method and system based on improved YOLOv12s

Abstract

The invention relates to the technical field of machine vision, in particular to a human ear detection and recognition method and system based on improved YOLOv12s. According to the method, human ear detection is carried out by utilizing a trained human ear recognition model. A backbone network of the human ear recognition model is provided with an SEFA attention module used for extracting channel attention features and frequency domain attention features. When an input ear image passes through the SEFA attention module in the backbone network, the SEFA attention module extracts channel attention features and frequency domain attention features of the input image. The channel attention features configure different weights for each channel of the input image, and adjust feature recognition weights of different channels in subsequent steps. The frequency domain attention feature captures the local texture and edge information of the input feature, and enhances the recognition ability of the model to the human ear texture. And finally, fusing the human ear feature vectors of multiple levels input by the backbone network through the neck network, and respectively inputting the human ear feature vectors into the three head networks to predict and obtain a human ear identification result.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a method and system for detecting and recognizing human ears based on an improved YOLOv12s. Background Art

[0002] In specific application scenarios, such as criminal investigation, only profile images of the target are often available, significantly reducing the effectiveness of traditional facial recognition technology. In contrast, ear recognition technology, with its stable morphological characteristics and strong anti-occlusion capabilities, can provide a reliable alternative authentication solution for such scenarios, effectively addressing the shortcomings of existing technologies.

[0003] Ear detection and recognition, an emerging biometric technology, shows great potential in fields such as security authentication and human-computer interaction. However, ear images captured by cameras often have obstructions such as hats, headphones, and hair, partially or completely hiding ear features. This results in low accuracy for existing neural network models for ear detection and recognition.

[0004] In order to improve the performance of the existing YOLOv12 model in human ear detection and recognition tasks, this application provides a human ear detection and recognition method and system based on an improved YOLOv12s. Summary of the Invention

[0005] To overcome the problems existing in the related art, the first aspect of the present application provides a human ear detection and recognition method based on an improved YOLOv12s, comprising: Build a human ear recognition model; Training the human ear recognition model using human ear image samples; Performing ear detection using the trained ear recognition model; The human ear recognition model includes a backbone network, a neck network, and a head network based on YOLOv12s; The backbone network is used to extract features from the input human ear image. The backbone network is provided with a SEFA attention module connected to the neck network. The SEFA attention module is used to extract channel attention features and frequency domain attention features. The neck network is used to fuse the image features extracted by the backbone network and input the feature fusion results into the head network; The head network is used to output human ear detection results.

[0006] In one embodiment, the SEFA attention module includes a channel attention submodule, a frequency domain attention submodule, and a gated fusion submodule; The channel attention submodule determines the channel attention feature according to the input human ear feature vector through the first fully connected layer; The frequency domain attention submodule maps the human ear feature vector to the frequency domain and determines the frequency domain attention feature through the second fully connected layer; The gated fusion submodule is used to fuse the channel attention features and the frequency domain attention features.

[0007] In one embodiment, the channel attention submodule performs global average pooling on the input features, extracts nonlinear channel dependencies through the first fully connected layer, and determines the channel attention features based on the nonlinear channel dependencies; The function of the first fully connected layer is expressed as:

[0008] in, for activation function, for activation function, is the channel weight vector, is the trainable parameter matrix.

[0009] In one embodiment, the frequency domain attention submodule maps the human ear feature vector to the frequency domain, determines the frequency domain attention through the second fully connected layer through a two-dimensional discrete cosine transform, and determines the frequency domain attention feature through the frequency domain attention; The function of the second fully connected layer is expressed as:

[0010] in, It is the result of global averaging of frequency domain features in the spatial dimension. is the frequency domain attention, for Activation function.

[0011] In one embodiment, in the gated fusion submodule, the input channel attention features, the frequency domain attention features, and the original input features are fused to obtain human ear fusion features.

[0012] In one embodiment, the human ear recognition model introduces a regional attention mechanism in the feature extraction layer, divides the input features into multiple spatial regions through the regional attention mechanism, and calculates the attention output for each of the spatial regions separately.

[0013] In one embodiment, the human ear recognition model uses a FlashAttention mechanism to accelerate attention calculation.

[0014] In one embodiment, the convolutional layer of the human ear recognition model is a separable convolutional layer.

[0015] In one embodiment, the separable convolution layer consists of a depthwise convolution module and a pointwise convolution module; The depth convolution module makes each channel of the input feature convolution with a Convolution kernel performs convolution; The point-by-point convolution module is implemented by 1 The convolution kernels linearly combine the outputs of the depthwise separable modules.

[0016] The second aspect of the present application provides a human ear detection and recognition system based on improved YOLOv12s, which is used to perform the human ear detection and recognition method described in the first aspect of the present application.

[0017] The technical solution provided by this application may have the following beneficial effects: In the present application, the backbone network of the human ear recognition model performs feature extraction on the input human ear image. When the human ear image is input into the SEFA attention module in the backbone network, the SEFA attention module extracts the channel attention features and frequency domain attention features of the input image respectively. The channel attention features assign different weights to each channel of the human ear feature vector, and adjust the feature recognition weights of different channels in subsequent steps. The frequency domain attention features capture the local texture and edge information of the human ear feature vector, and enhance the model's ability to recognize human ear texture. The gated fusion submodule outputs the human ear fusion feature and inputs it into the neck network. The neck network splices the inputs of multiple feature extraction layers of the backbone network and inputs them into the head network to predict the human ear recognition results.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0020] Figure 1 This is a flowchart of the human ear detection and recognition method based on the improved YOLOv12s shown in the embodiment of the present application; Figure 2 This is a structural diagram of the human ear recognition model shown in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The preferred embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0022] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0023] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0024] Example 1 In order to solve the technical problem that the existing YOLO model is not accurate enough for the task of human ear image detection and recognition, the embodiment of the present application provides a method for human ear detection and recognition based on improved YOLOv12s, such as Figure 1 As shown, the following steps are included: S100, building a human ear recognition model; S200, training the human ear recognition model using human ear image samples; S300: Perform ear detection using the trained ear recognition model.

[0025] The human ear recognition model is improved based on the existing YOLOv12 framework, including a backbone network, a neck network and a head network.

[0026] The backbone network is used to extract features from the input human ear image, the neck network is used to fuse the image features extracted by the backbone network, and the feature fusion results are input into the head network; the head network is used to output the human ear detection results.

[0027] In actual application scenarios, the ear area in the human ear image input to the model is very small, and objects such as hats, headphones, and hair obstruct the model, making it difficult to recognize ear features. To better extract ear features, the ear recognition model in the embodiment of the present application improves the attention mechanism of the backbone network based on the original YOLOv12.

[0028] Figure 2 This is a structural diagram of the human ear recognition model shown in an embodiment of the present application.

[0029] like Figure 2 As shown, the backbone network of the human ear recognition model shown is provided with a SEFA attention module connected to the neck network, and the SEFA attention module is used to extract channel attention features and frequency domain attention features.

[0030] Specifically, the SEFA attention module includes a channel attention submodule, a frequency domain attention submodule and a gated fusion submodule.

[0031] In the embodiment of the present application, the first few layers extract the features of the input ear image to obtain the ear feature vector The input to the SEFA attention module is further extracted through the channel attention submodule and the frequency domain attention submodule, and feature fusion is performed in the gated fusion submodule.

[0032] Furthermore, the ear feature vector is processed as follows when input into the channel attention submodule: S101. Human ear feature vector Perform global average pooling and get ; S102: Extracting nonlinear channel dependencies through the first fully connected layer .

[0033] In this embodiment of the present application, the function of the first fully connected layer is expressed as:

[0034] in, for activation function, for activation function, is the channel weight vector, is the trainable parameter matrix.

[0035] S103, determining the channel attention feature according to the nonlinear channel dependency; The channel attention feature is:

[0036] in, represents the channel attention feature, is the channel weight vector.

[0037] Furthermore, the ear feature vector is processed as follows when input into the frequency domain attention submodule: S201, mapping the human ear feature vector to the frequency domain, and obtaining the frequency domain feature through two-dimensional discrete cosine transform; Exemplarily, the function of frequency domain transformation is expressed as:

[0038] in, is the frequency domain feature, is a two-dimensional discrete cosine transform operation.

[0039] S202, outputting a frequency domain weight vector through a second fully connected layer; In this embodiment of the present application, the function of the second fully connected layer is expressed as:

[0040] in, It is the result of global averaging of frequency domain features in the spatial dimension. is the frequency domain weight vector, for activation function, is the trainable weight matrix.

[0041] S203: Determine frequency domain attention features based on frequency domain weight vectors .

[0042] For example, the output frequency domain attention features are:

[0043] in, is the frequency domain attention feature, is the input human ear feature vector, is the frequency domain weight vector.

[0044] Furthermore, the gated fusion submodule is used to fuse the channel attention feature and the frequency domain attention feature. The frequency domain attention feature and the channel attention feature described above are input into the gated fusion submodule to output the human ear fusion feature.

[0045] The function of the gated fusion submodule is expressed as:

[0046]

[0047] in, is the human ear fusion feature, is the human ear feature vector, is the attention fusion feature, is the channel fusion weight, is the frequency domain fusion weight.

[0048] In the embodiment of the present application, the gated fusion submodule introduces a gate mechanism for dynamic weighting to construct a weight vector and through Normalization , then fuse the results of the two attention branches, and finally perform element-wise multiplication operation on the attention fusion feature and the original input feature to complete the enhancement and obtain the human ear fusion feature.

[0049] In an embodiment of the present application, the backbone network of the human ear recognition model performs feature extraction on the input human ear image. When the human ear image is input into the SEFA attention module in the backbone network, the SEFA attention module extracts the channel attention features and frequency domain attention features of the input image respectively. The channel attention features assign different weights to each channel of the human ear feature vector, and adjust the feature recognition weights of different channels in subsequent steps. The frequency domain attention features capture the local texture and edge information of the human ear feature vector, and enhance the model's ability to recognize human ear texture. The gated fusion submodule outputs the human ear fusion feature and inputs it into the neck network. The neck network splices the inputs of multiple feature extraction layers of the backbone network and inputs them into the head network to predict the human ear recognition results.

[0050] The human ear recognition model designed in the embodiment of the present application also has the following characteristics: (1) R-ELAN (Residual-enhanced Layer Aggregation Network) is used as the backbone structure for feature extraction. This structure improves the information flow between shallow and deep features by introducing extended residual paths and feature aggregation mechanisms.

[0051] The core structure of the R-ELAN module includes: 1. Enhance the expressiveness of features based on multi-path residual blocks; 2. Introducing the Cross-Stage Partial (CSP) design to reduce redundant calculations and improve the lightweightness of the model; 3. In the neck network, feature map outputs from different levels are aggregated to enhance the robustness and training stability of the network.

[0052] (2) The human ear recognition model introduces a regional attention mechanism in the feature extraction stage, which divides the input features into multiple spatial regions and calculates the attention output for each spatial region separately.

[0053] In the embodiment of the present application, the input human ear feature vector Divided into non-overlapping regions , respectively calculated The query vector, key vector, and value vector of the non-overlapping regions are then normalized into a probability distribution using the Softmax function.

[0054] For example, the function of the regional attention mechanism is expressed as:

[0055]

[0056]

[0057]

[0058]

[0059] in, is the trainable weight matrix, is a non-overlapping area, is the query vector of the rth non-overlapping region, is the key vector of the rth non-overlapping region, is the transposed vector of the key vector of the rth non-overlapping region, is the value vector of the rth non-overlapping region.

[0060] (3) Use FlashAttention mechanism to accelerate attention calculation.

[0061] FlashAttention is an improved attention mechanism with efficient memory access and cache-friendly performance. It does not change the mathematical expression of the essential attention calculation, but improves performance through pipeline reordering and block-level reduction at the implementation level. Its core operations include: 1. Calculate matrix multiplication in blocks to avoid large-scale matrix writing; 2. Introducing numerical stability Computational strategies; Exemplarily, the calculation strategy is:

[0062] in, For the i The input vector of the class.

[0063] 3. Rewrite the Attention module through the CUDA kernel to achieve GPU hardware-level acceleration.

[0064] In an embodiment of the present application, FlashAttention significantly reduces the video memory overhead and running time of the attention module while maintaining accuracy.

[0065] (4) The convolutional layer of the human ear recognition model is a separable convolutional layer.

[0066] Furthermore, the convolution module of the ear recognition model can be separated into two parts: depth convolution Make each input channel independent of a The convolution kernel performs convolution, and the point-by-point convolution is , the total computational cost is , where K=7 and C' is the number of output channels.

[0067] The combined formula is:

[0068] The separable convolutional layer significantly reduces the number of parameters and computational complexity while providing a sufficiently large receptive field.

[0069] (5) The neck network is constructed through a multi-scale feature fusion module: To enhance the detector's ability to recognize objects of different sizes, the ear recognition model uses a structure similar to the Feature Pyramid Network (FPN) to fuse feature maps of different scales.

[0070] In the upsampling module, the high-level feature map Perform upsampling to align its spatial resolution with the low-level feature map. The upsampling method can be bilinear interpolation or deconvolution to obtain the upsampled feature vector .

[0071] The upsampled feature map and low-level feature maps of corresponding scales Perform concatenation operations in the channel dimension The concatenated feature map is input to the convolutional layer for feature fusion to obtain the final fusion feature representation .

[0072] The following is the experimental process of the human ear recognition model: (1) Constructing experimental datasets; Table 1 is a data set information table. As shown in Table 1, the embodiment of the present application adopts a three-stage division method of training set (70%), validation set (15%) and test set (15%).

[0073]

[0074] The EAR dataset contains 2,631 ear images from 10 individuals. During data collection, various lighting conditions and scenarios involving earring wear were considered to enhance the dataset's real-world applicability. Due to individual ear feature variations and varying shooting conditions, the dataset exhibits significant variations in image resolution and quality. This heterogeneity increases sample diversity but also complicates recognition.

[0075] Use the LabelImg tool to manually label the image dataset. First, import the image data into the LabelImg software and set the labeling format to YOLO. Then manually select the ear area in the image and label each image one by one. Then save the corresponding label file. Each image will generate a .txt file with the same name. The file contains the target category number, the coordinates of the center point of the bounding box, and the width and height of the bounding box. All values ​​are normalized to the image size range [0,1]. (2) Set experimental parameters During the experiment, the SGD optimizer was used to train for 325 epochs, with a batch size of 64 and an image size of 640. The initial learning rate was set to 0.01, a linear learning rate decay strategy was adopted, and a warm-up was performed in the first 3 epochs.

[0076] (3) Select evaluation indicators Precision, recall, average precision, and intersection-over-union ratio are selected as evaluation indicators. The higher the value, the better the detection performance of the model.

[0077] (4) Experimental results The experimental results of the human ear recognition model described in the embodiment of the present application are shown in Table 2.

[0078]

[0079] It can be seen that the human ear recognition model was experimented on the EAR dataset, and achieved 91.3% precision, 90.9% recall rate and 85.8% mAP50-95 for human ear detection and recognition in this dataset, which is better than the existing attention mechanism alternatives.

[0080] (5) Analysis of ablation experiment results

[0081] The embodiments of the present application tested the five model configurations shown in Table 3 through ablation experiments. When the SEFA module was completely removed, the model's Precision was 84.4%, Recall was 90.1%, and IoU was only 40.3%. The overall performance was significantly reduced, indicating that the SEFA module plays a key role in improving the model's boundary positioning ability and stability. When SEFA was replaced with the CBAM or FA mechanism, the IoU further decreased to 39.7% and 40.8%, respectively, indicating that these two mechanisms have certain limitations in spatial-channel modeling and cannot effectively take into account both global and frequency domain structural features. In contrast, when the ECA or EGA attention mechanism is used to replace SEFA, the model's precision and recall rate are slightly improved, but the IoU is still significantly lower than the solution proposed in this patent, indicating that although this type of lightweight attention mechanism performs well in the precision dimension, its modeling ability for complex texture areas is not as comprehensive as SEFA.

[0082] Judging from comprehensive indicators, the ear recognition model proposed in this technical solution achieved the best results in the ear recognition task for the EAR dataset, significantly outperforming all alternative solutions.

[0083] Example 2 A human ear detection and recognition system based on an improved YOLOv12s algorithm is used to perform the steps in the human ear detection and recognition method described in Example 1.

[0084] The scheme of the present application has been described in detail above with reference to the accompanying drawings. In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. Those skilled in the art should also be aware that the actions and modules involved in the description are not necessarily required for this application. In addition, it is understood that the steps in the method of the embodiment of the present application can be adjusted in sequence, merged and deleted according to actual needs, and the modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0085] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0086] Alternatively, the present application can also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or electronic device, server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0087] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the application herein may be implemented as electronic hardware, computer software, or combinations of both.

[0088] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems and methods according to multiple embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0089] The embodiments of the present application have been described above. The above description is illustrative and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein.

Claims

1. A human ear detection and recognition method based on improved YOLOv12s, characterized in that: include: Build a human ear recognition model; Training the human ear recognition model using human ear image samples; Performing ear detection using the trained ear recognition model; The human ear recognition model includes a backbone network based on YOLOv12s, a neck network, and a head network; The backbone network is used to extract features from the input human ear image. The backbone network is provided with a SEFA attention module connected to the neck network. The SEFA attention module is used to extract channel attention features and frequency domain attention features. The neck network is used to fuse the image features extracted by the backbone network and input the feature fusion results into the head network; The head network is used to output human ear detection results.

2. A human ear detection and recognition method based on improved YOLOv12s according to claim 1, characterized in that The SEFA attention module includes a channel attention submodule, a frequency domain attention submodule and a gated fusion submodule; The channel attention submodule determines the channel attention feature according to the input human ear feature vector through the first fully connected layer; The frequency domain attention submodule maps the human ear feature vector to the frequency domain and determines the frequency domain attention feature through the second fully connected layer; The gated fusion submodule is used to fuse the channel attention features and the frequency domain attention features.

3. A human ear detection and recognition method based on improved YOLOv12s according to claim 2, characterized in that, The channel attention submodule performs global average pooling on the input features, extracts nonlinear channel dependencies through the first fully connected layer, and determines the channel attention features according to the nonlinear channel dependencies; The function of the first fully connected layer is expressed as: in, for activation function, for activation function, is the channel weight vector, is the trainable parameter matrix.

4. A human ear detection and recognition method based on improved YOLOv12s according to claim 2, characterized in that The frequency domain attention submodule maps the human ear feature vector to the frequency domain, determines the frequency domain attention through the second fully connected layer through a two-dimensional discrete cosine transform, and determines the frequency domain attention feature through the frequency domain attention; The function of the second fully connected layer is expressed as: in, It is the result of global averaging of frequency domain features in the spatial dimension. is the frequency domain attention, for Activation function.

5. The human ear detection and recognition method based on improved YOLOv12s according to claim 2, characterized in that In the gated fusion submodule, the input channel attention features, the frequency domain attention features and the original input features are fused to obtain the human ear fusion features.

6. The human ear detection and recognition method based on improved YOLOv12s according to claim 1, characterized in that The human ear recognition model introduces a regional attention mechanism in the feature extraction layer, which divides the input features into multiple spatial regions and calculates the attention output for each spatial region separately.

7. The human ear detection and recognition method based on improved YOLOv12s according to claim 1, characterized in that The human ear recognition model adopts the FlashAttention mechanism to accelerate attention calculation.

8. The human ear detection and recognition method based on improved YOLOv12s according to claim 1, characterized in that The convolutional layer of the human ear recognition model is a separable convolutional layer.

9. The human ear detection and recognition method based on improved YOLOv12s according to claim 8, characterized in that The separable convolution layer consists of a depthwise convolution module and a pointwise convolution module; The depth convolution module makes each channel of the input feature convolution with a The convolution kernel performs convolution; The point-by-point convolution module is implemented by 1 The convolution kernels linearly combine the outputs of the depthwise separable modules.

10. A human ear detection and recognition system based on improved YOLOv12s, characterized in that: Used to perform the human ear detection and recognition method according to any one of claims 1 to 9.

Images