Palm Vein Detection Method, Mobile Terminal and Storage Medium Based on Feature Stabilizer

Abstract

The present application provides a palm vein detection method, a mobile terminal, and a storage medium based on a feature stabilizer. The method includes the steps of: collecting a plurality of palm vein images; inputting the palm vein images into a convolutional neural network model M1 for training, and using a multi-model network detection head Hn to output target information during training, and obtaining a prior model M2 after training; optimizing the multi-model network detection head Hn into a single-model network detection head H1, and correspondingly designing a network feature stabilizer module M3; importing the parameter information of the prior model M2 as pre-training weights into the network feature stabilizer module M3 to obtain a network feature stabilizer module M4; inputting the palm vein images into the network feature stabilizer module M4 for training, and using the network feature stabilizer module M4 to output the same type of target information during training, and obtaining a trained palm vein feature stabilizer M5. The method of the present application not only reduces the storage space and computing resources and meets the deployment requirements of mobile terminals, but also maintains the high detection accuracy of palm veins.

Description

Technical Field

[0001] This application belongs to the technical field of palm vein detection. More specifically, it relates to a palm vein detection method, a mobile terminal, and a storage medium based on a feature stabilizer. Background Art

[0002] With the continuous development of computer vision and artificial intelligence technologies, the application of convolutional neural networks (CNNs) based on deep learning has become increasingly widespread. While CNN models are constantly refreshing the accuracy of computer vision tasks, they also increase a large amount of storage space and computing resources.

[0003] In addition, palm vein features based on the palm can be used as an object for personal identification and are widely applied in fields such as security and rapid customs clearance in security checks.

[0004] However, in the prior art, when a convolutional neural network based on deep learning is deployed on an edge computing device for palm vein recognition on a mobile terminal, due to limited storage space and computing resources, there are many cases of missed detection of targets or misdetection of the background in the actual scene of palm vein images by existing palm vein recognition detection algorithms, or the recognition speed is very slow, which violates the high standards in fields such as security and rapid customs clearance in security checks, and it is impossible to achieve a balance between speed and accuracy on a mobile terminal with limited resources. Summary of the Invention

[0005] The purpose of the embodiments of this application is to provide a palm vein detection method, a mobile terminal, and a storage medium based on a feature stabilizer to solve the technical problem in the prior art that it is impossible to achieve a balance between speed and accuracy during the palm vein recognition process on a mobile terminal.

[0006] To achieve the above purpose, the technical solution adopted in this application is: to provide a palm vein detection method based on a feature stabilizer, including:

[0007] Step S01, collect a number of palm vein images;

[0008] Step S02, input the palm vein images into the convolutional neural network model M1 for training. When training, use the multi-model network detection head Hn to output target information, and obtain the prior model M2 after training;

[0009] Step S03, optimize the multi-model network detection head Hn into a single-model network detection head H1, and correspondingly design a network feature stabilizer module M3;

[0010] Step S04, import the parameter information of the prior model M2 as pre-training weights into the network feature stabilizer module M3 to obtain the network feature stabilizer module M4;

[0011] Step S05: Input the palm vein image into the network feature stabilizer module M4 for training. During training, use the target information of the same type output by the network feature stabilizer module M4 to obtain the trained palm vein feature stabilizer M5.

[0012] Preferably, in step S01, after collecting a number of palm vein images, it may further include preprocessing the palm vein images, and the preprocessing includes denoising and enhancing the images and performing maximum curvature transformation processing on the images in sequence.

[0013] Preferably, the target information includes the target category and the target position.

[0014] Preferably, the target category includes palm vein and background.

[0015] Preferably, the target position includes the upper left coordinate of the target detection box and the lower right coordinate of the target detection box.

[0016] Preferably, the palm vein detection method based on the feature stabilizer further includes:

[0017] Step S06: Test the accuracy and speed of the prior model M2 and the trained palm vein feature stabilizer M5.

[0018] Preferably, in step S03, the method for optimizing the multi-model network detection head Hn into a single-model network detection head H1 includes the steps of:

[0019] Retain the model network detection head with the largest output feature map resolution in the prior model M2 and its associated convolutional layers;

[0020] Remove the remaining model network detection heads and their associated convolutional layers.

[0021] This application also provides a mobile terminal, and the mobile terminal includes a processor, and the processor is used to run the palm vein detection method based on the feature stabilizer as described above.

[0022] This application also provides a storage medium, and the storage medium stores a computer program, and when the computer program is executed by a processor, it implements the palm vein detection method based on the feature stabilizer as described above.

[0023] Compared with the prior art, the palm vein detection method based on a feature stabilizer provided in this application optimizes the multi-model network detection head Hn into a single-model network detection head H1, correspondingly designs a network feature stabilizer module M3, and then imports the parameter information of the prior model M2 as pre-trained weights into the network feature stabilizer module M3. Based on the characteristics of the above-mentioned palm vein images, optimized network training is achieved by combining the designed feature stabilizer on the basis of the prior network model before optimization. Finally, the palm vein feature stabilizer M5 is obtained, which not only reduces the storage space and computing resources and meets the deployment requirements of mobile terminals, but also maintains the high detection accuracy of palm veins and meets the requirements of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of this application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0025] Figure 1 It is a schematic flowchart of the palm vein detection method based on a feature stabilizer provided in the embodiment of this application;

[0026] Figure 2 It is Figure 1 a schematic diagram of the network structure of the prior model M2 in

[0027] Figure 3 It is Figure 1 a schematic diagram of the network structure of the palm vein feature stabilizer M5 in

[0028] Figure 4 It is a schematic diagram of the mobile terminal provided in the embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] In order to make the technical problems, technical solutions and beneficial effects to be solved by this application clearer, the following further describes this application in detail with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0030] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0031] It should be understood that the orientation or positional relationship indicated by terms such as "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present application.

[0032] In addition, the terms "first" and "second" are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, "a plurality of" means two or more, unless otherwise specifically defined.

[0033] Please refer to Figures 1 to 3 , and a palm vein detection method based on a feature stabilizer provided by an embodiment of the present application will be described below. The palm vein detection method based on the feature stabilizer includes:

[0034] Step S01, collecting a plurality of palm vein images;

[0035] Step S02, inputting the palm vein images into a convolutional neural network model M1 for training. When training, a multi-model network detection head Hn is used to output target information, and a prior model M2 is obtained after training;

[0036] Step S03, optimizing the multi-model network detection head Hn into a single-model network detection head H1, and correspondingly designing a network feature stabilizer module M3;

[0037] Step S04, importing the parameter information of the prior model M2 as pre-training weights into the network feature stabilizer module M3 to obtain a network feature stabilizer module M4;

[0038] Step S05, inputting the palm vein images into the network feature stabilizer module M4 for training. When training, the network feature stabilizer module M4 is used to output the same type of target information, and a trained palm vein feature stabilizer M5 is obtained.

[0039] In step S01, after collecting a plurality of palm vein images, it may further include preprocessing the palm vein images, and the preprocessing includes performing denoising enhancement and maximum curvature transformation processing on the images in sequence.

[0040] Among them, denoising and enhancement can specifically include using denoising algorithms such as BM3D (Block-matching and 3D filtering), DCT (Discrete Cosine Transform), PCA (Principal Component Analysis), K-SVD (K-singular value decomposition), and non-local means denoising to denoise the image, and using image enhancement algorithms such as histogram equalization, logarithmic image enhancement, and exponential image enhancement to enhance the image. The maximum curvature transform processing includes calculating the image with the maximum curvature algorithm after denoising and enhancement to obtain the palmar vein contour in the image and get clear palmar vein patterns.

[0041] In step S02, the palmar vein image is input into the convolutional neural network model M1 for training. When training, the multi-model network detection head Hn is used to output target information, and the prior model M2 is obtained after training. Among them, when selecting the convolutional neural network model M1, it is preferably to select more than 3 model network detection heads. In this way, the more the number of model network detection heads, the more parameters can be obtained. When using the multi-model network detection head Hn during training, more target information can be output, so as to extract finer palmar vein features.

[0042] Furthermore, as an implementation manner of this embodiment, the target information includes the target category and the target location. Among them, the target category includes palmar veins and the background; the target location includes the upper left coordinate of the target detection box and the lower right coordinate of the target detection box.

[0043] In step S03, the multi-model network detection head Hn is optimized into a single-model network detection head H1, and the network feature stabilizer module M3 is correspondingly designed. The single-model network detection head H1 can be obtained by structural deletion based on the multi-model network detection head Hn according to the requirements of mobile terminal deployment. The deleted single-model network detection head H1 has the characteristics of simple structure and small model parameter quantity. Of course, the correspondingly designed network feature stabilizer module M3 will also have a decrease in model accuracy.

[0044] It should be noted that since the palmar vein image is collected based on near-infrared light, and the near-infrared camera is usually 5 to 10 cm away from the palm center during collection. Thus, compared with the images collected based on visible light conventionally, the palmar vein image has the characteristics of simple scene and single background. At the same time, the overall vein situation of each person's palm is approximately the same, and the feature difference from the background is obvious.

[0045] Thus, based on the characteristics of the above-mentioned palm vein images, in step S04, the method of importing the parameter information of the prior model M2 as the pre-trained weights into the network feature stabilizer module M3 is adopted to assign the model parameters of the prior model M2 to the network feature stabilizer module M3, obtaining the network feature stabilizer module M4; thus, the network feature stabilizer module M4 is optimized and loaded with the model parameters of the prior model M2.

[0046] Furthermore, in step S05, the palm vein image is input into the network feature stabilizer module M4 for training, and the same type of target information output by the network feature stabilizer module M4 is used during training. Among them, the same type of target information refers to the target information output by the multi-model network detection head Hn in step S02, thus solving the adaptability problem of the model parameters of the prior model M2 in the network feature stabilizer module M4. In this way, it can be ensured that the accuracy of the trained palm vein feature stabilizer M5 will not be lost.

[0047] Finally, the obtained palm vein feature stabilizer M5 not only reduces the number of parameters and computational complexity, meeting the requirements for deployment on mobile terminals, but also maintains the high detection accuracy of palm veins, meeting the requirements of the application scenario.

[0048] Furthermore, as an implementation manner of this embodiment, the palm vein detection method based on the feature stabilizer further includes:

[0049] Step S06, testing the accuracy and speed of the prior model M2 and the trained palm vein feature stabilizer M5.

[0050] In actual application, taking the model of a three-model network detection head as an example, after finally optimizing the three detection head models into a single detection head model, according to the above method, the obtained palm vein feature stabilizer M5 has an accuracy comparable to that of the prior model M2 (the accuracy of the prior model is 0.85, and the accuracy of the palm vein feature stabilizer M5 is 0.847). However, the number of parameters of the palm vein feature stabilizer M5 is reduced by 33.3% compared with that of the prior model M2, and the computational complexity is reduced by 16.4%. After transplantation to the terminal, the inference time can be reduced to one-third of the original.

[0051] The palm vein detection method based on a feature stabilizer provided by this application, compared with the prior art, optimizes the multi-model network detection head Hn into a single-model network detection head H1, correspondingly designs a network feature stabilizer module M3, and then imports the parameter information of the prior model M2 as pre-trained weights into the network feature stabilizer module M3. Based on the characteristics of the above palm vein images, optimized network training is achieved by combining the designed feature stabilizer on the basis of the prior network model before optimization. Finally, the palm vein feature stabilizer M5 is obtained, which not only reduces the storage space and computing resources and meets the deployment requirements of mobile terminals, but also maintains the high detection accuracy of palm veins and meets the application scenario requirements.

[0052] Further, as an implementation manner of this embodiment, in step S03, the method for optimizing the multi-model network detection head Hn into a single-model network detection head H1 includes the steps of:

[0053] Retain the model network detection head with the largest output feature map resolution in the prior model M2 and its associated convolutional layers;

[0054] Remove the remaining model network detection heads and their associated convolutional layers.

[0055] It can be understood that, usually, in the prior model M2 of the multi-model network detection head Hn, the first model network detection head outputs the feature map with the largest resolution. Thus, some detection heads at the tail of the prior model M2 and their associated convolutional layers are removed at the same time. Based on the characteristics of the above palm vein images, it can make the performance close to that before optimization while reducing the number of network parameters and the amount of computation, effectively solving the problem of high time consumption in model deployment.

[0056] Please refer to Figure 4 , this application also provides a mobile terminal, and the mobile terminal includes a processor, and the processor is used to run the above-mentioned palm vein detection method based on a feature stabilizer.

[0057] It can be understood that the so-called processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the terminal and connects various parts of the entire terminal through various interfaces and lines.

[0058] If the modules / units integrated in the mobile terminal are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present invention, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0059] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A palm vein detection method based on a feature stabilizer, characterized in that, Including: Step S01: Collect a number of palm vein images; Step S02: Input the palm vein images into the convolutional neural network model M1 for training. When training, use the multi-model network detection head Hn to output target information, and obtain the prior model M2 after training; Step S03: Optimize the multi-model network detection head Hn into a single-model network detection head H1, and correspondingly design the network feature stabilizer module M3; Step S04: Import the parameter information of the prior model M2 as pre-training weights into the network feature stabilizer module M3 to obtain the network feature stabilizer module M4; Step S05: Input the palm vein images into the network feature stabilizer module M4 for training. When training, use the network feature stabilizer module M4 to output the same type of target information, and obtain the trained palm vein feature stabilizer M5.

2. The palm vein detection method based on a feature stabilizer according to claim 1, characterized in that, In step S03, the method of optimizing the multi-model network detection head Hn into a single-model network detection head H1 includes the steps of: Retain the model network detection head with the largest output feature map resolution in the prior model M2 and its associated convolutional layers; Remove the remaining model network detection heads and their associated convolutional layers.

3. The palm vein detection method based on a feature stabilizer according to claim 1, characterized in that, The target information includes the target category and the target location.

4. The palm vein detection method based on a feature stabilizer according to claim 3, characterized in that, The target category includes palm vein and background.

5. The palm vein detection method based on a feature stabilizer according to claim 3, characterized in that, The target location includes the upper left corner coordinates of the target detection box and the lower right corner coordinates of the target detection box.

6. The palm vein detection method based on a feature stabilizer according to claim 1, characterized in that, In step S01, after collecting a number of palm vein images, it further includes preprocessing the palm vein images. The preprocessing includes denoising and enhancing the images and performing maximum curvature transformation processing in sequence.

7. The palm vein detection method based on a feature stabilizer according to claim 1, characterized in that, It also includes: Step S06: Test the accuracy and speed of the prior model M2 and the trained palm vein feature stabilizer M5.

8. A mobile terminal, characterized in that, The mobile terminal includes a processor, and the processor is used to run the palm vein detection method based on the feature stabilizer according to any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by the processor, it implements the palm vein detection method based on the feature stabilizer according to any one of claims 1 to 7.

Images