Device and method suitable for face image acquisition of old people

Through multi-view cameras and deep neural networks combined with traditional features, the problem of insufficient acquisition efficiency and ease of use of existing equipment in the acquisition of face images of elderly people is solved, and high-precision image quality is achieved Judgment and collection success rate.

CN119007260APending Publication Date: 2024-11-22CHINA ELECTRONICS STANDARDIZATION INST
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411005861.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing face acquisition equipment lacks acquisition efficiency and ease of use in the face image acquisition of elderly people, lacks aging-friendly functions, cannot effectively identify the identity information of the elderly, and lacks quality judgment steps, resulting in poor image quality.

Method used

Multi-view adjustable camera and deep neural network combined with traditional features (LBP, SIFT, BEM) are used to judge image quality, guide the elderly to collect high-quality face images through voice prompts, and obtain the image quality score through weighted fusion to ensure that Collection was successful.

Benefits of technology

It improves the quality and efficiency of face image acquisition for the elderly, provides high-precision image quality judgment function, facilitates operation of the elderly, reduces the number of acquisition failures, and enhances the aging-friendly nature of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007260A_ABST
    Figure CN119007260A_ABST
Patent Text Reader

Abstract

The invention discloses a device and method suitable for face image acquisition of old people, the device comprises a computer, an acquisition panel, multi-view adjustable cameras, a height adjustable support and a red fixation point, the acquisition panel is connected with the computer, the acquisition panel is arranged on the height adjustable support, the acquisition panel is provided with a plurality of multi-view adjustable cameras, and the multi-view adjustable cameras are connected with the red fixation point. A red fixation point is arranged on an acquisition panel, a face image containing a human face is acquired, human face features corresponding to the face image are extracted, a sample quality judgment function is realized, sample quality judgment is performed on the acquired face image, and a face image is acquired again by providing an image with a quality score and low sample quality. According to the method, the quality of the image is judged by fusing the features of the deep learning network with the traditional LBP, SIFT, BEM and other features, in the deep learning process, the training data of the old people is added, the face image features of the old people are analyzed, and a judgment basis is provided for the quality of the collected image of the old people.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to a device and method suitable for collecting face images of the elderly. Background Art

[0002] Face recognition is a very important part of computer vision and is widely used in security, finance and other fields. Face recognition aims to identify people's identity information by analyzing facial images. Face recognition mainly includes several important links: face data acquisition, signal processing, data storage, comparison and fusion decision-making. The face image acquisition system collects images or videos containing faces through devices such as cameras or cameras, and automatically detects and tracks faces in the images, and then processes the detected face images accordingly. The working principle of the face recognition system mainly includes steps such as face image acquisition, face detection, feature extraction and comparison.

[0003] In recent years, the global aging trend has become obvious, and how to collect facial images in a friendly way for the elderly has attracted more and more attention. The performance of the face analysis and recognition system depends on the quality of the collected face data. The quality of the face image is affected by factors such as expression, posture, and light. The sample quality assessment algorithm can automatically evaluate the biometric utility of the face data and filter out low-quality data. The deep learning-based method is an important part of the current sample quality assessment. In addition to image selection, face image quality assessment can also be used in various other application scenarios.

[0004] Face recognition technology recognizes input face images or videos based on human facial features and has broad application prospects. At present, the aging trend is serious, and the collection of face images of the elderly also faces many problems. For example, the elderly are not focused and cannot accurately focus the collection equipment. The elderly have reduced vision and cannot see clearly effectively. Voice prompts should provide assistance, and the volume of voice prompts for the elderly should be higher. Many face collection devices do not have the quality judgment function, and the quality of the images collected of the elderly is poor, and the identity information of the elderly cannot be effectively identified. The existing face image collection devices still have many shortcomings in terms of collection efficiency and ease of use, and do not have the problem of aging-friendly functions. The purpose of the present invention is to provide a convenient and effective face image collection device suitable for the elderly to solve the above problems.

[0005] The existing face acquisition equipment, such as a face recognition method, device, storage medium and electronic device, provides a face recognition method, including acquiring a facial image containing a face and extracting facial features corresponding to the facial image; searching for a target facial feature matching the facial feature in a plurality of pre-stored facial feature groups; obtaining the number of acquisitions of the facial image when the target facial feature is found; if the number of acquisitions is greater than or equal to a preset threshold, determining a supplementary facial feature based on the facial feature, and entering the supplementary facial feature into the facial feature group corresponding to the target facial feature. This method improves the pass rate of face recognition in complex environments.

[0006] Most face collection technologies and equipment do not fully consider the needs of facial data collection for the elderly. As they age, the physiological functions of the elderly in vision, hearing, comprehension, etc. decline. Some aspects lack sample quality assessment and judgment steps, and only provide single camera collection. Usually, the elderly are required to cooperate repeatedly to collect high-quality facial images, which is not user-friendly for the elderly.

[0007] The quality assessment method based on neural network is an important part of facial quality assessment at present. [1] The paper introduces the deep neural network-based work in detail, including FaceQnet v0, FaceQnet v1, etc., which utilize modern deep neural network architectures such as ResNet, which have been proven to be very effective in image recognition tasks. [3] Three variations of symmetry metrics based on SIFT (Scale-invariant feature transform) are proposed, including 1) counting the number of SIFT points in the left and right halves of the image, 2) counting the number of SIFT points that have matching points in the other half based on their position, and 3) further adding Euclidean distance comparison of SIFT feature vectors. [4] Sample quality assessment methods for asymmetry, inter-eye distance, illumination intensity, contrast and blur are proposed. Illumination / pose asymmetry is calculated as the sum of straight-line distances between pairs of histograms of multiple LBP (Local binary pattern) features at specified positions in half of the face image. Illumination intensity is calculated as the difference between the histogram of the input image and the fixed standard illumination histogram, contrast is calculated as the standard deviation of pixel values, and sharpness is calculated as the sum of gradient amplitudes. This type of method takes into account the quality of face samples of specific factors. Based on traditional statistical methods, the effective information of sample quality has not been fully mined. Work [5] BEM (block energy map) was proposed to represent information such as facial posture.

[0008] In view of the above problems, the present invention provides a device and method for collecting facial images of the elderly with high-precision facial image collection and image quality judgment functions. Summary of the invention

[0009] The present invention provides a device and method for collecting facial images of the elderly with high-precision facial image collection and image quality judgment functions.

[0010] The purpose of the present invention is to provide a device suitable for collecting facial images of the elderly, comprising a computer, a collection panel, a multi-view adjustable camera, a support base, a height-adjustable bracket and a red gaze point. The collection panel is connected to the computer, the collection panel is arranged on the height-adjustable bracket, the height-adjustable bracket is arranged on the support base, a plurality of multi-view adjustable cameras are arranged on the collection panel, and the red gaze point is arranged on the collection panel.

[0011] A method for collecting facial images of the elderly, the method comprising collecting facial images, judging the quality of facial images, uploading facial images and importing facial image data sets, the method comprising the following steps:

[0012] Step 1: The subject of the collection comes in front of the collection device and prepares for facial image collection;

[0013] Step 2: After the system automatically detects the face, a voice prompt will be given: "Start face image collection, please look at the camera." The multi-view camera will be used to collect face images of the elderly from different angles.

[0014] Step 3: Process the multi-view facial images collected from the elderly separately, that is, perform face detection and feature point extraction on the facial images at each angle to obtain the cropped facial images and the corresponding feature point maps;

[0015] Step 4: The quality of the cropped face image is judged by extracting LBP, SIFT, and BEM features to obtain the quality score Sv1 of the traditional method corresponding to the viewing angle v. In addition, the deep neural network model FaceQnet is used to score the image quality Sv2. The two scores are then weightedly fused to obtain the score Sv of the face image at the viewing angle v.

[0016] Step 5: If one of the viewing angles meets the quality requirement through the quality judgment in step 4, the acquisition is considered successful, and the viewing angle images that meet the quality requirement are stored and subsequently recognized and processed;

[0017] Step 6. The facial image acquisition is completed. The successful acquisition will be prompted by voice. The successfully acquired facial image can be displayed through the system interface. Please ask the elderly to check and confirm to complete the facial image acquisition task.

[0018] Furthermore, the specific steps of quality judgment in step 4 are as follows:

[0019] Step 1: Normalize the cropped face image to a fixed size. The image is denoted as I, which represents the face image of the i-th perspective. As the input of the subsequent quality evaluation algorithm, I is sent to two different types of quality evaluation processes, denoted as P1 and P2. P1 represents the face sample quality estimation based on traditional face statistical features and SVR. The calculation formula of LBP is:

[0020]

[0021] Among them, (x c ,y c ) is the pixel coordinate of the center of the window, i c is the window center before LBP calculation (x c ,y c ) pixel value, i p is the gray value of the neighborhood of the window center, p is the code of the neighborhood, s(x) is the sign function, when x>=0, s(x)=1, otherwise, s(x)=0;

[0022] Step 2: The calculation formula of BEM is:

[0023]

[0024] Where H is the image height, W is the image width, and the image is divided into m×n blocks, each block has a size of h×w, h = H / m, w = W / n;

[0025] Step 3: For SIFT features, a scale space and multi-scale Gaussian filter are constructed through Gaussian function. Each key point is described by 4×4 seed points with a total of 16 points. Each key point generates a 128-dimensional SIFT feature vector. After extracting LBP features, BEM features, and SIFT features, these three features are cascaded as the overall image description. The support vector regression model is used for quality assessment. The support vector regression model is trained in advance through the training data set. The test sample is regressed based on the trained model. The Gaussian radial basis is used in SVR regression. The modeling idea of ​​SVR is as follows:

[0026] f(x)=<w,x> +b

[0027] Step 4: Based on the SVR in step 3, we can get the sample quality score S v1The cropped face image is used as input and put into the deep neural network faceQnet for quality assessment. The vgg2face pre-trained network is used. A fully connected layer and regression are added after dropout before the fully connected layer to reduce overfitting and regress the quality score. For the regression network, the input image is 224×224×3. After multiple convolutional layers, the sizes of the convolution kernels are 1×1, 3×3, and 1×1 respectively. The subsequent dropout ratio is set to 0.25. The dimension of the first fully connected layer is 2048 dimensions, and the dimension of the second fully connected layer is 32 dimensions. The final output result is a 1-dimensional sample quality score S v2 , and then the two scores are weightedly fused to obtain the score Sv of the face image of view v:

[0028] S v =W v1 ×S v1 +W v2 ×S v2

[0029] The present invention has the following advantages: the present invention judges the quality of an image by integrating the features of a deep learning network with traditional features such as LBP, SIFT, and BEM. In the process of deep learning, the present invention adds training data of the elderly, analyzes the characteristics of facial images of the elderly, and provides a basis for judging the quality of the collected images of the elderly.

[0030] The present invention follows the principles of aging-friendly, such as significant presentation, easy to understand, and easy to operate, so that the elderly can cooperate in collecting image data. In addition, after the quality judgment is passed, a voice prompt will be given that the face image collection is successful. If the face image quality judgment score is too low, the system will prompt that the image collection failed and ask for cooperation in recollecting the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A flow chart of the face image quality judgment of a specific viewing angle of the present invention;

[0032] Figure 2 It is a sample quality assessment network diagram based on FaceQnet V1 of the present invention;

[0033] Figure 3 It is the image acquisition flow chart of the present invention;

[0034] Figure 4 It is a structural schematic diagram of the present invention.

[0035] In the figure: 1. Computer; 2. Collection panel; 3. Multi-angle adjustable camera; 4. Red gaze point; 5. Height-adjustable bracket; 6. Support base. DETAILED DESCRIPTION

[0036] The present invention provides a device suitable for collecting facial images of the elderly, comprising a computer 1, a collection panel 2, a multi-view adjustable camera 3, a support base 6, a height-adjustable bracket 5 and a red gaze point 4. The collection panel 2 is connected to the computer 1, the collection panel 2 is arranged on the height-adjustable bracket 5, the height-adjustable bracket 5 is arranged on the support base 6, a plurality of multi-view adjustable cameras 3 are arranged on the collection panel 2, and the red gaze point 4 is arranged on the collection panel 2.

[0037] A method for collecting facial images of the elderly, the method comprising collecting facial images, judging the quality of facial images, uploading facial images and importing facial image data sets, the method comprising the following steps:

[0038] Step 1: The subject of the collection comes in front of the collection device and prepares for facial image collection;

[0039] Step 2: After the system automatically detects the face, a voice prompt "Start face image acquisition, please look at the camera" is given, and the multi-angle adjustable camera 3 is used to collect face images of the elderly at different angles;

[0040] Step 3: Process the multi-view facial images collected from the elderly separately, that is, perform face detection and feature point extraction on the facial images at each angle to obtain the cropped facial images and the corresponding feature point maps;

[0041] Step 4: The quality of the cropped face image is judged by extracting LBP, SIFT, and BEM features to obtain the quality score Sv1 of the traditional method corresponding to the viewing angle v. In addition, the deep neural network model FaceQnet is used to score the image quality Sv2. The two scores are then weightedly fused to obtain the score Sv of the face image at the viewing angle v.

[0042] Step 5: If one of the viewing angles meets the quality requirement through the quality judgment in step 4, the acquisition is considered successful, and the viewing angle images that meet the quality requirement are stored and subsequently recognized and processed;

[0043] Step 6. The facial image acquisition is completed. The successful acquisition will be prompted by voice. The successfully acquired facial image can be displayed through the system interface. Please ask the elderly to check and confirm to complete the facial image acquisition task.

[0044] In this embodiment, the specific steps of quality judgment in step 4 are as follows:

[0045] Step 1: Normalize the cropped face image to a fixed size. The image is denoted as I, which represents the face image of the i-th perspective. As the input of the subsequent quality evaluation algorithm, I is sent to two different types of quality evaluation processes, denoted as P1 and P2. P1 represents the face sample quality estimation based on traditional face statistical features and SVR. The calculation formula of LBP is:

[0046]

[0047] Among them, (x c ,y c ) is the pixel coordinate of the center of the window, i c is the window center before LBP calculation (x c ,y c ) pixel value, i p is the gray value of the neighborhood of the window center, p is the code of the neighborhood, s(x) is the sign function, when x>=0, s(x)=1, otherwise, s(x)=0;

[0048] Step 2: The calculation formula of BEM is:

[0049]

[0050] Where H is the image height, W is the image width, and the image is divided into m×n blocks, each block has a size of h×w, h = H / m, w = W / n;

[0051] Step 3: For SIFT features, a scale space and multi-scale Gaussian filter are constructed through Gaussian function. Each key point is described by 4×4 seed points with a total of 16 points. Each key point generates a 128-dimensional SIFT feature vector. After extracting LBP features, BEM features, and SIFT features, these three features are cascaded as the overall image description. The support vector regression model is used for quality assessment. The support vector regression model is trained in advance through the training data set. The test sample is regressed based on the trained model. The Gaussian radial basis is used in SVR regression. The modeling idea of ​​SVR is as follows:

[0052] f(x)=<w,x> +b

[0053] Step 4: Based on the SVR in step 3, we can get the sample quality score S v1The cropped face image is used as input and put into the deep neural network faceQnet for quality assessment. The vgg2face pre-trained network is used. A fully connected layer and regression are added after dropout before the fully connected layer to reduce overfitting and regress the quality score. For the regression network, the input image is 224×224×3. After multiple convolutional layers, the sizes of the convolution kernels are 1×1, 3×3, and 1×1 respectively. The subsequent dropout ratio is set to 0.25. The dimension of the first fully connected layer is 2048 dimensions, and the dimension of the second fully connected layer is 32 dimensions. The final output result is a 1-dimensional sample quality score S v2 , and then the two scores are weightedly fused to obtain the score Sv of the face image of view v:

[0054] S v =W v1 ×S v1 +W v2 ×S v2

[0055] In this embodiment, there are clear voice prompts when the collection starts and ends. Considering that the hearing function of the elderly has declined, the collection system prompt sound is set at about 40db, and the volume is adjustable.

[0056] In this embodiment, not only real-time acquisition is supported, but also the import of collected facial image data from local devices is supported, and operations such as quality judgment of the imported facial image data are performed, involving facial image uploading, facial image quality evaluation, etc., and also provides import and export data management functions for facial image data sets.

[0057] In this embodiment, the quality of the facial images collected from each perspective is judged. As long as the quality of one or two of the images meets the requirements, the collection is considered successful, and the facial images that meet the requirements are saved for subsequent processing. If the quality of all samples is unqualified, the system voice prompts "Image collection failed, please adjust the status and re-collect". Multi-perspective facial image collection and processing is conducive to extracting high-quality multi-perspective facial information for later processing. At the same time, as long as the image of one perspective meets the quality requirements, the collection is considered successful. The collection will not be repeated due to the influence of the elderly's posture, expression, lighting, etc., which is conducive to improving data collection efficiency and facilitating data collection for the elderly.

[0058] In this embodiment, in terms of image quality evaluation, traditional features such as LBP and SIFT are used, and then the sample quality is judged by support vector regression to obtain Si1. In addition, the cropped image is used as the face input, and the FaceQnet V1 network model is used to judge the sample quality to obtain Si2. The model training reference work of this experiment is trained on the VGGFace2 database. During training, 2,000 facial image data of the elderly are added to calibrate the network model. For the test of input face, ResNet-50 is used as the main network structure for training, and the final quality score is output through the last 2 fully connected layers.

[0059] Although the specific implementation of the present invention is described in detail in conjunction with the drawings, it should not be understood as limiting the scope of protection of this patent. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still fall within the scope of protection of this patent.

Claims

1. A device for collecting facial images of the elderly, comprising a computer, a collection panel, a multi-view adjustable camera, a support base, a height adjustable bracket and a red gaze point, characterized in that: The acquisition panel is connected to a computer, the acquisition panel is arranged on a height-adjustable bracket, the height-adjustable bracket is arranged on a supporting base, a plurality of multi-view adjustable cameras are arranged on the acquisition panel, and the red gaze point is arranged on the acquisition panel.

2. A method for collecting facial images of the elderly, characterized in that: The method includes face image acquisition, face image quality judgment, face image upload and face image data set import, and the method includes the following steps: Step 1: The subject of the collection comes in front of the collection device and prepares for facial image collection; Step 2: After the system automatically detects the face, a voice prompt will be given: "Start face image acquisition, please look at the camera", and the multi-view camera will be used to collect face images of the elderly from different angles. Step 3: Process the multi-view facial images collected from the elderly separately, that is, perform face detection and feature point extraction on the facial images at each angle to obtain the cropped facial images and the corresponding feature point maps; Step 4: The quality of the cropped face image is judged by extracting LBP, SIFT, and BEM features to obtain the quality score Sv1 of the traditional method corresponding to the viewing angle v. In addition, the deep neural network model FaceQnet is used to score the image quality Sv2. The two scores are then weightedly fused to obtain the score Sv of the face image at the viewing angle v. Step 5: If one of the viewing angles meets the quality requirement through the quality judgment in step 4, the acquisition is considered successful, and the viewing angle images that meet the quality requirement are stored and subsequently recognized and processed; Step 6. The facial image acquisition is completed. The successful acquisition will be prompted by voice. The successfully acquired facial image can be displayed through the system interface. Please ask the elderly to check and confirm to complete the facial image acquisition task.

3. A method for collecting facial images of the elderly as claimed in claim 2, characterized in that: The specific steps of quality judgment in step 4 are as follows: Step 1: Normalize the cropped face image to a fixed size. The image is denoted as I, which represents the face image of the i-th perspective. As the input of the subsequent quality evaluation algorithm, I is sent to two different types of quality evaluation processes, denoted as P1 and P2. P1 represents the face sample quality estimation based on traditional face statistical features and SVR. The calculation formula of LBP is: Among them, (x c ,y c ) is the pixel coordinate of the center of the window, i c is the window center before LBP calculation (x c ,y c ) pixel value, i p is the gray value of the neighborhood of the window center, p is the code of the neighborhood, s(x) is the sign function, when x>=0, s(x)=1, otherwise, s(x)=0; Step 2: The calculation formula of BEM is: Where H is the image height, W is the image width, and the image is divided into m×n blocks, each block has a size of h×w, h = H / m, w = W / n; Step 3: For SIFT features, a scale space and multi-scale Gaussian filter are constructed through Gaussian function. Each key point is described by 4×4 seed points with a total of 16 points. Each key point generates a 128-dimensional SIFT feature vector. After extracting LBP features, BEM features, and SIFT features, these three features are cascaded as the overall image description. The support vector regression model is used for quality assessment. The support vector regression model is trained in advance through the training data set. The test sample is regressed based on the trained model. The Gaussian radial basis is used in SVR regression. The modeling idea of ​​SVR is as follows: f(x)=<w,x> +b Step 4: Based on the SVR in step 3, we can get the sample quality score S v1 The cropped face image is used as input and put into the deep neural network faceQnet for quality assessment. The vgg2face pre-trained network is used. A fully connected layer and regression are added after dropout before the fully connected layer to reduce overfitting and regress the quality score. For the regression network, the input image is 224×224×3. After multiple convolutional layers, the sizes of the convolution kernels are 1×1, 3×3, and 1×1 respectively. The subsequent dropout ratio is set to 0.

25. The dimension of the first fully connected layer is 2048 dimensions, and the dimension of the second fully connected layer is 32 dimensions. The final output result is a 1-dimensional sample quality score S v2 , and then the two scores are weightedly fused to obtain the score Sv of the face image of view v: S v =W v1 ×S v1 +W v2 ×S v2 。

Citation Information

Patent Citations

  • Scene adaptive human face recognition method

    CN108280399A

  • Human face pose estimation method fusing manual design descriptor and depth features

    CN109858342A

  • Face image quality evaluation method in unconstrained environment

    CN111696090A

  • Face image quality evaluation method based on lightweight regression network

    CN112215822A

  • Face recognition method, electronic equipment and readable medium

    CN113051978A