Electronic monitoring method based on face recognition and voiceprint recognition

Abstract

The present invention relates to the technical field of data recognition, and discloses an electronic monitoring method based on face recognition and voiceprint recognition, comprising the steps of: collecting a number of face images of a user, and performing noise reduction processing on the face images; marking the labels of each face image to form an initial training set, and using the initial training set to train a face recognition model; using the trained face recognition model to collect the face image of the user and then identify the user's identity, and at the same time, passing the collected face image into the initial training set to form an echo training set, and updating the face recognition model; collecting the original voiceprint signal of the user, performing discrete wavelet decomposition on the original voiceprint signal to remove the noise signal and obtain a clean voiceprint signal; calculating the discrete power spectrum of the clean voiceprint signal, and performing Fourier transform to obtain a voiceprint time domain signal; judging whether the user's identity and voiceprint match, and if they match, the monitoring is passed. The present invention improves the accuracy of face recognition and completes monitoring in combination with voiceprints.

Description

Technical Field

[0001] The present invention relates to the technical field of data recognition, and in particular to an electronic monitoring method based on face recognition and voiceprint recognition. Background Art

[0002] Biometrics is a new identity authentication technology. In real life and industrial applications, everyone has unique biological characteristics that are different from others. With the development of computer technology, people can extract their own biometric information, such as face, fingerprint, iris, voiceprint, etc. This technology that relies on human physical characteristics for identity recognition is called biometrics recognition technology.

[0003] In monitoring scenarios such as access control, clocking in, and alarm, the commonly used and low-cost method is to collect faces through high-definition cameras for identity recognition, such as using deep learning and other methods. However, the face recognition model has been trained before it is used. If there are some subtle changes in the face, such as dark circles under the eyes due to lack of rest, the face recognition model may take a long time to recognize, or even fail to recognize, making the monitoring efficiency low. In addition, for some places with strict access control requirements, the security of face recognition alone is not high. Therefore, a solution is needed that can update the face recognition model in real time and complete monitoring with the help of voiceprints. Summary of the invention

[0004] The purpose of the present invention is to improve the accuracy of face recognition and complete monitoring in combination with voiceprint, and to provide an electronic monitoring method based on face recognition and voiceprint recognition.

[0005] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0006] The electronic monitoring method based on face recognition and voiceprint recognition includes the following steps:

[0007] Step 1: collect several facial images of the user and perform noise reduction on the facial images; mark the labels of each facial image to form an initial training set, and use the initial training set to train the facial recognition model;

[0008] Step 2: Use the trained face recognition model to collect the user's face image and identify the user's identity. At the same time, the collected face image is passed into the initial training set to form an echo training set, and the face recognition model is updated;

[0009] Step 3: collect the original voiceprint signal of the user, perform discrete wavelet decomposition on the original voiceprint signal to remove the noise signal and obtain a clean voiceprint signal;

[0010] Step 4, calculate the discrete power spectrum of the clean voiceprint signal, and perform Fourier transform to obtain the voiceprint time domain signal;

[0011] Step 5: Determine whether the user's identity and voiceprint match. If they do, the monitoring is passed.

[0012] Compared with the prior art, the present invention has the following beneficial effects: when the face recognition model formally recognizes facial features, the present invention collects new facial features and corresponding labels and adds them to the echo training set, and uses the echo training set to update the face recognition model, thereby optimizing the recognition accuracy of the face recognition model; and matches the user's voiceprint with the identity to determine whether the user can pass the monitoring, thereby further improving the monitoring accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0014] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0016] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance, or implying any such actual relationship or order between these entities or operations.

[0017] The present invention is achieved through the following technical solutions: Figure 1As shown, the electronic monitoring method based on face recognition and voiceprint recognition includes the following steps:

[0018] Step 1: collect several facial images of the user and perform noise reduction on the facial images; mark the labels of each facial image to form an initial training set, and use the initial training set to train the facial recognition model.

[0019] Before using the face recognition model, it is necessary to train the face recognition model. This solution uses the R3D residual network as the basic network and uses the initial training set to train the R3D residual network. The initial training set consists of several face images of the user and the corresponding labels. When collecting the user's face image, each face is first subjected to noise reduction processing:

[0020] ;

[0021] Among them, (u,v) is the pixel coordinate; f(u,v) is the pixel after noise reduction processing; is the projection sampling interval; Indicates the current projection sampling times; T is the total projection sampling times; is the noise filtering weight, which is used to determine the degree of noise filtering; m is the mth channel of the image; M is the total number of image channels; is the projection parameter of the mth channel of the image; For the The weight of the subprojection sampling; is the projection data; is the interpolation weight of the channel passing through the pixel point (u, v); D is the convolution kernel in the convolution projection; k is the kth convolution kernel.

[0022] Noise reduction processing is done by projecting the pixels of the original image. During the projection process, the pixels containing noise will have an unstable noise distribution. The size of the image can be reduced.

[0023] After each face image has been subjected to the above denoising process, the Sobel algorithm is used to extract edge features in the face image to obtain face features, which can reduce the amount of model calculation. The face features are manually labeled, and the labeled labels are used to identify the user's identity. Since this solution collects several face images of the same user, when labeling several face features of the same user, only one labeling is required to complete batch labeling, reducing the complexity of manual labeling.

[0024] The facial features with labels form the initial training set {(x1,y1), (x2,y2), ..., (x n ,y n )}, where x irepresents the facial features extracted from the i-th face image, y i Represents x i The corresponding labels, n represents the total number of face images collected in the initial training set, i=1,2,...,n. The R3D residual network is trained using the initial training set so that the R3D residual network can predict the labels corresponding to the output face features, thereby obtaining a trained face recognition model.

[0025] Step 2: Use the trained face recognition model to collect the user's face image and identify the user's identity. At the same time, the collected face image is passed into the initial training set to form an echo training set, and the face recognition model is updated.

[0026] When the face recognition model is officially used, the face image is collected and recognized to identify the user's identity. n+1 and the corresponding identity information y n+1 Add to the initial training set to form the echo training set {(x1,y1),...,(x n ,y n ), (x n+1 ,y n+1 )}, which is equivalent to updating the initial training set. Then the face recognition model is iteratively updated and trained using the echo training set through the improved echo state network:

[0027]

[0028]

[0029] Among them, W in is the first connection weight; W res is the second connection weight; W b is the third connection weight; W out is the fourth connection weight; z i For x i The corresponding sparse coding function; z i+1 For x i+1 The corresponding sparse coding functions are:

[0030]

[0031] Among them, H is the basis vector; p is the sparse coefficient; || ||2 represents the two-norm.

[0032] When i=1, W in , W res , W b , W outTo initialize the generated connection weights, since (x1, y1) and (x2, y2) are known, at each iteration, the sparse coding function z is updated i To redistribute the values ​​of the four connection weights. After n iterations, let W in , W res , W b , W out The value of is continuously optimized, and then the connection weights are used to update the face recognition model to obtain the optimized face recognition model.

[0033] It is easy to understand that the (x n+2 ,y n+2 )Add the echo training set {(x1,y1),...,(x n ,y n ), (x n+1 ,y n+1 ), (x n+2 ,y n+2 )}, when using the echo state network for iterative update training, we only need to calculate:

[0034]

[0035]

[0036] Among them, z n+2 For x n+2 The corresponding sparse coding function does not need to be iterated from (x1, y1), so the amount of calculation is very small even if the face recognition model is constantly updated during use. In addition, the amount of data in the echo training set can be reduced by deleting the front face features and labels each time new face features and labels are added to the echo training set, thereby reducing the storage occupied by the face recognition model training.

[0037] Step 3: collect the user's original voiceprint signal, perform discrete wavelet decomposition on the original voiceprint signal to remove the noise signal and obtain a clean voiceprint signal.

[0038] For example, when a user passes through the access control, he / she says a short sentence or word that is the same as the one recorded, thereby collecting the user's original voiceprint signal D voice , the original voiceprint signal contains the clean voiceprint signal D clear and noise signal D noise ,Right now:

[0039]

[0040] For the original voiceprint signal D voice Perform a discrete wavelet transform:

[0041]

[0042] Among them, q(a,b) is the wavelet function; a is the scale parameter; b is the translation parameter; D voice (t) is the original voiceprint signal at time t; D clear (t) is the clean voiceprint signal at time t; D noise (t) is the noise signal at time t; is discrete wavelet; r a,b D clear (t) wavelet transform coefficient; o a,b D noise (t) is the wavelet transform coefficient of .

[0043] Generate threshold function , filter the noise signal:

[0044]

[0045] in, is the threshold value; is the lower threshold; is the upper threshold; ; ; ; sgn represents the sign function.

[0046] Through wavelet transform, the noise signal D noise Filter to get the clean voiceprint signal D clear .

[0047] Step 4: Calculate the discrete power spectrum of the clean voiceprint signal and perform Fourier transform to obtain the voiceprint time domain signal.

[0048] Calculate the clean voiceprint signal D clear The discrete power spectrum of :

[0049]

[0050] in, D is a clean voiceprint signal clear The discrete power spectrum of the K-th frequency point; l is the l-th amplitude; L is the total number of amplitudes; D clear (l) is the clean voiceprint signal at the lth amplitude; j is an imaginary number.

[0051] Perform inverse discrete Fourier transform on the discrete power spectrum to obtain the voiceprint time domain signal. First, calculate the modulus of the discrete power spectrum:

[0052]

[0053]

[0054] Among them, |X(k)| is Generate a random phase for each frequency point k , construct the complex spectrum:

[0055]

[0056] Among them, X(k) is the complex spectrum. Use the inverse discrete Fourier transform to convert the complex spectrum X(k) back to the voiceprint time domain signal:

[0057]

[0058] Among them, X(t) is the voiceprint time domain signal; t is the time domain; K is the length of the signal.

[0059] Step 5: Determine whether the user's identity and voiceprint match. If they do, the monitoring is passed.

[0060] After identifying the user's face image in step 2, the user's identity is obtained, and then it is matched with the voiceprint time domain signal obtained in step 4. If the user identity and voiceprint matched during registration are the same, the user is matched and the door is opened, or other monitoring scenes are passed.

[0061] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An electronic monitoring method based on face recognition and voiceprint recognition, characterized in that: The following steps are involved: Step 1: collect several facial images of the user and perform noise reduction on the facial images; mark the labels of each facial image to form an initial training set, and use the initial training set to train the facial recognition model; In step 1, the labels of each face image are marked to form an initial training set, and the steps of using the initial training set to train the face recognition model specifically include: Sobel algorithm is used to extract edge features in face images to obtain face features; By manually marking facial features, the marked labels are used to identify the user's identity; The facial features with labels form the initial training set {(x1,y1), (x2,y2), ..., (x n ,y n )}, where x i represents the facial features extracted from the i-th face image, y i Represents x i The corresponding label, n represents the total number of face images collected, i=1,2,...,n; Using the initial training set to train the R3D residual network, so that the R3D residual network can predict and output labels corresponding to facial features, thereby obtaining a trained face recognition model; Step 2: Use the trained face recognition model to collect the user's face image and identify the user's identity. At the same time, the collected face image is passed into the initial training set to form an echo training set, and the face recognition model is updated; The step 2 specifically includes the following steps: When the face recognition model is officially used, the face image is collected and recognized, and the recognized face features are x n+1 and the corresponding identity information y n+1 Add to the initial training set to form the echo training set {(x1,y1),...,(x n ,y n ), (x n+1 ,y n+1 )}; The face recognition model is iteratively updated and trained using the echo training set through the echo state network: Among them, W in is the first connection weight; W res is the second connection weight; W b is the third connection weight; W out is the fourth connection weight; z i For x i The corresponding sparse coding function; z i+1 For x i+1 The corresponding sparse coding functions are: Among them, H is the basis vector; p is the sparse coefficient; || ||2 represents the two-norm; Step 3: collect the original voiceprint signal of the user, perform discrete wavelet decomposition on the original voiceprint signal to remove the noise signal and obtain a clean voiceprint signal; Step 4, calculate the discrete power spectrum of the clean voiceprint signal, and perform Fourier transform to obtain the voiceprint time domain signal; Step 5: Determine whether the user's identity and voiceprint match. If they do, the monitoring is passed.

2. The electronic monitoring method based on face recognition and voiceprint recognition according to claim 1, characterized in that: In step 1, the step of collecting a plurality of facial images of the user and performing noise reduction processing on the facial images specifically includes: Collect several facial images of the user and perform noise reduction on each face: ； Among them, (u,v) is the pixel coordinate; f(u,v) is the pixel after noise reduction processing; is the projection sampling interval; Indicates the current projection sampling times; T is the total projection sampling times; is the noise filtering weight, which is used to determine the degree of noise filtering; m is the mth channel of the image; M is the total number of image channels; is the projection parameter of the mth channel of the image; For the The weight of the subprojection sampling; is the projection data; is the interpolation weight of the channel passing through the pixel point (u, v); D is the convolution kernel in the convolution projection; k is the kth convolution kernel.

3. The electronic monitoring method based on face recognition and voiceprint recognition according to claim 1, characterized in that: The step 3 specifically comprises the following steps: Collect the user's original voiceprint signal D voice , the original voiceprint signal contains the clean voiceprint signal D clear and noise signal D noise ,Right now: For the original voiceprint signal D voice Perform a discrete wavelet transform: Among them, q(a,b) is the wavelet function; a is the scale parameter; b is the translation parameter; D voice (t) is the original voiceprint signal at time t; D clear (t) is the clean voiceprint signal at time t; D noise (t) is the noise signal at time t; is discrete wavelet; r a,b D clear (t) wavelet transform coefficient; o a,b D noise (t) wavelet transform coefficients; Generate threshold function , filter the noise signal: in, is the threshold value; is the lower threshold; is the upper threshold; ; ; ; sgn represents the sign function.

Images