Access control method, system and access control terminal based on multi-source data

Abstract

The present invention relates to the field of access control technology, and in particular to an access control method, system and access control terminal based on multi-source data. The method comprises the following steps: through dynamic optical flow analysis, audio-visual auxiliary authentication and virtual coincidence model verification, multimodal fusion verification of user authentication images and audio is realized to defend against live deception attacks. Specifically, it comprises: extracting facial features of the authentication image and performing optical flow analysis to generate a portrait fluency index; when it is lower than a threshold, collecting user audio for audio-visual auxiliary authentication; when it is higher than a threshold, constructing a virtual coincidence model based on the fluency index to verify the access face model; the present invention makes full use of multimodal biometrics to achieve efficient and safe access control.

Description

Technical Field

[0001] The present invention relates to the field of door access technology, and in particular to a door access method, system and door access terminal based on multi-source data. Background Art

[0002] In the early days, people used access control, which was a mechanical password lock based on a digital combination to open the door. Users had to enter the correct digital combination to unlock the access control. The development of this technology can be traced back to the late 19th century. However, this password is easy to crack, and managers need to change the password regularly to ensure security, and users forget the password. In the 1960s, magnetic card access control systems began to appear. This system uses magnetic stripe cards or radio frequency identification cards as a means of identity authentication. Users swipe or place the card on the card reader to open the access control, but the magnetic card is easy to be copied, lost or damaged, and the security is not high. The card needs to be replaced regularly to ensure safety. With the development of technology, access control methods have begun to add biometric technology, using the unique physiological or behavioral characteristics of individuals for identity authentication, including fingerprint recognition, iris recognition, and face recognition. However, the initial stage of biometric technology has low accuracy and is easily affected or deceived by the environment, such as fingerprint recognition being contaminated and iris recognition being affected by light. In addition, biometric information is stolen or imitated. Summary of the invention

[0003] Based on this, it is necessary to provide a door access method, system and door access terminal based on multi-source data to solve at least one of the above technical problems.

[0004] To achieve the above purpose, the door access method based on multi-source data includes the following steps:

[0005] Step S1: obtaining a user authentication image; performing facial feature highlighting on the user authentication image to generate an authentication feature image; performing dynamic optical flow analysis on the authentication feature image to generate image optical flow field data;

[0006] Step S2: Calculate the optical vector amplitude of the image optical flow field data to obtain the optical flow vector motion intensity; perform adjacent frame angle change analysis on the optical flow vector field and the optical flow vector change angle; perform optical flow comprehensive measurement on the optical flow vector motion intensity and the optical flow vector change angle to generate a portrait smoothness index;

[0007] Step S3: when the portrait fluency index is less than a preset fluency threshold, the portrait fluency index is defined as a suspicious portrait fluency index and reported; user authentication audio is collected; audio-visual auxiliary authentication is performed on the user authentication audio and authentication feature image based on the suspicious portrait fluency index, so as to execute a door access method for defending against liveness spoofing attacks;

[0008] Step S4: When the portrait fluency index is greater than the preset fluency threshold, the portrait fluency index is defined as the standard portrait fluency index, and the authentication feature image is spatially overlapped and simulated based on the standard portrait fluency index and the preset access face model to obtain a virtual overlap model; based on the virtual overlap model, the preset access face model is model-difference verified to execute the standard access control method.

[0009] The present invention obtains the authentication image, extracts the facial features, and performs dynamic optical flow analysis to generate image optical flow field data. This step realizes the multi-dimensional collection and feature extraction of the user's biometric data, laying the foundation for the subsequent fluency analysis. The optical flow field data is deeply analyzed, and the motion intensity and angle change of the optical flow vector are calculated to generate the portrait fluency index. This step makes full use of the cutting-edge technology in the field of computer vision to achieve complex quantitative analysis of the user's facial behavior characteristics. When the portrait fluency index is lower than the threshold, it is identified as a suspicious portrait, and the user audio is collected at the same time to perform multi-modal authentication of audio and video fusion. This can effectively defend against live body deception attacks and improve authentication security. The generated suspicious portrait fluency index is an innovative data output of the method. When the portrait fluency index is higher than the threshold, it is identified as a standard portrait, and the face model is verified based on the virtual coincidence model to achieve standard access control access. This step makes full use of the technology of face modeling and comparative analysis to further improve the accuracy and adaptability of the access control system. The generated "virtual coincidence model" is another innovative data output. It integrates cutting-edge technologies such as computer vision, pattern recognition, and multimodal fusion, and realizes more intelligent, safe and reliable access control through complex analysis and verification of multi-source data. Each step generates innovative intermediate data, which provides important support for the execution of subsequent steps. This method based on multi-source data fusion is undoubtedly the development direction of future intelligent access control systems.

[0010] In this specification, a multi-source data-based access control system is provided, which is used to execute the multi-source data-based access control method as described above. The multi-source data-based access control system includes:

[0011] The optical flow analysis module is used to obtain the user authentication image; brighten the facial features of the user authentication image to generate the authentication feature image; perform dynamic optical flow analysis on the authentication feature image to generate image optical flow field data;

[0012] The smoothness measurement module is used to calculate the optical vector amplitude of the image optical flow field data to obtain the optical flow vector motion intensity; analyze the adjacent frame angle change of the optical flow vector field and the optical flow vector change angle; perform optical flow comprehensive measurement on the optical flow vector motion intensity and the optical flow vector change angle to generate the portrait smoothness index;

[0013] The biometric authentication module is used to define the portrait fluency index as a suspicious portrait fluency index and report it when the portrait fluency index is less than a preset fluency threshold; collect user authentication audio; perform audio-visual auxiliary authentication on the user authentication audio and authentication feature image based on the suspicious portrait fluency index to implement a door access method for defending against liveness spoofing attacks;

[0014] The standard authentication module is used to define the portrait fluency index as the standard portrait fluency index when the portrait fluency index is greater than the preset fluency threshold, and to perform spatial overlapping simulation on the authentication feature image based on the standard portrait fluency index and the preset access face model to obtain a virtual overlap model; based on the virtual overlap model, perform model difference verification on the preset access face model to execute the standard access control method.

[0015] The benefit of the present invention lies in that through dynamic optical flow analysis, not only the intensity of portrait motion is obtained, but also the change in motion angle is analyzed, thereby comprehensively reflecting the smoothness characteristics of the portrait. This multi-dimensional fluency evaluation can more accurately describe the dynamic characteristics of the portrait than a single indicator. When the abnormal fluency of the portrait is detected, the audio acquisition and multimodal fusion verification are triggered in time. This dynamic and static combined defense mechanism can effectively identify and troubleshoot live deception attack behaviors. For portraits with normal fluency, a virtual overlap model is used for further spatial feature verification. This verification method based on model differences can more accurately judge the degree of match between the portrait and the preset model. According to the level of portrait fluency, the appropriate verification method is automatically selected to achieve hierarchical management and control, which not only improves security, but also takes into account user experience, reflecting the intelligent and humanized design concept. The advantages of various technical means are fully utilized to build a comprehensive, intelligent and secure multi-level verification system.

[0016] The present invention also provides an access control terminal, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, for executing the access control method based on multi-source data as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of the steps of a door access control method based on multi-source data;

[0018] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;

[0019] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0020] Figure 4 for Figure 1Detailed implementation steps of step S4 in FIG.

[0021] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0023] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0024] It should be understood that, although the terms "first" and "second" are used here to describe each unit, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiment, the first unit can be referred to as the second unit, and similarly the second unit can be referred to as the first unit. The term "and / or" used here includes any and all combinations of one or more of the listed associated items.

[0025] To achieve this, please refer to Figures 1 to 4 , a door access method based on multi-source data, comprising the following steps:

[0026] Step S1: obtaining a user authentication image; performing facial feature highlighting on the user authentication image to generate an authentication feature image; performing dynamic optical flow analysis on the authentication feature image to generate image optical flow field data;

[0027] Step S2: Calculate the optical vector amplitude of the image optical flow field data to obtain the optical flow vector motion intensity; perform adjacent frame angle change analysis on the optical flow vector field and the optical flow vector change angle; perform optical flow comprehensive measurement on the optical flow vector motion intensity and the optical flow vector change angle to generate a portrait smoothness index;

[0028] Step S3: when the portrait fluency index is less than a preset fluency threshold, the portrait fluency index is defined as a suspicious portrait fluency index and reported; user authentication audio is collected; audio-visual auxiliary authentication is performed on the user authentication audio and authentication feature image based on the suspicious portrait fluency index, so as to execute a door access method for defending against liveness spoofing attacks;

[0029] Step S4: When the portrait fluency index is greater than the preset fluency threshold, the portrait fluency index is defined as the standard portrait fluency index, and the authentication feature image is spatially overlapped and simulated based on the standard portrait fluency index and the preset access face model to obtain a virtual overlap model; based on the virtual overlap model, the preset access face model is model-difference verified to execute the standard access control method.

[0030] The present invention obtains the authentication image, extracts the facial features, and performs dynamic optical flow analysis to generate image optical flow field data. This step realizes the multi-dimensional collection and feature extraction of the user's biometric data, laying the foundation for the subsequent fluency analysis. The optical flow field data is deeply analyzed, and the motion intensity and angle change of the optical flow vector are calculated to generate the portrait fluency index. This step makes full use of the cutting-edge technology in the field of computer vision to achieve complex quantitative analysis of the user's facial behavior characteristics. When the portrait fluency index is lower than the threshold, it is identified as a suspicious portrait, and the user audio is collected at the same time to perform multi-modal authentication of audio and video fusion. This can effectively defend against live body deception attacks and improve authentication security. The generated suspicious portrait fluency index is an innovative data output of the method. When the portrait fluency index is higher than the threshold, it is identified as a standard portrait, and the face model is verified based on the virtual coincidence model to achieve standard access control access. This step makes full use of the technology of face modeling and comparative analysis to further improve the accuracy and adaptability of the access control system. The generated "virtual coincidence model" is another innovative data output. It integrates cutting-edge technologies such as computer vision, pattern recognition, and multimodal fusion, and realizes more intelligent, safe and reliable access control through complex analysis and verification of multi-source data. Each step generates innovative intermediate data, which provides important support for the execution of subsequent steps. This method based on multi-source data fusion is undoubtedly the development direction of future intelligent access control systems.

[0031] In the embodiment of the present invention, reference Figure 1 The above is a schematic flow chart of the steps of the access control method based on multi-source data of the present invention. In this example, the access control method based on multi-source data includes the following steps:

[0032] Step S1: obtaining a user authentication image; performing facial feature highlighting on the user authentication image to generate an authentication feature image; performing dynamic optical flow analysis on the authentication feature image to generate image optical flow field data;

[0033] In the embodiment of the present invention, a near-infrared high-definition camera is used to capture the frontal portrait image of the user at a sampling rate of 30 frames per second. The image resolution is not less than 1920x1080, the color depth is 24-bit RGB, and the Haar cascade feature classifier is used to locate the face area in the image. Then, a multi-scale Retinex algorithm based on Retinex theory is used to perform nonlinear dynamic brightness enhancement on the face area to eliminate the influence of ambient lighting and generate a clear authentication feature image. The Farneback algorithm is used to calculate the pixel motion vector field between adjacent frames to obtain the dense optical flow field data of the image. The optical flow field data contains two components: the motion direction and motion amplitude of each pixel.

[0034] Step S2: Calculate the optical vector amplitude of the image optical flow field data to obtain the optical flow vector motion intensity; perform adjacent frame angle change analysis on the optical flow vector field and the optical flow vector change angle; perform optical flow comprehensive measurement on the optical flow vector motion intensity and the optical flow vector change angle to generate a portrait smoothness index;

[0035] In an embodiment of the present invention, each vector in the optical flow field is traversed and its Euclidean norm is calculated. The Euclidean norm is the modulus of the vector, which represents the motion intensity of the vector. The motion intensity values ​​of all vectors are averaged to obtain the average motion intensity of the entire optical flow field. For the corresponding vectors of two adjacent frames in the optical flow field, the angle difference between them is calculated. The angle differences of all vectors are averaged to obtain the average angle change value of the entire optical flow field. The average motion intensity and the average angle change values ​​are normalized to the [0, 1] interval respectively. Then, the comprehensive fluency index is calculated in a weighted average manner, where the weight of the average motion intensity is 0.6 and the weight of the average angle change is 0.4. The value range of the comprehensive index is also in the [0, 1] interval.

[0036] Step S3: when the portrait fluency index is less than a preset fluency threshold, the portrait fluency index is defined as a suspicious portrait fluency index and reported; user authentication audio is collected; audio-visual auxiliary authentication is performed on the user authentication audio and authentication feature image based on the suspicious portrait fluency index, so as to execute a door access method for defending against liveness spoofing attacks;

[0037] In an embodiment of the present invention, the system pre-sets the threshold value of the portrait fluency to 0.85. After performing the optical flow analysis, if the calculated portrait fluency index is lower than 0.85, it is determined to be suspicious fluency, and the index is reported to the subsequent processing module. For example, if the fluency index of a certain face verification is only 0.72, it will be marked as suspicious fluency. While verifying the face, a high-fidelity microphone with a coding sampling rate of 48kHz and a quantization bit depth of 24bit is used to synchronously collect the user's voice input. The collected audio data includes the verification password or password spoken by the user. When the portrait fluency index is marked as suspicious, the system will simultaneously analyze the user's voice features and facial dynamic information, and perform identity authentication through audio and video fusion. Specifically, voice recognition and voiceprint analysis technology are used to extract features from voice data, and compared and analyzed with the dynamic optical flow field information of facial microexpression. Only when both are verified can it be finally determined to be a real person, and then the access control authority can be opened. This multimodal biometric feature fusion method can effectively defend against various live attacks.

[0038] Step S4: When the portrait fluency index is greater than the preset fluency threshold, the portrait fluency index is defined as the standard portrait fluency index, and the authentication feature image is spatially overlapped and simulated based on the standard portrait fluency index and the preset access face model to obtain a virtual overlap model; based on the virtual overlap model, the preset access face model is model-difference verified to execute the standard access control method.

[0039] In an embodiment of the present invention, the threshold of portrait fluency is preset to 0.85. When the fluency index of a certain face verification is higher than 0.85, it is judged to be standard fluency. Next, the system performs spatial superposition simulation on the standard fluency index and the qualified face model trained in advance to generate a virtual face overlap image. This virtual overlap model can reflect the degree of consistency between the current user's facial features and the standard model. The newly generated virtual overlap model is compared and analyzed pixel by pixel and feature point by feature point with the pre-established qualified face model. Only when the difference between the two models is less than the preset threshold can it be determined that the current user has passed the standard face verification and finally the access control authority is opened. For example, if the position deviation between the virtual overlap model and the standard model on the key feature points of eyebrows, eyes, and nose is less than 3 pixels, it can be deemed to have passed the verification.

[0040] Preferably, step S1 comprises the following steps:

[0041] Step S11: collecting user authentication images in real time through the access control camera;

[0042] Step S12: performing facial tracking on the user authentication image to obtain a user facial image;

[0043] Step S13: highlighting the features of the user's facial image to generate an authentication feature image;

[0044] Step S14: extracting pixel values ​​of the authentication feature image to obtain feature image pixels;

[0045] Step S15: Performing dynamic optical flow analysis on the authentication feature image based on the feature image pixels to generate image optical flow field data.

[0046] The invention is beneficial in that the user authentication image is collected in real time by the access control camera, and the latest and reliable user identity information can be obtained, which lays the foundation for subsequent feature extraction and analysis. The facial tracking of the authentication image can accurately locate and extract the user's facial image, effectively eliminate background interference, and create a purer data input for feature extraction. Through the feature brightening technology, the key features in the user's face can be highlighted, and the authentication feature image richer in identification information can be generated, providing a more valuable data source for subsequent feature analysis. The pixel value extraction of the authentication feature image can obtain the original digital feature data, which lays the foundation for pixel-based dynamic analysis. Based on the feature image pixels, the dynamic optical flow analysis technology can be used to monitor the slight movement changes of the user's face during the authentication process in real time, and generate optical flow field data reflecting facial activity. This provides an important basis for identifying live deception behavior. The image acquisition, feature extraction, and dynamic analysis techniques are fully utilized to generate various types of data containing rich user identity information, which lays a solid foundation for subsequent credibility assessment based on multi-source data fusion. This innovative method can not only effectively prevent live deception attacks, but also reflects the in-depth application of current cutting-edge technologies in computer vision and image analysis.

[0047] In the embodiment of the present invention, a 1080P high-definition camera is used to capture the facial image of the user during face verification in real time at a capture frequency of 30 frames per second. The camera is installed above the access control system and can clearly capture the facial features of the user's front face. The captured image frames will be transmitted to the subsequent image processing module in real time. After receiving the real-time captured user facial image, the system will use the Viola-Jones face detection algorithm to quickly locate the face area in the image. Then, a target tracking algorithm based on KCF (Kernelized Correlation Filters) is used to track the detected face in real time and capture a clear front face image. After obtaining the user's facial image, the system will use the Retinex algorithm to dynamically enhance the contrast and color correct the image to highlight the key feature points of the face, such as eyebrows, eyes, nose, and lips. These facial images that have been processed with feature brightening constitute the authentication feature images that are ultimately used for identity verification. Based on the authentication feature image, the image processing function of OpenCV is used to convert the image data into a pixel matrix in numpy format. This matrix records the RGB three-channel pixel values ​​of each pixel in the image. The Farneback optical flow algorithm is used to perform real-time optical flow analysis on the authentication feature image converted into a pixel matrix format. This method based on the corresponding point displacement calculation between two frames of images can extract the movement information of the user's facial muscles and form a dynamic optical flow field data. These optical flow field data reflect the slight changes in the user's face and provide a basis for subsequent liveness detection.

[0048] Preferably, step S15 comprises the following steps:

[0049] Step S151: performing optical change analysis on the authentication feature image to obtain optical feature variables;

[0050] Step S152: optically mapping the optical characteristic variables and characteristic image pixels to generate independent pixel optical variables;

[0051] Step S153: performing optical flow field analysis on the optical variables of the independent pixels to obtain an optical flow change heat map;

[0052] Step S154: performing optical flow vector analysis on the optical flow change heat map based on a preset optical flow change threshold to generate image optical flow field data.

[0053] The benefit of the present invention is that by performing optical change analysis on the authentication feature image, key optical feature variables reflecting the dynamic changes of the face can be extracted, laying the foundation for subsequent correlation mapping and optical flow field analysis. This optical feature analysis helps to capture the live features more accurately. By correlating and mapping the optical feature variables with the feature image pixels, independent pixel optical variables can be obtained. This method of associating global optical features with local pixel features can more carefully depict various aspects of the dynamic changes of the face and provide richer data input for optical flow field analysis. Based on the independent pixel optical variables, an optical flow change heat map reflecting the dynamic changes of the entire face area can be obtained through optical flow field analysis technology. This visualization method helps to intuitively observe and analyze the live features of the face during the authentication process. Further, for the optical flow change heat map, combined with the preset optical flow change threshold, accurate optical flow vector data can be extracted to form more quantitative image optical flow field data. This quantified optical flow field data provides a more reliable basis for subsequent credibility assessment. The optical feature analysis, pixel correlation mapping, and optical flow field analysis techniques are fully utilized to generate richer and more accurate dynamic feature data of the face. This not only provides a reliable basis for preventing live body spoofing attacks, but also reflects the in-depth application of cutting-edge technologies in computer vision and image analysis. This innovative method can greatly improve the security and reliability of door access.

[0054] In an embodiment of the present invention, after obtaining the authentication feature image, the Lambert-Beer law is used to analyze the light intensity of the image, and a series of optical parameters such as color saturation, reflectivity, and refractive index are extracted. These optical feature variables can reflect the subtle changes in the skin color and texture biometrics of the user's face. An association mapping model between the optical feature variables and the pixels of the authentication feature image is established. This model can convert the aforementioned optical variables into independent optical variables for each pixel. These independent pixel optical variables can more accurately describe the optical characteristics of each position in the image. The Farneback optical flow algorithm is used to compare and analyze the optical variables of two consecutive frames of images. This optical flow calculation method based on optical features can obtain a heat map reflecting the change of optical flow. This heat map highlights the degree of change of optical flow in different areas of the user's face. A pre-trained optical flow change threshold is set, and the aforementioned optical flow heat map is analyzed. Only those areas where the amplitude of the optical flow change exceeds the threshold will be identified as having effective optical flow changes. The system will extract the optical flow vectors of these areas to form the final image optical flow field data. These data provide a key basis for subsequent liveness detection.

[0055] Preferably, step S2 comprises the following steps:

[0056] Step S21: Visualizing the optical flow vectors of adjacent frames of the image optical flow field data to obtain an optical flow vector field;

[0057] Step S22: Calculate the vector amplitude of the optical flow vector field to obtain the optical flow vector motion intensity;

[0058] Step S23: Analyze the change of the direction of the optical flow vector field in adjacent frames to obtain the change direction of the optical flow vector;

[0059] Step S24: Calculate the initial angle difference of the optical flow vector change direction to generate the optical flow vector change angle;

[0060] Step S25: Performing comprehensive optical flow measurement on the optical flow vector motion intensity and the optical flow vector change angle based on the preset weight distribution data to generate a portrait smoothness index.

[0061] The benefit of the present invention is that by visualizing the adjacent frame optical flow vectors of the image optical flow field data, the dynamic changes of the user's face during the authentication process can be intuitively displayed. This intuitive visualization effect helps to better understand and analyze the live features of the human face. By calculating the amplitude of the visualized optical flow vector field, the motion intensity of the optical flow vector reflecting the degree of dynamic changes of the face can be obtained. This quantitative index provides important input data for subsequent comprehensive measurement. By analyzing the direction changes of the adjacent frames of the optical flow vector field and calculating the initial angle difference, the change direction data of the optical flow vector can be obtained. This analysis of facial dynamic features from the perspective of direction change can supplement the motion intensity index and provide a more comprehensive data basis for comprehensive measurement. Based on the preset weight distribution, the motion intensity and change angle of the optical flow vector are comprehensively measured to generate a portrait smoothness index reflecting the dynamic features of the user's face. This comprehensive index provides a more comprehensive and reliable basis for subsequent credibility evaluation. The optical flow vector visualization, motion intensity calculation, and direction change analysis techniques are fully utilized to generate richer and more quantitative facial dynamic feature data. This not only helps to intuitively observe and analyze facial liveness features, but also provides more comprehensive and reliable data input for credibility assessment based on multi-source data fusion. This innovative method greatly improves the security and reliability of access control.

[0062] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:

[0063] Step S21: Visualizing the optical flow vectors of adjacent frames of the image optical flow field data to obtain an optical flow vector field;

[0064] In an embodiment of the present invention, the cv2.calcOpticalFlowFarneback() function in the OpenCV library is used to calculate and visualize the optical flow field data of two consecutive frames, and the optical flow field data is converted into a numpy array containing two channels, horizontal and vertical, in which each element represents the optical flow vector at that position. Then, the cv2.visualizeOpticalFlow() function is used to visualize these optical flow vectors as a colorful vector field image. In this image, the length and direction of the arrow at each position correspond to the amplitude and direction of the optical flow change at that position. For example, if the user's face has a slight blinking movement between two consecutive frames, the corresponding eye area will show an obvious optical flow vector. Similarly, facial movements such as opening and closing the lips and raising and lowering the eyebrows will also be reflected in the optical flow vector field.

[0065] Step S22: Calculate the vector amplitude of the optical flow vector field to obtain the optical flow vector motion intensity;

[0066] In an embodiment of the present invention, the amplitude of the optical flow vector at each position is calculated to obtain the motion intensity of the optical flow vector of the entire image, and each pixel position in the optical flow vector field is traversed to extract the optical flow vector components in the horizontal and vertical directions of the position. Then, the amplitude of the optical flow vector at the position is calculated using the Pythagorean theorem. An intensity matrix of the same size as the original image is obtained, in which the value of each element represents the amplitude of the optical flow vector at the position. For example, if the optical flow vector at a certain position is (3, 4), then its amplitude is √(3^2+4^2)=5. The intensity value at the position is set to 5. In this matrix, areas with higher brightness represent locations with larger optical flow changes and larger facial movements, while dark areas represent relatively static areas.

[0067] Step S23: Analyze the change of the direction of the optical flow vector field in adjacent frames to obtain the change direction of the optical flow vector;

[0068] In the embodiment of the present invention, the optical flow vector field data of two consecutive frames are converted into polar coordinates, and the optical flow vector at each position is represented by its amplitude and direction angle. The direction angles of the optical flow vectors of the two frames at the corresponding positions are compared one by one, and the angle difference between them is calculated. This angle difference reflects the degree of change in the direction of the optical flow vector at the position between the two frames. For example, if the angle of the optical flow vector at a certain position is 30 degrees in the previous frame and becomes 45 degrees in the next frame, then the angle change at the position is 15 degrees. By performing this adjacent frame direction change analysis on the entire optical flow vector field, the system can obtain a direction change matrix with the same size as the original image. In this matrix, the area with higher brightness represents the position where the optical flow direction changes greatly, while the dark area represents a relatively stable area.

[0069] Step S24: Calculate the initial angle difference of the optical flow vector change direction to generate the optical flow vector change angle;

[0070] In the embodiment of the present invention, for each position in the matrix, the system extracts the optical flow vector change angle value in the 8 neighborhoods of the position, calculates the absolute angle difference between the position and its 8 neighborhood positions, and obtains 8 angle differences. These 8 angle differences are sorted, the maximum and minimum values ​​are removed, and only the middle 6 values ​​are retained. The average value of these 6 angle differences is taken as the final optical flow vector change angle of the position. For example, assuming that the optical flow vector change angle of a certain position is 30 degrees, and its 8 neighborhood positions are: 25 degrees, 28 degrees, 32 degrees, 35 degrees, 38 degrees, 42 degrees, 45 degrees, 48 ​​degrees. The angle difference between these 8 positions and the center position is calculated to obtain: 5 degrees, 3 degrees, 2 degrees, 5 degrees, 8 degrees, 12 degrees, 15 degrees, 18 degrees, these 8 angle differences are sorted, the maximum value 18 degrees and the minimum value 2 degrees are removed, and only the middle 6 values ​​are retained: 3 degrees, 5 degrees, 5 degrees, 8 degrees, 12 degrees, 15 degrees. Take the average of these 6 values, which is 8 degrees, as the final optical flow vector change angle at this position.

[0071] Step S25: Performing comprehensive optical flow measurement on the optical flow vector motion intensity and the optical flow vector change angle based on the preset weight distribution data to generate a portrait smoothness index.

[0072] In an embodiment of the present invention, the weight coefficients of the optical flow vector motion intensity and the optical flow vector change angle are read from the preset weight distribution data, for example, set to 0.6 and 0.4, and the optical flow vector motion intensity matrix and the optical flow vector change angle matrix are traversed to perform weighted sum calculation on the corresponding values ​​of each position. That is, the optical flow vector motion intensity at the corresponding position is multiplied by 0.6, and the optical flow vector change angle is multiplied by 0.4, and the two are added to obtain the comprehensive measurement value of the position. The comprehensive measurement values ​​of all positions are averaged to obtain the average fluency index of the entire portrait area. For example, assuming that the optical flow vector motion intensity at a certain position is 10 and the optical flow vector change angle is 20 degrees. The comprehensive measurement values ​​of all positions in the entire portrait area are averaged to obtain the final portrait fluency index.

[0073] Preferably, step S3 comprises the following steps:

[0074] Step S31: when the portrait fluency index is less than a preset fluency threshold, the portrait fluency index is defined as a suspicious portrait fluency index, and the suspicious portrait fluency index is visually broadcasted;

[0075] Step S32: Collecting user authentication audio through a sound collector;

[0076] Step S33: performing audio and video frequency matching on the user authentication audio based on the suspicious portrait fluency index to generate suspicious portrait matching audio;

[0077] Step S34: Perform audio-visual assisted authentication on the suspicious person's matching audio and authentication feature image to implement a door access method for defending against liveness spoofing attacks.

[0078] The benefit of the present invention lies in that by setting a fluency threshold, when the portrait fluency index is less than the threshold, it is defined as a suspicious portrait fluency index. This method can effectively identify authentication requests with a risk of live deception, and perform a visual broadcast for them, in preparation for the next step of audio-visual assisted authentication. When a suspicious portrait fluency index is found, the user's authentication audio is collected through a sound collector. This multimodal fusion method can comprehensively obtain the user's live feature information and provide a basis for subsequent audio and video frequency matching. Based on the suspicious portrait fluency index, the user's authentication audio is subjected to audio and video frequency matching analysis to generate suspicious portrait matching audio. This method of associating video and audio features can more accurately verify the user's live features and help identify counterfeit live attack attempts. The suspicious portrait matching audio and authentication feature image are subjected to audio-visual assisted authentication to implement a door access method for defending against live deception attacks. This multimodal fusion authentication method can effectively improve the security and reliability of authentication and greatly reduce the risk of passive attacks. It makes full use of the human fluency index recognition, audio collection, and audio and video frequency matching technologies to achieve three-dimensional authentication based on multi-source data. This can not only effectively identify suspicious live deception attempts, but also greatly improve the security and reliability of the entire access control system through audio and video auxiliary authentication.

[0079] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:

[0080] Step S31: when the portrait fluency index is less than a preset fluency threshold, the portrait fluency index is defined as a suspicious portrait fluency index, and the suspicious portrait fluency index is visually broadcasted;

[0081] In an embodiment of the present invention, a fluency threshold is read from a preset parameter, for example, it is set to 80, and the portrait fluency index calculated in step S25 is compared with the preset threshold. If the portrait fluency index is less than 80, it is defined as a suspicious portrait fluency index. The suspicious portrait fluency index is visually broadcast on the display interface in a striking manner, for example, highlighted in red. At the same time, a corresponding prompt message can be given to warn the user that the current portrait is abnormal. The relevant information of the suspicious portrait fluency index is saved for further processing by the subsequent liveness detection module. For example, suppose that the calculation result of the fluency index of a certain portrait is 75, which is lower than the preset threshold of 80. The system will define 75 as the suspicious portrait fluency index, and highlight it in a striking red font on the display interface, and give a prompt message "A suspicious portrait is detected, please treat it with caution".

[0082] Step S32: Collecting user authentication audio through a sound collector;

[0083] In an embodiment of the present invention, a built-in or external high-quality sound collector on the device is activated, such as a microphone or a pickup. These sound collectors need to have wide-band response and high signal-to-noise ratio characteristics to ensure that high-quality audio signals are collected. A voice prompt is issued to the user, requiring the user to read out a random text distributed by the system, such as "The weather is so nice today." The user needs to read this text clearly and standardly, with the duration controlled at about 5-10 seconds. The sound collector captures the user's voice input, converts it into a digital audio signal, and transmits it to the sound processing module of the system in real time. The collected audio signal is preliminarily filtered and noise suppressed to ensure that the sound quality meets the subsequent liveness verification requirements.

[0084] Step S33: performing audio and video frequency matching on the user authentication audio based on the suspicious portrait fluency index to generate suspicious portrait matching audio;

[0085] In an embodiment of the present invention, the fluency index of the suspicious portrait is collected and converted into corresponding audio spectrum feature parameters. These parameters include but are not limited to frequency, amplitude, and phase, which can accurately describe the sound characteristics of the suspicious portrait. The user authentication audio signal is compared and matched with the audio spectrum characteristics of the above-mentioned suspicious portrait frame by frame. By adopting the dynamic time warping (DTW) algorithm, the similarity between the two is accurately calculated. If the user authentication audio is highly consistent with the audio spectrum characteristics of the suspicious portrait, the system will define the user authentication audio as suspicious portrait matching audio and store relevant data. For example, suppose that the audio spectrum characteristics of the suspicious portrait include the average frequency of the voiced vowel segment of 400Hz and the maximum amplitude of -12dB. The system will compare the user authentication audio with these parameters frame by frame. If the two are highly consistent, the user authentication audio will be determined to be suspicious portrait matching audio.

[0086] Step S34: Perform audio-visual assisted authentication on the suspicious person's matching audio and authentication feature image to implement a door access method for defending against liveness spoofing attacks.

[0087] In an embodiment of the present invention, the high-quality portrait feature image generated in step S31 is comprehensively analyzed with the user authentication audio. This includes cross-comparison of portrait features such as skin color and facial texture with audio features such as speech speed and pronunciation rhythm. A joint feature model for audio and video synchronization is established using deep learning artificial intelligence technology. The model can accurately detect the naturalness of user behavior and identify whether there is deceptive recording or video behavior. Only when the user's portrait features and audio features are completely consistent and pass the strict liveness detection of the model, the system will finally confirm that the user's identity is legitimate and allow him to access through the access control. For example, suppose that the user's portrait features show uniform skin color and clear facial texture, while the audio features show a natural and smooth pronunciation rhythm and normal speech speed changes. These features can be perfectly matched, and after comprehensive analysis by the system, it can be determined that the user is the real identity and allowed to pass through the access control.

[0088] Preferably, step S33 includes the following steps:

[0089] Step S331: tracing the suspicious portrait fluency index to obtain the suspicious portrait data;

[0090] Step S332: performing timestamp matching on the user authentication audio based on the suspicious portrait data to generate a matching timestamp;

[0091] Step S333: intercept the user authentication audio for a certain period of time according to the matching timestamp to obtain the suspicious person portrait matching audio.

[0092] The benefit of the present invention is that for the fluency index of the suspicious portrait found, the corresponding suspicious portrait data is obtained through the portrait tracing technology. This tracing process can provide the required visual feature data for the subsequent timestamp matching and lay the foundation for audio analysis. Based on the suspicious portrait data, the user authentication audio is timestamped and matched. This method of time-aligning the video and audio data can ensure that the subsequent intercepted audio clips are consistent with the time of the suspicious portrait data, thereby improving the accuracy of audio-visual fusion. According to the aforementioned timestamp matching results, the user authentication audio is time-intercepted to obtain the suspicious portrait matching audio. This precise audio interception process ensures that the audio data used for the subsequent audio-visual auxiliary authentication is highly correlated with the suspicious portrait data, which helps to more accurately verify the user's living characteristics. The portrait tracing and timestamp matching technical means are fully utilized to provide high-quality data input for the subsequent audio-visual auxiliary authentication. By time-aligning and segment-intercepting the video and audio data, the accuracy and reliability of the audio-visual fusion analysis are ensured, greatly improving the security of the entire access control system.

[0093] In the embodiment of the present invention, the fluency index of the suspicious portrait is read, and it is matched and compared one by one with the numerous portrait feature models pre-stored in the system database. A face recognition algorithm based on deep learning is used to accurately locate the portrait model that is most similar to the fluency index of the current suspicious portrait from the database, extract the detailed feature data contained in the portrait model, such as facial contour, skin color, and eye shape, and store it as suspicious portrait data to provide a basis for subsequent steps. The suspicious portrait data is converted into corresponding audio spectrum feature parameters, including the average frequency and maximum amplitude of the voiced vowel segment, and the user authentication audio signal is subjected to dynamic time warping (DTW) matching analysis frame by frame with the above-mentioned suspicious portrait audio features. Through comparative analysis, the system can accurately find the time point in the user authentication audio that is most consistent with the suspicious portrait audio feature, and record the corresponding timestamp. Using the matching timestamp, the corresponding time period is accurately intercepted from the user authentication audio. The audio content in this time period is the suspicious portrait matching audio that the system determines to be highly correlated with the suspicious portrait. The suspicious portrait matching audio is saved separately to provide a basis for subsequent liveness detection.

[0094] Preferably, step S34 includes the following steps:

[0095] Step S341: Perform Fourier transform on the authentication feature image to obtain a frequency domain feature tensor;

[0096] Step S342: performing spectrum template matching on the frequency domain feature tensor based on a preset frequency domain feature database of the accessed face, and generating a portrait access verification result;

[0097] Step S343: Perform bioacoustic driving processing on the suspicious portrait matching audio to generate a user voiceprint model;

[0098] Step S344: performing voiceprint feature verification on the user voiceprint model based on the preset access voiceprint data to obtain a voiceprint access verification result;

[0099] Step S345: Perform access control determination based on the portrait access verification result and the voiceprint access verification result to execute the access control method for defending against liveness spoofing attacks.

[0100] The invention is beneficial in that Fourier transform is performed on the authentication feature image to obtain a frequency domain feature tensor. This method of converting image data into frequency domain space can more effectively extract the user's facial texture and contour features, laying the foundation for subsequent spectrum template matching. Based on the preset access face frequency domain feature database, the frequency domain feature tensor is template matched and analyzed to generate a portrait access verification result. This face verification mechanism based on spectrum features is more robust than the traditional time domain feature analysis, and has stronger anti-interference ability against illumination and occlusion factors. The suspicious portrait matching audio is bioacoustically driven to generate a user voiceprint model, and feature matching is performed based on the preset access voiceprint data to obtain a voiceprint access verification result. This method of fusing visual and sound features can more comprehensively verify the user's live features and make up for the limitations of a single biometric feature. Finally, access control access is determined based on the portrait access verification result and the voiceprint access verification result. This multimodal biometric fusion mechanism gives full play to the complementary advantages between different features, greatly improves the defense capability of the entire access control system against attacks of forged live features, and ensures more reliable access control. By making full use of frequency domain feature extraction, spectrum template matching, and voiceprint feature analysis techniques, a multi-dimensional biometric verification mechanism has been constructed. By integrating and analyzing visual and sound features, the user's living features can be verified more comprehensively, greatly enhancing the security and anti-attack capabilities of the entire access control system.

[0101] In an embodiment of the present invention, a face image collected by the user in real time before the access control is captured as an authentication feature image, and a two-dimensional discrete Fourier transform is performed on the face image to convert it from the time domain to the frequency domain representation. This process generates a three-dimensional frequency domain feature tensor, which contains the amplitude and phase information of the image at different frequency components. For example, the fft2() function in MATLAB can be used to complete this Fourier transform operation. This frequency domain feature tensor will be used as the input feature of subsequent face recognition for accurate comparison with the preset access face frequency domain feature database. From the pre-established access face frequency domain feature database, the standard frequency domain feature template associated with the current user is read, and the template matching algorithm is used to compare the similarity between the frequency domain feature tensor collected by the user in real time and the standard template. This comparison process can be performed using a variety of methods such as correlation coefficient and Euclidean distance. Based on the matching results, the system will generate a portrait access verification score to characterize the similarity between the current user's face and the standard template. If the score is higher than the preset threshold, the portrait access verification is determined to be successful. Collect the audio matching of suspicious portraits, perform bioacoustic modeling on this audio data, extract the user's voiceprint features, and generate a user voiceprint model. This model contains the multi-dimensional physiological characteristics of the user's voice, such as the spectrum, resonance peak, and sound pressure. For example, it can be implemented using a voiceprint modeling algorithm based on a Gaussian mixture model (GMM). Read the standard voiceprint model associated with the current user from the pre-established access voiceprint database, and use the voiceprint matching algorithm to compare the similarity between the voiceprint model generated by the user in real time and the standard model. This comparison process can be performed using multiple methods such as log-likelihood ratio and Euclidean distance. Based on the matching results, the system will generate a voiceprint access verification score to characterize the similarity between the current user's voiceprint and the standard model. If the score is higher than the preset threshold, the voiceprint access verification is determined to be successful. Comprehensively consider the portrait access verification results and the voiceprint access verification results to obtain the final access control decision. Only when both the portrait verification and the voiceprint verification are passed, the system will determine that the current user is alive and allow him to perform the standard access process through the access control. If any verification fails, the system will determine that the current user is suspected of liveness fraud, deny their access request, and take further security measures, such as triggering alarms and intercepting users.

[0102] Preferably, step S4 comprises the following steps:

[0103] Step S41: when the portrait fluency index is greater than a preset fluency threshold, the portrait fluency index is defined as a standard portrait fluency index, and based on the standard portrait fluency index, a three-dimensional face reconstruction is performed on the authentication feature image to generate a simulated three-dimensional face model;

[0104] Step S42: Overlapping the preset admission face model and the simulated three-dimensional face model in virtual space to obtain a virtual overlap model;

[0105] Step S43: performing model difference analysis on the preset allowed face model based on the virtual overlap model to generate a difference index;

[0106] Step S44: When the difference index is higher than the preset difference threshold, the access control authentication is judged as failed; when the difference index is lower than the preset difference threshold, the access control authentication is judged as successful, so as to execute the standard access control access method.

[0107] The benefit of the present invention is that when the portrait fluency index is greater than the preset fluency threshold, it is defined as the standard portrait fluency index. Based on this standard portrait data, a simulated three-dimensional face model is generated by three-dimensional reconstruction technology. This three-dimensional reconstruction process can more accurately capture the three-dimensional features of the user's face, providing a high-quality data basis for subsequent virtual space overlap. The preset access face model and the simulated three-dimensional face model are overlapped in virtual space to generate a virtual overlap model. This method of aligning models in virtual space can more intuitively compare the differences between the user's facial features and the preset standards, laying the foundation for subsequent model difference analysis. Based on the virtual overlap model, the preset access face model is analyzed for differences and a difference index is generated. This quantitative difference measurement method can more objectively evaluate the degree of fit between the user's facial features and the standard model, providing a basis for the final access authentication judgment. When the difference index is higher than the preset difference threshold, it is determined that the access authentication fails; when the difference index is lower than the preset difference threshold, it is determined that the access authentication succeeds and the standard access method is executed. This adaptive judgment mechanism based on the difference index can flexibly adapt to the security requirements in different scenarios and improve the reliability of the entire access control system. It makes full use of the technical means of portrait 3D reconstruction, virtual space overlap, and model difference analysis to build a three-dimensional face feature verification mechanism. By accurately comparing the user's facial features with the preset standards, it can more reliably identify the attack attempt of forging living features, greatly improving the security of the entire access control system.

[0108] As an example of the present invention, refer to Figure 4 As shown, in this example, step S4 includes:

[0109] Step S41: when the portrait fluency index is greater than a preset fluency threshold, the portrait fluency index is defined as a standard portrait fluency index, and based on the standard portrait fluency index, a three-dimensional face reconstruction is performed on the authentication feature image to generate a simulated three-dimensional face model;

[0110] In an embodiment of the present invention, the portrait fluency index is read and compared with a preset fluency threshold. If the portrait fluency index is greater than the preset threshold, the system will define it as a standard portrait fluency index. This means that the portrait has a sufficiently high reliability and can be used as a benchmark for subsequent three-dimensional reconstruction. Based on the standard portrait fluency index, the system will use a deep learning-based face three-dimensional reconstruction algorithm to perform three-dimensional reconstruction of the two-dimensional authentication feature image, and comprehensively use the face geometry features and expression dynamic multi-dimensional information to generate a realistic three-dimensional face model. The model can accurately reflect the shape, skin color, and texture detail features of the user's face. For example, through three-dimensional reconstruction, the system can construct the height of the user's nose bridge, the curve of the cheekbones, the three-dimensional structure of the curvature of the eyebrows, as well as the skin fineness and pore distribution micro-texture features to form a realistic and credible three-dimensional face model.

[0111] Step S42: Overlapping the preset admission face model and the simulated three-dimensional face model in virtual space to obtain a virtual overlap model;

[0112] In an embodiment of the present invention, a pre-stored access face model is read from a database. This model is a standard three-dimensional face model collected and generated by a high-precision three-dimensional scanner when the user registers. The simulated three-dimensional face model is aligned and overlapped with the access face model in a virtual three-dimensional space. This requires the use of a complex image registration algorithm to accurately match the differences in position, angle, and proportion between the two models so that they completely overlap. For example, a rigid registration algorithm based on feature points can be used to identify the key feature points (corners of the eyes, nose tip, corners of the mouth) on the two models, and calculate the optimal translation, rotation, and scaling parameters to make the two models completely consistent. Through this virtual overlap process, the system will obtain a brand new virtual overlap model. This model accurately describes the correspondence between the user's real face and the standard face model at the time of registration.

[0113] Step S43: performing model difference analysis on the preset allowed face model based on the virtual overlap model to generate a difference index;

[0114] In an embodiment of the present invention, a virtual coincidence model is read, which accurately describes the correspondence between the user's real-time face and the standard face model during registration, and the virtual coincidence model is deeply analyzed to compare the differences between it and the preset access face model in multiple dimensions. Specifically, the system compares the corresponding feature points of the two models one by one from multiple angles of shape, texture, and volume ratio. For example, the system can measure the difference values ​​of the two models in the key features of nose bridge height, cheekbone curvature, and lip thickness, and analyze the statistical distribution characteristics of these differences. Based on the above comparative analysis, the system generates a comprehensive difference index. This index quantitatively reflects the degree of difference between the user's real-time face and the standard face model, and the calculation formula of the difference index can be: difference index = Σ (weighted sum of the absolute values ​​of each feature difference) / total number of features. Among them, the weight of each feature difference can be dynamically adjusted according to its importance to face recognition.

[0115] Step S44: When the difference index is higher than the preset difference threshold, the access control authentication is judged as failed; when the difference index is lower than the preset difference threshold, the access control authentication is judged as successful, so as to execute the standard access control access method.

[0116] In an embodiment of the present invention, a difference index is read. This index quantitatively reflects the degree of difference between the user's real-time face and the standard face model. This difference index is compared with a preset difference threshold. This threshold is a reasonable value determined by combining a large amount of actual test data with the performance characteristics of the face recognition algorithm. For example, the difference threshold can be set to 0.2. In other words, if the difference index between the user's face and the standard model exceeds 0.2, it is judged to be suspicious. If the difference index is higher than the preset 0.2 threshold, the system will directly determine that the access control authentication has failed. At this time, the subsequent standard access control access process will no longer be executed, but further liveness detection measures will be taken. For example, the user can be required to perform more rigorous liveness verification such as random action detection and voiceprint recognition to ensure the authenticity of the user's identity. On the contrary, if the difference index is lower than the 0.2 threshold, the system will determine that the access control authentication is successful, and execute the standard access control access process to allow the user to enter.

[0117] In this specification, a multi-source data-based access control system is provided, which is used to execute the multi-source data-based access control method as described above. The multi-source data-based access control system includes:

[0118] The optical flow analysis module is used to obtain the user authentication image; brighten the facial features of the user authentication image to generate the authentication feature image; perform dynamic optical flow analysis on the authentication feature image to generate image optical flow field data;

[0119] The smoothness measurement module is used to calculate the optical vector amplitude of the image optical flow field data to obtain the optical flow vector motion intensity; analyze the adjacent frame angle change of the optical flow vector field and the optical flow vector change angle; perform optical flow comprehensive measurement on the optical flow vector motion intensity and the optical flow vector change angle to generate the portrait smoothness index;

[0120] The biometric authentication module is used to define the portrait fluency index as a suspicious portrait fluency index and report it when the portrait fluency index is less than a preset fluency threshold; collect user authentication audio; perform audio-visual auxiliary authentication on the user authentication audio and authentication feature image based on the suspicious portrait fluency index to implement a door access method for defending against liveness spoofing attacks;

[0121] The standard authentication module is used to define the portrait fluency index as the standard portrait fluency index when the portrait fluency index is greater than the preset fluency threshold, and to perform spatial overlapping simulation on the authentication feature image based on the standard portrait fluency index and the preset access face model to obtain a virtual overlap model; based on the virtual overlap model, perform model difference verification on the preset access face model to execute the standard access control method.

[0122] The benefit of the present invention lies in that through dynamic optical flow analysis, not only the intensity of portrait motion is obtained, but also the change in motion angle is analyzed, thereby comprehensively reflecting the smoothness characteristics of the portrait. This multi-dimensional fluency evaluation can more accurately describe the dynamic characteristics of the portrait than a single indicator. When the abnormal fluency of the portrait is detected, the audio acquisition and multimodal fusion verification are triggered in time. This dynamic and static combined defense mechanism can effectively identify and troubleshoot live deception attack behaviors. For portraits with normal fluency, a virtual overlap model is used for further spatial feature verification. This verification method based on model differences can more accurately judge the degree of match between the portrait and the preset model. According to the level of portrait fluency, the appropriate verification method is automatically selected to achieve hierarchical management and control, which not only improves security, but also takes into account user experience, reflecting the intelligent and humanized design concept. The advantages of various technical means are fully utilized to build a comprehensive, intelligent and secure multi-level verification system.

[0123] The present invention also provides an access control terminal, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, for executing the access control method based on multi-source data as described above.

[0124] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points of view, and the scope of the present invention is limited by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the same elements of the application documents are included in the present invention.

[0125] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A door access method based on multi-source data, characterized in that: The following steps are involved: Step S1: Obtain a user authentication image; highlight the facial features of the user authentication image to generate an authentication feature image; Perform dynamic optical flow analysis on the authentication feature image to generate image optical flow field data; wherein step S1 includes the following steps: Step S11: collecting user authentication images in real time through the access control camera; Step S12: performing facial tracking on the user authentication image to obtain a user facial image; Step S13: highlighting the features of the user's facial image to generate an authentication feature image; Step S14: extracting pixel values ​​of the authentication feature image to obtain feature image pixels; Step S15: Performing dynamic optical flow analysis on the authentication feature image based on the feature image pixels to generate image optical flow field data; wherein step S15 includes the following steps: Step S151: performing optical change analysis on the authentication feature image to obtain optical feature variables; Step S152: optically mapping the optical characteristic variables and characteristic image pixels to generate independent pixel optical variables; Step S153: performing optical flow field analysis on the optical variables of the independent pixels to obtain an optical flow change heat map; Step S154: performing optical flow vector analysis on the optical flow change heat map based on a preset optical flow change threshold to generate image optical flow field data; Step S2: calculating the optical vector amplitude of the image optical flow field data to obtain the optical flow vector motion intensity; analyzing the adjacent frame angle change of the image optical flow field data to obtain the optical flow vector change angle; performing optical flow comprehensive measurement on the optical flow vector motion intensity and the optical flow vector change angle to generate a portrait smoothness index; wherein step S2 includes the following steps: Step S21: Visualizing the optical flow vectors of adjacent frames of the image optical flow field data to obtain an optical flow vector field; Step S22: Calculate the vector amplitude of the optical flow vector field to obtain the optical flow vector motion intensity; Step S23: Analyze the change of the direction of the optical flow vector field in adjacent frames to obtain the change direction of the optical flow vector; Step S24: Calculate the initial angle difference of the optical flow vector change direction to generate the optical flow vector change angle; Step S25: performing optical flow comprehensive measurement on the optical flow vector motion intensity and the optical flow vector change angle based on the preset weight distribution data to generate a portrait smoothness index, wherein step S25 includes the following steps: The weight coefficients of the optical flow vector motion intensity and the optical flow vector change angle are read from the preset weight distribution data, the optical flow vector motion intensity matrix and the optical flow vector change angle matrix are traversed, the corresponding values ​​of each position are weighted summed up to obtain the comprehensive measurement value of the position, the comprehensive measurement values ​​of all positions are averaged to obtain the average fluency index of the entire portrait area, and the comprehensive measurement values ​​of all positions in the entire portrait area are averaged to obtain the final portrait fluency index; Step S3: when the portrait fluency index is less than a preset fluency threshold, the portrait fluency index is defined as a suspicious portrait fluency index and reported; user authentication audio is collected; audio-visual auxiliary authentication is performed on the user authentication audio and authentication feature image based on the suspicious portrait fluency index, so as to execute a door access method for defending against liveness spoofing attacks; Step S4: when the portrait fluency index is greater than a preset fluency threshold, the portrait fluency index is defined as a standard portrait fluency index, and a spatial overlap simulation is performed on the authentication feature image based on the standard portrait fluency index and a preset access face model to obtain a virtual overlap model; based on the virtual overlap model, a model difference verification is performed on the preset access face model to execute a standard access control method; wherein step S4 includes the following steps: Step S41: when the portrait fluency index is greater than a preset fluency threshold, the portrait fluency index is defined as a standard portrait fluency index, and based on the standard portrait fluency index, a three-dimensional face reconstruction is performed on the authentication feature image to generate a simulated three-dimensional face model; Step S42: Overlapping the preset admission face model and the simulated three-dimensional face model in virtual space to obtain a virtual overlap model; Step S43: performing model difference verification on the preset allowed face model based on the virtual overlap model to generate a difference index; Step S44: When the difference index is higher than the preset difference threshold, the access control authentication is judged as failed; when the difference index is lower than the preset difference threshold, the access control authentication is judged as successful, so as to execute the standard access control access method.

2. The door access method based on multi-source data according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: when the portrait fluency index is less than a preset fluency threshold, the portrait fluency index is defined as a suspicious portrait fluency index, and the suspicious portrait fluency index is visually broadcasted; Step S32: Collecting user authentication audio through a sound collector; Step S33: performing audio and video frequency matching on the user authentication audio based on the suspicious portrait fluency index to generate suspicious portrait matching audio; Step S34: Perform audio-visual assisted authentication on the suspicious person's matching audio and authentication feature image to implement a door access method for defending against liveness spoofing attacks.

3. The door access method based on multi-source data according to claim 2, characterized in that: Step S33 includes the following steps: Step S331: tracing the suspicious portrait fluency index to obtain the suspicious portrait data; Step S332: performing timestamp matching on the user authentication audio based on the suspicious portrait data to generate a matching timestamp; Step S333: intercept the user authentication audio for a certain period of time according to the matching timestamp to obtain the suspicious person portrait matching audio.

4. The door access method based on multi-source data according to claim 2, characterized in that: Step S34 includes the following steps: Step S341: Perform Fourier transform on the authentication feature image to obtain a frequency domain feature tensor; Step S342: performing spectrum template matching on the frequency domain feature tensor based on a preset frequency domain feature database of the accessed face, and generating a portrait access verification result; Step S343: Perform bioacoustic driving processing on the suspicious portrait matching audio to generate a user voiceprint model; Step S344: performing voiceprint feature verification on the user voiceprint model based on the preset access voiceprint data to obtain a voiceprint access verification result; Step S345: Perform access control determination based on the portrait access verification result and the voiceprint access verification result to execute the access control method for defending against liveness spoofing attacks.

5. A door access control system based on multi-source data, characterized in that: Used to execute the door access method based on multi-source data as claimed in claim 1, the door access system based on multi-source data comprises: The optical flow analysis module is used to obtain the user authentication image; brighten the facial features of the user authentication image to generate the authentication feature image; perform dynamic optical flow analysis on the authentication feature image to generate image optical flow field data; The smoothness measurement module is used to calculate the optical vector amplitude of the image optical flow field data to obtain the optical flow vector motion intensity; analyze the adjacent frame angle change of the optical flow vector field and the optical flow vector change angle; perform optical flow comprehensive measurement on the optical flow vector motion intensity and the optical flow vector change angle to generate the portrait smoothness index; The biometric authentication module is used to define the portrait fluency index as a suspicious portrait fluency index and report it when the portrait fluency index is less than a preset fluency threshold; collect user authentication audio; perform audio-visual auxiliary authentication on the user authentication audio and authentication feature image based on the suspicious portrait fluency index to implement a door access method for defending against liveness spoofing attacks; The standard authentication module is used to define the portrait fluency index as the standard portrait fluency index when the portrait fluency index is greater than the preset fluency threshold, and to perform spatial overlapping simulation on the authentication feature image based on the standard portrait fluency index and the preset access face model to obtain a virtual overlap model; based on the virtual overlap model, perform model difference verification on the preset access face model to execute the standard access control method.

6. An access control terminal, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program is used to execute the door access method based on multi-source data as claimed in any one of claims 1 to 4.

Images