Biometric method, apparatus, computer device, storage medium and program product

Abstract

The application relates to a biometric identification method, a biometric identification device, a computer device, a storage medium and a computer program product. The biometric identification method comprises: acquiring an audio and video formed by photographing a user in a current scene; determining biological characteristics of the user in multiple dimensions and scene information of the user in the current scene based on the audio and video; determining an identification weight value of the biological characteristics in each dimension under the scene information, and determining a scene score of the scene information; the scene score is used to represent a score of the scene information of the current scene compared with preset optimal scene information; the scene score and a biological characteristic score are fused, and an identification result of the user is determined based on a fusion result; the biological characteristic score is determined by the biological characteristics in each dimension and the identification weight value of the corresponding dimension. The method can enhance the accuracy and security of user identification when identifying the user.

Description

Technical Field

[0001] This application relates to the field of biometric technology, and in particular to a biometric method, a biometric device, a computer equipment, a storage medium, and a computer program product. Background Technology

[0002] Biometric technology combines computers with advanced technologies such as optics, acoustics, biosensors, and biostatistics to identify individuals using their inherent physiological characteristics (such as fingerprints, faces, and irises) and behavioral features. Current biometric identification technologies are playing a crucial role in population management, counter-terrorism, airport security, border control, access control and security systems, financial anti-counterfeiting, and e-commerce.

[0003] Taking facial recognition and fingerprint recognition technologies as examples, facial / fingerprint recognition mainly includes two parts: biometric registration and verification / identification. The registration process involves pre-registering users and collecting and storing their facial / fingerprint templates. After one registration, verification / identification can be performed multiple times. The identification process involves capturing the user's facial / fingerprint image and comparing it with the facial / fingerprint template; if the comparison is successful, the facial / fingerprint recognition is complete.

[0004] However, the recognition rate of current face / fingerprint recognition methods is easily affected by the user's specific circumstances (e.g., the recognition rate of face / fingerprint recognition for users with burns may be lower than that for normal users), which in turn leads to low accuracy in biometric identification of users. Summary of the Invention

[0005] This disclosure provides a biometric identification method, a biometric identification device, a computer equipment, a storage medium, and a computer program product to at least solve the problems of low accuracy and security in biometric identification in related technologies. The technical solution of this disclosure is as follows:

[0006] According to a first aspect of the present disclosure, a biometric identification method is provided, comprising:

[0007] Acquire audio and video recordings of users within the current scene;

[0008] Based on the audio and video, determine the user's biometric features in multiple dimensions and the user's scene information in the current scene; the biometric features in multiple dimensions include at least two of the user's facial features, iris features, voice features and fingerprint features;

[0009] The recognition weight values ​​of the biometric features of each dimension under the scene information are determined, and the scene score of the scene information is determined; the scene score is used to characterize the score of the scene information of the current scene compared with the preset optimal scene information.

[0010] The scene score and biometric score are fused together, and the user's identification result is determined based on the fusion result; the biometric score is determined by the biometric features of each dimension and the identification weight value of the corresponding dimension.

[0011] In an exemplary embodiment, determining the user's biometric characteristics across multiple dimensions based on the audio and video includes:

[0012] The audio and video data are analyzed to obtain the user's image and audio data;

[0013] Based on the image data and the audio data, the biometric features of the multiple dimensions are extracted.

[0014] In an exemplary embodiment, determining the user's scene information in the current scene based on the audio and video includes:

[0015] Based on the image data and the audio data, at least one category of scene factors is determined, and the factor score of each category of scene factors is determined; the factor score is used to characterize the score of the scene factors in the current scene compared with the preset optimal scene factors;

[0016] By aggregating the scene factors of at least one category and the factor scores of the scene factors of each category, the scene information under the current scene is obtained.

[0017] In an exemplary embodiment, the category of the scene factor includes at least one of the following: business form category, business process category, user environment category, user physiological category, user behavior category, and user emotion category when the user is being photographed;

[0018] Determining the recognition weight values ​​of the biometric features in each dimension under the scene information includes:

[0019] Obtain the correlation between scene factors of each category and biometric features of each dimension; the correlation is used to characterize the degree of influence of scene factors of each category on the user identification result when using biometric features of each dimension for user identification.

[0020] Based on the factor score of each category of scene factors and the correlation of the biometric features of the corresponding dimensions of each category of scene factors, the recognition weight value of the biometric features of each dimension is determined.

[0021] In an exemplary embodiment, determining the scene score of the scene information includes:

[0022] Obtain the mean and variance values ​​of the scene factors for each category;

[0023] The probability density of the scene information is calculated using the factor scores of at least one category of scene factors and the mean and variance of each category of scene factors, and the scene score is determined.

[0024] In an exemplary embodiment, after determining the user's biometric features across multiple dimensions based on the audio and video, the method further includes:

[0025] Based on the image data and / or the audio data, calculate the biopsy value corresponding to each dimension of the biometric feature, as the first type of feature score; and

[0026] Based on the image data and / or the audio data, the feature similarity corresponding to each dimension of biometric features is calculated as the second type of feature score; the feature similarity is used to characterize the degree of similarity between the user's biometric features and the preset template features;

[0027] After determining the recognition weight values ​​of the biometric features of each dimension under the scene information, the method further includes:

[0028] The biometric score is determined based on the first and second feature scores corresponding to each dimension of biometric features, and the recognition weight value of each dimension of biometric features under the scene information.

[0029] In an exemplary embodiment, determining the user's identification result based on the fusion result includes:

[0030] If the score corresponding to the fusion result is less than the preset fusion score, then the user's recognition result is recognition failure;

[0031] If the score corresponding to the fusion result is greater than or equal to the preset fusion score, then the user's recognition result is successful.

[0032] In one exemplary embodiment, after determining the user's identification result based on the fusion result, the method further includes:

[0033] If the recognition result is a recognition failure, an adjustment prompt will be issued so that the user can adjust the user's current scene according to the adjustment prompt;

[0034] Based on the adjusted current scenario, the user is re-identified biometrically, resulting in a new identification result.

[0035] According to a second aspect of the present disclosure, a biometric device is provided, comprising:

[0036] The image acquisition unit is configured to acquire images of users within the current scene.

[0037] The first processing unit is configured to perform an action based on the audio and video to determine the user's biometric features in multiple dimensions and the user's scene information in the current scene; the biometric features in multiple dimensions include at least two of the user's facial features, iris features, voice features and fingerprint features;

[0038] The second processing unit is configured to determine the recognition weight values ​​of the biometric features of each dimension under the scene information, and to determine the scene score of the scene information; the scene score is used to characterize the score of the scene information of the current scene compared with the preset optimal scene information.

[0039] The fusion recognition unit is configured to fuse the scene score and the biometric score, and determine the user's recognition result based on the fusion result; the biometric score is determined by the biometric features of each dimension and the recognition weight value of the corresponding dimension.

[0040] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0041] processor;

[0042] Memory for storing the executable instructions of the processor;

[0043] The processor is configured to execute the executable instructions to implement the biometric method as described in any of the preceding claims.

[0044] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, the computer-readable storage medium including program data that, when executed by a processor of an electronic device, enables the electronic device to perform the biometric method as described in any of the preceding claims.

[0045] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including program instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the biometric method as described in any of the preceding claims.

[0046] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0047] First, audio and video recordings of the user within the current scene are acquired. Then, based on the audio and video, multiple dimensions of the user's biometric features and scene information within the current context are determined. These multiple dimensions of biometric features include at least two of the user's facial features, iris features, voice features, and fingerprint features. Next, the recognition weight values ​​of each dimension of biometric features within the scene information are determined, along with a scene score. The scene score represents the score of the current scene information compared to a preset optimal scene information. Finally, the scene score and biometric feature score are fused, and the user's identification result is determined based on the fusion result. The biometric feature score is determined by each dimension of biometric features and its corresponding recognition weight value. In this way, on the one hand, using scene information corresponding to the user's dynamic environment for user identification improves the accuracy of biometric identification and optimizes the user identification experience; on the other hand, using multiple dimensions of user features and fusing feature weights within the corresponding scene for user identification improves the security and reliability of biometric identification.

[0048] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0050] Figure 1 This is an application environment diagram illustrating a biometric identification method according to an exemplary embodiment.

[0051] Figure 2 This is a flowchart illustrating a biometric method according to an exemplary embodiment.

[0052] Figure 3 This is a flowchart illustrating a step for determining multiple dimensions of biometrics according to an exemplary embodiment.

[0053] Figure 4 This is a flowchart illustrating a step for determining scene information in the current scene according to an exemplary embodiment.

[0054] Figure 5 This is a flowchart illustrating a step for determining the identification weight values ​​of each biometric feature according to an exemplary embodiment.

[0055] Figure 6 This is a flowchart illustrating a step for determining a scene score based on an exemplary embodiment.

[0056] Figure 7 This is a flowchart illustrating a process of re-performing biometric identification on a user according to an exemplary embodiment.

[0057] Figure 8 This is a flowchart illustrating a biometric method according to another exemplary embodiment.

[0058] Figure 9 This is a block diagram illustrating a biometric method according to another exemplary embodiment.

[0059] Figure 10 This is a block diagram illustrating a biometric device according to an exemplary embodiment.

[0060] Figure 11 This is a block diagram illustrating an electronic device for biometric identification according to an exemplary embodiment.

[0061] Figure 12 This is a block diagram illustrating a computer-readable storage medium for biometric identification according to an exemplary embodiment.

[0062] Figure 13 This is a block diagram illustrating a computer program product for biometric identification according to an exemplary embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] The terms "first," "second," etc., used in this application are used to distinguish different objects, not to describe a specific order. Furthermore, although the terms "first," "second," etc., are used repeatedly to describe various operations (or various thresholds, or various applications, or various instructions, or various elements), these operations (or thresholds, applications, instructions, or elements) should not be limited by these terms. These terms are only used to distinguish one operation (or threshold, application, instruction, or element) from another operation (or threshold, application, instruction, or element). For example, a first fusion score can be referred to as a second fusion score, and a second fusion score can be referred to as a first fusion score, without departing from the scope of this application. Both the first fusion score and the second fusion score are scores of a biometric feature obtained by fusing scores of corresponding biometric features; however, they are not scores of the same biometric feature.

[0065] It should be noted that if the technical solution of this application involves users' personal information, the user must be clearly informed of the rules for processing personal information and their consent obtained before the product corresponding to the technical solution of this application processes the user's personal information. If the technical solution of this application involves users' sensitive personal information, the user's individual consent must be obtained before the product corresponding to the technical solution of this application processes the user's sensitive personal information, and the requirement of "express consent" must also be met. For example, at the user's personal information collection device such as a camera, a clear and prominent sign should be set up to inform the user that they have entered the scope of personal information collection, and that the user's personal information will be collected within this scope. If the user voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or, on the personal information processing device, the personal information processing rules should be clearly informed through signs / information, and the user's personal authorization should be obtained through pop-up information or by asking the individual to upload their personal information. The personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0066] The biometric identification method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server.

[0067] In one embodiment, reference Figure 1 First, server 104 acquires audio and video footage of the user in the current scene. Then, based on the audio and video, server 104 determines the user's biometric features across multiple dimensions and the user's scene information in the current scene. The biometric features across multiple dimensions include at least two of the user's facial features, iris features, voice features, and fingerprint features. Next, server 104 determines the recognition weight values ​​of each of the biometric features in the scene information and determines the scene score of the scene information. The scene score represents the score of the scene information in the current scene compared to a preset optimal scene information. Finally, server 104 fuses the scene score and the biometric score, and determines the user's recognition result based on the fusion result. The biometric score is determined by the biometric features of each dimension and the corresponding recognition weight values.

[0068] In some embodiments, terminal 102 (such as a mobile terminal or a fixed terminal) can be implemented in various forms. Terminal 102 can be a mobile terminal, including mobile phones, smartphones, laptops, portable handheld devices, personal digital assistants (PDAs), tablet computers (PADs), etc., capable of capturing images of the user in the current scene and using those images for biometric identification. Terminal 102 can also be a fixed terminal, including automated teller machines (ATMs), access control machines, digital TVs, desktop computers, fixed-line computers, etc., capable of capturing images of the user in the current scene and using those images for biometric identification. Hereinafter, it is assumed that terminal 102 is a fixed terminal. However, those skilled in the art will understand that, if there are operations or elements specifically designed for mobile purposes, the construction according to the embodiments disclosed in this application can also be applied to mobile type terminals 102.

[0069] In some embodiments, the image processing and data processing components running on server 104 may load any of the various additional server applications and / or middleware applications being executed, such as HTTP (Hypertext Transfer Protocol), FTP (File Transfer Protocol), CGI (Common Gateway Interface), RDBMS (Relational Database Management System), etc.

[0070] In some embodiments, server 104 may be implemented using a standalone server or a server cluster consisting of multiple servers. Server 104 may be adapted to run one or more application services or software components that provide the terminal 102 described in the foregoing disclosure.

[0071] In some embodiments, the application service may include, for example, a photography service for capturing images of the user in the current scene, and follow-up services provided to the user after successful biometric authentication. The software component may include, for example, an app or client with biometric authentication capabilities.

[0072] In some embodiments, an app or client with biometric functionality includes a portal that provides one-to-one application services to users in the foreground and multiple business systems that process data in the background, so as to extend the application of biometric functionality to the app or client, thereby enabling users to use and access biometric functionality anytime and anywhere.

[0073] In some embodiments, the biometric function of the APP or client can be a computer program running in user mode to complete one or more specific tasks. It can interact with the user and has a visual user interface. The APP or client can include two parts: a graphical user interface (GUI) and an engine, which together provide users with a variety of application services in the form of a user interface, creating a digital customer system.

[0074] In some embodiments, users can input corresponding code data or control parameters into an app or client via an input device to execute application services of a computer program and display application services in the user interface. For example, if it is necessary to capture the user's current scene, the user can operate the input device and display the image through the user interface. Optionally, the input device can be a touchscreen input, button input, voice input, or pupil focusing input, etc.

[0075] In some embodiments, the operating system running on the app or client may include various versions of Microsoft... Apple and / or Linux operating system, various commercial or similar Operating systems (including but not limited to various GNU / Linux operating systems, Google) OS and / or mobile operating systems, such as Phone OS OS OS operating systems, as well as other online or offline operating systems.

[0076] In one embodiment, such as Figure 2 As shown, a biometric method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the method includes the following steps:

[0077] Step S11: Acquire the audio and video footage captured from the user in the current scene.

[0078] In one embodiment, the user uses a camera device in the terminal to capture audio and video of the user in the current scene in real time. Then, the terminal sends the captured audio and video to the server for subsequent data processing.

[0079] The camera device may include a face camera, an iris camera, a fingerprint scanner, a recorder, etc., so that the audio and video captured by the camera device may include the user's environment, the user's face, the user's iris, and the recorded user's voice, the user's fingerprint, etc.

[0080] In some embodiments, the user environment, user face, user iris, and user fingerprint in the audio and video can be carried in the form of images and / or videos.

[0081] Images can include bitmaps, JPEGs, PNGs (Portable Web Graphics), GIFs, JPGs, PDFs, or depth maps, while videos can include Microsoft Video, Real Player, MPEG videos, mobile videos, or Apple Video.

[0082] As an example, the camera device can capture audio and video of a user's face and iris by first locating and acquiring the user's face image using a face camera, then locating and acquiring the user's iris image by focusing on the user's eyes. While acquiring the user's face and iris images, the camera device can simultaneously acquire sound from the scene (including the user's voice and ambient sound) using a recorder. At any moment during audio and video acquisition, the server can issue an operation command to the user to collect fingerprint information. The user can then follow the operation command, and the camera device can use a fingerprint scanner to record the user's fingerprint information.

[0083] Step S12: Based on audio and video, determine the user's biometric features in multiple dimensions and the user's scene information in the current scenario.

[0084] In one embodiment, the multi-dimensional biometrics include at least two of the user's facial features, iris features, voice features, and fingerprint features.

[0085] In some embodiments, a user's fingerprint features may be hand fingerprint features and finger vein features, and iris features may be left eye iris features and right eye iris features, etc.

[0086] In some embodiments, the method for acquiring biometric features, taking facial features and iris features as examples, can extract facial and iris images from audio and video using a face camera and an iris camera, thereby obtaining facial features and iris features. During acquisition, a single image can be acquired, or a video can be acquired and multiple frames obtained from the video.

[0087] In some embodiments, biometric features can be extracted automatically, for example, through methods such as convolutional neural networks (CNNs).

[0088] In one embodiment, the user's scenario information in the current scenario may include application type information of the user's application terminal (including the user's near-field offline application terminal and remote online application terminal), environmental information (including ambient light intensity, light source position, noise, etc.), service type information of the terminal used, user's physiological information, user's behavioral information, user's emotional information, etc.

[0089] Step S13: Determine the recognition weight values ​​of each dimension of biometric features under scene information, and determine the scene score of scene information.

[0090] In one embodiment, the recognition weight value of each dimension of biometric features refers to the reference weight value assigned by the server to the recognition results of the user's facial features, iris features, voice features, and fingerprint features based on the current scene information during user identification. The sum of each weight value is 1.

[0091] In one embodiment, the scene score is used to characterize the score of the scene information of the current scene compared to the preset optimal scene information.

[0092] The optimal scene information refers to the scene in which the server achieves the highest success rate and the highest accuracy in biometric identification. Different scene information corresponds to different success rates and accuracy rates.

[0093] In one embodiment, the server obtains a score for the current scene information by comparing the degree of difference between the current scene information and the preset optimal scene information. The greater the difference, the lower the scene score; the smaller the difference, the higher the scene score.

[0094] Step S14: The scene score and biometric score are fused together, and the user's identification result is determined based on the fusion result.

[0095] In one embodiment, the server can input the scene score and biometric score into a pre-set fusion model for fusion to obtain a fusion result. The fusion model can be a pre-set computational model or a pre-trained neural network.

[0096] In one embodiment, the biometric score is determined by the biometric features of each dimension and the corresponding recognition weight value of the dimension.

[0097] In one embodiment, the biometric score is used to characterize the score obtained by fusing the biometric recognition result obtained by the server and the recognition weight value of the corresponding dimension in the current scenario. Specifically, the higher the fused score, the higher the recognition success rate of the corresponding dimension of biometrics; conversely, the lower the fused score, the lower the recognition success rate of the corresponding dimension of biometrics.

[0098] As an example, the server calculates the user's facial feature recognition score as A, iris feature recognition score as B, voice feature recognition score as C, and fingerprint feature recognition score as D. Based on the current scene information, the server calculates the user's facial feature weight as X1, iris feature weight as X2, voice feature weight as X3, and fingerprint feature weight as X4. Then, the server calculates the user's facial feature biometric score as A×X1, iris feature biometric score as B×X2, voice feature biometric score as C×X3, and fingerprint feature biometric score as D×X4.

[0099] In the aforementioned biometric identification method, firstly, audio and video recordings of the user within the current scene are acquired. Then, based on the audio and video, multiple dimensions of the user's biometric features and scene information within the current scene are determined. These multiple dimensions of biometric features include at least two of the user's facial features, iris features, voice features, and fingerprint features. Next, the recognition weight values ​​of each dimension of biometric features under the scene information are determined, along with a scene score. The scene score represents the score of the current scene information compared to a preset optimal scene information. Finally, the scene score and biometric feature score are fused, and the user's identification result is determined based on the fusion result. The biometric feature score is determined by each dimension of biometric features and its corresponding recognition weight value. Thus, on the one hand, using scene information corresponding to the user's dynamic scene for user identification improves the accuracy of biometric identification and optimizes the user identification experience; on the other hand, using multiple dimensions of user features and fusing feature weights under the corresponding scene for user identification improves the security and reliability of biometric identification.

[0100] Those skilled in the art will understand that the methods disclosed in the above-described specific embodiments can be implemented in more concrete ways. For example, the above-described embodiments of the biometric process are merely illustrative descriptions.

[0101] For example, the process by which the server determines multiple dimensions of a user's biometric features and the user's scene information in the current scenario based on audio and video; or the process by which the terminal captures images of the user in the current scene to form audio and video, etc., is just one way of combining data. In actual implementation, there can be other ways of dividing the data. For example, the methods by which the server determines the recognition weight values ​​of each dimension of biometric features in the scene information and the methods by which the scene information is determined can be combined or integrated into another system, or some features can be ignored or not executed.

[0102] In one exemplary embodiment, see Figure 3 , Figure 3 This is a flowchart illustrating an embodiment of determining multiple dimensions of biometrics in this application. In step S12, the server determines multiple dimensions of a user's biometrics based on audio and video, which can be implemented in the following ways:

[0103] Step S121: Analyze the audio and video to obtain the user's image data and audio data.

[0104] In one embodiment, the user's audio data can be obtained by the server extracting the audio stream separately from the audio and video, or by the server controlling a recorder to collect the audio stream separately as part of the audio and video. Similarly, the user's image data can be obtained by the server extracting the video stream separately from the audio and video, or by the server controlling a camera to capture the image stream separately as part of the audio and video.

[0105] In one embodiment, the image data may include image data in formats such as bitmap, JPEG image, PNG image (portable web graphics), GIF image, JPG image, PDF image or depth map, and may also include video data in formats such as Microsoft Video, RealPlayer, MPEG video, mobile video or Apple Video.

[0106] Step S122: Extract biometric features from multiple dimensions based on image and audio data.

[0107] In some embodiments, the server needs to preprocess the image data before extracting multi-dimensional biometric features. This preprocessing includes three parts: localization, normalization, and image enhancement.

[0108] As an example, the localization of face image data uses an algorithm combining Haar features and Adaboost. Normalization of the face image data employs translation, rotation, and scaling. Histogram equalization is used for face image data enhancement. Similarly, the localization of iris image data uses an algorithm combining Hough transform and edge detection. Normalization of the iris image data uses polar coordinate transformation, and histogram equalization is used for iris image data enhancement.

[0109] In some embodiments, the method for acquiring biometric features, continuing to take facial and iris features as examples, can extract facial and iris images from audio and video using a face camera and an iris camera, thereby obtaining facial and iris features. During acquisition, a single image can be acquired, or a video can be acquired and multiple frames obtained from the video.

[0110] In some embodiments, biometric features can be extracted automatically, for example, through methods such as convolutional neural networks (CNNs).

[0111] In one exemplary embodiment, see Figure 4 , Figure 4 This is a flowchart illustrating an embodiment of determining scene information in the current scene according to this application. In step S12, the process by which the server determines the user's scene information in the current scene based on audio and video can be implemented in the following way:

[0112] Step S123: Based on image data and audio data, determine at least one category of scene factors and the factor score of each category of scene factors.

[0113] In one embodiment, the server extracts multiple scene factors from image data and audio data including the user's current scene, and calculates a corresponding factor score for each scene factor based on the user's current scene.

[0114] In some embodiments, the scene factors extracted by the server from the current scene may include: application type information of the user's application terminal (including whether the user is a near-field offline application terminal or a remote online application terminal), environmental information of the user's location (including ambient light intensity, light source position, noise, etc.), business type information of the user's terminal (including whether the user is performing account transactions, logging in, identity verification, financial fraud, etc.), physiological information of the user (including the user's gender, age, ethnicity, height, weight, etc.), behavioral information of the user (including whether the user is wearing makeup, whether their eyes are closed, whether they are making faces, whether they are wearing colored contact lenses, whether they are wearing glasses, whether they are wearing a mask, whether they are wearing a hat, etc.), and emotional information of the user (including the user's expressions of joy, anger, sorrow, and happiness).

[0115] In one embodiment, the factor score is used to characterize the score of the scene factor in the current scene compared to the preset optimal scene factor.

[0116] As an example, the scene factors extracted by the server from the current scenario include: the user is a remote online application terminal, the light intensity of the user's environment is A, the noise intensity of the environment is B, the service type of the user's terminal is identity recognition, and the user is wearing glasses, colored contact lenses, and a hat. The server's preset optimal scenario is the user being near an offline application terminal, the light intensity of the user's environment is A0, the noise intensity of the environment is B0, the service type of the user's terminal is login, and the user is not wearing glasses, colored contact lenses, or a hat. Therefore, based on preset distance calculation rules, the server calculates the following factor scores: application type of the user's terminal (X1), environment (X2), service type of the user's terminal (X3), physiological information (X4), behavioral information (X5), and emotional information (X6).

[0117] Step S124: Aggregate at least one category of scene factors and the factor scores of each category of scene factors to obtain scene information under the current scene.

[0118] In one embodiment, the server aggregates all scene factors extracted from the audio and video and their corresponding factor scores to form a scene combination. This scene combination represents the user's current scene, and as scene variables change, the aggregated scene combination can adaptively change to obtain a scene combination form suitable for the user's current scenario.

[0119] As an example, the server calculates the following factor scores: X1 for the application type of the user's terminal, X2 for the environment, X3 for the service type of the user's terminal, X4 for the user's physiological information, X5 for the user's behavioral information, and X6 for the user's emotional information. The server then aggregates these six scenario factors into a single row-column expression X, where X = [X1, X2, X3, X4, X5, X6].

[0120] In one exemplary embodiment, see Figure 5 , Figure 5 This is a flowchart illustrating an embodiment of determining the recognition weight values ​​of each biometric feature in this application. In step S13, the process by which the server determines the recognition weight values ​​of each dimension of biometric features under scene information can be implemented in the following way:

[0121] Step S131: Obtain the correlation between scene factors for each category and biological characteristics for each dimension.

[0122] In one embodiment, relevance is used to characterize the degree of influence of various categories of scene factors on the results of user identification when using biometric features of various dimensions for user identification.

[0123] In one embodiment, the correlation between each category of scene factors and each dimension of biometric features is automatically calculated by a computer program that is pre-configured by the development engineer on a server and stored in a storage medium.

[0124] As an example, a development engineer can set the correlation between the user's application terminal's application type and facial features as P1, the correlation with iris features as P2, the correlation with voice features as P3, and the correlation with fingerprint features as P4; and set the correlation between the user's environmental factors and facial features as P5, the correlation with iris features as P6, the correlation with voice features as P7, and the correlation with fingerprint features as P8. The correlation values ​​P1-P8 can be the same or different.

[0125] Step S132: Based on the factor score of each category of scene factor and the correlation of the biometric features of the corresponding dimension of each category of scene factor, determine the recognition weight value of the biometric features of each dimension.

[0126] In one embodiment, the server can determine the recognition weight value of each dimension of biometric features based on the factor scores of scene factors and the correlation between the corresponding dimensions of biometric features using a pre-set reinforcement learning model.

[0127] In some embodiments, the reinforcement learning model learns a large number of historical scene factors of various categories and their correlation with corresponding biometric features to obtain the cumulative feedback values ​​of various biometric features (i.e., the recognition weight values ​​of a large number of biometric features), and then obtains the optimal feedback value range (i.e., the optimal recognition weight value range of biometric features) for different action strategies under each initial state value (i.e., the input factor score and biometric feature correlation).

[0128] As an example, the reinforcement learning model sets all initial input parameters for each biometric feature to sp = (x1n, x2n, ..., xnn) × (y1n, y2n, ..., ynn), where sp is the biometric feature recognition weight, xnn is the initial input parameter for a factor score, ynn is the initial input parameter for the relevance of a biometric feature, and xi / yi ∈ [li, hi] is the initial input parameter for the i-th factor score. A total of n ∈ N initial input parameters need to be learned. The optimal feedback value range under different action strategies of the reinforcement learning model is rt = score(st+1) - score(st), where the step size for each parameter tuning is set to 1, i.e., a = (±1, ±1, ..., ±1), and score is the optimal feedback value range (i.e., the optimal recognition weight range for each dimension of biometric features) determined by the engineer under the current action strategy.

[0129] In one exemplary embodiment, see Figure 6 , Figure 6 This is a flowchart illustrating an embodiment of determining the scene score of scene information in this application. In step S13, the process of the server determining the scene score of the scene information can be implemented in the following way:

[0130] Step S133: Obtain the mean and variance of the factor scores for each category of scene factors.

[0131] In one embodiment, the server can calculate the mean and variance of scene factors for each category using a large amount of historical data.

[0132] As an example, the server derives the aggregate expression X of the scene factors for the user in the current scenario as: X = [X1, X2, X3, X4, X5, X6]. Then, the server retrieves from the database the mean expression P of the factor scores for each category of scene factors, calculated from a large amount of historical data: P = [P1, P2, P3, P4, P5, P6], and the variance expression б of the factor scores for each category of scene factors: б = [б1, б2, б3, б4, б5, б6].

[0133] Step S134: Calculate the probability density of scene information and determine the scene score by using the factor scores of at least one category of scene factors and the mean and variance of each category of scene factors.

[0134] In one embodiment, the server can calculate the mean and variance of scene factors for each category using a large amount of historical data.

[0135] In one embodiment, the aggregation expression X of the scene factor follows an N(P, σ) group. 2The normal distribution N(x) is given by the normal distribution N(x), where the data distribution of the normal distribution N(x) can be described by an n-dimensional column vector, and the mean P and variance б of the other dimensions can be P1, P2, P3, P4, P5, P6 and б1, б2, б3, б4, б5, б6, respectively.

[0136] Furthermore, the server inputs the factor scores of various categories of scene factors, as well as the mean and variance of each category of scene factors, into the normal distribution N(x) to calculate the probability density. Finally, the server uses the probability density value output by the normal distribution N(x) as the scene score under the user's current scene information.

[0137] In one embodiment, after determining the user's biometric features in multiple dimensions based on audio and video, the server also needs to calculate the biopsy value corresponding to each dimension of biometric features based on image data and / or audio data, as the first type of feature score.

[0138] Specifically, the server can use several biodetection algorithms to perform liveness verification on each dimension of the user's biometrics to obtain the corresponding biodetection value εi for each dimension of the biometrics. The server uses each biodetection value εi as the first type of feature score of the biometrics.

[0139] In another embodiment, after the server determines the user's biometric features in multiple dimensions based on audio and video, it also needs to calculate the feature similarity corresponding to each dimension of biometric features based on image data and / or audio data, as a second type of feature score.

[0140] Specifically, the server can use several similarity algorithms to compare the similarity of each dimension of the user's biometric features to obtain the feature similarity θi of each dimension of the biometric features. The server uses each feature similarity θi as the second type of feature score of the biometric features.

[0141] Feature similarity is used to characterize the degree of similarity between a user's biometric features and preset template features.

[0142] In one embodiment, after the server determines the user's biometric features in multiple dimensions based on audio and video, it also needs to determine the biometric feature score based on the first type feature score and the second type feature score corresponding to each biometric feature, as well as the recognition weight value of each biometric feature in the context information.

[0143] As an example, suppose the first category of feature scores corresponding to the biometric features are [ε1, ε2, ε3…εn]. T The first category of features corresponding to the biological characteristics are scored as [θ1, θ2, θ3…θn]. TThe recognition weight of each dimension of biometric features under scene information is [P1, P2, P3...Pn]. T Then, the server fuses the first type of feature score with the recognition weight value to obtain the first fused score L(x) (i.e., LiveValu(x)), where L(x) = [ε1, ε2, ε3…εn]. T ×[P1, P2, P3…Pn] T The server combines the second type of feature score with the recognition weight value to obtain the second fusion score F(x) (i.e., Feature(x)), where L(x) = [θ1, θ2, θ3…θn]. T ×[P1, P2, P3…Pn] T Finally, the server introduces a preset error parameter α for the first fusion score L(x) and a preset error parameter β for the second fusion score F(x). The first and second fusion scores with the error parameters are then fused to obtain the biometric score P(x), i.e., P(x) = α × L(x) + β × F(x).

[0144] In one embodiment, the server fuses the scene score and the biometric score by adding the scene score N(x) and the biometric score P(x) to obtain the fusion optimization score R(x).

[0145] In one embodiment, the server determines the user's recognition result based on the fusion result, including: if the score corresponding to the fusion result is less than a preset fusion score, the user's recognition result is recognition failure; or if the score corresponding to the fusion result is greater than or equal to the preset fusion score, the user's recognition result is recognition success.

[0146] As an example, if the fusion optimization score R(x) corresponding to the fusion result is less than the preset fusion score Sp, the user's recognition result is recognition failure; or if the fusion optimization score R(x) corresponding to the fusion result is greater than or equal to the preset fusion score Sp, the user's recognition result is recognition success.

[0147] In one exemplary embodiment, see Figure 7 , Figure 7 This is a flowchart illustrating an embodiment of re-identifying a user's biometrics in this application. After the server determines the user's identification result based on the fusion result in step S14, there is still a process where the server needs to re-identify the user's biometrics. This can be implemented in the following ways:

[0148] Step a1: If the recognition result is recognition failure, an adjustment prompt is issued so that the user can adjust the current scene according to the adjustment prompt.

[0149] In one embodiment, the adjustment prompt issued by the server is used to prompt the user to adjust the scene factors and their scores. This adjustment prompt can be a voice prompt, a video prompt, or a prompt to assist in calling staff.

[0150] Step a2: Re-perform biometric identification on the user based on the adjusted current scenario to obtain a new identification result for the user.

[0151] In one embodiment, after the customer adjusts their scene factors and factor scores according to the adjustment prompts, the server re-acquires the audio and video footage of the user in the current scene based on the method described in the above embodiment, and performs subsequent biometric identification. If the re-biometric identification still fails, identification continues until the user's scene factors and factor scores reach a level that allows for successful identification.

[0152] To more clearly illustrate the biometric method provided in the embodiments of this disclosure, the following specific embodiment will be used to describe the biometric method in detail. In an exemplary embodiment, reference is made to... Figure 8 and Figure 9 , Figure 8 This is a flowchart illustrating a biometric method according to another exemplary embodiment. Figure 9 This is a block diagram illustrating a biometric method according to another exemplary embodiment. The biometric method is used in server 104 and specifically includes the following:

[0153] Step 21: Use the audio and video acquisition device through the information acquisition module 001 to collect biometric information of living personnel.

[0154] The collected biometric information includes facial information, iris information, and voiceprint information.

[0155] Step 22: The collected biometric information is extracted and identified by the data construction module 002 to generate multi-dimensional feature information of sound image.

[0156] Among them, the multidimensional feature information of voice image includes: facial feature information, iris feature information, and voiceprint feature information.

[0157] Step 23: The adaptive scene module 003 adaptively adjusts the scene factors based on the user's scene information to simulate the scores of each scene factor in the current scene, so as to determine the scene combination formula X.

[0158] Commonly used scenario information includes:

[0159] Business scenario information: On-site recognition or remote recognition;

[0160] User environment information: light intensity, light source location, etc.;

[0161] Business process information: whether it is a transaction involving account activity, login, identity verification, financial fraud, etc.;

[0162] User physiological information: gender, age, ethnicity, height, weight, etc.;

[0163] User behavior information: whether they wear makeup, whether they close their eyes, whether they make funny faces, and whether they wear colored contact lenses, glasses, masks, hats, etc.

[0164] User emotional information: joy, anger, sorrow, and happiness.

[0165] Among them, the adaptive scene module 003 adaptively adjusts the score of the scene factor Xi by simulating the current scene. Each score of Xi represents the above 1-6 scenes. For example, a score of 0-9 can be obtained according to the different light intensity to obtain the scene combination X.

[0166] The expression for the scene combination X is: x = [x1, x2, ..., x...]. n ] T .

[0167] Step 24: Using the modality selection module 004 and scene combination X, calculate the weight ratio for recognizing each biometric feature and the scene score of scene combination X.

[0168] The modality selection module 004 inputs the scene combination X into the weight factor Param(w) function for calculation, and then the weight factor Param(w) function outputs the weight ratio of the three recognition methods: face recognition, voiceprint recognition, and iris recognition.

[0169] For example, in remote identity recognition, the weight of iris can be reduced and the weight of voiceprint can be increased. The weight ratio of each biometric feature can be set to voiceprint:face:iris = 7:2:1. In close-range (usually 40CM, the maximum working distance of mainstream iris recognition devices on the market) on-site identity recognition, the weight of iris can be increased and the weight of face can be reduced. The weight ratio of each biometric feature can be set to voiceprint:face:iris = 2:2:6.

[0170] The modality selection module 004 inputs the scene combination X into the scene adaptation function N(x) for calculation, and then the scene adaptation function N(x) outputs the scene score under this scene combination.

[0171] Wherein, the scene combination X follows N(μ,σ) 2 The data distribution of scene combination X follows a normal distribution. It can be described by an n-dimensional column vector, where the mean and variance of each type of scene factor Xi can be expressed as μ1, μ2, ..., μ...n and σ1,σ2,...,σ n To express.

[0172] in, The normal distribution can be characterized by the expression of the scene adaptation function N(x), which is the Normal Distribution(x).

[0173] in,

[0174] The scene combination X is input into the scene adaptation function N(x) for calculation, and then the scene adaptation function N(x) outputs the scene score under this scene combination.

[0175] Step 25: The biopsy value and feature similarity of each single-modal biometric feature are fused with the recognition weight ratio corresponding to each biometric feature through the modality fusion module 005 to obtain the biopsy fusion score and the similarity fusion score.

[0176] In this process, the server identifies the real-time collected biometric information at the front end and calculates the biopsy value of each single-modality biometric εi. Then, the biopsy value of each single-modality biometric εi is fused with the weight ratio output by the weight factor Param(w) function to obtain the fused biopsy fusion score L(x).

[0177] In this process, the server performs feature comparison on the acquired multidimensional feature information in the backend, and calculates the feature similarity of each single-modal biometric feature εi. Then, the feature similarity of each single-modal biometric feature εi is fused with the weight ratio output by the weight factor Param(w) function to obtain the fused similarity score F(x).

[0178] The biopsy fusion score L(x) represents the fusion value calculated for biopsies under scene factor x. The calculation function for the biopsy fusion score L(x) is as follows:

[0179] L(x) = [L1, L2, ..., L n ] T =[ε1,ε2,...,ε n ]×Paras L (w)

[0180] Paras L (w) = [w1, w2, ..., w k ] T

[0181] In practical applications, it is necessary to ensure that n = k, which means that a row and column transformation to fill with 1s is required.

[0182] The similarity fusion score F(x) represents the fusion value calculated for features under scene factor x. The calculation function for the similarity fusion score F(x) is as follows:

[0183] F(x) = [F1, F2, ..., F n ] T =[θ1,θ2,...,θ n ]×Paras F (w)

[0184] Paras F (w) = [w1, w2, ..., w k ] T

[0185] Step 26: The error parameters α and β are introduced into the live detection fusion score L(x) and the similarity fusion score F(x) respectively through the model optimization module 006 to obtain the fusion score P(x). Then, the fusion score P(x) is mapped to the adaptive scene function N(x) to obtain the final optimization score R(x).

[0186] Among them, the error parameters α and β have a mapping relationship with L(x) and F(x).

[0187] The calculation function for the fusion score P(x) is as follows:

[0188] P(x) = α × L(x) + β × F(x)

[0189] The final optimization score R(x) is calculated using the following function:

[0190]

[0191] Step 27: Through the result return module 007, output the score (accuracy) corresponding to the fusion score P(x) based on the optimization score R(x) and the scene score corresponding to the scene adaptive function N(x).

[0192] As an example, under scene combination X, the user obtains the final recognition score P(X) = 0.965 and the scene score N(x) = 0.924 for scene combination X.

[0193] It should be understood that, although Figures 2-9 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 2-9At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0194] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.

[0195] Figure 10 This is a block diagram illustrating a biometric identification method apparatus according to an exemplary embodiment. (Refer to...) Figure 10 The device 10 includes an image acquisition unit 11, a first processing unit 12, a second processing unit 13, and a fusion recognition unit 14.

[0196] The image acquisition unit 11 is configured to acquire images formed by taking pictures of users in the current scene.

[0197] The first processing unit 12 is configured to perform the following based on the audio and video: determine the user's biometric features in multiple dimensions and the user's scene information in the current scene; the biometric features in multiple dimensions include at least two of the user's facial features, iris features, voice features and fingerprint features.

[0198] The second processing unit 13 is configured to determine the recognition weight values ​​of the biometric features of each dimension under the scene information, and to determine the scene score of the scene information; the scene score is used to characterize the score of the scene information of the current scene compared with the preset optimal scene information.

[0199] The fusion recognition unit 14 is configured to perform the fusion of the scene score and the biometric score, and determine the user's recognition result based on the fusion result; the biometric score is determined by the biometric features of each dimension and the recognition weight value of the corresponding dimension.

[0200] In one exemplary embodiment, the first processing unit 12 is further configured to perform parsing of the audio and video to obtain the user's image data and audio data;

[0201] Based on the image data and the audio data, the biometric features of the multiple dimensions are extracted.

[0202] In an exemplary embodiment, the first processing unit 12 is further configured to perform the following: determining at least one category of scene factors and factor scores for each category of scene factors based on the image data and the audio data; the factor scores are used to characterize the score of the scene factors in the current scene compared to a preset optimal scene factor.

[0203] By aggregating the scene factors of at least one category and the factor scores of the scene factors of each category, the scene information under the current scene is obtained.

[0204] In an exemplary embodiment, the second processing unit 13 is further configured to perform the determination of the recognition weight values ​​of the biometric features of each dimension under the scene information, including:

[0205] Obtain the correlation between scene factors of each category and biometric features of each dimension; the correlation is used to characterize the degree of influence of scene factors of each category on the user identification result when using biometric features of each dimension for user identification.

[0206] Based on the factor score of each category of scene factors and the correlation of the biometric features of the corresponding dimensions of each category of scene factors, the recognition weight value of the biometric features of each dimension is determined.

[0207] The categories of the scene factors include at least one of the following: business form category, business process category, user environment category, user physiological category, user behavior category, and user emotion category when the user is being photographed.

[0208] In one exemplary embodiment, the second processing unit 13 is further configured to perform the acquisition of the mean and variance values ​​of the scene factors for each category;

[0209] The probability density of the scene information is calculated using the factor scores of at least one category of scene factors and the mean and variance of each category of scene factors, and the scene score is determined.

[0210] In an exemplary embodiment, the fusion recognition unit 14 is further configured to perform the following: calculating the biopsy value corresponding to each dimension of biometrics based on the image data and / or the audio data, as a first-class feature score; and

[0211] Based on the image data and / or the audio data, the feature similarity corresponding to each dimension of biometric features is calculated as a second type of feature score; the feature similarity is used to characterize the degree of similarity between the user's biometric features and preset template features.

[0212] In an exemplary embodiment, the fusion recognition unit 14 is further configured to determine the biometric score based on the first type of feature score and the second type of feature score corresponding to each dimension of biometric features, and the recognition weight value of each dimension of biometric features under the scene information.

[0213] In an exemplary embodiment, the fusion recognition unit 14 is further configured to execute the following: if the score corresponding to the fusion result is less than a preset fusion score, then the user's recognition result is a recognition failure.

[0214] If the score corresponding to the fusion result is greater than or equal to the preset fusion score, then the user's recognition result is successful.

[0215] In an exemplary embodiment, the fusion recognition unit 14 is further configured to issue an adjustment prompt if the recognition result is a recognition failure, so that the user can adjust the user's current scene according to the adjustment prompt;

[0216] Based on the adjusted current scenario, the user is re-identified biometrically, resulting in a new identification result.

[0217] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0218] Figure 11 This is a block diagram illustrating an electronic device 20 for biometric identification according to an exemplary embodiment. For example, the electronic device 20 can be a server. (Refer to...) Figure 11 The electronic device 20 includes a processing component 21, which further includes one or more processors, and memory resources represented by memory 22 for storing executable instructions, such as application programs, that can be executed by the processing component 21. The application programs stored in memory 22 may include one or more modules, each corresponding to a set of executable instructions. Furthermore, the processing component 21 is configured to execute the executable instructions to perform the methods described above.

[0219] In one embodiment, the electronic device 20 is a server, and the computing system within the server can run one or more operating systems, including any of the operating systems discussed above and any commercially available server operating system. The server can also run any of a variety of additional server applications and / or middleware applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, database servers, etc. Exemplary database servers include, but are not limited to, commercially available database servers from companies such as IBM.

[0220] In one embodiment, the processing component 21 typically controls the overall operation of the electronic device 20, such as operations associated with display, data processing, data communication, and recording operations. The processing component 21 may include one or more processors to execute instructions to perform all or part of the steps of the methods described above. Furthermore, the processing component 21 may include one or more modules to facilitate interaction between the processing component 21 and other components. For example, the processing component 21 may include a multimedia module to facilitate control of the interaction between the user terminal and the processing component 21 using multimedia components.

[0221] In one embodiment, the processor in processing component 21 may also be referred to as a CPU (Central Processing Unit). The processor may be an electronic chip with signal processing capabilities. The processor may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, the processor may be implemented using integrated circuit chips.

[0222] In one embodiment, memory 22 is configured to store various types of data to support the operation of electronic device 20. Examples of such data include instructions for any application or method operating on electronic device 20, acquired data, messages, images, videos, etc. Memory 22 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, optical disk, or graphene memory.

[0223] In one embodiment, the memory 22 can be a memory module, TF card, etc., and can store all the information in the electronic device 20, including the input raw data, computer programs, intermediate running results, and final running results. It stores and retrieves information according to the location specified by the processor. With the memory 22 in this embodiment, the electronic device 20 has a memory function and can ensure normal operation. In one embodiment of the electronic device 20, the memory 22 can be classified according to its purpose as main memory (RAM) and auxiliary memory (external memory), or it can be classified as external memory and internal memory. External memory is usually magnetic media or optical discs, which can store information for a long time. RAM refers to the storage components on the motherboard, used to store currently executing data and programs, but it is only used for temporary storage of programs and data; the data will be lost when the power is turned off or disconnected.

[0224] Electronic device 20 may further include: a power supply component 23 configured to perform power management of electronic device 20, a wired or wireless network interface 24 configured to connect electronic device 20 to a network, and an input / output (I / O) interface 25. Electronic device 20 may operate on an operating system stored in memory 22, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or similar.

[0225] In one embodiment, power supply component 23 provides power to various components of electronic device 20. Power supply component 23 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 20.

[0226] In one embodiment, the wired or wireless network interface 24 is configured to facilitate wired or wireless communication between the electronic device 20 and other devices. The electronic device 20 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof.

[0227] In one exemplary embodiment, the wired or wireless network interface 24 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In another exemplary embodiment, the wired or wireless network interface 24 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0228] In one embodiment, the input / output (I / O) interface 25 provides an interface between the processing component 21 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a power button, and a lock button.

[0229] Figure 12 This is a block diagram illustrating a computer-readable storage medium 30 for biometric identification according to an exemplary embodiment. The computer-readable storage medium 30 stores program data 31 capable of implementing the methods described above.

[0230] If the integrated units of the various functional units in the various embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium 30. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer-readable storage medium 30 includes a program data 31, which includes several instructions to cause a computer device (which may be a personal computer, system server, or network device, etc.), an electronic device (e.g., MP3, MP4, etc., or a mobile phone, tablet computer, wearable device, etc., or a desktop computer, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of this application.

[0231] Figure 13 This is a block diagram illustrating a computer program product 40 for biometric identification according to an exemplary embodiment. The computer program product 40 includes program instructions 41, and the program data can be executed by the processor of the electronic device 20 to perform the described method.

[0232] Those skilled in the art will understand that embodiments of this application can be provided as a method for biometric identification, an electronic resource verification device 10, an electronic device 20, a computer-readable storage medium 30, or a computer program product 40. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product 40 embodied on one or more computer program instructions 41 (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0233] This application is described with reference to flowchart illustrations and / or block diagrams of a biometric identification method, an electronic resource verification device 10, an electronic device 20, a computer-readable storage medium 30, or a computer program product 40 according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by the computer program product 40. These computer program products 40 can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that program instructions 41, executable by the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0234] These computer program products 40 may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing device to function in a particular manner, such that program instructions 41 stored in the computer program product 40 produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0235] These program instructions 41 may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing the program instructions 41 that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0236] It should be noted that the various methods, apparatuses, electronic devices, computer-readable storage media, computer program products, etc. described above may also include other implementation methods according to the description of the method embodiments. For specific implementation methods, please refer to the description of the relevant method embodiments, which will not be elaborated here.

[0237] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0238] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A biometric identification method, characterized in that, The method includes: Acquire audio and video recordings of users within the current scene; Based on the audio and video, determine the user's biometric features in multiple dimensions and the user's scene information in the current scene; the biometric features in multiple dimensions include at least two of the user's facial features, iris features, voice features and fingerprint features; The recognition weight values ​​of the biometric features of each dimension under the scene information are determined, and the scene score of the scene information is determined; the scene score is used to characterize the score of the scene information of the current scene compared with the preset optimal scene information. The scene score and biometric score are fused together, and the user's identification result is determined based on the fusion result; the biometric score is determined by the biometric features of each dimension and the identification weight value of the corresponding dimension; Determining the user's scene information in the current scenario based on the audio and video includes: Based on the audio and video, at least one category of scene factors is determined, and the factor score of each category of scene factors is determined; the factor score is used to characterize the score of the scene factors in the current scene compared with the preset optimal scene factors; By aggregating the scene factors of at least one category and the factor scores of the scene factors of each category, the scene information under the current scene is obtained; The categories of the scene factors include at least one of the following: business form category, business process category, user environment category, user physiological category, user behavior category, and user emotion category when the user is being photographed; Accordingly, determining the recognition weight values ​​of the biometric features of each dimension under the scene information includes: Obtain the correlation between scene factors of each category and biometric features of each dimension; the correlation is used to characterize the degree of influence of scene factors of each category on the user identification result when using biometric features of each dimension for user identification. Based on the factor score of each category of scene factors and the correlation of the biometric features of the corresponding dimensions of each category of scene factors, the recognition weight value of the biometric features of each dimension is determined.

2. The method according to claim 1, characterized in that, The determination of the user's biometric characteristics across multiple dimensions based on the audio and video includes: The audio and video data are analyzed to obtain the user's image and audio data; Based on the image data and the audio data, the biometric features of the multiple dimensions are extracted.

3. The method according to claim 1, characterized in that, Determining the scene score of the scene information includes: Obtain the mean and variance of the factor scores for each category of scene factors; The probability density of the scene information is calculated using the factor scores of at least one category of scene factors and the mean and variance of each category of scene factors, and the scene score is determined.

4. The method according to claim 2, characterized in that, After determining the user's biometric characteristics across multiple dimensions based on the audio and video, the process further includes: Based on the image data and / or the audio data, calculate the biopsy value corresponding to each dimension of the biometric feature, as the first type of feature score; and Based on the image data and / or the audio data, the feature similarity corresponding to each dimension of biometric features is calculated as the second type of feature score; the feature similarity is used to characterize the degree of similarity between the user's biometric features and the preset template features; After determining the recognition weight values ​​of the biometric features of each dimension under the scene information, the method further includes: The biometric score is determined based on the first and second feature scores corresponding to each dimension of biometric features, and the recognition weight value of each dimension of biometric features under the scene information.

5. The method according to any one of claims 1 to 4, characterized in that, The process of determining the user's identification result based on the fusion result includes: If the score corresponding to the fusion result is less than the preset fusion score, then the user's recognition result is recognition failure; If the score corresponding to the fusion result is greater than or equal to the preset fusion score, then the user's recognition result is successful.

6. The method according to claim 5, characterized in that, After determining the user's identification result based on the fusion result, the method further includes: If the recognition result is a recognition failure, an adjustment prompt will be issued so that the user can adjust the user's current scene according to the adjustment prompt; Based on the adjusted current scenario, the user is re-identified biometrically, resulting in a new identification result.

7. A biometric identification device, characterized in that, include: The image acquisition unit is configured to acquire images of users within the current scene. The first processing unit is configured to perform an action based on audio and video to determine the user's biometric features in multiple dimensions and the user's scene information in the current scene; the biometric features in multiple dimensions include at least two of the user's facial features, iris features, voice features and fingerprint features; The second processing unit is configured to determine the recognition weight values ​​of the biometric features of each dimension under the scene information, and to determine the scene score of the scene information; The scene score is used to characterize the score of the scene information of the current scene compared to the preset optimal scene information; The fusion recognition unit is configured to fuse the scene score and the biometric score, and determine the user's recognition result based on the fusion result; the biometric score is determined by the biometric features of each dimension and the recognition weight value of the corresponding dimension; The first processing unit is further configured to: Based on the audio and video, at least one category of scene factors is determined, and the factor score of each category of scene factors is determined; the factor score is used to characterize the score of the scene factors in the current scene compared with the preset optimal scene factors; By aggregating the scene factors of at least one category and the factor scores of the scene factors of each category, the scene information under the current scene is obtained; The categories of the scene factors include at least one of the following: business form category, business process category, user environment category, user physiological category, user behavior category, and user emotion category when the user is being photographed; The second processing unit is further configured to: acquire the correlation between scene factors of each category and biometric features of each dimension; the correlation is used to characterize the degree of influence of scene factors of each category on the result of user identification when using biometric features of each dimension. Based on the factor score of each category of scene factors and the correlation of the biometric features of the corresponding dimensions of each category of scene factors, the recognition weight value of the biometric features of each dimension is determined.

8. An electronic device, characterized in that, include: processor; Memory for storing the executable instructions of the processor; The processor is configured to execute the executable instructions to implement the biometric method as described in any one of claims 1 to 6.

9. A computer-readable storage medium comprising program data, characterized in that, When the program data is executed by the processor of the electronic device, the electronic device is able to perform the biometric method as described in any one of claims 1 to 6.

10. A computer program product, the computer program product comprising program instructions, characterized in that, When the program instructions are executed by the processor of the electronic device, the electronic device is enabled to perform the biometric method as described in any one of claims 1 to 6.

Images