Live detection method, live detection model training method and device

Abstract

This disclosure provides a liveness detection method, a training method for a liveness detection model, and an apparatus, comprising: in response to a liveness detection task being triggered, acquiring user attack risk information of the target user triggering the task and device attack risk information of the target device triggering the task; determining the liveness category of the task based on the user attack risk information and the device attack risk information, wherein the liveness category is either liveness or attack. This avoids the disadvantage of low hardware cost caused by liveness detection methods based on multimodal face images, which is beneficial for saving hardware costs. It also avoids the disadvantage of detection efficiency caused by liveness detection methods based on multi-frame images of multi-action interactions, which is beneficial for improving detection efficiency. Furthermore, it can improve the accuracy and reliability of liveness detection.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology and can be applied to image recognition, particularly to a liveness detection method, a training method for a liveness detection model, and a device. Background Technology

[0002] Facial recognition technology has rapidly become one of the main methods of identity authentication due to its convenience. However, as a relatively new technology, it also faces many new security threats. Among these, liveness detection is a common and high-risk threat, which can be mitigated through liveness detection methods.

[0003] In related technologies, there are two main types of liveness detection methods: one is based on multimodal face images, and the other is based on multi-frame images with multi-action interactions.

[0004] However, liveness detection methods based on multimodal face images rely on multimodal camera modules, which have the disadvantage of high hardware costs; liveness detection methods based on multi-frame images of multi-action interactions require multiple user interactions, which have the disadvantage of low efficiency.

[0005] It should be noted that the content in the above-mentioned related technologies is merely information known only to the inventor and does not mean that the above information had entered the public domain before the date of this application, nor does it mean that it can be considered prior art in this disclosure. Summary of the Invention

[0006] This disclosure provides a liveness detection method, a liveness detection model training method, and an apparatus to avoid at least one of the aforementioned drawbacks.

[0007] In a first aspect, this disclosure provides a method for detecting live organisms, the method comprising:

[0008] In response to the liveness detection task being triggered, user attack risk information of the target user who triggered the task and device attack risk information of the target device that triggered the task are obtained.

[0009] Based on the user attack risk information and the device attack risk information, the liveness category of the task is determined, wherein the liveness category is either live or attack.

[0010] In some embodiments, the target user has target user characteristics; the user attack risk information is determined based on the target user characteristics;

[0011] The target device has target device characteristics; the device attack risk information is determined based on the target device characteristics.

[0012] In some embodiments, the target user characteristics and the target device characteristics are determined from a pre-built user device relationship graph;

[0013] The user device relationship diagram includes user nodes and device nodes. The user nodes include user features, and the device nodes include device features. The user nodes include nodes of the target user, and the device nodes include nodes of the target device. The user features include the target user features, and the device features include the target device features.

[0014] In some embodiments, the user device relationship graph further includes edges between the user nodes and the device nodes; the user device relationship graph is constructed based on acquired sample data; the sample data includes user characteristics, device characteristics, and user behavior information of the user using the device;

[0015] The user node is constructed based on the user, the device node is constructed based on the device, and the edge is constructed based on the usage behavior information.

[0016] In some embodiments, the task has a type attribute, which is either a login type or an authentication type; the user device relationship diagram includes a user device login relationship diagram and a user device authentication relationship diagram.

[0017] If the type attribute is the login type, then the target user characteristics and the target device characteristics are determined from the user device login relationship graph;

[0018] If the type attribute is the verification type, then the target user feature and the target device feature are determined from the user device verification relationship graph.

[0019] In some embodiments, the user attack risk information is obtained by performing attack risk prediction processing on the target user characteristics based on a pre-trained user risk perception model.

[0020] The device attack risk information is obtained by performing attack risk prediction processing on the characteristics of the target device based on a pre-trained device risk perception model.

[0021] In some embodiments, the target user feature has a target user class attribute, and the target device feature has a target device class attribute; the user risk perception model includes a risk perception model corresponding to each user class attribute, and the user class attribute includes the target user class attribute; the device risk perception model includes a risk perception model corresponding to each device class attribute, and the device class attribute includes the target device class attribute;

[0022] The user attack risk information is obtained by performing attack risk prediction processing on the target user characteristics based on the risk perception model corresponding to the target user class attribute in the user risk perception model.

[0023] The device attack risk information is obtained by performing attack risk prediction processing on the characteristics of the target device based on the risk perception model corresponding to the target device class attribute in the device risk perception model.

[0024] In some embodiments, the user risk perception model is obtained by clustering the user features to obtain user class attributes, and training the user features corresponding to each user class attribute.

[0025] The equipment risk perception model is obtained by clustering the equipment features to obtain equipment class attributes, and training the equipment features corresponding to each equipment class attribute.

[0026] In some embodiments, the user features include user features in the user device login relationship graph and user features in the user device verification relationship graph; the device features include device features in the user device login relationship graph and device features in the user device verification relationship graph.

[0027] The user type attributes are obtained by fusing the user features of the user in the user device login relationship graph and the user features of the user in the user device verification relationship graph for each user, and then performing clustering processing on the user fusion features corresponding to each user.

[0028] The device type attributes are obtained by fusing the device features of the device in the user device login relationship diagram and the device features of the device in the user device verification relationship diagram for each device, and then performing clustering processing on the device fusion features corresponding to each device.

[0029] In some embodiments, the method further includes:

[0030] The acquired target image that triggers the task is subjected to attack risk prediction processing to obtain image attack risk information;

[0031] In addition, determining the liveness category of the task based on the user attack risk information and the device attack risk information includes: determining the liveness category based on the image attack risk information, the user attack risk information, and the device attack risk information.

[0032] In some embodiments, the image attack risk information includes the image attack probability, the user attack risk information includes the number of user attacks, and the device attack risk information includes the number of device attacks; determining the liveness category based on the image attack risk information, the user attack risk information, and the device attack risk information includes:

[0033] The total number of attacks is calculated based on the number of user attacks and the number of device attacks.

[0034] The liveness category is determined based on the total number of attacks and the image attack probability.

[0035] In some embodiments, determining the liveness category based on the total number of attacks and the image attack probability includes:

[0036] Determine the predicted attack probability corresponding to the total number of attacks;

[0037] The total attack probability is calculated based on the predicted attack probability and the image attack probability.

[0038] If the total probability of the attack reaches a preset threshold, the liveness category is determined as an attack; if the total probability of the attack does not reach the preset threshold, the liveness category is determined as a liveness.

[0039] Secondly, this disclosure provides a training method for a liveness detection model, the method comprising:

[0040] Obtain user characteristics and device characteristics from a pre-defined user device relationship diagram;

[0041] A user risk perception model is trained based on the user characteristics, and the user risk perception model is used to predict the user attack risk information of the target user in the liveness detection task.

[0042] A device risk perception model is trained based on the device characteristics, and the device risk perception model is used to predict the device attack risk information of the target device in the task.

[0043] The user attack risk information and the device attack risk information are used to determine the liveness category of the task; the liveness detection model includes the user risk perception model and the device risk perception model.

[0044] In some embodiments, the user device relationship graph includes user nodes, device nodes, and edges between the user nodes and the device nodes. The user nodes include the user features, the device nodes include the device features, and the edges are used to characterize user behavior information when using the device.

[0045] The user features and the device features are obtained by optimizing adjacent nodes in the user-device relationship graph, where adjacent nodes are user nodes and device nodes connected by the edges.

[0046] In some embodiments, the user features and the device features are determined based on the multivariate graph feature prediction model, which is based on the user-device relationship graph to predict the features corresponding to each node and is trained according to the feature difference information between adjacent nodes.

[0047] In some embodiments, a user risk perception model is trained based on the user characteristics, including:

[0048] The user features are clustered to obtain the attributes of each user class;

[0049] For each user class attribute, a risk perception model corresponding to that user class attribute is trained based on the user characteristics of that user class attribute.

[0050] The user risk perception model includes risk perception models corresponding to each user class attribute.

[0051] In some embodiments, the user equipment relationship graph includes a user equipment login relationship graph and a user equipment verification relationship graph; the user features are obtained by fusing user features in the user equipment login relationship graph and user features in the user equipment verification relationship graph.

[0052] In some embodiments, a device risk perception model is trained based on the device characteristics, including:

[0053] The device features are clustered to obtain the attributes of each device class;

[0054] For each equipment class attribute, a risk perception model corresponding to that equipment class attribute is trained based on the equipment characteristics of that equipment class attribute.

[0055] The equipment risk perception model includes risk perception models corresponding to each equipment class attribute.

[0056] In some embodiments, the user equipment relationship graph includes a user equipment login relationship graph and a user equipment verification relationship graph; the device feature is a feature obtained by fusing the device features in the user equipment login relationship graph and the device features in the user equipment verification relationship graph.

[0057] In some embodiments, a user risk perception model is trained based on the user characteristics, including:

[0058] The user characteristics are input into the first network model, and the predicted user attack risk information is output.

[0059] Calculate the first prediction loss between the predicted user attack risk information and the preset user attack truth value, and generate the user risk perception model based on the first prediction loss.

[0060] In some embodiments, a device risk perception model is trained based on the device characteristics, including:

[0061] The device characteristics are input into the second network model, which outputs predicted device attack risk information.

[0062] Calculate the second prediction loss between the predicted device attack risk information and the preset device attack truth value, and generate the device risk perception model based on the second prediction loss.

[0063] Thirdly, this disclosure provides a liveness detection device, the device comprising:

[0064] The acquisition unit is configured to acquire user attack risk information of the target user who triggered the task and device attack risk information of the target device that triggered the task in response to the triggering of the liveness detection task.

[0065] The determining unit is configured to determine the liveness category of the task based on the user attack risk information and the device attack risk information, wherein the liveness category is either live or attack.

[0066] In some embodiments, the target user has target user characteristics; the user attack risk information is determined based on the target user characteristics;

[0067] The target device has target device characteristics; the device attack risk information is determined based on the target device characteristics.

[0068] In some embodiments, the target user characteristics and the target device characteristics are determined from a pre-built user device relationship graph;

[0069] The user device relationship diagram includes user nodes and device nodes. The user nodes include user features, and the device nodes include device features. The user nodes include nodes of the target user, and the device nodes include nodes of the target device. The user features include the target user features, and the device features include the target device features.

[0070] In some embodiments, the user device relationship graph further includes edges between the user nodes and the device nodes; the user device relationship graph is constructed based on acquired sample data; the sample data includes user characteristics, device characteristics, and user behavior information of the user using the device;

[0071] The user node is constructed based on the user, the device node is constructed based on the device, and the edge is constructed based on the usage behavior information.

[0072] In some embodiments, the task has a type attribute, which is either a login type or an authentication type; the user device relationship diagram includes a user device login relationship diagram and a user device authentication relationship diagram.

[0073] If the type attribute is the login type, then the target user characteristics and the target device characteristics are determined from the user device login relationship graph;

[0074] If the type attribute is the verification type, then the target user feature and the target device feature are determined from the user device verification relationship graph.

[0075] In some embodiments, the user attack risk information is obtained by performing attack risk prediction processing on the target user characteristics based on a pre-trained user risk perception model.

[0076] The device attack risk information is obtained by performing attack risk prediction processing on the characteristics of the target device based on a pre-trained device risk perception model.

[0077] In some embodiments, the target user feature has a target user class attribute, and the target device feature has a target device class attribute; the user risk perception model includes a risk perception model corresponding to each user class attribute, and the user class attribute includes the target user class attribute; the device risk perception model includes a risk perception model corresponding to each device class attribute, and the device class attribute includes the target device class attribute;

[0078] The user attack risk information is obtained by performing attack risk prediction processing on the target user characteristics based on the risk perception model corresponding to the target user class attribute in the user risk perception model.

[0079] The device attack risk information is obtained by performing attack risk prediction processing on the characteristics of the target device based on the risk perception model corresponding to the target device class attribute in the device risk perception model.

[0080] In some embodiments, the user risk perception model is obtained by clustering the user features to obtain user class attributes, and training the user features corresponding to each user class attribute.

[0081] The equipment risk perception model is obtained by clustering the equipment features to obtain equipment class attributes, and training the equipment features corresponding to each equipment class attribute.

[0082] In some embodiments, the user features include user features in the user device login relationship graph and user features in the user device verification relationship graph; the device features include device features in the user device login relationship graph and device features in the user device verification relationship graph.

[0083] The user type attributes are obtained by fusing the user features of the user in the user device login relationship graph and the user features of the user in the user device verification relationship graph for each user, and then performing clustering processing on the user fusion features corresponding to each user.

[0084] The device type attributes are obtained by fusing the device features of the device in the user device login relationship diagram and the device features of the device in the user device verification relationship diagram for each device, and then performing clustering processing on the device fusion features corresponding to each device.

[0085] In some embodiments, the apparatus further includes:

[0086] The prediction unit is used to perform attack risk prediction processing on the acquired target image that triggers the task, and obtain image attack risk information.

[0087] Furthermore, the determining unit is configured to determine the liveness category based on the image attack risk information, the user attack risk information, and the device attack risk information.

[0088] In some embodiments, the image attack risk information includes the image attack probability, the user attack risk information includes the number of user attacks, and the device attack risk information includes the number of device attacks; the determining unit includes:

[0089] The calculation subunit is used to calculate the total number of attacks based on the number of user attacks and the number of device attacks;

[0090] A subunit is defined to determine the liveness category based on the total number of attacks and the image attack probability.

[0091] In some embodiments, the determining subunit includes:

[0092] The first determining module is used to determine the predicted attack probability corresponding to the total number of attacks;

[0093] The calculation module is used to calculate the total attack probability based on the predicted attack probability and the image attack probability;

[0094] The second determining module is used to determine the liveness category as an attack if the total probability of the attack reaches a preset threshold, and to determine the liveness category as a liveness if the total probability of the attack does not reach the preset threshold.

[0095] Fourthly, this disclosure provides a training device for a liveness detection model, the device comprising:

[0096] The acquisition unit is used to acquire user features and device features from a preset user device relationship diagram;

[0097] The first training unit is used to train a user risk perception model based on the user characteristics. The user risk perception model is used to predict the user attack risk information of the target user in the liveness detection task.

[0098] The second training unit is used to train a device risk perception model based on the device characteristics. The device risk perception model is used to predict the device attack risk information of the target device in the task.

[0099] The user attack risk information and the device attack risk information are used to determine the liveness category of the task; the liveness detection model includes the user risk perception model and the device risk perception model.

[0100] In some embodiments, the user device relationship graph includes user nodes, device nodes, and edges between the user nodes and the device nodes. The user nodes include the user features, the device nodes include the device features, and the edges are used to characterize user behavior information when using the device.

[0101] The user features and the device features are obtained by optimizing adjacent nodes in the user-device relationship graph, where adjacent nodes are user nodes and device nodes connected by the edges.

[0102] In some embodiments, the user features and the device features are determined based on the multivariate graph feature prediction model, which is based on the user-device relationship graph to predict the features corresponding to each node and is trained according to the feature difference information between adjacent nodes.

[0103] In some embodiments, the first training unit includes:

[0104] The first clustering subunit is used to perform clustering processing on the user features to obtain the attributes of each user class.

[0105] The first training subunit is used to train the risk perception model corresponding to each user class attribute based on the user characteristics of that user class attribute.

[0106] The user risk perception model includes risk perception models corresponding to each user class attribute.

[0107] In some embodiments, the user equipment relationship graph includes a user equipment login relationship graph and a user equipment verification relationship graph; the user features are obtained by fusing user features in the user equipment login relationship graph and user features in the user equipment verification relationship graph.

[0108] In some embodiments, the second training unit includes:

[0109] The second clustering subunit is used to perform clustering processing on the device features to obtain the attributes of each device class.

[0110] The second training subunit is used to train the risk perception model corresponding to each device class attribute based on the device characteristics of that device class attribute.

[0111] The equipment risk perception model includes risk perception models corresponding to each equipment class attribute.

[0112] In some embodiments, the user equipment relationship graph includes a user equipment login relationship graph and a user equipment verification relationship graph; the device feature is a feature obtained by fusing the device features in the user equipment login relationship graph and the device features in the user equipment verification relationship graph.

[0113] In some embodiments, the first training unit includes:

[0114] The first input subunit is used to input the user features into the first network model and output predicted user attack risk information.

[0115] The first calculation subunit is used to calculate the first prediction loss between the predicted user attack risk information and the preset user attack truth value.

[0116] The first generation subunit is used to generate the user risk perception model based on the first predicted loss.

[0117] In some embodiments, the second training unit includes:

[0118] The second input subunit is used to input the device features into the second network model and output predicted device attack risk information.

[0119] The second calculation subunit is used to calculate the second prediction loss between the predicted device attack risk information and the preset device attack truth value;

[0120] The second generation subunit is used to generate the equipment risk perception model based on the second predicted loss.

[0121] Fifthly, this disclosure provides a processor-readable storage medium storing a computer program configured to cause the processor to perform the method described in any of the first aspects; or, the computer program configured to cause the processor to perform the method described in any of the second aspects.

[0122] In a sixth aspect, this disclosure provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0123] The memory stores computer-executed instructions;

[0124] The processor executes computer execution instructions stored in the memory to implement the method as described in any of the first aspects; or to implement the method as described in any of the second aspects.

[0125] In a seventh aspect, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in either the first or second aspect.

[0126] Eighthly, this disclosure provides a liveness detection system, comprising:

[0127] At least one memory, the memory including at least one set of instructions for pushing information;

[0128] At least one processor communicates with the at least one memory;

[0129] Wherein, when the at least one processor executes the at least one set of instructions, the method as described in any one of the first aspects above is implemented.

[0130] Ninthly, this disclosure provides a training system for a liveness detection model, comprising:

[0131] At least one memory, the memory including at least one set of instructions for pushing information;

[0132] At least one processor communicates with the at least one memory;

[0133] Wherein, when the at least one processor executes the at least one set of instructions, the method as described in any one of the second aspects above is implemented.

[0134] This disclosure provides a liveness detection method, a training apparatus for a liveness detection model, and includes: in response to a liveness detection task being triggered, acquiring user attack risk information of the target user triggering the task and device attack risk information of the target device triggering the task; determining the liveness category of the task based on the user attack risk information and the device attack risk information, wherein the liveness category is either liveness or attack. In this embodiment, by acquiring user attack risk information and device attack risk information separately when the task is triggered, and combining user attack risk information and device attack risk information to determine the liveness detection result, on the one hand, there is no need to deploy a multimodal camera module, thereby avoiding the disadvantage of low hardware cost caused by the liveness detection method based on multimodal face images in the above-mentioned related technologies, which is conducive to saving hardware costs; on the other hand, there is no need to perform multi-action interaction, avoiding the disadvantage of detection efficiency caused by the liveness detection method based on multi-frame images of multi-action interaction in the above-mentioned related technologies, which is conducive to improving detection efficiency; furthermore, by combining attack risk information from two different dimensions (i.e., the target user dimension and the target device dimension) to determine the liveness detection result, the accuracy and reliability of liveness detection can be improved. Attached Figure Description

[0135] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0136] Figure 1 This is a schematic diagram illustrating an application scenario of the liveness detection method according to an embodiment of this disclosure;

[0137] Figure 2 This is a schematic diagram of a liveness detection method according to an embodiment of the present disclosure;

[0138] Figure 3 This is a schematic diagram of a liveness detection method according to another embodiment of the present disclosure;

[0139] Figure 4 This is a schematic diagram of a training method for a liveness detection model according to an embodiment of the present disclosure;

[0140] Figure 5 This is a schematic diagram of a liveness detection device according to an embodiment of the present disclosure;

[0141] Figure 6This is a schematic diagram of a training device for a liveness detection model according to an embodiment of the present disclosure;

[0142] Figure 7 This is a hardware structure diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0143] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0144] It should be understood that the terms “comprising” and “having” and any variations thereof in the embodiments of this disclosure are intended to cover but not exclude inclusion. For example, a product or device that includes a series of components is not necessarily limited to those components that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0145] In this disclosure, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0146] In this disclosure, the term "multiple" refers to two or more, and other quantifiers are similar.

[0147] The terms “first,” “second,” “third,” etc., used in this disclosure are used to distinguish similar or related objects or entities and do not necessarily imply a specific order or sequence, unless otherwise indicated. It should be understood that such terms can be used interchangeably where appropriate, for example, in situations where implementation can proceed in an order other than those given in the illustrations or descriptions of embodiments of this disclosure.

[0148] As used in this disclosure, the term "unit / module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0149] To facilitate the reader's understanding of this disclosure, at least some of the terms used in this disclosure are explained below:

[0150] A facial recognition system is a system built on facial recognition technology. Facial recognition technology refers to the use of computer technology to analyze and compare human faces. Facial recognition technology belongs to biometric identification technology, which distinguishes individuals based on their biological characteristics (generally referring to humans).

[0151] Liveness attacks refer to attack methods that bypass facial recognition systems by using screens, paper, masks, or other means.

[0152] Liveness detection, also known as liveness attack prevention, refers to the use of artificial intelligence models to detect and intercept liveness attacks to verify whether the user is a real, living person operating the system.

[0153] Clustering is the process of dividing a dataset into different classes or clusters according to a predefined criterion (such as distance), so that the similarity of data objects within the same cluster is maximized, while the differences between data objects in different clusters are maximized. In other words, after clustering, data of the same class are grouped together as much as possible, while data of different classes are separated as much as possible.

[0154] There are various clustering analysis methods, and different methods will yield different results when used to analyze the same object. Clustering analysis methods include: hierarchical clustering, k-means clustering, two-step clustering, etc. Hierarchical clustering is also known as agglomerative hierarchical clustering. K-means clustering is also known as fast clustering.

[0155] Facial recognition payment refers to a new type of payment method based on technologies such as artificial intelligence, machine vision, 3D sensing, and big data. It has advantages such as being more convenient, safer, and providing a better user experience.

[0156] In recent years, with the rapid development of Internet technology, facial recognition technology has been applied to various identity recognition and authentication scenarios, such as facial recognition payment, facial recognition attendance, and facial recognition entry.

[0157] Although facial recognition technology has rapidly become a primary method of identity authentication due to its convenience, it also faces many new security threats as it is an emerging technology. Among these, liveness detection attacks are a relatively common and high-risk security threat.

[0158] Correspondingly, liveness detection methods can be used to determine the type of liveness, so as to avoid liveness attacks as much as possible.

[0159] In related technologies, there are two main types of liveness detection methods: one is based on multimodal face images, and the other is based on multi-frame images with multi-action interactions.

[0160] For example, the implementation principle of the liveness detection method based on multimodal face images is as follows: a multimodal camera module is installed in the terminal device, the terminal device acquires the user's multimodal face images through the multimodal camera module, and inputs the multimodal face images into a pre-trained multimodal liveness detection model to output the liveness detection results.

[0161] Multimodal face images include RGB images, modern near-infrared (NIR) images, depth images, and thermal imaging images.

[0162] Although this liveness detection method, which uses multimodal face images to determine the liveness detection result, can improve liveness detection performance, the cost of multimodal camera modules is high. Therefore, using multimodal face image liveness detection method for liveness detection has the disadvantage of high hardware cost.

[0163] The principle of the liveness detection method based on multi-frame images of multi-action interaction is as follows: the face recognition system collects the face images corresponding to each interactive action performed by the user during the interaction, that is, it collects multiple frames of face images, and completes liveness detection based on the multiple frames of face images.

[0164] Interactive actions include blinking, shaking your head, and so on.

[0165] However, in this type of liveness detection method, the face recognition system needs to collect various interactive actions. Therefore, the face recognition system takes a long time to collect face images and requires the user's cooperation for a long time. This results in the drawbacks of low efficiency in liveness detection and poor user experience.

[0166] It should be noted that the content in the above-mentioned related technologies is merely information known only to the inventor and does not mean that the above information had entered the public domain before the date of this application, nor does it mean that it can be considered prior art in this disclosure.

[0167] To avoid at least one of the above problems, this disclosure proposes a technical concept developed through inventive effort: when a liveness detection task is triggered, the liveness detection device acquires attack risk information corresponding to the user dimension and the device dimension, respectively, and combines the attack risk information from the two different dimensions to determine the liveness category (i.e., the liveness detection result), thereby completing the liveness detection.

[0168] Before explaining the implementation principle of the liveness detection method of this disclosure, an exemplary application scenario of the liveness detection method of this disclosure will be described to deepen the reader's understanding of the liveness detection method of this disclosure.

[0169] Based on the above analysis, it can be seen that the liveness detection method disclosed herein can be applied to different scenarios. For example, in the facial recognition payment scenario, the liveness detection method disclosed herein can be used to perform liveness detection on the collected facial images of users to be paid; as well as in the facial recognition attendance scenario, the liveness detection method disclosed herein can be used to perform liveness detection on the collected facial images of users, and so on, which will not be elaborated here.

[0170] Figure 1 This is a schematic diagram illustrating an application scenario of the liveness detection method according to an embodiment of the present disclosure. The liveness detection method of the present disclosure can be applied to, for example... Figure 1 The liveness detection system 100 shown is an example. Figure 1 As shown, the liveness detection system 100 may include a target user 101, a client 102, a server 103, and a network 104.

[0171] The target user 101 can be the user who triggers the liveness detection of the target facial image (i.e., the face image of the target user 101), and the target user 101 can perform the liveness detection operation on the client 102.

[0172] Client 102 can be a device for performing liveness detection on a target facial image in response to a liveness detection operation by target user 102. That is, the liveness detection method can be executed on client 102. At this time, client 102 can store data or instructions for executing the liveness detection method described in this specification, and can execute or be used to execute the data or instructions.

[0173] In some embodiments, client 102 may include a hardware device with data processing capabilities and the necessary programs to drive the hardware device. For example... Figure 1 As shown, client 102 can establish a communication connection with server 103. Server 103 can establish a communication connection with one client 102 or with multiple clients 102.

[0174] In some embodiments, client 102 can interact with server 103 via network 104 to receive or send messages, etc.

[0175] In some embodiments, client 102 may include mobile devices, tablets, laptops, built-in devices in motor vehicles, or similar content, or any combination thereof.

[0176] In some embodiments, the mobile device may include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device, or a similar device, or any combination thereof.

[0177] In some embodiments, smart home devices may include smart TVs, desktop computers, or any combination thereof.

[0178] In some embodiments, a smart mobile device may include a smartphone, a personal digital assistant, a gaming device, a navigation device, or any combination thereof.

[0179] In some embodiments, a virtual reality device or augmented reality device may include a virtual reality headset, virtual reality glasses, a virtual reality patch, an augmented reality headset, augmented reality glasses, an augmented reality patch, or similar content, or any combination thereof. For example, the virtual reality device or the augmented reality device may include Google Glass, a head-mounted display, virtual reality technology (VR), etc.

[0180] In some embodiments, the built-in devices in a motor vehicle may include an onboard computer, an onboard television, etc.

[0181] In some embodiments, client 102 may include an image acquisition device and an audio acquisition device for collecting user data of an account.

[0182] In some embodiments, the image acquisition device may be a two-dimensional image acquisition device (such as an RGB camera), or a two-dimensional image acquisition device (such as an RGB camera) and a depth image acquisition device (such as a 3D structured light camera, a laser detector, etc.).

[0183] In some embodiments, client 102 may be a device with positioning technology for locating the position of client 102.

[0184] In some embodiments, client 102 may have one or more applications (APPs) installed. APPs provide target user 101 with the ability and interface to interact with the outside world via network 104. APPs include, but are not limited to: web browser APPs, search APPs, chat APPs, shopping APPs, video APPs, financial management APPs, instant messaging tools, email clients, social media platform software, etc.

[0185] In some embodiments, a target app may be installed on the client 102. The target app can collect facial images and corresponding audio information of users corresponding to multiple accounts for the client 102, thereby obtaining a user data set.

[0186] In some embodiments, the target user 101 may also trigger a liveness detection request (i.e., trigger a liveness detection task) through the target app. The target app may respond to the liveness detection request by executing the liveness detection method described in this specification. The liveness detection method will be described in detail later.

[0187] Server 103 can be a server that provides various services, such as a backend server that supports the collection of user data sets and account login information corresponding to multiple accounts on client 102, and performs liveness detection on multiple accounts.

[0188] In some embodiments, the liveness detection method can be executed on server 103. In this case, server 103 may store data or instructions for executing the liveness detection method described herein, and may execute or be used to execute the data or instructions.

[0189] In some embodiments, server 103 may include a hardware device with data processing capabilities and the necessary programs to drive the hardware device. Similarly, server 103 may communicate with one client 103 and receive data sent by that client 103, or it may communicate with multiple clients 103 and receive data sent by each client 103.

[0190] Network 104 is a medium used to provide a communication connection between client 102 and server 103. Network 104 can facilitate the exchange of information or data. Figure 1 As shown, client 102 and server 103 can connect to network 104 respectively and transmit information or data to each other through network 104.

[0191] In some embodiments, network 104 can be any type of wired or wireless network, or a combination thereof. For example, network 104 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network™, a ZigBee™ short-range wireless network, a near field communication (NFC) network, or a similar network.

[0192] In some embodiments, network 104 may include one or more network access points. For example, network 104 may include wired or wireless network access points, such as base stations or internet switching points, through which one or more components of client 102 and server 103 can connect to network 104 to exchange data or information.

[0193] It should be understood that, Figure 1 The number of clients 102, servers 103, and networks 104 shown is merely illustrative. Depending on implementation needs, there can be any number of clients 102, servers 103, and networks 104.

[0194] It should be noted that the liveness detection method provided in this disclosure can be executed entirely on the client 102, entirely on the server 103, or partially on the client 102 and partially on the server 103.

[0195] In other words, Figure 1 and targeting Figure 1 The above description is only used to illustrate the possible application scenarios for the liveness detection method of this disclosure, and should not be construed as limiting the application scenarios.

[0196] Please see Figure 2 , Figure 2 This is a schematic diagram of a liveness detection method according to an embodiment of this disclosure. Figure 2 As shown, the method includes:

[0197] S201: In response to the liveness detection task being triggered, obtain user attack risk information of the target user who triggered the task and device attack risk information of the target device that triggered the task.

[0198] For example, the execution subject of this embodiment can be a liveness detection device, which can be a server, a terminal device, a processor, a chip, etc., and will not be listed here.

[0199] If the liveness detection device is a server, it can be a standalone server or a cluster server; it can be a cloud server or a local server, and this embodiment does not impose any limitations.

[0200] For example, in combination Figure 1 In the application scenarios shown, the liveness detection device can be a client, a server, or both.

[0201] In this context, "target user" can be understood as the user targeted by the liveness detection method. "Target device" can be understood as the device used by the target user when liveness detection is performed. "User attack risk information" can be understood as information about the target user that indicates they are an attacker, such as representing the probability or confidence level of the target user being an attacker. "Device attack risk information" can be understood as information about the target device that indicates it is an attacker, such as representing the probability or confidence level of the target device being an attacker.

[0202] This embodiment does not limit the method by which the liveness detection device obtains user attack risk information and device attack risk information. For example, it can obtain them through prediction or through models, etc.

[0203] Specifically, the prediction-based acquisition method allows the liveness detection device to determine user attack risk information by acquiring historical attack data of the target user, and the liveness detection device to determine device attack risk information by acquiring historical attack data of the target device. The model-based acquisition method enables the liveness detection device to train a network model based on sample data to determine user attack risk information, and vice versa.

[0204] Based on the above analysis, it can be seen that the liveness detection method of this embodiment can be applied to different application scenarios. Accordingly, the way the task is triggered may be different for different application scenarios. That is, this embodiment does not limit the way the task is triggered, and specifically, the way the task is triggered may be different for different application scenarios.

[0205] For example, if the liveness detection method of this disclosure is applied to a facial recognition payment scenario, the task is triggered when the target user enables the facial recognition payment function of the payment app on the target device (such as a terminal device). Similarly, if the liveness detection method of this disclosure is applied to a facial recognition attendance scenario, the task is triggered when the target user enables the facial recognition attendance function of the attendance app on the target device (such as a terminal device).

[0206] S202: Based on user attack risk information and device attack risk information, determine the liveness category of the task, where the liveness category is either live or attack.

[0207] For example, the liveness category can be understood as the liveness detection result, which may indicate a live person or an attack.

[0208] Based on the above analysis, this disclosure provides a liveness detection method. The method includes: in response to a liveness detection task being triggered, acquiring user attack risk information of the target user triggering the task and device attack risk information of the target device triggering the task; determining the liveness category of the task based on the user attack risk information and device attack risk information, wherein the liveness category is either liveness or attack. In this embodiment, by acquiring user attack risk information and device attack risk information respectively when the task is triggered, and combining the user attack risk information and device attack risk information to determine the liveness detection result, the following technical features are achieved: Firstly, there is no need to deploy a multimodal camera module, thus avoiding the high hardware cost drawback of the liveness detection method based on multimodal face images in the aforementioned related technologies, which is beneficial for saving hardware costs. Secondly, there is no need to perform multi-action interaction, avoiding the low detection efficiency drawback of the liveness detection method based on multi-frame images of multi-action interaction in the aforementioned related technologies, which is beneficial for improving detection efficiency. Furthermore, by combining attack risk information from two different dimensions (i.e., the target user dimension and the target device dimension) to determine the liveness detection result, the accuracy and reliability of liveness detection can be improved.

[0209] To help readers gain a deeper understanding of the implementation principles of this disclosure, the following is combined with... Figure 3 The live detection method of this disclosure is described in detail. Among other things, Figure 3 This is a schematic diagram of a liveness detection method according to another embodiment of the present disclosure, as shown below. Figure 3 As shown, the method includes:

[0210] S301: In response to the liveness detection task being triggered, acquire the target image, target user features, and target device features of the triggering task.

[0211] For example, if a target user triggers a task through a target device, the liveness detection device acquires the target user's facial image, which can be called the target image, and acquires the target user's features, which can be called the target user features, and acquires the target device's features, which can be called the target device features.

[0212] Among them, target user characteristics are used to describe the characteristics of the target user, such as the target user's age, gender, number of times the task was triggered, and the account that triggered the task. Target device characteristics include the characteristics of the target device itself and the characteristics of the target user's use of the target device. For example, the characteristics of the target device itself include its identifier, model, and location, while the characteristics of the target user's use of the target device include the frequency and time distribution of the target user's use of the target device.

[0213] Taking a liveness detection device deployed on a cloud platform as a cloud server and a facial recognition payment scenario as an example, this step can be understood as follows: The target user can open the facial recognition payment APP on the target device (such as a mobile phone) by touch, so as to trigger the liveness detection task through the target device. The target device is equipped with an image acquisition device (such as a camera). The target device acquires the target user's facial image (i.e., the target image) through the image acquisition device and sends the target image to the cloud server. Correspondingly, the cloud server obtains the target image and obtains the characteristics of the target user (i.e., the target user characteristics) and the characteristics of the target device (i.e., the target device characteristics).

[0214] This embodiment does not limit the method by which the cloud server obtains the target user characteristics and target device characteristics. For example, the cloud server can obtain them online or offline.

[0215] The online acquisition method can be understood as follows: the target user characteristics and target device characteristics are not pre-stored in the cloud server. When the task is triggered, the cloud server obtains the target user characteristics and target device characteristics through network communication.

[0216] The offline method of acquisition can be understood as the target user characteristics and target device characteristics being pre-stored in the cloud server. When the task is triggered, the cloud server retrieves the pre-stored target user characteristics and target device characteristics.

[0217] If the data is obtained offline, it can be pre-stored in a table or a relationship diagram (such as a node diagram). This embodiment does not limit the method.

[0218] For example, in some embodiments, the target user characteristics and target device characteristics are determined by a cloud server from a pre-built user device relationship graph. This user device relationship graph includes user nodes and device nodes; user nodes include user characteristics, and device nodes include device characteristics; user nodes include nodes representing target users, and device nodes include nodes representing target devices; user characteristics include target user characteristics, and device characteristics include target device characteristics.

[0219] For example, based on the above analysis, the cloud server can pre-build and store a user device relationship graph in order to obtain target user characteristics and target device characteristics from the user device relationship graph.

[0220] The user device relationship diagram represents the relationship between users and devices, specifically the usage relationship between users and devices. A user may use one device or multiple devices. A device may be used by one user or multiple users.

[0221] The user device relationship graph can be a node graph, which includes multiple nodes, each corresponding to a user or a device. For example, if the user device relationship graph includes K (K is a positive integer greater than 1) nodes, namely node 1, node 2, up to node K. If node 1 is a node of user 1, then node 1 stores the characteristics of user 1; if node 2 is a node of device 2, then node 2 stores the characteristics of device 2, and so on. These will not be listed individually here.

[0222] Correspondingly, when a target user triggers a task through a target device, the cloud server can determine the target user's node from the user device relationship graph and obtain the target user's characteristics from that node; the cloud server can also determine the target device's node from the user device relationship graph and obtain the target device's characteristics from that node.

[0223] In this embodiment, by pre-constructing a user equipment relationship graph to obtain target user features and target device features from the user equipment relationship graph, the efficiency of obtaining target user features and target device features can be improved, thereby improving the efficiency of liveness detection.

[0224] In some embodiments, the user device relationship graph further includes edges between user nodes and device nodes; the user device relationship graph is constructed based on acquired sample data; the sample data includes user characteristics, device characteristics, and user behavior information of the device.

[0225] Among them, user nodes are built based on users, device nodes are built based on devices, and edges are built based on usage behavior information.

[0226] For example, based on the above analysis, the sample data includes data from A users and data from B devices, where A + B = K. Nodes are constructed based on the number of users and devices. Nodes corresponding to users can be called user nodes, and nodes corresponding to devices can be called device nodes. For each node, the cloud server stores the data corresponding to that node. For example, if the node is user node 1 corresponding to user 1, the cloud server stores user 1's user data in user node 1; if the node is device node 2 corresponding to device 2, the cloud server stores device 2's device data in device node 2.

[0227] Behavioral information is used to characterize user behavior when using the device. The cloud server constructs edges based on whether the user uses the device. For example, if behavioral information is used to characterize that device 2 is used by user 1, the cloud server constructs an edge to connect user node 1 and device node 2.

[0228] In some embodiments, usage behavior information includes usage time. If the cloud server determines that a user has used the device within a preset time period based on the usage time, then an edge is constructed based on the usage behavior information.

[0229] Similarly, the preset duration can be determined based on needs, historical records, and experiments, and this embodiment does not impose any limitations.

[0230] In this embodiment, the cloud server constructs a user device relationship graph by combining sample data including user characteristics, device characteristics, and usage behavior information, thereby achieving reliability and accuracy in constructing the user device relationship graph.

[0231] In some embodiments, the task has a type attribute, which is either a login type or an authentication type; the user device relationship diagram includes a user device login relationship diagram and a user device authentication relationship diagram.

[0232] If the type attribute is login type, then the target user characteristics and target device characteristics are determined from the user device login relationship diagram.

[0233] If the type attribute is a verification type, then the target user characteristics and target device characteristics are determined from the user device verification relationship diagram.

[0234] For example, if the type attribute is login, then the task is a login task; if the type attribute is authentication, then the task is an authentication task. For tasks with different type attributes, the cloud server can construct different user device relationship diagrams. For instance, the cloud server can call the user device relationship diagram corresponding to the login task a user device login relationship diagram, and the user device relationship diagram corresponding to the authentication task a user device authentication relationship diagram.

[0235] Accordingly, for login tasks, the cloud server obtains the target user characteristics and target device characteristics from the user device login relationship graph. For verification tasks, the cloud server obtains the target user characteristics and target device characteristics from the user device verification relationship graph.

[0236] It should be understood that the above is only a illustrative description of type attributes using login type and verification type as examples, and should not be interpreted as a limitation on type attributes.

[0237] In this embodiment, the cloud server constructs user device relationship graphs corresponding to different type attributes to determine the user device relationship graphs corresponding to the type attributes of the task from each user device relationship graph, and obtains target user features and target device features from the determined user device relationship graphs, thereby improving the flexibility and diversity of liveness detection and meeting the needs of different scenarios. Furthermore, by performing targeted feature acquisition operations, the accuracy and reliability of liveness detection can be improved.

[0238] S302: Input the target user features into the pre-trained user risk perception model and output user attack risk information; input the target device features into the pre-trained device risk perception model and output device attack risk information.

[0239] This embodiment does not limit the type, architecture, or parameters of the user risk perception model used by the cloud server; these can be determined based on requirements, historical records, and experiments. Similarly, this embodiment does not limit the training method and execution of the user risk perception model. For example, the cloud server can collect sample data, which is a dataset including user features, and train a preset network model based on the sample data. This allows the preset network model to learn and predict the user attack risk corresponding to the user features in the sample data, such as the probability or level of the user corresponding to the user feature being an attacker, thereby obtaining the user risk perception model.

[0240] Based on the above analysis, it can be seen that cloud servers obtain user attack risk information through prediction or modeling. In this step, cloud servers obtain user attack risk information through modeling, which can improve the efficiency and reliability of obtaining user attack risk information.

[0241] Similarly, this embodiment does not limit the type, architecture, or parameters of the device risk perception model used by the cloud server; these can be determined based on requirements, historical records, and experiments. Likewise, this embodiment does not limit the training method or the executing entity of the device risk perception model. For example, the cloud server can collect sample data, which is a dataset including device features. A preset network model can be trained based on the sample data, enabling the preset network model to learn and predict the device attack risk corresponding to the device features in the sample data, such as the probability or level of the device corresponding to the device feature being an attacking device, thereby obtaining the device risk perception model.

[0242] Based on the above analysis, it can be seen that cloud servers obtain device attack risk information in two ways: through prediction and through models. In this step, cloud servers obtain device attack risk information through models, which can improve the efficiency and reliability of obtaining such information.

[0243] In some embodiments, the target user feature has a target user class attribute, and the target device feature has a target device class attribute; the user risk perception model includes a risk perception model corresponding to each user class attribute, and the user class attribute includes the target user class attribute; the device risk perception model includes a risk perception model corresponding to each device class attribute, and the device class attribute includes the target device class attribute.

[0244] User attack risk information is obtained by performing attack risk prediction processing on the characteristics of target users based on the risk perception model corresponding to the target user class attributes in the user risk perception model.

[0245] Device attack risk information is obtained by performing attack risk prediction processing on the characteristics of the target device based on the risk perception model corresponding to the target device class attributes in the device risk perception model.

[0246] For example, a user class attribute can be understood as a class corresponding to a user characteristic. Different users may have the same user characteristics or different user characteristics. Different user characteristics may correspond to the same class or different classes. Similarly, a device class attribute can be understood as a class corresponding to a device characteristic. Different devices may have the same device characteristics or different device characteristics. Different device characteristics may correspond to the same class or different classes.

[0247] Accordingly, for different classes (including user classes and device classes), risk perception models corresponding to each class can be pre-trained (including risk perception models corresponding to user classes and risk perception models corresponding to device classes).

[0248] For example, if there are N (N is a positive integer greater than or equal to 1) classes of user features, numbered from Class 1 to Class N, then for each class of user features, the cloud server pre-constructs a user risk perception model for that class of user features, resulting in N user risk perception models. Correspondingly, if the target user class attribute of a target user feature indicates that the target user feature belongs to Class 1, then the cloud server inputs the target user feature into the risk perception model corresponding to the first class among the N user risk perception models, and outputs user risk perception information.

[0249] Similarly, if there are M (M is a positive integer greater than or equal to 1) classes of device features, categorized from Class 1 to Class M, then for each class of device features, the cloud server pre-constructs a device risk perception model for that class of device features, resulting in M ​​device risk perception models. Correspondingly, if the target device class attribute of a target device feature indicates that the target device feature belongs to Class 1, then the cloud server inputs the target device feature into the risk perception model corresponding to the first class among the M device risk perception models, and outputs device risk perception information.

[0250] In this embodiment, the cloud server determines the user risk perception information corresponding to the target user characteristics by combining the target user class attributes, thereby achieving targeted determination of user risk perception information and improving the accuracy and reliability of user risk perception information. Similarly, the cloud server determines the device risk perception information corresponding to the target device characteristics by combining the target device class attributes, thereby achieving targeted determination of device risk perception information and improving the accuracy and reliability of device risk perception information.

[0251] In some embodiments, the user risk perception model is obtained by clustering user features to obtain user class attributes, and training the user features corresponding to each user class attribute.

[0252] The equipment risk perception model is obtained by clustering equipment features to obtain the attributes of each equipment class, and then training the equipment features corresponding to each equipment class attribute.

[0253] This embodiment does not limit the clustering analysis method used for cloud server clustering. For example, cloud servers can use hierarchical clustering, K-means clustering, or second-order clustering.

[0254] By combining clustering processing to train user risk perception models and device risk perception models, cloud servers can construct risk perception models corresponding to different categories of attributes (including user category attributes and device category attributes). This allows for the construction of user risk perception models corresponding to user category attributes and device risk perception models corresponding to device category attributes. By leveraging the correspondence between user features and user risk perception models, as well as the correspondence between device features and device risk perception models, the targeting of risk perception (including user risk perception and device risk perception) can be improved, thereby enhancing the accuracy and effectiveness of risk perception.

[0255] In some embodiments, user features include user features in the user device login relationship graph and user features in the user device authentication relationship graph; device features include device features in the user device login relationship graph and device features in the user device authentication relationship graph.

[0256] Each user type attribute is obtained by fusing the user features in the user device login relationship graph and the user features in the user device verification relationship graph for each user, and then clustering the user fusion features corresponding to each user.

[0257] Each device type attribute is obtained by fusing the device features of the device in the user device login relationship graph and the device features of the device in the user device verification relationship graph for each device, and then performing clustering processing on the corresponding device fusion features of each device.

[0258] Based on the above analysis, the user device relationship graph can include both a user device login relationship graph and a user device authentication relationship graph. Therefore, when performing clustering on the cloud server, features from both graphs can be combined. Taking the user type attribute obtained through cloud server clustering as an example, the following explanation is provided:

[0259] There are multiple users. For each user, the cloud server may include the user's characteristics in the user device login relationship graph and the user device verification relationship graph. The cloud server then merges the user's characteristics in the user device login relationship graph and the user's characteristics in the user device verification relationship graph to obtain the user's merged characteristics. This process is repeated to obtain the user merged characteristics for each user. The merged characteristics are then clustered, for example, using K-means clustering to obtain the attributes of each user class.

[0260] Taking the device type attribute obtained through clustering as an example, the explanation is as follows:

[0261] There are multiple devices. For each device, its device characteristics may be included in both the user device login relationship graph and the user device verification relationship graph. The cloud server then merges the device characteristics in the user device login relationship graph and the device characteristics in the user device verification relationship graph to obtain the device fused characteristics. This process is repeated to obtain the device fused characteristics for each device. The fused characteristics of each device are then clustered, such as by using K-means clustering, to obtain the attributes of each device class.

[0262] It should be noted that for any given user, the user characteristics of that user may be included in the user device login relationship graph but not in the user device verification relationship graph. In this case, the user's user fusion characteristics are the same as the user characteristics of that user in the user device login relationship graph. Conversely, the user characteristics of that user may not be included in the user device login relationship graph but are included in the user device verification relationship graph. In this case, the user's user fusion characteristics are the same as the user characteristics of that user in the user device verification relationship graph.

[0263] Similarly, for any given device, if the user device login relationship graph includes the device's device features but the user device verification relationship graph does not, then the device's fusion feature is the same as the device's device features in the user device login relationship graph. Conversely, if the user device login relationship graph does not include the device's device features but the user device verification relationship graph does, then the device's fusion feature is the same as the device's device features in the user device verification relationship graph.

[0264] In other words, in this embodiment, for situations where different relationship graphs include user features of the same user and device features of the same device, the cloud server first performs fusion processing on the user features of the same user and fusion processing on the device features of the same device before performing clustering processing, so that the fused user features and fused device features have higher comprehensiveness, thereby improving the effectiveness and accuracy of clustering processing.

[0265] The above example illustrates how to construct a user device relationship graph. When the user device relationship graph includes a user device login relationship graph and a user device authentication relationship graph, the cloud server can construct the user device login relationship graph and the user device authentication relationship graph based on the sample data.

[0266] For example, usage behavior information may include usage categories, which include login usage categories and authentication usage categories. A login usage category can be understood as a user using the device based on a login requirement, and an authentication usage category can be understood as a user using the device based on an authentication requirement.

[0267] Correspondingly, the cloud server can construct a user device login relationship graph based on login usage categories. For example, the edges in the user device login relationship graph that connect user nodes and device nodes represent usage behavior information for login usage categories. Similarly, the cloud server can construct a user device verification relationship graph based on verification usage categories. For example, the edges in the user device verification relationship graph that connect user nodes and device nodes represent usage behavior information for verification usage categories.

[0268] Based on the above analysis, it can be seen that usage behavior information has usage time. Taking login usage category as an example, if user 1 uses device 2 for usage time, and user 1 performs login operation on device 2 within a preset time period (such as Q days, where Q is a positive integer greater than or equal to 1), then the cloud server constructs a point connecting user node 1 and device node 2.

[0269] Taking the verification of usage category as an example, if user 1 uses device 2 to represent the usage time, and user 1 performs verification operations on device 2 within a preset duration (such as Q days, where Q is a positive integer greater than or equal to 1), then the cloud server constructs a connection between user node 1 and device node 2.

[0270] S303: Perform attack risk prediction processing on the target image to obtain image attack risk information.

[0271] Image attack risk information can be understood as information about the target user being an attacker, determined by the cloud server from the dimensions of the target image. For example, image attack risk information can characterize the probability or confidence that the target user is an attacker.

[0272] Based on the above description of the relevant technologies, it is clear that images may also be used to determine the user's attack risk. Therefore, this embodiment does not limit the implementation method of obtaining image attack risk information. For example, it can be implemented using the liveness detection method of multimodal face images in the relevant technologies, or it can be implemented using the liveness detection method of unimodal face images in the relevant technologies (the liveness detection method of multimodal face images is a further improvement of this method, so it is not described in this disclosure; please refer to the prior art for details).

[0273] S304: Determine the liveness category based on image attack risk information, user attack risk information, and device attack risk information, where the liveness category is either live or attack.

[0274] Compared to Figure 2 In the embodiment shown, the cloud server can further incorporate image attack risk information to combine it with user attack risk information and device attack risk information to determine the liveness category based on three dimensions of risk information, thereby improving the effectiveness and accuracy of the liveness category.

[0275] In some embodiments, image attack risk information includes image attack probability, user attack risk information includes the number of user attacks, and device attack risk information includes the number of device attacks; S304 may include the following steps:

[0276] First step: Calculate the total number of attacks based on the number of user attacks and the number of device attacks.

[0277] For example, image attack risk information can be represented by image attack probability, while user attack risk information and device attack risk information can be represented by the number of attacks, with the total number of attacks being the sum of the number of user attacks and the number of device attacks.

[0278] The second step is to determine the liveness category based on the total number of attacks and the probability of image attacks.

[0279] The cloud server can convert the total number of attacks into attack probabilities, and then combine the image attack probability with the converted attack probability to determine the liveness category. For example, the sum of the image attack probability and the converted attack probability can be calculated to obtain the overall attack probability. If the overall attack probability is high, the liveness category is determined to be an attack; if the overall attack probability is low, the liveness category is determined to be a live.

[0280] For example, the second step may include the following sub-steps:

[0281] First sub-step: Determine the predicted attack probability corresponding to the total number of attacks.

[0282] The conversion between the total number of attacks and the predicted attack probability can be determined based on a formula. For example, a cloud server can divide the total number of attacks by a preset conversion value to obtain the predicted attack probability.

[0283] Similarly, the preset conversion value can be determined based on requirements, historical records, and experiments. For example, if the total number of attacks is n and the preset conversion value is 10, then the predicted attack probability = n / 10.

[0284] The second sub-step: Calculate the total attack probability based on the predicted attack probability and the image attack probability.

[0285] For example, if the probability of an image attack is p, then, based on the example above, the total probability of an attack is s = p + n / 10.

[0286] The third sub-step: If the total attack probability reaches the preset threshold, the liveness category is determined as an attack; if the total attack probability does not reach the preset threshold, the liveness category is determined as a liveness.

[0287] Similarly, the preset threshold can be determined based on demand, historical records, and experiments. For example, if the preset threshold is T, if s reaches T (i.e., s is greater than or equal to T), then the liveness category is attack; conversely, if s does not reach T (i.e., s is less than T), then the liveness category is live.

[0288] It is worth noting that the image attack probability, user attack count, and device attack count in this embodiment can also be other types of parameters, such as their respective risk levels or their respective confidence levels.

[0289] Taking other types of parameters as their corresponding risk levels as an example, the cloud server can determine the liveness category based on each risk level, and can determine the overall risk level by adding up the risk levels, and then determine the liveness category based on the overall risk level. For example, if the overall risk level is less than a preset level threshold, the liveness category is live; conversely, if the overall risk level is greater than or equal to the preset level threshold, the liveness category is attack.

[0290] Based on the above analysis, it can be seen that before performing liveness detection, the cloud server can pre-train user risk perception model and device risk perception model to complete liveness detection based on user risk perception model and device risk perception model. Therefore, we can call user risk perception model and device risk perception model the liveness detection model, that is, the liveness detection model includes both user risk perception model and device risk perception model.

[0291] In other embodiments, the liveness detection model may also include other models, such as a multimodal face image perception model, to further determine the liveness category based on the image attack risk information determined by the multimodal face image perception model in the case of liveness detection.

[0292] Accordingly, based on the above technical concept, this disclosure also provides a training method for a liveness detection model. Since the multimodal face image perception model adopts the training method in related technologies, this embodiment mainly takes the liveness detection model including the user risk perception model and the risk perception model as examples to illustrate the training of the liveness detection model.

[0293] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating a training method for a liveness detection model according to an embodiment of this disclosure. Figure 4 As shown, the method includes:

[0294] S401: Obtain user characteristics and device characteristics from the preset user equipment relationship diagram.

[0295] The execution entity in this embodiment can be a training device for a liveness detection model (hereinafter referred to as the training device). The training device can be the same as the liveness detection device or a different device.

[0296] If the training device and the liveness detection device are different devices, they can be connected through a communication link. After the training device trains the liveness detection model, it can transmit the liveness detection model to the liveness detection device through the communication link. Accordingly, the liveness detection device can perform liveness detection based on the liveness detection model.

[0297] It should be understood that this embodiment is not limited to the same or similar technical features as those in the above embodiments. For example, the descriptions of user equipment relationship diagrams, user features, and device features can be found in the examples above, and will not be repeated here.

[0298] In some embodiments, the user device relationship graph includes user nodes, device nodes, and edges between user nodes and device nodes. User nodes include user features, device nodes include device features, and edges are used to characterize user behavior information when using the device.

[0299] User features and device features are obtained by optimizing adjacent nodes in the user-device relationship graph, where adjacent nodes are user nodes and device nodes connected by edges.

[0300] Since there is a usage relationship between two adjacent nodes, the features corresponding to the two adjacent nodes are relatively correlated. In this embodiment, the user features and device features are obtained by the training device through the optimization processing of adjacent nodes. This is equivalent to considering the correlation between two adjacent nodes when the training device obtains user features and device features, thereby making the user features and device features have higher accuracy and reliability.

[0301] In some embodiments, user features and device features are determined based on a multivariate graph feature prediction model, which is based on predicting the features corresponding to each node in the user-device relationship graph and trained according to the feature difference information between adjacent nodes.

[0302] For example, the training device can pre-train a multivariate graph feature prediction model based on the user-device relationship graph. This multivariate graph feature prediction model can learn to optimize user features and device features, specifically by learning to optimize user features and device features based on the difference information between adjacent nodes.

[0303] Based on the above analysis, it can be seen that the user device relationship graph includes the user device login relationship graph and the user device verification relationship graph. For each relationship graph, the training device can construct a multivariate graph feature prediction model corresponding to that relationship graph.

[0304] Taking the training of a multivariate graph feature prediction model corresponding to the user device login relationship graph as an example:

[0305] The training device extracts the user features (referred to as original user features for easy distinction) and the device features (referred to as original device features for easy distinction) of each user node from the user device login relationship graph. It inputs the original user features and original device features into a preset network model and outputs the predicted user features and predicted device features corresponding to each original user feature. The loss function is determined based on the feature difference information (i.e., consistency) between the predicted features of adjacent nodes (including predicted user features and predicted device features), and the preset network model is adjusted based on the loss function to obtain the multivariate graph feature prediction model corresponding to the user device login relationship graph.

[0306] Similarly, this embodiment does not limit the type, architecture, and parameters of the preset network model, which can be determined by the training device based on requirements, historical records, and experiments. For example, the preset network model can be a multilayer perceptron (MLP), specifically a 3-layer MLP.

[0307] The implementation principle of the multivariate graph feature prediction model corresponding to the user equipment verification relationship graph trained by the training device can be found in the above-mentioned implementation principle of the multivariate graph feature prediction model corresponding to the user equipment login relationship graph trained by the training device, which will not be repeated here.

[0308] In this embodiment, the training device obtains optimized user features and device features by training a multivariate graph feature prediction model, which can improve the efficiency of optimization.

[0309] S402: A user risk perception model is trained based on user characteristics. The user risk perception model is used to predict user attack risk information of target users in the liveness detection task.

[0310] In some embodiments, S402 may include the following steps:

[0311] The first step is to cluster user features to obtain the attributes of each user class.

[0312] For example, in conjunction with the above analysis example, if the sample data includes user features corresponding to A users, the training device can perform clustering processing on the user features corresponding to A users to obtain N (N is a positive integer greater than or equal to 1) classes, and obtain the user class attributes corresponding to the user features of each user.

[0313] Based on the above analysis, it can be seen that the user device relationship graph includes the user device login relationship graph and the user device verification relationship graph. The user features are obtained by the training device through the fusion processing of the user features in the user device login relationship graph and the user features in the user device verification relationship graph.

[0314] In other words, the user features used for clustering are based on the same user. The training device fuses the user features of the same user in the user device login relationship graph and the user features in the user device verification relationship graph to obtain the fused features of each user and then performs clustering.

[0315] In this embodiment, the training device first fuses user features from different relationship graphs based on the same user for clustering processing, so that the fused features have higher comprehensiveness, thereby improving the effectiveness and reliability of clustering processing.

[0316] For example, taking a user device relationship graph that includes a user device login relationship graph and a user device authentication relationship graph, and using K-means clustering as the clustering method, the following explanation is provided:

[0317] User features include user features predicted by a multivariate graph feature prediction model based on the user device login relationship graph (for ease of distinction, these user features can be called login user features) and user features predicted by a multivariate graph feature prediction model based on the user device verification relationship graph (for ease of distinction, these user features can be called verification user features).

[0318] For each of the A users, the training device determines the user features corresponding to that user from the login user features and the user features corresponding to that user from the verification user features. The user features corresponding to that user determined from the login user features and the user features corresponding to that user determined from the verification user features are then fused. The fusion process can be understood as performing concatenation and normalization processes in sequence to obtain the user fusion features corresponding to that user. This process is repeated to obtain the user fusion features corresponding to each of the A users, such as the A user fusion features.

[0319] The training device uses K-means clustering to cluster A user fusion features, resulting in N classes, thus obtaining the user class attribute corresponding to each user fusion feature.

[0320] The second step is to train a risk perception model for each user class attribute based on the user characteristics of that user class attribute.

[0321] The user risk perception model includes risk perception models corresponding to each user class attribute.

[0322] In this embodiment, the training device first performs clustering to obtain the attributes of each user class, and then trains the corresponding risk perception model for each user class attribute, so that the training of the risk perception model is more targeted, thereby improving the effectiveness of the risk perception model.

[0323] In some embodiments, S402 may include the following steps:

[0324] The first step is to input user characteristics into the first network model and output predicted user attack risk information.

[0325] Similarly, this embodiment does not limit the type, architecture, parameters, etc. of the first network model, which can be determined by the training device based on requirements, historical records, and experiments. For example, the first network model can be an MLP, and specifically, it can be a 3-layer MLP.

[0326] The second step is to calculate the first prediction loss between the predicted user attack risk information and the preset user attack truth value, and generate a user risk perception model based on the first prediction loss.

[0327] The default user attack truth value is pre-labeled.

[0328] As can be seen from the above examples, the information for predicting user attack risk can be the number of user attacks, the probability of user attacks, the level of user attacks, and so on.

[0329] Taking the prediction of user attack risk information as the prediction of the number of user attacks as an example, the preset user attack truth value is the user attack count truth value. Then the first prediction loss is the difference information between the predicted user attack count and the user attack count truth value. The training device iteratively optimizes the first network model based on the first prediction loss until the first network model converges, thus obtaining the user risk perception model.

[0330] S403: A device risk perception model is trained based on device characteristics. The device risk perception model is used to predict the device attack risk information of the target device in the task.

[0331] Among them, user attack risk information and device attack risk information are used to determine the liveness category of the task; the liveness detection model includes a user risk perception model and a device risk perception model.

[0332] In some embodiments, S403 may include the following steps:

[0333] The first step is to cluster the device features to obtain the attributes of each device class.

[0334] For example, in conjunction with the above analysis example, if the sample data includes the device features corresponding to each of the B devices, the training device can perform clustering processing on the device features corresponding to each of the B devices to obtain N (N is a positive integer greater than or equal to 1) classes, and obtain the device class attribute corresponding to the device features of each device.

[0335] Based on the above analysis, it can be seen that the user device relationship graph includes a user device login relationship graph and a user device verification relationship graph. The device features are obtained by fusing the device features in the user device login relationship graph and the device features in the user device verification relationship graph.

[0336] In other words, the device features used for clustering are based on the same device. The training device fuses the device features of the same device in the user device login relationship graph and the device features in the user device verification relationship graph to obtain the fused features of each device for clustering.

[0337] In this embodiment, the training device uses the same device as the basis for clustering and first fuses the device features in different relationship graphs to make the fused features more comprehensive, thereby improving the effectiveness and reliability of clustering.

[0338] For example, taking a user device relationship graph that includes a user device login relationship graph and a user device authentication relationship graph, and using K-means clustering as the clustering method, the following explanation is provided:

[0339] Device features include device features predicted by a multivariate graph feature prediction model based on the user device login relationship graph (for ease of distinction, these device features can be called login device features) and device features predicted by a multivariate graph feature prediction model based on the user device verification relationship graph (for ease of distinction, these device features can be called verification device features).

[0340] For each of the B devices, the training device determines the device features corresponding to that device from the login device features and the verification device features. It then performs a fusion process on the device features determined from the login device features and the device features determined from the verification device features. This fusion process can be understood as performing a series processing and a normalization process in sequence to obtain the device fusion features corresponding to that user. This process is repeated to obtain the device fusion features corresponding to each of the B devices, such as the B device fusion features.

[0341] The training device uses K-means clustering to cluster the fusion features of B devices, resulting in M ​​classes, and thus obtaining the user class attribute corresponding to each fusion feature of the device.

[0342] The second step is to train a risk perception model for each device class attribute based on the device characteristics of that attribute.

[0343] The equipment risk perception model includes risk perception models corresponding to each equipment category attribute.

[0344] In this embodiment, the training device first performs clustering to obtain the attributes of each device class, and then trains the corresponding risk perception model for each device class attribute, so that the training of the risk perception model is more targeted, thereby improving the effectiveness of the risk perception model.

[0345] In some embodiments, S403 may include the following steps:

[0346] The first step is to input the device characteristics into the second network model and output the predicted device attack risk information.

[0347] Similarly, this embodiment does not limit the type, architecture, parameters, etc. of the second network model, which can be determined by the training device based on requirements, historical records, and experiments. For example, the second network model can be an MLP, and specifically, it can be a 3-layer MLP.

[0348] The second step is to calculate the second prediction loss between the predicted device attack risk information and the preset device attack truth value, and generate a device risk perception model based on the second prediction loss.

[0349] The preset device attack truth value is pre-labeled.

[0350] As can be seen from the above examples, the information on the predicted device attack risk can be the predicted number of device attacks, the predicted probability of device attacks, the predicted attack level, and so on.

[0351] Taking the predicted number of device attacks as an example, the preset true value of the device attack is the true value of the number of device attacks. Then the second prediction loss is the difference information between the predicted number of device attacks and the true value of the number of device attacks. The training device iteratively optimizes the second network model based on the second prediction loss until the second network model converges, thus obtaining the user risk perception model.

[0352] Based on the above technical concept, this disclosure also provides a liveness detection device.

[0353] Please see Figure 5 , Figure 5 This is a schematic diagram of a liveness detection device according to an embodiment of the present disclosure, as shown below. Figure 5 As shown, the liveness detection device 500 includes:

[0354] The acquisition unit 501 is used to acquire user attack risk information of the target user who triggered the task and device attack risk information of the target device that triggered the task in response to the triggering of the liveness detection task.

[0355] The determining unit 502 is used to determine the liveness category of the task based on the user attack risk information and the device attack risk information, wherein the liveness category is either liveness or attack.

[0356] In some embodiments, the target user has target user characteristics; the user attack risk information is determined based on the target user characteristics.

[0357] The target device has target device characteristics; the device attack risk information is determined based on the target device characteristics.

[0358] In some embodiments, the target user characteristics and the target device characteristics are determined from a pre-built user device relationship graph.

[0359] The user device relationship diagram includes user nodes and device nodes. The user nodes include user features, and the device nodes include device features. The user nodes include nodes of the target user, and the device nodes include nodes of the target device. The user features include the target user features, and the device features include the target device features.

[0360] In some embodiments, the user device relationship graph further includes edges between the user nodes and the device nodes; the user device relationship graph is constructed based on acquired sample data; the sample data includes user characteristics, device characteristics, and user behavior information of the user using the device.

[0361] The user node is constructed based on the user, the device node is constructed based on the device, and the edge is constructed based on the usage behavior information.

[0362] In some embodiments, the task has a type attribute, which is a login type or an authentication type; the user device relationship diagram includes a user device login relationship diagram and a user device authentication relationship diagram.

[0363] If the type attribute is the login type, then the target user characteristics and the target device characteristics are determined from the user device login relationship graph.

[0364] If the type attribute is the verification type, then the target user feature and the target device feature are determined from the user device verification relationship graph.

[0365] In some embodiments, the user attack risk information is obtained by performing attack risk prediction processing on the target user characteristics based on a pre-trained user risk perception model.

[0366] The device attack risk information is obtained by performing attack risk prediction processing on the characteristics of the target device based on a pre-trained device risk perception model.

[0367] In some embodiments, the target user feature has a target user class attribute, and the target device feature has a target device class attribute; the user risk perception model includes a risk perception model corresponding to each user class attribute, and the user class attribute includes the target user class attribute; the device risk perception model includes a risk perception model corresponding to each device class attribute, and the device class attribute includes the target device class attribute.

[0368] The user attack risk information is obtained by performing attack risk prediction processing on the target user characteristics based on the risk perception model corresponding to the target user class attributes in the user risk perception model.

[0369] The device attack risk information is obtained by performing attack risk prediction processing on the characteristics of the target device based on the risk perception model corresponding to the target device class attribute in the device risk perception model.

[0370] In some embodiments, the user risk perception model is obtained by clustering the user features to obtain user class attributes, and training the user features corresponding to each user class attribute.

[0371] The equipment risk perception model is obtained by clustering the equipment features to obtain equipment class attributes, and training the equipment features corresponding to each equipment class attribute.

[0372] In some embodiments, the user features include user features in the user device login relationship graph and user features in the user device verification relationship graph; the device features include device features in the user device login relationship graph and device features in the user device verification relationship graph.

[0373] The user type attributes are obtained by fusing the user features of the user in the user device login relationship diagram and the user features of the user in the user device verification relationship diagram for each user, and then performing clustering processing on the user fusion features corresponding to each user.

[0374] The device type attributes are obtained by fusing the device features of the device in the user device login relationship diagram and the device features of the device in the user device verification relationship diagram for each device, and then performing clustering processing on the device fusion features corresponding to each device.

[0375] In some embodiments, such as Figure 5 As shown, the device 500 further includes:

[0376] The prediction unit 503 is used to perform attack risk prediction processing on the acquired target image that triggers the task, and obtain image attack risk information.

[0377] Furthermore, the determining unit 502 is configured to determine the liveness category based on the image attack risk information, the user attack risk information, and the device attack risk information.

[0378] In some embodiments, the image attack risk information includes the image attack probability, the user attack risk information includes the number of user attacks, and the device attack risk information includes the number of device attacks; the determining unit 502 includes:

[0379] The calculation subunit 5021 is used to calculate the total number of attacks based on the number of user attacks and the number of device attacks.

[0380] The determination subunit 5022 is used to determine the liveness category based on the total number of attacks and the image attack probability.

[0381] In some embodiments, the determining subunit 5022 includes:

[0382] The first determining module is used to determine the predicted attack probability corresponding to the total number of attacks.

[0383] The calculation module is used to calculate the total attack probability based on the predicted attack probability and the image attack probability.

[0384] The second determining module is used to determine the liveness category as an attack if the total probability of the attack reaches a preset threshold, and to determine the liveness category as a liveness if the total probability of the attack does not reach the preset threshold.

[0385] Based on the above technical concept, this disclosure also provides a training device for a liveness detection model.

[0386] Please see Figure 6 , Figure 6 This is a schematic diagram of a training apparatus for a liveness detection model according to an embodiment of the present disclosure, as shown below. Figure 6 As shown, the training device 600 for the liveness detection model includes:

[0387] The acquisition unit 601 is used to acquire user features and device features from a preset user device relationship diagram.

[0388] The first training unit 602 is used to train a user risk perception model based on the user characteristics. The user risk perception model is used to predict user attack risk information of the target user in the liveness detection task.

[0389] The second training unit 603 is used to train a device risk perception model based on the device characteristics. The device risk perception model is used to predict the device attack risk information of the target device in the task.

[0390] The user attack risk information and the device attack risk information are used to determine the liveness category of the task; the liveness detection model includes the user risk perception model and the device risk perception model.

[0391] In some embodiments, the user device relationship graph includes user nodes, device nodes, and edges between the user nodes and the device nodes. The user nodes include the user characteristics, the device nodes include the device characteristics, and the edges are used to characterize user behavior information when using the device.

[0392] The user features and the device features are obtained by optimizing adjacent nodes in the user-device relationship graph, where adjacent nodes are user nodes and device nodes connected by the edges.

[0393] In some embodiments, the user features and the device features are determined based on the multivariate graph feature prediction model, which is based on the user-device relationship graph to predict the features corresponding to each node and is trained according to the feature difference information between adjacent nodes.

[0394] In some embodiments, the first training unit 602 includes:

[0395] The first clustering subunit 6021 is used to perform clustering processing on the user features to obtain the attributes of each user class.

[0396] The first training subunit 6022 is used to train the risk perception model corresponding to each user class attribute based on the user characteristics of that user class attribute.

[0397] The user risk perception model includes risk perception models corresponding to each user class attribute.

[0398] In some embodiments, the user equipment relationship graph includes a user equipment login relationship graph and a user equipment verification relationship graph; the user features are obtained by fusing user features in the user equipment login relationship graph and user features in the user equipment verification relationship graph.

[0399] In some embodiments, the second training unit 603 includes:

[0400] The second clustering subunit 6031 is used to perform clustering processing on the device features to obtain the attributes of each device class.

[0401] The second training subunit 6032 is used to train the risk perception model corresponding to each device class attribute based on the device characteristics of that device class attribute.

[0402] The equipment risk perception model includes risk perception models corresponding to each equipment class attribute.

[0403] In some embodiments, the user equipment relationship graph includes a user equipment login relationship graph and a user equipment verification relationship graph; the device feature is a feature obtained by fusing the device features in the user equipment login relationship graph and the device features in the user equipment verification relationship graph.

[0404] In some embodiments, the first training unit 602 includes:

[0405] The first input subunit 6023 is used to input the user features into the first network model and output predicted user attack risk information.

[0406] The first calculation subunit 6024 is used to calculate the first prediction loss between the predicted user attack risk information and the preset user attack truth value.

[0407] The first generation subunit 6025 is used to generate the user risk perception model based on the first predicted loss.

[0408] In some embodiments, the second training unit 603 includes:

[0409] The second input subunit 6033 is used to input the device features into the second network model and output predicted device attack risk information.

[0410] The second calculation subunit 6034 is used to calculate the second prediction loss between the predicted device attack risk information and the preset device attack truth value.

[0411] The second generation subunit 6035 is used to generate the equipment risk perception model based on the second predicted loss.

[0412] Based on the above-described technical concept, this disclosure also provides a processor-readable storage medium storing a computer program for causing the processor to execute the liveness detection method as described in any of the above embodiments; or, the computer program for causing the processor to execute the training method for the liveness detection model as described in any of the above embodiments.

[0413] Based on the above technical concept, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the liveness detection method as described in any of the above embodiments; or, implements the training method for the liveness detection model as described in any of the above embodiments.

[0414] Based on the above technical concept, this disclosure also provides a liveness detection system, comprising:

[0415] At least one memory, the memory including at least one set of instructions for pushing information;

[0416] At least one processor communicates with the at least one memory;

[0417] When the at least one processor executes the at least one set of instructions, the liveness detection method as described in any of the above embodiments is implemented.

[0418] Based on the above technical concept, this disclosure also provides a training system for a liveness detection model, comprising:

[0419] At least one memory, the memory including at least one set of instructions for pushing information;

[0420] At least one processor communicates with the at least one memory;

[0421] When the at least one processor executes the at least one set of instructions, the training method for the liveness detection model as described in any of the above embodiments is implemented.

[0422] Based on the above technical concept, this disclosure also provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0423] The memory stores computer-executed instructions;

[0424] The processor executes computer execution instructions stored in the memory to implement the training method of the liveness detection model as described in any of the above embodiments; or, to implement the training method of the liveness detection model as described in any of the above embodiments.

[0425] in, Figure 7This is a hardware structure diagram of an electronic device 700 according to an embodiment of the present disclosure. The electronic device 700 can execute the liveness detection method or the liveness detection model training method as described in any of the above embodiments.

[0426] The liveness detection described in this disclosure is applied to, for example, Figure 1 Taking the application scenario shown as an example, the electronic device 700 is described as follows:

[0427] When the liveness detection method described in any of the above embodiments is executed on client 102, electronic device 700 can be client 102. When the liveness detection method described in any of the above embodiments is executed on server 103, electronic device 700 can be server 300. When the liveness detection method described in any of the above embodiments is partially executed on client 102 and partially executed on server 103, electronic device 700 can be both client 102 and server 103.

[0428] like Figure 7 As shown, the electronic device 700 may include at least one storage medium 701 and at least one processor 702. In some embodiments, the electronic device 700 may also include a communication port 703 and an internal communication bus 704. Additionally, the electronic device 700 may also include an input / output (I / O) component 705.

[0429] The internal communication bus 704 can connect different system components, including storage medium 701, processor 702 and communication port 703.

[0430] I / O component 705 supports input / output between electronic device 700 and other components.

[0431] Communication port 703 is used for data communication between electronic device 700 and the outside world. For example, communication port 703 can be used for data communication between electronic device 700 and network 104. Communication port 703 can be a wired communication port or a wireless communication port.

[0432] Storage medium 701 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 7011, a read-only memory (ROM) 7012, or a random access memory (RAM) 7013. Storage medium 701 also includes at least one instruction set stored in the data storage device. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the liveness detection method provided in this specification.

[0433] At least one processor 702 can be communicatively connected to at least one storage medium 701 and a communication port 703 via an internal communication bus 704. The at least one processor 702 is used to execute at least one instruction set described above. When the electronic device 700 is running, the at least one processor 702 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the liveness detection method provided in this specification. The processor 702 can execute all the steps included in the liveness detection method. Processor 702 may take the form of one or more processors. In some embodiments, processor 702 may include one or more hardware processors, such as microcontrollers, microprocessors, reduced instruction set computers (RISC), application-specific integrated circuits (ASICs), application-specific instruction processors (ASIPs), central processing units (CPUs), graphics processing units (GPUs), physical processing units (PPUs), microcontroller units, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), advanced RISC machines (ARMs), programmable logic devices (PLDs), any circuitry or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 702 is described in the electronic device 700 of this specification. However, it should be noted that the electronic device 700 in this specification may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described in this specification, or they may be executed jointly by multiple processors. For example, if the processor 702 of the electronic device 700 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 702 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0434] It should be noted that the multimodal liveness detection model in this embodiment is not designed for a specific user's face image and cannot reflect the personal information of a specific user. It should also be noted that the face images in this embodiment are from a publicly available dataset.

[0435] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0436] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0437] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0438] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0439] These processors can execute instructions that can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0440] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for detecting liveness, characterized in that, The method includes: In response to the triggering of a liveness detection task, the target user characteristics and target device characteristics of the target user who triggered the task are obtained, wherein the target device characteristics include the characteristics of the target user using the target device; The attack risk prediction process is performed on the target user characteristics based on a pre-trained user risk perception model to obtain user attack risk information, wherein the user attack risk information is used to characterize the probability or confidence level that the target user is an attacking user. Based on a pre-trained device risk perception model, attack risk prediction processing is performed on the characteristics of the target device to obtain device attack risk information, wherein the device attack risk information is used to characterize the probability or confidence level that the target device is an offensive device; and Based on the user attack risk information and the device attack risk information, the liveness category of the task is determined, wherein the liveness category is either live or attack; The target user features and target device features are determined from a pre-constructed user device relationship graph. This graph includes user nodes and device nodes; each user node includes user features, and each device node includes device features. The user nodes include nodes representing the target user, and each device node includes nodes representing the target device. The user features include the target user features, and the device features include the target device features. The graph also includes edges between user nodes and device nodes. The graph is constructed based on acquired sample data. This sample data includes user features, device features, and user behavior information related to the device. User nodes are constructed based on the user, device nodes are constructed based on the device, and edges are constructed based on the user behavior information.

2. The method according to claim 1, characterized in that, The task has a type attribute, which is either a login type or an authentication type; the user device relationship diagram includes a user device login relationship diagram and a user device authentication relationship diagram. If the type attribute is the login type, then the target user characteristics and the target device characteristics are determined from the user device login relationship graph; If the type attribute is the verification type, then the target user feature and the target device feature are determined from the user device verification relationship graph.

3. The method according to claim 2, characterized in that, The target user feature has a target user class attribute, and the target device feature has a target device class attribute; the user risk perception model includes a risk perception model corresponding to each user class attribute, and the user class attribute includes the target user class attribute; the device risk perception model includes a risk perception model corresponding to each device class attribute, and the device class attribute includes the target device class attribute; The user attack risk information is obtained by performing attack risk prediction processing on the target user characteristics based on the risk perception model corresponding to the target user class attribute in the user risk perception model. The device attack risk information is obtained by performing attack risk prediction processing on the characteristics of the target device based on the risk perception model corresponding to the target device class attribute in the device risk perception model.

4. The method according to claim 3, characterized in that, The user risk perception model is obtained by clustering the user features to obtain user class attributes, and training the user features corresponding to each user class attribute. The equipment risk perception model is obtained by clustering the equipment features to obtain equipment class attributes, and training the equipment features corresponding to each equipment class attribute.

5. The method according to claim 4, characterized in that, The user features include the user features in the user device login relationship graph and the user features in the user device verification relationship graph; the device features include the device features in the user device login relationship graph and the device features in the user device verification relationship graph. Each user type attribute is obtained by fusing the user features of the user in the user device login relationship diagram and the user features of the user in the user device verification relationship diagram for each user, and then performing clustering processing on the user fusion features corresponding to each user. Each device type attribute is obtained by fusing the device features of the device in the user device login relationship diagram and the device features of the device in the user device verification relationship diagram for each device, and then performing clustering processing on the device fusion features corresponding to each device.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: The acquired target image that triggers the task is subjected to attack risk prediction processing to obtain image attack risk information; In addition, determining the liveness category of the task based on the user attack risk information and the device attack risk information includes: determining the liveness category based on the image attack risk information, the user attack risk information, and the device attack risk information.

7. The method according to claim 6, characterized in that, The image attack risk information includes the image attack probability, the user attack risk information includes the number of user attacks, and the device attack risk information includes the number of device attacks. Based on the image attack risk information, the user attack risk information, and the device attack risk information, the liveness category is determined, including: The total number of attacks is calculated based on the number of user attacks and the number of device attacks. The liveness category is determined based on the total number of attacks and the image attack probability.

8. The method according to claim 7, characterized in that, The liveness category is determined based on the total number of attacks and the image attack probability, including: Determine the predicted attack probability corresponding to the total number of attacks; The total attack probability is calculated based on the predicted attack probability and the image attack probability. If the total probability of the attack reaches a preset threshold, the liveness category is determined as an attack; if the total probability of the attack does not reach the preset threshold, the liveness category is determined as a liveness.

9. A training method for a liveness detection model, characterized in that, The method includes: Obtain user characteristics and device characteristics from a pre-defined user device relationship diagram; A user risk perception model is trained based on the user characteristics. The user risk perception model is used to predict the user attack risk information of the target user in the liveness detection task. The user attack risk information is used to characterize the probability or confidence that the target user is an aggressive user. A device risk perception model is trained based on the device characteristics. The device risk perception model is used to predict the device attack risk information of the target device in the task. The device attack risk information is used to characterize the probability or confidence that the target device is an attack device. The user attack risk information and the device attack risk information are used to determine the liveness category of the task; the liveness detection model includes the user risk perception model and the device risk perception model. The user device relationship graph includes user nodes, device nodes, and edges between the user nodes and the device nodes. The user nodes include user features, the device nodes include device features, and the edges are used to characterize user behavior information when using the device. The user features and the device features are obtained by optimizing adjacent nodes in the user device relationship graph, where adjacent nodes are user nodes and device nodes connected by the edges.

10. The method according to claim 9, characterized in that, The user features and the device features are determined based on a multivariate graph feature prediction model. The multivariate graph feature prediction model is based on the user-device relationship graph to predict the features corresponding to each node, and is trained based on the feature difference information between adjacent nodes.

11. The method according to claim 9, characterized in that, A user risk perception model is trained based on the user characteristics, including: The user features are clustered to obtain the attributes of each user class; For each user class attribute, a risk perception model corresponding to that user class attribute is trained based on the user characteristics of that user class attribute. The user risk perception model includes risk perception models corresponding to each user class attribute.

12. The method according to claim 9, characterized in that, The user equipment relationship graph includes a user equipment login relationship graph and a user equipment verification relationship graph; the user features are obtained by fusing the user features in the user equipment login relationship graph and the user features in the user equipment verification relationship graph.

13. The method according to claim 9, characterized in that, A device risk perception model is trained based on the device characteristics, including: The device features are clustered to obtain the attributes of each device class; For each equipment class attribute, a risk perception model corresponding to that equipment class attribute is trained based on the equipment characteristics of that equipment class attribute. The equipment risk perception model includes risk perception models corresponding to each equipment class attribute.

14. The method according to claim 9, characterized in that, The user equipment relationship diagram includes a user equipment login relationship diagram and a user equipment verification relationship diagram; the device feature is a feature obtained by fusing the device features in the user equipment login relationship diagram and the device features in the user equipment verification relationship diagram.

15. The method according to claim 9, characterized in that, A user risk perception model is trained based on the user characteristics, including: The user characteristics are input into the first network model, and the predicted user attack risk information is output. Calculate the first prediction loss between the predicted user attack risk information and the preset user attack truth value, and generate the user risk perception model based on the first prediction loss.

16. The method according to any one of claims 9-15, characterized in that, A device risk perception model is trained based on the device characteristics, including: The device characteristics are input into the second network model, which outputs predicted device attack risk information. Calculate the second prediction loss between the predicted device attack risk information and the preset device attack truth value, and generate the device risk perception model based on the second prediction loss.

17. A liveness detection device, characterized in that, The device includes: The acquisition unit is configured to, in response to the triggering of a liveness detection task, acquire the target user characteristics and target device characteristics of the target user who triggered the task, perform attack risk prediction processing on the target user characteristics based on a pre-trained user risk perception model to obtain user attack risk information, and perform attack risk prediction processing on the target device characteristics based on a pre-trained device risk perception model to obtain device attack risk information. The target device characteristics include the characteristics of the target user using the target device, the user attack risk information is used to characterize the probability or confidence level of the target user being an attacking user, and the device attack risk information is used to characterize the probability or confidence level of the target device being an attacking device. The determining unit is configured to determine the liveness category of the task based on the user attack risk information and the device attack risk information, wherein the liveness category is either liveness or attack; The target user features and target device features are determined from a pre-constructed user device relationship graph. This graph includes user nodes and device nodes; each user node includes user features, and each device node includes device features. The user nodes include nodes representing the target user, and each device node includes nodes representing the target device. The user features include the target user features, and the device features include the target device features. The graph also includes edges between user nodes and device nodes. The graph is constructed based on acquired sample data. This sample data includes user features, device features, and user behavior information related to the device. User nodes are constructed based on the user, device nodes are constructed based on the device, and edges are constructed based on the user behavior information.

18. A training device for a liveness detection model, characterized in that, The device includes: The acquisition unit is used to acquire user features and device features from a preset user device relationship diagram; The first training unit is used to train a user risk perception model based on the user characteristics. The user risk perception model is used to predict the user attack risk information of the target user in the liveness detection task. The user attack risk information is used to characterize the probability or confidence that the target user is an aggressive user. The second training unit is used to train a device risk perception model based on the device characteristics. The device risk perception model is used to predict the device attack risk information of the target device in the task. The device attack risk information is used to characterize the probability or confidence that the target device is an attack device. The user attack risk information and the device attack risk information are used to determine the liveness category of the task; the liveness detection model includes the user risk perception model and the device risk perception model; the user-device relationship graph includes user nodes, device nodes, and edges between the user nodes and the device nodes, the user nodes include user features, the device nodes include device features, and the edges are used to characterize user behavior information when using the device; the user features and the device features are obtained by optimizing adjacent nodes in the user-device relationship graph, wherein the adjacent nodes are user nodes and device nodes connected by the edges.

19. A liveness detection system, characterized in that, include: At least one memory, the memory including at least one set of instructions for pushing information; At least one processor communicates with the at least one memory; Wherein, when the at least one processor executes the at least one set of instructions, the method as described in any one of claims 1 to 8 is implemented.

20. A training system for a liveness detection model, characterized in that, include: At least one memory, the memory including at least one set of instructions for pushing information; At least one processor communicates with the at least one memory; Wherein, when the at least one processor executes the at least one set of instructions, the method as described in any one of claims 9 to 16 is implemented.

Images