A method and system for liveness detection

Abstract

The live body recognition method provided in the specification obtains a target face image of a target user and a voice database in a target geographical area within a preset time period, performs cross-modal comparison on the target face image and the voice database to determine a probability that the target user belongs to a basic user in the voice database, and determines an attack risk in live body recognition. In this way, the target face image of the target user is verified through low-cost voice cross-modal comparison, which improves the accuracy of live body recognition while reducing the cost of live body recognition.

Description

Technical Field

[0001] This specification relates to the field of image processing, and in particular to a liveness detection method and system. Background Technology

[0002] Biometric systems, such as facial recognition systems, have been widely used in recent years, achieving success in scenarios such as facial recognition payment, facial recognition entry, and facial recognition attendance. However, facial recognition systems also face security challenges, with liveness detection attacks being one of the main security threats. Examples of such attacks include photos displayed on electronic screens, printed photos, and 3D masks.

[0003] Therefore, in order to accurately detect liveness attacks during the biometric process, a new liveness detection method is urgently needed. Summary of the Invention

[0004] The liveness detection method and system provided in this specification improve the accuracy of liveness detection while reducing its cost by performing low-cost cross-modal voice verification of the target user's face image.

[0005] In a first aspect, this specification provides a liveness detection method, comprising: obtaining a target face image of a target user through a target terminal; obtaining a voice database, the voice database including voices corresponding to multiple basic users located within a target geographical area within a preset time period, the target terminal being located within the target geographical area; performing a cross-modal comparison between the target face image and the voice database, and determining the comparison result, the comparison result including the probability that the target user belongs to the multiple basic users; and determining a liveness detection result corresponding to the target user based on the comparison result.

[0006] In some embodiments, the voice includes data in which voice content information is removed but identity information is retained.

[0007] In some embodiments, the voice database is obtained by: obtaining the original voices of the plurality of basic users; and performing desensitization processing on the original voices of the plurality of basic users to obtain desensitized voices; and obtaining the voice database based on the desensitized voices.

[0008] In some embodiments, the process of desensitizing the original speech of the plurality of basic users to obtain the desensitized speech includes: inputting the original speech into a pre-trained content desensitization module and outputting the desensitized speech. The training objectives of the content desensitization module during the training process include: the pre-trained content recognition module being unable to recognize the content in the original speech of the training sample from the desensitized speech corresponding to the training sample, and the pre-trained voiceprint recognition module being able to recognize the identity corresponding to the original speech of the training sample from the desensitized speech corresponding to the training sample.

[0009] In some embodiments, obtaining the voice database based on the de-identified speech includes: extracting identity features from the de-identified speech to obtain first identity features; and storing multiple first identity features corresponding to the multiple basic users into the voice database.

[0010] In some embodiments, the step of performing cross-modal comparison between the target face image and the voice database and determining the comparison result includes: determining the image features of the target face image; performing cross-modal comparison between the image features and each of the plurality of first identity features, and determining the comparison result.

[0011] In some embodiments, the step of performing a cross-modal comparison between the image feature and each of the plurality of first identity features to determine the comparison result includes: determining the feature similarity between the image feature and each of the first identity features; and determining the maximum similarity with the highest feature similarity as the comparison result.

[0012] In some embodiments, the image features include speech modality information.

[0013] In some embodiments, the image features are obtained through an image feature encoder. The training objective of the image feature encoder during the training process includes: the pre-trained modality transfer module is able to generate cross-modal speech features that satisfy a preset first transfer condition based on the image features corresponding to the face image training samples. The first transfer condition includes that the difference between the cross-modal speech features and the real speech features corresponding to the face image training samples is less than a preset first difference threshold. The first identity feature is obtained based on an identity feature extraction model, and the real speech features corresponding to the face image training samples are obtained by extracting identity features from the real speech corresponding to the face image training samples through the identity feature extraction model.

[0014] In some embodiments, determining the feature similarity between the image feature and each first identity feature includes, for each first identity feature: inputting the image feature and the current first identity feature into a cross-modal verification module, and outputting the feature similarity, wherein the feature similarity includes the probability that the image feature and the current first identity feature belong to the same identity.

[0015] In some embodiments, the step of performing cross-modal comparison between the image features and each of the first identity features to determine the comparison result includes: determining a plurality of second identity features corresponding to the plurality of first identity features; determining the feature similarity between the image features and each of the plurality of second identity features; and determining the maximum similarity with the highest feature similarity as the comparison result.

[0016] In some embodiments, the second identity feature includes image modality information.

[0017] In some embodiments, the second identity feature is obtained through a speech feature encoder. The training objective of the speech feature encoder during the training process includes: the pre-trained modality transfer module is able to generate cross-modal image features that satisfy a preset second transfer condition based on the second identity feature corresponding to the speech training sample. The second transfer condition includes that the difference between the cross-modal image feature and the real face image feature corresponding to the speech training sample is less than a preset second difference threshold. The image feature is obtained by feature extraction of the target face image based on the image feature encoder, and the real face image feature corresponding to the speech training sample is obtained by feature extraction of the real face image corresponding to the speech training sample based on the image feature encoder.

[0018] In some embodiments, determining the feature similarity between the image feature and each of the plurality of second identity features includes, for each second identity feature: inputting the image feature and the current second identity feature into a cross-modal verification module, and outputting the feature similarity, wherein the feature similarity includes the probability that the image feature and the current second identity feature belong to the same identity.

[0019] In some embodiments, determining the liveness detection result corresponding to the target user based on the comparison result includes: determining a liveness detection threshold based on the comparison result; determining the attack probability that the target user belongs to an attack based on the target face image; and determining the liveness detection result based on the attack probability and the liveness detection threshold.

[0020] In some embodiments, determining the liveness detection result based on the attack probability and the liveness detection threshold includes: determining that the attack probability is greater than the liveness detection threshold and determining that the liveness detection result is an attack; or determining that the attack probability is less than the liveness detection threshold and determining that the liveness detection result is a live entity.

[0021] Secondly, this specification also provides a liveness detection system, including at least one storage medium storing at least one set of instructions for implementing liveness detection; and at least one processor communicatively connected to the at least one storage medium, wherein when the liveness detection system is running, the at least one processor reads the at least one set of instructions and implements the liveness detection method described in the first aspect.

[0022] As can be seen from the above technical solutions, the liveness detection method and system provided in this specification acquire a target user's facial image and a voice database within a target geographical area over a preset time period. By performing a cross-modal comparison between the target facial image and the voice database, the probability that the target user belongs to a base user in the voice database is determined, thereby assessing the attack risk during liveness detection. The method and system can also dynamically adjust the liveness detection threshold based on the probability that the target user belongs to a base user. Since the base users in a fixed location are relatively fixed and fluctuate little, when the probability that the target user belongs to the base user is high, the possibility that the target user is an attacker is low. In this case, the threshold can be adjusted to make the target user more easily identified as a normal user; conversely, when the probability that the target user belongs to the base user is low, the possibility that the target user is an attacker is high. In this case, the threshold can be adjusted to make the target user more easily identified as an attacker. Thus, by using low-cost cross-modal voice verification of the target user's facial image, the accuracy of liveness detection is improved while reducing the cost of liveness detection.

[0023] Other functions of the liveness detection methods and systems provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the liveness detection methods and systems provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 The diagram illustrates an application scenario of a liveness detection system according to some embodiments of this specification.

[0026] Figure 2 A hardware structure diagram of a computing device provided according to some embodiments of this specification is shown;

[0027] Figure 3 A flowchart of a liveness detection method according to some embodiments of this specification is shown; and

[0028] Figure 4 A flowchart of a liveness detection method provided according to some embodiments of this specification is shown. Detailed Implementation

[0029] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0030] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0031] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0032] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0033] Before describing the specific embodiments in this specification, the application scenarios of this specification will be introduced as follows:

[0034] Multiple liveness detection devices can be installed within a certain geographical area. For example, a liveness detection device can be installed in locations such as vending machines, restaurants, and supermarkets within an office park, or at the entrance of the office park. The liveness detection device can be a fixed device, i.e., permanently installed in a certain location, or a portable device. The geographical area can be an open area such as a country, a city, a subway line, or a station area, or a relatively closed small area such as a park, a company, or a campus. For ease of description, the following text primarily uses a relatively closed small area as an example to introduce the liveness detection method in this specification. It should be understood that the liveness detection method and system provided in this specification can also be applied to other scenarios, and are not limited to the scenarios described above.

[0035] For relatively enclosed, small-scale areas, the users active within are relatively fixed and their activity is minimal. Examples include company buildings with access control, industrial parks, residential communities, and campuses. Those who can enter these areas are mostly regular users of that area. Generally, liveness detection attacks randomly select users to attack. When a target user undergoes liveness detection within such a relatively enclosed, small-scale area, if the target user is a regular user within that area, the probability that the target user is an attacker is relatively low; conversely, if the target user is not a regular user within that area, the probability that the target user is an attacker is relatively high.

[0036] For ease of description, this manual explains the terms that will appear in the following descriptions:

[0037] Attacks / Liveness Attacks: Non-liveness attacks refer to attack methods targeting liveness detection systems, including electronic photos displayed on mobile phone screens, printed paper photos, high-precision masks, molds, prostheses, etc.

[0038] Liveness detection / liveness attack prevention: Algorithm technology that uses artificial intelligence models to detect and judge liveness and attacks in order to detect and intercept attacks;

[0039] Voiceprint recognition: In this specification, it refers to identifying a user's identity through voice signals; and

[0040] Cross-modal verification: In this specification, it refers to matching the facial features and voice features of users with the same identity.

[0041] Figure 1The diagram illustrates an application scenario of a liveness detection system 001 according to some embodiments of this specification. For example... Figure 1 As shown, the application scenario may include system 001, target user 100, and network 400. System 001 may include target terminal 200 and server 300.

[0042] The target user 100 can be any user who uses the target terminal 200 for liveness detection.

[0043] The target terminal 200 may include an image acquisition device that can acquire a target user 100's facial image. Of course, the image acquisition device can also acquire other images of the target user 100, such as fingerprint images, iris images, retinal images, hand images, etc. The image acquisition device 210 can also acquire behavioral information of the target user 100, such as voice, behavior, gait, etc. The image acquisition device can be one or more of a visible light camera (such as an RGB camera), a near-infrared camera, a depth camera (such as a 3D structured light camera, a laser detector), and a thermal imaging camera.

[0044] The target terminal 200 can be any liveness detection device located within the target geographical area A. For example... Figure 1 As shown, target terminal 200 is located at the entrance of target geographical area A. Target user 100 needs to pass the liveness detection of target terminal 200 to enter target geographical area A. There can be many basic users within target geographical area A. For example, target geographical area A is an office park, and the basic users can be employees, logistics personnel, security personnel, cleaning personnel, etc., of various companies within the office park. When the basic users speak, the voice acquisition device (not shown in the figure) within target geographical area A can collect the voice of each basic user. It should be noted that the collection of user voices in this specification is authorized by the users, and the collected voices have undergone content anonymization processing; the liveness detection process does not involve user privacy. To enrich the voice database, multiple voice acquisition devices can be set up within target geographical area A. These multiple voice acquisition devices can send the collected voices to a voice collection device (not shown in the figure) for storage. The voice acquisition device can be any type of device capable of collecting voices. The voice collection device can establish a voice database based on the collected voice set, or the voice collection device can send the voice set to a device executing the liveness detection method, which then establishes a voice database based on the voice set. The voice collection device can be a target terminal 200 or a server 300, or it can be other target terminals or other servers.

[0045] The target terminal 200 or server 300 can use the liveness detection method described in this specification to perform liveness detection on a target face image and output the liveness detection result. In some embodiments, the liveness detection method can be executed on the target terminal 200. In this case, the target terminal 200 may store data or instructions for executing the liveness detection method described in this specification, and may execute or be used to execute the data or instructions. In some embodiments, the target terminal 200 may include a hardware device with data information processing capabilities and the necessary programs required to drive the hardware device to work. Figure 1 As shown, the target terminal 200 can communicate with the server 300. In some embodiments, the server 300 can communicate with multiple target terminals 200. In some embodiments, the target terminal 200 can interact with the server 300 through the network 400 to receive or send messages, such as receiving or sending target facial images or various feature information, such as two-dimensional images / features and / or three-dimensional images / features. In some embodiments, the target terminal 200 may include a mobile device, tablet computer, laptop computer, built-in device of a motor vehicle or similar content, Alipay's Dragonfly device, vending machine, vending cabinet, or any combination thereof. In some embodiments, the mobile device may include a smart home device, smart mobile device, virtual reality device, augmented reality device or similar device, or any combination thereof. In some embodiments, the smart home device may include a smart TV, desktop computer, etc., or any combination thereof. In some embodiments, the smart mobile device may include a smartphone, personal digital assistant, gaming device, navigation device, etc., or any combination thereof. In some embodiments, the virtual reality device or augmented reality device may include a virtual reality headset, virtual reality glasses, virtual reality patch, augmented reality headset, augmented reality glasses, augmented reality patch or similar content, or any combination thereof. For example, the virtual reality device or the augmented reality device may include Google Glass, a head-mounted display, VR, etc. In some embodiments, the built-in device in the motor vehicle may include an in-vehicle computer, an in-vehicle television, etc. In some embodiments, the target terminal 200 may be a device with positioning technology for locating the position of the target terminal 200. In some embodiments, the target terminal 200 may have one or more of the following functions: NFC (Near Field Communication), WIFI (Wireless Fidelity), 3G / 4G / 5G, POS (Point of Sale) card swiping function, QR code scanning function, barcode scanning function, Bluetooth, infrared, SMS (Short Message Service), MMS (Multimedia Message Service).

[0046] In some embodiments, the target terminal 200 may have one or more applications (APPs) installed. The APPs provide the target user 100 with the ability and interface to interact with the outside world via the network 400. The APPs include, but are not limited to: web browser APPs, search APPs, chat APPs, shopping APPs, video APPs, financial management APPs, instant messaging tools, email target terminals, social media platform software, etc. In some embodiments, the target terminal 200 may have a target APP installed. The target APP can capture target facial images for the target terminal 200. In some embodiments, the target APP can also recognize the target facial images. The target user 100 can trigger a liveness detection request through the target APP. The target APP can respond to the liveness detection request and execute a liveness detection method.

[0047] Server 300 may be a server providing various services, such as a backend server supporting the pages displayed by the target APP on the target terminal 200. In some embodiments, the liveness detection method may be executed on server 300. In this case, server 300 may store data or instructions for executing the liveness detection method described herein, and may execute or be used to execute the data or instructions. In some embodiments, server 300 may include a hardware device with data processing capabilities and the necessary programs required to drive the hardware device. Server 300 may be communicatively connected to multiple target terminals 200 and receive data sent by the target terminals 200.

[0048] Network 400 serves as a medium to provide a communication connection between target terminal 200 and server 300. Network 400 can facilitate the exchange of information or data. For example... Figure 1 As shown, target terminal 200 and server 300 can connect to network 400 and transmit information or data to each other through network 400. In some embodiments, network 400 can be any type of wired or wireless network, or a combination thereof. For example, network 400 may include cable networks, wired networks, fiber optic networks, telecommunications networks, intranets, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), Bluetooth networks, ZigBee networks, near field communication (NFC) networks, or similar networks. In some embodiments, network 400 may include one or more network access points. For example, network 400 may include wired or wireless network access points, such as base stations or Internet switching points, through which one or more components of target terminal 200 and server 300 can connect to network 400 to exchange data or information.

[0049] It should be understood that Figure 1 The number of target terminals 200, servers 300, and networks 400 shown is merely illustrative. Depending on implementation needs, there can be any number of target terminals 200, servers 300, and networks 400.

[0050] It should be noted that the liveness detection method can be executed entirely on the target terminal 200, entirely on the server 300, or partially on the target terminal 200 and partially on the server 300.

[0051] Figure 2 A hardware structure diagram of a computing device 600 according to some embodiments of this specification is shown. The computing device 600 can execute the liveness detection method described in this specification. The liveness detection method is described in other parts of this specification. When the liveness detection method is executed on a target terminal 200, the computing device 600 can be the target terminal 200. When the liveness detection method is executed on a server 300, the computing device 600 can be the server 300. When the liveness detection method can be executed partly on the target terminal 200 and partly on the server 300, the computing device 600 can be either the target terminal 200 or the server 300.

[0052] like Figure 2 As shown, the computing device 600 may include at least one storage medium 630 and at least one processor 620. In some embodiments, the computing device 600 may also include a communication port 650 and an internal communication bus 610. Additionally, the computing device 600 may include I / O components 660.

[0053] The internal communication bus 610 can connect different system components, including storage medium 630, processor 620 and communication port 650.

[0054] I / O component 660 supports input / output between computing device 600 and other components.

[0055] Communication port 650 is used for data communication between computing device 600 and external sources. For example, communication port 650 can be used for data communication between computing device 600 and network 400. Communication port 650 can be a wired communication port or a wireless communication port.

[0056] Storage medium 630 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 632, a read-only storage medium (ROM) 634, or a random access storage medium (RAM) 636. Storage medium 630 may store at least one set of instructions for implementing liveness detection. The instructions are computer program code, which may include programs, routines, objects, components, data structures, processes, modules, etc., that execute the liveness detection methods provided in this specification. Storage medium 630 may also store a liveness detection model for implementing the liveness detection method, such as storing a content desensitization module, a voiceprint recognition module, etc. In this case, the model / module may be one or more instruction sets stored in storage medium 630 that execute corresponding instructions, and is executed by processor 620 in computing device 600. Of course, the model may also be a part of the circuitry, hardware device, or module in computing device 600. For example, the content desensitization module can be a hardware device / module in the computing device 600 that implements content desensitization, and the voiceprint recognition module can be a hardware device / module in the computing device 600 that implements identity recognition from voice, and so on. In this case, the processor 620 can store at least one set of instructions for controlling one or more of the models / modules.

[0057] At least one processor 620 can be communicatively connected to at least one storage medium 630 and a communication port 650 via an internal communication bus 610. At least one processor 620 is used to execute the at least one instruction set described above. When the computing device 600 is running, at least one processor 620 can read the at least one instruction set and, according to the instructions of the at least one instruction set, execute the liveness detection method provided in this specification. Processor 620 can execute all the steps included in the liveness detection method. Processor 620 can be in the form of one or more processors. In some embodiments, processor 620 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 620 is described in this specification for the computing device 600. However, it should be noted that the computing device 600 may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described herein, or they may be executed jointly by multiple processors. For example, if processor 620 of the computing device 600 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 620 (e.g., a first processor executes step A, a second processor executes step B, or the first and second processors jointly execute steps A and B).

[0058] Figure 3 A flowchart of a liveness detection method P100 according to some embodiments of this specification is shown. As previously described, computing device 600 can execute the liveness detection method P100 described herein. Specifically, processor 620 can read an instruction set stored in its local storage medium and then execute the liveness detection method P100 described herein according to the provisions of the instruction set. Figure 3 As shown, method P100 may include:

[0059] S120: Obtain the target face image of the target user 100 through the target terminal 200.

[0060] When target user 100 is in front of target terminal 200, the image acquisition device on target terminal 200 can acquire a target facial image of target user 100. Target user 100 can be a real user or a malicious user. A malicious user refers to a user who uses attack methods such as paper photos, mobile phone screens, or masks for liveness detection. A real user refers to a normal user who uses their real identity for liveness detection without using attack methods. If target user 100 is a real user, the liveness detection result is normally a live image; if target user 100 is a malicious user, the liveness detection result is normally an attack. That is, the liveness detection result corresponding to target user 100 includes either a live image or an attack.

[0061] The target face image can be a pre-processed image. For example, the original image acquired by the image acquisition device contains the face region and background region of the target user 100. The computing device 600 can pre-process the original image to remove the background region, thereby obtaining the target face image containing the face region. The target face image can be a single face image, such as a single frontal face image, or multiple face images, such as a frontal face image, a side face image, an open mouth image, a smiling image, a teeth-showing image, etc. To improve the accuracy of liveness detection, the computing device 600 can acquire multiple modal images of the target user 100, such as RGB images, near-infrared images, 3D images, etc., and use multiple modal images as the target face image. This embodiment of the specification does not limit this.

[0062] S140: Obtain the voice database.

[0063] The voice database includes voices corresponding to multiple basic users located within a target geographical area during a preset time period, and the target terminal 200 is located within the target geographical area.

[0064] The computing device 600 can acquire the voice database in advance, that is, acquire the voice database before the target user 100 performs liveness detection, or acquire the voice database when the target user 100 triggers a liveness detection command, for example, when the liveness detection "start" option is selected, or acquire the voice database after the liveness detection command is detected and the target user 100's face image is acquired. This specification does not limit the timing of acquiring the voice database.

[0065] The preset time period can be the period most recent to the current liveness detection action, such as the past week, the past half month, or the past month. The preset time period can be a natural cycle or a cycle calculated backward from the date of the current liveness detection. For example, if the computing device 600 performs the current liveness detection action on Tuesday, the preset time period can be the previous natural week (from last Monday to last Sunday) or a week calculated backward from the current Tuesday (such as from last Tuesday to the current Tuesday).

[0066] If the target geographical area is a small, enclosed region with a relatively fixed number of people, the speech in the speech database is relatively stable and controllable, including relatively fixed background noise. Therefore, it is convenient to use the speech database to verify the target face image, thereby improving the accuracy of liveness detection. Furthermore, the controllable number of people within a small area reduces the probability of different people having the same speech, thus improving the accuracy of cross-modal verification of speech and image, and further improving the accuracy of liveness detection. If the target geographical area is a large, enclosed region, with sufficient hardware and software resources, the method described in this specification can also achieve high-accuracy liveness detection. The enclosed region can refer to an area where all personnel entering the target geographical area must be recorded in a personnel registration database to ensure the security of the target geographical area. Users in the personnel registration database can be users who have passed the regional security authentication of the target geographical area, or users who temporarily entered the target geographical area without passing the regional security authentication but are temporarily registered.

[0067] Before acquiring the voice database, the computing device 600 may first establish the voice database. It should be noted that the computing device 600 may establish the voice database, or a different computing device may establish the voice database and send it to the computing device 600. Taking the establishment of the voice database by the computing device 600 as an example... Figure 4 This illustrates a method for building a speech database, such as... Figure 4 As shown, the method may include:

[0068] S142: Obtain the original voices of the multiple basic users.

[0069] The computing device 600 can acquire the original voice recordings of each basic user. Each basic user can correspond to one original voice recording or multiple original voice recordings. When a basic user corresponds to multiple original voice recordings, the identity characteristics of these multiple original voice recordings are consistent. As mentioned above, multiple voice acquisition devices can be set up within the target geographical area. The original voice recordings can be acquired by multiple voice acquisition devices within the target geographical area. These multiple voice acquisition devices can be communicatively connected to the computing device 600.

[0070] S144: Desensitize the original voices of the multiple basic users to obtain desensitized voices.

[0071] When the basic user corresponds to multiple original voice messages, the computing device 600 can randomly select one message from these messages for desensitization processing to obtain desensitized voice messages, or it can perform desensitization processing on each of the original voice messages, and then use the results of multiple desensitization processes as the desensitized voice messages. That is, the basic user can correspond to one desensitized voice message or multiple desensitized voice messages. The original voice messages and the desensitized voice messages are in the form of a single voice signal, which can be understood as a time-series signal.

[0072] The desensitization process can remove content information from the original speech while retaining the identity information, ensuring that the desensitized speech and subsequent data derived from it do not contain speech content information but do contain identity information, thus protecting user privacy while enabling liveness detection in conjunction with speech. The high-frequency signals in the original speech generally contain content information. The computing device 600 can perform desensitization processing on the high-frequency signals. For example, the computing device 600 can remove content information by deleting the high-frequency signals from the original speech. Alternatively, the computing device 600 can transform the high-frequency signals into signals completely different from the original high-frequency signals. Thus, even if content information exists in the transformed signal, it will be completely different from the content information in the original speech, thereby achieving the purpose of removing content information from the original speech. The high-frequency signals can be speech signals above a preset frequency in the original speech. The preset frequency can be a fixed frequency, such as 1000Hz, 1500Hz, etc. The preset frequency can also adaptively change according to the different original signals. For example, the high-frequency signals in the original male speech are different from those in the original female speech; therefore, the preset frequency corresponding to the original male speech can also be different from that corresponding to the original female speech. For example, a male's high-frequency signal is a speech signal above 1000Hz, while a female's high-frequency signal is a speech signal above 1500Hz. A person's identity is unrelated to the content of their speech; therefore, even if the computing device 600 removes the content information from the original speech, it can still obtain identity information based on the speech after removing the content information.

[0073] In some embodiments, the computing device 600 can input the original speech into a pre-trained content desensitization module and output the desensitized speech. It should be noted that the executing entity for the liveness detection method and the executing entity for model training in this specification can be the same or different. For example, the executing entity for both the liveness detection method and the executing entity for model training may be the target terminal 200; or, for another example, the executing entity for the liveness detection method may be the target terminal 200, while the executing entity for model training may be the server 300.

[0074] The content desensitization module can be trained under the constraints of two pre-trained models: a content recognition module and a pre-trained voiceprint recognition module. For example, the input to the content desensitization module during training is the original speech of the training sample (i.e., the training sample is the original speech), and the output is the desensitized speech corresponding to the training sample. The computing device 600 can input the original speech of the training sample and the corresponding desensitized speech into the pre-trained content recognition module. This content recognition module can identify the content information in the original speech of the training sample, outputting a first content recognition result, and identify the content information in the desensitized speech of the training sample, outputting a second content recognition result. The first and second content recognition results can be, for example, text, or high-frequency signals containing content information, or feature vectors corresponding to the high-frequency signals. The computing device 600 can calculate the difference between the first and second content recognition results, which can be a similarity score. When the difference is less than a preset third difference threshold, the content desensitization module continues to train until the difference between the second content recognition result corresponding to the desensitized speech output by the content desensitization module and the first content recognition result is greater than the third difference threshold. At this point, the difference between the first content recognition result and the second content recognition result is large enough that the pre-trained content recognition module cannot recognize the content in the original speech of the training sample from the anonymized speech corresponding to the training sample. In other words, it cannot obtain the first content recognition result from the anonymized speech corresponding to the training sample, thus achieving the effect of content removal. Therefore, one training objective of the content anonymization module during training can be: the pre-trained content recognition module cannot recognize the content in the original speech of the training sample from the anonymized speech corresponding to the training sample.

[0075] The content desensitization module uses a transformer model. The content recognition module can use models such as CTC (Connectionist Temporal Classification), RNN-T (Recurrent Neural Network Transducer), LAS (Listen Attend and Spell) (a neural network that learns to convert speech into characters), etc.

[0076] The computing device 600 can also input the original speech and corresponding anonymized speech of the training sample into a pre-trained voiceprint recognition module. This module can identify the identity information in the original speech of the training sample, outputting a first identity recognition result, and identify the identity information in the anonymized speech of the training sample, outputting a second identity recognition result. The first and second identity recognition results can be, for example, identity features (such as identity feature vectors). The computing device 600 can calculate the difference between the first and second identity recognition results, which can be a similarity score. When the difference exceeds a preset fourth difference threshold, the content anonymization module continues to train until the difference between the second identity recognition result and the first identity recognition result corresponding to the anonymized speech output by the content anonymization module is less than the fourth difference threshold. At this point, the difference between the first and second identity recognition results is sufficiently small, and the pre-trained voiceprint recognition module can identify the identity corresponding to the original speech of the training sample from the anonymized speech, that is, it can obtain the first identity recognition result from the anonymized speech of the training sample, thereby achieving the effect of preserving the identity. Therefore, another training objective of the content desensitization module during the training process can be that the pre-trained voiceprint recognition module can identify the identity corresponding to the original voice of the training sample from the desensitized voice of the training sample.

[0077] The voiceprint recognition module can be an LSTM (Long Short Term Memory) model, and the computing device 600 can train the voiceprint recognition model using open-source datasets and multi-classification loss.

[0078] The loss function of the content anonymization module during training can include content anonymization loss and identity preservation loss. The content anonymization loss constrains the first content recognition result and the second content recognition result to be as inconsistent as possible, for example, constraining the difference between the two content recognition results to be greater than a third difference threshold. The identity preservation loss constrains the first identity recognition result and the second identity recognition result to be as consistent as possible, for example, constraining the difference between the two identity recognition results to be less than a fourth difference threshold.

[0079] S146: Based on the de-identified speech, obtain the speech database.

[0080] To determine the identity corresponding to each de-identified speech, the computing device 600 can extract identity features from the de-identified speech to obtain a first identity feature. In some embodiments, the computing device 600 can input the de-identified speech into a pre-trained identity feature extraction model and output the first identity feature. The first identity feature is a feature vector or a feature matrix. The loss function of the identity feature extraction model during training can be an identity consistency loss. The identity consistency loss can constrain that for de-identified speech of a training sample (the training sample is de-identified speech), the predicted value of the identity feature corresponding to that training sample output by the identity feature extraction model gradually approaches the true value of the identity feature corresponding to that training sample. It can also constrain that training samples belonging to the same identity have consistent identity features, while training samples belonging to different identities have inconsistent identity features. The model structure of the identity feature extraction model can be an LSTM (Long Short Term Memory) model.

[0081] The computing device 600 can store multiple primary identity features corresponding to multiple basic users in a voice database. The computing device 600 can also store one or more anonymized voice messages corresponding to the multiple basic users in the voice database. In this way, the voice messages in the voice database are data with the voice content information removed but the identity information retained. Of course, the computing device 600 can also store the original voice messages corresponding to the multiple basic users in the voice database.

[0082] S160: Perform cross-modal comparison between the target face image and the voice database, and determine the comparison result, wherein the comparison result includes the probability that the target user 100 belongs to the plurality of basic users.

[0083] There is a correlation between a person's voice and their facial image; a natural mapping exists between the two. Voice contains feature parameters that can annotate the speaker's physiological structure, such as age, gender, and ethnicity. Therefore, the computing device 600 can characterize the speaker's facial features based on the voice data, and then obtain the speaker's facial image based on these features. For example, a clear, cheerful, and high-pitched voice may correspond to the face of a young woman without wrinkles, while a deep, resonant, and low-pitched voice may correspond to the face of a middle-aged man with slight wrinkles. Therefore, by performing a cross-modal comparison between the target facial image and the voice database, the computing device 600 can determine whether the two belong to the same user, thereby facilitating liveness detection.

[0084] The computing device 600 can determine the image features of the target face image. For example, the computing device 600 can input the target face image into a pre-trained image feature encoder and output the image features. Furthermore, the computing device 600 can perform a cross-modal comparison between the image features and each of the plurality of first identity features to determine the comparison result.

[0085] The first identity feature is actually a voice feature. To achieve comparability between the voice feature and the image feature, the computing device 600 can compare the first identity feature and the image feature on the same dimension. For example, the computing device 600 can compare the two on the dimension of image modality or on the dimension of voice modality.

[0086] (1) The computing device 600 compares the first identity feature and the image feature in the dimension of the voice modality.

[0087] When the first identity feature and the image feature are compared in the dimension of speech modality, the image feature needs to have the ability to generate cross-modal speech features, that is, the image feature needs to contain speech modality information. In some embodiments, the computing device 600 can pre-train a modality transfer module, which is used to transfer features of one modality to features of another modality corresponding to the same identity. The modality may include image modality and speech modality. The computing device 600 can use the pre-trained modality transfer module to train and constrain the image feature encoder. For example, the input of the image feature encoder during the training process is a face image training sample, and the output is the image feature corresponding to the training sample. The computing device 600 can input the image feature corresponding to the training sample into the pre-trained modality transfer module, output the cross-modal speech feature corresponding to the training sample, and determine whether the cross-modal speech feature meets the preset first transfer condition. When the cross-modal speech features do not meet the first transfer condition, the image feature encoder continues to be trained until the pre-trained modal transfer module can generate cross-modal speech features that meet the first transfer condition from the image features output by the image feature encoder. At this point, the image features output by the image feature encoder can contain speech modal information, enabling cross-modal comparison between the image features and the first identity feature in the application stage. Therefore, the training objective of the image feature encoder during training can include: the pre-trained modal transfer module can generate cross-modal speech features that meet the preset first transfer condition based on the image features corresponding to the face image training samples.

[0088] The first transfer condition can be: the difference between the cross-modal speech features and the real speech features corresponding to the face image training samples is less than a preset first difference threshold. As mentioned above, the first identity feature is obtained based on an identity feature extraction model. For the real speech features corresponding to the face image training samples, the computing device 600 can acquire the real speech corresponding to the face image training samples, input the real speech into the identity feature extraction model, and thus output the real speech features. That is, the real speech features corresponding to the face image training samples are obtained by extracting identity features from the real speech corresponding to the face image training samples through the identity feature extraction model. The loss function of the image feature encoder during training can be a modality transfer loss, which can constrain the cross-modal speech features to be as consistent as possible with the real speech features.

[0089] In a specific comparison, the computing device 600 can determine the feature similarity between the image feature and each of the first identity features, and determine the maximum similarity as the comparison result. For example, for each of the first identity features: the computing device 600 can input the image feature and the current first identity feature into the cross-modal verification module and output the feature similarity. The feature similarity can include the probability that the image feature and the current first identity feature belong to the same identity. In this way, the probability that the image feature and each of the first identity features belong to the same identity is obtained, where the maximum probability can be considered as the maximum similarity, and this maximum probability can also be the probability that the target user 100 belongs to the multiple basic users. For example, if the maximum probability is 80%, then it is considered that the target user 100 has an 80% probability of belonging to the multiple basic users. The loss function of the cross-modal verification module during training can be the cross-modal verification loss, which can constrain the predicted value of the probability that the image feature of the training sample and the first identity feature of the training sample belong to the same identity to gradually approach the true value of the probability. Of course, the computing device 600 can also calculate feature similarity in other ways, such as Euclidean distance, Pearson correlation coefficient, etc. This manual does not limit the calculation method of feature similarity.

[0090] (2) The computing device 600 compares the first identity feature and the image feature in the dimension of the image modality.

[0091] When comparing the first identity feature and the image feature in the dimension of image modality, the first identity feature needs to have the ability to generate cross-modal image features. Since the first identity feature is a speech feature, it needs to be converted into a feature that includes image modality information. For example, the computing device 600 can re-encode the first identity feature, such as by inputting it into a speech feature encoder for re-encoding, and outputting a second identity feature that includes image modality information.

[0092] The speech feature encoder can be constrained by the modality transfer module during training. For example, the input to the speech feature encoder during training is a speech training sample, which can be a first identity feature of the training sample, and the output is a second identity feature corresponding to the training sample. The computing device 600 can input the second identity feature corresponding to the training sample into the pre-trained modality transfer module, output the cross-modal image feature corresponding to the training sample, and determine whether the cross-modal image feature meets a preset second transfer condition. When the cross-modal image feature does not meet the second transfer condition, the speech feature encoder continues to be trained until the pre-trained modality transfer module can generate cross-modal image features that meet the second transfer condition from the second identity feature output by the speech feature encoder. At this time, the second identity feature output by the speech feature encoder can contain image modality information, and cross-modal comparison of the image feature and the second identity feature can be realized in the application stage. Therefore, the training objective of the speech feature encoder during training can include: the pre-trained modality transfer module can generate cross-modal image features that meet the preset second transfer condition based on the second identity feature corresponding to the speech training sample.

[0093] The second transfer condition can be that the difference between the cross-modal image features and the real face image features corresponding to the speech training samples is less than a preset second difference threshold. For the real face image features corresponding to the speech training samples, the computing device 600 can acquire the real face image corresponding to the speech training samples, input the real face image into the image feature encoder, and then output the real face image features. That is, the real face image features corresponding to the speech training samples are obtained based on feature extraction from the real face image corresponding to the speech training samples by the image feature encoder. The loss function of the speech feature encoder during training can be a modality transfer loss, which can constrain the cross-modal image features to be as consistent as possible with the real face image features.

[0094] In a specific comparison, the computing device 600 can determine the feature similarity between the image feature and each of the second identity features, and determine the maximum similarity as the comparison result. For example, for each second identity feature: the computing device 600 can input the image feature and the current second identity feature into the cross-modal verification module and output the feature similarity. The feature similarity can include the probability that the image feature and the current second identity feature belong to the same identity. In this way, the probability that the image feature and each of the second identity features belong to the same identity is obtained, where the highest probability can be considered as the maximum similarity, and this maximum probability can also be the probability that target user 100 belongs to the multiple basic users. For example, if the maximum probability is 90%, then target user 100 is considered to have a 90% probability of belonging to the multiple basic users. The loss function of the cross-modal verification module during training can be the cross-modal verification loss, which can constrain the predicted value of the probability that the image feature of the training sample and the second identity feature of the training sample belong to the same identity to gradually approach the true value of the probability. Of course, the computing device 600 can also calculate feature similarity in other ways, such as Euclidean distance, Pearson correlation coefficient, Jaccard coefficient, etc. This manual does not limit the calculation method of feature similarity.

[0095] S180: Based on the comparison results, determine the liveness detection result corresponding to the target user 100.

[0096] The comparison result can be either a first maximum probability that the image feature and the first identity feature belong to the same identity, or a second maximum probability that the image feature and the second identity feature belong to the same identity. Both the first maximum probability and the second maximum probability can reflect the maximum probability that the target user 100 belongs to the plurality of basic users. The computing device 600 can determine a liveness detection threshold based on the comparison result of the target user 100; that is, the liveness detection threshold is personalized for the target user 100. For example, the computing device 600 can input the comparison result into a dynamic threshold module and output the liveness detection threshold.

[0097] In some embodiments, different comparison results may correspond to different liveness detection thresholds. The probability that the target user 100 belongs to the plurality of basic users may be directly proportional to the liveness detection threshold, which can be a threshold compared to the attack probability. That is, the higher the probability that the target user 100 belongs to the plurality of basic users, the lower the probability that the target user 100 is an attacker. Therefore, the computing device 600 can set a higher (more lenient) liveness detection threshold for the target user 100 so that the target user 100 passes the liveness detection. For example, if the probability that the target user 100 belongs to the plurality of basic users is 98%, the liveness detection threshold is 90%. Conversely, the lower the probability that the target user 100 belongs to the plurality of basic users, the higher the probability that the target user 100 is an attacker. Therefore, the computing device 600 can set a lower liveness detection threshold for the target user 100 to strictly control the passage of the target user 100. For example, if the probability that the target user 100 belongs to the plurality of basic users is 10%, the liveness detection threshold is 20%. The proportional relationship between the probability that the target user 100 belongs to the plurality of basic users and the liveness detection threshold can be linear or non-linear. In some embodiments, the dynamic adjustment of the liveness detection threshold can be performed within a range. That is, the liveness detection threshold can be dynamically adjusted within a preset range based on the probability that the target user 100 belongs to the plurality of basic users. The preset range can be obtained in various ways, such as machine learning methods, experimental methods, statistical methods, empirical methods, etc. In some embodiments, the preset range can be related to the fault tolerance rate of the liveness detection. The higher the fault tolerance rate, the larger the preset range.

[0098] In some embodiments, the computing device 600 can determine two types of attribution results based on the comparison results: a first type of attribution result is that the target user 100 belongs to the plurality of basic users, and a second type of attribution result is that the target user 100 does not belong to the plurality of basic users. Then, the liveness detection threshold is determined based on these two types of attribution results. For example, the computing device 600 can compare the probability that the target user 100 belongs to the plurality of basic users with a preset comparison threshold. When the probability that the target user 100 belongs to the plurality of basic users is greater than the comparison threshold, the target user 100 is determined to correspond to the first type of attribution result. When the probability that the target user 100 belongs to the plurality of basic users is less than the comparison threshold, the target user 100 is determined to correspond to the second type of attribution result. When the probability that the target user 100 belongs to the plurality of basic users is equal to the comparison threshold, the target user 100 is determined to correspond to either the first type of attribution result or the second type of attribution result. Furthermore, if the target user 100 corresponds to the first type of attribution result, the liveness detection threshold is a first liveness detection threshold. If the target user 100 corresponds to the second type of attribution result, the liveness detection threshold is a second liveness detection threshold. The first liveness detection threshold is more lenient than the second liveness detection threshold. That is, the target user 100 is more likely to be identified as a normal user or a live user than the second liveness detection threshold.

[0099] The computing device 600 can determine the probability that the target user 100 is an attacker based on the target face image. In some embodiments, the computing device 600 can input the target face image into a feature encoding module, output the liveness feature corresponding to the target face image, and then input the liveness feature into a liveness detection module to output the attack probability. Furthermore, the computing device 600 can determine the liveness detection result based on the attack probability and the liveness detection threshold. For example, the attack probability and the liveness detection threshold can be input into the liveness detection module, and the liveness detection result can be output. If the attack probability is determined to be greater than the liveness detection threshold, the liveness detection result is determined to be an attack. If the attack probability is determined to be less than the liveness detection threshold, the liveness detection result is determined to be a live user. If the attack probability is determined to be equal to the liveness detection threshold, the liveness detection result is determined to be either an attack or a live user, at which point the computing device 600 can instruct the target user 100 to re-perform liveness detection. The loss function of the liveness detection module during training includes a liveness detection loss. This liveness detection loss can constrain the predicted value of the liveness detection result during the training phase to gradually approach its true value. The true value of the liveness detection result during the training phase can include liveness and attacks.

[0100] It should be noted that the liveness feature can be the same as or different from the aforementioned image features. When the computing device 600 compares the first identity feature and the image features in the dimension of image modality, the image features do not need to undergo modality transfer. In this case, the liveness feature can be the same as the aforementioned image features. The computing device 600 can directly use the image features when determining the attack probability without extracting liveness features from the target face image again, thereby reducing the cost of liveness detection. When the computing device 600 compares the first identity feature and the image features in the dimension of speech modality, since the image features have undergone modality transfer to include speech modality information for comparison with the first identity feature, the image features may not be suitable for determining the attack probability. Therefore, the computing device 600 can re-extract features from the target face image when determining the attack probability to extract the liveness feature, thereby improving the accuracy of liveness detection. In this case, the liveness feature can be different from the aforementioned image features.

[0101] In some embodiments, the liveness feature may differ from the aforementioned image features. The liveness feature can be a feature used for liveness detection, such as texture features, color features, spectral features, motion features, image quality features, etc. The aforementioned image features can be features used for identity recognition, such as facial feature distribution features, facial feature size features, contour features, etc.

[0102] It should also be noted that this specification includes a scenario where target user 100 belongs to the multiple basic users, but the final liveness detection result is an attack. For example, target user 100 is a malicious user who uses a photo (attack) of an employee within the target geographical area for liveness detection. In this case, the comparison result shows that target user 100 belongs to the multiple basic users, but the liveness detection result is an attack. Because the comparison result shows that target user 100 belongs to the multiple basic users, it means that the real person in the photo (the attacked user) appeared within the target geographical area within the preset time period. The attacked user is also very likely to be near the liveness detection device at that time. Therefore, the malicious user is easily detected and held accountable. In some embodiments, when the target user 100 belongs to the plurality of basic users, but the final liveness detection result is an attack, the computing device 600 can send an alarm message to the terminal device of the attacked user, reminding the attacked user that a malicious user is impersonating the attacked user and conducting liveness detection in the target geographical area, so that the attacked user can discover and deal with it in time. Therefore, the method of this specification can provide better security for users / employees in the target geographical area and have a certain deterrent effect on attackers / malicious users.

[0103] In summary, the liveness detection method and system provided in this specification acquire a target facial image of a target user 100 and a voice database within a target geographical area over a preset time period. By performing a cross-modal comparison between the target facial image and the voice database, the probability that the target user 100 belongs to a base user in the voice database is determined, thereby assessing the attack risk during liveness detection. Thus, by performing low-cost cross-modal voice verification of the target user 100's facial image, the accuracy of liveness detection is improved while reducing its cost.

[0104] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for performing liveness detection. When the executable instructions are executed by a processor, they instruct the processor to implement the steps of the liveness detection method P100 described herein. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a liveness detection system 001, the program code causes the liveness detection system 001 to perform the steps of the liveness detection method P100 described herein. The program product for implementing the above method may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on the liveness detection system 001. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system. The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium can also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the liveness detection system 001, partially on the liveness detection system 001, as a standalone software package, partially on the liveness detection system 001 and partially on a remote computing device, or entirely on a remote computing device.

[0105] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0106] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0107] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0108] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0109] Every patent, patent application, publication of a patent application, and other material such as articles, books, specifications, publications, documents, articles, etc., cited herein, except for any related historical prosecution documents, any identical ones that may be inconsistent with or conflict with this document, or any identical historical prosecution documents that may have a limiting effect on the widest scope of the claims, may be incorporated herein by reference and used for all purposes now or hereafter in connection with this document. Furthermore, in the event of any inconsistency or conflict between the description, definition, and / or use of terms related to any included material and those related to this document, the terminology used herein shall prevail.

[0110] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A liveness detection method, comprising: Obtain the target user's facial image through the target terminal; Obtain a voice database, which includes voices corresponding to multiple basic users located within a target geographical area during a preset time period, wherein the target terminal is located within the target geographical area; The image features of the target face image are compared with the identity features in the voice database in a cross-modal manner, and the comparison result is determined. The comparison result includes the probability that the target user belongs to the multiple basic users. The cross-modal comparison includes: The image features are compared with multiple first identity features included in the identity features in the dimension of speech modality. The image features are obtained through an image feature encoder. The training objectives of the image feature encoder during the training process include: the pre-trained modality transfer module can generate cross-modal speech features that meet preset first transfer conditions based on the image features corresponding to the face image training samples. or, The image features are compared with multiple second identity features included in the identity features along the image modality dimension. The second identity features are obtained through a speech feature encoder. The training objectives of the speech feature encoder during training include: the pre-trained modality transfer module being able to generate cross-modal image features that satisfy preset second transfer conditions based on the second identity features corresponding to the speech training samples; and Based on the comparison results, the liveness detection result corresponding to the target user is determined.

2. The method as described in claim 1, wherein, The voice data includes data with the voice content removed but retaining the identity information.

3. The method as described in claim 2, wherein, The speech database was obtained through the following method: Obtain the original voice recordings of the aforementioned multiple basic users; The original voices of the multiple basic users are de-identified to obtain de-identified voices; as well as The voice database is obtained based on the anonymized speech.

4. The method of claim 3, wherein, The process of de-sensitizing the original voice data of the multiple basic users to obtain the de-sensitized voice data includes: The original speech is input into a pre-trained content desensitization module, which outputs the desensitized speech. The training objectives of the content desensitization module during the training process include: the pre-trained content recognition module cannot identify the content in the original speech of the training sample from the desensitized speech corresponding to the training sample, and the pre-trained voiceprint recognition module can identify the identity corresponding to the original speech of the training sample from the desensitized speech corresponding to the training sample.

5. The method of claim 3, wherein, The process of obtaining the voice database based on the de-identified voice includes: The de-identified speech is subjected to identity feature extraction to obtain a first identity feature; and The multiple primary identity features corresponding to the multiple basic users are stored in the voice database.

6. The method of claim 5, wherein, The step of performing a cross-modal comparison between the image features of the target face image and the identity features in the voice database, and determining the comparison result, includes: Determine the feature similarity between the image features and each of the plurality of first identity features; and The maximum similarity with the highest feature similarity is determined as the comparison result.

7. The method of claim 6, wherein, The image features include speech modality information.

8. The method of claim 7, wherein, The first migration condition includes that the difference between the cross-modal speech features and the real speech features corresponding to the face image training samples is less than a preset first difference threshold. The first identity feature is obtained based on an identity feature extraction model. The real speech features corresponding to the face image training samples are obtained by extracting identity features from the real speech corresponding to the face image training samples using the identity feature extraction model.

9. The method of claim 7, wherein, Determining the feature similarity between the image feature and each of the plurality of first identity features includes, for each first identity feature: The image features and the current first identity features are input into the cross-modal verification module, and the feature similarity is output. The feature similarity includes the probability that the image features and the current first identity features belong to the same identity.

10. The method of claim 5, wherein, The step of performing a cross-modal comparison between the image features of the target face image and the identity features in the voice database, and determining the comparison result, includes: Determine the plurality of second identity features corresponding to the plurality of first identity features; Determine the feature similarity between the image features and each of the plurality of second identity features; and The maximum similarity with the highest feature similarity is determined as the comparison result.

11. The method of claim 10, wherein, The second identity feature includes image modality information.

12. The method of claim 11, wherein, The second migration condition includes that the difference between the cross-modal image features and the real face image features corresponding to the speech training samples is less than a preset second difference threshold. The image features are obtained by extracting features from the target face image based on an image feature encoder, and the real face image features corresponding to the speech training samples are obtained by extracting features from the real face image corresponding to the speech training samples based on the image feature encoder.

13. The method of claim 10, wherein, Determining the feature similarity between the image feature and each of the plurality of second identity features includes, for each second identity feature: The image features and the current second identity features are input into the cross-modal verification module, and the feature similarity is output. The feature similarity includes the probability that the image features and the current second identity features belong to the same identity.

14. The method of claim 1, wherein, The step of determining the liveness detection result corresponding to the target user based on the comparison result includes: Based on the comparison results, a liveness detection threshold is determined; Based on the target facial image, determine the probability that the target user belongs to the attack; and The liveness detection result is determined based on the attack probability and the liveness detection threshold.

15. The method of claim 14, wherein, The determination of the liveness detection result based on the attack probability and the liveness detection threshold includes: If the attack probability is greater than the liveness detection threshold, the liveness detection result is determined to be an attack; or If the attack probability is determined to be less than the liveness detection threshold, the liveness detection result is determined to be live.

16. A liveness detection system, comprising: At least one storage medium stores at least one set of instructions for implementing liveness detection; as well as At least one processor is communicatively connected to the at least one storage medium. When the liveness detection system is running, the at least one processor reads the at least one instruction set and implements the liveness detection method according to any one of claims 1-15.

Images