A method, apparatus, and electronic device for detecting attacks targeting identity authentication.

Abstract

This application provides a method, apparatus, and electronic device for detecting attacks on identity authentication, relating to the field of identity authentication technology. The method for detecting attacks on identity authentication includes: performing specified segmentation processing on target audio and target video to obtain multiple data segments, wherein the multiple data segments include various audio segments and various video segments, and the target audio and target video are content recorded during user authentication based on reading specified verification text; extracting speech features from each audio segment to obtain feature vectors for each audio segment, and extracting facial motion features from each video segment to obtain feature vectors for each video segment; determining the deviation result corresponding to each target data segment of a specified media type based on the obtained feature vectors; and determining the detection result based on the deviation result corresponding to each target data segment. Therefore, this solution can improve the accuracy of attack detection.

Description

Technical Field

[0001] This application relates to the field of identity authentication technology, and in particular to a method, apparatus and electronic device for detecting attacks on identity authentication. Background Technology

[0002] In recent years, with the rise of smart devices, the demand for identity authentication has been increasing. Identity authentication can be applied to fields such as access control, intercom, and e-commerce. During identity authentication, attack detection against spoofing attacks is usually required.

[0003] In related technologies, for identity authentication based on reading a specified verification text, when performing attack detection, the audio and video recorded by the user during identity authentication are identified and analyzed to obtain the user's facial movement start time and the audio speech start time. For example, the facial movement start time can be the lip movement start time. By using whether the facial movement start time and the speech start time match, it is analyzed whether there is a spoofing attack.

[0004] However, the relevant technologies rely solely on whether the start time of facial movements and the start time of speech match for attack detection, which is a relatively simplistic criterion and results in low accuracy in attack detection. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, and electronic device for detecting attacks on identity authentication, so as to improve the accuracy of attack detection. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a method for detecting attacks on identity authentication, including:

[0007] The target audio and target video are processed into multiple data segments by specifying segmentation; wherein, the multiple data segments include each audio segment and each video segment; the target audio and target video are the content entered when user identity authentication is performed based on reading a specified verification text.

[0008] Speech features are extracted from each audio segment to obtain the feature vector of each audio segment, and facial motion features are extracted from each video segment to obtain the feature vector of each video segment.

[0009] Based on the obtained feature vectors, the deviation result corresponding to each target data segment of the specified media type is determined; wherein, the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment regarding the input time; the target reference segment corresponding to each target data segment is a data segment among the various reference segments for the specified media type whose represented reading content matches the reading content represented by the target data segment; the various reference segments for the specified media type include data segments belonging to media types other than the specified media type among multiple data segments;

[0010] Based on the deviation results corresponding to each target data segment, the detection result is determined; wherein, the detection result characterizes whether there is a spoofing attack in the user identity authentication process.

[0011] Optionally, determining the deviation result corresponding to each target data segment of a specified media type based on the obtained feature vectors includes:

[0012] For each target data segment of a specified media type, based on the feature vector of the target data segment and the feature vectors of each reference segment, select the target reference segment corresponding to the target data segment from each reference segment;

[0013] For each target data segment, the deviation result corresponding to that target data segment is determined based on the target reference segment corresponding to that target data segment.

[0014] Optionally, the step of selecting a target reference segment corresponding to the target data segment from among the reference segments for each target data segment of a specified media type, based on the feature vector of the target data segment and the feature vectors of each reference segment, includes:

[0015] For each target data segment of a specified media type, based on the feature vector of the target data segment, a feature template that meets the specified selection conditions is selected from multiple pre-built feature templates, and based on the selected feature template and the feature vectors of each reference segment, the target reference segment corresponding to the target data segment is determined.

[0016] Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial motion features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation process on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation process on the sample video; both the sample audio and the sample video are content recorded when performing sample user identity authentication based on reading sample verification text, and the sample audio segment and the sample video have the same time information;

[0017] The specified selection criteria include: the specified feature vector is closest to the feature vector of the target data segment; the specified feature vector is the feature vector of the sample segment of the specified media type from the first type of feature vector and the second type of feature vector.

[0018] Optionally, determining the target reference segment corresponding to the target data segment based on the selected feature template and the feature vectors of each reference segment includes:

[0019] Calculate the distance between the feature vector to be used and the feature vector of each reference segment; wherein, the feature vector to be used is the feature vector in the selected feature template other than the specified feature vector;

[0020] Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment.

[0021] Optionally, the audio segments are stored in sequence and the video segments are stored in sequence.

[0022] For each target data segment, based on the corresponding target reference segment, determine the deviation result for that target data segment, including:

[0023] For each target data segment, determine the first position of the target data segment in its sequence and the second position of the target reference segment corresponding to the target data segment in its sequence, and determine the position difference between the first position and the second position to obtain the deviation result corresponding to the target data segment.

[0024] Optionally, determining the detection result based on the deviation result corresponding to each target data segment includes:

[0025] Based on the deviation results corresponding to each target data segment, the deviation results corresponding to each data segment group are determined; wherein, each data segment group includes audio segments and video segments with the same corresponding time information;

[0026] Based on the deviation results corresponding to each data segment group, determine the proportion of the data segment group that is within the preset deviation range in each data segment group, and use it as the target proportion.

[0027] The detection result is determined based on the relationship between the target proportion and the preset threshold.

[0028] Optionally, determining the detection result based on the relationship between the target proportion and a preset threshold includes:

[0029] If the target proportion is greater than a preset threshold, the first result is determined as the detection result;

[0030] If the target proportion is not greater than the preset threshold, the second result is determined as the detection result;

[0031] The first result is a result indicating that the user identity authentication process is free from spoofing attacks;

[0032] The second result indicates that a spoofing attack exists in the user authentication process.

[0033] Optionally, the number of specified media types is two;

[0034] The step of determining the deviation results for each data segment group based on the deviation results corresponding to each target data segment includes:

[0035] For each data segment group, the deviation results corresponding to the target data segments belonging to that data segment group are weighted and averaged to obtain the deviation result corresponding to the data segment group.

[0036] Optionally, the process of performing specified segmentation on the target audio and target video to obtain multiple data segments includes:

[0037] The target audio and target video are segmented according to a predetermined step size and a predetermined segment length to obtain multiple data segments; wherein the predetermined step size is less than the predetermined segment length.

[0038] Secondly, embodiments of this application provide an attack detection device for identity authentication, comprising:

[0039] The specified segmentation processing module is used to perform specified segmentation processing on the target audio and target video respectively to obtain multiple data segments; wherein, the multiple data segments include each audio segment and each video segment; the target audio and target video are the content entered when user identity authentication is performed based on reading specified verification text.

[0040] The speech feature extraction module is used to extract speech features from each audio segment to obtain the feature vector of each audio segment, and to extract facial motion features from each video segment to obtain the feature vector of each video segment.

[0041] The first determining module is used to determine the deviation result corresponding to each target data segment of a specified media type based on the obtained feature vectors; wherein, the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment regarding the input time; the target reference segment corresponding to each target data segment is a data segment among the various reference segments for the specified media type whose represented reading content matches the reading content represented by the target data segment; the various reference segments for the specified media type include data segments belonging to media types other than the specified media type among multiple data segments;

[0042] The second determining module is used to determine the detection result based on the deviation result corresponding to each target data segment; wherein the detection result characterizes whether there is a spoofing attack in the user identity authentication process.

[0043] Optionally, the first determining module includes:

[0044] The selection submodule is used to select the target reference segment corresponding to the target data segment from each reference segment for each target data segment of a specified media type, based on the feature vector of the target data segment and the feature vector of each reference segment.

[0045] The first determining submodule is used to determine the deviation result corresponding to each target data segment based on the target reference segment corresponding to that target data segment.

[0046] Optionally, the selection submodule includes:

[0047] The first determining unit is used to select a feature template that meets the specified selection conditions from a plurality of pre-built feature templates for each target data segment of a specified media type, based on the feature vector of the target data segment, and determine the target reference segment corresponding to the target data segment based on the selected feature template and the feature vector of each reference segment.

[0048] Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial motion features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation process on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation process on the sample video; both the sample audio and the sample video are content recorded when performing sample user identity authentication based on reading sample verification text, and the sample audio segment and the sample video have the same time information;

[0049] The specified selection criteria include: the specified feature vector is closest to the feature vector of the target data segment; the specified feature vector is the feature vector of the sample segment of the specified media type from the first type of feature vector and the second type of feature vector.

[0050] Optionally, the first determining unit determines the target reference segment corresponding to the target data segment based on the selected feature template and the feature vectors of each reference segment, including:

[0051] Calculate the distance between the feature vector to be used and the feature vector of each reference segment; wherein, the feature vector to be used is the feature vector in the selected feature template other than the specified feature vector;

[0052] Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment.

[0053] Optionally, the audio segments are stored in sequence and the video segments are stored in sequence.

[0054] The first determining submodule includes:

[0055] The second determining unit is used to determine, for each target data segment, the first position of the target data segment in its sequence and the second position of the target reference segment corresponding to the target data segment in its sequence, and to determine the position difference between the first position and the second position, thereby obtaining the deviation result corresponding to the target data segment.

[0056] Optionally, the second determining module includes:

[0057] The second determining submodule is used to determine the deviation result corresponding to each data segment group based on the deviation result corresponding to each target data segment; wherein, each data segment group includes audio segments and video segments with the same corresponding time information;

[0058] The third determination submodule is used to determine the proportion of data segment groups within the preset deviation range in each data segment group based on the deviation results corresponding to each data segment group, and use this proportion as the target proportion.

[0059] The fourth determination submodule is used to determine the detection result based on the relationship between the target proportion and the preset threshold.

[0060] Optionally, the fourth determining submodule includes:

[0061] The third determining unit is used to determine the first result as the detection result if the target proportion is greater than a preset threshold.

[0062] The fourth determining unit is used to determine the second result as the detection result if the proportion of the target is not greater than the preset threshold.

[0063] The first result is a result indicating that the user identity authentication process is free from spoofing attacks;

[0064] The second result indicates that a spoofing attack exists in the user authentication process.

[0065] Optionally, the number of specified media types is two;

[0066] The second determining submodule includes:

[0067] The weighted averaging unit is used to perform a weighted average of the deviation results corresponding to the target data segments belonging to each data segment group, so as to obtain the deviation result corresponding to the data segment group.

[0068] Optionally, the designated segmentation processing module includes:

[0069] The segmentation processing submodule is used to segment the target audio and target video according to a predetermined step size and a predetermined segment length to obtain multiple data segments; wherein the value of the predetermined step size is less than the value of the predetermined segment length.

[0070] Thirdly, embodiments of this application provide an electronic device, including:

[0071] Memory, used to store computer programs;

[0072] The processor, when executing a program stored in memory, implements any of the aforementioned attack detection methods for authentication.

[0073] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the aforementioned attack detection methods for identity authentication.

[0074] Beneficial effects of the embodiments in this application:

[0075] The attack detection method for identity authentication provided in this application embodiment can perform specified segmentation processing on target audio and target video to obtain multiple data segments. Feature extraction is performed on each audio and video segment to obtain feature vectors for each audio segment and each video segment. Based on each feature vector, the deviation result corresponding to each target data segment of a specified media type is determined. Based on the deviation result corresponding to each target data segment, the detection result is determined. Therefore, this application embodiment can perform specified segmentation processing on target audio and target video, thus using the deviation result corresponding to each target data segment as the judgment basis when performing attack detection for identity authentication, thereby obtaining a detection result characterizing whether a spoofing attack exists in the user identity authentication process; rather than simply judging the consistency between the effective start and end times of facial movements and the effective start and end times of speech to obtain the detection result. Therefore, compared with the prior art, this application embodiment can provide richer judgment criteria for each target data segment, resulting in higher accuracy of the detection result, thereby improving the accuracy of attack detection.

[0076] Furthermore, in this embodiment, the input content can be segmented for different speaking speeds to facilitate attack detection against identity authentication. Since this embodiment uses the offset of each target data segment as the basis for judgment, the judgment criteria are relatively rich, thereby improving the accuracy of attack detection and the ability to prevent spoofing in identity authentication. Additionally, this embodiment employs a multi-point data statistical method, which can reduce the probability of false positives from single-point / few-point data, and decrease the probability of breaching the identity authentication device.

[0077] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0079] Figure 1 A flowchart illustrating an attack detection method for identity authentication provided in an embodiment of this application;

[0080] Figure 2 A flowchart illustrating another attack detection method for identity authentication provided in an embodiment of this application;

[0081] Figure 3(a) is a schematic diagram of an attack detection device provided in an embodiment of this application;

[0082] Figure 3(b) is a flowchart illustrating the calculation of deviation results provided in an embodiment of this application;

[0083] Figure 3(c) is a schematic diagram of a process for constructing a feature template according to an embodiment of this application;

[0084] Figure 3(d) is a schematic diagram of a process for obtaining detection results provided in an embodiment of this application;

[0085] Figure 4 A schematic diagram of an attack detection device for identity authentication provided in an embodiment of this application;

[0086] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0087] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0088] Below, we will first introduce the technical terms involved in the embodiments of this application:

[0089] Dynamic password: An authentication password generated by an authentication device. Dynamic passwords can also be called verification text.

[0090] Recording replay attack: By replaying recordings of a target through a playback device (such as a speaker), the attacker can then attack the authentication device.

[0091] Voice splicing attack: An attack method based on audio playback attack, which splices and synthesizes the audio recording of a valid target to obtain audio data that matches the dynamic password, and then the attacker imitates the audio data to read it aloud.

[0092] To better understand this solution, before introducing the solution provided in the embodiments of this application, a brief introduction will be given to existing attack detection methods for identity authentication:

[0093] It should be emphasized that, for ease of understanding, the following introduction will use the lip movement initiation time as the facial movement initiation time.

[0094] An identity authentication device is used for identity authentication. This device can input and save a user's identity information. By recognizing and analyzing the audio and video recorded during user identity authentication, the device can obtain the speech start time and audio text content of the recorded audio, and the lip movement start time and lip-reading text content of the recorded video.

[0095] The onset time of audio speech can be determined using techniques such as VAD (Voice Activity Detector), and speech recognition can utilize existing speech recognition technologies. Lip movement localization and feature extraction are prerequisites for lip reading recognition. Specific recognition methods can include shape-based methods—variable templates and the Snake method—image-based methods—principal component analysis, and motion-based methods—optical flow analysis, etc. It is understood that the above-mentioned lip reading recognition methods are all relatively conventional methods in existing technologies, and therefore will not be elaborated upon further here.

[0096] Understandably, based on the audio's speech start time and audio text content, and the video's lip movement start time and lip-reading text content, it can be determined whether the video's lip movement start time matches the audio's speech start time, and whether both the lip-reading text content and the audio text content match the verification text. Specifically, matching the video's lip movement start time with the audio's speech start time means that the difference between the video's lip movement start time and the audio's speech start time is within an acceptable range, for example, within 1 second.

[0097] It should be noted that when an identity authentication device verifies a user's identity, if the user reads the verification text quickly, the identifiable lip movements and the start time of the speech will be compressed into a single point in time. In this case, the authentication device only detects attacks at this single point in time. If an attacker pre-records this fast-paced recording and happens to read it correctly, the probability of breaching the authentication device is high. Using only a single starting point as the basis for judgment is weak, resulting in low accuracy in attack detection. Conversely, if the user reads the verification text slowly or rhythmically, if an attacker pre-records this slow-paced recording, the difficulty of matching the recording is low. Again, judging solely on the matching of lip movements and the start time of the speech is also weak, leading to a high probability of breaching the authentication device and low accuracy in attack detection.

[0098] Based on the problems existing in related technologies, this application provides an attack detection method, device and electronic device for identity authentication to improve the accuracy of attack detection. Specific comparisons are described below.

[0099] The following describes an attack detection method for identity authentication provided in an embodiment of this application.

[0100] The attack detection method for identity authentication provided in this application can be applied to electronic devices. In specific applications, the electronic device can be a terminal device or a server, and can also be called an identity authentication device. For example, the terminal device can be a computer device, etc. This application does not limit the specific form of the electronic device.

[0101] Specifically, the execution entity of this method for detecting attacks on identity authentication can be an attack detection device for identity authentication. For example, when the method is applied to a terminal device, the attack detection device can be a client running on the terminal device, which can be used to perform attack detection for identity authentication. For example, when the method is applied to a server, the attack detection device can be a computer program running on the server, which can be used to perform attack detection for identity authentication.

[0102] It is understood that the embodiments of this application can analyze target audio and target video, therefore the attack detection method for identity authentication provided in the embodiments of this application is a multimodal method.

[0103] One method for detecting attacks on identity authentication may include:

[0104] The target audio and target video are processed into multiple data segments by specifying segmentation; wherein, the multiple data segments include each audio segment and each video segment; the target audio and target video are the content entered when user identity authentication is performed based on reading a specified verification text.

[0105] Speech features are extracted from each audio segment to obtain the feature vector of each audio segment, and facial motion features are extracted from each video segment to obtain the feature vector of each video segment.

[0106] Based on the obtained feature vectors, the deviation result corresponding to each target data segment of the specified media type is determined; wherein, the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment regarding the input time; the target reference segment corresponding to each target data segment is a data segment among the various reference segments for the specified media type whose represented reading content matches the reading content represented by the target data segment; the various reference segments for the specified media type include data segments belonging to media types other than the specified media type among multiple data segments;

[0107] Based on the deviation results corresponding to each target data segment, the detection result is determined; wherein, the detection result characterizes whether there is a spoofing attack in the user identity authentication process.

[0108] As can be seen, the embodiments of this application can perform specified segmentation processing on the target audio and target video, so that when performing attack detection against identity authentication, the deviation result corresponding to each target data segment is used as the judgment basis to obtain the detection result characterizing whether there is a spoofing attack in the user identity authentication process; rather than simply judging the consistency between the effective start and end time of facial movement and the effective start and end time of speech to obtain the detection result. Therefore, compared with the prior art, the embodiments of this application can have richer judgment basis for each target data segment, and the accuracy of the detection result is higher, thereby improving the accuracy of attack detection.

[0109] Furthermore, in this embodiment, the input content can be segmented for different speaking speeds to facilitate attack detection against identity authentication. Since this embodiment uses the offset of each target data segment as the basis for judgment, the judgment criteria are relatively rich, thereby improving the accuracy of attack detection and the ability to prevent spoofing in identity authentication. Additionally, this embodiment employs a multi-point data statistical method, which can reduce the probability of false positives from single-point / few-point data, and decrease the probability of breaching the identity authentication device.

[0110] The following describes, with reference to the accompanying drawings, an embodiment of this application providing a method for detecting attacks on identity authentication.

[0111] Figure 1 This is a flowchart illustrating the attack detection method for identity authentication provided in an embodiment of this application, as shown below. Figure 1 As shown, the method may include steps S101-S104:

[0112] S101, Perform specified segmentation processing on the target audio and target video respectively to obtain multiple data segments;

[0113] The multiple data segments include various audio segments and various video segments; the target audio and target video are the content entered when user identity authentication is performed based on reading a specified verification text.

[0114] In this process, during user authentication, the user can be instructed to read aloud a specified verification text, i.e., a dynamic password. Audio and video are captured during the reading, resulting in audio and video of the same duration, i.e., the target audio and target video. Since the target audio and target video have the same duration and are both segmented using specified segmentation, performing specified segmentation on the target audio and target video separately will also result in each audio segment and each video segment having the same duration. For example, performing specified segmentation on target audio A and target video B will yield 4 audio segments and 4 video segments, each with the same duration. It is understood that this application embodiment does not limit the specific content of the target audio and target video.

[0115] It should be noted that, for ease of analysis and processing, each audio segment can be formed into an audio segment sequence, and each video segment can be formed into a video segment sequence, or other forms. This application does not impose specific limitations on these forms. Furthermore, in the audio segment sequence, the audio segments can be sorted according to their corresponding recording time; similarly, in the video segment sequence, the video segments can be sorted according to their corresponding recording time.

[0116] Optionally, in one implementation, the target audio and target video are processed into specified segments to obtain multiple data segments, which may include step A1:

[0117] Step A1: The target audio and target video are segmented according to a predetermined step size and a predetermined segment length to obtain multiple data segments; wherein the value of the predetermined step size is less than the value of the predetermined segment length.

[0118] Understandably, the predetermined step size and predetermined segment length are pre-set by relevant personnel. By pre-setting the predetermined step size and predetermined segment length, these personnel can set the interval duration and duration of each audio and video segment. It should be noted that the predetermined segment length can be understood as a sliding window, while the predetermined step size is the distance the sliding window slides each time. This sliding window, by sliding over the target audio or video, achieves segmentation processing of the target audio or video. For example, if the duration of target video A is 1 second, the predetermined step size is 0.1 seconds, and the predetermined segment length is 0.5 seconds, then target video A can be divided into 6 segments: the first segment is from 0 to 0.5 seconds, the second segment is from 0.1 to 0.6 seconds, the third segment is from 0.2 to 0.7 seconds, the fourth segment is from 0.3 to 0.8 seconds, the fifth segment is from 0.4 to 0.9 seconds, and the sixth segment is from 0.5 to 1 second.

[0119] Furthermore, because the area of ​​the predetermined step size is smaller than the value of the predetermined segment length when setting the predetermined step size and predetermined segment length, each audio segment and video segment has overlapping content. It should be noted that since users naturally pause when reading the specified verification text, the target audio and video will contain content collected during quiet periods. Therefore, if the audio and video segments do not overlap and are only segmented according to segment length, there may be instances where an audio or video segment's content falls within a quiet period. In subsequent steps, this audio or video segment will have fewer features to collect, potentially making it impossible to determine the corresponding target reference segment, thus affecting the accuracy of attack detection. Based on this consideration, by setting the step size to be smaller than the segment length, it is possible to ensure that adjacent audio segments and video segments have overlapping content, reducing the possibility of data segments lacking valid information. It should be noted that not all audio and video segments necessarily have overlapping parts, but adjacent audio segments or video segments will have overlapping parts.

[0120] As can be seen, since each audio segment and video segment has overlapping parts, it can be ensured that each audio segment and video segment has more features, reducing the probability of lacking useful information due to quiet periods, which would affect the accuracy of attack detection. Therefore, the embodiments of this application can improve the accuracy of attack detection and the anti-spoofing capability for identity authentication.

[0121] S102, extract speech features from each audio segment to obtain feature vectors for each audio segment, and extract facial motion features from each video segment to obtain feature vectors for each video segment.

[0122] It is understood that the feature vectors of each video segment can be facial motion feature vectors, which can be lip feature vectors or other feature vectors, such as eyes, nose, etc. This application does not specifically limit this.

[0123] For example, speech feature extraction for each audio segment can be achieved using MFCC (Mel-scale Frequency Cepstral Coefficients), thereby obtaining the feature vector for each audio segment. These feature vectors can also be called speech feature vectors or audio feature vectors. Of course, other techniques can also be used to extract speech feature vectors, and this application does not specifically limit their application.

[0124] For example, for each video segment, facial motion features can be extracted using techniques such as face detection and keypoint localization, thereby obtaining a feature vector of facial motion for that video segment. For example, the features extracted by facial motion feature extraction may include one or more of the following: lip width, lip height, lip corner angle, lip change information, etc. Furthermore, the feature vector for each video segment can be a normalized feature vector, but is not limited to this; additionally, lip change information may include information belonging to first-order changes and / or information belonging to second-order changes; for example, information belonging to first-order changes may be the velocity of lip changes, and information belonging to second-order changes may be the acceleration of lip changes. Of course, this application does not limit the specific implementation method of facial motion feature extraction.

[0125] It should be noted that the feature vectors of each audio segment and each video segment mentioned above may or may not be in the form of a sequence, and this application does not specifically limit this.

[0126] S103, Based on the obtained feature vectors, determine the deviation result corresponding to each target data segment of the specified media type;

[0127] The deviation result corresponding to each target data segment represents the deviation between the target data segment and the corresponding target reference segment regarding the input time. The target reference segment corresponding to each target data segment is a data segment among the various reference segments for a specified media type whose represented reading content matches the reading content represented by the target data segment. The various reference segments for a specified media type include data segments belonging to media types other than the specified media type among multiple data segments. Each target data segment of a specified media type is a data segment among multiple data segments.

[0128] It's understandable that the number of specified media types can be one or two. When there's only one specified media type, it can be an audio type. In this case, each audio segment can be used as a target data segment, and each audio segment has a corresponding deviation result. The reference segments for each specified media type are video segments, and the target reference segment for any target data segment is a video segment. Alternatively, the specified media type can be a video type. In this case, each video segment can be used as a target data segment, and each video segment has a corresponding deviation result. The reference segments for each specified media type are audio segments, and the target reference segment for each video segment is an audio segment. When there are two specified media types, both audio and video types can be used as a specified media type. Therefore, each audio segment and each video segment can be used as a target data segment, and each audio segment and video segment has its own deviation result. Furthermore, for each target data segment belonging to the audio type, the reference segments are video segments, and the target reference segment is a video segment; for each target data segment belonging to the video type, the reference segments are audio segments, and the target reference segment is an audio segment.

[0129] It is understandable that the deviation result corresponding to each target data segment is essentially the deviation between the target data segment and the corresponding target reference segment regarding the input time. Therefore, the deviation result can take various forms. The deviation result for the target data segment can be the deviation between the input time of the target data segment and the input time of the corresponding target reference segment; it can also be the deviation between the position of the target data segment in its sequence and the position of the corresponding target reference segment in the sequence. It should be noted that the input time here refers to the time information of the data segment; the input time can be understood as a time period. Therefore, the deviation result can be calculated based on the start time of the input time of the target data segment and the corresponding target reference segment. Of course, it can also be calculated based on the stop time of the input time, or an intermediate time point; this application embodiment does not limit this. For example, if the starting point of the input time of target data segment a is 0.5s, and the starting point of the input time of the target reference segment b corresponding to target data segment a is 0.6s, then the deviation result can be 0.1s. Of course, the deviation results can take other forms, but in essence they are all deviations related to the entry time. This application does not limit the deviation results.

[0130] It is important to emphasize that the specified media type of the target data segment differs from that of the corresponding target reference segment. When the specified media type of the target data segment is video, the specified media type of the target reference segment is audio, and vice versa. In other words, when the specified media type of the target data segment is audio, the specified media type of the target reference segment is video. This means that the target reference segment is one of the various reference segments for a specified media type, and the reading content represented by the target reference segment matches the reading content represented by the target data segment. Matching can also be understood as the most similar segment. Furthermore, since the various reference segments for a specified media type include data segments from multiple data segments that belong to media types other than the specified media type, when the specified media type is audio, the various reference segments for that media type include video segments from multiple data segments; similarly, when the specified media type is video, the various reference segments for that media type include audio segments from multiple data segments.

[0131] It is understood that, in order to ensure clarity of the layout, the specific process of determining the deviation result corresponding to each target data segment will be described in other embodiments, and will not be elaborated on in this embodiment.

[0132] S104, Determine the detection result based on the deviation result corresponding to each target data segment;

[0133] The detection results indicate whether a spoofing attack exists in the user authentication process.

[0134] It is understandable that, based on the deviation results corresponding to each target data segment, it can be determined whether a spoofing attack exists in the user authentication process or whether a spoofing attack does not exist in the user authentication process.

[0135] It is understood that, for the sake of clarity, the specific process of the deviation results corresponding to each target data segment will be described in other embodiments, and this application will not elaborate on this aspect.

[0136] As can be seen, the embodiments of this application can perform specified segmentation processing on the target audio and target video, so that when performing attack detection against identity authentication, the deviation result corresponding to each target data segment is used as the judgment basis to obtain the detection result characterizing whether there is a spoofing attack in the user identity authentication process; rather than simply judging the consistency between the effective start and end time of facial movement and the effective start and end time of speech to obtain the detection result. Therefore, compared with the prior art, the embodiments of this application can have richer judgment basis for each target data segment, and the accuracy of the detection result is higher, thereby improving the accuracy of attack detection.

[0137] Furthermore, in this embodiment, the input content can be segmented for different speaking speeds to facilitate attack detection against identity authentication. Since this embodiment uses the offset of each target data segment as the basis for judgment, the judgment criteria are relatively rich, thereby improving the accuracy of attack detection and the ability to prevent spoofing in identity authentication. Additionally, this embodiment employs a multi-point data statistical method, which can reduce the probability of false positives from single-point / few-point data, and decrease the probability of breaching the identity authentication device.

[0138] Based on the above method embodiments, the process of determining the deviation result corresponding to each target data segment of a specified media type is described below:

[0139] Optionally, in one implementation, determining the deviation result corresponding to each target data segment of a specified media type based on the obtained feature vectors may include steps B1-B2:

[0140] Step B1: For each target data segment of a specified media type, based on the feature vector of the target data segment and the feature vectors of each reference segment, select the target reference segment corresponding to the target data segment from each reference segment;

[0141] It is understandable that when the specified media type is audio, the feature vector of the target audio segment is a speech feature vector, and the feature vectors of each reference segment are facial motion feature vectors; similarly, when the specified media type is video, the feature vector of the target video segment is a facial motion feature vector, and the feature vectors of each reference segment are speech feature vectors.

[0142] Understandably, based on the feature vector of the target data segment, the data segment whose reading content is most similar to that represented by the target data segment can be selected from various reference segments as the target reference segment.

[0143] For example, for each target data segment of the target video, based on the facial motion feature vector of the target video segment a and the audio feature vector of each reference segment, the target audio segment b, whose reading content is closest to the reading content represented by the target video segment a, is selected from each reference segment as the target reference segment corresponding to the target video segment a.

[0144] Additionally, it is understood that, in one alternative implementation, selecting the target reference segment corresponding to the target data segment from the various reference segments can utilize deep learning to identify the degree of difference between the semantics represented by the feature vector of the target data segment and the semantics represented by the feature vectors of each reference segment, and then selecting the reference segment with the smallest degree of difference as the target reference segment. Other implementations for selecting the target reference segment corresponding to the target data segment from the various reference segments will be described later in conjunction with other embodiments for clarity.

[0145] Step B2: For each target data segment, determine the deviation result corresponding to the target data segment based on the target reference segment corresponding to the target data segment;

[0146] It should be noted that when determining the deviation result corresponding to the target data segment, the deviation between the target data segment and the corresponding target reference segment regarding the recording time can be used. Therefore, the processing approach can be understood as using the target video reference segment corresponding to the target audio segment to determine the deviation result corresponding to that target audio segment. This deviation result is related to the target video segment; that is, using the target audio segment to determine the deviation result related to the target video segment. Similarly, the deviation result corresponding to the target video segment can be understood as using the target video segment to determine the deviation result related to the target audio segment. For example, for each target video segment, based on the deviation in recording time between the target reference segment b corresponding to target video segment a, the deviation result corresponding to target video segment a can be determined.

[0147] As can be seen, the embodiments of this application can select the target reference segment corresponding to the target data segment based on the feature vector of the target data segment and the feature vector of each reference segment, and determine the deviation result corresponding to the target data segment based on the target reference segment. The embodiments of this application adopt a cross-processing approach, that is, using the target audio segment to determine the deviation result about the target video segment, and using the target video segment to determine the deviation result about the target audio segment. This processing approach can improve the accuracy of the deviation result, thereby improving the accuracy of attack detection.

[0148] Optionally, in one implementation, step B1 may include step B11:

[0149] Step B11: For each target data segment of a specified media type, select a feature template that meets the specified selection conditions from a plurality of pre-built feature templates based on the feature vector of the target data segment, and determine the target reference segment corresponding to the target data segment based on the selected feature template and the feature vector of each reference segment.

[0150] Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial motion features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation process on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation process on the sample video; both the sample audio and the sample video are content recorded when performing sample user identity authentication based on reading sample verification text, and the sample audio segment and the sample video have the same time information;

[0151] The specified selection criteria include: the specified feature vector is closest to the feature vector of the target data segment; the specified feature vector is the feature vector of the sample segment with respect to the specified media type from the first type of feature vector and the second type of feature vector.

[0152] It is important to emphasize that the sample verification text can be the same as or different from the specified verification text.

[0153] It should be noted that the first and second feature vectors can be simply feature vectors of sample segments obtained by processing the sample video and audio recorded by the sample user reading the sample verification text in a specified segment, or they can be averaged facial motion feature vectors and average speech feature vectors obtained by averaging the feature vectors of multiple sample video and audio segments. This application does not specifically limit the content of the first and second feature vectors. Furthermore, each feature template includes a first feature vector and a second feature vector; the two feature vectors exist in pairs, therefore the time information represented by the sample audio segment and the sample video segment are the same, and the reading content represented is the same.

[0154] Understandably, when the specified media type is audio, for each target audio segment, based on the speech feature vector of that target audio segment, a feature template is selected from a pre-constructed set of multiple feature templates. This template contains a first feature vector whose distance to the speech feature vector of the target audio segment is the shortest. Based on the selected feature template and the facial motion feature vectors of each video segment, the target video reference segment corresponding to that target audio segment is determined, thus obtaining the target reference segment. For example, using the speech feature vector c of the target audio segment C, a feature template 1 is selected from a pre-constructed set of multiple feature templates. This template includes a first feature vector c' whose distance to the speech feature vector c is the shortest. Based on feature template 1 and the facial motion feature vectors of each video segment, the target video reference segment D corresponding to the target audio segment C is determined.

[0155] Understandably, when the specified media type is video, for each target video segment, based on the facial motion feature vector of that target video segment, a feature template whose second feature vector is closest to the facial motion feature vector of that target video segment is selected from a pre-constructed set of multiple feature templates. Then, based on the selected feature template and the speech feature vectors of each audio segment, the target audio reference segment corresponding to that target video segment is determined. For example, using the facial motion feature vector e of the target video segment E, a feature template 2 including the second feature vector e' that is closest to the facial motion feature vector e is selected from a pre-constructed set of multiple feature templates. Based on feature template 2 and the speech feature vectors of each audio segment, the target audio reference segment F corresponding to that target video segment E is determined.

[0156] It should be noted that if the specified media type is audio and video, the process of determining the target video reference segment and the target audio reference segment described above should be executed once when the specified media type is audio and once when the specified media type is video.

[0157] As can be seen, for each target data segment of a specified media type, this application embodiment selects a feature template that meets the specified conditions from a plurality of pre-constructed feature templates, thereby determining the target reference segment corresponding to the target data segment. This application embodiment determines the target reference segment corresponding to the target data segment by finding the closest feature template, which facilitates the determination of the deviation result corresponding to the target data segment in subsequent steps based on the target reference segment, thereby improving the accuracy of the deviation result and the accuracy of attack detection.

[0158] Based on the above description of selecting feature templates that meet the specified selection criteria, the following describes the process of determining the target reference segment corresponding to the target data segment based on the selected feature templates and the feature vectors of each reference segment.

[0159] Optionally, in one implementation, determining the target reference segment corresponding to the target data segment based on the selected feature template and the feature vectors of each reference segment may include steps C1-C2:

[0160] Step C1: Calculate the distance between the feature vector to be used and the feature vectors of each reference segment; wherein, the feature vector to be used is the feature vector in the selected feature template other than the specified feature vector;

[0161] Understandably, when the specified media type is audio, the feature vector to be utilized can be the second feature vector in the selected feature template. Therefore, the distance between the second feature vector in the selected feature template and the facial motion feature vectors of each video segment can be calculated. For example, calculating the distance between the second feature vector c” in the selected feature template and the facial motion feature vectors of each video segment reveals that the distance between the second feature vector c” and the facial motion feature vector a of video segment A is 1, and the distance between the second feature vector c” and the facial motion feature vector b of video segment B is 2.

[0162] Understandably, when the specified media type is video, the feature vector to be utilized can be the first feature vector in the selected feature template. Therefore, the distance between the first feature vector in the selected feature template and the speech feature vectors of each audio segment can be calculated. For example, calculating the distance between the first feature vector d” in the selected feature template and the speech feature vectors of each audio segment reveals that the distance between the first feature vector d” and the speech feature vector e of audio segment E is 1, and the distance between the first feature vector d” and the speech feature vector f of audio segment F is 2.

[0163] It should be noted that if the specified media type is audio and video, the process of calculating the distance between the feature vector to be used and the feature vector of each reference segment described above should be performed once when the specified media type is audio and once when the specified media type is video.

[0164] Step C2: Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment;

[0165] It is understandable that, based on the distance between the feature vector to be used and the feature vectors of each reference segment obtained from the above calculation, the feature vector with the shortest distance can be determined, and the reference segment to which the feature vector with the shortest distance belongs can be determined, thereby obtaining the target reference segment corresponding to the target data segment.

[0166] Understandably, when the specified media type is audio, the facial motion feature vector with the shortest distance can be determined from the distances between the second feature vector in the selected feature template and the facial motion feature vectors of each video segment. This facial motion feature vector is then used to determine the video segment to which it belongs, thus obtaining the target reference segment corresponding to the audio segment. For example, if the distance between the second feature vector c” and the facial motion feature vector a of video segment A is 1, and the distance between the second feature vector c” and the facial motion feature vector b of video segment B is 2, it can be determined that facial motion feature vector b is the feature vector with the shortest distance. Therefore, the reference segment to which facial motion feature vector b belongs can be determined as video segment B, thus obtaining the target video reference segment corresponding to the audio segment.

[0167] Understandably, when the specified media type is video, the shortest distance speech feature vector can be determined from the distances between the first feature vector in the selected feature template and the speech feature vectors of each audio segment, and the audio segment to which that speech feature vector belongs can be determined, thereby obtaining the target reference segment corresponding to the video segment. For example, if the distance between the first feature vector d” and the speech feature vector e of audio segment E is 1, and the distance between the first feature vector d” and the speech feature vector f of audio segment F is 2, it can be determined that speech feature vector e is the shortest distance feature vector. Therefore, the reference segment to which speech feature vector e belongs can be determined as audio segment E, thus obtaining the target audio reference segment corresponding to the video segment.

[0168] It should be noted that if the specified media type is audio or video, the process of determining the target reference segment corresponding to the target data segment described above should be executed once when the specified media type is audio and once when the specified media type is video.

[0169] Of course, there is no single way to determine the target reference segment here. Alternatively, the reference segment can be determined by identifying the feature vector with the closest similarity to obtain the target reference segment corresponding to the target data segment. This application does not specifically limit this method.

[0170] As can be seen, the embodiments of this application also adopt a cross-processing approach when determining the target reference segment. The target reference segment corresponding to the target audio segment is the video segment, and the target reference segment corresponding to the target video segment is the audio segment. By using the target reference segment, the deviation result corresponding to the target data segment can be determined, thereby improving the accuracy of the deviation result and thus improving the accuracy of attack detection.

[0171] Alternatively, in one implementation, the audio segments are stored as a sequence, and the video segments are stored as a sequence.

[0172] Accordingly, step B2 may include step B21:

[0173] B21. For each target data segment, determine the first position of the target data segment in its sequence and the second position of the target reference segment corresponding to the target data segment in its sequence, and determine the position difference between the first position and the second position to obtain the deviation result corresponding to the target data segment.

[0174] When the target data segment is an audio segment, the sequence it belongs to can be an audio segment sequence; when the target data segment is a video segment, the sequence it belongs to can be a video segment sequence.

[0175] It should be noted that the first position and the second position can be represented as the sorting position of the target data segment and the corresponding target reference segment in their respective sequences, and the difference between the first position and the second position can be understood as the positional difference between the sorting positions. It is understood that this sequence can be sorted according to the corresponding entry time.

[0176] It is understood that the target data segment is an audio segment. For each audio segment, the first position of the audio segment in its respective audio segment sequence and the second position of the corresponding target reference segment in its respective video segment sequence are determined. The position difference between the first and second positions is then determined, yielding the deviation result for that audio segment. For example, audio segment 'a' has the first sorting position in its respective audio segment sequence, and the corresponding target video reference segment 'b' has the second sorting position in its respective video segment sequence. Therefore, the position difference for audio segment 'a' is 1, and this position difference can be used as the deviation result for audio segment 'a'.

[0177] It is understood that the target data segment is a video segment. For each video segment, the first position of the video segment in its sequence and the second position of the corresponding target reference segment in its audio sequence are determined. The position difference between the first and second positions is then determined to obtain the deviation result for that video segment. For example, video segment c has the first sorting position in its sequence, and the corresponding target audio reference segment d has the third sorting position. Therefore, the position difference for video segment c is 2, and this position difference can be used as the deviation result for video segment c.

[0178] It should be noted that if the target data segment is an audio segment or a video segment, the process described above for determining the deviation result corresponding to the target data segment should be executed once when the target data segment is an audio segment and once when the target data segment is a video segment.

[0179] As can be seen, this embodiment determines the position difference based on the first position of the sequence to which the target data segment belongs and the second position of the sequence to which the corresponding target reference segment belongs, thereby obtaining the deviation result corresponding to the target data segment. This embodiment can use the deviation result of each target data segment as the judgment basis, rather than simply judging the consistency between the effective start and end times of facial movements and the effective start and end times of speech. Therefore, this embodiment can provide richer judgment criteria for each target data segment, thereby improving the accuracy of attack detection.

[0180] Alternatively, in one implementation, the construction of multiple feature templates may include steps D1-D5:

[0181] Step D1: Obtain the sample audio and sample video corresponding to each sample user; wherein, the sample audio and sample video corresponding to each sample user are the contents recorded when authenticating the identity of the sample user based on reading the sample verification text.

[0182] It is understandable that the sample audio for each sample user can be the content recorded when the user reads the sample verification text to authenticate their identity within a certain period of time, i.e., the recorded content obtained under non-attack scenarios. All of the above-mentioned recorded content is saved by the identity authentication device. In addition, in order to facilitate the construction of multiple feature templates, the number of sample users can also be multiple.

[0183] Step D2: For each sample user, perform the specified segmentation process on the sample audio and sample video of that sample user to obtain the sample audio segment sequence and sample video segment sequence of that sample user.

[0184] It is understandable that the process of performing specified segmentation processing on the sample audio and sample video of the sample users is the same as the process of performing specified segmentation processing on the target video and target audio, so it will not be described in detail here.

[0185] In addition, the obtained sample audio segments and sample video segments can be combined to obtain sample audio segment sequences and sample video segment sequences.

[0186] Step D3: For each sample user, extract speech features from the sample audio segments in the sample audio segment sequence of the sample user to obtain the first feature vector sequence corresponding to the sample audio segment sequence of the sample user, and extract facial motion features from the sample video segments in the sample video segment sequence of the sample user to obtain the second feature vector sequence corresponding to the sample video segment sequence of the sample user.

[0187] It is understandable that the process of extracting speech features from the sample audio segments in the sample audio segment sequence of the sample user to obtain the first feature vector sequence corresponding to the sample audio segment sequence is the same as the process of extracting speech features from each audio segment to obtain the feature vector of each audio segment; similarly, the process of extracting facial motion features from the sample video segments in the sample video segment sequence of the sample user to obtain the second feature vector sequence corresponding to the sample video segment sequence is the same as the process of extracting facial motion features from each video segment to obtain the feature vector of each video segment, so it will not be elaborated here.

[0188] Step D4: Scale the length of each template pair in the segment, delete features from template pairs that exceed the target feature length, and pad template pairs that are less than the target feature length with zeros to obtain template pairs with consistent feature lengths.

[0189] It is understandable that each template pair in the segment can be composed of speech feature vectors and facial motion feature vectors at the same sequence position in the first feature vector sequence and the second feature vector sequence, and the speech feature vectors and facial motion feature vectors at the same sequence position have the same feature length.

[0190] It is important to emphasize that the scaling ratio can be 1.1 times, 0.9 times, 0.8 times, etc., and this application does not specifically limit it. After scaling each template pair, the feature length of the scaled template pair can be compared with the target feature length, which is a feature length preset by the user. It is understood that if the feature length of the scaled template pair exceeds the target feature length, feature deletion is performed on the template pair. The deleted part can be a part without feature content, such as silence or pauses, so that the feature length of the template pair matches the target feature length. If the feature length of the scaled template pair is less than the target feature length, zero padding is performed on the template pair. The length of silence can be added to the template pair, so that the feature length of the template pair matches the target feature length.

[0191] Step D5: Based on the template pairs with consistent feature lengths mentioned above, feature templates are obtained through dictionary learning.

[0192] Understandably, a dictionary is constructed for each template with the same feature length. The matching pursuit algorithm is used to obtain dictionary atoms through dictionary learning. Then, the dictionary is updated through the dictionary training algorithm K-SVD. After dictionary learning and dictionary update iterations, the final dictionary atoms are obtained, which serve as feature templates.

[0193] Of course, deep learning, dictionary learning, and feature learning methods can also be used to construct feature templates, and this application does not specifically limit them. Any method capable of constructing feature templates can be applied to this solution.

[0194] As can be seen, by constructing multiple feature templates as described above, a feature template can be obtained. Using this feature template, it is easier to determine the offset result in subsequent steps, thereby improving the accuracy of attack detection.

[0195] Optionally, in one implementation, this application embodiment also provides another attack detection method for identity authentication, such as... Figure 2 As shown, it may include the following steps:

[0196] S201, Perform specified segmentation processing on the target audio and target video respectively to obtain multiple data segments;

[0197] S202, extract speech features from each audio segment to obtain feature vectors for each audio segment, and extract facial motion features from each video segment to obtain feature vectors for each video segment.

[0198] S203, Based on the obtained feature vectors, determine the deviation result corresponding to each target data segment of the specified media type;

[0199] It is understood that steps S201-S203 are the same as steps S101-S103, so they will not be described in detail here.

[0200] S204, Based on the deviation results corresponding to each target data segment, determine the deviation results corresponding to each data segment group;

[0201] Each data segment group contains audio segments and video segments with the same corresponding time information;

[0202] Understandably, based on the deviation results corresponding to each target data segment, the deviation results corresponding to each audio segment and video segment containing the same corresponding time information can be determined. It should be noted that the deviation results can depend on the specified media type. When the specified media type is audio, the deviation results for each data segment group can be the deviation results for the audio segment; when the specified media type is video, the deviation results can be the deviation results for the video segment; and when the specified media type is both audio and video, the deviation results can be the deviation results for both audio and video segments.

[0203] Optionally, in one implementation, the number of specified media types is two;

[0204] Accordingly, determining the deviation results for each data segment group based on the deviation results corresponding to each target data segment may include step E1:

[0205] Step E1: For each data segment group, the deviation results corresponding to the target data segments belonging to that data segment group are weighted and averaged to obtain the deviation results corresponding to the data segment group.

[0206] It is understandable that when the specified media type is audio and video, the deviation results corresponding to the target data segments belonging to that data segment group can include the deviation results corresponding to the audio segment and the deviation results corresponding to the video segment. Therefore, the two deviation results can be weighted and averaged to obtain the deviation result corresponding to the data segment group. It should be noted that the weights in the weighting calculation can be preset by relevant personnel. For example, relevant personnel can preset the deviation results corresponding to the audio segment to account for 60% and the deviation results corresponding to the video segment to account for 40%. Then, the deviation results corresponding to the target data segments can be weighted and averaged according to the above weights to obtain the deviation result corresponding to the data segment group.

[0207] Of course, when the number of specified media types is two, other methods can also be used to obtain the deviation results corresponding to the data segment group. This application embodiment does not specifically limit this.

[0208] As can be seen, when the number of specified media types is two, the deviation results corresponding to the target data segments of each data segment group can be weighted and averaged to obtain the deviation results corresponding to the data segment group. Therefore, the embodiments of this application can be applied to the case where the number of specified media types is two, and the two deviation results are weighted and averaged to make the accuracy of the deviation results corresponding to the final data segment group higher, thereby improving the accuracy of attack detection.

[0209] S205, Based on the deviation results corresponding to each data segment group, determine the proportion of the data segment group within the preset deviation range in each data segment group, and use it as the target proportion;

[0210] It is understandable that each data segment group corresponds to a deviation result. Therefore, by statistically analyzing the deviation results for each data segment group, we can obtain an interval graph showing the deviation results for each data segment group, where each deviation result represents a point on the interval graph. Then, using a preset deviation range, we can calculate the proportion of the deviation results within that range on the interval graph and use this proportion as the target proportion. For example, there are 10 data segment groups with corresponding deviation results, the deviation result interval is -10 to -10, and the preset deviation range is -2 to -2. Since there are 8 deviation results within the preset deviation range, the target proportion can be calculated to be 80%.

[0211] Optionally, in one implementation, the media types are specified as audio and video, and the deviation results corresponding to the audio segment and the video segment are not merged. In this case, the deviation result can be a deviation plane, where the horizontal axis can be the deviation result corresponding to the video segment, and the vertical axis can be the deviation result corresponding to the audio segment. It is understood that by comparing the deviation plane with the theoretical plane, the overlap ratio between the two can be obtained. This overlap ratio can be used as the target ratio. Since the deviation results within the theoretical plane are all within the normal deviation range, the portion of the deviation plane that overlaps with the theoretical plane represents the deviation results within the normal deviation range.

[0212] Furthermore, the meaning of the target percentage is not unique; it can also represent the percentage of extremely large values ​​in the total deviation. A brief introduction to this situation follows:

[0213] Optionally, in one implementation, the target percentage is the proportion of extremely large values ​​to the total deviation. Therefore, the target percentage can be obtained by counting the number of extremely large values ​​in the interval graph of the deviation results for each data segment group. For example, if there are 10 data segment groups with deviation results in the range of -10 to 10, and an extremely large value of 10, with 2 deviation results reaching the extremely large value, then the proportion of extremely large values ​​to the total deviation can be calculated as 20%.

[0214] Of course, there are other ways to calculate the target proportion, such as fitting the deviation distribution region and the training distribution curve to calculate their similarity. This similarity can be understood as the target proportion, where the training distribution curve is a curve obtained through multiple training iterations showing deviations within the normal range. This application does not specifically limit the method for obtaining the target proportion.

[0215] S206, Based on the relationship between the target proportion and the preset threshold, determine the detection result.

[0216] The preset threshold is used to characterize whether a spoofing attack exists during the identity authentication process.

[0217] It is understandable that the relationship between the target percentage and the preset threshold can determine whether a spoofing attack exists during the identity authentication process. However, since the meaning represented by the target percentage is different, the detection results determined based on the relationship between the target percentage and the preset threshold will also be different.

[0218] The following describes how the target percentage represents the proportion of the deviation within the normal deviation range to the total deviation:

[0219] Optionally, in one implementation, the detection result is determined based on the relationship between the target proportion and a preset threshold, including steps F1-F2:

[0220] Step F1: If the target proportion is greater than a preset threshold, the first result is determined as the detection result;

[0221] The first result is a result indicating that the user identity authentication process is free from spoofing attacks;

[0222] It should be noted that the preset threshold is set in advance by relevant personnel, and can be a baseline value obtained based on multiple determinations.

[0223] Understandably, if the target percentage is greater than the preset threshold, it indicates that the proportion of deviations within the normal range to the total deviations is higher than the preset threshold. Since the proportion of deviations within the normal range is relatively high, the first result can be determined as the detection result, meaning that there is no spoofing attack in the user authentication process. For example, if the target percentage is 80%, the proportion of deviations within the normal range to the total deviations is 80%, and the preset threshold is 75%, then any deviation within the normal range exceeding 75% of the total deviations can be considered as indicating that there is no spoofing attack in the user authentication process. Since the target percentage is greater than the preset threshold, the first result can be determined as the detection result, signifying that there is no spoofing attack in the user authentication process.

[0224] Step F2: If the proportion of the target is not greater than the preset threshold, the second result is determined as the detection result.

[0225] The second result is a result indicating that the user authentication process is susceptible to spoofing attacks.

[0226] Understandably, if the target percentage is not greater than the preset threshold, it indicates that the proportion of deviations within the normal range to the total deviation is lower than the preset threshold. Since the proportion of deviations within the normal range is low, the second result can be determined as the detection result, meaning that a spoofing attack exists in the user authentication process. For example, if the target percentage is 50%, the proportion of deviations within the normal range to the total deviation is 50%, and the preset threshold is 75%, since the target percentage is less than the preset threshold, the second result can be determined as the detection result, indicating that a spoofing attack exists in the user authentication process.

[0227] The following describes how the target percentage represents the proportion of extremely large values ​​in the total deviation:

[0228] Alternatively, in one implementation, the target percentage represents the proportion of the extreme skewness to the total deviation;

[0229] Correspondingly, if the target proportion is not greater than the preset threshold, the first result will be determined as the detection result;

[0230] If the target proportion is greater than the preset threshold, the second result will be determined as the detection result;

[0231] Understandably, if the target percentage is greater than the preset threshold, it indicates that the proportion of extremely large values ​​in the total deviation is higher than the preset threshold, and the proportion of extremely large values ​​in the total deviation is relatively high. Since many deviation results are maximal values, the second result can be determined as the detection result. Conversely, if the target percentage is less than the preset threshold, it indicates that the proportion of extremely large values ​​in the total deviation is lower than the preset threshold, and the proportion of extremely large values ​​in the total deviation is relatively low. Therefore, the first result can be determined as the detection result. For example, if the target percentage is 40%, the proportion of extremely large values ​​in the total deviation is 40%, and the preset threshold is 20%, then any extreme large value exceeding 20% ​​in the total deviation can be considered as indicating a spoofing attack in the user authentication process. Since the target percentage is greater than the preset threshold, the second result can be determined as the detection result, indicating a spoofing attack in the user authentication process.

[0232] As can be seen, the embodiments of this application statistically analyze the deviation results of each target data segment to obtain the target proportion, and then determine the detection result by the relationship between the target proportion and the preset threshold. Compared with the prior art, this application does not only judge the consistency between the effective start and end time of facial movement and the effective start and end time of audio. Therefore, the judgment basis of the embodiments of this application is more abundant, which can improve the accuracy of the attack detection.

[0233] In addition, the embodiments of this application adopt a statistical method of multi-point data, which can reduce the probability of misjudgment of single-point / few-point data and reduce the probability of breaking the identity authentication device.

[0234] Furthermore, the attack detection device implementing the attack detection method provided in this application embodiment can be implemented by multiple functional units working together. For example, in one implementation, as shown in FIG3(a), the attack detection device may include:

[0235] The system includes a target video segmentation unit 310, a facial motion feature extraction unit 320, a target audio segmentation unit 330, a speech feature extraction unit 340, a search and matching unit 350, and a statistical scoring unit 360.

[0236] The target video segmentation unit 310 can perform specified segmentation processing on the input target video to obtain various video segments, and there is information overlap between the various video segments.

[0237] Among them, the facial motion feature extraction unit 320 can extract facial motion feature points from each input video segment to obtain facial motion feature vectors, and construct a facial motion feature vector sequence based on the facial motion feature vectors of each segment.

[0238] The target audio segmentation unit 330 can perform specified segmentation processing on the input target audio to obtain various audio segments, and there is information overlap between the various audio segments.

[0239] The speech feature extraction unit 340 can extract speech features from each input audio segment to obtain speech feature vectors, and construct a speech feature vector sequence based on the speech feature vectors of each segment.

[0240] The search matching unit 350 can determine the deviation result corresponding to each target data segment of the specified media type based on the input facial motion feature vector and speech feature vector.

[0241] Among them, the statistical scoring unit 360 can statistically analyze the deviation results corresponding to each target data segment, thereby obtaining the detection results that characterize whether there is a spoofing attack in the user identity authentication process.

[0242] To better understand the part about search matching unit 350, a brief introduction will be given below with reference to the accompanying drawings, as shown in Figures 3(b) and 3(c):

[0243] Understandably, the search matching unit 350 has built-in feature templates. From the feature templates, the feature vectors that are closest to the speech feature vector and the facial motion feature vector can be searched respectively, thereby determining the corresponding target reference segment, and then determining the deviation result corresponding to the target data segment.

[0244] As shown in Figure 3(b), the deviation results corresponding to the target data segment can include:

[0245] S351, Input facial motion feature vector;

[0246] S352, based on the facial motion feature vector, search for the first feature template and determine the corresponding target reference segment;

[0247] S353, Input speech feature vector;

[0248] S354, based on the speech feature vector, search for the second feature template to determine the corresponding target reference segment;

[0249] S355, For each target data segment, determine the deviation result corresponding to the target data segment based on the target reference segment corresponding to the target data segment;

[0250] Understandably, facial motion feature vectors correspond to video segments. Using these facial motion feature vectors, a first feature template can be searched. This first feature template includes the facial motion feature vector closest to the given facial motion feature vector and its corresponding speech feature vector. Based on this corresponding speech feature vector, the reference segment to which the feature vector closest to the corresponding speech feature vector belongs can be determined, and this segment is used as the target reference segment corresponding to the video segment. By determining the second position of the target reference segment corresponding to the video segment in its sequence, and the first position of the video segment in its sequence, and then determining the positional difference between the first and second positions, a first deviation result can be obtained.

[0251] Understandably, a speech feature vector corresponds to an audio segment. Using the speech feature vector, a second feature template can be searched. This second feature template includes the speech feature vector closest to the original speech feature vector and its corresponding facial motion feature vector. Based on the corresponding facial motion feature vector, the reference segment to which the feature vector closest to the corresponding facial motion feature vector belongs can be determined, and this segment is used as the target reference segment corresponding to the audio segment. Determining the second position of the target reference segment corresponding to the audio segment in its sequence, and the first position of the audio segment in its sequence, and then determining the positional difference between the first and second positions, yields the second deviation result.

[0252] It is understandable that, based on the first deviation result and the second deviation result, the first deviation result and the second deviation result can be weighted and averaged to obtain the deviation result corresponding to the target data segment group.

[0253] It should be noted that the method for constructing the feature template is shown in Figure 3(c):

[0254] S301, input a large number of sample audio files;

[0255] S302, Input a large number of sample videos;

[0256] S303, after dictionary learning, obtains the feature template;

[0257] It is understandable that the sample audio and sample video are the contents recorded when the sample user is authenticated by reading the specified verification text.

[0258] It should be noted that the method for constructing feature templates has already been described in the above embodiments, and will not be repeated here.

[0259] To better understand the section on statistical scoring unit 360, a brief introduction will be provided below with reference to the attached figure, as shown in Figure 3(d):

[0260] S361, Calculate the deviation results for each target data segment;

[0261] Understandably, by statistically analyzing the deviation results corresponding to each of the above target data segments, a range chart or curve graph can be generated.

[0262] S362, Calculate the target percentage;

[0263] The target percentage represents the proportion of the deviation within the normal deviation range to the total deviation. It is understood that the calculation process for this target percentage has already been described in the above embodiments, and therefore will not be repeated here.

[0264] S363, determine whether the target percentage meets the predetermined threshold;

[0265] Understandably, based on the relationship between the target percentage and the predetermined threshold, it can be determined whether the target percentage meets the predetermined threshold. If the target percentage is greater than the predetermined threshold, then the target percentage meets the predetermined threshold, and step S365 is executed. If the target percentage is less than the predetermined threshold, then the target percentage does not meet the predetermined threshold, and step S364 is executed.

[0266] S364, a spoofing attack exists during the identity authentication process;

[0267] S365 eliminates the possibility of spoofing attacks during the identity authentication process;

[0268] It is understandable that since the above steps are all performed by the statistical scoring unit 360, the steps are numbered S361-S365.

[0269] Understandably, after the judgment is completed, the judgment result will be used as the detection result of the attack.

[0270] As can be seen, the embodiments of this application can perform specified segmentation processing on the target audio and target video, so that when performing attack detection against identity authentication, the deviation result corresponding to each target data segment is used as the judgment basis to obtain the detection result characterizing whether there is a spoofing attack in the user identity authentication process; rather than simply judging the consistency between the effective start and end time of facial movement and the effective start and end time of speech to obtain the detection result. Therefore, compared with the prior art, the embodiments of this application can have richer judgment basis for each target data segment, and the accuracy of the detection result is higher, thereby improving the accuracy of attack detection.

[0271] In addition, in this embodiment of the application, if the input content is spoken at a fast or slow speed, it can be segmented first to achieve attack detection for identity authentication. Since this embodiment of the application uses the offset result of each target data segment as the judgment basis, the judgment basis in this embodiment of the application is relatively rich, thereby improving the accuracy of attack detection and the anti-spoofing capability for identity authentication.

[0272] Based on the above method embodiments, such as Figure 4 As shown in the figure, this application embodiment provides an attack detection device for identity authentication, including:

[0273] The specified segmentation processing module 410 is used to perform specified segmentation processing on the target audio and target video respectively to obtain multiple data segments; wherein, the multiple data segments include each audio segment and each video segment; the target audio and target video are both contents entered when user identity authentication is performed based on reading specified verification text.

[0274] The feature extraction module 420 is used to extract speech features from each audio segment to obtain the feature vector of each audio segment, and to extract facial motion features from each video segment to obtain the feature vector of each video segment.

[0275] The first determining module 430 is used to determine the deviation result corresponding to each target data segment of a specified media type based on the obtained feature vectors; wherein, the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment regarding the input time; the target reference segment corresponding to each target data segment is a data segment among the various reference segments for the specified media type whose represented reading content matches the reading content represented by the target data segment; the various reference segments for the specified media type include data segments belonging to media types other than the specified media type among multiple data segments;

[0276] The second determining module 440 is used to determine the detection result based on the deviation result corresponding to each target data segment; wherein the detection result indicates whether there is a spoofing attack in the user identity authentication process.

[0277] Optionally, the first determining module includes:

[0278] The selection submodule is used to select the target reference segment corresponding to the target data segment from each reference segment for each target data segment of a specified media type, based on the feature vector of the target data segment and the feature vector of each reference segment.

[0279] The first determining submodule is used to determine the deviation result corresponding to each target data segment based on the target reference segment corresponding to that target data segment.

[0280] Optionally, the selection submodule includes:

[0281] The first determining unit is used to select a feature template that meets the specified selection conditions from a plurality of pre-built feature templates for each target data segment of a specified media type, based on the feature vector of the target data segment, and determine the target reference segment corresponding to the target data segment based on the selected feature template and the feature vector of each reference segment.

[0282] Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial motion features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation process on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation process on the sample video; both the sample audio and the sample video are content recorded when performing sample user identity authentication based on reading sample verification text, and the sample audio segment and the sample video have the same time information;

[0283] The specified selection criteria include: the specified feature vector is closest to the feature vector of the target data segment; the specified feature vector is the feature vector of the sample segment with respect to the specified media type from the first type of feature vector and the second type of feature vector.

[0284] Optionally, the first determining unit determines the target reference segment corresponding to the target data segment based on the selected feature template and the feature vectors of each reference segment, including:

[0285] Calculate the distance between the feature vector to be used and the feature vector of each reference segment; wherein, the feature vector to be used is the feature vector in the selected feature template other than the specified feature vector;

[0286] Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment;

[0287] Optionally, the audio segments are stored in sequence and the video segments are stored in sequence.

[0288] The first determining submodule includes:

[0289] The second determining unit is used to determine, for each target data segment, the first position of the target data segment in its sequence and the second position of the target reference segment corresponding to the target data segment in its sequence, and to determine the position difference between the first position and the second position to obtain the deviation result corresponding to the target data segment.

[0290] Optionally, the second determining module includes:

[0291] The second determining submodule is used to determine the deviation result corresponding to each data segment group based on the deviation result corresponding to each target data segment; wherein, each data segment group includes audio segments and video segments with the same corresponding time information;

[0292] The third determination submodule is used to determine the proportion of data segment groups within the preset deviation range in each data segment group based on the deviation results corresponding to each data segment group, and use this proportion as the target proportion.

[0293] The fourth determining submodule is used to determine the detection result based on the relationship between the target proportion and the preset threshold.

[0294] Optionally, the fourth determining submodule includes:

[0295] The third determining unit is used to determine the first result as the detection result if the target proportion is greater than a preset threshold.

[0296] The fourth determining unit is used to determine the second result as the detection result if the proportion of the target is not greater than the preset threshold.

[0297] The first result is a result indicating that the user identity authentication process is free from spoofing attacks;

[0298] The second result indicates that a spoofing attack exists in the user authentication process;

[0299] Optionally, the number of specified media types is two;

[0300] The second determining submodule includes:

[0301] The weighted averaging unit is used to perform a weighted average of the deviation results corresponding to the target data segments belonging to each data segment group, so as to obtain the deviation result corresponding to the data segment group.

[0302] Optionally, the designated segmentation processing module includes:

[0303] The segmentation processing submodule is used to segment the target audio and target video according to a predetermined step size and a predetermined segment length to obtain multiple data segments; wherein the value of the predetermined step size is less than the value of the predetermined segment length.

[0304] In the technical solution of this application, the operations of obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0305] This application also provides an electronic device, such as... Figure 5 As shown, it includes:

[0306] Memory 501 is used to store computer programs;

[0307] The processor 502, when executing the program stored in the memory 501, implements the above-mentioned attack detection method for identity authentication.

[0308] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 502, communication interface, and memory 501 communicating with each other via the communication bus.

[0309] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0310] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0311] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0312] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0313] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements any of the above-described attack detection methods for identity authentication.

[0314] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the attack detection methods for identity authentication described in the above embodiments.

[0315] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape) or an optical medium (e.g., DVD).

[0316] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0317] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0318] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for detecting attacks on identity authentication, characterized in that, include: The target audio and target video are processed into multiple data segments by specifying segmentation; wherein, the multiple data segments include each audio segment and each video segment; the target audio and target video are the content entered when user identity authentication is performed based on reading a specified verification text. Speech features are extracted from each audio segment to obtain the feature vector of each audio segment, and facial motion features are extracted from each video segment to obtain the feature vector of each video segment. Based on the obtained feature vectors, the deviation result corresponding to each target data segment of the specified media type is determined; wherein, the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment regarding the input time; the target reference segment corresponding to each target data segment is a data segment among the various reference segments for the specified media type whose represented reading content matches the reading content represented by the target data segment; the various reference segments for the specified media type include data segments belonging to media types other than the specified media type among multiple data segments; Based on the deviation results corresponding to each target data segment, the detection result is determined; wherein, the detection result characterizes whether there is a spoofing attack in the user identity authentication process.

2. The method according to claim 1, characterized in that, The step of determining the deviation result corresponding to each target data segment of a specified media type based on the obtained feature vectors includes: For each target data segment of a specified media type, based on the feature vector of the target data segment and the feature vectors of each reference segment, select the target reference segment corresponding to the target data segment from each reference segment; For each target data segment, the deviation result corresponding to that target data segment is determined based on the target reference segment corresponding to that target data segment.

3. The method according to claim 2, characterized in that, For each target data segment of a specified media type, based on the feature vector of the target data segment and the feature vectors of each reference segment, the selection of the target reference segment corresponding to the target data segment from the reference segments includes: For each target data segment of a specified media type, based on the feature vector of the target data segment, a feature template that meets the specified selection conditions is selected from multiple pre-built feature templates, and based on the selected feature template and the feature vectors of each reference segment, the target reference segment corresponding to the target data segment is determined. Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial motion features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation process on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation process on the sample video; both the sample audio and the sample video are content recorded when performing sample user identity authentication based on reading sample verification text, and the sample audio segment and the sample video have the same time information; The specified selection criteria include: the specified feature vector is closest to the feature vector of the target data segment; the specified feature vector is the feature vector of the sample segment of the specified media type from the first type of feature vector and the second type of feature vector.

4. The method according to claim 3, characterized in that, The step of determining the target reference segment corresponding to the target data segment based on the selected feature template and the feature vectors of each reference segment includes: Calculate the distance between the feature vector to be used and the feature vector of each reference segment; wherein, the feature vector to be used is the feature vector in the selected feature template other than the specified feature vector; Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment.

5. The method according to claim 2, characterized in that, The audio segments are stored in sequence, and the video segments are stored in sequence. For each target data segment, based on the corresponding target reference segment, determine the deviation result for that target data segment, including: For each target data segment, determine the first position of the target data segment in its sequence and the second position of the target reference segment corresponding to the target data segment in its sequence, and determine the position difference between the first position and the second position to obtain the deviation result corresponding to the target data segment.

6. The method according to claim 1, characterized in that, The determination of the detection result based on the deviation result corresponding to each target data segment includes: Based on the deviation results corresponding to each target data segment, the deviation results corresponding to each data segment group are determined; wherein, each data segment group includes audio segments and video segments with the same corresponding time information; Based on the deviation results corresponding to each data segment group, determine the proportion of the data segment group that is within the preset deviation range in each data segment group, and use it as the target proportion. The detection result is determined based on the relationship between the target proportion and the preset threshold.

7. The method according to claim 6, characterized in that, The determination of the detection result based on the relationship between the target proportion and the preset threshold includes: If the target proportion is greater than a preset threshold, the first result is determined as the detection result; If the target proportion is not greater than the preset threshold, the second result is determined as the detection result; The first result is a result indicating that the user identity authentication process is free from spoofing attacks; The second result indicates that a spoofing attack exists in the user authentication process.

8. The method according to claim 6, characterized in that, The number of specified media types is two; The step of determining the deviation results for each data segment group based on the deviation results corresponding to each target data segment includes: For each data segment group, the deviation results corresponding to the target data segments belonging to that data segment group are weighted and averaged to obtain the deviation result corresponding to the data segment group.

9. The method according to claim 1, characterized in that, The target audio and target video are processed into specified segments to obtain multiple data segments, including: The target audio and target video are segmented according to a predetermined step size and a predetermined segment length to obtain multiple data segments; wherein the predetermined step size is less than the predetermined segment length.

10. An attack detection device for identity authentication, characterized in that, include: The specified segmentation processing module is used to perform specified segmentation processing on the target audio and target video respectively to obtain multiple data segments; wherein, the multiple data segments include each audio segment and each video segment; the target audio and target video are the content entered when user identity authentication is performed based on reading specified verification text. The speech feature extraction module is used to extract speech features from each audio segment to obtain the feature vector of each audio segment, and to extract facial motion features from each video segment to obtain the feature vector of each video segment. The first determining module is used to determine the deviation result corresponding to each target data segment of a specified media type based on the obtained feature vectors; wherein, the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment regarding the input time; the target reference segment corresponding to each target data segment is a data segment among the various reference segments for the specified media type whose represented reading content matches the reading content represented by the target data segment; the various reference segments for the specified media type include data segments belonging to media types other than the specified media type among multiple data segments; The second determining module is used to determine the detection result based on the deviation result corresponding to each target data segment; wherein the detection result indicates whether there is a spoofing attack in the user identity authentication process.

11. The apparatus according to claim 10, characterized in that, The first determining module includes: The selection submodule is used to select the target reference segment corresponding to the target data segment from each reference segment for each target data segment of a specified media type, based on the feature vector of the target data segment and the feature vector of each reference segment. The first determining submodule is used to determine the deviation result corresponding to each target data segment based on the target reference segment corresponding to that target data segment. or, The selection submodule includes: The first determining unit is used to select a feature template that meets the specified selection conditions from a plurality of pre-built feature templates for each target data segment of a specified media type, based on the feature vector of the target data segment, and determine the target reference segment corresponding to the target data segment based on the selected feature template and the feature vector of each reference segment. Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial motion features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation process on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation process on the sample video; both the sample audio and the sample video are content recorded when performing sample user identity authentication based on reading sample verification text, and the sample audio segment and the sample video have the same time information; The specified selection criteria include: the specified feature vector is closest to the feature vector of the target data segment; the specified feature vector is the feature vector of the sample segment with respect to the specified media type from the first type of feature vector and the second type of feature vector. or, The first determining unit determines the target reference segment corresponding to the target data segment based on the selected feature template and the feature vectors of each reference segment, including: Calculate the distance between the feature vector to be used and the feature vector of each reference segment; wherein, the feature vector to be used is the feature vector in the selected feature template other than the specified feature vector; Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment; or, The audio segments are stored in sequence, and the video segments are stored in sequence. The first determining submodule includes: The second determining unit is used to determine, for each target data segment, the first position of the target data segment in its sequence and the second position of the target reference segment corresponding to the target data segment in its sequence, and to determine the position difference between the first position and the second position to obtain the deviation result corresponding to the target data segment. or, The second determining module includes: The second determining submodule is used to determine the deviation result corresponding to each data segment group based on the deviation result corresponding to each target data segment; wherein, each data segment group includes audio segments and video segments with the same corresponding time information; The third determination submodule is used to determine the proportion of data segment groups within the preset deviation range in each data segment group based on the deviation results corresponding to each data segment group, and use this proportion as the target proportion. The fourth determining submodule is used to determine the detection result based on the relationship between the target proportion and the preset threshold. or, The fourth determining submodule includes: The third determining unit is used to determine the first result as the detection result if the target proportion is greater than a preset threshold. The fourth determining unit is used to determine the second result as the detection result if the proportion of the target is not greater than the preset threshold. The first result is a result indicating that the user identity authentication process is free from spoofing attacks; The second result indicates that a spoofing attack exists in the user authentication process; or, The number of specified media types is two; The second determining submodule includes: The weighted averaging unit is used to perform a weighted average of the deviation results corresponding to the target data segments belonging to each data segment group, so as to obtain the deviation result corresponding to the data segment group. or, The designated segmentation processing module includes: The segmentation processing submodule is used to segment the target audio and target video according to a predetermined step size and a predetermined segment length to obtain multiple data segments; wherein the value of the predetermined step size is less than the value of the predetermined segment length.

12. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

Images