Attack detection method and device for identity authentication and electronic equipment

Abstract

The embodiment of the invention provides an attack detection method and device for identity authentication and electronic equipment, and relates to the technical field of identity authentication. The attack detection method for identity authentication can comprise the steps that specified segmentation processing is carried out on a target audio and a target video to obtain a plurality of data segments, the data segments comprise audio segments and video segments, and when the target audio and the target video are subjected to user identity authentication based on reading of a specified verification text, the audio segments and the video segments are subjected to user identity authentication based on reading of the specified verification text. According to the recorded content, voice feature extraction is carried out on each audio segment to obtain a feature vector of each audio segment, facial motion feature extraction is carried out on each video segment to obtain a feature vector of each video segment, and a deviation result corresponding to each target data segment of the specified media type is determined based on each obtained feature vector; and determining a detection result based on the deviation result corresponding to each target data segment. Therefore, the scheme can improve the accuracy of attack detection.

Description

Technical Field

[0001] The present application relates to the field of identity authentication technology, and in particular to an attack detection method, device and electronic device for identity authentication. Background Art

[0002] In recent years, with the rise of smart devices, the demand for identity authentication has increased. Identity authentication can be applied to fields such as access control, intercom, and e-commerce. During identity authentication, it is usually necessary to perform attack detection against prosthetic attacks.

[0003] In the related technology, with regard to the method of performing identity authentication based on reading aloud a specified verification text, when performing attack detection, the audio and video recorded when the user performs identity authentication are recognized and analyzed to obtain the user's facial movement start time and the audio voice start time. Exemplarily, the facial movement start time may be the lip movement start time; whether the facial movement start time and the voice start time match is used to analyze whether there is a prosthesis attack.

[0004] However, in the related art, attack detection is performed only based on whether the start time of facial movement matches the start time of speech. The judgment basis is relatively simple, resulting in low accuracy of attack detection. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an attack detection method, device and electronic device for identity authentication to improve the accuracy of attack detection. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides an attack detection method for identity authentication, comprising:

[0007] The target audio and the target video are respectively subjected to designated segmentation processing to obtain a plurality of data segments; wherein the plurality of data segments include respective audio segments and respective video segments; the target audio and the target video are both contents entered when performing user identity authentication based on reading a designated verification text;

[0008] Extracting speech features from each audio segment to obtain a feature vector of each audio segment, and extracting facial motion features from each video segment to obtain a feature vector of each video segment;

[0009] Based on each obtained feature vector, determine the deviation result corresponding to each target data segment of the specified media type; wherein the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment regarding the input time, the target reference segment corresponding to each target data segment is a data segment in each reference segment for the specified media type, the represented reading content matches the reading content represented by the target data segment; each reference segment for the specified media type includes a data segment in a plurality of data segments, which belongs to a media type other than the specified media type;

[0010] Based on the deviation result corresponding to each target data segment, a detection result is determined; wherein the detection result indicates whether there is a prosthetic attack in the user identity authentication process.

[0011] Optionally, determining the deviation result corresponding to each target data segment of the specified media type based on each obtained feature vector includes:

[0012] For each target data segment of the specified media type, based on the feature vector of the target data segment and the feature vectors of each reference segment, a target reference segment corresponding to the target data segment is selected from each reference segment;

[0013] For each target data segment, based on the target reference segment corresponding to the target data segment, a deviation result corresponding to the target data segment is determined.

[0014] Optionally, for each target data segment of the specified media type, selecting a target reference segment corresponding to the target data segment from each reference segment based on a feature vector of the target data segment and feature vectors of each reference segment includes:

[0015] For each target data segment of the specified media type, according to the feature vector of the target data segment, a feature template that meets the specified selection condition is selected from a plurality of pre-constructed feature templates, and based on the selected feature template and the feature vectors of each reference segment, a target reference segment corresponding to the target data segment is determined;

[0016] Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial movement features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation processing on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation processing on the sample video. The sample audio and sample video are both contents recorded when performing sample user identity authentication based on reading aloud sample verification text, and the time information corresponding to the sample audio segment and the sample video is the same;

[0017] The specified selection conditions include: the distance between the specified feature vector included and the feature vector of the target data segment is the shortest; the specified feature vector is a feature vector of a sample segment of the specified media type among the first and second feature vectors.

[0018] Optionally, determining the target reference segment corresponding to the target data segment based on the selected feature template and the feature vectors of each reference segment includes:

[0019] Calculating the distance between the feature vector to be used and the feature vector of each reference segment respectively; wherein the feature vector to be used is a feature vector in the selected feature template except the specified feature vector;

[0020] Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment.

[0021] Optionally, each audio segment is stored in a sequence form, and each video segment is stored in a sequence form;

[0022] For each target data segment, based on a target reference segment corresponding to the target data segment, determining a deviation result corresponding to the target data segment includes:

[0023] For each target data segment, determine the first position of the target data segment in the corresponding sequence, and the second position of the target reference segment corresponding to the target data segment in the corresponding sequence, and determine the position difference between the first position and the second position to obtain the deviation result corresponding to the target data segment.

[0024] Optionally, determining the detection result based on the deviation result corresponding to each target data segment includes:

[0025] Based on the deviation results corresponding to each target data segment, determining the deviation results corresponding to each data segment group; wherein each data segment group includes an audio segment and a video segment having the same corresponding time information;

[0026] According to the deviation result corresponding to each data segment group, the proportion of the data segment groups within the preset deviation range in each data segment group is determined as the target proportion;

[0027] The detection result is determined based on the relationship between the target proportion and the preset threshold.

[0028] Optionally, determining the detection result based on the magnitude relationship between the target proportion and a preset threshold value includes:

[0029] If the target proportion is greater than a preset threshold, the first result is determined as the detection result;

[0030] If the target proportion is not greater than the preset threshold, the second result is determined as the detection result;

[0031] The first result is a result indicating that there is no pseudo attack in the user identity authentication process;

[0032] The second result is a result indicating that a pseudo-attack exists in the user identity authentication process.

[0033] Optionally, the number of the specified media types is two;

[0034] The step of determining the deviation results corresponding to each data segment group based on the deviation results corresponding to each target data segment includes:

[0035] For each data segment group, the deviation results corresponding to the target data segments belonging to the data segment group are weighted and averaged to obtain the deviation result corresponding to the data segment group.

[0036] Optionally, the target audio and target video are respectively subjected to designated segmentation processing to obtain multiple data segments, including:

[0037] According to a predetermined step length and a predetermined segment length, the target audio and the target video are segmented to obtain a plurality of data segments; wherein the value of the predetermined step length is smaller than the value of the predetermined segment length.

[0038] In a second aspect, an embodiment of the present application provides an attack detection device for identity authentication, comprising:

[0039] A designated segment processing module is used to perform designated segment processing on the target audio and the target video respectively to obtain multiple data segments; wherein the multiple data segments include each audio segment and each video segment; the target audio and the target video are both the content entered when the user identity authentication is performed based on reading a designated verification text;

[0040] A speech feature extraction module is used to extract speech features from each audio segment to obtain a feature vector of each audio segment, and to extract facial motion features from each video segment to obtain a feature vector of each video segment;

[0041] A first determination module is used to determine the deviation result corresponding to each target data segment of the specified media type based on each obtained feature vector; wherein the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment about the input time, and the target reference segment corresponding to each target data segment is a data segment in each reference segment of the specified media type, the represented reading content of which matches the reading content represented by the target data segment; each reference segment of the specified media type includes a data segment of a media type other than the specified media type among a plurality of data segments;

[0042] The second determination module is used to determine the detection result based on the deviation result corresponding to each target data segment; wherein the detection result represents whether there is a prosthesis attack in the user identity authentication process.

[0043] Optionally, the first determining module includes:

[0044] A selection submodule, for selecting, for each target data segment of a specified media type, a target reference segment corresponding to the target data segment from each reference segment based on a feature vector of the target data segment and feature vectors of each reference segment;

[0045] The first determination submodule is used to determine, for each target data segment, a deviation result corresponding to the target data segment based on a target reference segment corresponding to the target data segment.

[0046] Optionally, the selection submodule includes:

[0047] A first determination unit is used to select, for each target data segment of a specified media type, a feature template that meets a specified selection condition from a plurality of pre-constructed feature templates according to a feature vector of the target data segment, and determine a target reference segment corresponding to the target data segment based on the selected feature template and feature vectors of each reference segment;

[0048] Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial movement features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation processing on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation processing on the sample video. The sample audio and sample video are both contents recorded when performing sample user identity authentication based on reading aloud sample verification text, and the time information corresponding to the sample audio segment and the sample video is the same;

[0049] The specified selection conditions include: the distance between the specified feature vector included and the feature vector of the target data segment is the shortest; the specified feature vector is a feature vector of a sample segment of the specified media type among the first and second feature vectors.

[0050] Optionally, the first determining unit determines a target reference segment corresponding to the target data segment based on the selected feature template and feature vectors of each reference segment, including:

[0051] Calculating the distance between the feature vector to be used and the feature vector of each reference segment respectively; wherein the feature vector to be used is a feature vector in the selected feature template except the specified feature vector;

[0052] Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment.

[0053] Optionally, each audio segment is stored in a sequence form, and each video segment is stored in a sequence form;

[0054] The first determining submodule includes:

[0055] The second determination unit is used to determine, for each target data segment, a first position of the target data segment in the sequence to which it belongs, and a second position of a target reference segment corresponding to the target data segment in the sequence to which it belongs, and determine a position difference between the first position and the second position to obtain a deviation result corresponding to the target data segment.

[0056] Optionally, the second determining module includes:

[0057] A second determination submodule is used to determine the deviation results corresponding to each data segment group based on the deviation results corresponding to each target data segment; wherein each data segment group includes an audio segment and a video segment having the same corresponding time information;

[0058] The third determination submodule is used to determine the proportion of data segment groups within a preset deviation range in each data segment group according to the deviation result corresponding to each data segment group, as the target proportion;

[0059] The fourth determination submodule is used to determine the detection result based on the size relationship between the target proportion and a preset threshold.

[0060] Optionally, the fourth determining submodule includes:

[0061] A third determining unit, configured to determine the first result as a detection result if the target proportion is greater than a preset threshold;

[0062] A fourth determining unit, configured to determine the second result as a detection result if the target proportion is not greater than the preset threshold;

[0063] The first result is a result indicating that there is no pseudo attack in the user identity authentication process;

[0064] The second result is a result indicating that a pseudo-attack exists in the user identity authentication process.

[0065] Optionally, the number of the specified media types is two;

[0066] The second determining submodule includes:

[0067] The weighted averaging unit is used to perform weighted averaging on the deviation results corresponding to the target data segments belonging to each data segment group to obtain the deviation results corresponding to the data segment group.

[0068] Optionally, the designated segment processing module includes:

[0069] The segment processing submodule is used to segment the target audio and target video according to a predetermined step length and a predetermined segment length to obtain multiple data segments; wherein the value of the predetermined step length is smaller than the value of the predetermined segment length.

[0070] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0071] Memory, used to store computer programs;

[0072] The processor is used to implement any of the above-mentioned attack detection methods for identity authentication when executing the program stored in the memory.

[0073] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements any of the above-mentioned attack detection methods for identity authentication.

[0074] Beneficial effects of the embodiments of the present application:

[0075] The attack detection method for identity authentication provided by the embodiment of the present application can perform designated segmentation processing on the target audio and target video respectively to obtain multiple data segments, perform feature extraction on each audio segment and video segment to obtain the feature vector of each audio segment and the feature vector of each video segment, determine the deviation result corresponding to each target data segment of the designated media type based on each feature vector, and determine the detection result based on the deviation result corresponding to each target data segment. It can be seen that the embodiment of the present application can perform designated segmentation processing on the target audio and target video, so that when performing attack detection on identity authentication, the deviation result corresponding to each target data segment is used as the judgment basis, so as to obtain the detection result characterizing whether there is a prosthesis attack in the user identity authentication process; instead of just judging the consistency between the effective start and end time of the facial movement and the effective start and end time of the voice, so as to obtain the detection result, compared with the prior art, the embodiment of the present application can have a richer judgment basis for each target data segment, and the accuracy of the detection result is higher, so as to improve the accuracy of attack detection.

[0076] In addition, the embodiment of the present application can process the input content in segments first, whether the speed is fast or slow, so as to achieve attack detection for identity authentication. Therefore, since the embodiment of the present application uses the offset result of each target data segment as the judgment basis, the judgment basis in the embodiment of the present application is relatively rich, so as to improve the accuracy of attack detection and the anti-counterfeiting ability for identity authentication. In addition, the embodiment of the present application adopts a statistical method of multi-point data, which can reduce the probability of misjudgment of single-point / few-point data and reduce the probability of breaking the identity authentication device.

[0077] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0079] Figure 1 A flowchart of an attack detection method for identity authentication provided in an embodiment of the present application;

[0080] Figure 2 A flowchart of another attack detection method for identity authentication provided in an embodiment of the present application;

[0081] FIG3( a ) is a schematic diagram of the structure of an attack detection device provided in an embodiment of the present application;

[0082] FIG3( b ) is a schematic diagram of a process for calculating a deviation result provided in an embodiment of the present application;

[0083] FIG3( c ) is a schematic diagram of a process of constructing a feature template provided in an embodiment of the present application;

[0084] FIG3( d ) is a schematic diagram of a process for obtaining a test result provided in an embodiment of the present application;

[0085] Figure 4 A schematic diagram of the structure of an attack detection device for identity authentication provided in an embodiment of the present application;

[0086] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0088] Below, the professional terms involved in the embodiments of the present application are first introduced:

[0089] Dynamic password: An identity authentication password generated by an identity authentication device. A dynamic password can also be called a verification text.

[0090] Recording playback attack: Through a playback device (such as a speaker), the recording of a valid target is played back to attack the identity authentication device.

[0091] Voice splicing attack: An attack method based on recording playback attack, in which the recordings of valid targets are spliced ​​and synthesized to obtain recording data that matches the dynamic password, and then the attacker imitates the recording data to read along.

[0092] In order to better understand this solution, before introducing the solution provided by the embodiment of the present application, a brief introduction is given to the attack detection method for identity authentication in the prior art:

[0093] It should be emphasized that for ease of understanding, the following introduction will use the starting time of lip movement as the starting time of facial movement.

[0094] The identity authentication device is a device used for identity authentication, which can record and save the user's identity information. By identifying and analyzing the audio and video recorded by the user during identity authentication, the voice start time and audio text content of the recorded audio, the lip movement start time and lip reading text content of the video can be obtained;

[0095] Among them, the speech start time of the audio can be realized by VAD (Voice Activity Detector) and other technologies, and speech recognition can use the existing speech recognition technology; and lip movement positioning and feature extraction are prerequisites for lip reading recognition. Specific recognition methods can be shape-based methods - variable templates and snake methods, image-based methods - principal component analysis methods, motion-based methods - optical flow analysis methods, etc. It can be understood that the above lip reading recognition methods are relatively conventional methods in the prior art, so they will not be described in detail here.

[0096] It is understandable that based on the above-mentioned audio speech start time and audio text content, video lip movement start time and lip reading text content, it can be determined whether the video lip movement start time matches the audio speech start time, and whether the lip reading text content and audio text content both match the verification text. Among them, the video lip movement start time matches the audio speech start time when the difference between the video lip movement start time and the audio speech start time is within an allowable range, for example: within 1 second.

[0097] It should be noted that when the identity authentication device authenticates the user, if the user reads the verification text at a fast speed, the start time of the identifiable lip movement and voice will be compressed to a time point, then the identity authentication device only performs attack detection on one time point. When the faster recording is pre-recorded by the attacker, and the attacker accidentally reads it in place, the probability of breaking the identity authentication device is high, and only a single starting point is used as the basis for judgment. The judgment basis is relatively weak, and the accuracy of attack detection is low; in addition, if the user reads the verification text at a slow speed or reads it rhythmically, when the slower recording is pre-recorded by the attacker, it is less difficult for the attacker to follow the recording, and only the start time of the lip movement and voice is judged to be consistent. The judgment basis is relatively simple, the probability of breaking the identity authentication device is high, and the accuracy of attack detection is low.

[0098] Based on the problems existing in the related technologies, the embodiments of the present application provide an attack detection method, device and electronic device for identity authentication to improve the accuracy of attack detection. The specific comparative description is introduced below.

[0099] The following is an introduction to an attack detection method for identity authentication provided in an embodiment of the present application.

[0100] The attack detection method for identity authentication provided in the embodiment of the present application can be applied to electronic devices. In specific applications, the electronic device can be a terminal device or a server, and the electronic device can also be called an identity authentication device. For example, the terminal device can be a computer device, etc. The embodiment of the present application does not limit the specific form of the electronic device.

[0101] Specifically, the execution subject of the attack detection method for identity authentication may be an attack detection device for identity authentication. Exemplarily, when the attack detection method for identity authentication is applied to a terminal device, the attack detection device for identity authentication may be a client running in the terminal device, and the client may be used to perform attack detection for identity authentication. Exemplarily, when the attack detection method for identity authentication is applied to a server, the attack detection device for identity authentication may be a computer program running in the server, and the computer program may be used to perform attack detection for identity authentication.

[0102] It can be understood that the embodiment of the present application can analyze the target audio and the target video, so the attack detection method for identity authentication provided by the embodiment of the present application is a multimodal method.

[0103] Among them, a method for detecting attacks on identity authentication may include:

[0104] The target audio and the target video are respectively subjected to designated segmentation processing to obtain a plurality of data segments; wherein the plurality of data segments include respective audio segments and respective video segments; the target audio and the target video are both contents entered when performing user identity authentication based on reading a designated verification text;

[0105] Extracting speech features from each audio segment to obtain a feature vector of each audio segment, and extracting facial motion features from each video segment to obtain a feature vector of each video segment;

[0106] Based on each obtained feature vector, determine the deviation result corresponding to each target data segment of the specified media type; wherein the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment regarding the input time, the target reference segment corresponding to each target data segment is a data segment in each reference segment for the specified media type, the represented reading content matches the reading content represented by the target data segment; each reference segment for the specified media type includes a data segment in a plurality of data segments, which belongs to a media type other than the specified media type;

[0107] Based on the deviation result corresponding to each target data segment, a detection result is determined; wherein the detection result indicates whether there is a prosthetic attack in the user identity authentication process.

[0108] It can be seen that the embodiments of the present application can perform designated segmentation processing on the target audio and target video, so that when performing attack detection on identity authentication, the deviation result corresponding to each target data segment is used as the basis for judgment, thereby obtaining a detection result that characterizes whether there is a prosthesis attack in the user identity authentication process; instead of simply judging the consistency between the effective start and end time of the facial movement and the effective start and end time of the voice to obtain the detection result, so compared with the prior art, the embodiments of the present application can have richer judgment basis for each target data segment, and the accuracy of the detection result is higher, thereby improving the accuracy of attack detection.

[0109] In addition, the embodiment of the present application can process the input content in segments first, whether the speed is fast or slow, so as to achieve attack detection for identity authentication. Therefore, since the embodiment of the present application uses the offset result of each target data segment as the judgment basis, the judgment basis in the embodiment of the present application is relatively rich, so as to improve the accuracy of attack detection and the anti-counterfeiting ability for identity authentication. In addition, the embodiment of the present application adopts a statistical method of multi-point data, which can reduce the probability of misjudgment of single-point / few-point data and reduce the probability of breaking the identity authentication device.

[0110] An attack detection method for identity authentication provided by an embodiment of the present application is described below in conjunction with the accompanying drawings.

[0111] Figure 1 A flow chart of an attack detection method for identity authentication provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method may include steps S101-S104:

[0112] S101, performing designated segmentation processing on the target audio and target video respectively to obtain multiple data segments;

[0113] The multiple data segments include audio segments and video segments; the target audio and target video are both the content entered when the user identity authentication is performed based on reading aloud the specified verification text;

[0114] Among them, when performing user identity authentication, the user can be instructed to read a specified verification text, that is, a dynamic password, and audio and picture collection are performed during the user's reading to obtain audio and video of the same duration, that is, the target audio and target video. Then, since the duration of the target audio and target video is the same and both are segmented using a specified segmentation process, then, the target audio and target video are respectively subjected to a specified segmentation process, and the duration of each audio segment and each video segment obtained is also the same. Exemplarily, the target audio A and the target video B are subjected to a specified segmentation process, then 4 audio segments and 4 video segments can be obtained, and the duration of each audio segment and each video segment is the same. It can be understood that the embodiment of the present application does not limit the specific content of the target audio and target video.

[0115] It should be noted that, in order to facilitate analysis and processing, for each audio segment, an audio segment sequence can be formed, and for each video segment, a video segment sequence can be formed, or other forms can be formed, and the embodiments of the present application do not specifically limit this. In addition, in the audio segment sequence, the order of each audio segment can be sorted according to the corresponding recording time; similarly, in the video segment sequence, the order of each video segment can be sorted according to the corresponding recording time.

[0116] Optionally, in one implementation, performing designated segmentation processing on the target audio and the target video respectively to obtain multiple data segments may include step A1:

[0117] Step A1, segmenting the target audio and target video according to a predetermined step length and a predetermined segment length to obtain multiple data segments; wherein the value of the predetermined step length is smaller than the value of the predetermined segment length.

[0118] It is understandable that the predetermined step length and the predetermined segment length are pre-set by the relevant personnel. The relevant personnel can set the interval duration of each audio segment and each video segment, and the duration of each audio segment and each video segment by pre-setting the predetermined step length and the predetermined segment length. It should be noted that the predetermined segment length can be understood as a sliding window, and the predetermined step length is the distance that the sliding window slides each time. The sliding window slides on the target audio or target video to achieve segmentation of the target audio or target video. Exemplarily, the duration of the target video A is 1s, the predetermined step length is 0.1s, and the predetermined segment length is 0.5s. Then, the target video A can be divided into 6 segments, the first target video is from 0-0.5s, the second target video is from 0.1-0.6s, the third target video is from 0.2-0.7s, the fourth target video is from 0.3-0.8s, the fifth target video is from 0.4-0.9s, and the sixth target video is 0.5-1s.

[0119] In addition, since the area of ​​the predetermined step length is smaller than the value of the predetermined segment length when setting the predetermined step length and the predetermined segment length, each audio segment and video segment has overlapping content. It should be noted that since the user will naturally pause when reading the specified verification text, the target audio and target video will contain content collected during the quiet period. If each audio segment and video segment do not have overlapping parts and are only segmented according to the segment length, it is possible that the content of an audio segment and a video segment is just in the quiet period. In this way, in the subsequent steps, the audio segment and the video segment have fewer features that can be collected, and the corresponding target reference segment may not be determined, thereby affecting the accuracy of attack detection; based on this consideration, the step length can be set to a value smaller than the segment length to ensure that there is overlapping content between adjacent audio segments and adjacent video segments, thereby reducing the possibility that there is no valid information in the data segment. It should be noted that not all audio segments and video segments must have overlapping parts, but there are overlapping parts between two adjacent audio segments or two adjacent video segments.

[0120] It can be seen that since each audio segment and video segment has overlapping parts, it can be ensured that each audio segment and video segment has more features, reducing the probability of lack of useful information due to quiet periods, thereby affecting the accuracy of attack detection. Therefore, the embodiment of the present application can improve the accuracy of attack detection and the anti-counterfeiting capability for identity authentication.

[0121] S102, extracting speech features from each audio segment to obtain a feature vector for each audio segment, and extracting facial motion features from each video segment to obtain a feature vector for each video segment;

[0122] It is understandable that the feature vector of each video segment may be a facial motion feature vector, the facial motion feature vector may be a lip feature vector, or may be other feature vectors, such as eyes, nose, etc., and the embodiments of the present application do not specifically limit this.

[0123] Exemplarily, for the speech feature extraction of each audio segment, MFCC (Mel-scale Frequency Cepstral Coefficients) can be used to obtain the feature vector of each audio segment, and the feature vector of each audio segment can also be called a speech feature vector or an audio feature vector. Of course, other technologies can also be used to extract the speech feature vector, which is not specifically limited in the embodiments of the present application.

[0124] Exemplarily, for each video segment, facial motion feature extraction can be performed based on face detection, key point positioning and other technologies to obtain a feature vector of facial motion for the video segment. Exemplarily, the feature content extracted by facial motion feature extraction may include one or more of lip width, lip height, lip angle, lip change information, and the like. Moreover, the feature vector of each video segment may be a normalized feature vector, but is certainly not limited thereto; in addition, the lip change information may include information belonging to first-order changes, and / or information belonging to second-order changes; exemplary, the information belonging to first-order changes may be the speed of lip changes, and the information belonging to second-order changes may be the acceleration of lip changes. Of course. The present application does not limit the specific implementation method of facial motion feature extraction.

[0125] It should be noted that the feature vectors of the above-mentioned audio segments and the feature vectors of the above-mentioned video segments may be in the form of a sequence or may not be in the form of a sequence, and the embodiments of the present application do not make any specific limitation on this.

[0126] S103, determining a deviation result corresponding to each target data segment of a specified media type based on each obtained feature vector;

[0127] Among them, the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment regarding the entry time, the target reference segment corresponding to each target data segment is a data segment in each reference segment for the specified media type, the reading content represented by which matches the reading content represented by the target data segment; each reference segment for the specified media type includes data segments of multiple data segments that belong to media types other than the specified media type; each target data segment of the specified media type is a data segment among multiple data segments.

[0128] It can be understood that the number of specified media types can be one or two. Then, when the number of specified media types is one, the specified media type can be an audio type. In this case, each audio segment can be used as a target data segment. In this case, each audio segment corresponds to a deviation result. Each reference segment for the specified media type is a video segment. The target reference segment corresponding to any target data segment is a video segment. Of course, the specified media type can also be a video type. In this case, each video segment can be used as a target data segment. In this case, each video segment corresponds to a deviation result. Each reference segment for the specified media type is an audio segment. The target reference segment corresponding to each video segment is an audio segment. When the number of specified media types is two, both the audio type and the video type can be used as a specified media type. Then, each audio segment and each video segment can be used as a target data segment. Each audio segment and each video segment has its own deviation result. Moreover, for each target data segment belonging to the audio type, each reference segment is a video segment, and the target reference segment is a video segment. For each target data segment belonging to the video type, each reference segment is an audio segment, and the target reference segment is an audio segment.

[0129] It can be understood that the essence of the deviation result corresponding to each target data segment is the deviation between the target data segment and the corresponding target reference segment about the entry time. Then, the deviation result can have multiple forms. The deviation result corresponding to the target data segment can be the entry time of the target data segment and the deviation between the entry time of the target reference segment corresponding to the target data segment; it can also be the position of the target data segment in the sequence to which it belongs and the deviation between the position of the target reference segment corresponding to the target data segment in the sequence. It should be noted that the entry time here is the time information of the data segment; the entry time can be understood as a time period, so the deviation result can be calculated based on the start time of the entry time of the target data segment and the corresponding target reference segment. Of course, it can also be calculated based on the stop time of the entry time, or the middle time point, which is not limited in the embodiment of the present application. Exemplarily, the starting point of the entry time of the target data segment a is 0.5s, and the starting point of the entry time of the target reference segment b corresponding to the target data segment a is 0.6s, so the deviation result can be 0.1s. Of course, the deviation results may have other forms, but their essence is the deviation about the input time, and the embodiments of the present application do not limit the deviation results.

[0130] It should be emphasized that the designated media type of the target data segment is different from that of the corresponding target reference segment. When the designated media type of the target data segment is video, the designated media type of the target reference segment corresponding to the target data segment is audio. Similarly, when the designated media type of the target data segment is audio, the designated media type of the target reference segment corresponding to the target data segment is video. It can be understood that the target reference segment is one of the reference segments for the designated media type, and the reading content represented by the target reference segment matches the reading content represented by the target data segment. Matching can also be understood as the most similar. In addition, each reference segment for the designated media type includes data segments of a media type other than the designated media type among multiple data segments. Then, when the designated media type is audio, each reference segment for the designated media type includes video segments among multiple data segments; similarly, when the designated media type is video, each reference segment for the designated media type includes audio segments among multiple data segments.

[0131] It can be understood that for the sake of clarity of layout, the specific process of determining the deviation result corresponding to each target data segment is introduced in other embodiments, and the embodiments of the present application will not go into details about this.

[0132] S104, determining a detection result based on the deviation result corresponding to each target data segment;

[0133] The detection result indicates whether there is a pseudo attack in the user identity authentication process.

[0134] It can be understood that, based on the deviation results corresponding to each target data segment, it can be determined that there is a prosthetic attack in the user authentication process, or it can be determined that there is no prosthetic attack in the user authentication process.

[0135] It is understandable that for the sake of clarity of the layout, the specific process of the deviation result corresponding to each target data segment is introduced in other embodiments, and the embodiments of the present application will not go into details about this.

[0136] It can be seen that the embodiments of the present application can perform designated segmentation processing on the target audio and target video, so that when performing attack detection on identity authentication, the deviation result corresponding to each target data segment is used as the basis for judgment, thereby obtaining a detection result that characterizes whether there is a prosthesis attack in the user identity authentication process; instead of simply judging the consistency between the effective start and end time of the facial movement and the effective start and end time of the voice to obtain the detection result, so compared with the prior art, the embodiments of the present application can have richer judgment basis for each target data segment, and the accuracy of the detection result is higher, thereby improving the accuracy of attack detection.

[0137] In addition, the embodiment of the present application can process the input content in segments first, whether the speed is fast or slow, so as to achieve attack detection for identity authentication. Therefore, since the embodiment of the present application uses the offset result of each target data segment as the judgment basis, the judgment basis in the embodiment of the present application is relatively rich, so as to improve the accuracy of attack detection and the anti-counterfeiting ability for identity authentication. In addition, the embodiment of the present application adopts a statistical method of multi-point data, which can reduce the probability of misjudgment of single-point / few-point data and reduce the probability of breaking the identity authentication device.

[0138] Based on the above method embodiment, the process of determining the deviation result corresponding to each target data segment of a specified media type is introduced below:

[0139] Optionally, in an implementation, determining the deviation result corresponding to each target data segment of a specified media type based on each obtained feature vector may include steps B1-B2:

[0140] Step B1, for each target data segment of a specified media type, based on a feature vector of the target data segment and feature vectors of each reference segment, selecting a target reference segment corresponding to the target data segment from each reference segment;

[0141] It can be understood that when the specified media type is audio, the feature vector of the target audio segment is a speech feature vector, and the feature vector of each reference segment is a facial motion feature vector; similarly, when the specified media type is video, the feature vector of the target video segment is a facial motion feature vector, and the feature vector of each reference segment is a speech feature vector.

[0142] It is understandable that, based on the feature vector of the target data segment, a data segment whose reading content is most similar to the reading content represented by the target data segment can be selected from each reference segment as the target reference segment corresponding to the target data segment.

[0143] Exemplarily, for each target data segment of the target video, based on the facial motion feature vector of the target video segment a and the audio feature vectors of each reference segment, a target audio segment b whose reading content is closest to the reading content represented by the target video segment a is selected from each reference segment as the target reference segment corresponding to the target video segment a.

[0144] In addition, it can be understood that in an optional implementation, the target reference segment corresponding to the target data segment can be selected from each reference segment by using a deep learning method to identify the semantics represented by the feature vector of the target data segment and the degree of difference between the semantics represented by the feature vectors of each reference segment, and the reference segment with the smallest degree of difference is used as the target reference segment. For the sake of clarity, other implementations of selecting the target reference segment corresponding to the target data segment from each reference segment will be introduced in conjunction with other embodiments later.

[0145] Step B2, for each target data segment, based on the target reference segment corresponding to the target data segment, determining the deviation result corresponding to the target data segment;

[0146] It should be noted that when determining the deviation result corresponding to the target data segment, the deviation between the target data segment and the corresponding target reference segment regarding the recording time can be used. Then, the processing idea can be understood as using the target video reference segment corresponding to the target audio segment to determine the deviation result corresponding to the target audio segment. The deviation result is about the target video segment, that is, the target audio segment is used to determine the deviation result about the target video segment; similarly, for the deviation result corresponding to the target video segment, it can be understood as using the target video segment to determine the deviation result about the target audio segment. Exemplarily, for each target video segment, based on the deviation between the target reference segment b corresponding to the target video segment a regarding the recording time, the deviation result corresponding to the target video segment a can be determined.

[0147] It can be seen that the embodiment of the present application can select the target reference segment corresponding to the target data segment based on the feature vector of the target data segment and the feature vectors of each reference segment, and determine the deviation result corresponding to the target data segment based on the target reference segment. The embodiment of the present application adopts a cross-processing processing idea, that is, using the target audio segment to determine the deviation result about the target video segment, and using the target video segment to determine the deviation result about the target audio segment. This processing idea can improve the accuracy of the deviation result, thereby improving the accuracy of attack detection.

[0148] Optionally, in one implementation, step B1 may include step B11:

[0149] Step B11, for each target data segment of the specified media type, according to the feature vector of the target data segment, a feature template that meets the specified selection condition is selected from a plurality of pre-constructed feature templates, and based on the selected feature template and the feature vectors of each reference segment, a target reference segment corresponding to the target data segment is determined;

[0150] Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial movement features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation processing on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation processing on the sample video. The sample audio and sample video are both contents recorded when performing sample user identity authentication based on reading aloud sample verification text, and the time information corresponding to the sample audio segment and the sample video is the same;

[0151] The specified selection condition includes: the distance between the specified feature vector and the feature vector of the target data segment is the shortest; the specified feature vector is a feature vector of the sample segment of the specified media type in the first type of feature vectors and the second type of feature vectors;

[0152] It should be emphasized that the sample verification text may be the same as the specified verification text or may be different from the specified verification text.

[0153] It should be noted that the first feature vector and the second feature vector can be simply the feature vector of the sample segment obtained by performing specified segmentation processing on the sample video and sample audio recorded by the sample user reading the sample verification text, or can be the average facial movement feature vector and average speech feature vector obtained after averaging the feature vectors of multiple sample video segments and sample audio segments. The embodiment of the present application does not specifically limit the specific content of the first feature vector and the second feature vector. In addition, each feature template includes a first feature vector and a second feature vector, and the two feature vectors exist in pairs, so the time information corresponding to the represented sample audio segment and the sample video segment is also the same, and the represented reading content is the same.

[0154] It is understandable that when the specified media type is audio, for each target audio segment, according to the speech feature vector of the target audio segment, a feature template whose first feature vector has the shortest distance to the speech feature vector of the target audio segment is selected from a plurality of pre-constructed feature templates, and based on the selected feature template and the facial motion feature vectors of each video segment, a target video reference segment corresponding to the target audio segment is determined to obtain a target reference segment. Exemplarily, using the speech feature vector c of the target audio segment C, a feature template 1 including a first feature vector c' having the shortest distance to the speech feature vector c is selected from a plurality of pre-constructed feature templates, and based on the feature template 1 and the facial motion feature vectors of each video segment, a target video reference segment D corresponding to the target audio segment C is determined.

[0155] It is understandable that when the specified media type is video, for each target video segment, according to the facial motion feature vector of the target video segment, a feature template whose second feature vector has the shortest distance to the facial motion feature vector of the target video segment is selected from a plurality of pre-constructed feature templates, and the target audio reference segment corresponding to the target video segment is determined based on the selected feature template and the speech feature vectors of each audio segment. Exemplarily, using the facial motion feature vector e of the target video segment E, a feature template 2 including the second feature vector e' having the shortest distance to the facial motion feature vector e is selected from a plurality of pre-constructed feature templates, and the target audio reference segment F corresponding to the target video segment E is determined based on the feature template 2 and the speech feature vectors of each audio segment.

[0156] It should be noted that if the specified media type is audio and video, the above-mentioned process of determining the target video reference segment and determining the target audio reference segment must be performed once with the specified media type as audio, and once with the specified media type as video.

[0157] It can be seen that the embodiment of the present application selects a feature template that meets the specified conditions from multiple pre-constructed feature templates for each target data segment of the specified media type, thereby determining the target reference segment corresponding to the target data segment. The embodiment of the present application determines the target reference segment corresponding to the target data segment by finding the closest feature template, which facilitates determining the deviation result corresponding to the target data segment based on the target reference segment in the subsequent steps, thereby improving the accuracy of the deviation result and improving the accuracy of attack detection.

[0158] Based on the above introduction of selecting feature templates that meet the specified selection conditions, the following introduces a process of determining a target reference segment corresponding to the target data segment based on the selected feature templates and feature vectors of each reference segment.

[0159] Optionally, in an implementation, determining a target reference segment corresponding to the target data segment based on the selected feature template and feature vectors of each reference segment may include steps C1-C2:

[0160] Step C1, respectively calculating the distance between the feature vector to be used and the feature vector of each reference segment; wherein the feature vector to be used is a feature vector in the selected feature template except the specified feature vector;

[0161] It can be understood that when the specified media type is audio, the feature vector to be used can be the second feature vector in the selected feature template, then the distance between the second feature vector in the selected feature template and the facial motion feature vector of each video segment can be calculated respectively. Exemplarily, the distance between the second feature vector c'' in the selected feature template and the facial motion feature vector of each video segment is calculated respectively, and it can be obtained that the distance between the second feature vector c'' and the facial motion feature vector a of video segment A is 1, and the distance between the second feature vector c'' and the facial motion feature vector b of video segment B is 2.

[0162] It can be understood that when the specified media type is video, the feature vector to be used can be the first feature vector in the selected feature template, then the distance between the first feature vector in the selected feature template and the speech feature vector of each audio segment can be calculated respectively. Exemplarily, the distance between the first feature vector d" in the selected feature template and the speech feature vector of each audio segment is calculated respectively, and it can be obtained that the distance between the first feature vector d" and the speech feature vector e of audio segment E is 1, and the distance between the first feature vector d" and the speech feature vector f of audio segment F is 2.

[0163] It should be noted that if the specified media type is audio and video, the above-mentioned process of calculating the distance between the feature vector to be used and the feature vector of each reference segment must be executed once with the specified media type as audio, and once with the specified media type as video.

[0164] Step C2, determining the reference segment to which the feature vector with the shortest distance belongs, and obtaining the target reference segment corresponding to the target data segment;

[0165] It can be understood that based on the distance between the feature vector to be used and the feature vectors of each reference segment obtained by the above calculation, the feature vector with the shortest distance can be determined, and the reference segment to which the feature vector with the shortest distance belongs can be determined, thereby obtaining the target reference segment corresponding to the target data segment.

[0166] It is understandable that when the specified media type is audio, the facial motion feature vector with the shortest distance can be determined from the distance between the second feature vector in the selected feature template and the facial motion feature vectors of each video segment, and the video segment described by the facial motion feature vector can be determined, thereby obtaining the target reference segment corresponding to the audio segment. Exemplarily, the distance between the second feature vector c" and the facial motion feature vector a of video segment A is 1, and the distance between the second feature vector c" and the facial motion feature vector b of video segment B is 2. It can be determined that the facial motion feature vector b is the feature vector with the shortest distance. Then, it can be determined that the reference segment to which the facial motion feature vector b belongs is video segment B, and the target video reference segment corresponding to the audio segment is obtained.

[0167] It can be understood that when the specified media type is video, the speech feature vector with the shortest distance can be determined from the distance between the first feature vector in the selected feature template and the speech feature vectors of each audio segment, and the audio segment described by the speech feature vector can be determined, thereby obtaining the target reference segment corresponding to the video segment. Exemplarily, the distance between the first feature vector d" and the speech feature vector e of the audio segment E is 1, and the distance between the first feature vector d" and the speech feature vector f of the audio segment F is 2. It can be determined that the speech feature vector e is the feature vector with the shortest distance. Then, it can be determined that the reference segment to which the speech feature vector e belongs is the audio segment E, and the target audio reference segment corresponding to the video segment is obtained.

[0168] It should be noted that if the specified media type is audio and video, the process of determining the target reference segment corresponding to the target data segment described above must be performed once when the specified media type is audio, and once when the specified media type is video.

[0169] Of course, the method for determining the target reference segment here is not unique. The reference segment described by the feature vector with the closest similarity can also be used to obtain the target reference segment corresponding to the target data segment. The embodiment of the present application does not make specific limitations on this.

[0170] It can be seen that the embodiment of the present application also adopts a cross-processing approach when determining the target reference segment. The target reference segment corresponding to the target audio segment is the video segment, and the target reference segment corresponding to the target video segment is the audio segment. The target reference segment can be used to determine the deviation result corresponding to the target data segment, thereby improving the accuracy of the deviation result, and further improving the accuracy of attack detection.

[0171] Optionally, in one implementation, each audio segment is stored in a sequence form, and each video segment is stored in a sequence form;

[0172] Accordingly, step B2 may include step B21:

[0173] B21, for each target data segment, determine the first position of the target data segment in the corresponding sequence, and the second position of the target reference segment corresponding to the target data segment in the corresponding sequence, and determine the position difference between the first position and the second position to obtain the deviation result corresponding to the target data segment.

[0174] When the target data segment is an audio segment, the sequence to which it belongs may be an audio segment sequence; when the target data segment is a video segment, the sequence to which it belongs may be a video segment sequence.

[0175] It should be noted that the first position and the second position can be represented as the sorting position of the target data segment and the corresponding target reference segment in the corresponding sequence, and the position difference between the first position and the second position can be understood as the position difference between the sorting positions. It can be understood that the sequence can be sorted according to the corresponding entry time.

[0176] It can be understood that the target data segment is an audio segment. For each audio segment, the first position of the audio segment in the audio segment sequence to which it belongs and the second position of the target reference segment corresponding to the audio segment in the video segment sequence to which it belongs are determined, thereby determining the position difference between the first position and the second position to obtain the deviation result corresponding to the audio segment. Exemplarily, the audio segment a is in the first sorting position in the audio segment sequence to which it belongs, and the target video reference segment b corresponding to the audio segment a is in the second sorting position in the video segment sequence to which it belongs, so the position difference corresponding to the audio segment a can be obtained as 1, and the position difference can be used as the deviation result corresponding to the audio segment a.

[0177] It can be understood that the target data segment is a video segment. For each video segment, the first position of the video segment in the video segment sequence to which it belongs and the second position of the target reference segment corresponding to the video segment in the audio segment sequence to which it belongs are determined, thereby determining the position difference between the first position and the second position to obtain the deviation result corresponding to the video segment. Exemplarily, the video segment c is in the first sorting position in the video segment sequence to which it belongs, and the target audio reference segment d corresponding to the video segment c is in the third sorting position in the audio segment sequence to which it belongs, so the position difference corresponding to the video segment c can be obtained as 2, and the position difference can be used as the deviation result corresponding to the video segment c.

[0178] It should be noted that if the target data segment is an audio segment and a video segment, the above-described process of determining the deviation result corresponding to the target data segment must be performed once with the target data segment being the audio segment and once with the target data segment being the video segment.

[0179] It can be seen that the embodiment of the present application determines the position difference based on the first position of the sequence to which the target data segment belongs and the second position of the sequence to which the corresponding target reference segment belongs, thereby obtaining the deviation result corresponding to the target data segment. The embodiment of the present application can use the deviation result of each target data segment as a basis for judgment, rather than just judging the consistency between the effective start and end time of the facial movement and the effective start and end time of the voice. Therefore, the embodiment of the present application can have a richer basis for judgment for each target data segment, thereby improving the accuracy of attack detection.

[0180] Optionally, in one implementation, the method of constructing multiple feature templates may include steps D1-D5:

[0181] Step D1, obtaining sample audio and sample video corresponding to each sample user; wherein the sample audio and sample video corresponding to each sample user are the content recorded when the sample user identity authentication is performed based on reading aloud the sample verification text;

[0182] It is understandable that the sample audio and sample audio corresponding to each sample user can be the content recorded when the user reads the sample verification text to authenticate the sample user in the past period of time, that is, the recorded content obtained in a non-attack scenario, and the above-mentioned recorded content is saved by the identity authentication device. In addition, in order to facilitate the construction of multiple feature templates, the number of sample users can also be multiple.

[0183] Step D2, for each sample user, performing the specified segmentation processing on the sample audio and sample video of the sample user respectively, to obtain a sample audio segment sequence and a sample video segment sequence of the sample user;

[0184] It is understandable that the process of performing designated segment processing on the sample audio and sample video of the sample user is consistent with the process of performing designated segment processing on the target video and target audio, so it will not be described in detail here.

[0185] In addition, the obtained sample audio segments and sample video segments may be aggregated to obtain a sample audio segment sequence and a sample video segment sequence.

[0186] Step D3: for each sample user, performing speech feature extraction on a sample audio segment in a sample audio segment sequence of the sample user to obtain a first feature vector sequence corresponding to the sample audio segment sequence of the sample user, and performing facial motion feature extraction on a sample video segment in a sample video segment sequence of the sample user to obtain a second feature vector sequence corresponding to the sample video segment sequence of the sample user;

[0187] It can be understood that the process of performing speech feature extraction on the sample audio segments in the sample audio segment sequence of the sample user to obtain the first feature vector sequence corresponding to the sample audio segment sequence of the sample user is consistent with the process of performing speech feature extraction on each audio segment to obtain the feature vector of each audio segment; the process of performing facial motion feature extraction on the sample video segments in the sample video segment sequence of the sample user to obtain the second feature vector sequence corresponding to the sample video segment sequence of the sample user is consistent with the process of performing facial motion feature extraction on each video segment to obtain the feature vector of each video segment, so it will not be elaborated here.

[0188] Step D4, scaling the length of each segmented template pair, deleting the features of the template pairs that exceed the target feature length, and padding the features of the template pairs that are less than the target feature length with zeros, to obtain template pairs with consistent feature lengths.

[0189] It can be understood that each segmented template pair can be composed of speech feature vectors and facial motion feature vectors at the same sequence position in the first feature vector sequence and the second feature vector sequence, and the feature lengths of the speech feature vectors and facial motion feature vectors at the same sequence position are the same.

[0190] It should be emphasized that the scaling ratio length can be 1.1 times, 0.9 times, 0.8 times, etc., and the embodiments of the present application do not specifically limit it. Then, after scaling each template pair, the feature length of the scaled template pair can be compared with the target feature length, and the target feature length is the feature length pre-set by the user. It can be understood that if the feature length of the enlarged template pair exceeds the target feature length, the template pair is feature deleted, and the deleted part can be a part without feature content, such as a silent or paused part, so that the feature length of the template pair is consistent with the target feature length; if the feature length of the reduced template pair is less than the target feature length, the template pair is feature-filled with zeros, and the length of silence can be added to the template pair, so that the feature length of the template pair is consistent with the target feature length.

[0191] Step D5, based on the template pairs with the same feature length, a feature template is obtained by dictionary learning.

[0192] It can be understood that a dictionary is constructed for each template with the same feature length as above, and a matching pursuit algorithm is used to obtain dictionary atoms through dictionary learning. The dictionary is then updated through the dictionary training algorithm K-SVD. After dictionary learning and dictionary updating iterations, the final dictionary atoms are obtained as feature templates.

[0193] Of course, for the method of constructing the feature template, deep learning, dictionary learning, feature learning and other learning methods can also be used, and the embodiment of the present application does not specifically limit them. Any method that can construct a feature template can be applied to this solution.

[0194] It can be seen that, through the method of constructing multiple feature templates introduced above, a feature template can be obtained. Using the feature template, it is convenient to determine the offset result in the subsequent steps, thereby improving the accuracy of attack detection.

[0195] Optionally, in one implementation, the present application embodiment also provides another attack detection method for identity authentication, such as Figure 2 As shown, the following steps may be included:

[0196] S201, performing designated segment processing on the target audio and target video respectively to obtain multiple data segments;

[0197] S202, extracting speech features from each audio segment to obtain a feature vector for each audio segment, and extracting facial motion features from each video segment to obtain a feature vector for each video segment;

[0198] S203, determining a deviation result corresponding to each target data segment of a specified media type based on each obtained feature vector;

[0199] It is understandable that steps S201-S203 are the same as steps S101-S103, so they are not described here in detail.

[0200] S204, determining the deviation results corresponding to each data segment group based on the deviation results corresponding to each target data segment;

[0201] Wherein, each data segment group includes an audio segment and a video segment having the same corresponding time information;

[0202] It can be understood that, based on the deviation results corresponding to each target data segment, the deviation results corresponding to each audio segment and video segment containing the same corresponding time information can be determined. It should be noted that the deviation results may depend on the specified media type. When the specified media type is audio, the deviation results corresponding to each data segment group may be the deviation results corresponding to the audio segment. When the specified media type is video, the deviation results may be the deviation results corresponding to the video segment. When the specified media type is audio and video, the deviation results may be the deviation results corresponding to the audio segment and the video segment.

[0203] Optionally, in one implementation, the number of the specified media types is two;

[0204] Accordingly, based on the deviation results corresponding to the target data segments, determining the deviation results corresponding to the data segment groups may include step E1:

[0205] Step E1, for each data segment group, weighted average the deviation results corresponding to the target data segments belonging to the data segment group to obtain the deviation result corresponding to the data segment group.

[0206] It is understandable that when the specified media type is audio and video, the deviation result corresponding to the target data segment belonging to the data segment group may include the deviation result corresponding to the audio segment and the deviation result corresponding to the video segment. Then, the two deviation results may be weighted and averaged to obtain the deviation result corresponding to the data segment group. It should be noted that the weights in the weighted calculation may be set in advance by relevant personnel. For example, relevant personnel may set in advance that the deviation result corresponding to the audio segment accounts for 60% and the deviation result corresponding to the video segment accounts for 40%. Then, the deviation results corresponding to the target data segment may be weighted and averaged according to the above weights to obtain the deviation result corresponding to the data segment group.

[0207] Of course, when the number of specified media types is two, other methods may also be used to obtain the deviation result corresponding to the data segment group, and the embodiment of the present application does not specifically limit this.

[0208] It can be seen that the embodiment of the present application can perform weighted averaging of the deviation results corresponding to the target data segments of each data segment group when the number of specified media types is two, thereby obtaining the deviation results corresponding to the data segment group. Therefore, the embodiment of the present application can be applicable to the case where the number of specified media types is two, and the two deviation results can be weighted averaged, so that the final deviation results corresponding to the data segment group are more accurate, thereby improving the accuracy of attack detection.

[0209] S205, according to the deviation result corresponding to each data segment group, determining the proportion of the data segment groups within the preset deviation range in each data segment group as the target proportion;

[0210] It can be understood that each data segment group corresponds to a deviation result. Then, by statistically analyzing the deviation results corresponding to each data segment group, an interval graph of the deviation results corresponding to each data segment group can be obtained. The deviation results corresponding to each data segment group are a point on the interval graph. Then, using the preset deviation range, the deviation results in the preset deviation range can be calculated, and the proportion in the interval graph can be used as the target proportion. Exemplarily, there are 10 deviation results corresponding to data segment groups, the interval of the deviation results is -10-10, the preset deviation range is -2-2, and there are 8 deviation results in the preset deviation range. Then, the target proportion can be calculated as 80%.

[0211] Optionally, in one implementation, the specified media types are audio and video, and the deviation results corresponding to the audio segment and the deviation results corresponding to the video segment are not fused. In this case, the deviation result can be a deviation plane, and the horizontal coordinate of the deviation plane can be the deviation result corresponding to the video segment, and the vertical coordinate can be the deviation result corresponding to the audio segment. It can be understood that by comparing the deviation plane with the theoretical plane, the overlap ratio of the deviation plane and the theoretical plane can also be obtained, and the overlap ratio can be used as the target ratio, wherein the deviation results within the theoretical plane are all deviation results within the normal deviation range, then the portion where the deviation plane overlaps with the theoretical plane represents the deviation result within the normal deviation range.

[0212] In addition, the target ratio does not have a unique meaning. It can also represent the ratio of extremely large values ​​to the total deviation. The following is a brief introduction to this situation:

[0213] Optionally, in one implementation, the target proportion is the proportion of extremely large values ​​in the total deviation. Then, the number of extremely large values ​​can be counted in the interval diagram of the deviation results corresponding to each data segment group to obtain the target proportion. Exemplarily, there are deviation results corresponding to 10 data segment groups, the interval of the deviation results is -10-10, the extremely large value is 10, and there are 2 deviation results reaching extremely large values. Then, it can be calculated that the proportion of extremely large values ​​in the total deviation is 20%.

[0214] Of course, there are other ways to calculate the target proportion, for example, fitting the deviation distribution area and the training distribution curve to calculate the similarity between the two, which can be understood as the target proportion, wherein the training distribution curve is a curve obtained through multiple trainings with deviations within a normal range. The specific implementation method for obtaining the target proportion is not specifically limited in the embodiments of the present application.

[0215] S206: Determine a detection result based on a relationship between the target proportion and a preset threshold.

[0216] The preset threshold is used to characterize and determine whether there is a prosthesis attack during the identity authentication process.

[0217] It is understandable that based on the size relationship between the target proportion and the preset threshold, it can be determined whether there is a prosthesis attack during the identity authentication process, but due to the different meanings represented by the target proportion, the detection results determined based on the size relationship between the target proportion and the preset threshold are also different.

[0218] The following describes the target ratio, which represents the ratio of the deviation results within the normal deviation range to the total deviation:

[0219] Optionally, in one implementation, determining the detection result based on the magnitude relationship between the target proportion and a preset threshold value includes steps F1-F2:

[0220] Step F1, if the target proportion is greater than a preset threshold, the first result is determined as the detection result;

[0221] The first result is a result indicating that there is no pseudo attack in the user identity authentication process;

[0222] It should be noted that the preset threshold is set in advance by relevant personnel and may be a benchmark value obtained based on multiple determinations.

[0223] It can be understood that if the target ratio is greater than the preset threshold, it means that the proportion of the deviation results in the normal range to the total deviation is higher than the preset threshold, and the deviation results in the normal range account for a higher proportion of the total deviation, so the first result can be determined as the detection result, that is, there is no prosthesis attack in the user authentication process. Exemplarily, the target ratio is 80%, the deviation results in the normal range account for 80% of the total deviation, and the preset threshold is 75%, that is, the deviation results in the normal range account for more than 75% of the total deviation. It can be considered that there is no prosthesis attack in the user authentication process. Since the target ratio is greater than the preset threshold, at this time, the first result can be determined as the detection result, that is, the result that characterizes the absence of prosthesis attack in the user authentication process.

[0224] Step F2, if the target proportion is not greater than the preset threshold, determining the second result as the detection result;

[0225] The second result is a result indicating that a pseudo attack exists in the user identity authentication process.

[0226] It can be understood that if the target ratio is not greater than the preset threshold, it indicates that the ratio of the deviation results in the normal range to the total deviation is lower than the preset threshold, and the deviation results in the normal range account for a low proportion of the total deviation, so the second result can be determined as the detection result, that is, there is a prosthetic attack in the user authentication process. Exemplarily, the target ratio is 50%, the deviation results in the normal range account for 50% of the total deviation, and the preset threshold is 75%. Since the target ratio is less than the preset threshold, at this time, the second result can be determined as the detection result, that is, the result of the prosthetic attack in the user authentication process.

[0227] The following describes the target ratio, which represents the ratio of extreme large values ​​to the total deviation:

[0228] Optionally, in one implementation, the target proportion represents the proportion of the extremely large value to the total deviation;

[0229] Correspondingly, if the target proportion is not greater than the preset threshold, the first result is determined as the detection result;

[0230] If the target proportion is greater than the preset threshold, the second result is determined as the detection result;

[0231] It can be understood that if the target proportion is greater than the preset threshold, it indicates that the proportion of extreme large values ​​in the total deviation is higher than the preset threshold, the proportion of extreme large values ​​in the total deviation is relatively high, and more deviation results are maximum values, so the second result can be determined as the detection result; conversely, if the target proportion is less than the preset threshold, it indicates that the proportion of extreme large values ​​in the total deviation is lower than the preset threshold, and the proportion of extreme large values ​​in the total deviation is relatively low, so the first result can be determined as the detection result. Exemplarily, the target proportion is 40%, the extreme large values ​​account for 40% of the total deviation, and the preset threshold is 20%, that is, the extreme large values ​​account for more than 20% of the total deviation, which can be considered that there is a prosthetic attack in the user authentication process. Since the target proportion is greater than the preset threshold, at this time, the second result can be determined as the detection result, that is, the result of the characterization of the existence of a prosthetic attack in the user authentication process.

[0232] It can be seen that the embodiment of the present application counts the deviation results of each target data segment to obtain the target proportion, and then determines the detection result through the relationship between the target proportion and the preset threshold. Compared with the prior art, the present application does not only judge the consistency between the effective start and end time of the facial movement and the effective start and end time of the audio. Therefore, the judgment basis of the embodiment of the present application is relatively rich, thereby improving the accuracy of the detection.

[0233] In addition, the embodiment of the present application adopts a statistical method of multi-point data, which can reduce the probability of misjudgment of single-point / few-point data and reduce the probability of breaking through the identity authentication device.

[0234] In addition, the attack detection device that implements the attack detection method provided in the embodiment of the present application can be implemented by multiple functional units working together. Exemplarily, in one implementation, as shown in FIG3(a), the attack detection device may include:

[0235] Target video segmentation unit 310, facial motion feature extraction unit 320, target audio segmentation unit 330, speech feature extraction unit 340, search and match unit 350 and statistical scoring unit 360;

[0236] The target video segmentation unit 310 may perform designated segmentation processing on the input target video to obtain various video segments, and there is information overlap between the various video segments.

[0237] The facial motion feature extraction unit 320 may extract facial motion feature points from each input video segment to obtain a facial motion feature vector, and construct a facial motion feature vector sequence based on the facial motion feature vector of each segment.

[0238] The target audio segmentation unit 330 may perform designated segmentation processing on the input target audio to obtain various audio segments, and there is information overlap between the various audio segments.

[0239] The speech feature extraction unit 340 may extract speech features from each input audio segment to obtain a speech feature vector, and construct a speech feature vector sequence based on the speech feature vector of each segment.

[0240] The search and matching unit 350 may determine the deviation result corresponding to each target data segment of the specified media type based on the input facial movement feature vector and voice feature vector.

[0241] The statistical scoring unit 360 may perform statistics on the deviation results corresponding to each target data segment, thereby obtaining a detection result indicating whether there is a prosthetic attack in the user identity authentication process.

[0242] In order to better understand the search and matching unit 350, a brief introduction will be given below in conjunction with the accompanying drawings, as shown in FIG. 3(b) and FIG. 3(c):

[0243] It can be understood that the search and matching unit 350 has a built-in feature template, from which the feature vectors closest to the speech feature vector and the facial movement feature vector can be searched respectively, so as to determine the corresponding target reference segment, and then determine the deviation result corresponding to the target data segment.

[0244] As shown in FIG3(b), the deviation result corresponding to the search target data segment may include:

[0245] S351, input facial motion feature vector;

[0246] S352, searching for a first feature template based on the facial motion feature vector to determine a corresponding target reference segment;

[0247] S353, input speech feature vector;

[0248] S354, searching for a second feature template based on the speech feature vector to determine a corresponding target reference segment;

[0249] S355, for each target data segment, determining a deviation result corresponding to the target data segment based on a target reference segment corresponding to the target data segment;

[0250] It can be understood that the facial motion feature vector corresponds to a video segment, and the facial motion feature vector can be used to search for a first feature template, which includes the facial motion feature vector closest to the facial motion feature vector and the corresponding voice feature vector. Based on the corresponding voice feature vector, the reference segment to which the feature vector closest to the corresponding voice feature vector belongs can be determined, and used as the target reference segment corresponding to the video segment. The second position of the target reference segment corresponding to the video segment in the sequence to which it belongs and the first position of the video segment in the sequence to which it belongs are determined, and the position difference between the first position and the second position are determined, and a first deviation result can be obtained.

[0251] It can be understood that the speech feature vector corresponds to an audio segment, and the speech feature vector can be used to search for a second feature template, which includes a speech feature vector closest to the speech feature vector and a corresponding facial motion feature vector. Based on the corresponding facial motion feature vector, a reference segment to which a feature vector closest to the corresponding facial motion feature vector belongs can be determined, and the reference segment can be used as a target reference segment corresponding to the audio segment. The second position of the target reference segment corresponding to the audio segment in the sequence to which it belongs and the first position of the audio segment in the sequence to which it belongs are determined, and the position difference between the first position and the second position are determined, so as to obtain a second deviation result.

[0252] It can be understood that based on the first deviation result and the second deviation result, the first deviation result and the second deviation result can be weighted and averaged to obtain the deviation result corresponding to the target data segment group.

[0253] It should be noted that the construction method of the feature template is as shown in Figure 3(c):

[0254] S301, input a large amount of sample audio;

[0255] S302, input a large number of sample videos;

[0256] S303, obtaining a feature template through dictionary learning;

[0257] It can be understood that the sample audio and sample video are both content recorded when the sample user's identity is authenticated based on reading aloud the designated verification text.

[0258] It should be noted that the method for constructing the feature template has been introduced in the above embodiment and will not be repeated here.

[0259] In order to better understand the statistical scoring unit 360, a brief introduction will be given below in conjunction with the accompanying drawings, as shown in FIG3(d):

[0260] S361, counting the deviation results corresponding to each target data segment;

[0261] It can be understood that the deviation results corresponding to each of the above target data segments are counted to form an interval graph or a curve graph, etc.

[0262] S362, calculating the target proportion;

[0263] The target ratio represents the ratio of the deviation results within the normal deviation range to the total deviation. It is understandable that the calculation process of the target ratio has been introduced in the above embodiment, so it will not be repeated here.

[0264] S363, determining whether the target proportion meets a predetermined threshold;

[0265] It can be understood that, based on the relationship between the target proportion and the predetermined threshold, it can be determined whether the target proportion meets the predetermined threshold. If the target proportion is greater than the predetermined threshold, the target proportion meets the predetermined threshold, and at this time, step S365 is executed; if the target proportion is less than the predetermined threshold, the target proportion does not meet the predetermined threshold, and at this time, step S364 is executed.

[0266] S364, there is a fake attack in the identity authentication process;

[0267] S365, there is no fake attack in the identity authentication process;

[0268] It can be understood that, since the above steps are all executed by the statistical scoring unit 360, the steps are numbered S361-S365.

[0269] It can be understood that after the determination is completed, the determination result is used as the detection result of the attack detection.

[0270] It can be seen that the embodiments of the present application can perform designated segmentation processing on the target audio and target video, so that when performing attack detection on identity authentication, the deviation result corresponding to each target data segment is used as the basis for judgment, thereby obtaining a detection result that characterizes whether there is a prosthesis attack in the user identity authentication process; instead of simply judging the consistency between the effective start and end time of the facial movement and the effective start and end time of the voice to obtain the detection result, so compared with the prior art, the embodiments of the present application can have richer judgment basis for each target data segment, and the accuracy of the detection result is higher, thereby improving the accuracy of attack detection.

[0271] In addition, the embodiments of the present application can segment the input content regardless of whether it is spoken at a fast or slow speed, so as to achieve attack detection for identity authentication. Since the embodiments of the present application use the offset result of each target data segment as the basis for judgment, the judgment basis in the embodiments of the present application is relatively rich, thereby improving the accuracy of attack detection and the anti-counterfeiting capability for identity authentication.

[0272] Based on the above method embodiment, Figure 4 As shown, the embodiment of the present application provides an attack detection device for identity authentication, including:

[0273] The designated segment processing module 410 is used to perform designated segment processing on the target audio and the target video respectively to obtain multiple data segments; wherein the multiple data segments include each audio segment and each video segment; the target audio and the target video are the content entered when the user identity authentication is performed based on reading the designated verification text;

[0274] A feature extraction module 420 is used to extract speech features from each audio segment to obtain a feature vector of each audio segment, and to extract facial motion features from each video segment to obtain a feature vector of each video segment;

[0275] The first determination module 430 is used to determine the deviation result corresponding to each target data segment of the specified media type based on each obtained feature vector; wherein the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment about the input time, and the target reference segment corresponding to each target data segment is a data segment in each reference segment of the specified media type, the reading content represented by which matches the reading content represented by the target data segment; each reference segment of the specified media type includes a data segment of a media type other than the specified media type among a plurality of data segments;

[0276] The second determination module 440 is used to determine a detection result based on the deviation result corresponding to each target data segment; wherein the detection result indicates whether there is a prosthetic attack in the user identity authentication process.

[0277] Optionally, the first determining module includes:

[0278] A selection submodule, for selecting, for each target data segment of a specified media type, a target reference segment corresponding to the target data segment from each reference segment based on a feature vector of the target data segment and feature vectors of each reference segment;

[0279] A first determination submodule is used to determine, for each target data segment, a deviation result corresponding to the target data segment based on a target reference segment corresponding to the target data segment;

[0280] Optionally, the selection submodule includes:

[0281] A first determination unit is used to select, for each target data segment of a specified media type, a feature template that meets a specified selection condition from a plurality of pre-constructed feature templates according to a feature vector of the target data segment, and determine a target reference segment corresponding to the target data segment based on the selected feature template and feature vectors of each reference segment;

[0282] Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial movement features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation processing on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation processing on the sample video. The sample audio and sample video are both contents recorded when performing sample user identity authentication based on reading aloud sample verification text, and the time information corresponding to the sample audio segment and the sample video is the same;

[0283] The specified selection condition includes: the distance between the specified feature vector and the feature vector of the target data segment is the shortest; the specified feature vector is a feature vector of the sample segment of the specified media type in the first type of feature vectors and the second type of feature vectors;

[0284] Optionally, the first determining unit determines a target reference segment corresponding to the target data segment based on the selected feature template and feature vectors of each reference segment, including:

[0285] Calculating the distance between the feature vector to be used and the feature vector of each reference segment respectively; wherein the feature vector to be used is a feature vector in the selected feature template except the specified feature vector;

[0286] Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment;

[0287] Optionally, each audio segment is stored in a sequence form, and each video segment is stored in a sequence form;

[0288] The first determining submodule includes:

[0289] a second determination unit, configured to determine, for each target data segment, a first position of the target data segment in the sequence to which it belongs, and a second position of a target reference segment corresponding to the target data segment in the sequence to which it belongs, and determine a position difference between the first position and the second position to obtain a deviation result corresponding to the target data segment;

[0290] Optionally, the second determining module includes:

[0291] A second determination submodule is used to determine the deviation results corresponding to each data segment group based on the deviation results corresponding to each target data segment; wherein each data segment group includes an audio segment and a video segment having the same corresponding time information;

[0292] The third determination submodule is used to determine the proportion of data segment groups within a preset deviation range in each data segment group according to the deviation result corresponding to each data segment group, as the target proportion;

[0293] A fourth determination submodule, configured to determine a detection result based on a magnitude relationship between the target proportion and a preset threshold;

[0294] Optionally, the fourth determining submodule includes:

[0295] A third determining unit, configured to determine the first result as a detection result if the target proportion is greater than a preset threshold;

[0296] A fourth determining unit, configured to determine the second result as a detection result if the target proportion is not greater than the preset threshold;

[0297] The first result is a result indicating that there is no pseudo attack in the user identity authentication process;

[0298] The second result is a result indicating that a pseudo attack exists in the user identity authentication process;

[0299] Optionally, the number of the specified media types is two;

[0300] The second determining submodule includes:

[0301] A weighted averaging unit is used to perform weighted averaging on the deviation results corresponding to the target data segments belonging to the data segment group for each data segment group, so as to obtain the deviation result corresponding to the data segment group;

[0302] Optionally, the designated segment processing module includes:

[0303] The segment processing submodule is used to segment the target audio and target video according to a predetermined step length and a predetermined segment length to obtain multiple data segments; wherein the value of the predetermined step length is smaller than the value of the predetermined segment length.

[0304] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0305] The present application also provides an electronic device, such as Figure 5 As shown, including:

[0306] Memory 501, used for storing computer programs;

[0307] The processor 502 is used to implement the above-mentioned attack detection method for identity authentication when executing the program stored in the memory 501.

[0308] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 502, the communication interface, and the memory 501 communicate with each other via the communication bus.

[0309] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0310] The communication interface is used for communication between the above electronic device and other devices.

[0311] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0312] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0313] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned attack detection methods for identity authentication is implemented.

[0314] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the attack detection methods for identity authentication in the above embodiments.

[0315] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD).

[0316] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0317] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0318] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method for detecting attacks on identity authentication, characterized in that: include: The target audio and the target video are respectively subjected to designated segmentation processing to obtain a plurality of data segments; wherein the plurality of data segments include respective audio segments and respective video segments; the target audio and the target video are both contents entered when performing user identity authentication based on reading a designated verification text; Extracting speech features from each audio segment to obtain a feature vector of each audio segment, and extracting facial motion features from each video segment to obtain a feature vector of each video segment; Based on each obtained feature vector, determine the deviation result corresponding to each target data segment of the specified media type; wherein the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment regarding the input time, the target reference segment corresponding to each target data segment is a data segment in each reference segment for the specified media type, the represented reading content matches the reading content represented by the target data segment; each reference segment for the specified media type includes a data segment in a plurality of data segments, which belongs to a media type other than the specified media type; Based on the deviation result corresponding to each target data segment, a detection result is determined; wherein the detection result indicates whether there is a prosthetic attack in the user identity authentication process.

2. The method according to claim 1, characterized in that The step of determining the deviation result corresponding to each target data segment of the specified media type based on each obtained feature vector includes: For each target data segment of the specified media type, based on the feature vector of the target data segment and the feature vectors of each reference segment, a target reference segment corresponding to the target data segment is selected from each reference segment; For each target data segment, based on the target reference segment corresponding to the target data segment, a deviation result corresponding to the target data segment is determined.

3. The method according to claim 2, characterized in that The step of selecting, for each target data segment of the specified media type, a target reference segment corresponding to the target data segment from each reference segment based on the feature vector of the target data segment and the feature vectors of each reference segment, comprises: For each target data segment of the specified media type, according to the feature vector of the target data segment, a feature template that meets the specified selection condition is selected from a plurality of pre-constructed feature templates, and based on the selected feature template and the feature vectors of each reference segment, a target reference segment corresponding to the target data segment is determined; Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial movement features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation processing on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation processing on the sample video. The sample audio and sample video are both contents recorded when performing sample user identity authentication based on reading aloud sample verification text, and the time information corresponding to the sample audio segment and the sample video is the same; The specified selection conditions include: the distance between the specified feature vector included and the feature vector of the target data segment is the shortest; the specified feature vector is a feature vector of a sample segment of the specified media type among the first and second feature vectors.

4. The method according to claim 3, characterized in that The step of determining a target reference segment corresponding to the target data segment based on the selected feature template and the feature vectors of each reference segment includes: Calculating the distance between the feature vector to be used and the feature vector of each reference segment respectively; wherein the feature vector to be used is a feature vector in the selected feature template except the specified feature vector; Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment.

5. The method according to claim 2, characterized in that: The audio segments are stored in a sequence form, and the video segments are stored in a sequence form; For each target data segment, based on a target reference segment corresponding to the target data segment, determining a deviation result corresponding to the target data segment includes: For each target data segment, determine the first position of the target data segment in the corresponding sequence, and the second position of the target reference segment corresponding to the target data segment in the corresponding sequence, and determine the position difference between the first position and the second position to obtain the deviation result corresponding to the target data segment.

6. The method according to claim 1, characterized in that The step of determining the detection result based on the deviation result corresponding to each target data segment includes: Based on the deviation results corresponding to each target data segment, determining the deviation results corresponding to each data segment group; wherein each data segment group includes an audio segment and a video segment having the same corresponding time information; According to the deviation result corresponding to each data segment group, the proportion of the data segment groups within the preset deviation range in each data segment group is determined as the target proportion; The detection result is determined based on the relationship between the target proportion and the preset threshold.

7. The method according to claim 6, characterized in that The determining of the detection result based on the magnitude relationship between the target proportion and a preset threshold value includes: If the target proportion is greater than a preset threshold, the first result is determined as the detection result; If the target proportion is not greater than the preset threshold, the second result is determined as the detection result; The first result is a result indicating that there is no pseudo attack in the user identity authentication process; The second result is a result indicating that a pseudo-attack exists in the user identity authentication process.

8. The method according to claim 6, characterized in that The number of the specified media types is two; The step of determining the deviation results corresponding to each data segment group based on the deviation results corresponding to each target data segment includes: For each data segment group, the deviation results corresponding to the target data segments belonging to the data segment group are weighted and averaged to obtain the deviation result corresponding to the data segment group.

9. The method according to claim 1, characterized in that: The target audio and target video are respectively processed into designated segments to obtain multiple data segments, including: According to a predetermined step length and a predetermined segment length, the target audio and the target video are segmented to obtain a plurality of data segments; wherein the value of the predetermined step length is smaller than the value of the predetermined segment length.

10. An attack detection device for identity authentication, characterized in that: include: A designated segment processing module is used to perform designated segment processing on the target audio and the target video respectively to obtain multiple data segments; wherein the multiple data segments include each audio segment and each video segment; the target audio and the target video are both the content entered when the user identity authentication is performed based on reading a designated verification text; A speech feature extraction module is used to extract speech features from each audio segment to obtain a feature vector of each audio segment, and to extract facial motion features from each video segment to obtain a feature vector of each video segment; A first determination module is used to determine the deviation result corresponding to each target data segment of the specified media type based on each obtained feature vector; wherein the deviation result corresponding to each target data segment represents: the deviation between the target data segment and the corresponding target reference segment about the input time, and the target reference segment corresponding to each target data segment is a data segment in each reference segment of the specified media type, the represented reading content of which matches the reading content represented by the target data segment; each reference segment of the specified media type includes a data segment of a media type other than the specified media type among a plurality of data segments; The second determination module is used to determine the detection result based on the deviation result corresponding to each target data segment; wherein the detection result represents whether there is a prosthesis attack in the user identity authentication process.

11. The device according to claim 10, characterized in that The first determining module includes: A selection submodule, for selecting, for each target data segment of a specified media type, a target reference segment corresponding to the target data segment from each reference segment based on a feature vector of the target data segment and feature vectors of each reference segment; A first determination submodule is used to determine, for each target data segment, a deviation result corresponding to the target data segment based on a target reference segment corresponding to the target data segment; or, The selection submodule includes: A first determination unit is used to select, for each target data segment of a specified media type, a feature template that meets a specified selection condition from a plurality of pre-constructed feature templates according to a feature vector of the target data segment, and determine a target reference segment corresponding to the target data segment based on the selected feature template and feature vectors of each reference segment; Each feature template includes a first feature vector and a second feature vector; the first feature vector represents the audio features in the sample audio segment; the second feature vector represents the facial movement features in the sample video segment; the sample audio segment is a sample segment obtained by performing the specified segmentation processing on the sample audio, and the sample video segment is a sample segment obtained by performing the specified segmentation processing on the sample video. The sample audio and sample video are both contents recorded when performing sample user identity authentication based on reading aloud sample verification text, and the time information corresponding to the sample audio segment and the sample video is the same; The specified selection condition includes: the distance between the specified feature vector and the feature vector of the target data segment is the shortest; the specified feature vector is a feature vector of the sample segment of the specified media type in the first type of feature vectors and the second type of feature vectors; or, The first determining unit determines the target reference segment corresponding to the target data segment based on the selected feature template and the feature vectors of each reference segment, including: Calculating the distance between the feature vector to be used and the feature vector of each reference segment respectively; wherein the feature vector to be used is a feature vector in the selected feature template except the specified feature vector; Determine the reference segment to which the feature vector with the shortest distance belongs, and obtain the target reference segment corresponding to the target data segment; or, The audio segments are stored in a sequence form, and the video segments are stored in a sequence form; The first determining submodule includes: a second determination unit, configured to determine, for each target data segment, a first position of the target data segment in the sequence to which it belongs, and a second position of a target reference segment corresponding to the target data segment in the sequence to which it belongs, and determine a position difference between the first position and the second position to obtain a deviation result corresponding to the target data segment; or, The second determining module includes: A second determination submodule is used to determine the deviation results corresponding to each data segment group based on the deviation results corresponding to each target data segment; wherein each data segment group includes an audio segment and a video segment having the same corresponding time information; The third determination submodule is used to determine the proportion of data segment groups within a preset deviation range in each data segment group according to the deviation result corresponding to each data segment group, as the target proportion; A fourth determination submodule, configured to determine a detection result based on a magnitude relationship between the target proportion and a preset threshold; or, The fourth determining submodule includes: A third determining unit, configured to determine the first result as a detection result if the target proportion is greater than a preset threshold; A fourth determining unit, configured to determine the second result as a detection result if the target proportion is not greater than the preset threshold; The first result is a result indicating that there is no pseudo attack in the user identity authentication process; The second result is a result indicating that a pseudo attack exists in the user identity authentication process; or, The number of the specified media types is two; The second determining submodule includes: A weighted averaging unit is used to perform weighted averaging on the deviation results corresponding to the target data segments belonging to the data segment group for each data segment group, so as to obtain the deviation result corresponding to the data segment group; or, The designated segment processing module comprises: The segment processing submodule is used to segment the target audio and target video according to a predetermined step length and a predetermined segment length to obtain multiple data segments; wherein the value of the predetermined step length is smaller than the value of the predetermined segment length.

12. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-9 when executing a program stored in a memory.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images