Identity verification method and device, model training method and device and electronic equipment

Abstract

The embodiment of the invention discloses an identity verification method and device, a model training method and device and electronic equipment. The method comprises the steps that a plurality of vector sequences corresponding to a plurality of users are obtained, each vector sequence comprises a plurality of vectors, and each vector represents touch data of one touch action and corresponding sensor data; based on the plurality of vector sequences corresponding to the plurality of users, carrying out multiple times of training on a to-be-trained extraction model to obtain a target extraction model; and based on the plurality of vector sequences corresponding to the plurality of users and the target extraction model, training a to-be-trained classification model for multiple times to obtain a target classification model. In this way, the privacy information of the user is prevented from being used in the identity verification process.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to an identity authentication method, a model training method, a device, and an electronic device. Background Art

[0002] During the operation of electronic devices, identity verification can reduce the probability of unauthorized users using the electronic devices. However, during the verification process, the user's private information (such as fingerprint information or facial information) is usually required, which may affect the security of the user's privacy. Summary of the Invention

[0003] In view of the above problems, this application proposes an identity authentication method, a model training method, a device and an electronic device to improve the above problems.

[0004] In the first aspect, the present application provides a model training method, which includes: obtaining multiple vector sequences corresponding to multiple users, wherein the vector sequence includes multiple vectors, each vector representing touch data of a touch action and corresponding sensor data; based on the multiple vector sequences corresponding to the multiple users, training the extraction model to be trained multiple times to obtain a target extraction model; based on the multiple vector sequences corresponding to the multiple users and the target extraction model, training the classification model to be trained multiple times to obtain a target classification model.

[0005] In the second aspect, the present application provides an identity authentication method, which includes: obtaining data to be verified, the data to be verified including touch data collected multiple times and corresponding sensor data; obtaining a corresponding vector sequence based on the data to be verified, wherein the vector sequence includes multiple vectors, each vector representing the touch data of a touch action and the corresponding sensor data; obtaining a multidimensional feature vector to be verified through a target extraction model and the vector sequence; verifying the multidimensional feature vector to be verified through a target classification model and a pre-registered multidimensional feature vector to verify the legitimacy of the user to whom the data to be verified belongs.

[0006] In the third aspect, the present application provides a model training device, which includes: a sequence acquisition unit, used to obtain multiple vector sequences corresponding to multiple users, wherein the vector sequence includes multiple vectors, each vector represents touch data of a touch action and corresponding sensor data; a first training unit, used to train the extraction model to be trained multiple times based on the multiple vector sequences corresponding to the multiple users to obtain a target extraction model; a second training unit, used to train the classification model to be trained multiple times based on the multiple vector sequences corresponding to the multiple users and the target extraction model to obtain a target classification model.

[0007] In a fourth aspect, the present application provides an identity authentication device, which includes: a data acquisition unit for acquiring data to be verified, which is used to acquire data to be verified, and the data to be verified includes touch data collected multiple times and corresponding sensor data; a sequence acquisition unit, which is used to obtain a corresponding vector sequence based on the data to be verified, wherein the vector sequence includes multiple vectors, each vector representing touch data of a touch action and corresponding sensor data; a vector acquisition unit, which is used to obtain a multidimensional feature vector to be verified through a target extraction model and the vector sequence; a verification unit, which is used to verify the multidimensional feature vector to be verified through a target classification model and a pre-registered multidimensional feature vector, so as to verify the legitimacy of the user to whom the data to be verified belongs.

[0008] In a fifth aspect, the present application provides an electronic device, which includes at least a processor and a memory; one or more programs are stored in the memory and configured to be executed by the processor to implement the above method.

[0009] In a sixth aspect, the present application provides a computer-readable storage medium, in which program code is stored, wherein the above method is executed when the program code is executed by a processor.

[0010] The present application proposes an identity authentication method, a model training method, an apparatus and an electronic device, which can first obtain multiple vector sequences corresponding to multiple users, wherein the vector sequence includes multiple vectors, each vector representing the touch data of a touch action and the corresponding sensor data. Then, based on the multiple vector sequences corresponding to the multiple users, the extraction model to be trained can be trained multiple times to obtain a target extraction model. Finally, based on the multiple vector sequences corresponding to the multiple users and the target extraction model, the classification model to be trained can be trained multiple times to obtain a target classification model. Thus, through the above method, the extraction model to be trained and the classification model to be trained can be trained by the multiple vector sequences of multiple users to obtain the target extraction model and the target classification model, and then when identity authentication is required, the target extraction model can be used to extract feature vectors from the collected data of the user to be authenticated (touch data when the user performs a touch action and the corresponding sensor data), and then the target classification model can be used to authenticate the user based on the feature vectors extracted by the target extraction model, thereby avoiding the use of the user's privacy information in the identity authentication process. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A schematic diagram showing an application scenario proposed in an embodiment of the present application is shown;

[0013] Figure 2 A schematic diagram showing another application scenario proposed in an embodiment of the present application is shown;

[0014] Figure 3 A flow chart of a model training method provided in an embodiment of the present application is shown;

[0015] Figure 4 A schematic diagram showing the method of determining the first touch operation from touch data collected multiple times in the implementation of the present application is shown;

[0016] Figure 5 A flow chart of a model training method provided by another embodiment of the present application is shown;

[0017] Figure 6 A flow chart of a model training method provided in another embodiment of the present application is shown;

[0018] Figure 7A flowchart of an identity authentication method provided by another embodiment of the present application is shown;

[0019] Figure 8 The following is an architecture diagram of the application verification system;

[0020] Figure 9 A structural block diagram of a model training device proposed in an embodiment of the present application is shown;

[0021] Figure 10 The following is a structural block diagram of an identity verification device proposed in an embodiment of the present application;

[0022] Figure 11 A structural block diagram of an electronic device for executing a model training method or an identity authentication method according to an embodiment of the present application is shown;

[0023] Figure 12 It is a storage unit of an embodiment of the present application for storing or carrying program codes for implementing the model training method or identity authentication method according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] Identity authentication technology is an important part of the modern information security field, and the development of its background technology is closely related to multiple fields such as information technology, cryptography, and biometrics. With the popularization of the Internet and the improvement of the degree of digitization, the importance of identity authentication technology has become increasingly prominent. During the operation of electronic devices, the probability of illegal users using electronic devices can be reduced by performing identity authentication. However, the inventors found in their research that in the verification process of relevant verification methods, it is usually necessary to use the user's private information (for example, fingerprint information or face information, etc.), which may affect the security of the user's privacy.

[0026] Therefore, the inventors have proposed the identity authentication method, model training method, device and electronic device in the present application. In this method, a plurality of vector sequences corresponding to a plurality of users can be first obtained, wherein the vector sequence includes a plurality of vectors, each vector representing the touch data of a touch action and the corresponding sensor data. Then, based on the plurality of vector sequences corresponding to the plurality of users, the extraction model to be trained can be trained multiple times to obtain a target extraction model. Finally, based on the plurality of vector sequences corresponding to the plurality of users and the target extraction model, the classification model to be trained can be trained multiple times to obtain a target classification model. Thus, through the above-mentioned method, the extraction model to be trained and the classification model to be trained can be trained by using the plurality of vector sequences of the plurality of users to obtain the target extraction model and the target classification model. Then, when identity authentication is required, the target extraction model can be used to extract feature vectors from the collected data of the user to be authenticated (touch data when the user performs a touch action and the corresponding sensor data). Then, the target classification model can be used to perform identity authentication based on the feature vectors extracted by the target extraction model, thereby avoiding the use of the user's privacy information during the identity authentication process.

[0027] Before further describing the embodiments of the present application in detail, an application environment involved in the embodiments of the present application is introduced.

[0028] The following first introduces the application scenarios involved in the embodiments of this application.

[0029] In the embodiment of the present application, the model training method or identity authentication method provided can be executed by an electronic device. In this way, all steps in the model training method or identity authentication method provided by the embodiment of the present application can be executed by the electronic device. For example, Figure 1 As shown, when all steps in the model training method or identity authentication method provided in the embodiment of the present application can be executed by an electronic device, all steps can be executed by the processor of the electronic device 100.

[0030] Furthermore, the model training method or identity authentication method provided in the embodiment of the present application can also be executed by a server. Correspondingly, in this mode of execution by the server, the server can start executing the steps in the model training method or identity authentication method provided in the embodiment of the present application in response to a trigger instruction. The trigger instruction can be sent by the electronic device used by the user, or it can be triggered locally by the server in response to some automated events.

[0031] In addition, if Figure 2As shown, the model training method or identity authentication method provided in the embodiment of the present application can also be performed collaboratively by an electronic device and a server. In this manner performed collaboratively by an electronic device and a server, some steps in the identity authentication method provided in the embodiment of the present application are performed by the electronic device, while the other steps are performed by the server. Exemplarily, taking the model training method as an example, the electronic device 100 can perform the model training method including: obtaining multiple vector sequences of users of the electronic device. Afterwards, the electronic device 100 transmits the multiple vector sequences of the users to the server 200. For the server 200, multiple vector sequences uploaded by multiple electronic devices (electronic device 100 and other electronic devices) can be received, and then multiple vector sequences corresponding to each of the multiple users can be obtained. Then, the server 200 performs subsequent training steps to obtain a target extraction model and a target classification model, and returns the target extraction model and the target classification model to the electronic device 100, so that the electronic device 100 performs the identity authentication method provided in the embodiment of the present application based on the target extraction model and the target classification model. Alternatively, after obtaining the target extraction model and the target classification model, the server 200 can also retain the target extraction model and the target classification model locally.

[0032] It should be noted that in this method of collaborative execution by the electronic device and the server, the steps respectively executed by the electronic device and the server are not limited to the methods introduced in the above examples. In actual applications, the steps respectively executed by the electronic device and the server can be dynamically adjusted according to actual conditions.

[0033] It should be noted that the electronic device 100 is Figure 1 and Figure 2 In addition to the smartphone shown in the figure, it can also be a tablet computer, a smart watch, smart glasses and other devices. Server 200 can be an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers. In particular, when the model training method or identity authentication method provided in the embodiment of the present application is executed by a server cluster or distributed system composed of multiple physical servers, different steps in the model training method or identity authentication method can be executed by different physical servers respectively, or can be executed in a distributed manner by a server built based on a distributed system.

[0034] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0035] See also Figure 3 , an embodiment of the present application provides a model training method, the method comprising:

[0036] S110: Acquire multiple vector sequences corresponding to multiple users, wherein the vector sequence includes multiple vectors, and each vector represents touch data of a touch action and corresponding sensor data.

[0037] In the embodiments of the present application, the multiple sequence vectors corresponding to multiple users can be understood as multiple vector sequences collected by the electronic devices used by the multiple users. For a single user, the corresponding multiple vector sequences can be obtained based on touch data and corresponding sensor data from multiple touch operations. The corresponding sensor data can be understood as sensor data corresponding to the touch data. For example, sensor data and touch data collected at the same time can be understood as data having a corresponding relationship with each other.

[0038] In one embodiment, for an electronic device used by a user, touch data and corresponding sensor data can be collected in real time. Optionally, taking a sliding operation as an example, each collected data may include the following three parts:

[0039] Auxiliary data: user number UserId; timestamp timeStamp; screen orientation flag screenOrientation (indicates whether the user uses the electronic device horizontally or vertically); screen resolution screenResolution; screen refresh rate screenRefreshRate.

[0040] Touch data (sliding data): finger sliding action type actionType (representing finger dropping, finger sliding on the screen, and finger lifting respectively); finger number finger_num; coordinates of finger touch tapX and tapY; area of ​​finger touch size. Among them, the finger number is used to represent the Nth finger in the sliding operation. It should be noted that in some cases, the sliding operation can be a multi-finger sliding operation. In this case, the present application can collect sliding data for each finger in the multi-finger sliding operation separately, and then configure the corresponding finger number for each mobile phone separately.

[0041] Sensor data: accelerometer data accX, accY, accZ; gyroscope data gyroX, gyroY, gyroZ; magnetometer data magX, magY, magZ; gravimeter data graX, graY, graZ; compass data oriX, oriY, oriZ; rotation vector meter data roX, roY, roZ; linear accelerometer linX, linY, linZ.

[0042] It should be noted that an electronic device can collect touch data and sensor data in a chronological order. After obtaining multiple collections of touch data and corresponding sensor data, the electronic device can traverse each collection of touch data based on the chronological order, and then obtain touch data for a touch operation based on the touch category in each collection of touch data. After obtaining the touch data for a touch operation, the sensor data collected at the same time as the touch data can be used as the corresponding sensor data. For example, if the touch operation is a sliding operation, the touch category can be the finger sliding action type actionType, and accordingly, the touch data for a touch operation can be understood as the touch data for a sliding operation.

[0043] For example, Figure 4 As shown, the touch data collected by the electronic device in sequence include: touch data D1, touch data D2, touch data D3, touch data D4, touch data D5, touch data D6, touch data D7, touch data D8, and touch data D9. Among them, the action category number corresponding to touch data D1 is 0, the action category number corresponding to touch data D2 is 1, the action category number corresponding to touch data D3 is 2, the action category number corresponding to touch data D4 is 0, the action category number corresponding to touch data D5 is 1, the action category number corresponding to touch data D6 is 2, the action category number corresponding to touch data D7 is 0, the action category number corresponding to touch data D8 is 1, and the action category number corresponding to touch data D9 is 1. If the action category numbers 0, 1, and 2 can form a complete touch operation, then the resulting touch operation can be as follows. Figure 4 shown.

[0044] When the touch data of a touch operation is obtained, a corresponding vector can be obtained based on the touch data of the touch operation and the corresponding sensor data, so that each vector can represent the touch data of the touch operation and the corresponding sensor data. In addition, in one embodiment, vector sequences can be obtained for different finger numbers. In this case, the multiple vector sequences corresponding to the user can be understood as at least one vector sequence corresponding to each of the user's multiple finger numbers. Furthermore, the multiple vectors in each vector sequence represent the relevant data corresponding to the touch operation (for example, a sliding operation) corresponding to the finger with the same number.

[0045] S120: Based on the multiple vector sequences corresponding to the multiple users, the extraction model to be trained is trained multiple times to obtain a target extraction model.

[0046] After obtaining multiple vector sequences for each of the multiple users, the multiple vector sequences for each of the multiple users can be used as training data to train the extraction model to be trained to obtain a target extraction model. In an embodiment of the present application, the target extraction model can be used to perform feature extraction on the user's vector sequence to obtain a multidimensional feature vector of a specified dimension. Alternatively, the target extraction model can further perform feature extraction on the features obtained based on the vector sequence to obtain a multidimensional feature vector of a specified dimension.

[0047] As an approach, the target extraction model can be a feature extractor from the DNN (Deep Neural Networks) category. Optionally, the target extraction model is constructed to consist of five fully connected layers, each containing 128, 256, 128, 64, and 32 nodes, respectively. Optionally, during training, the extraction model can be trained using TripletLoss. This loss function (TripletLoss) minimizes intra-class distances and maximizes inter-class distances.

[0048] S130: Based on the multiple vector sequences corresponding to the multiple users and the target extraction model, the classification model to be trained is trained multiple times to obtain a target classification model.

[0049] It should be noted that when authenticating a user, the user's pre-entered registration data can be obtained, and then the user's data to be authenticated can be compared with the registration data to determine whether the user's identity is legitimate. In the embodiment of the present application, the target classification model is used to compare the user's data to be authenticated with the registration data to determine whether the user's identity is legitimate.

[0050] In the process of training the classification model to be trained, the output of the target extraction model and the multiple vector sequences of the multiple users can be input into the classification model to be trained to perform model training.

[0051] As one approach, in an embodiment of the present application, the target classification model may be a KNN (K-Nearest Neighbors) classifier.

[0052] This embodiment proposes a model training method, so that the extraction model to be trained and the classification model to be trained can be trained through multiple vector sequences of multiple users to obtain a target extraction model and a target classification model. As a result, when identity authentication is required, the target extraction model can be used to extract feature vectors from the collected data of the user to be authenticated (touch data when the user performs touch actions and corresponding sensor data) and then the target classification model can be used to authenticate the identity based on the feature vectors extracted by the target extraction model, thereby avoiding the use of the user's privacy information during the identity authentication process.

[0053] See also Figure 5 , an embodiment of the present application provides a model training method, the method comprising:

[0054] S210: Acquire multiple vector sequences corresponding to multiple users, wherein the vector sequence includes multiple vectors, each vector representing touch data of a touch action and corresponding sensor data.

[0055] S220: Perform feature extraction on each vector in the multiple vector sequences to obtain a first feature of each vector.

[0056] S230: Based on the first feature of each vector in the multiple vector sequences, obtain a plurality of first multidimensional feature vectors corresponding to the multiple users, wherein the plurality of first multidimensional feature vectors correspond to the first vector sequences one-to-one.

[0057] For example, feature extraction can be performed on each vector in a vector sequence to obtain the corresponding feature f i As the first feature, we can then get the total feature F corresponding to a vector sequence = {f1,f2…f n Then, the feature vector of the specified dimension is obtained for the total feature to obtain a first multidimensional feature vector corresponding to a vector sequence, and then the first multidimensional feature vector corresponding to each first vector sequence can be obtained. The specified dimension can be understood as the dimension corresponding to the feature included in the first multidimensional feature vector. Alternatively, in the process of obtaining the total feature corresponding to the vector sequence based on the first feature corresponding to each first vector in the same vector sequence, the total feature can be directly obtained based on the specified dimension, so that the obtained total feature will include a feature vector of the specified dimension. In this case, the obtained total feature can be used as the first multidimensional feature vector.

[0058] S240: Based on the multiple first multidimensional feature vectors corresponding to the multiple users, the extraction model to be trained is trained multiple times to obtain a target extraction model, wherein, in each training process, the multiple first multidimensional feature vectors of one user are selected as legal data, and the multiple first multidimensional feature vectors of users other than the user are selected as illegal data.

[0059] In an embodiment of the present application, the dimension of the vector included in the first multidimensional feature vector may be 46. Optionally, taking the touch operation as a sliding operation as an example, the features included in the 46-dimensional first multidimensional feature vector may include:

[0060] Action duration duration, relative distance of the sliding action in the X and Y directions relativeStopPositionX, relativeStopPositionY; range of the sliding action in the X and Y directions Xrange, Yrange; maximum and minimum values ​​of the sliding action in the X and Y directions Xmax, Xmin, Ymax, Ymin; vector distance end2endDistance, trajectory distance trajectoryDistance of the sliding action; trajectory direction end2endDirection of the sliding action; area downSize when the finger starts to contact the screen; variance stdSize of the contact area; meanSize of the contact area; vector velocity meanVelocity; path velocity trajVelocity; mean acceleration meanAcc; variance stdAcc of acceleration; displacement DisplacementDownX, DisplacementDownY of the finger in the X and Y directions when two consecutive drops are made; displacement of the finger in the X and Y directions when the finger is lifted and the next drop is made displacement; mean pitch angle pitchMean; mean yaw angle yawMean; mean roll angle rollMean; variance of pitch angle pitchStd; variance of yaw angle yawStd; variance of roll angle rollStd; minimum pitch angle pitchMin; minimum yaw angle yawMin; minimum roll angle rollMin; maximum pitch angle pitchMax; maximum yaw angle yawMax; maximum roll angle rollMax; root mean square of pitch angle pitchRMS; yaw angle The root mean square of the heading angle yawRMS; the root mean square of the roll angle rollRMS; the average deviation of the pitch angle pitchAd, the average deviation of the yaw angle yawAd; the average deviation of the roll angle rollAd; the distance from the starting point to the farthest point Start2LDPlen; the distance from the farthest point to the end point LDP2Endlen; the time interval from the starting point to the farthest point Start2LDPduration; the time interval from the farthest point to the end point LDP2Endduration; the distance LDP from the farthest point to the connecting line.

[0061] As a method, before executing S240, the multidimensional feature vectors to be processed can be deleted from the multiple first multidimensional feature vectors of the user to obtain multiple second multidimensional feature vectors of the user, wherein the characteristic value of the target feature included in the multidimensional feature vector to be processed does not meet the first condition. Optionally, the target feature can be trajectory distance trajectoryDistance. In this case, the first condition can be that the trajectory distance is less than 5. After deleting the multidimensional feature vector to be processed, the extraction model to be trained can be trained multiple times based on the multiple second multidimensional feature vectors corresponding to each of the multiple users to obtain the target extraction model.

[0062] In an embodiment of the present application, after obtaining the initial training data, data augmentation can be performed on the initial training data, and then training can be performed based on the data augmented training data. Data augmentation is a commonly used technique in machine learning and deep learning. Its core idea is to expand the data set through a series of transformations, thereby making limited data produce the equivalent value of more data without substantially increasing the data. This technique aims to increase the number and diversity of training samples to improve the generalization ability and performance of the model.

[0063] As a method, data enhancement can be performed based on the sliding feature vector and / or sensor feature vector of at least one first multidimensional feature vector among the multiple first multidimensional feature vectors to obtain multiple third multidimensional feature vectors; in this case, the extraction model to be trained can be trained multiple times based on the multiple first multidimensional feature vectors and the multiple third multidimensional feature vectors corresponding to each of the multiple users to obtain a target extraction model.

[0064] In the embodiment of the present application, the obtained first multi-dimensional feature vector may include multiple features, wherein different data enhancement methods may be used for different features.

[0065] Optionally, the performing data enhancement on the sliding feature vectors and / or sensor feature vectors of at least one first multidimensional feature vector among the multiple first multidimensional feature vectors to obtain multiple third multidimensional feature vectors includes: performing data enhancement on the sensor feature vectors of at least one first multidimensional feature vector among the multiple first multidimensional feature vectors to obtain multiple third multidimensional feature vectors, wherein the distribution of the multiple third multidimensional feature vectors obeys a Gaussian distribution.

[0066] It should be noted that, assuming that the noise added to the first multidimensional feature vectors all obeys the Gaussian distribution The mean μ is 0, and the variance σ is 1 times the variance of a certain sensor data, which can be manually specified. After selecting an optimal parameter based on experience in multiple experiments, the constructed Gaussian distribution A value is collected within a specific interval as the newly generated noise, and the noise is superimposed on the sensor data to synthesize the enhanced data.

[0067] It should be noted that, when data enhancement is performed, data enhancement can also be performed on multiple second multidimensional feature vectors to obtain multiple third multidimensional feature vectors. In this manner, data enhancement can be performed based on the sensor feature vector of at least one second multidimensional feature vector among the multiple second multidimensional feature vectors to obtain multiple third multidimensional feature vectors.

[0068] Optionally, the data enhancement is performed on the sliding feature vector and / or sensor feature vector of at least one first multidimensional feature vector among the multiple first multidimensional feature vectors to obtain multiple third multidimensional feature vectors, including: based on data scaling and data displacement, data enhancement is performed on the sliding feature vector of at least one first multidimensional feature vector among the multiple first multidimensional feature vectors to obtain multiple third multidimensional feature vectors.

[0069] As a scaling method, we can select a data augmentation factor of N1, where the augmented data is N1+1 times the original data. Based on the comparison of experimental results, the optimal scaling factor is 0.02.

[0070] As a displacement method, the position shift ratio N2 can be selected, and the position after enhancement is calculated by the following formula:

[0071] X new =X origin ×(1+N2)

[0072] Y new =Y origin ×(1+N2)

[0073] Among them, X new 、Y new For the data after adding, X origin and Y origin To enhance the previous data, optionally, the position shift magnification N2 may be 0.02.

[0074] S250: Based on the multiple vector sequences corresponding to the multiple users and the target extraction model, the classification model to be trained is trained multiple times to obtain a target classification model.

[0075] As one approach, the method of training the classification model to be trained multiple times based on the multiple vector sequences corresponding to the multiple users and the target extraction model to obtain the target classification model includes:

[0076] Performing feature extraction on each vector in the multiple vector sequences based on the target extraction model to obtain a third multidimensional feature vector corresponding to each vector sequence, wherein the third multidimensional feature vector includes the second feature corresponding to each vector in the corresponding vector sequence;

[0077] Based on the third multidimensional feature vectors of the multiple vector sequences of each of the multiple users, the classification model to be trained is trained multiple times to obtain a target classification model, wherein, in each training process, the third multidimensional feature vectors of the multiple vector sequences of one user are selected as positive example data, and the third multidimensional feature vectors of the multiple vector sequences of users other than the user are selected as negative example data.

[0078] This embodiment proposes a model training method, so that the extraction model to be trained and the classification model to be trained can be trained through multiple vector sequences of multiple users to obtain a target extraction model and a target classification model. As a result, when identity authentication is required, the target extraction model can be used to extract feature vectors from the collected data of the user to be authenticated (touch data when the user performs touch actions and corresponding sensor data) and then the target classification model can be used to authenticate the identity based on the feature vectors extracted by the target extraction model, thereby avoiding the use of the user's privacy information during the identity authentication process.

[0079] Furthermore, in this embodiment, a preliminary feature extraction can be performed on multiple vector sequences to obtain a first feature corresponding to each vector in the multiple vector sequences. Then, based on the obtained first features, multiple first multidimensional feature vectors can be further obtained, thereby making the obtained first multidimensional feature vectors more refined and comprehensive, thereby also improving the performance of the trained target extraction model. In addition, in this embodiment, after obtaining the multiple first multidimensional feature vectors, data augmentation can be performed on the multiple first multidimensional feature vectors, thereby improving the data generalization capability of the trained target extraction model.

[0080] See also Figure 6 , an embodiment of the present application provides a model training method, the method comprising:

[0081] S310: Acquire a collection data sequence corresponding to each of a plurality of users, wherein the collection data sequence includes a plurality of collection data obtained in sequence, and the collection data includes touch data and corresponding sensor data, wherein each touch data has a corresponding touch category.

[0082] As one approach, the sensor data includes raw data collected by a sensor of the electronic device and sensor features derived based on the raw data. Optionally, the sensor features may include at least one of pitch, roll, and raw data.

[0083] S320: sequentially identifying the collected data in the collected data sequence, wherein touch data of a first touch category is identified as representing a start action of a touch action, and touch data of a second touch category is identified as representing an end action of a touch action, so as to obtain a plurality of touch actions represented by the collected data sequence;

[0084] S330: Obtaining multiple vector sequences based on the touch data of each of the multiple touch actions and the corresponding sensor data;

[0085] S340: Based on the multiple vector sequences corresponding to the multiple users, the extraction model to be trained is trained multiple times to obtain a target extraction model;

[0086] S350: Based on the multiple vector sequences corresponding to the multiple users and the target extraction model, the classification model to be trained is trained multiple times to obtain a target classification model.

[0087] This embodiment proposes a model training method, so that through the above-mentioned method, the extraction model to be trained and the classification model to be trained can be trained through multiple vector sequences of multiple users to obtain a target extraction model and a target classification model. Therefore, when identity authentication is required, the target extraction model can be used to extract feature vectors from the collected data of the user to be authenticated (touch data when the user performs touch actions and corresponding sensor data). Then, the target classification model is used to authenticate the user based on the feature vectors extracted by the target extraction model, thereby avoiding the use of the user's privacy information during the identity authentication process. Moreover, in this embodiment, each vector obtained involves data about a touch operation (for example, a sliding operation), thereby avoiding interference between different touch operations, so as to improve the performance of the target extraction model finally trained.

[0088] See also Figure 7 , an embodiment of the present application provides an identity authentication method, the method comprising:

[0089] S410: Acquire data to be verified, where the data to be verified includes touch data collected multiple times and corresponding sensor data;

[0090] S420: Obtaining a corresponding vector sequence based on the data to be verified, wherein the vector sequence includes a plurality of vectors, each vector representing touch data of a touch action and corresponding sensor data;

[0091] S430: Obtaining a multidimensional feature vector to be verified through the target extraction model and the vector sequence;

[0092] S440: Verifying the multidimensional feature vector to be verified by using a target classification model and a pre-registered multidimensional feature vector to verify the legitimacy of the user to whom the data to be verified belongs.

[0093] As a method, the multidimensional feature vector to be verified is obtained through the target extraction model and the vector sequence, including: obtaining multiple first features corresponding to the vector sequence, the multiple first features including the first features corresponding to each vector in the vector sequence; inputting the multiple first features into the target extraction model to obtain the multidimensional feature vector to be verified output by the target extraction model.

[0094] like Figure 8 As shown, in Figure 8 The data obtained through data collection and preprocessing shown in the left portion of the figure can be understood as the data used to train the classification model (KNN classification model) to be trained in the aforementioned embodiment. For example, it can be multiple vector sequences corresponding to multiple users in the aforementioned embodiment, or it can be multiple first multidimensional feature vectors.

[0095] See also Figure 9 , an embodiment of the present application provides a model training device 500, the device 500 comprising:

[0096] The sequence acquisition unit 510 is configured to acquire a plurality of vector sequences corresponding to a plurality of users, wherein the vector sequence includes a plurality of vectors, each vector representing touch data of a touch action and corresponding sensor data.

[0097] The first training unit 520 is configured to perform multiple training on the extraction model to be trained based on the multiple vector sequences corresponding to the multiple users to obtain a target extraction model.

[0098] The second training unit 530 is configured to perform multiple training on the classification model to be trained based on the multiple vector sequences corresponding to the multiple users and the target extraction model to obtain a target classification model.

[0099] As a method, the first training unit 520 is specifically used to perform feature extraction on each vector in the multiple vector sequences to obtain the first feature of each vector; based on the first feature of each vector in the multiple vector sequences, a plurality of first multidimensional feature vectors corresponding to each of the multiple users are obtained, wherein the plurality of first multidimensional feature vectors correspond one-to-one to the first vector sequence; based on the plurality of first multidimensional feature vectors corresponding to each of the multiple users, the extraction model to be trained is trained multiple times to obtain a target extraction model, wherein, in each training process, the plurality of first multidimensional feature vectors of one user are selected as legal data, and the plurality of first multidimensional feature vectors of users other than the user are selected as illegal data.

[0100] As one approach, the first training unit 520 is further configured to delete the to-be-processed multidimensional feature vectors from the plurality of first multidimensional feature vectors of the user to obtain a plurality of second multidimensional feature vectors of the user, wherein the feature values ​​of the target features included in the to-be-processed multidimensional feature vectors do not satisfy the first condition. In this approach, the first training unit 520 is further configured to train the to-be-trained extraction model multiple times based on the plurality of second multidimensional feature vectors corresponding to each of the plurality of users to obtain a target extraction model.

[0101] As one approach, the first training unit 520 is further configured to perform data augmentation based on a sliding feature vector and / or a sensor feature vector of at least one of the multiple first multidimensional feature vectors to obtain multiple third multidimensional feature vectors. In this manner, the first training unit 520 is further configured to train the extraction model to be trained multiple times based on the multiple first multidimensional feature vectors and the multiple third multidimensional feature vectors corresponding to each of the multiple users to obtain a target extraction model.

[0102] Optionally, the first training unit 520 is further used to perform data enhancement based on the sensor feature vector of at least one first multidimensional feature vector among the multiple first multidimensional feature vectors to obtain multiple third multidimensional feature vectors, wherein the distribution of the multiple third multidimensional feature vectors obeys a Gaussian distribution.

[0103] Optionally, the first training unit 520 is further configured to perform data enhancement on a sliding feature vector of at least one of the multiple first multidimensional feature vectors based on data scaling and data shifting to obtain multiple third multidimensional feature vectors.

[0104] As a method, the second training unit 530 is specifically used to perform feature extraction on each vector in the multiple vector sequences based on the target extraction model to obtain a third multidimensional feature vector corresponding to each vector sequence, wherein the third multidimensional feature vector includes the second feature corresponding to each vector in the corresponding vector sequence; based on the third multidimensional feature vectors of the multiple vector sequences of the multiple users, the classification model to be trained is trained multiple times to obtain a target classification model, wherein, in each training process, the third multidimensional feature vectors of the multiple vector sequences of one user are selected as positive example data, and the third multidimensional feature vectors of the multiple vector sequences of users other than the user are selected as negative example data.

[0105] As a method, the sequence acquisition unit 510 is specifically used to obtain a collection data sequence corresponding to each of multiple users, wherein the collection data sequence includes a plurality of collection data obtained in sequence, and the collection data includes touch data and corresponding sensor data, wherein each touch data has a corresponding touch category; the collection data in the collection data sequence is identified in sequence, wherein the touch data of the first touch category is identified as a start action representing a touch action, and the touch data of the second touch category is identified as an end action representing a touch action, so as to obtain multiple touch actions represented by the collection data sequence; based on the touch data of each of the multiple touch actions and the corresponding sensor data, multiple vector sequences are obtained.

[0106] See also Figure 10 , an embodiment of the present application provides an identity authentication device 600, the device 600 comprising:

[0107] The data to be verified acquiring unit 610 is configured to acquire the data to be verified, where the data to be verified includes touch data collected multiple times and corresponding sensor data.

[0108] The sequence acquisition unit 620 is configured to obtain a corresponding vector sequence based on the data to be verified, wherein the vector sequence includes a plurality of vectors, each vector representing touch data of a touch action and corresponding sensor data.

[0109] The vector acquisition unit 630 is configured to obtain a multi-dimensional feature vector to be verified through the target extraction model and the vector sequence.

[0110] The verification unit 640 is configured to verify the multidimensional feature vector to be verified by using a target classification model and a pre-registered multidimensional feature vector, so as to verify the legitimacy of the user to whom the data to be verified belongs.

[0111] The following will be combined Figure 11An electronic device provided by this application is described.

[0112] See also Figure 11 Based on the above-mentioned identity authentication method, model training method, and device, the embodiment of the present application also provides an electronic device 1000 that can execute the above-mentioned identity authentication method and model training method. The electronic device 1000 includes one or more (only one is shown in the figure) processors 105, a memory 104, an audio playback module 106, and an audio acquisition device 108 that are coupled to each other. Among them, the memory 104 stores a program that can execute the content of the above-mentioned embodiments, and the processor 105 can execute the program stored in the memory 104.

[0113] The processor 105 may include one or more processing cores. The processor 105 utilizes various interfaces and circuits to connect various components within the electronic device 1000. It executes instructions, programs, code sets, or instruction sets stored in the memory 104, and accesses data stored in the memory 104 to perform various functions and process data within the electronic device 1000. Optionally, the processor 105 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 105 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 105 and may be implemented separately via a communication chip.

[0114] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.

[0115] Furthermore, in addition to the aforementioned components, the electronic device 1000 may further include a network module 110 and a sensor module 112 .

[0116] The network module 110 is used to realize information interaction between the electronic device 1000 and other devices. For example, a connection can be established with other audio playback devices or other electronic devices, and information interaction is performed based on the established connection. As a way, the network module 110 of the electronic device 1000 is a radio frequency module, which is used to receive and send electromagnetic waves, realize the mutual conversion of electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The radio frequency module may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM) cards, memories, etc. For example, the radio frequency module can interact with external devices by sending or receiving electromagnetic waves.

[0117] The sensor module 112 may include at least one sensor. Specifically, the sensor module 112 may include, but is not limited to: a pressure sensor, a motion sensor, an acceleration sensor, and other sensors.

[0118] Among them, the pressure sensor can be a sensor that detects pressure generated by pressing on the electronic device 1000. That is, the pressure sensor detects pressure generated by contact or pressing between the user and the electronic device 1000, for example, pressure generated by contact or pressing between the user's ear and the electronic device 1000. Therefore, the pressure sensor can be used to determine whether contact or pressing occurs between the user and the electronic device 1000, and the magnitude of the pressure.

[0119] The accelerometer can detect the magnitude of acceleration in all directions (generally three axes) and the magnitude and direction of gravity when stationary. This can be used for applications that recognize the posture of the electronic device 1000 (such as switching between landscape and portrait modes, related games, and magnetometer posture calibration), vibration recognition-related functions (such as pedometers and tapping), etc. In addition, the electronic device 1000 may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, and thermometer, which will not be detailed here.

[0120] The audio collection device 108 is used to collect audio signals. Optionally, the audio collection device 108 includes multiple audio collection devices, which may be microphones.

[0121] Please refer to Figure 12 , which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0122] The computer-readable storage medium 800 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 800 comprises a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 for executing any of the method steps described above. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in a suitable form.

[0123] In summary, the present application proposes an identity authentication method, a model training method, a device and an electronic device, which can first obtain multiple vector sequences corresponding to multiple users, wherein the vector sequence includes multiple vectors, each vector representing the touch data of a touch action and the corresponding sensor data. Then, based on the multiple vector sequences corresponding to the multiple users, the extraction model to be trained can be trained multiple times to obtain a target extraction model. Finally, based on the multiple vector sequences corresponding to the multiple users and the target extraction model, the classification model to be trained can be trained multiple times to obtain a target classification model. Thus, through the above method, the extraction model to be trained and the classification model to be trained can be trained by using the multiple vector sequences of multiple users to obtain the target extraction model and the target classification model, and then when identity authentication is required, the target extraction model can be used to extract feature vectors from the collected data of the user to be authenticated (touch data when the user performs a touch action and the corresponding sensor data), and then the target classification model can be used to perform identity authentication based on the feature vectors extracted by the target extraction model, thereby avoiding the use of the user's privacy information in the identity authentication process.

[0124] Among them, an embodiment of the present application discloses an implicit identity authentication system based on touch and sensors of a mobile device. By designing a DNN feature extractor (target extraction model) based on a fully connected layer, multiple data enhancement methods and a KNN classifier (target classification model), it is possible to achieve identity authentication when an unknown user uses the mobile phone only by entering registration data when using the mobile phone for the first time, allowing authenticated users to use the mobile phone, and unauthenticated users cannot unlock the mobile phone.

[0125] Compared with identity authentication methods such as fingerprints and faces, the solution provided in the embodiments of the present application does not involve sensitive privacy information such as faces and fingerprints. Instead, it realizes user identity authentication without infringing on privacy by collecting touch and sensor data when the user uses the mobile phone.

[0126] Currently, mobile phones on the market are equipped with fingerprint, face and other authentication methods, but these methods can only perform one-time authentication and cannot continuously authenticate the user's identity after the electronic device is unlocked. The touch and sensor-based authentication method provided in the embodiment of the present application can continuously perform identity authentication while the user is using the electronic device, preventing the electronic device from being illegally used after being unlocked, and improving the security of the system.

[0127] In addition, the Gaussian, scaling, and moving techniques used in the data enhancement part of the embodiment of the present application maintain the discreteness and difference of the original data while ensuring the increase in data volume, providing data-level support for knowledge extraction during the training process of the user identity authentication model.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A model training method, characterized in that: The method comprises: Acquire multiple vector sequences corresponding to multiple users, wherein the vector sequences include multiple vectors, each vector representing touch data of a touch action and corresponding sensor data; Based on the multiple vector sequences corresponding to the multiple users, the extraction model to be trained is trained multiple times to obtain a target extraction model; Based on the multiple vector sequences corresponding to the multiple users and the target extraction model, the classification model to be trained is trained multiple times to obtain a target classification model.

2. The method according to claim 1, characterized in that The step of training the extraction model to be trained multiple times based on the multiple vector sequences corresponding to the multiple users to obtain a target extraction model includes: Performing feature extraction on each vector in the plurality of vector sequences to obtain a first feature of each vector; Based on the first feature of each vector in the plurality of vector sequences, a plurality of first multidimensional feature vectors corresponding to each of the plurality of users are obtained, wherein the plurality of first multidimensional feature vectors correspond one-to-one to the first vector sequences; Based on the multiple first multidimensional feature vectors corresponding to the multiple users, the extraction model to be trained is trained multiple times to obtain a target extraction model, wherein, in each training process, the multiple first multidimensional feature vectors of one user are selected as legal data, and the multiple first multidimensional feature vectors of users other than the user are selected as illegal data.

3. The method according to claim 2, characterized in that Before the extraction model to be trained is trained multiple times based on the multiple first multi-dimensional feature vectors corresponding to the multiple users to obtain a target extraction model, the method further includes: Deleting a to-be-processed multidimensional feature vector from the plurality of first multidimensional feature vectors of the user to obtain a plurality of second multidimensional feature vectors of the user, wherein the feature value of the target feature included in the to-be-processed multidimensional feature vector does not satisfy the first condition; The step of training the extraction model to be trained multiple times based on the multiple first multi-dimensional feature vectors corresponding to the multiple users to obtain a target extraction model includes: Based on the multiple second multi-dimensional feature vectors corresponding to the multiple users, the extraction model to be trained is trained multiple times to obtain a target extraction model.

4. The method according to claim 2, characterized in that Before the extraction model to be trained is trained multiple times based on the multiple first multi-dimensional feature vectors corresponding to the multiple users to obtain a target extraction model, the method further includes: performing data enhancement based on a sliding feature vector and / or a sensor feature vector of at least one of the plurality of first multidimensional feature vectors to obtain a plurality of third multidimensional feature vectors; The step of training the extraction model to be trained multiple times based on the multiple first multi-dimensional feature vectors corresponding to the multiple users to obtain a target extraction model includes: Based on the multiple first multidimensional feature vectors and the multiple third multidimensional feature vectors corresponding to the multiple users, the extraction model to be trained is trained multiple times to obtain a target extraction model.

5. The method according to claim 4, characterized in that The performing data enhancement based on the sliding feature vector and / or the sensor feature vector of at least one of the plurality of first multidimensional feature vectors to obtain a plurality of third multidimensional feature vectors includes: Data enhancement is performed on a sensor feature vector of at least one of the multiple first multidimensional feature vectors to obtain multiple third multidimensional feature vectors, wherein distribution of the multiple third multidimensional feature vectors obeys Gaussian distribution.

6. The method according to claim 4, characterized in that The performing data enhancement based on the sliding feature vector and / or the sensor feature vector of at least one of the plurality of first multidimensional feature vectors to obtain a plurality of third multidimensional feature vectors includes: Based on data scaling and data shifting, data enhancement is performed on a sliding feature vector of at least one first multidimensional feature vector among the multiple first multidimensional feature vectors to obtain multiple third multidimensional feature vectors.

7. The method according to claim 1, characterized in that The training of the classification model to be trained multiple times based on the multiple vector sequences corresponding to the multiple users and the target extraction model to obtain the target classification model includes: Performing feature extraction on each vector in the multiple vector sequences based on the target extraction model to obtain a third multidimensional feature vector corresponding to each vector sequence, wherein the third multidimensional feature vector includes the second feature corresponding to each vector in the corresponding vector sequence; Based on the third multidimensional feature vectors of the multiple vector sequences of each of the multiple users, the classification model to be trained is trained multiple times to obtain a target classification model, wherein, in each training process, the third multidimensional feature vectors of the multiple vector sequences of one user are selected as positive example data, and the third multidimensional feature vectors of the multiple vector sequences of users other than the user are selected as negative example data.

8. The method according to claim 1, characterized in that The obtaining of multiple vector sequences corresponding to multiple users includes: Acquire a collection data sequence corresponding to each of the plurality of users, wherein the collection data sequence includes a plurality of sequentially obtained collection data, the collection data including touch data and corresponding sensor data, wherein each touch data has a corresponding touch category; Sequentially identifying the collected data in the collected data sequence, wherein touch data of a first touch category is identified as representing a start action of a touch action, and touch data of a second touch category is identified as representing an end action of a touch action, so as to obtain a plurality of touch actions represented by the collected data sequence; A plurality of vector sequences are obtained based on the touch data of each of the plurality of touch actions and the corresponding sensor data.

9. The method according to claim 8, characterized in that The sensor data includes raw data collected by a sensor of the electronic device and sensor features obtained based on the raw data.

10. An identity authentication method, characterized in that: The method comprises: Acquiring data to be verified, where the data to be verified includes touch data collected multiple times and corresponding sensor data; Obtaining a corresponding vector sequence based on the data to be verified, wherein the vector sequence includes a plurality of vectors, each vector representing touch data of a touch action and corresponding sensor data; Obtaining a multidimensional feature vector to be verified through the target extraction model and the vector sequence; The multidimensional feature vector to be verified is verified by using a target classification model and a pre-registered multidimensional feature vector to verify the legitimacy of the user to whom the data to be verified belongs.

11. The method according to claim 10, characterized in that The method of obtaining a multidimensional feature vector to be verified by using the target extraction model and the vector sequence includes: Acquire a plurality of first features corresponding to the vector sequence, wherein the plurality of first features include a first feature corresponding to each vector in the vector sequence; The plurality of first features are input into the target extraction model to obtain a multi-dimensional feature vector to be verified output by the target extraction model.

12. A model training device, characterized in that: The device comprises: A sequence acquisition unit, configured to acquire a plurality of vector sequences corresponding to respective users, wherein the vector sequence includes a plurality of vectors, each vector representing touch data of a touch action and corresponding sensor data; A first training unit is configured to perform multiple training on the extraction model to be trained based on the multiple vector sequences corresponding to the multiple users to obtain a target extraction model; The second training unit is configured to perform multiple training on the classification model to be trained based on the multiple vector sequences corresponding to the multiple users and the target extraction model to obtain a target classification model.

13. An identity verification device, characterized in that: The device comprises: a data acquisition unit for obtaining data to be verified, configured to obtain data to be verified, wherein the data to be verified includes touch data collected multiple times and corresponding sensor data; A sequence acquisition unit, configured to obtain a corresponding vector sequence based on the data to be verified, wherein the vector sequence includes a plurality of vectors, each vector representing touch data of a touch action and corresponding sensor data; A vector acquisition unit, configured to obtain a multidimensional feature vector to be verified through a target extraction model and the vector sequence; The verification unit is used to verify the multidimensional feature vector to be verified by using a target classification model and a pre-registered multidimensional feature vector to verify the legitimacy of the user to whom the data to be verified belongs.

14. An electronic device, characterized in that: The invention comprises a processor and a memory; one or more programs are stored in the memory and are configured to be executed by the processor to implement the method of any one of claims 1 to 9, or to be executed by the processor to implement the method of any one of claims 10 to 11.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 9 is executed, or the method according to any one of claims 10 to 11 is implemented by the processor.

Images