User identity recognition method and device, computer program product and electronic equipment
Through image and audio feature extraction, a parallel convolution module combined with self-attention neural network model, the problem of low security of account name and password authentication is solved, the accuracy and security of user identity recognition is improved, and the risk of password theft is reduced.
Patent Information
- Application Number
- CN202410961081.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-17
AI Technical Summary
In the prior art, the security of authenticating user identity through account names and passwords is low, and there is a risk of password stolen or guessing, affecting the security of user property.
The image information of the target user is obtained through the image acquisition device, and the image information is extracted in feature, combined with the audio acquisition device to obtain audio information, and identity recognition is used using a neural network model based on the self-attention mechanism. The parallel convolution module in the model is from spatial features. The feature vectors are processed in two dimensions and channel features to improve the diversity and richness of features, thereby improving the accuracy and security of identity recognition.
It improves the accuracy and security of user identity identification, reduces the risk of password stolen or guessing, and enhances the authentication security of user identity.
Smart Images

Figure CN118940244A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence, and more specifically, to a user identity recognition method, device, computer program product, and electronic device. Background Art
[0002] As the number of users of digital banking services gradually increases, the business volume of digital banking services is also increasing. Currently, most of the business that can be handled in offline banks can also be handled in digital banks.
[0003] In order to ensure the security of banking services, when using digital banking services, users need to authenticate their identities first, and can only conduct subsequent business after passing the authentication. Currently, when users authenticate their identities, they usually use usernames and passwords for authentication. However, although the method of authenticating by entering a pre-registered username and password is simple and direct, there is a risk of password theft or guessing, which in turn affects the user's property safety.
[0004] Currently, no effective solution has been proposed to address the low security problem of user identity authentication through account names and passwords in related technologies. Summary of the invention
[0005] The present application provides a user identity identification method, device, computer program product and electronic device to solve the problem of low security in authenticating user identity through account name and password in the related art.
[0006] According to one aspect of the present application, a user identity recognition method is provided. The method includes: acquiring image information of a target user through an image acquisition device, and performing a feature extraction operation on the image information to obtain a first feature information set of the target user; acquiring audio information of the target user through an audio acquisition device, and performing a feature extraction operation on the audio information to obtain a second feature information set of the target user; inputting the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module; determining the identity information of the target user according to the recognition result.
[0007] Optionally, in the parallel convolution module, the convolution layer is a one-dimensional convolution layer, which is used to adjust the number of channels of the input features to obtain processed input features, and input the processed input features into the spatial feature transformation module and the channel feature transformation module respectively; the spatial feature transformation module is used to obtain the spatial information of the input features to obtain the first processed data; the channel feature transformation module is used to obtain the channel information of the input features to obtain the second processed data; the first processed data and the second processed data are spliced according to the channel to obtain the output data of the parallel convolution module.
[0008] Optionally, obtaining spatial information of input features to obtain first processed data includes: performing a two-dimensional convolution operation on the processed input features to obtain first feature data; inputting the first feature data into a normalization layer to obtain a first normalized feature vector, and inputting the first normalized feature vector into an activation function layer to obtain a first feature score vector; inputting the first feature data into a normalization layer to obtain a second normalized feature vector, and inputting the second normalized feature vector into an activation function layer to obtain a second feature score vector; and splicing the first feature score vector and the second feature score vector according to channels to obtain the first processed data.
[0009] Optionally, obtaining channel information of the input feature to obtain the second processed data includes: performing a one-dimensional convolution operation on the processed input feature to obtain second feature data, and inputting the second feature data into a normalization layer to obtain a third normalized feature vector, and inputting the third normalized feature vector into an activation function layer to obtain a third feature score vector; multiplying the third feature score vector and the processed input feature to obtain a weighted feature vector, and processing the weighted feature vector through a pooling layer to obtain a reduced-dimensional feature vector, and concatenating the reduced-dimensional feature vector and the processed input feature according to the channel to obtain the second processed data.
[0010] Optionally, the target recognition model is trained in the following manner: obtaining user information of multiple historical users, wherein the user information includes multiple groups of image information and multiple groups of audio information; using the user information as sample data to train an initial recognition model until the recognition accuracy of the initial recognition model is higher than a preset threshold, thereby obtaining a target recognition model.
[0011] Optionally, performing a feature extraction operation on the image information to obtain a first feature information set of the target user includes: determining whether the size of the image information meets a preset size; when the size of the image information is larger than the preset size, changing the size of the image information to the preset size, and normalizing the pixel values of the image information to obtain an initial image; when the size of the image information is smaller than the preset size, padding the image information, and normalizing the pixel values of the image information to obtain an initial image; and inputting the initial image into a convolutional neural network model to perform a feature extraction operation to obtain a first feature information set.
[0012] Optionally, performing a feature extraction operation on the audio information to obtain a second feature information set of the target user includes: preprocessing the audio information to obtain audio data, wherein the preprocessing includes at least one of the following: noise reduction, removal of silent audio, and audio information segmentation; performing a feature extraction operation on the audio information through a recurrent neural network model to obtain multiple initial feature information; and normalizing the multiple initial feature information to obtain a second feature information set.
[0013] According to another aspect of the present application, a user identity recognition device is provided. The device includes: a first acquisition unit, which is used to acquire image information of a target user through an image acquisition device, and perform a feature extraction operation on the image information to obtain a first feature information set of the target user; a second acquisition unit, which is used to acquire audio information of the target user through an audio acquisition device, and perform a feature extraction operation on the audio information to obtain a second feature information set of the target user; an input unit, which is used to input the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module; a determination unit, which is used to determine the identity information of the target user according to the recognition result.
[0014] According to another aspect of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, a user identity recognition method in the present application is implemented.
[0015] According to another aspect of the present invention, an electronic device is provided, comprising one or more processors and a memory; the memory stores computer-readable instructions, and the processor is used to execute the computer-readable instructions, wherein a user identity recognition method is executed when the computer-readable instructions are executed.
[0016] Through this application, the following steps are adopted: the image information of the target user is obtained through an image acquisition device, and a feature extraction operation is performed on the image information to obtain a first feature information set of the target user; the audio information of the target user is obtained through an audio acquisition device, and a feature extraction operation is performed on the audio information to obtain a second feature information set of the target user; the first feature information set and the second feature information set are input into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module; the identity information of the target user is determined according to the recognition result. The problem of low security in authenticating user identity through account name and password in the related art is solved. By obtaining the image information and audio information of the target user, and obtaining the first feature information set and the second feature information set respectively according to the image information and the audio information, and processing the feature vector from the two dimensions of spatial features and channel features through the parallel convolution module in the target recognition model, the diversity and richness of the features are improved, thereby improving the performance and generalization ability of the target recognition model, and achieving the effect of improving the accuracy and security of user identity recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0018] Figure 1 is a flow chart of a user identity identification method provided according to an embodiment of the present application;
[0019] Figure 2 is a schematic diagram of an encoder part provided according to an embodiment of the present application;
[0020] Figure 3 is a schematic diagram of a parallel convolution module provided according to an embodiment of the present application;
[0021] Figure 4 is a schematic diagram of a spatial feature transformation module provided according to an embodiment of the present application;
[0022] Figure 5 is a schematic diagram of a channel feature transformation module provided according to an embodiment of the present application;
[0023] Figure 6 is a schematic diagram of a user identity recognition device provided according to an embodiment of the present application;
[0024] Figure 7It is a schematic diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION
[0025] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0028] It should be noted that the user identity recognition method, device, computer program product and electronic device determined in the present disclosure can be used in the field of artificial intelligence, and can also be used in any field other than the field of artificial intelligence. The application field of the user identity recognition method, device, computer program product and electronic device determined in the present disclosure is not limited.
[0029] It should be noted that the collected information, user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) used in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated. Corresponding operation portals are provided for users to choose to authorize or refuse to use. For example, an interface is set up between this system and relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.
[0030] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:
[0031] RNN model: Recurrent Neural Network model, which is used to process sequence data, has memory function, can capture the time dependency in sequence data, and is suitable for tasks such as natural language processing, speech recognition, and time series prediction.
[0032] CNN model: Convolutional Neural Network model is a deep learning model that is mainly used to process and recognize image data.
[0033] Neural network model based on self-attention mechanism: Transformer model. Transformer is a neural network architecture based on attention mechanism, which was originally used for natural language processing tasks such as machine translation and text generation. The Transformer model captures the dependencies between positions in the input sequence by using the self-attention mechanism without using recurrent neural networks or convolutional neural networks. The Transformer model consists of an encoder and a decoder, each of which contains multiple layers, and each layer consists of a multi-head attention mechanism and a feedforward neural network. During the training process, the Transformer model optimizes the model parameters by minimizing the loss function so that the correct prediction results are generated at the output.
[0034] 2D convolution: In image processing or computer vision, 2D convolution is a mathematical operation that performs a weighted sum of each pixel in an image (or feature map). This operation is often used to extract spatial features in an image.
[0035] Normalization layer: Normalization layer is often used in deep learning models to adjust the output of neurons to a specific range or have a specific distribution, which helps to accelerate model training and improve the generalization ability of the model.
[0036] Sigmoid activation function: The Sigmoid function is a logical function that maps any real value to an output between 0 and 1. In neural networks, it is often used to convert the output of a neuron into a probability value or to control the flow of information.
[0037] MaxPooling: Max pooling is a downsampling operation used to reduce the dimension of a feature map. It creates a new, smaller feature map by selecting the maximum value from each non-overlapping region of the input feature map.
[0038] According to an embodiment of the present application, a user identity identification method is provided.
[0039] Figure 1 is a flow chart of a user identity identification method provided according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:
[0040] Step S101: acquiring image information of a target user through an image acquisition device, and performing a feature extraction operation on the image information to obtain a first feature information set of the target user.
[0041] Specifically, in order to perform identity recognition operations on users without requiring the users to input their account and password, it is possible to determine whether the user's identity is accurate by performing facial recognition on the user.
[0042] Furthermore, the image information of the target user can be obtained through an image acquisition device, for example, the face of the target user can be photographed through a camera to obtain the image information. After obtaining the image information, the image information can be preprocessed, such as image enhancement, resizing, and normalization, to ensure that the image input to the model has a consistent format and size, and the image is input into the CNN convolutional neural network model. The CNN model processes the image information and outputs an image feature vector, which is also the feature vector in the first feature information set.
[0043] Step S102: Acquire audio information of the target user through an audio acquisition device, and perform a feature extraction operation on the audio information to obtain a second feature information set of the target user.
[0044] Specifically, in order to perform identity identification operations on users without the need for users to enter their account and password, it is also possible to determine whether the user's identity is accurate by identifying the user's audio information. For example, the target user uses a terminal device to read a fixed sentence in the terminal device, and the user's identity is identified based on the voice information collected by the terminal device.
[0045] After obtaining the audio information, the voice signal in the audio information can be converted into initial data and input into the RNN recurrent neural network model. The RNN model outputs feature vectors (that is, the second feature information set). These feature vectors usually include information such as audio energy, spectrum, and time domain characteristics of the sound, so that the user's identity can be identified using the above feature vectors.
[0046] Step S103, input the first feature information set and the second feature information set into the target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module.
[0047] Specifically, in order to improve the accuracy of user identification, the first feature information obtained from the image information and the second feature information obtained from the audio information can be input into the target identification model, and interactive fusion is performed in the target identification model to learn the association information between the two modalities. For example, the model can learn the consistency and mutual complementarity between voice and image, thereby improving the accuracy and reliability of identity authentication.
[0048] It should be noted that the model used in this application for processing feature information is a neural network model based on the self-attention mechanism, that is, the Transformer model. However, in order to enable the model to capture information of different scales at the same time, thereby enhancing the diversity and richness of the model's features, a parallel convolution module can be added to the encoder part of the conventional Transformer model. Figure 2 is a schematic diagram of an encoder part provided according to an embodiment of the present application, such as Figure 2As shown, in the conventional Transformer model, the encoder part is composed of multiple encoders, each of which is composed of a multi-head attention mechanism and a feedforward neural network. In this application, a parallel convolution module is added after the multi-head attention mechanism module, so as to further enhance the expressiveness of the Transformer model and the ability to capture local features in the model input information through the parallel convolution module. Among them, in the encoder of the Transformer, the parallel convolution module can extract local features in the input sequence through the convolution kernel (Convolution Kernel), which may be ignored or not fully represented in the multi-head attention mechanism, and by adding the parallel convolution module, the model can capture global and local information at the same time, thereby improving its expressiveness and processing ability for complex sequence data, and the parallel convolution module allows the Transformer model to perform parallel calculations, which can significantly improve the computational efficiency of the model. After the multi-head attention mechanism, the parallel convolution module can process input data from multiple positions at the same time, thereby speeding up the training and reasoning speed of the model.
[0049] That is, in the encoder of the Transformer model, the role of adding a parallel convolution module after the multi-head attention mechanism module is to enhance the model's ability to extract local features, supplement the local information that may be ignored by the global attention mechanism, and improve the computational efficiency of the model through parallel computing, so that the Transformer model can capture global and local information at the same time when processing sequence data, thereby improving its expressiveness and performance.
[0050] Step S104: determining the identity information of the target user according to the recognition result.
[0051] Specifically, after the model outputs the recognition result, the user identity can be determined based on the recognition result. If the user identity authentication is successful, the digital banking system can allow the user to perform corresponding transactions or query operations. If the identity authentication fails, the user's operation request will be rejected, thereby ensuring the security of the user's assets in the digital banking system.
[0052] The user identification method provided in the embodiment of the present application obtains the image information of the target user through an image acquisition device, and performs a feature extraction operation on the image information to obtain a first feature information set of the target user; obtains the audio information of the target user through an audio acquisition device, and performs a feature extraction operation on the audio information to obtain a second feature information set of the target user; inputs the first feature information set and the second feature information set into the target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module; and determines the identity information of the target user according to the recognition result. The problem of low security in authenticating user identity through account name and password in the related art is solved. By obtaining the image information and audio information of the target user, and obtaining the first feature information set and the second feature information set respectively according to the image information and the audio information, and processing the feature vector from the two dimensions of spatial features and channel features through the parallel convolution module in the target recognition model, the diversity and richness of the features are improved, thereby improving the performance and generalization ability of the target recognition model, and achieving the effect of improving the accuracy and security of user identity recognition.
[0053] Optionally, in the user identity identification method provided in the embodiment of the present application, in the parallel convolution module, the convolution layer is a one-dimensional convolution layer, which is used to adjust the number of channels of the input features to obtain processed input features, and input the processed input features into the spatial feature transformation module and the channel feature transformation module respectively; the spatial feature transformation module is used to obtain the spatial information of the input features to obtain the first processed data; the channel feature transformation module is used to obtain the channel information of the input features to obtain the second processed data; the first processed data and the second processed data are spliced according to the channel to obtain the output data of the parallel convolution module.
[0054] It should be noted that in convolutional neural networks, the spatial feature transformation module mainly focuses on the information transformation on the spatial dimensions (height and width) of the image or feature map. This type of module extracts or strengthens the spatial structural information in the image, such as edges, textures, etc., through specific convolution kernel operations; while the channel feature transformation module focuses on the information transformation on the channel dimension (depth or number of features) of the feature map. In convolutional neural networks, each channel usually represents a specific feature, such as color, shape, etc. The channel feature transformation module enhances or suppresses certain features by adjusting the weights or combinations between different channels, thereby optimizing the model's processing of different types of information.
[0055] Specifically, Figure 3 is a schematic diagram of a parallel convolution module provided according to an embodiment of the present application, such as Figure 3 As shown in the figure, the main structures in the parallel convolution module are the spatial feature transformation module and the channel feature transformation module. When the input feature enters the parallel convolution module, it first adjusts the number of channels through a 1×1 convolution layer to meet the needs of the subsequent two parallel branches. Among them, the 1×1 convolution is also called point convolution, which is mainly used to change the number of channels (depth) of the feature map. Through the 1×1 convolution kernel, each position of the feature map can be linearly transformed without changing the spatial size of the feature map, thereby adjusting the number of channels or realizing cross-channel information fusion.
[0056] Furthermore, the adjusted feature map is divided into two paths, one entering the spatial feature transformation module, and the other entering the channel feature transformation module. The spatial feature transformation module focuses on the spatial dimension of the image, and uses the convolution kernel to capture and enhance the spatial structural information such as edges and textures in the image, while the channel feature transformation module focuses on the channel dimension of the feature map, and enhances or suppresses certain features by adjusting the weights or combinations between different channels, thereby optimizing the model's processing of different types of information.
[0057] Finally, the first processed data and the second processed data are spliced according to channels to obtain the output data of the parallel convolution module, thereby completing the processing operation of the feature information.
[0058] It should be noted that when splicing the output features of the spatial feature transformation module and the channel feature transformation module, the channel splicing method can be adopted. For example, assuming that the output of the spatial feature transformation module is a feature vector A, and the output of the channel feature transformation module is a feature vector B, where the dimension of A is [height, width, channels_A], and the dimension of B is [height, width, channels_B]. When the two feature maps are merged by channels, the dimension of the output feature C will be [height, width, channels_A+channels_B], thereby completing the splicing of the features and forming a richer and more diverse feature representation.
[0059] In this embodiment, by setting a parallel convolution module, not only the computational efficiency of the model is improved, but also the diversity and richness of the model's features are enhanced, which helps to improve the performance and generalization ability of the model.
[0060] Optionally, in the user identity recognition method provided in the embodiment of the present application, obtaining the spatial information of the input features and obtaining the first processed data includes: performing a two-dimensional convolution operation on the processed input features to obtain first feature data; inputting the first feature data into a normalization layer to obtain a first normalized feature vector, and inputting the first normalized feature vector into an activation function layer to obtain a first feature score vector; inputting the first feature data into a normalization layer to obtain a second normalized feature vector, and inputting the second normalized feature vector into the activation function layer to obtain a second feature score vector; and splicing the first feature score vector and the second feature score vector according to channels to obtain the first processed data.
[0061] Specifically, Figure 4 is a schematic diagram of a spatial feature transformation module provided according to an embodiment of the present application, such as Figure 4 As shown in the figure, in the spatial feature reconstruction module, the input data is first divided into two parallel processing branches through a 2D convolution operation. The two branches extract features from the data independently, and the feature map of each branch is adjusted through a normalization layer to output normalized feature vectors W and Y. Subsequently, these feature vectors are nonlinearly transformed through the Sigmoid activation function to generate feature score vectors, which are then multiplied element-by-element with the original input vector to achieve feature weighting.
[0062] Next, the feature vectors of the two branches undergo a spatial reorganization operation. The core idea of this operation is to fuse the feature vectors of the two branches in a specific way by utilizing the feature vectors with rich information. Specifically, the spatial reorganization operation is implemented by inserting the two feature vectors according to the channel dimension, which can effectively improve the richness and expressiveness of the features without increasing the computational complexity.
[0063] It should be noted that splicing by channel can be a spatial reorganization operation, thereby spatially rearranging or combining features, which can be achieved by changing the size, position or channel order of the feature vector to improve the model's understanding of the spatial structure or characteristics of the input data.
[0064] This application processes the input data through a spatial feature transformation module so that the model can better capture the spatial structure and feature information in the input data, thereby improving its performance in subsequent tasks.
[0065] Optionally, in the user identity identification method provided in an embodiment of the present application, obtaining channel information of the input feature to obtain the second processed data includes: performing a one-dimensional convolution operation on the processed input feature to obtain second feature data, and inputting the second feature data into a normalization layer to obtain a third normalized feature vector, and inputting the third normalized feature vector into an activation function layer to obtain a third feature score vector; multiplying the third feature score vector and the processed input feature to obtain a weighted feature vector, and processing the weighted feature vector through a pooling layer to obtain a reduced dimensionality feature vector, and splicing the reduced dimensionality feature vector and the processed input feature according to the channel to obtain the second processed data.
[0066] Specifically, Figure 5 is a schematic diagram of a channel feature transformation module provided according to an embodiment of the present application, such as Figure 5 As shown,
[0067] It should be noted that in a convolutional neural network, each convolution layer outputs multiple feature maps (FeatureMaps), which are usually called channels. Each channel contains the response of the input data under a specific convolution kernel, that is, information about a certain feature. The design of the channel feature transformation module is to optimize the utilization and expression of channel feature information in neural networks. First, the module uses 1×1 convolution to reduce the dimension and transform the original features. This step reduces redundant information while retaining key features, providing a more refined and efficient feature representation for subsequent processing.
[0068] Furthermore, the transformed features will undergo layer normalization to stabilize the training process and improve the generalization ability of the model. The layer normalization operation normalizes all activation values of a single sample to ensure that they are in a similar scale range, thereby reducing the impact of scale differences between different samples on model training.
[0069] Furthermore, the features are mapped to the range of 0 to 1 through the Sigmoid function to obtain a series of channel weights W1 to Wn. These weights actually reflect the importance of different channel features in the current task. The larger the weight of the channel, the more critical the feature information it contains is to the current task. After obtaining the weights, these weights will be multiplied element by element with the original features to achieve the weighting of the feature map. This step is equivalent to selectively emphasizing and suppressing the original features, thereby further optimizing the expression of the features. Important feature channels are strengthened, while unimportant channels are weakened.
[0070] Furthermore, the module uses pooling operations to further process the weighted features. Pooling operations extract key feature information by selecting the maximum value from each non-overlapping area and reduce the dimension of the feature at the same time. This not only helps to reduce the amount of calculation, but also enables the model to focus on more abstract and global features.
[0071] Finally, the module adds the pooled features to the original features to achieve the fusion and reorganization of channel features. This fusion method not only retains the information of the original features, but also incorporates the feature information after weighting and pooling, making the output features richer and more comprehensive. In this way, the channel feature transformation module can significantly improve the utilization efficiency and expression ability of neural networks for feature information.
[0072] Optionally, in the user identity recognition method provided in an embodiment of the present application, the target recognition model is trained in the following manner: obtaining user information of multiple historical users, wherein the user information includes multiple groups of image information and multiple groups of audio information; using the user information as sample data to train an initial recognition model until the recognition accuracy of the initial recognition model is higher than a preset threshold, thereby obtaining a target recognition model.
[0073] Specifically, when training the target recognition model, user information of multiple historical users can be used, for example, multiple image information and multiple audio information of each user, and the user information of multiple historical users can be used as samples to train the initial recognition model, so that the initial recognition model learns the feature information in the image information and audio information in the sample data, and after the learning is completed, when the accuracy of the initial recognition model in identifying whether the input information is a user in the database based on the newly added input information is greater than a preset threshold, the training of the model is completed to obtain the target recognition model.
[0074] In order to ensure that the model can more accurately identify image information, optionally, in the user identity recognition method provided in the embodiment of the present application, a feature extraction operation is performed on the image information to obtain a first feature information set of the target user, including: determining whether the size of the image information meets the preset size; when the size of the image information is larger than the preset size, changing the size of the image information to the preset size, and normalizing the pixel values of the image information to obtain an initial image; when the size of the image information is smaller than the preset size, padding the image information, and normalizing the pixel values of the image information to obtain the initial image; and inputting the initial image into the convolutional neural network model for feature extraction to obtain the first feature information set.
[0075] Specifically, in order to ensure that the model can recognize image information more accurately, the image needs to be preprocessed. First, it can be determined whether the size of the image information meets the preset size, for example, checking whether its size meets the preset size (such as 224x224 pixels).
[0076] When the size of the image information is larger than the preset size, an image scaling algorithm (such as bilinear interpolation, bicubic interpolation, etc.) can be used to change the size of the image information to the preset size. Then, the scaled image information is normalized and the pixel values are scaled to the range of [0, 1] or [-1, 1] to obtain the initial image.
[0077] Furthermore, if the size of the image information is smaller than the preset size, padding can be performed to ensure that the image meets the requirements of the model in terms of size, thereby avoiding loss or deformation of image information due to size mismatch. For example, a preset padding method (such as edge padding, mean padding, etc.) is used to pad the image information to reach the preset size. Similarly, the padded image information is normalized to obtain the initial image.
[0078] It should be noted that normalization is to normalize the pixel values of image information, which can ensure that the pixel values are within a reasonable range and avoid some pixel values being too large or too small to have an adverse effect on model training. At the same time, normalization can also accelerate the model training process.
[0079] After image processing and normalization, you can also adjust the channel order of the image or perform image enhancement operations according to the requirements of the model. For example, for some models, the channel order of the image may be required to be RGB, while for other models, the channel order may be required to be BGR, or image enhancement operations such as random cropping, random rotation, random flipping, etc. may be performed to increase the diversity of the data and improve the generalization ability of the model.
[0080] Finally, the preprocessed image is input into the CNN convolutional neural network model. The CNN model processes the image information and outputs the image feature vector, which is the feature vector in the first feature information set.
[0081] In order to ensure that the model can more accurately identify audio information, optionally, in the user identity identification method provided in the embodiment of the present application, a feature extraction operation is performed on the audio information to obtain a second feature information set of the target user, including: preprocessing the audio information to obtain audio data, wherein the preprocessing includes at least one of the following: noise reduction, removal of silent audio, and audio information segmentation; performing a feature extraction operation on the audio information through a recurrent neural network model to obtain multiple initial feature information; and normalizing the multiple initial feature information to obtain a second feature information set.
[0082] Specifically, in order to ensure that the model can more accurately identify audio information, the audio information needs to be preprocessed. When preprocessing the audio information, appropriate noise reduction algorithms (such as spectral subtraction, Wiener filtering, etc.) can be used to reduce the noise of the audio information to reduce the interference of background noise. And by setting a threshold or using silence detection technology, identify and remove silent segments in the audio to reduce redundant data. And divide the audio information into multiple shorter audio segments according to needs to facilitate subsequent feature extraction and processing.
[0083] Furthermore, the preprocessed audio data is subjected to feature extraction. Common features include: Mel-frequency spectrum coefficients, Mel-frequency cepstral coefficients (MFCC), etc. These features can better represent the characteristic information of speech.
[0084] Furthermore, the extracted features are normalized and scaled to a fixed range to better meet the model's input requirements and improve the model's stability and convergence speed. Time series processing and audio data enhancement operations are also performed. For example, for time series data, it may be necessary to perform time series processing, such as cutting it into segments of fixed length, performing sequence padding, etc., to meet the model's input requirements. For some audio data, data enhancement operations can be performed, such as adding white noise, speed distortion, volume changes, etc., to increase data diversity and improve the model's generalization ability.
[0085] Furthermore, after completing the preprocessing of the audio information, the preprocessed audio information is input into the RNN recurrent neural network model, and the RNN model outputs feature vectors (i.e., the second feature information set). These feature vectors generally include information such as the energy, spectrum, and time domain characteristics of the audio, so that the user's identity is identified using the above feature vectors.
[0086] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0087] The embodiment of the present application also provides a user identification device. It should be noted that the user identification device of the embodiment of the present application can be used to execute the user identification method provided by the embodiment of the present application. The user identification device provided by the embodiment of the present application is introduced below.
[0088] Figure 6 Schematic diagram of a user identification device provided according to an embodiment of the present application. Figure 6As shown, the device includes: a first acquisition unit 61, a second acquisition unit 62, an input unit 63, and a determination unit 64.
[0089] The first acquisition unit 61 is used to acquire the image information of the target user through an image acquisition device, and perform a feature extraction operation on the image information to obtain a first feature information set of the target user.
[0090] The second acquisition unit 62 is used to acquire audio information of the target user through an audio acquisition device, and perform a feature extraction operation on the audio information to obtain a second feature information set of the target user.
[0091] The input unit 63 is used to input the first feature information set and the second feature information set into the target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module.
[0092] The determination unit 64 is used to determine the identity information of the target user according to the recognition result.
[0093] The user identity recognition device provided in the embodiment of the present application obtains the image information of the target user through the image acquisition device through the first acquisition unit 61, and performs a feature extraction operation on the image information to obtain a first feature information set of the target user; the second acquisition unit 62 obtains the audio information of the target user through the audio acquisition device, and performs a feature extraction operation on the audio information to obtain a second feature information set of the target user; the input unit 63 inputs the first feature information set and the second feature information set into the target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on the self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module; the determination unit 64 determines the identity information of the target user according to the recognition result. The problem of low security in user identity authentication through account name and password in related technologies is solved. By obtaining image information and audio information of the target user, and obtaining a first feature information set and a second feature information set according to the image information and audio information, respectively, and processing the feature vector from two dimensions of spatial features and channel features through a parallel convolution module in the target recognition model, the diversity and richness of the features are improved, thereby improving the performance and generalization ability of the target recognition model, and achieving the effect of improving the accuracy and security of user identity recognition.
[0094] Optionally, in the user identity recognition device provided in the embodiment of the present application, in the parallel convolution module, the convolution layer is a one-dimensional convolution layer, which is used to adjust the number of channels of the input features to obtain processed input features, and input the processed input features into the spatial feature transformation module and the channel feature transformation module respectively; the spatial feature transformation module is used to obtain the spatial information of the input features to obtain the first processed data; the channel feature transformation module is used to obtain the channel information of the input features to obtain the second processed data; the splicing unit is used to splice the first processed data and the second processed data according to the channel to obtain the output data of the parallel convolution module.
[0095] Optionally, in the user identity identification device provided in the embodiment of the present application, the spatial feature transformation module includes: a first convolution submodule, used to perform a two-dimensional convolution operation on the processed input features to obtain first feature data; a first input submodule, used to input the first feature data into a normalization layer to obtain a first normalized feature vector, and input the first normalized feature vector into an activation function layer to obtain a first feature score vector; a second input submodule, used to input the first feature data into the normalization layer to obtain a second normalized feature vector, and input the second normalized feature vector into the activation function layer to obtain a second feature score vector; a splicing submodule, used to splice the first feature score vector and the second feature score vector according to channels to obtain first processed data.
[0096] Optionally, in the user identity identification device provided in the embodiment of the present application, the channel feature transformation module includes: a second convolution submodule, used to perform a one-dimensional convolution operation on the processed input features to obtain second feature data, and input the second feature data into the normalization layer to obtain a third normalized feature vector, and input the third normalized feature vector into the activation function layer to obtain a third feature score vector; a pooling submodule, used to multiply the third feature score vector and the processed input features to obtain a weighted feature vector, and process the weighted feature vector through the pooling layer to obtain a reduced dimensionality feature vector, and splice the reduced dimensionality feature vector and the processed input features according to the channel to obtain second processed data.
[0097] Optionally, in the user identity identification device provided in the embodiment of the present application, the target recognition model is trained in the following manner: a third acquisition unit is used to obtain user information of multiple historical users, wherein the user information includes multiple groups of image information and multiple groups of audio information; a training unit is used to train an initial recognition model using the user information as sample data until the recognition accuracy of the initial recognition model is higher than a preset threshold, thereby obtaining a target recognition model.
[0098] Optionally, in the user identity recognition device provided in the embodiment of the present application, the first acquisition unit 61 includes: a judgment module, used to judge whether the size of the image information meets the preset size; a change module, used to change the size of the image information to the preset size when the size of the image information is larger than the preset size, and normalize the pixel values of the image information to obtain an initial image; a filling module, used to fill the image information when the size of the image information is smaller than the preset size, and normalize the pixel values of the image information to obtain the initial image; a first extraction module, used to input the initial image into a convolutional neural network model for feature extraction operation to obtain a first feature information set.
[0099] Optionally, in the user identity identification device provided in the embodiment of the present application, the second acquisition unit 62 includes: a preprocessing module, used to preprocess the audio information to obtain audio data, wherein the preprocessing includes at least one of the following: noise reduction, removal of silent audio, and audio information segmentation; a second extraction module, used to perform feature extraction operations on the audio information through a recurrent neural network model to obtain multiple initial feature information; a normalization module, used to perform normalization operations on multiple initial feature information to obtain a second feature information set.
[0100] The user identification device includes a processor and a memory. The first acquisition unit 61, the second acquisition unit 62, the input unit 63, the determination unit 64, etc. are all stored in the memory as program units. The processor executes the program units stored in the memory to implement corresponding functions.
[0101] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem of low security in authenticating user identities through account names and passwords in related technologies is solved by adjusting kernel parameters.
[0102] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0103] An embodiment of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the user identity recognition method is implemented.
[0104] An embodiment of the present invention provides a processor, which is used to run a program, wherein the user identity recognition method is executed when the program is running.
[0105] Figure 7 is a schematic diagram of an electronic device provided according to an embodiment of the present application, such as Figure 7 As shown, an embodiment of the present invention provides an electronic device, the electronic device 70 includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned user identification method when executing the program. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.
[0106] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes the steps of the above-mentioned user identification method.
[0107] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0108] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1A step that specifies a function in one or more boxes.
[0111] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0112] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0113] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0114] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0115] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.
Claims
1. A user identification method, characterized in that: include: Acquire image information of a target user through an image acquisition device, and perform a feature extraction operation on the image information to obtain a first feature information set of the target user; Acquire the audio information of the target user through an audio acquisition device, and perform a feature extraction operation on the audio information to obtain a second feature information set of the target user; Inputting the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module; The identity information of the target user is determined according to the recognition result.
2. The method according to claim 1, characterized in that In the parallel convolution module, the convolution layer is a one-dimensional convolution layer, which is used to adjust the number of channels of the input features to obtain processed input features, and input the processed input features into the spatial feature transformation module and the channel feature transformation module respectively; The spatial feature transformation module is used to obtain the spatial information of the input feature to obtain first processed data; The channel feature transformation module is used to obtain the channel information of the input feature to obtain the second processed data; The first processed data and the second processed data are concatenated according to channels to obtain output data of the parallel convolution module.
3. The method according to claim 2, characterized in that Acquiring the spatial information of the input feature to obtain the first processed data includes: Performing a two-dimensional convolution operation on the processed input features to obtain first feature data; Inputting the first feature data into a normalization layer to obtain a first normalized feature vector, and inputting the first normalized feature vector into an activation function layer to obtain a first feature score vector; Inputting the first feature data into the normalization layer to obtain a second normalized feature vector, and inputting the second normalized feature vector into the activation function layer to obtain a second feature score vector; The first feature score vector and the second feature score vector are concatenated according to channels to obtain the first processed data.
4. The method according to claim 2, characterized in that: Acquiring channel information of the input feature to obtain second processed data includes: Performing a one-dimensional convolution operation on the processed input features to obtain second feature data, inputting the second feature data into a normalization layer to obtain a third normalized feature vector, and inputting the third normalized feature vector into an activation function layer to obtain a third feature score vector; The third feature score vector is multiplied by the processed input feature to obtain a weighted feature vector, and the weighted feature vector is processed through a pooling layer to obtain a reduced-dimensional feature vector, and the reduced-dimensional feature vector and the processed input feature are concatenated according to channels to obtain the second processed data.
5. The method according to claim 1, characterized in that The target recognition model is trained in the following way: Acquire user information of multiple historical users, wherein the user information includes multiple sets of image information and multiple sets of audio information; The user information is used as sample data to train an initial recognition model until the recognition accuracy of the initial recognition model is higher than a preset threshold, thereby obtaining the target recognition model.
6. The method according to claim 1, characterized in that Performing a feature extraction operation on the image information to obtain a first feature information set of the target user includes: Determining whether the size of the image information meets a preset size; When the size of the image information is larger than the preset size, the size of the image information is changed to the preset size, and pixel values of the image information are normalized to obtain an initial image; When the size of the image information is smaller than the preset size, padding the image information and normalizing the pixel values of the image information to obtain the initial image; The initial image is input into a convolutional neural network model to perform feature extraction operations to obtain the first feature information set.
7. The method according to claim 1, characterized in that Performing a feature extraction operation on the audio information to obtain a second feature information set of the target user includes: Preprocessing the audio information to obtain audio data, wherein the preprocessing includes at least one of the following: noise reduction, removal of silent audio, and audio information segmentation; Performing a feature extraction operation on the audio information through a recurrent neural network model to obtain a plurality of initial feature information; A normalization operation is performed on the multiple initial feature information to obtain the second feature information set.
8. A user identity recognition device, characterized in that: include: A first acquisition unit, configured to acquire image information of a target user through an image acquisition device, and perform a feature extraction operation on the image information to obtain a first feature information set of the target user; A second acquisition unit, configured to acquire audio information of the target user through an audio acquisition device, and perform a feature extraction operation on the audio information to obtain a second feature information set of the target user; An input unit, used for inputting the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module; A determination unit is used to determine the identity information of the target user according to the recognition result.
9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the user identification method described in any one of claims 1 to 7 is implemented.
10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the user identity identification method described in any one of claims 1 to 7.
Citation Information
Patent Citations
Identity authentication method and device fusing face and voiceprint, equipment and storage medium
CN117852007A
Real person authentication method and device and electronic equipment
CN118260740A
Method and device for depth image completion
US20230245282A1
