User identity identification method, device, computer program product, and electronic device

Abstract

The present application discloses a user identity recognition method, device, computer program product and electronic device. It relates to the field of artificial intelligence. The method comprises: acquiring image information of a target user through an image acquisition device, and performing a feature extraction operation on the image information to obtain a first feature information set of the target user; acquiring audio information of the target user through an audio acquisition device, and performing a feature extraction operation on the audio information to obtain a second feature information set of the target user; inputting the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the encoder part in the target recognition model includes a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module; determining the identity information of the target user based on the recognition result. Through this application, the problem of low security of user identity authentication through account name and password in the related art is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a method, device, computer program product, and electronic device for user identity recognition. Background Art

[0002] As the number of users using digital banking services gradually increases, the business volume in digital banking services is also increasing. Currently, most of the business that can be handled in offline banks can also be handled in digital banks.

[0003] To ensure banking security, users must first authenticate their identity before using digital banking services. Only after successful authentication can they proceed with subsequent transactions. Currently, users typically authenticate using a username and password. While this simple and straightforward method of entering a pre-registered username and password carries the risk of password theft or guessing, potentially compromising the user's financial security.

[0004] Currently, no effective solution has been proposed to address the low security issue of user identity authentication through account names and passwords in related technologies. Summary of the Invention

[0005] The present application provides a user identity identification method, device, computer program product and electronic device to solve the problem of low security in authenticating user identity through account name and password in related technologies.

[0006] According to one aspect of the present application, a user identity recognition method is provided. The method includes: acquiring image information of a target user through an image acquisition device and performing a feature extraction operation on the image information to obtain a first feature information set of the target user; acquiring audio information of the target user through an audio acquisition device and performing a feature extraction operation on the audio information to obtain a second feature information set of the target user; inputting the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder portion of the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module, and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module, and a channel feature transformation module; and determining the identity information of the target user based on the recognition result.

[0007] Optionally, in the parallel convolution module, the convolution layer is a one-dimensional convolution layer, which is used to adjust the number of channels of the input features to obtain processed input features, and input the processed input features into the spatial feature transformation module and the channel feature transformation module respectively; the spatial feature transformation module is used to obtain the spatial information of the input features to obtain first processed data; the channel feature transformation module is used to obtain the channel information of the input features to obtain second processed data; the first processed data and the second processed data are spliced ​​according to the channels to obtain the output data of the parallel convolution module.

[0008] Optionally, obtaining spatial information of the input features to obtain first processed data includes: performing a two-dimensional convolution operation on the processed input features to obtain first feature data; inputting the first feature data into a normalization layer to obtain a first normalized feature vector, and inputting the first normalized feature vector into an activation function layer to obtain a first feature score vector; inputting the first feature data into a normalization layer to obtain a second normalized feature vector, and inputting the second normalized feature vector into the activation function layer to obtain a second feature score vector; and splicing the first feature score vector and the second feature score vector according to channels to obtain the first processed data.

[0009] Optionally, obtaining channel information of the input feature to obtain the second processed data includes: performing a one-dimensional convolution operation on the processed input feature to obtain second feature data, and inputting the second feature data into a normalization layer to obtain a third normalized feature vector, and inputting the third normalized feature vector into an activation function layer to obtain a third feature score vector; multiplying the third feature score vector and the processed input feature to obtain a weighted feature vector, and processing the weighted feature vector through a pooling layer to obtain a reduced dimensionality feature vector, and splicing the reduced dimensionality feature vector and the processed input feature according to the channel to obtain the second processed data.

[0010] Optionally, the target recognition model is trained in the following manner: obtaining user information of multiple historical users, wherein the user information includes multiple sets of image information and multiple sets of audio information; using the user information as sample data to train the initial recognition model until the recognition accuracy of the initial recognition model is higher than a preset threshold, thereby obtaining the target recognition model.

[0011] Optionally, performing a feature extraction operation on the image information to obtain a first feature information set of the target user includes: determining whether the size of the image information meets a preset size; if the size of the image information is larger than the preset size, changing the size of the image information to the preset size, and normalizing the pixel values ​​of the image information to obtain an initial image; if the size of the image information is smaller than the preset size, padding the image information, and normalizing the pixel values ​​of the image information to obtain an initial image; inputting the initial image into a convolutional neural network model for a feature extraction operation to obtain a first feature information set.

[0012] Optionally, performing a feature extraction operation on the audio information to obtain a second feature information set of the target user includes: preprocessing the audio information to obtain audio data, wherein the preprocessing includes at least one of the following: noise reduction, removal of silent audio, and audio information segmentation; performing a feature extraction operation on the audio information through a recurrent neural network model to obtain multiple initial feature information; and performing a normalization operation on the multiple initial feature information to obtain a second feature information set.

[0013] According to another aspect of the present application, a user identity recognition device is provided. The device includes: a first acquisition unit for acquiring image information of a target user through an image acquisition device and performing a feature extraction operation on the image information to obtain a first feature information set of the target user; a second acquisition unit for acquiring audio information of the target user through an audio acquisition device and performing a feature extraction operation on the audio information to obtain a second feature information set of the target user; an input unit for inputting the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part of the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module, and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module, and a channel feature transformation module; and a determination unit for determining the identity information of the target user based on the recognition result.

[0014] According to another aspect of the present invention, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the user identity recognition method in the present application is implemented.

[0015] According to another aspect of the present invention, an electronic device is provided, comprising one or more processors and a memory; the memory stores computer-readable instructions, and the processor is used to execute the computer-readable instructions, wherein the computer-readable instructions execute a user identity identification method when executed.

[0016] This application adopts the following steps: acquiring the target user's image information through an image acquisition device and performing a feature extraction operation on the image information to obtain a first feature information set of the target user; acquiring the target user's audio information through an audio acquisition device and performing a feature extraction operation on the audio information to obtain a second feature information set of the target user; inputting the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part of the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module, and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module, and a channel feature transformation module; and determining the target user's identity information based on the recognition result. This solves the problem of low security of user identity authentication through account name and password in related technologies. By obtaining the image information and audio information of the target user, and obtaining the first feature information set and the second feature information set respectively based on the image information and audio information, and processing the feature vector from the two dimensions of spatial features and channel features through the parallel convolution module in the target recognition model, the diversity and richness of the features are improved, thereby improving the performance and generalization ability of the target recognition model, and achieving the effect of improving the accuracy and security of user identity recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0018] Figure 1 is a flowchart of a user identity identification method provided according to an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of an encoder portion provided according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a parallel convolution module provided according to an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a spatial feature transformation module provided according to an embodiment of the present application;

[0022] Figure 5 Schematic diagram of a channel feature transformation module provided according to an embodiment of the present application;

[0023] Figure 6 is a schematic diagram of a user identity recognition device provided according to an embodiment of the present application;

[0024] Figure 7This is a schematic diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] It should be noted that the user identity identification method, device, computer program product and electronic device determined in the present disclosure can be used in the field of artificial intelligence, and can also be used in any field other than the field of artificial intelligence. The application field of the user identity identification method, device, computer program product and electronic device determined in the present disclosure is not limited.

[0029] It should be noted that the collected information, user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) used in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize use or refuse use. For example, an interface is set up between this system and relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0030] For ease of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0031] RNN model: Recurrent Neural Network model, which is used to process sequence data, has memory function, can capture the temporal dependencies in sequence data, and is suitable for tasks such as natural language processing, speech recognition, and time series prediction.

[0032] CNN model: Convolutional Neural Network model is a deep learning model mainly used to process and recognize image data.

[0033] Neural network model based on self-attention mechanism: Transformer model. Transformer is a neural network architecture based on attention mechanism, originally used for natural language processing tasks such as machine translation and text generation. The Transformer model uses the self-attention mechanism to capture the dependencies between positions in the input sequence without the need for recurrent neural networks or convolutional neural networks. The Transformer model consists of an encoder and a decoder, each of which contains multiple layers, and each layer consists of a multi-head attention mechanism and a feedforward neural network. During the training process, the Transformer model optimizes the model parameters by minimizing the loss function in order to generate correct prediction results at the output.

[0034] 2D convolution: In image processing or computer vision, 2D convolution is a mathematical operation that performs a weighted sum on each pixel of an image (or feature map). This operation is often used to extract spatial features from an image.

[0035] Normalization layer: Normalization layer is commonly used in deep learning models to adjust the output of neurons to a specific range or have a specific distribution, which helps to accelerate model training and improve the generalization ability of the model.

[0036] Sigmoid activation function: The Sigmoid function is a logical function that maps any real value to an output between 0 and 1. In neural networks, it is often used to convert the output of a neuron into a probability value or to control the flow of information.

[0037] MaxPooling: Max pooling is a downsampling operation used to reduce the dimensionality of feature maps. It creates a new, smaller feature map by selecting the maximum value from each non-overlapping region of the input feature map.

[0038] According to an embodiment of the present application, a user identity identification method is provided.

[0039] Figure 1 This is a flow chart of a user identification method according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0040] Step S101: acquiring image information of a target user through an image acquisition device, and performing a feature extraction operation on the image information to obtain a first feature information set of the target user.

[0041] Specifically, in order to perform identity recognition operations on users without requiring the users to enter their account and password, the accuracy of the user's identity can be determined by performing facial recognition on the user.

[0042] Furthermore, image information of the target user can be obtained through an image acquisition device, for example, by taking a photo of the target user's face with a camera to obtain image information. After obtaining the image information, the image information can be pre-processed, such as image enhancement, resizing, and normalization, to ensure that the image input to the model has a consistent format and size. The image is then input into a CNN convolutional neural network model, which processes the image information and outputs an image feature vector, i.e., a feature vector in the first feature information set.

[0043] Step S102: Acquire audio information of the target user through an audio acquisition device, and perform a feature extraction operation on the audio information to obtain a second feature information set of the target user.

[0044] Specifically, in order to perform identity identification operations on users without requiring the users to enter their account and password, it is also possible to determine whether the user's identity is accurate by identifying the user's audio information. For example, the target user uses the terminal device to read aloud fixed sentences in the terminal device, and the user's identity is identified based on the voice information collected by the terminal device.

[0045] After obtaining the audio information, the voice signal in the audio information can be converted into initial data and input into the RNN recurrent neural network model. The RNN model outputs feature vectors (that is, the second feature information set). These feature vectors usually include information such as the energy, spectrum, and time domain characteristics of the audio, so that the user's identity can be identified using the above feature vectors.

[0046] In step S103, the first feature information set and the second feature information set are input into the target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on the self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module.

[0047] Specifically, to improve the accuracy of user identification, the first feature information derived from image information and the second feature information derived from audio information can be input into the target recognition model, where they are interactively fused to learn the correlation between the two modalities. For example, the model can learn the consistency and complementarity between voice and image, thereby improving the accuracy and reliability of identity authentication.

[0048] It should be noted that the model used in this application for processing feature information is a neural network model based on the self-attention mechanism, that is, the Transformer model. However, in order to enable the model to capture information of different scales at the same time, thereby enhancing the diversity and richness of the model's features, a parallel convolution module can be added to the encoder part of the conventional Transformer model. Figure 2 is a schematic diagram of an encoder portion provided according to an embodiment of the present application, such as Figure 2As shown, in the conventional Transformer model, the encoder part is composed of multiple encoders, each of which is composed of a multi-head attention mechanism and a feedforward neural network. In this application, a parallel convolution module is added after the multi-head attention mechanism module, thereby further enhancing the expressive power of the Transformer model and the ability to capture local features in the model input information through the parallel convolution module. Among them, in the encoder of the Transformer, the parallel convolution module can extract local features in the input sequence through the convolution kernel (Convolution Kernel), which may be ignored or not fully represented in the multi-head attention mechanism, and by adding the parallel convolution module, the model can capture global and local information at the same time, thereby improving its expressive power and ability to process complex sequence data, and the parallel convolution module allows the Transformer model to perform parallel calculations, which can significantly improve the computational efficiency of the model. After the multi-head attention mechanism, the parallel convolution module can process input data at multiple positions at the same time, thereby speeding up the training and reasoning speed of the model.

[0049] That is, in the encoder of the Transformer model, the role of adding a parallel convolution module after the multi-head attention mechanism module is to enhance the model's ability to extract local features, supplement local information that may be ignored by the global attention mechanism, and improve the computational efficiency of the model through parallel computing, so that the Transformer model can capture global and local information at the same time when processing sequence data, thereby improving its expressiveness and performance.

[0050] Step S104: determining the identity information of the target user according to the recognition result.

[0051] Specifically, after the model outputs the recognition result, the user identity can be determined based on the recognition result. If the user identity authentication is successful, the digital banking system can allow the user to perform corresponding transactions or query operations. If the identity authentication fails, the user's operation request will be rejected, thereby ensuring the security of the user's assets in the digital banking system.

[0052] The user identity recognition method provided in an embodiment of the present application obtains image information of a target user through an image acquisition device and performs a feature extraction operation on the image information to obtain a first feature information set of the target user; obtains audio information of the target user through an audio acquisition device and performs a feature extraction operation on the audio information to obtain a second feature information set of the target user; inputs the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part of the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module, and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module, and a channel feature transformation module; and determines the identity information of the target user based on the recognition result. This solves the problem of low security of user identity authentication through account name and password in the related art. By obtaining the image information and audio information of the target user, and obtaining the first feature information set and the second feature information set respectively based on the image information and audio information, and processing the feature vector from the two dimensions of spatial features and channel features through the parallel convolution module in the target recognition model, the diversity and richness of the features are improved, thereby improving the performance and generalization ability of the target recognition model, and achieving the effect of improving the accuracy and security of user identity recognition.

[0053] Optionally, in the user identity recognition method provided in an embodiment of the present application, in the parallel convolution module, the convolution layer is a one-dimensional convolution layer, which is used to adjust the number of channels of the input features to obtain processed input features, and input the processed input features into the spatial feature transformation module and the channel feature transformation module respectively; the spatial feature transformation module is used to obtain the spatial information of the input features to obtain first processed data; the channel feature transformation module is used to obtain the channel information of the input features to obtain second processed data; the first processed data and the second processed data are spliced ​​according to the channels to obtain the output data of the parallel convolution module.

[0054] It's important to note that in convolutional neural networks, spatial feature transformation modules primarily focus on transforming information within the spatial dimensions (height and width) of an image or feature map. These modules extract or enhance spatial structural information within an image, such as edges and textures, through specific convolutional kernel operations. Channel-wise feature transformation modules, on the other hand, focus on transforming information within the channel dimension (depth or number of features) of a feature map. In convolutional neural networks, each channel typically represents a specific feature, such as color or shape. Channel-wise feature transformation modules enhance or suppress certain features by adjusting the weights or combinations between different channels, thereby optimizing the model's processing of different types of information.

[0055] Specifically, Figure 3 is a schematic diagram of a parallel convolution module provided according to an embodiment of the present application, such as Figure 3 As shown in the figure, the main structures in the parallel convolution module are the spatial feature transformation module and the channel feature transformation module. When the input feature enters the parallel convolution module, it first passes through a 1×1 convolution layer to adjust the number of channels to meet the needs of the subsequent two parallel branches. Among them, 1×1 convolution is also called point convolution, which is mainly used to change the number of channels (depth) of the feature map. Through the 1×1 convolution kernel, each position of the feature map can be linearly transformed without changing the spatial size of the feature map, thereby adjusting the number of channels or achieving cross-channel information fusion.

[0056] Furthermore, the adjusted feature map is divided into two paths, one entering the spatial feature transformation module and the other entering the channel feature transformation module. The spatial feature transformation module focuses on the spatial dimension of the image and uses the convolution kernel to capture and enhance the spatial structural information such as edges and textures in the image, while the channel feature transformation module focuses on the channel dimension of the feature map and enhances or suppresses certain features by adjusting the weights or combinations between different channels, thereby optimizing the model's processing of different types of information.

[0057] Finally, the first processed data and the second processed data are spliced ​​according to channels to obtain the output data of the parallel convolution module, thereby completing the processing operation of the feature information.

[0058] It should be noted that when splicing the output features of the spatial feature transformation module and the channel feature transformation module, the channel-by-channel splicing method can be used. For example, assuming that the output of the spatial feature transformation module is a feature vector A and the output of the channel feature transformation module is a feature vector B, where the dimension of A is [height, width, channels_A] and the dimension of B is [height, width, channels_B]. When the two feature maps are merged through channels, the dimension of the output feature C will be [height, width, channels_A+channels_B], thereby completing the feature splicing and forming a richer and more diverse feature representation.

[0059] In this embodiment, by setting up a parallel convolution module, not only the computational efficiency of the model is improved, but also the diversity and richness of the model's features are enhanced, which helps to improve the performance and generalization ability of the model.

[0060] Optionally, in the user identity recognition method provided in the embodiment of the present application, obtaining the spatial information of the input features and obtaining the first processed data includes: performing a two-dimensional convolution operation on the processed input features to obtain first feature data; inputting the first feature data into the normalization layer to obtain a first normalized feature vector, and inputting the first normalized feature vector into the activation function layer to obtain a first feature score vector; inputting the first feature data into the normalization layer to obtain a second normalized feature vector, and inputting the second normalized feature vector into the activation function layer to obtain a second feature score vector; and splicing the first feature score vector and the second feature score vector according to the channel to obtain the first processed data.

[0061] Specifically, Figure 4 is a schematic diagram of a spatial feature transformation module provided according to an embodiment of the present application, such as Figure 4 As shown in the figure, in the spatial feature reconstruction module, the input data is first divided into two parallel branches through a 2D convolution operation. These two branches independently extract features from the data. The feature maps of each branch are adjusted through a normalization layer to output normalized feature vectors W and Y. These feature vectors are then nonlinearly transformed using the Sigmoid activation function to generate feature score vectors. These score vectors are then element-wise multiplied with the original input vector to achieve feature weighting.

[0062] Next, the feature vectors of the two branches undergo a spatial recombination operation. The core concept of this operation is to combine the information-rich feature vectors of the two branches in a specific way. Specifically, the spatial recombination operation is implemented by interpolating the two feature vectors along the channel dimension. This effectively improves the richness and expressiveness of features without increasing computational complexity.

[0063] It should be noted that splicing by channel can be a spatial reorganization operation, which can spatially rearrange or combine features. This can be achieved by changing the size, position or channel order of the feature vector to improve the model's understanding of the spatial structure or characteristics of the input data.

[0064] This application processes the input data through a spatial feature transformation module, so that the model can better capture the spatial structure and feature information in the input data, thereby improving its performance in subsequent tasks.

[0065] Optionally, in the user identity identification method provided in an embodiment of the present application, obtaining channel information of the input feature and obtaining the second processed data includes: performing a one-dimensional convolution operation on the processed input feature to obtain second feature data, and inputting the second feature data into a normalization layer to obtain a third normalized feature vector, and inputting the third normalized feature vector into an activation function layer to obtain a third feature score vector; multiplying the third feature score vector and the processed input feature to obtain a weighted feature vector, and processing the weighted feature vector through a pooling layer to obtain a reduced dimensionality feature vector, and splicing the reduced dimensionality feature vector and the processed input feature according to the channel to obtain the second processed data.

[0066] Specifically, Figure 5 is a schematic diagram of a channel feature transformation module provided according to an embodiment of the present application, such as Figure 5 As shown,

[0067] It's important to note that in convolutional neural networks, each convolutional layer outputs multiple feature maps, often referred to as channels. Each channel contains the response of the input data to a specific convolution kernel, i.e., information about a specific feature. The channel feature transformation module is designed to optimize the utilization and representation of channel feature information in neural networks. First, the module uses 1×1 convolution to reduce the dimensionality and transform the original features. This step reduces redundant information while retaining key features, providing a more refined and efficient feature representation for subsequent processing.

[0068] Furthermore, the transformed features undergo layer normalization to stabilize the training process and improve the model's generalization capabilities. Layer normalization normalizes all activation values ​​of a single sample to ensure they are within a similar scale range, thereby reducing the impact of scale differences between different samples on model training.

[0069] Furthermore, the features are mapped to a range of 0 to 1 using a Sigmoid function, resulting in a series of channel weights W1 to Wn. These weights reflect the importance of different channel features in the current task. The larger the weight, the more critical the feature information contained in the channel is to the current task. After the weights are obtained, they are element-wise multiplied with the original features to achieve a weighted feature map. This step is equivalent to selectively emphasizing and suppressing the original features, further optimizing the feature representation. Important feature channels are strengthened, while less important channels are weakened.

[0070] Furthermore, the module applies pooling to further process the weighted features. Pooling extracts key feature information by selecting the maximum value from each non-overlapping region, while simultaneously reducing the feature dimension. This not only helps reduce computational effort but also enables the model to focus on more abstract and global features.

[0071] Finally, the module adds the pooled features to the original features, achieving the fusion and reorganization of channel features. This fusion method preserves the original feature information while incorporating the weighted and pooled feature information, making the output features richer and more comprehensive. In this way, the channel feature transformation module can significantly improve the efficiency and expressiveness of neural networks in utilizing feature information.

[0072] Optionally, in the user identity recognition method provided in an embodiment of the present application, the target recognition model is trained in the following manner: obtaining user information of multiple historical users, wherein the user information includes multiple sets of image information and multiple sets of audio information; using the user information as sample data to train the initial recognition model until the recognition accuracy of the initial recognition model is higher than a preset threshold, thereby obtaining the target recognition model.

[0073] Specifically, when training the target recognition model, user information of multiple historical users can be used, for example, multiple image information and multiple audio information of each user, and the user information of multiple historical users can be used as samples to train the initial recognition model, so that the initial recognition model learns the feature information in the image information and audio information in the sample data. After the learning is completed, when the accuracy of the initial recognition model in identifying whether the input information is from a user in the database based on the newly added input information is greater than a preset threshold, the training of the model is completed to obtain the target recognition model.

[0074] In order to ensure that the model can more accurately identify image information, optionally, in the user identity recognition method provided in the embodiment of the present application, a feature extraction operation is performed on the image information to obtain a first feature information set of the target user, including: determining whether the size of the image information meets the preset size; when the size of the image information is larger than the preset size, changing the size of the image information to the preset size, and normalizing the pixel values ​​of the image information to obtain an initial image; when the size of the image information is smaller than the preset size, padding the image information, and normalizing the pixel values ​​of the image information to obtain an initial image; and inputting the initial image into the convolutional neural network model for feature extraction operation to obtain a first feature information set.

[0075] Specifically, in order to ensure that the model can recognize image information more accurately, the image needs to be preprocessed. First, it can be determined whether the size of the image information meets the preset size, for example, checking whether its size meets the preset size (such as 224x224 pixels).

[0076] When the size of the image information is larger than the preset size, an image scaling algorithm (such as bilinear interpolation, bicubic interpolation, etc.) can be used to change the size of the image information to the preset size. Then, the scaled image information is normalized and the pixel values ​​are scaled to the range of [0, 1] or [-1, 1] to obtain the initial image.

[0077] Furthermore, if the image size is smaller than the preset size, padding can be performed to ensure that the image size meets the model requirements, thereby avoiding image information loss or deformation caused by size mismatch. For example, a preset padding method (such as edge padding or mean padding) can be used to pad the image information to the preset size. Similarly, the padded image information is normalized to obtain the initial image.

[0078] It should be noted that normalization is the process of normalizing the pixel values ​​of image information. This ensures that pixel values ​​are within a reasonable range, preventing excessively large or small pixel values ​​from adversely affecting model training. Normalization also accelerates the model training process.

[0079] After image processing and normalization, you can adjust the image channel order or perform image enhancement operations based on the model's requirements. For example, some models may require the image channel order to be RGB, while others may require it to be BGR. Alternatively, you can perform image enhancement operations such as random cropping, random rotation, and random flipping to increase data diversity and improve the model's generalization ability.

[0080] Finally, the preprocessed image is input into the CNN convolutional neural network model. The CNN model processes the image information and outputs the image feature vector, which is the feature vector in the first feature information set.

[0081] In order to ensure that the model can more accurately identify audio information, optionally, in the user identity identification method provided in the embodiment of the present application, a feature extraction operation is performed on the audio information to obtain a second feature information set of the target user, including: preprocessing the audio information to obtain audio data, wherein the preprocessing includes at least one of the following: noise reduction, removal of silent audio, and audio information segmentation; performing a feature extraction operation on the audio information through a recurrent neural network model to obtain multiple initial feature information; and performing a normalization operation on the multiple initial feature information to obtain a second feature information set.

[0082] Specifically, to ensure the model can more accurately identify audio information, the audio information needs to be preprocessed. During preprocessing, appropriate noise reduction algorithms (such as spectral subtraction and Wiener filtering) can be used to reduce background noise interference. Silent segments in the audio can be identified and removed by setting thresholds or using silence detection technology to reduce redundant data. Furthermore, the audio information can be divided into multiple shorter audio segments as needed to facilitate subsequent feature extraction and processing.

[0083] Furthermore, the preprocessed audio data is subjected to feature extraction. Common features include Mel-spectral coefficients and Mel-Frequency Cepstral Coefficients (MFCC), which can better represent the characteristic information of speech.

[0084] Furthermore, the extracted features are normalized and scaled to a fixed range to better meet the model's input requirements and improve the model's stability and convergence speed. Time series processing and audio data enhancement operations are also performed. For example, for time series data, time series processing may be required, such as splitting it into fixed-length segments and performing sequence padding to meet the model's input requirements. For some audio data, data enhancement operations such as adding white noise, speed distortion, and volume changes can be performed to increase data diversity and improve the model's generalization ability.

[0085] Furthermore, after completing the preprocessing of the audio information, the preprocessed audio information is input into the RNN recurrent neural network model, and the RNN model outputs feature vectors (i.e., the second feature information set). These feature vectors generally include information such as the energy, spectrum, and time domain characteristics of the audio, so that the user identity is identified using the above feature vectors.

[0086] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0087] The present application also provides a user identification device. It should be noted that the user identification device of the present application can be used to execute the user identification method provided in the present application. The user identification device provided in the present application is introduced below.

[0088] Figure 6 Schematic diagram of a user identification device according to an embodiment of the present application. Figure 6As shown, the device includes: a first acquiring unit 61 , a second acquiring unit 62 , an input unit 63 , and a determining unit 64 .

[0089] The first acquiring unit 61 is configured to acquire image information of a target user through an image acquisition device, and perform a feature extraction operation on the image information to obtain a first feature information set of the target user.

[0090] The second acquiring unit 62 is configured to acquire audio information of the target user through an audio acquisition device, and perform a feature extraction operation on the audio information to obtain a second feature information set of the target user.

[0091] The input unit 63 is used to input the first feature information set and the second feature information set into the target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on the self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module.

[0092] The determining unit 64 is configured to determine the identity information of the target user according to the recognition result.

[0093] The user identity recognition device provided in the embodiment of the present application obtains the image information of the target user through the image acquisition device through the first acquisition unit 61, and performs a feature extraction operation on the image information to obtain a first feature information set of the target user; the second acquisition unit 62 obtains the audio information of the target user through the audio acquisition device, and performs a feature extraction operation on the audio information to obtain a second feature information set of the target user; the input unit 63 inputs the first feature information set and the second feature information set into the target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on the self-attention mechanism, and the encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module and a channel feature transformation module; the determination unit 64 determines the identity information of the target user according to the recognition result. The problem of low security in authenticating user identities through account names and passwords in related technologies is solved. By obtaining the image information and audio information of the target user, and obtaining a first feature information set and a second feature information set based on the image information and audio information respectively, and processing the feature vector from two dimensions of spatial features and channel features through the parallel convolution module in the target recognition model, the diversity and richness of the features are improved, thereby improving the performance and generalization ability of the target recognition model, and achieving the effect of improving the accuracy and security of user identity recognition.

[0094] Optionally, in the user identity recognition device provided in the embodiment of the present application, in the parallel convolution module, the convolution layer is a one-dimensional convolution layer, which is used to adjust the number of channels of the input features to obtain processed input features, and input the processed input features into the spatial feature transformation module and the channel feature transformation module respectively; the spatial feature transformation module is used to obtain the spatial information of the input features to obtain first processed data; the channel feature transformation module is used to obtain the channel information of the input features to obtain second processed data; the splicing unit is used to splice the first processed data and the second processed data according to the channels to obtain the output data of the parallel convolution module.

[0095] Optionally, in the user identity recognition device provided in the embodiment of the present application, the spatial feature transformation module includes: a first convolution submodule, used to perform a two-dimensional convolution operation on the processed input features to obtain first feature data; a first input submodule, used to input the first feature data into the normalization layer to obtain a first normalized feature vector, and input the first normalized feature vector into the activation function layer to obtain a first feature score vector; a second input submodule, used to input the first feature data into the normalization layer to obtain a second normalized feature vector, and input the second normalized feature vector into the activation function layer to obtain a second feature score vector; a splicing submodule, used to splice the first feature score vector and the second feature score vector according to channels to obtain first processed data.

[0096] Optionally, in the user identity identification device provided in the embodiment of the present application, the channel feature transformation module includes: a second convolution submodule, used to perform a one-dimensional convolution operation on the processed input features to obtain second feature data, and input the second feature data into the normalization layer to obtain a third normalized feature vector, and input the third normalized feature vector into the activation function layer to obtain a third feature score vector; a pooling submodule, used to multiply the third feature score vector and the processed input features to obtain a weighted feature vector, and process the weighted feature vector through the pooling layer to obtain a reduced dimensionality feature vector, and splice the reduced dimensionality feature vector and the processed input features according to the channel to obtain second processed data.

[0097] Optionally, in the user identity identification device provided in the embodiment of the present application, the target recognition model is trained in the following manner: a third acquisition unit is used to obtain user information of multiple historical users, wherein the user information includes multiple groups of image information and multiple groups of audio information; a training unit is used to train the initial recognition model using the user information as sample data until the recognition accuracy of the initial recognition model is higher than a preset threshold, thereby obtaining a target recognition model.

[0098] Optionally, in the user identity recognition device provided in the embodiment of the present application, the first acquisition unit 61 includes: a judgment module, used to judge whether the size of the image information meets the preset size; a change module, used to change the size of the image information to the preset size when the size of the image information is larger than the preset size, and normalize the pixel values ​​of the image information to obtain an initial image; a filling module, used to fill the image information when the size of the image information is smaller than the preset size, and normalize the pixel values ​​of the image information to obtain the initial image; a first extraction module, used to input the initial image into the convolutional neural network model for feature extraction operation to obtain a first feature information set.

[0099] Optionally, in the user identity identification device provided in the embodiment of the present application, the second acquisition unit 62 includes: a preprocessing module, used to preprocess the audio information to obtain audio data, wherein the preprocessing includes at least one of the following: noise reduction, removal of silent audio, and audio information segmentation; a second extraction module, used to perform feature extraction operations on the audio information through a recurrent neural network model to obtain multiple initial feature information; a normalization module, used to perform normalization operations on multiple initial feature information to obtain a second feature information set.

[0100] The above-mentioned user identity recognition device includes a processor and a memory. The above-mentioned first acquisition unit 61, second acquisition unit 62, input unit 63, determination unit 64, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0101] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be configured, and adjusting kernel parameters solves the low security issue of user identity authentication using account names and passwords in related technologies.

[0102] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0103] An embodiment of the present invention provides a computer-readable storage medium having a program stored thereon, which implements the user identity recognition method when executed by a processor.

[0104] An embodiment of the present invention provides a processor, which is used to run a program, wherein the user identity recognition method is executed when the program is run.

[0105] Figure 7 is a schematic diagram of an electronic device provided according to an embodiment of the present application, such as Figure 7 As shown, an embodiment of the present invention provides an electronic device. The electronic device 70 includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of the user identification method described above are implemented. The device herein may be a server, a PC, a PAD, a mobile phone, or the like.

[0106] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes the steps of the above-mentioned user identification method.

[0107] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0108] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1A step that specifies a function in one or more boxes.

[0111] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0112] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0113] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0114] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0115] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A user identification method, characterized in that: include: Acquire image information of a target user through an image acquisition device, and perform a feature extraction operation on the image information to obtain a first feature information set of the target user; Acquire audio information of the target user through an audio acquisition device, and perform a feature extraction operation on the audio information to obtain a second feature information set of the target user; Inputting the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and an encoder part in the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module, and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module, and a channel feature transformation module; Determining the identity information of the target user according to the recognition result; In which, in the parallel convolution module, the convolution layer is a one-dimensional convolution layer, which is used to adjust the number of channels of the input features to obtain processed input features, and input the processed input features into the spatial feature transformation module and the channel feature transformation module respectively; the spatial feature transformation module is used to obtain the spatial information of the input features to obtain first processed data; the channel feature transformation module is used to obtain the channel information of the input features to obtain second processed data; the first processed data and the second processed data are spliced ​​according to channels to obtain the output data of the parallel convolution module.

2. The method according to claim 1, characterized in that Acquiring spatial information of the input feature to obtain first processed data includes: Performing a two-dimensional convolution operation on the processed input features to obtain first feature data; Inputting the first feature data into a normalization layer to obtain a first normalized feature vector, and inputting the first normalized feature vector into an activation function layer to obtain a first feature score vector; Inputting the first feature data into the normalization layer to obtain a second normalized feature vector, and inputting the second normalized feature vector into the activation function layer to obtain a second feature score vector; The first feature score vector and the second feature score vector are concatenated according to channels to obtain the first processed data.

3. The method according to claim 1, characterized in that Acquiring channel information of the input feature to obtain second processed data includes: Performing a one-dimensional convolution operation on the processed input features to obtain second feature data, inputting the second feature data into a normalization layer to obtain a third normalized feature vector, and inputting the third normalized feature vector into an activation function layer to obtain a third feature score vector; Multiplying the third feature score vector and the processed input feature to obtain a weighted feature vector, processing the weighted feature vector through a pooling layer to obtain a reduced dimensionality feature vector, and concatenating the reduced dimensionality feature vector and the processed input feature according to channels to obtain the second processed data.

4. The method according to claim 1, wherein The target recognition model is trained in the following way: Acquire user information of multiple historical users, wherein the user information includes multiple sets of image information and multiple sets of audio information; The user information is used as sample data to train an initial recognition model until the recognition accuracy of the initial recognition model is higher than a preset threshold, thereby obtaining the target recognition model.

5. The method according to claim 1, wherein Performing a feature extraction operation on the image information to obtain a first feature information set of the target user includes: Determining whether the size of the image information meets a preset size; When the size of the image information is larger than the preset size, the size of the image information is changed to the preset size, and pixel values ​​of the image information are normalized to obtain an initial image; When the size of the image information is smaller than the preset size, padding the image information and normalizing the pixel values ​​of the image information to obtain the initial image; The initial image is input into a convolutional neural network model to perform feature extraction operations to obtain the first feature information set.

6. The method according to claim 1, characterized in that Performing a feature extraction operation on the audio information to obtain a second feature information set of the target user includes: Preprocessing the audio information to obtain audio data, wherein the preprocessing includes at least one of the following: noise reduction, removal of silent audio, and audio information segmentation; Performing a feature extraction operation on the audio information through a recurrent neural network model to obtain a plurality of initial feature information; A normalization operation is performed on the multiple initial feature information to obtain the second feature information set.

7. A user identification device, characterized in that: include: A first acquiring unit is configured to acquire image information of a target user through an image acquisition device, and perform a feature extraction operation on the image information to obtain a first feature information set of the target user; A second acquiring unit is configured to acquire audio information of the target user through an audio acquisition device, and perform a feature extraction operation on the audio information to obtain a second feature information set of the target user; an input unit, configured to input the first feature information set and the second feature information set into a target recognition model to obtain a recognition result, wherein the target recognition model is a neural network model based on a self-attention mechanism, and the encoder part of the target recognition model is composed of a feedforward neural network module, a multi-head self-attention mechanism module, and a parallel convolution module, and the parallel convolution module is composed of a convolution layer, a spatial feature transformation module, and a channel feature transformation module; a determination unit, configured to determine the identity information of the target user according to the recognition result; In which, in the parallel convolution module, the convolution layer is a one-dimensional convolution layer, which is used to adjust the number of channels of the input features to obtain processed input features, and input the processed input features into the spatial feature transformation module and the channel feature transformation module respectively; the spatial feature transformation module is used to obtain the spatial information of the input features to obtain first processed data; the channel feature transformation module is used to obtain the channel information of the input features to obtain second processed data; the first processed data and the second processed data are spliced ​​according to channels to obtain the output data of the parallel convolution module.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the user identification method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: The method comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the user identity identification method described in any one of claims 1 to 6.

Images