User identity recognition system, method and related device

Abstract

The invention discloses a user identity recognition system and method and a related device, and relates to the field of image recognition, and the method comprises the steps that a data collection module obtains a monitoring image including hand actions of a target user and bank card information of the target user, extracts pre-stored hand action features and pre-stored hand biological features corresponding to the bank card information, and transmits the pre-stored hand action features and the pre-stored hand biological features to a server; outputting an image segmentation feature by using a preset residual module in a preset prediction branch, outputting a hand image feature map based on the image segmentation feature by using a preset feature fusion module, and processing the hand image feature map by using a preset attention module to obtain an identity recognition associated feature map; and the identity recognition module outputs an identity recognition result based on a pre-stored hand action feature, a pre-stored hand biological feature, and a consistency comparison result of the hand action feature and the hand biological feature, and extracts a hand action feature and a hand biological feature of the target user from the pre-stored hand action feature and the pre-stored hand biological feature. According to the invention, the ATM identifies the user identity based on the image identification technology.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a user identity recognition system, method and related devices. Background Art

[0002] In the current financial transaction scenario of automated teller machines (ATMs), fingerprints are commonly used to identify users. However, when the fingerprint collection device is contaminated due to frequent use, it is impossible to accurately collect fingerprints, which in turn affects the accuracy of user identification.

[0003] With the development of image recognition technology, researchers in this field have proposed the idea of ​​using monitoring equipment to collect hand images of users in specific postures, and using image recognition algorithms to perform feature recognition of hand images to complete identity recognition. However, since the external light in the ATM scene has a great impact on the quality of hand images, the image recognition algorithm needs to have a higher recognition accuracy, but the higher recognition accuracy makes the image recognition algorithm generate higher computing power requirements, and ATM machines cannot provide the computing power requirements of high-precision image recognition algorithms. Therefore, how to realize user identity recognition based on image recognition technology in the ATM scene has become an urgent problem to be solved. Summary of the invention

[0004] In view of the above problems, this application provides a user identity recognition system, method and related devices to achieve the purpose of user identity recognition based on image recognition technology in ATM machine scenarios. The specific solution is as follows:

[0005] The first aspect of the present application provides a user identity recognition system, which is applied to an ATM machine. The user identity recognition system includes:

[0006] A data acquisition module, used to obtain a monitoring image including a target user's hand movements and the target user's bank card information, and extract pre-stored hand movement features and pre-stored hand biometric features corresponding to the bank card information from a database;

[0007] A preset YOLACT instance segmentation model is used to output image segmentation features of a hand image region in the monitoring image based on an input monitoring image using a preset residual module in a preset prediction branch, and to output a hand image feature map of the target user based on the image segmentation features using a preset feature fusion module;

[0008] A GhostNetV2 lightweight visual model is preset to perform information retention and weight assignment processing on the hand image feature map using a preset attention module to obtain an identity recognition associated feature map, and to extract hand motion features and hand biometric features of the target user from the identity recognition associated feature map;

[0009] The identity recognition module is used to output the identity recognition result of the target user based on the pre-stored hand motion features, the pre-stored hand biometric features, and the consistency comparison result of the hand motion features and the hand biometric features.

[0010] In a possible implementation, the preset YOLACT instance segmentation model includes: a basic network, a feature pyramid network, a prototype generation branch, the prediction branch and the feature fusion module. The preset YOLACT instance segmentation model is set to: when using a preset residual module in a preset prediction branch to output image segmentation features of a hand image area in the monitoring image based on an input monitoring image, and using a preset feature fusion module to output a hand image feature map of the target user based on the image segmentation features:

[0011] The basic network is used to adjust the resolution of the monitoring image and output initial images of multiple resolutions;

[0012] The feature pyramid network is used to perform initial image fusion based on the initial images of each resolution to obtain a basic feature map;

[0013] The prototype generation branch is used to extract features from the basic feature map to obtain an initial feature map;

[0014] The prediction branch is used to extract features from the basic feature map using the preset residual module to obtain the image segmentation features;

[0015] The feature fusion module is used to perform feature fusion on the image segmentation features and the initial feature map to obtain the hand image feature map.

[0016] In a possible implementation, the preset residual module includes a first convolutional layer and a second convolutional layer connected in sequence, and the prediction branch is used to extract features from the basic feature map using the preset residual module to obtain the image segmentation feature, including:

[0017] The prediction branch inputs the basic feature map into the preset residual module to determine whether the gradient value calculated by the preset residual module based on the number of channels of the basic feature map is within a preset gradient interval, and if so, performs the feature extraction on the basic feature map to obtain the image segmentation feature;

[0018] When the gradient value is not within the preset gradient range, the basic feature map is input into the first convolution layer, so that the first convolution layer and the second convolution layer adjust the number of channels of the basic feature map to obtain an adjusted basic feature map, and perform the feature extraction on the adjusted basic feature map to obtain the image segmentation feature.

[0019] In a possible implementation, when the feature fusion module performs feature fusion on the image segmentation feature and the initial feature map to obtain the hand image feature map, the feature fusion module is set to:

[0020] The mask weight coefficients in the image segmentation features are linearly weighted with the channels associated with the feature categories in the initial feature map to generate the hand image feature map.

[0021] In a possible implementation, the preset attention module includes: a preprocessing layer, an average pooling layer, a maximum pooling layer, a feature fusion layer, a splicing layer, a convolution layer and an output layer. The preset GhostNetV2 lightweight visual model is set to: when the preset attention module is used to perform information retention and weight assignment processing on the hand image feature map to obtain the identity recognition associated feature map:

[0022] The preprocessing layer is used to perform information retention processing on the hand image feature map to obtain an initial identity recognition associated feature map after the information retention processing;

[0023] The average pooling layer is used to perform average pooling processing on the initial identity recognition associated feature map to obtain a first feature map;

[0024] The maximum pooling layer is used to perform maximum pooling processing on the initial identity recognition associated feature map to obtain a second feature map;

[0025] The feature fusion layer is used to perform weighted fusion on the first feature and the second feature based on a preset weight value to obtain a mixed feature map;

[0026] The splicing layer is used to splice the first feature map, the second feature map and the mixed feature map to obtain an intermediate feature map;

[0027] The convolution layer is used to perform a standard convolution operation and a dilated convolution operation on the intermediate feature map, and perform a weighted sum operation on the obtained standard convolution operation result features and the dilated convolution operation result features to obtain an identity recognition associated feature set;

[0028] The output layer is used to multiply the identity recognition associated feature set by the initial identity recognition associated feature map to obtain the identity recognition associated feature map.

[0029] In a possible implementation, the preprocessing layer includes: a global pooling layer, a fully connected layer, an activation function, a product layer, and a summation layer. When the preprocessing layer performs information retention processing on the hand image feature map to obtain an initial identity recognition associated feature map after information retention processing, the preprocessing layer is set to:

[0030] Using the global pooling layer, the fully connected layer and the activation function to extract features from the hand image feature map, to obtain an initial identity recognition associated feature set;

[0031] The product layer is used to obtain the product feature of the hand image feature map and the initial identity recognition associated feature set;

[0032] The summation layer is used to superimpose the product feature with the hand image feature map to obtain the initial identity recognition associated feature map.

[0033] A second aspect of the present application provides a user identity recognition method, which is applied to the user identity recognition system described in the first aspect and any one of its implementations. The user identity recognition method includes:

[0034] Obtaining a monitoring image including a target user's hand motion and the target user's bank card information, and extracting pre-stored hand motion features and pre-stored hand biometric features corresponding to the bank card information from a database;

[0035] The preset YOLACT instance segmentation model uses a preset residual module in a preset prediction branch to output image segmentation features of a hand image region in the monitoring image based on an input monitoring image, and uses a preset feature fusion module to output a hand image feature map of the target user based on the image segmentation features;

[0036] The preset GhostNetV2 lightweight visual model uses a preset attention module to perform information retention and weight assignment processing on the hand image feature map to obtain an identity recognition associated feature map, and extracts the hand motion features and hand biometric features of the target user from the identity recognition associated feature map;

[0037] The identity recognition result of the target user is output based on the consistency comparison result of the pre-stored hand motion feature, the pre-stored hand biometric feature, the hand motion feature and the hand biometric feature.

[0038] A third aspect of the present application provides a computer program product, comprising computer-readable instructions, which, when executed on an electronic device, enables the electronic device to implement the user identity identification method of the second aspect.

[0039] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0040] The memory is used to store computer programs;

[0041] The processor is used to execute the computer program so that the electronic device can implement the user identity recognition method of the second aspect mentioned above.

[0042] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the user identity identification method of the second aspect mentioned above.

[0043] By means of the above technical scheme, the user identity recognition system, method and related device provided by the present application output the image segmentation features of the hand image area in the monitoring image by configuring the preset YOLACT instance segmentation model and using the preset residual module in the preset prediction branch. The residual module can avoid the characteristic of gradient descent, thereby improving the accuracy of the output image segmentation features. At the same time, by configuring the preset YOLACT instance segmentation model and using the preset feature fusion module to output the hand image feature map of the target user based on the image segmentation features, the rough extraction of the hand image features is realized. Subsequently, by configuring the preset GhostNetV2 lightweight visual model and using the preset attention module to retain information and assign weights to the identity recognition associated features in the hand image feature map, the accuracy of the extracted hand motion features and hand biometric features is improved. Finally, by configuring the above preset GhostNetV2 lightweight visual model for the extraction of hand motion features and hand biometric features, the low computing power requirement of the lightweight model is utilized to realize the recognition of user identity based on image recognition technology under the ATM machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0045] Figure 1 A block diagram of a user identification system provided for this application;

[0046] Figure 2 A schematic diagram of the structure of a preset YOLACT instance segmentation model provided for this application;

[0047] Figure 3 A schematic diagram of the structure of a preset residual module provided in this application;

[0048] Figure 4 A schematic diagram of a process for generating a hand image feature map by a Fusion module provided in this application;

[0049] Figure 5 A comparison chart of a mask generation strategy provided for this application;

[0050] Figure 6 A schematic diagram of the structure of a preset attention module provided in this application;

[0051] Figure 7 A schematic diagram of a process for obtaining an initial identity recognition associated feature map after information retention processing at a preprocessing layer provided in the present application;

[0052] Figure 8 A flowchart of a user identification method provided for this application;

[0053] Fig. 9 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0054] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0055] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0056] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0057] The first aspect of the present application provides a user identity recognition system, which is applied to an ATM machine, such as Figure 1 As shown, the user identification system includes:

[0058] The data acquisition module 101 is used to obtain monitoring images including the hand movements of the target user and the bank card information of the target user, and extract pre-stored hand movement features and pre-stored hand biometric features corresponding to the bank card information from the database.

[0059] It should be noted that, in actual application scenarios, there are multiple implementation methods for obtaining the monitoring image including the target user's hand movements and the target user's bank card information. An exemplary implementation is provided herein, including the following steps A1 to A4.

[0060] Step A1: The data acquisition module reads the bank card information in response to the card insertion signal received by the ATM card insertion module, and triggers step A2.

[0061] In step A2, the data acquisition module accesses the backend database according to the bank card information, obtains the pre-stored hand motion features and pre-stored hand biometric features corresponding to the bank card information, and triggers step A3.

[0062] In step A3, the data acquisition module analyzes and predicts the hand movement features, and sends the display movement instruction generated based on the analysis result to the ATM display module, so that the ATM display module displays the movement in the display movement instruction, and triggers step A4.

[0063] In a possible implementation, the display action instruction of step A3 stores a schematic diagram of the above action. The ATM display module guides the target user to perform an action similar to the displayed action by displaying the above action, and prompts the target user that if the above action is not completed within the countdown, the service will be terminated.

[0064] Step A4: the data acquisition module sends a screenshot instruction to the monitoring device, so that the monitoring device takes a screenshot of the monitoring video in response to the screenshot instruction, and feeds the obtained screenshot back to the data acquisition module.

[0065] In a possible implementation, the screenshot instruction in the above step A4 may be generated by the data acquisition module when the display interface enters a preset display countdown interval, thereby reserving time for the user to perform a display action.

[0066] The preset YOLACT instance segmentation model 102 is used to output image segmentation features of the hand image area in the monitoring image based on the input monitoring image using the preset residual module in the preset prediction branch, and to output a hand image feature map of the target user based on the image segmentation features using the preset feature fusion module.

[0067] It should be noted that in actual application scenarios, the above-mentioned preset YOLACT instance segmentation model is a model obtained by the researchers of this application after adding the above-mentioned preset residual module to the prediction branch of the original model on the basis of the existing YOLACT (You Only Look At CoefficienTs) instance segmentation model. Because the external light in the ATM machine scene will cause more interference features in the monitoring image, it is very easy to cause the YOLACT instance segmentation model to have a decrease in segmentation accuracy due to excessive gradient descent during the image segmentation process. However, this application configures a preset YOLACT instance segmentation model to use the preset residual module in the preset prediction branch to output the image segmentation features of the hand image area in the monitoring image. The residual module can avoid the characteristics of gradient descent, thereby improving the accuracy of the output image segmentation features.

[0068] It should be noted that in actual application scenarios, the above-mentioned image segmentation features include bounding box information, image feature categories and hand boundaries. Among them, the above-mentioned bounding box information can be implemented by the detection (Regression, Reg) task in the above-mentioned preset YOLACT instance segmentation model. The above-mentioned image feature category can be implemented by the category prediction (Class Prediction, cls) task in the above-mentioned preset YOLACT instance segmentation model. The above-mentioned hand boundary can be implemented by the segmentation (Mask) task in the above-mentioned preset YOLACT instance segmentation model. The detection (Regression, Reg) task, category prediction (Class Prediction, cls) task and segmentation (Mask) task in the above-mentioned preset YOLACT instance segmentation model are the same as the existing YOLACT instance segmentation model.

[0069] The preset GhostNetV2 lightweight visual model 103 is used to perform information retention and weight assignment processing on the hand image feature map using the preset attention module, obtain the identity recognition associated feature map, and extract the hand motion features and hand biometric features of the target user from the identity recognition associated feature map.

[0070] It should be noted that in actual application scenarios, the above-mentioned preset GhostNetV2 lightweight visual model is based on the existing GhostNetV2 lightweight visual model, and the researchers of this application replace the original SE channel attention module with the preset attention module provided by this application. Since the original SE channel attention module compresses the channel of the feature image and then expands it to achieve feature extraction, the channel information will be lost due to compression in this process, resulting in a decrease in the accuracy of the final feature obtained. However, this application configures the preset attention module to perform information retention and weight assignment processing on the hand image feature map to obtain an identity recognition associated feature map, thereby avoiding channel information loss, and assigning weights based on the degree of association of different features with hand motion features or hand biometrics, thereby improving the generation accuracy of the identity recognition associated feature map, and thereby improving the accuracy of the hand motion features and hand biometrics finally obtained. At the same time, since the lightweight visual model has the characteristic of low computing power requirements, this application configures the above-mentioned preset GhostNetV2 lightweight visual model for the extraction of hand movement features and hand biometric features, and utilizes the low computing power requirements of the lightweight model to realize the identification of user identity based on image recognition technology in the ATM machine.

[0071] The identity recognition module 104 is used to output the identity recognition result of the target user based on the pre-stored hand motion features, the pre-stored hand biometric features, and the consistency comparison result of the hand motion features and the hand biometric features.

[0072] It should be noted that, in actual application scenarios, the above-mentioned output of the target user's identity recognition result based on the pre-stored hand motion features, pre-stored hand biometrics, and the consistency comparison results of the hand motion features and hand biometrics can be implemented in the following manner: when the consistency comparison results of the pre-stored hand motion features and the hand motion features are passed, and the consistency comparison results of the pre-stored hand biometrics and the hand biometrics are passed, the output content is the identity recognition result of passed identity recognition. For all other situations except the above, the output content is the identity recognition result of failed identity recognition.

[0073] This application configures a preset YOLACT instance segmentation model and uses a preset residual module in a preset prediction branch to output the image segmentation features of the hand image area in the surveillance image. The residual module can avoid the characteristic of gradient descent, thereby improving the accuracy of the output image segmentation features. At the same time, by configuring the preset YOLACT instance segmentation model, a preset feature fusion module is used to output the hand image feature map of the target user based on the image segmentation features, thereby achieving a rough extraction of the hand image features. Subsequently, by configuring the preset GhostNetV2 lightweight visual model, a preset attention module is used to maintain information and assign weights to the identity recognition associated features in the hand image feature map, thereby improving the accuracy of the extracted hand motion features and hand biometric features. Finally, by configuring the above-mentioned preset GhostNetV2 lightweight visual model for the extraction of hand motion features and hand biometric features, the low computing power requirements of the lightweight model are utilized to achieve the identification of user identities based on image recognition technology under the ATM machine.

[0074] In a possible implementation, the preset YOLACT instance segmentation model includes: a basic network, a feature pyramid network, a prototype generation branch, a prediction branch and a feature fusion module. The preset YOLACT instance segmentation model is set to: when outputting image segmentation features of a hand image area in a monitoring image based on an input monitoring image using a preset residual module in a preset prediction branch, and outputting a hand image feature map of a target user based on the image segmentation features using a preset feature fusion module:

[0075] The basic network is used to adjust the resolution of surveillance images and output initial images of multiple resolutions;

[0076] The feature pyramid network is used to perform initial image fusion based on the initial images of each resolution to obtain the basic feature map;

[0077] The prototype generation branch is used to extract features from the basic feature map to obtain the initial feature map;

[0078] The prediction branch is used to extract features from the basic feature map using a preset residual module to obtain image segmentation features;

[0079] The feature fusion module is used to fuse the image segmentation features with the initial feature map to obtain the hand image feature map.

[0080] It should be noted that in actual application scenarios, the structural diagram of the above preset YOLACT instance segmentation model can be as follows: Figure 2As shown: In addition to the above-mentioned basic network 201, feature pyramid network 202, prototype generation branch 203, prediction branch 204 and feature fusion module (Fusion) 205, it also includes a non-maximum suppression module 206 and a cropping & binarization module 207. Among them, except for the two residual modules involved in the prediction branch 204, all are structures of the existing YOLACT instance segmentation model. C1 is a monitoring image, C2 to C5 are initial images of various resolutions output by the basic network 201 based on the monitoring image C1. P5 is C5, P7 and P6 are basic feature maps generated by the feature pyramid network 202 based on P5 to further improve the resolution, and P4 and P3 are basic feature maps generated by the feature pyramid network 202 by fusing P5 with C4 and C3 through the initial image. For the prediction branch 204, the box on the left side of the preset residual module is the basic feature map, and the boxes on the right side of the preset residual module are the detection (Regression, Reg) task, the category prediction (Class Prediction, cls) task and the segmentation (Mask) task.

[0081] In a possible implementation, the preset residual module includes a first convolutional layer and a second convolutional layer connected in sequence, and the prediction branch is used to extract features from the basic feature map using the preset residual module to obtain image segmentation features, including:

[0082] The prediction branch inputs the basic feature map into the preset residual module to determine whether the gradient value calculated by the preset residual module based on the number of channels of the basic feature map is within the preset gradient range. If so, feature extraction is performed on the basic feature map to obtain image segmentation features.

[0083] When the gradient value is not within the preset gradient range, the basic feature map is input into the first convolution layer, so that the first convolution layer and the second convolution layer adjust the number of channels of the basic feature map to obtain an adjusted basic feature map, and feature extraction is performed on the adjusted basic feature map to obtain image segmentation features.

[0084] It should be noted that in actual application scenarios, the structural diagram of the preset residual module is as follows: Figure 3 As shown, the first convolution layer and the second convolution layer are both 3×3 convolution layers. When the gradient value is within the preset gradient range, the basic feature map will enter the feature extraction process along the W2 branch. When the gradient value is not within the preset gradient range, the basic feature map will be input into the first convolution layer, so that the first and second convolution layers adjust the number of channels of the basic feature map, and enter the feature extraction process through the W1 branch.

[0085] It should be noted that in actual application scenarios, the essence of adjusting the number of channels of the basic feature map using the preset convolution layer is to change the mask weight coefficient of the basic feature map, so as to find a balance between complete coupling and complete decoupling of the prediction branch and improve the accuracy of image segmentation features.

[0086] In a possible implementation, the feature fusion module is set to:

[0087] The mask weight coefficients in the image segmentation features are linearly weighted with the channels associated with the feature categories in the initial feature map to generate a hand image feature map.

[0088] It should be noted that, in actual application scenarios, the above mask weight coefficients may be collected by a class prediction (ClassPrediction, cls) task.

[0089] It should be noted that in actual application scenarios, the feature fusion module can be a Fusion module. The Fusion module uses the mask weight coefficient and the initial feature map to perform linear weighting along the channel associated with the feature category to obtain a significant value, and generates a hand image feature map based on the significant value, thereby replacing the single-channel linear weighted summation process of the mask coefficient and the prototype feature in the existing YOLACT network to improve the quality of the generated instance mask, thereby improving the generation accuracy of the hand image feature map.

[0090] It should be noted that in actual application scenarios, the calculation formula of the above Fusion module is: Among them, m is the instance mask corresponding to the hand bounding box in the hand image feature map, i represents the category, C is the number of categories in the image segmentation feature, j represents the number of channels, k is the number of channels in the initial feature map, b is the feature in the initial feature map, a is the mask weight coefficient, d represents the anchor box, and p is the number of anchor boxes in the image segmentation feature after non-maximum suppression processing.

[0091] It should be noted that the flow chart of the above-mentioned Fusion module generating the hand image feature map is as follows: Figure 4 As shown: Extract each feature b from the initial feature map d ij , extract each mask weight coefficient a from the image segmentation feature d ij , the Fusion module is used to obtain the instance mask corresponding to each hand bounding box according to the above calculation formula, and the hand image feature map is generated based on the instance mask.

[0092] It should be noted that, compared with the existing image segmentation algorithms, the preset YOLACT instance segmentation model provided by the present application improves the image segmentation quality without introducing additional computational complexity. Specifically, BlendMask first condenses the prototype feature map through region of interest pooling (ROI pooling), then changes the mask coefficient from the scalar form in YOLACT to the form of a two-dimensional feature map, and finally superimposes the two pixel by pixel to improve the quality of the generated mask. However, due to the introduction of the ROI pooling operation, the computational cost of the algorithm itself is relatively high. SipMask divides the bounding box in the prototype feature map into multiple sub-regions, and then the network predicts different spatial coefficients for each sub-region, and finally multiplies the pixels of each sub-region in the prototype feature map by the corresponding spatial coefficient and then sums them to obtain the final mask. Although this method further mines the spatial information of the bounding box on the prototype feature map and achieves a certain accuracy improvement in the dense target segmentation scene, it is not very applicable to the identity recognition task of the ATM transaction scene where the target distribution is relatively sparse. However, the present application does not require ROI pooling and regional division, and will not generate additional computational complexity. In order to facilitate the understanding of the advantages of the preset YOLACT instance segmentation model provided in this application compared with BlendMask, SipMask and the existing YOLACT, a comparison chart of the mask generation strategies of the above models is provided, as shown in Figure 5 .

[0093] In a possible implementation, the preset attention module includes: a preprocessing layer, an average pooling layer, a maximum pooling layer, a feature fusion layer, a splicing layer, a convolution layer and an output layer. The preset GhostNetV2 lightweight visual model is set to: when the preset attention module is used to perform information retention and weight assignment processing on the hand image feature map to obtain the identity recognition associated feature map:

[0094] The preprocessing layer is used to perform information preservation processing on the hand image feature map to obtain an initial identity recognition associated feature map after information preservation processing;

[0095] The average pooling layer is used to perform average pooling processing on the initial identity recognition associated feature map to obtain a first feature map;

[0096] The maximum pooling layer is used to perform maximum pooling processing on the initial identity recognition associated feature map to obtain a second feature map;

[0097] The feature fusion layer is used to perform weighted fusion on the first feature and the second feature based on a preset weight value to obtain a mixed feature map;

[0098] The concatenation layer is used to concatenate the first feature map, the second feature map and the mixed feature map to obtain an intermediate feature map;

[0099] The convolution layer is used to perform standard convolution operations and dilated convolution operations on the intermediate feature maps, and perform weighted sum operations on the obtained standard convolution operation result features and dilated convolution operation result features to obtain a set of identity recognition related features;

[0100] The output layer is used to multiply the identity recognition associated feature set with the initial identity recognition associated feature map to obtain the identity recognition associated feature map.

[0101] It should be noted that in actual application scenarios, the structural diagram of the above preset attention module can be as follows: Figure 6 As shown:

[0102] The preprocessing layer 601 sends the initial identity recognition associated feature map after the information preservation process to the average pooling layer 602, the maximum pooling layer 603 and the output layer 607 respectively. The average pooling layer 602 sends the first feature map to the feature fusion layer 604 and the splicing layer 605 respectively, and the maximum pooling layer 603 sends the second feature map to the feature fusion layer 604 and the splicing layer 605 respectively. The feature fusion layer 604 sends the generated mixed feature map to the splicing layer 605. The splicing layer 605 sends the intermediate feature map to the convolution layer 606. The convolution layer 606 sends the identity recognition associated feature set to the output layer 607. The output layer 607 multiplies the identity recognition associated feature set with the initial identity recognition associated feature map to obtain the identity recognition associated feature map.

[0103] It should be noted that in actual application scenarios, a sigmoid activation function can also be configured in the above convolutional layer to convert the identity recognition associated feature set into a spatial attention score map to facilitate the subsequent generation of the identity recognition associated feature map.

[0104] In a possible implementation, the preprocessing layer includes: a global pooling layer, a fully connected layer, an activation function, a product layer, and a summation layer. The preprocessing layer is set to:

[0105] The global pooling layer, fully connected layer and activation function are used to extract the hand image feature map to obtain the initial identity recognition related feature set;

[0106] The product layer is used to obtain the product features of the hand image feature map and the initial identity recognition associated feature set;

[0107] The summation layer is used to superimpose the product features with the hand image feature map to obtain the initial identity recognition associated feature map.

[0108] It should be noted that in actual application scenarios, the flowchart of the preprocessing layer obtaining the initial identity recognition associated feature map after information retention processing can be as follows: Figure 7shown.

[0109] It should be noted that, by configuring the above-mentioned preprocessing layer, the present application achieves the goal of reducing the channel information loss caused by feature dimension compression while ensuring that the network focuses on the significant channels on the feature map, and by constructing a residual connection structure, it alleviates the problem of model performance degradation caused by the excessive depth of the network structure, improves the accuracy of the output initial identity recognition associated feature map, and thereby improves the accuracy of the final hand motion features and hand biometric features.

[0110] The second aspect of the present application provides a user identity recognition method, which is applied to the user identity recognition system of the first aspect and any implementation thereof, such as Figure 8 As shown, the user identity identification method includes:

[0111] S801, obtaining a monitoring image including a target user's hand motion and the target user's bank card information, and extracting pre-stored hand motion features and pre-stored hand biometric features corresponding to the bank card information from a database;

[0112] S802, the preset YOLACT instance segmentation model uses the preset residual module in the preset prediction branch to output the image segmentation features of the hand image area in the monitoring image based on the input monitoring image, and uses the preset feature fusion module to output the hand image feature map of the target user based on the image segmentation features;

[0113] S803, the preset GhostNetV2 lightweight visual model uses the preset attention module to perform information retention and weight assignment processing on the hand image feature map, obtains the identity recognition associated feature map, and extracts the hand motion features and hand biometric features of the target user from the identity recognition associated feature map;

[0114] S804: Output the target user's identity recognition result based on the pre-stored hand motion features, the pre-stored hand biometric features, and the consistency comparison result of the hand motion features and the hand biometric features.

[0115] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the user identity identification method of the second aspect.

[0116] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0117] The memory is used to store computer programs;

[0118] The processor is used to execute a computer program so that the electronic device can implement the user identity recognition method of the second aspect.

[0119] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the user identity identification method of the second aspect mentioned above.

[0120] The structural diagram of the electronic device provided in the third aspect of the present application is as follows Fig. 9 The electronic devices in the embodiments of the present application may include but are not limited to fixed terminals such as mobile phones, notebook computers, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, and the like. Fig. 9 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0121] like Fig. 9 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0122] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a memory card, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0123] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0124] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0125] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0126] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A user identification system, characterized in that: Applied to ATM machines, the user identification system comprises: A data acquisition module, used to obtain a monitoring image including a target user's hand movements and the target user's bank card information, and extract pre-stored hand movement features and pre-stored hand biometric features corresponding to the bank card information from a database; A preset YOLACT instance segmentation model is used to output image segmentation features of a hand image region in the monitoring image based on an input monitoring image using a preset residual module in a preset prediction branch, and to output a hand image feature map of the target user based on the image segmentation features using a preset feature fusion module; A GhostNetV2 lightweight visual model is preset to perform information retention and weight assignment processing on the hand image feature map using a preset attention module to obtain an identity recognition associated feature map, and to extract hand motion features and hand biometric features of the target user from the identity recognition associated feature map; The identity recognition module is used to output the identity recognition result of the target user based on the pre-stored hand motion features, the pre-stored hand biometric features, and the consistency comparison result of the hand motion features and the hand biometric features.

2. The user identification system according to claim 1, characterized in that: The preset YOLACT instance segmentation model includes: a basic network, a feature pyramid network, a prototype generation branch, the prediction branch and the feature fusion module. The preset YOLACT instance segmentation model is set to: when using a preset residual module in a preset prediction branch to output image segmentation features of a hand image area in the monitoring image based on an input monitoring image, and using a preset feature fusion module to output a hand image feature map of the target user based on the image segmentation features: The basic network is used to adjust the resolution of the monitoring image and output initial images of multiple resolutions; The feature pyramid network is used to perform initial image fusion based on the initial images of each resolution to obtain a basic feature map; The prototype generation branch is used to extract features from the basic feature map to obtain an initial feature map; The prediction branch is used to extract features from the basic feature map using the preset residual module to obtain the image segmentation features; The feature fusion module is used to perform feature fusion on the image segmentation features and the initial feature map to obtain the hand image feature map.

3. The user identification system according to claim 2, characterized in that: The preset residual module includes a first convolutional layer and a second convolutional layer connected in sequence, and the prediction branch is used to extract features from the basic feature map using the preset residual module to obtain the image segmentation features, including: The prediction branch inputs the basic feature map into the preset residual module to determine whether the gradient value calculated by the preset residual module based on the number of channels of the basic feature map is within a preset gradient interval, and if so, performs the feature extraction on the basic feature map to obtain the image segmentation feature; When the gradient value is not within the preset gradient range, the basic feature map is input into the first convolution layer, so that the first convolution layer and the second convolution layer adjust the number of channels of the basic feature map to obtain an adjusted basic feature map, and perform the feature extraction on the adjusted basic feature map to obtain the image segmentation feature.

4. The user identification system according to claim 2, characterized in that: When the feature fusion module performs feature fusion on the image segmentation feature and the initial feature map to obtain the hand image feature map, the feature fusion module is set to: The mask weight coefficients in the image segmentation features are linearly weighted with the channels associated with the feature categories in the initial feature map to generate the hand image feature map.

5. The user identification system according to claim 1, characterized in that: The preset attention module includes: a preprocessing layer, an average pooling layer, a maximum pooling layer, a feature fusion layer, a splicing layer, a convolution layer and an output layer. The preset GhostNetV2 lightweight visual model is set to: The preprocessing layer is used to perform information retention processing on the hand image feature map to obtain an initial identity recognition associated feature map after the information retention processing; The average pooling layer is used to perform average pooling processing on the initial identity recognition associated feature map to obtain a first feature map; The maximum pooling layer is used to perform maximum pooling processing on the initial identity recognition associated feature map to obtain a second feature map; The feature fusion layer is used to perform weighted fusion on the first feature and the second feature based on a preset weight value to obtain a mixed feature map; The splicing layer is used to splice the first feature map, the second feature map and the mixed feature map to obtain an intermediate feature map; The convolution layer is used to perform a standard convolution operation and a dilated convolution operation on the intermediate feature map, and perform a weighted sum operation on the obtained standard convolution operation result features and the dilated convolution operation result features to obtain an identity recognition associated feature set; The output layer is used to multiply the identity recognition associated feature set by the initial identity recognition associated feature map to obtain the identity recognition associated feature map.

6. The user identification system according to claim 5, characterized in that: The preprocessing layer includes: a global pooling layer, a fully connected layer, an activation function, a product layer and a summation layer. When the preprocessing layer performs information retention processing on the hand image feature map to obtain the initial identity recognition associated feature map after the information retention processing, the preprocessing layer is set to: Using the global pooling layer, the fully connected layer and the activation function to extract features from the hand image feature map, to obtain an initial identity recognition associated feature set; The product layer is used to obtain the product feature of the hand image feature map and the initial identity recognition associated feature set; The summation layer is used to superimpose the product feature with the hand image feature map to obtain the initial identity recognition associated feature map.

7. A user identification method, characterized in that: Applied to the user identification system according to claims 1 to 6, the user identification method comprises: Obtaining a monitoring image including a target user's hand motion and the target user's bank card information, and extracting pre-stored hand motion features and pre-stored hand biometric features corresponding to the bank card information from a database; The preset YOLACT instance segmentation model uses a preset residual module in a preset prediction branch to output image segmentation features of a hand image region in the monitoring image based on an input monitoring image, and uses a preset feature fusion module to output a hand image feature map of the target user based on the image segmentation features; The preset GhostNetV2 lightweight visual model uses a preset attention module to perform information retention and weight assignment processing on the hand image feature map to obtain an identity recognition associated feature map, and extracts the hand motion features and hand biometric features of the target user from the identity recognition associated feature map; The identity recognition result of the target user is output based on the consistency comparison result of the pre-stored hand motion feature, the pre-stored hand biometric feature, the hand motion feature and the hand biometric feature.

8. A computer program product, characterized in that It includes computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the user identity recognition method as claimed in claim 7.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the user identity recognition method as claimed in claim 7.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the user identity recognition method as claimed in claim 7.

Images