Feature extraction model processing method and apparatus, and feature extraction method and apparatus

Abstract

A feature extraction model processing method, comprising: acquiring a plurality of sample palm images of a sample palm, and determining a first sample image area comprising a preset part, and a second sample image area other than the first sample image area in each sample palm image (S202); mapping first sample embedding features of the first sample image areas to a key space to obtain key vectors (S204); mapping second sample embedding features of the second sample image areas to a query space to obtain query vectors (S206); performing self-attention prediction on the basis of the key vectors and the query vectors to obtain respective predicted self-attention weights of the first sample embedding features and the second sample embedding features (S208); fusing the first sample embedding features and the second sample embedding features according to the respective predicted self-attention weights to obtain predicted palm features (S210); and performing model training on the basis of predicted differences between the predicted palm features and tag palm features to obtain a feature extraction model (S212).

Description

Feature extraction model processing method, device and feature extraction method, device

[0001] Related applications

[0002] This application claims priority to Chinese patent application number 2023109319666, filed on July 27, 2023, entitled “Feature Extraction Model Processing Method, Device and Feature Extraction Method, Device,” the entire text of which is hereby incorporated by reference. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a feature extraction model processing method, apparatus, computer equipment, storage medium and computer program product, as well as a feature extraction method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0004] With the development of computer technology, feature extraction technology has emerged. Feature extraction technology is widely used in fields such as text translation, image recognition, and identity verification. For example, feature extraction is performed on palm images and used to perform identity verification.

[0005] Traditional feature extraction technology uses a neural network to extract palmprint features from a single palm image. If the number of extracted palmprint features is small, different features are extracted from other palm images. In other words, for the same region in multiple palm images, only the features of one region are extracted, while the same regions in the other palm images are ignored.

[0006] However, there may be detail differences between the same regions in multiple palm images. Directly ignoring the same regions in other palm images may lead to inaccurate extraction of detailed palmprint features, thereby affecting subsequent related applications of the extracted palmprint features.

[0007] Summary of the Invention

[0008] According to various embodiments provided in the present application, a feature extraction model processing method, apparatus, computer device, computer-readable storage medium and computer program product, as well as a feature extraction processing method, apparatus, computer device, computer-readable storage medium and computer program product are provided.

[0009] In one aspect, the present application provides a feature extraction model processing method, which is executed by a computer device, and the method includes:

[0010] Acquire multiple sample palm images of a sample palm, and determine a first sample image region and a second sample image region other than the first sample image region in each of the sample palm images, wherein each of the first sample image regions includes a preset portion of the sample palm;

[0011] Obtaining first sample embedding features of respective first sample image regions, and mapping each first sample embedding feature to a key space to obtain a key vector of each first sample embedding feature;

[0012] Obtaining a second sample embedding feature of each second sample image region, and mapping each second sample embedding feature to a query space to obtain a query vector for each second sample embedding feature;

[0013] Performing self-attention prediction based on each of the key vectors and each of the query vectors to obtain a predicted self-attention weight corresponding to each of the first sample embedding features and a predicted self-attention weight corresponding to each of the second sample embedding features;

[0014] Fusing each of the first sample embedded features and each of the second sample embedded features according to their respective predicted self-attention weights to obtain a predicted palm feature corresponding to the multiple sample palm images; and

[0015] The labeled palm features corresponding to the multiple sample palm images are obtained, and a model is trained based on the predicted difference between the predicted palm features and the labeled palm features to obtain a feature extraction model.

[0016] On the other hand, the present application also provides a feature extraction model processing device, the device comprising:

[0017] a sample acquisition module, configured to acquire a plurality of sample palm images of a sample palm, and determine a first sample image region and a second sample image region other than the first sample image region in each of the sample palm images, wherein each of the first sample image regions includes a preset portion of the sample palm;

[0018] a key space mapping module, configured to obtain a first sample embedding feature of each of the first sample image regions, and map each of the first sample embedding features to a key space to obtain a key vector of each of the first sample embedding features;

[0019] a query space mapping module, configured to obtain a second sample embedding feature of each second sample image region, and map each second sample embedding feature to a query space to obtain a query vector for each second sample embedding feature;

[0020] a weight prediction module, configured to perform self-attention prediction based on each of the key vectors and each of the query vectors, and obtain a predicted self-attention weight corresponding to each of the first sample embedding features and a predicted self-attention weight corresponding to each of the second sample embedding features;

[0021] a sample feature fusion module, configured to fuse each of the first sample embedded features and each of the second sample embedded features according to their respective predicted self-attention weights to obtain a predicted palm feature corresponding to the plurality of sample palm images; and

[0022] The training module is used to obtain the label palm features corresponding to the multiple sample palm images, perform model training based on the predicted differences between the predicted palm features and the label palm features, and obtain a feature extraction model.

[0023] On the other hand, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the method embodiments of the present application when executing the computer program.

[0024] On the other hand, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the various method embodiments of the present application are performed.

[0025] On the other hand, the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it performs the steps of the various method embodiments of the present application.

[0026] In another aspect, the present application provides a feature extraction method, performed by a computer device, comprising:

[0027] acquiring a plurality of palm images of the palm;

[0028] The feature extraction model obtained by training the feature extraction model processing method determines a first image region and a second image region other than the first image region in each of the palm images, wherein each of the first image regions includes a preset portion of the palm;

[0029] Obtaining, by means of the feature extraction model, a first embedding feature of each of the first image regions and a second embedding feature of each of the second image regions; and

[0030] Through the feature extraction model, each first embedded feature and each second embedded feature are fused according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model to obtain the palm features corresponding to the multiple palm images, wherein the self-attention weight is obtained through training of the feature extraction model.

[0031] On the other hand, the present application also provides a feature extraction device, comprising:

[0032] An image acquisition module, for acquiring multiple palm images of the palm;

[0033] a region determination module, configured to obtain the feature extraction model trained by the feature extraction model processing device, and determine a first image region and a second image region other than the first image region in each of the palm images, wherein each of the first image regions includes a preset portion of the palm;

[0034] a feature acquisition module, configured to acquire, through the feature extraction model, a first embedded feature of each of the first image regions and a second embedded feature of each of the second image regions; and

[0035] A feature fusion module is used to obtain, through the feature extraction model, the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model, and fuse each first embedded feature with each second embedded feature to obtain the palm features corresponding to the multiple palm images, wherein the self-attention weight is obtained through training of the feature extraction model.

[0036] On the other hand, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the method embodiments of the present application when executing the computer program.

[0037] On the other hand, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the various method embodiments of the present application are performed.

[0038] On the other hand, the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it performs the steps of the various method embodiments of the present application.

[0039] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without any creative work.

[0041] FIG1 is a diagram illustrating an application environment of a feature extraction model processing method and a feature extraction method according to an embodiment;

[0042] FIG2 is a schematic flow chart of a feature extraction model processing method according to an embodiment;

[0043] FIG3 is a schematic diagram of two sample palm images taken at different angles in one embodiment;

[0044] FIG4 is a schematic diagram of a process for mapping a first sample embedding feature to a key space and mapping a second sample embedding feature to a query space in one embodiment;

[0045] FIG5 is a schematic diagram of mapping a sample palm image into a vector space in one embodiment;

[0046] FIG6 is a schematic diagram of a process for mapping a first sample embedding feature and a second sample embedding feature to a value space in one embodiment;

[0047] 7 is a schematic diagram of a process for determining a first sample image region and a second sample image region other than the first sample image region in each sample palm image in one embodiment;

[0048] FIG8 is a schematic flow chart of a feature extraction model processing method in one embodiment;

[0049] FIG9 is a schematic flow chart of a feature extraction model processing method in one embodiment;

[0050] FIG10 is a schematic diagram of a process for parallel training of a feature extraction model in one embodiment;

[0051] FIG11 is a schematic diagram of a flow chart of a feature extraction method in one embodiment;

[0052] FIG12 is a schematic diagram of an application scenario of a feature extraction method according to an embodiment;

[0053] FIG13 is a structural block diagram of a feature extraction model processing device according to an embodiment;

[0054] FIG14 is a block diagram of a feature extraction device according to an embodiment;

[0055] FIG15 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0057] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0058] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0059] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0060] The solutions provided in the embodiments of this application involve natural language processing, machine learning and other technologies of artificial intelligence, which are specifically illustrated by the following embodiments.

[0061] The feature extraction model processing method provided in the embodiment of the present application can be applied in the application environment shown in Figure 1. In which, the terminal 102 communicates with the server 104 via a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 and the server 104 can both independently execute the feature extraction model processing method provided in the embodiment of the present application. The terminal 102 and the server 104 can also be used in conjunction to execute the feature extraction model processing method provided in the embodiment of the present application. When the terminal 102 and the server 104 are used in conjunction to execute the feature extraction model processing method provided in the embodiment of the present application, the terminal 102 obtains multiple sample palm images of the sample palm from the server 104, determines a first sample image area and a second sample image area other than the first sample image area in each sample palm image; each first sample image area includes a preset part of the sample palm. The terminal 102 obtains the first sample embedding features of each first sample image area, maps each first sample embedding feature to a key space, and obtains a key vector for each first sample embedding feature. The terminal 102 obtains the second sample embedding features of each second sample image region, maps each second sample embedding feature to the query space, and obtains a query vector for each second sample embedding feature. The terminal 102 performs self-attention prediction based on each key vector and each query vector, and obtains a predicted self-attention weight corresponding to each first sample embedding feature, as well as a predicted self-attention weight corresponding to each second sample embedding feature. The terminal 102 fuses each first sample embedding feature and each second sample embedding feature according to their respective predicted self-attention weights to obtain a predicted palm feature corresponding to multiple sample palm images; obtains a labeled palm feature corresponding to multiple sample palm images, and performs model training based on the predicted difference between the predicted palm feature and the labeled palm feature to obtain a feature extraction model. The feature extraction model can be deployed on the terminal 102 or the server 104.

[0062] The terminal 102 may be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, portable wearable devices, etc. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0063] In one embodiment, the feature extraction method can also be applied in the application environment shown in FIG1 . Both the terminal 102 and the server 104 can independently execute the feature extraction method provided in the embodiment of the present application. The terminal 102 and the server 104 can also be used in conjunction to execute the feature extraction method provided in the embodiment of the present application.

[0064] It should be noted that the quantity of “multiple sheets” etc. mentioned in the embodiments of the present application refers to the quantity of “at least two sheets”.

[0065] In one embodiment, as shown in FIG2 , a feature extraction model processing method is provided. The method is described by applying the method to the computer device in FIG1 (the computer device may be a terminal or a server in FIG1 ) as an example, and includes the following steps:

[0066] Step S202 : acquiring multiple sample palm images of a sample palm, determining a first sample image region and a second sample image region other than the first sample image region in each sample palm image, wherein each first sample image region includes a preset part of the sample palm.

[0067] The sample palm refers to the palm of the sample object, which can be a person or an animal. The sample palm can be the palm of a person's hand or foot, or the forepaw or hindpaw of an animal.

[0068] The sample palm image is an image obtained by photographing the sample palm. The sample palm may include the fingers, palm center, and palm edge. The fingers include the thumb, index finger, middle finger, ring finger, and pinky finger. The palm edge is located around the palm center. The sample palm includes a preset location, which may be the palm center of the sample palm.

[0069] In this embodiment, the computer device may obtain multiple sample palm images of the same sample palm of the sample object, for example, obtain multiple sample palm images of a person's left palm, or obtain multiple sample palm images of a person's right palm.

[0070] The computer device determines a predetermined portion in each sample palm image and determines the region within the sample palm image where the predetermined portion is located, and uses the region as the first sample image region. For each sample palm image, the computer device uses the region of the sample palm image other than the first sample image region as the second sample image region, thereby obtaining the first sample image region and the second sample image region for each sample palm image.

[0071] In this embodiment, the computer device can use a camera to shoot the same sample palm of the sample object at different angles to obtain multiple sample palm images shot at different angles. As shown in Figure 3, two sample palm images shot at different angles are displayed.

[0072] In this embodiment, the computer device can use a camera to capture a video of the same sample palm of the sample subject. The sample subject moves and rotates the sample palm during the video capture process to change the palm posture of the sample palm, thereby generating a palm video. Multiple video frames are obtained from the palm video, and each video frame is used as a sample palm image.

[0073] Step S204: Obtain the first sample embedding features of each first sample image region, map each first sample embedding feature to a key space, and obtain a key vector of each first sample embedding feature.

[0074] The key space refers to the projection space under the QKV (Query-Key-Value) mechanism. The QKV projection operation is an operation in the attention mechanism that projects the input vector into three different spaces, converting it into a vector in each of the three spaces. The three different spaces are the query space, the key space, and the value space. The value space is also called the content space. The first sample embedding feature includes the palm print feature of the first sample image region and may also include the posture feature of the first sample image region.

[0075] In this embodiment, the computer device performs feature mapping on each first sample image region to map the first sample image region to a vector space, thereby obtaining a first sample embedding feature for each first sample image region. The computer device maps each first sample embedding feature to a key space, thereby obtaining a key vector for each first sample embedding feature.

[0076] Step S206: Obtain the second sample embedding features of each second sample image region, map each second sample embedding feature to the query space, and obtain a query vector for each second sample embedding feature.

[0077] The query space refers to the projection space under the QKV mechanism. The query space is different from the key space. The second sample embedding feature includes a palm print feature of the second sample image region and may also include a posture feature of the second sample image region.

[0078] In this embodiment, the computer device performs feature mapping on each second sample image region to map the second sample image region to a vector space, thereby obtaining a second sample embedded feature for each second sample image region. The computer device maps each second sample embedded feature to a query space, thereby obtaining a query vector for each second sample embedded feature.

[0079] Step S208: Perform self-attention prediction based on each key vector and each query vector to obtain the predicted self-attention weight corresponding to each first sample embedding feature and the predicted self-attention weight corresponding to each second sample embedding feature.

[0080] The predicted self-attention weight refers to the self-attention weight determined by the model during training. The final self-attention weight is obtained after model training is complete. The self-attention weight represents the weight that represents the correlation between features under the self-attention mechanism. The correlation between features can be represented by the similarity between the features.

[0081] The self-attention mechanism is a commonly used mechanism in computer vision and natural language processing. It establishes associations and weights between features by calculating the similarity between the query (Q), key (K), and value (V). In this example, the self-attention mechanism is applied to feature extraction from multiple palm images of the same sample palm to obtain more feature information about the sample palm.

[0082] In this embodiment, for each key vector, the computer device determines the content similarity between the key vector and each query vector. Based on the content similarities, self-attention prediction is performed to obtain a predicted self-attention weight corresponding to each first sample embedding feature and a predicted self-attention weight corresponding to each second sample embedding feature.

[0083] Furthermore, the content similarities are normalized to obtain the predicted self-attention weight corresponding to each first sample embedding feature and the predicted self-attention weight corresponding to each second sample embedding feature.

[0084] In this embodiment, the computer device can determine the key similarity between each key vector, perform self-attention prediction based on the content similarity and the key similarity, and obtain the predicted self-attention weight corresponding to each first sample embedding feature and the predicted self-attention weight corresponding to each second sample embedding feature.

[0085] In this embodiment, the computer device can determine the query similarity between each query vector, perform self-attention prediction based on the content similarity and the query similarity, and obtain the predicted self-attention weight corresponding to each first sample embedding feature and the predicted self-attention weight corresponding to each second sample embedding feature.

[0086] In step S210 , each first sample embedding feature and each second sample embedding feature are fused according to their respective predicted self-attention weights to obtain predicted palm features corresponding to multiple sample palm images.

[0087] In this embodiment, the computer device fuses each first sample embedding feature and each second sample embedding feature according to their respective predicted self-attention weights to obtain a predicted palm feature corresponding to multiple sample palm images.

[0088] Furthermore, the computer device performs weighted summation on each first sample embedding feature, each second sample embedding feature, and the corresponding predicted self-attention weight to obtain the predicted palm feature corresponding to the multiple sample palm images.

[0089] Step S212: obtaining label palm features corresponding to multiple sample palm images, performing model training based on the predicted differences between the predicted palm features and the label palm features, and obtaining a feature extraction model.

[0090] Among them, the label palm feature is a pre-labeled feature that serves as a reference result for model training.

[0091] In this embodiment, a computer device obtains labeled palm features corresponding to multiple sample palm images, determines a predicted difference between the predicted palm features and the labeled palm features, and performs model training based on the predicted difference between the predicted palm features and the labeled palm features to adjust model parameters and continue training. Upon completion of training, a feature extraction model is obtained. The feature extraction model obtained through training includes self-attention weights.

[0092] In this embodiment, the computer device determines a predicted difference between the predicted palm features and the labeled palm features, obtains a preset iteration condition, matches the predicted difference with the preset iteration condition, and adjusts model parameters when the predicted difference meets the preset iteration condition. Training continues until the predicted difference determined during training no longer meets the preset iteration condition, thereby obtaining a feature extraction model.

[0093] The preset iteration condition refers to the condition for the next iteration, which may be a preset number of iterations, a preset difference threshold, etc. For example, when the predicted difference is greater than the difference threshold, the model parameters are adjusted and training continues until the predicted difference determined during training is less than or equal to the difference threshold, thereby terminating the training and obtaining a trained feature extraction model.

[0094] In this embodiment, multiple sample palm images of a sample palm are obtained, and a first sample image region and a second sample image region other than the first sample image region are determined in each sample palm image. Each first sample image region includes a preset portion of the sample palm, thereby determining the same image region in the multiple sample palm images and the different image regions in the multiple sample palm images. A first sample embedding feature is obtained for each first sample image region, and each first sample embedding feature is mapped to a key space. The feature representations of the same first sample image regions can be mapped from the vector space to the key space to obtain a key vector for each first sample embedding feature. A second sample embedding feature is obtained for each second sample image region, and each second sample embedding feature is mapped to a query space. The feature representations of different first sample image regions can be mapped from the vector space to the query space to obtain a query vector for each second sample embedding feature. Based on the key vectors for the same sample image region and the query vectors for different sample image regions, the predicted self-attention weight corresponding to each first sample embedded feature and the predicted self-attention weight corresponding to each second sample embedded feature of multiple sample palm images can be predicted. The predicted self-attention weights represent the importance of each key vector and each query vector in the fusion, that is, the model's attention to the features in each of the same sample image regions, thereby simultaneously considering the features in the same sample image region and the features in different sample image regions. By fusing each first sample embedded feature and each second sample embedded feature according to their respective predicted self-attention weights, the same features and different features in multiple sample palm images can be fused to obtain the predicted palm features corresponding to the multiple sample palm images. The model's different attention to features in each of the same sample image regions makes the features extracted from the same sample image region not completely identical, thereby extracting different detailed features existing in the same sample image region, thereby obtaining more feature information. The labeled palm features corresponding to multiple sample palm images are obtained. The labeled palm features are used as the palm features expected to be obtained in model training. Model training is performed based on the difference between the predicted palm features obtained by the model prediction and the expected palm features. The difference between the predicted palm features and the expected palm features is gradually reduced during training, thereby gradually improving the extraction accuracy of the model and finally obtaining a feature extraction model.

[0095] The final feature extraction model contains the self-attention weights obtained through training. The self-attention weights represent the degree of attention paid to the features of the same image areas and different image areas in multiple palm images, so that the features of the same image areas can be paid attention to, thereby extracting different detail features existing in the same image areas, thereby obtaining more feature information.

[0096] In one embodiment, performing self-attention prediction based on each key vector and each query vector to obtain a predicted self-attention weight corresponding to each first sample embedding feature and a predicted self-attention weight corresponding to each second sample embedding feature includes:

[0097] Determine the key similarity between each key vector and the query similarity between each query vector; for each key vector, determine the content similarity between the key vector and each query vector; perform self-attention prediction based on each key similarity, each query similarity, and each content similarity to obtain a predicted self-attention weight corresponding to each first sample embedding feature and a predicted self-attention weight corresponding to each second sample embedding feature.

[0098] In this embodiment, the computer device calculates the similarity between each pair of key vectors. The similarity between two key vectors is called key similarity. Key similarity can be cosine similarity or Euclidean distance. The computer device can also calculate the similarity between each pair of query vectors. The similarity between two query vectors is called query similarity. This query similarity can be cosine similarity or Euclidean distance.

[0099] For each key vector, the similarity between the key vector and each query vector is determined. The similarity between the key vector and the query vector is the content similarity. The content similarity can be cosine similarity or Euclidean distance.

[0100] In this embodiment, for each first sample embedding feature, the predicted self-attention weight corresponding to the first sample embedding feature is determined based on the key similarity between the key vector of the first sample embedding feature and each other key vector, and the content similarity between the key vector of the first sample embedding feature and each query vector; for each second sample embedding feature, the predicted self-attention weight corresponding to the second sample embedding feature is determined based on the query similarity between the query vector of the second sample embedding feature and other query vectors, and the content similarity between the query vector of the second sample embedding feature and each key vector.

[0101] The other key vectors refer to the key vectors of each second sample embedding feature except the key vector of the second sample embedding feature. The other query vectors refer to the query vectors of each second sample embedding feature except the query vector of the second sample embedding feature.

[0102] Furthermore, for each first sample embedding feature, the key similarity between the key vector of the first sample embedding feature and each other key vector, as well as the content similarity between the key vector of the first sample embedding feature and each query vector, is determined. The determined key similarity and content similarity are fused to obtain a predicted self-attention weight corresponding to the first sample embedding feature. For each second sample embedding feature, the query similarity between the query vector of the second sample embedding feature and other query vectors, as well as the content similarity between the query vector of the second sample embedding feature and each key vector, are fused to obtain a predicted self-attention weight corresponding to the second sample embedding feature.

[0103] In this embodiment, the cosine similarity between each key vector is determined, and the cosine similarity between two key vectors is used as the key similarity between the two key vectors. The cosine similarity between each query vector is determined, and the cosine similarity between two query vectors is used as the query similarity between the two query vectors.

[0104] In this embodiment, the distance between each key vector is determined, and the distance between two key vectors is used as the key similarity between the two key vectors. The distance between the two key vectors can be the Euclidean distance between the two key vectors.

[0105] The distance between each query vector is determined, and the distance between two query vectors is used as the query similarity between the two query vectors. The distance between the two query vectors can be the Euclidean distance between the two query vectors.

[0106] In this embodiment, for each iteration of model training, the computer device determines each key vector and each query vector for the iteration to determine the key similarity between each key vector and the query similarity between each query vector in the iteration. In the iteration, for each key vector, the content similarity between the key vector and each query vector is determined. Self-attention prediction is performed based on each key similarity, each query similarity, and each content similarity in the iteration to obtain a predicted self-attention weight corresponding to each first sample embedding feature in the iteration, as well as a predicted self-attention weight corresponding to each second sample embedding feature in the iteration.

[0107] In this embodiment, the key similarity between each key vector and the query similarity between each query vector are determined. The key similarity represents the degree of correlation between the key vectors. The higher the key similarity, the higher the degree of correlation between the key vectors. The query similarity represents the degree of correlation between the query vectors. The higher the query similarity, the higher the degree of correlation between the query vectors. For each key vector, the content similarity between the targeted key vector and each query vector is determined. The content similarity represents the degree of correlation between the key vector and the query vector. The higher the content similarity, the higher the degree of correlation between the key vector and the query vector. Self-attention prediction is performed based on each key similarity, each query similarity, and each content similarity to obtain the predicted self-attention weight corresponding to each first sample embedded feature and the predicted self-attention weight corresponding to each second sample embedded feature. Therefore, the corresponding weights can be calculated based on the degree of correlation between the vectors, thereby obtaining the self-attention weights of all features in multiple sample palm images.

[0108] In one embodiment, mapping each first sample embedding feature to a key space to obtain a key vector for each first sample embedding feature includes: obtaining an initial key weight, and mapping each first sample embedding feature to the key space based on the initial key weight to obtain a key vector for each first sample embedding feature;

[0109] Mapping each second sample embedding feature to a query space to obtain a query vector for each second sample embedding feature includes: obtaining an initial query weight, mapping each second sample embedding feature to a query space based on the initial query weight, and obtaining a query vector for each second sample embedding feature.

[0110] The initial key weight represents the conversion coefficient from the vector space to the key space during the training process, and the initial query weight represents the conversion coefficient from the vector space to the query space during the training process.

[0111] In this embodiment, the computer device obtains an initial key weight, performs a dot product process on the initial key weight and each first sample embedding feature to map each first sample embedding feature to a key space, and obtains a key vector for each first sample embedding feature.

[0112] The computer device obtains an initial query weight, performs a dot product process on the initial query weight and each second sample embedding feature to respectively map each second sample embedding feature to the query space, and obtains a query vector for each second sample embedding feature.

[0113] In this embodiment, the computer device obtains a feature extraction model through multiple iterative training, with each iteration corresponding to its own initial key weight and initial query weight. The initial key weight of each iteration is obtained based on the adjustment of the initial key weight of the previous iteration. The initial query weight of each iteration is obtained based on the adjustment of the initial query weight of the previous iteration.

[0114] For each iteration, the computer device obtains the initial key weight for the iteration, and based on the initial key weight for the iteration, maps each first sample embedding feature to the key space to obtain a key vector for each first sample embedding feature in the iteration.

[0115] For each iteration, the computer device obtains an initial query weight for the iteration, and based on the initial query weight for the iteration, maps each second sample embedding feature to the query space to obtain a query vector for each second sample embedding feature in the iteration.

[0116] The computer device determines the predicted self-attention weight corresponding to each first sample embedding feature in the targeted iteration and the predicted self-attention weight corresponding to each second sample embedding feature in the targeted iteration based on each key vector and each query vector in the targeted iteration. Each first sample embedding feature and each second sample embedding feature are fused according to their respective predicted self-attention weights in the targeted iteration to obtain the predicted palm features corresponding to multiple sample palm images in the targeted iteration. Based on the predicted difference between the predicted palm features in the targeted iteration and the label palm features, the initial key weight and initial query weight corresponding to the targeted iteration are adjusted to obtain the updated key weight and updated query weight in the targeted iteration. The updated key weight is used as the initial key weight of the next iteration, and the updated query weight is used as the initial query weight of the next iteration, thereby entering the next iteration until the feature extraction model is obtained through training.

[0117] As shown in Figure 4, the initial key weight is W k , the first sample embedding feature is X1, through the initial key weight W k Map the first sample embedding feature X1 to the key space to obtain the key vector K.

[0118] The initial query weight is W Q , the second sample embedding feature is X2, through the initial query weight W Q The second sample embedding feature X2 is mapped to the key space to obtain the query vector Q. According to each key vector K and each query vector Q, the predicted self-attention weight α is determined.

[0119] In this embodiment, initial key weights are obtained, and based on the initial key weights, each first sample embedding feature is mapped to the key space, thereby accurately projecting each first sample image region from the vector space to the key space, and obtaining a key vector for each first sample embedding feature. Initial query weights are obtained, and based on the initial query weights, each second sample embedding feature is mapped to the query space, thereby accurately projecting each second sample embedding feature from the vector space to the query space, and obtaining a query vector for each second sample embedding feature.

[0120] In one embodiment, model training is performed based on the predicted difference between the predicted palm features and the labeled palm features to obtain a feature extraction model, including:

[0121] Determine the predicted difference between the predicted palm feature and the labeled palm feature; when the predicted difference meets the preset iteration condition, determine the key similarity between each key vector, and adjust the initial key weight according to each key similarity to obtain an updated key weight; determine the query similarity between each query vector, adjust the initial query weight according to each query similarity, and obtain an updated query weight; use the updated key weight as the initial key weight of the next iteration, use the updated query weight as the initial query weight of the next iteration, return to the step of mapping each first sample embedding feature to the key space based on the initial key weight and continue executing until the determined prediction difference does not meet the preset iteration condition, and stop, thereby obtaining a feature extraction model.

[0122] In this embodiment, the computer device determines the predicted difference between the predicted palm features and the labeled palm features and matches the predicted difference with a preset iteration condition. When the predicted difference satisfies the preset iteration condition, the computer device determines the key similarity between each key vector and adjusts the initial key weight based on each key similarity to obtain an updated key weight. Furthermore, the computer device determines the query similarity between each query vector and adjusts the initial query weight based on each query similarity to obtain an updated query weight.

[0123] After obtaining the updated key weight and the updated query weight, the next iteration is entered, and the updated key weight is used as the initial key weight of the next iteration, and the updated query weight is used as the initial query weight of the next iteration.

[0124] In this embodiment, each iteration corresponds to its own initial key weight and initial query weight. For each iteration, the computer device obtains the initial key weight and initial query weight for the iteration, and based on the initial key weight for the iteration, maps each first sample embedded feature to the key space, thereby obtaining a key vector for each first sample embedded feature in the iteration. Based on the initial query weight for the iteration, each second sample embedded feature is mapped to the query space, thereby obtaining a query vector for each second image region in the iteration.

[0125] The computer device determines, based on each key vector and each query vector in the targeted iteration, a predicted self-attention weight corresponding to each first sample embedding feature and a predicted self-attention weight corresponding to each second sample embedding feature in the targeted iteration. Based on each predicted self-attention weight in the targeted iteration, each first sample embedding feature and each second sample embedding feature are fused to obtain the predicted palm features commonly corresponding to the multiple sample palm images in the targeted iteration. The computer device determines the predicted difference of the predicted palm features in the targeted iteration relative to the label palm features. When the predicted difference in the targeted iteration meets a preset iteration condition, the computer device determines the key similarity between each key vector in the targeted iteration, and adjusts the initial key weight in the targeted iteration based on each key similarity to obtain the updated key weight in the targeted iteration. Furthermore, the computer device determines the query similarity between each query vector in the targeted iteration, adjusts the initial query weight in the targeted iteration based on each query similarity, and obtains the updated query weight in the targeted iteration. The updated key weight is used as the initial key weight of the next iteration of the iteration being targeted, and the updated query weight is used as the initial query weight of the next iteration of the iteration being targeted, thereby entering the next iteration. In the next iteration, the step of mapping each first sample embedding feature to the key space based on the initial key weight is executed and continued until the determined prediction difference does not meet the preset iteration condition, and the feature extraction model is obtained.

[0126] The failure to meet the preset iteration condition may be that the number of iterations does not meet the preset number of iterations, the predicted difference for the iteration does not meet the preset difference threshold, etc. The failure of the predicted difference for the iteration to meet the preset difference threshold may be that the predicted difference for the iteration is less than or equal to the difference threshold.

[0127] In this embodiment, the predicted difference between the predicted palm features and the labeled palm features is determined, and the difference between the predicted palm features obtained by the model prediction and the expected palm features can be determined. When the predicted difference meets the preset iteration conditions, indicating that the next iteration is required, the key weight and query weight are first updated. That is, the key similarity between each key vector is determined. The key similarity represents the correlation between the key vectors. The higher the similarity, the higher the correlation. Therefore, the initial key weight can be adjusted according to each key similarity to obtain an updated key weight. The query similarity represents the correlation between the query vectors. Therefore, the initial query weight can be adjusted according to each query similarity to obtain an updated query weight. The updated key weight is used as the initial key weight of the next iteration, and the updated query weight is used as the initial query weight of the next iteration, so as to enter the next iteration and continue execution, so that the key weight and query weight are continuously updated in the iteration, and the predicted self-attention weights determined based on the key weight and query weight are also continuously updated, so that the difference between the predicted palm features and the expected palm features is gradually reduced in the iteration, until the determined prediction difference does not meet the preset iteration condition and stops. At this time, the final key weight, query weight and self-attention weight are obtained, thereby obtaining a feature extraction model.

[0128] In one embodiment, the method further comprises:

[0129] Obtaining an initial content weight, and based on the initial content weight, mapping each first sample embedding feature and each second sample embedding feature to a value space, respectively, to obtain a content vector for each first sample embedding feature and a content vector for each second sample embedding feature;

[0130] The first sample embedding features and the second sample embedding features are fused according to their respective predicted self-attention weights to obtain the predicted palm features corresponding to multiple sample palm images, including:

[0131] The content vector of each first sample embedding feature and the content vector of each second sample embedding feature are fused according to their respective predicted self-attention weights to obtain the predicted palm features corresponding to multiple sample palm images.

[0132] Among them, the initial content weight represents the conversion coefficient from vector space to value space during training.

[0133] In this embodiment, the computer device obtains the initial content weight, performs dot multiplication on the initial content weight and each first sample embedding feature to map each first sample embedding feature and each second sample embedding feature to the value space respectively, and obtains the content vector of each first sample embedding feature and the content vector of each second sample embedding feature.

[0134] The computer device fuses the content vector of each first sample embedding feature and the content vector of each second sample embedding feature based on the predicted self-attention weights to obtain predicted palm features corresponding to the multiple sample palm images.

[0135] In this embodiment, the computer device performs weighted summation on the content vector of each first sample embedding feature, the content vector of each second sample embedding feature, and the corresponding predicted self-attention weight to obtain the predicted palm features corresponding to multiple sample palm images.

[0136] As shown in Figure 5, the sample palm image is mapped to the vector space to obtain the corresponding sample embedding feature X. The sample embedding feature X includes a first sample embedding feature X1 and a second sample embedding feature X2. That is, the first sample image region and the second sample image region are mapped to the vector space respectively to obtain the first sample embedding feature X1 and the second sample embedding feature X2.

[0137] As shown in Figure 6, the predicted self-attention weight α is determined based on each key vector K and each query vector Q.

[0138] Based on the initial content weight W V , map the sample embedding feature X to the value space, that is, map each first sample embedding feature X1 and each second sample embedding feature X2 to the value space respectively, and obtain the content vector V.

[0139] Each content vector V is fused according to its own predicted self-attention weight to obtain the predicted palm feature.

[0140] In this embodiment, initial content weights are obtained, and based on the initial content weights, each first sample embedded feature and each second sample embedded feature are mapped to a value space. This allows all embedded features of multiple sample palm images to be accurately mapped to the value space, thereby obtaining a content vector for each first sample embedded feature and a content vector for each second sample embedded feature. The content vectors of the first sample embedded feature and the content vectors of each second sample embedded feature are then reduced and fused according to their respective predicted self-attention weights. This allows features of the same region in multiple sample palm images to be fused to varying degrees, resulting in different levels of attention to features of the same region in each sample palm image. This allows attention to be paid to the differences between features of the same region in multiple sample palm images, and the detailed features of these differences are obtained. Combined with features of other image regions that do not contain the preset region, the predicted palm features corresponding to the multiple sample palm images obtained contain more information, helping to improve the accuracy of palm feature recognition and verification in different scenarios.

[0141] In one embodiment, the method further comprises:

[0142] When the predicted difference meets the preset iteration condition, for each key vector, the content similarity between the key vector and each query vector is determined; the initial content weight is adjusted according to each content similarity to obtain an updated content weight; the updated content weight is used as the initial content weight for the next iteration, and the step of returning to the initial key weight and mapping each first sample embedding feature to the key space is executed.

[0143] In this embodiment, when the predicted difference satisfies a preset iteration condition, for each key vector, the content similarity between the key vector and each query vector is determined. The content similarity can be cosine similarity or distance, i.e., the cosine similarity between the key vector and the query vector is used as the content similarity, or the distance between the key vector and the query vector is used as the content similarity. The distance can be a Euclidean distance.

[0144] The computer device adjusts the initial content weights based on the similarities of the respective contents to obtain updated content weights. The updated content weights are used as the initial content weights for the next iteration, and the next iteration is entered, where the step of mapping each first sample embedding feature to the key space based on the initial key weights is performed.

[0145] In this embodiment, the computer device obtains the feature extraction model through multiple iterations of training, each iteration corresponding to its own initial key weight, initial query weight, and initial content weight. The initial content weight of each iteration is obtained based on the adjustment of the initial content weight of the previous iteration.

[0146] For each iteration, the computer device obtains the initial content weight of the iteration, and based on the initial content weight of the iteration, maps each first sample embedding feature and each second sample embedding feature to the key space respectively, to obtain the content vector of each first sample embedding feature and the content vector of each second sample embedding feature in the iteration.

[0147] After determining the predicted self-attention weight corresponding to each first sample embedding feature and each second sample embedding feature in the targeted iteration, the computer device fuses the content vectors in the targeted iteration based on the predicted self-attention weights in the targeted iteration to obtain the predicted palm features commonly corresponding to the multiple sample palm images in the targeted iteration. The computer device determines the predicted difference between the predicted palm features in the targeted iteration and the label palm features. When the predicted difference in the targeted iteration meets a preset iteration condition, the computer device adjusts the initial key weight in the targeted iteration based on the key similarity between the key vectors in the targeted iteration to obtain the updated key weight in the targeted iteration. Furthermore, the computer device adjusts the initial query weight in the targeted iteration based on the query similarity between the query vectors in the targeted iteration to obtain the updated query weight in the targeted iteration. Finally, the computer device adjusts the initial content weight in the targeted iteration based on the content similarity between the query vectors in the targeted iteration to obtain the updated content weight in the targeted iteration. The updated key weight is used as the initial key weight of the next iteration of the iteration being targeted, the updated query weight is used as the initial query weight of the next iteration, and the updated content weight is used as the initial content weight of the next iteration, thereby entering the next iteration. In the next iteration, the step of mapping each first sample embedding feature to the key space based on the initial key weight is executed and continued until the determined prediction difference does not meet the preset iteration condition, and the feature extraction model is obtained.

[0148] In this embodiment, when the predicted difference meets the preset iteration condition, for each key vector, the content similarity between the key vector and each query vector is determined, and the content similarity characterizes the degree of correlation between the key vector and the query vector, so that the initial content weight is adjusted according to each content similarity, and an updated content weight is obtained. The updated content weight is used as the initial content weight for the next iteration, and the return is executed based on the initial key weight to continuously update the content weight in the iteration, thereby gradually reducing the difference between the predicted palm features and the expected palm features in the iteration, thereby gradually improving the extraction accuracy of the model, and finally obtaining a feature extraction model.

[0149] In one embodiment, as shown in FIG7 , determining a first sample image region and a second sample image region other than the first sample image region in each sample palm image includes:

[0150] Step S702: extract feature points from each sample palm image.

[0151] Step S704 : identifying a preset part in each sample palm image based on the feature points of each sample palm image, and determining a first sample image region where the preset part in each sample palm image is located.

[0152] In this embodiment, the computer device extracts feature points from each sample palm image to obtain a plurality of feature points for each sample palm image.

[0153] For each sample palm image, the computer device identifies a preset part based on the feature points of the sample palm image, determines the positions of the feature points forming the preset part in the sample palm image, and obtains a first sample image area including the preset part based on the determined positions.

[0154] Step S706 : for each sample palm image, determine a second sample image region in the sample palm image except the corresponding first sample image region.

[0155] In this embodiment, the computer device uses the area of ​​the sample palm image other than the first sample image area as the second sample image area, thereby obtaining the first sample image area and the second sample image area of ​​the sample palm image. Following the same process, the computer device can obtain the first sample image area and the second sample image area of ​​each sample palm image.

[0156] In this embodiment, feature points are extracted from each sample palm image. Based on the feature points of each sample palm image, a predetermined portion of each sample palm image is identified, and a first sample image region within each sample palm image where the predetermined portion is located is determined. This allows for the identification of image regions containing the same predetermined portion across multiple sample palm images. These image regions contain identical or very similar features, facilitating further extraction of detailed features based on the identical or similar features, thereby improving the accuracy of detailed feature acquisition and identification. For each sample palm image, a second sample image region, excluding the corresponding first sample image region, is determined within the sample palm image, facilitating subsequent different processing for sample image regions containing the predetermined portion and sample image regions not containing the predetermined portion.

[0157] In one embodiment, the feature extraction model processing method includes multiple parts such as image acquisition, image mapping, self-attention mechanism processing, parallel training and feature storage.

[0158] As shown in Figure 8, the image acquisition component is used to capture multiple sample palm images of the same user. By acquiring sample palm images from different angles and time points, diverse data containing more information is obtained. Specifically, by using multiple cameras to capture the user's palm from multiple angles and in multiple temporal states, diverse data from multiple palm images of the same user is obtained. This data contains palm information from different angles, different time points, and different dynamics, providing a richer information foundation for subsequent feature extraction and processing.

[0159] Image mapping aims to map the sample palm image to a specific vector space to extract the feature representation of the sample palm image and retain more information about the sample palm image. This can be achieved by using a redundant sparse embedding method, where the vector may contain redundant features and the entire vector is sparse, which can better preserve the feature information of the sample palm image.

[0160] For example, a sample palm image is represented by a pixel matrix, where each element represents the pixel value of the sample palm image. This pixel matrix is ​​a 5*5 matrix, where the element values ​​range from 0 to 255. The pixel matrix can be flattened into a 25-dimensional vector, with each pixel value as a feature. This 25-dimensional vector is then reduced in dimensionality to map it to a lower-dimensional vector representation. This can be achieved by using dimensionality reduction methods such as principal component analysis (PCA) or autoencoders. This compresses the information of the sample palm image to a lower dimension while retaining important features. Ultimately, a low-dimensional embedding feature vector for the sample palm image is obtained, where each element represents the feature representation of the corresponding pixel of the sample palm image in a low-dimensional space. This vector can be used for subsequent palm feature extraction, matching, or recognition tasks.

[0161] By adopting a redundant and sparse embedding approach, more feature information of the sample palm images can be retained. This approach allows for the existence of redundant features, that is, multiple embedding representations of the same feature, while leveraging sparsity to represent the entire vector. This enables the model to better extract and represent the detailed features of the sample palm images, thereby enhancing the accuracy and robustness of palm recognition in subsequent processing and recognition stages.

[0162] The self-attention mechanism is a multi-head self-attention mechanism for multi-sample palm image attention processing. It uses the self-attention mechanism to calculate the correlation and weight distribution between multiple sample palm images, thereby obtaining more redundant information and increasing the information content of palm features. This unit uses the QKV mechanism to project the embedding features into multiple subspaces to learn sample palm image features in multiple subspaces in parallel, and utilizes the multi-head attention mechanism to enhance the computational efficiency. Specifically, the self-attention mechanism is used to calculate the correlation and weight distribution between multiple sample palm images. By converting the embedded features of each sample image region into query vectors, key vectors, and content vectors, and calculating the similarity between them, it can obtain redundant information between multiple sample palm images and capture the correlation between these sample palm images, thereby obtaining more redundant information and increasing the information content of palm features.

[0163] After calculating the predicted self-attention weights, the content vectors are fused to obtain the predicted palm features. The predicted palm features obtained by feature fusion are used as the input of the parallel training part in Figure 8.

[0164] As shown in Figure 9, parallel training uses a feedforward neural network as the base model. A feedforward neural network is a commonly used deep learning model that processes information and extracts features by passing signals layer by layer. To avoid problems like exploding gradients and improve model training performance, the Residual Network (ResNet) structure and batch normalization technology are used in training. ResNet introduces cross-layer connections and residual learning, addressing the issues of vanishing gradients and limited expressiveness in deep network training. Batch normalization normalizes each small batch of samples, making the network's input distribution more stable, improving training speed and the model's generalization capabilities.

[0165] Parallel training performs parallel computations on each layer to preserve the features of the sample palm image. This allows the features of multiple sample palm images to be processed simultaneously, preventing information loss. This approach helps speed up training and preserve the detailed features of the sample palm images.

[0166] As shown in Figure 10, by employing a feedforward neural network architecture, ResNet, batch normalization, and parallel computing, the parallel training module can improve the training speed of the palm swipe recognition system. To avoid problems such as exploding gradients, parallel computing is performed on each layer simultaneously, preserving the palm features of the sample palm image and improving computational efficiency. This also accelerates the training process while effectively extracting and learning palm features, enhancing the accuracy and performance of the palm swipe recognition system.

[0167] As shown in Figure 10, the first sample image area and the second sample image area of ​​the sample palm image can be mapped to the vector space to obtain sample embedding feature 1 and sample embedding feature 2. The sample embedding feature 1 and the sample embedding feature 2 can be normalized in parallel. The two normalized embedding features are respectively input into the residual block to realize parallel processing in the residual network.

[0168] Alternatively, sample embedded feature 1 and sample embedded feature 2 can be input separately into the hidden neurons of the hidden layer, so that hidden neuron 1 processes sample embedded feature 1 and hidden neuron 2 processes sample embedded feature 2, thereby achieving parallel processing. The features processed by hidden neuron 1 are input into residual block 1 of the residual network, and the features processed by hidden neuron 2 are input into residual block 2 of the residual network. This results in predicted palm feature 1 output by residual block 1 and predicted palm feature 2 output by residual block 2, thereby achieving parallel processing of various components during training. By fusing predicted palm feature 1 and predicted palm feature 2, the predicted palm feature corresponding to the sample palm image can be obtained.

[0169] In this embodiment, the sample embedding feature 1 and the sample embedding feature 2 may be the embedding features corresponding to the two sample palm images respectively, thereby realizing parallel training of multiple sample palm images.

[0170] In this embodiment, to improve computational efficiency and explore different subspaces, parallel learning during training allows for the simultaneous processing of multiple sample palm image features to retain more information and features. The multi-head attention mechanism allows the model to learn and calculate the degree of correlation and importance between sample palm images in multiple subspaces, thereby increasing the amount of information. By combining parallel learning and the multi-head attention mechanism, it is possible to more comprehensively capture the redundant information between different sample palm images, thereby extracting richer palm feature information to improve the accuracy of the palm swipe recognition system.

[0171] In one embodiment, as shown in FIG11 , a feature extraction method is provided. The method is described by applying the method to the computer device in FIG1 (the computer device may be a terminal or a server in FIG1 ) as an example. The method includes the following steps:

[0172] Step S1102: Acquire multiple palm images.

[0173] The palm refers to the palm of the object to be verified. The object to be verified can be a person or an animal, and the palm can be the palm of a person's hand or foot, or the forefoot or hindfoot of an animal.

[0174] A palm image is an image obtained by photographing the palm of the subject to be authenticated. The palm can include the fingers, palm center, and palm edge. The fingers include the thumb, index finger, middle finger, ring finger, and pinky finger. The palm includes a preset area, which can be the palm center.

[0175] In this embodiment, the computer device may obtain multiple palm images of the same palm of the object to be verified, for example, obtaining multiple palm images of a person's left palm, or obtaining multiple palm images of a person's right palm.

[0176] In this embodiment, the computer device may use a camera to shoot the same palm of the object to be verified at different angles to obtain multiple palm images shot at different angles.

[0177] In this embodiment, the computer device can use a camera to capture a video of the same palm of the subject to be verified. The subject to be verified moves or rotates the palm during the video capture process to change the palm posture, thereby generating a palm video. Multiple video frames are obtained from the palm video, each of which is used as a palm image.

[0178] Step S1104: determining a first image region and a second image region other than the first image region in each palm image using a feature extraction model trained by the feature extraction model processing method, wherein each first image region includes a preset part of the palm.

[0179] In this embodiment, the computer device inputs each palm image into a feature extraction model, and the feature extraction model is obtained by training through a feature extraction model processing method.

[0180] The feature extraction model identifies a predetermined location in each palm image and the region within the palm image where the predetermined location is located, which is used as the first image region. For each palm image, the region of the palm image other than the first image region is used as the second image region, thereby obtaining the first and second image regions for each palm image.

[0181] In this embodiment, the feature extraction model obtained by training the feature extraction model processing method determines the first image region and the second image region other than the first image region in each palm image, including:

[0182] The feature extraction model obtained by training the feature extraction model processing method extracts the feature points of each palm image respectively; the feature extraction model identifies the preset part in each palm image based on the feature points of each palm image, and determines the first image area where the preset part is located in each palm image; the feature extraction model determines, for each palm image, a second image area in the targeted palm image other than the corresponding first image area.

[0183] Step S1106: Obtain the first embedded features of each first image region and the second embedded features of each second image region through the feature extraction model.

[0184] In this embodiment, the computer device performs feature mapping on each first image region using the feature extraction model to map the first image region to a vector space, thereby obtaining a first embedded feature for each first image region. The computer device also performs feature mapping on each second image region using the feature extraction model to map the second image region to a vector space, thereby obtaining a second embedded feature for each second image region.

[0185] In step S1108, the first embedded features and the second embedded features are fused through the feature extraction model according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model to obtain the palm features corresponding to the multiple palm images.

[0186] In this embodiment, the feature extraction model obtains multiple self-attention weights through training. The multiple self-attention weights include a self-attention weight for each first embedded feature and a self-attention weight for each second embedded feature. The computer device, through the feature extraction model, fuses the first embedded features and the second embedded features based on their respective attention weights to obtain palm features corresponding to the multiple palm images.

[0187] Furthermore, the computer device performs weighted summation on each first embedding feature, each second embedding feature, and the corresponding self-attention weight to obtain a palm feature corresponding to multiple palm images.

[0188] In this embodiment, a feature extraction model trained using a feature extraction model processing method includes multiple self-attention weights, each of which represents the degree to which the feature extraction model focuses on features of identical image regions and different image regions in multiple palm images. Multiple palm images are obtained, and a first image region and a second image region other than the first image region are determined in each palm image using a feature extraction model. Each first image region includes a predetermined portion of the palm, thereby determining identical image regions and different image regions in the multiple palm images. The feature extraction model obtains a first embedded feature for each first image region and a second embedded feature for each second image region, thereby converting each image region from image space to vector space to obtain a corresponding embedded feature. The feature extraction model fuses each first embedded feature with each second embedded feature according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model. This allows the model to focus not only on features of different image regions but also on features of identical image regions, thereby extracting different detail features within the same image region and ensuring that the obtained palm features contain more feature quantities. Moreover, the extracted palm features contain more detailed features, which can improve the recognition accuracy when using palm features for feature recognition in different scenarios.

[0189] In one embodiment, the feature extraction model further includes content weights obtained through training; the method further includes: mapping each first embedded feature and each second embedded feature to a value space based on the content weights using the feature extraction model to obtain a content vector for each first embedded feature and a content vector for each second embedded feature;

[0190] Through the feature extraction model, according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model, each first embedded feature and each second embedded feature are fused to obtain palm features corresponding to multiple palm images, including: through the feature extraction model, according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model, the content vector of each first embedded feature and the content vector of each second embedded feature are fused to obtain palm features corresponding to multiple palm images.

[0191] The content weight represents the conversion coefficient from the vector space to the value space.

[0192] In this embodiment, the feature extraction model obtains multiple self-attention weights and content weights through training. The computer device, through the feature extraction model, performs a dot product between the content weights and each first embedded feature to map each first embedded feature and each second embedded feature to a value space, thereby obtaining a content vector for each first embedded feature and a content vector for each second embedded feature.

[0193] The computer device uses a feature extraction model to fuse the content vector of each first embedded feature and the content vector of each second embedded feature according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature, so as to obtain palm features corresponding to multiple palm images.

[0194] In this embodiment, the self-attention weight includes the weight corresponding to the content vector. The computer device uses a feature extraction model to perform weighted summation on the content vector of each first embedded feature and the content vector of each second embedded feature with the corresponding weight to obtain the palm features corresponding to multiple palm images.

[0195] In this embodiment, through a feature extraction model, each first embedded feature and each second embedded feature is mapped to a value space based on content weights. This allows all embedded features of multiple palm images to be accurately mapped to the value space to obtain a content vector for each first embedded feature and a content vector for each second embedded feature. The content vectors of each first embedded feature and each second embedded feature are fused according to the self-attention weights for each first embedded feature and each second embedded feature. This allows the features of the same part in multiple palm images to be fused to varying degrees, resulting in different levels of attention to the features of the same part in each palm image. This allows attention to be paid to the differences between the features of the same part in multiple palm images, and the detailed features of these differences to be obtained. Combined with the features of other image regions that do not contain the preset part, the palm features corresponding to the multiple palm images obtained contain more information, which helps improve the accuracy of palm feature recognition and verification in different scenarios.

[0196] In one embodiment, the feature extraction method is applied to a resource transfer scenario; obtaining multiple palm images includes:

[0197] In response to the resource transfer triggering operation, calling the camera to shoot the palm of the object to be verified at different angles to obtain multiple palm images;

[0198] The method further includes:

[0199] The authentication of the object to be authenticated is performed according to the palm features to obtain an authentication result; when the authentication result indicates that the authentication of the object to be authenticated is successful, resource transfer information is obtained, and a resource transfer operation is performed based on the resource transfer information.

[0200] In this embodiment, in a resource transfer scenario, the user to be verified can run a resource transfer application on a computer device, enter resource transfer information based on the application, and then click to trigger the resource transfer operation. In response to the resource transfer triggering operation, the resource transfer application uses a camera to capture the palm of the person to be verified at different angles, obtaining multiple palm images.

[0201] A computer device inputs multiple palm images into a feature extraction model, and uses the feature extraction model to determine a first image region and a second image region other than the first image region in each palm image, where each first image region includes a preset portion of the palm. The feature extraction model obtains a first embedded feature for each first image region and a second embedded feature for each second image region. The feature extraction model fuses each first embedded feature and each second embedded feature based on self-attention weights to obtain palm features corresponding to the multiple palm images. The computer device authenticates the object to be authenticated based on the palm features and obtains a verification result. When the verification result indicates that the authentication of the object to be authenticated is successful, resource transfer information is obtained, and a resource transfer operation is performed based on the resource transfer information.

[0202] In this embodiment, a feature extraction model is used to map each first embedded feature and each second embedded feature to a value space based on content weights, thereby obtaining a content vector for each first embedded feature and a content vector for each second embedded feature. The feature extraction model also fuses the content vectors of each first embedded feature and each second embedded feature according to the self-attention weights for each first embedded feature and each second embedded feature, thereby obtaining palm features corresponding to multiple palm images. The computer device authenticates the subject to be authenticated based on the palm features. Upon successful authentication of the subject to be authenticated, the computer device obtains resource transfer information and executes a resource transfer operation based on the resource transfer information.

[0203] In this embodiment, the feature extraction method is applied to the resource transfer scenario, so that in response to the resource transfer trigger operation, the camera is called to shoot the palm of the object to be verified at different angles to obtain multiple palm images, and then the palm features of the multiple palm images are obtained through the feature extraction model. The obtained palm features include features with detailed differences in the same parts of the multiple palm images, so that the identity authentication of the object to be verified based on the palm features can be more accurate, and the resource transfer operation is performed based on the resource transfer information, which can effectively improve the security of the resource transfer.

[0204] In other embodiments, the feature extraction method can be applied to access control, gate control, and other applications. As shown in Figure 12, a user places their palm on the gate's palm recognition area. The palm recognition area captures multiple palm images of the user's palm and processes them using a feature extraction model to obtain palm features. The user's identity is authenticated based on the palm features, and the gate is unlocked if successful. Otherwise, the gate is not unlocked and a prompt is issued.

[0205] It can be understood that the feature extraction method can be applied to any application scenario requiring feature extraction of a palm image, and is not limited to the examples in the above embodiments.

[0206] In one embodiment, a feature extraction model processing method is provided, which is applied to a computer device and includes:

[0207] A plurality of sample palm images of the sample palm are obtained, and feature points of each sample palm image are extracted respectively.

[0208] Next, based on the characteristic points of each sample palm image, a preset part in each sample palm image is identified, and a first sample image region where the preset part in each sample palm image is located is determined.

[0209] Next, for each sample palm image, a second sample image region other than the corresponding first sample image region in the sample palm image is determined.

[0210] Furthermore, a first sample embedding feature of each first sample image region is obtained, and a second sample embedding feature of each second sample image region is obtained.

[0211] Furthermore, the initial key weight, initial query weight, and initial content weight of the current iteration are obtained, and based on the initial key weight of the current iteration, each first sample embedding feature is mapped to the key space to obtain the key vector of each first sample embedding feature in the current iteration; based on the initial query weight of the current iteration, each second sample embedding feature is mapped to the query space to obtain the query vector of each second image region in the current iteration; based on the initial content weight of the current iteration, each first sample embedding feature and each second sample embedding feature is mapped to the value space to obtain the content vector of each first sample embedding feature and the content vector of each second sample embedding feature in the current iteration. Wherein, when the current iteration is the first iteration, the initial key weight, initial query weight, and initial content weight are all preset weights.

[0212] Then, in the current iteration, for each key vector, the content similarity between the key vector and each query vector is determined; each content similarity is normalized to obtain the predicted self-attention weight corresponding to each first sample embedding feature and the predicted self-attention weight corresponding to each second sample embedding feature.

[0213] Then, based on the predicted self-attention weights in the current iteration, the content vectors of the current iteration are fused to obtain the predicted palm features corresponding to multiple sample palm images in the current iteration.

[0214] Furthermore, label palm features corresponding to multiple sample palm images are obtained, and the predicted difference between the predicted palm features of the current iteration and the label palm features is determined.

[0215] Then, when the prediction difference of the current iteration is greater than the difference threshold, the key similarity between each key vector in the current iteration is determined, and the initial key weight of the current iteration is adjusted according to each key similarity to obtain the key weight updated in the current iteration; the query similarity between each query vector in the current iteration is determined, and the initial query weight of the current iteration is adjusted according to each query similarity to obtain the query weight updated in the current iteration; in the current iteration, for each key vector, the content similarity between the key vector and each query vector is determined; the initial content weight of the current iteration is adjusted according to each content similarity to obtain the content weight updated in the current iteration.

[0216] Furthermore, the updated key weight, updated query weight, and updated query weight in the current iteration are used as the initial key weight, initial query weight, and initial content weight corresponding to the next iteration of the current iteration, respectively. Entering the next iteration, returning to the step of mapping each first sample embedding feature to the key space based on the initial key weight and continuing until the determined prediction difference is less than or equal to the difference threshold, thereby obtaining a feature extraction model. The key weight, query weight, content weight, and self-attention weight of the feature extraction model are obtained when the last iteration stops.

[0217] This embodiment also provides a feature extraction method, including:

[0218] A plurality of palm images are obtained, and a first image region and a second image region other than the first image region in each palm image are determined through a feature extraction model; each first image region includes a preset part of the palm.

[0219] Next, the first embedded features of each first image region and the second embedded features of each second image region are obtained through the feature extraction model; based on the content weight, each first embedded feature and each second embedded feature are respectively mapped to the value space to obtain the content vector of each first embedded feature and the content vector of each second embedded feature.

[0220] Furthermore, through the feature extraction model, according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model, the content vector of each first embedded feature and the content vector of each second embedded feature are fused to obtain the palm features corresponding to multiple palm images.

[0221] In this embodiment, feature points are extracted from each sample palm image. Based on the feature points of each sample palm image, a predetermined portion of each sample palm image is identified, and a first sample image region within each sample palm image is determined. This allows for the identification of image regions containing the same predetermined portion across multiple sample palm images. These image regions contain identical or very similar features, facilitating further extraction of detailed features based on these identical or similar features, thereby improving the accuracy of detailed feature acquisition and identification. For each sample palm image, a second sample image region, excluding the corresponding first sample image region, is determined within the sample palm image, thereby identifying distinct image regions across the multiple sample palm images.

[0222] Based on the initial key weight, each first sample image region is accurately projected from the vector space to the key space to obtain the key vector of each first sample embedding feature. Based on the initial query weight, each second sample image region is accurately projected from the vector space to the query space to obtain the query vector of each second sample embedding feature. Based on the content query weight, each first sample embedding feature and each second sample embedding feature is accurately projected from the vector space to the value space to obtain their corresponding content vectors.

[0223] Based on the key vectors for the same sample image region and the query vectors for different sample image regions, a predicted self-attention weight corresponding to each first sample embedded feature and a predicted self-attention weight corresponding to each second sample embedded feature can be predicted. The predicted self-attention weights represent the importance of each key vector and each query vector in the fusion, that is, the model's attention to features in the same sample image region, thereby simultaneously considering features in the same sample image region and features in different sample image regions. By fusing the first sample embedded features and the second sample embedded features based on the predicted self-attention weights, it is possible to fuse both common and different features in multiple sample palm images to obtain predicted palm features corresponding to the multiple sample palm images. The model's varying attention to features in the same sample image region results in varying features extracted from the same sample image region. This allows for the extraction of different detailed features within the same sample image region, thereby obtaining more feature information. Labeled palm features corresponding to the multiple sample palm images are obtained and used as the desired palm features for model training. The differences between the predicted palm features predicted by the model and the desired palm features can then be determined. When the predicted difference meets the preset iteration conditions, indicating that the next iteration is necessary, the key weight and query weight are first updated. Specifically, the key similarity between each key vector is determined. Key similarity represents the correlation between key vectors, with higher similarities indicating higher correlation. This allows the initial key weight to be adjusted based on each key similarity, resulting in an updated key weight. Query similarity represents the correlation between query vectors, allowing the initial query weight to be adjusted based on each query similarity, resulting in an updated query weight. Content similarity represents the correlation between query vectors and key vectors, allowing the initial content weight to be adjusted based on each content similarity, resulting in an updated content weight.

[0224] The updated key weight, query weight, and content weight are used as the initial key weight, initial query weight, and initial content weight of the next iteration, thereby entering the next iteration and continuing to execute, so that the key weight, query weight, and content weight are continuously updated in the iteration, and the predicted self-attention weight determined based on the key weight and query weight is also continuously updated, so that the difference between the predicted palm features and the expected palm features is gradually reduced in the iteration, until the determined prediction difference does not meet the preset iteration condition and stops. At this time, the final key weight, query weight, and self-attention weight are obtained, thereby obtaining a feature extraction model.

[0225] The final feature extraction model contains multiple self-attention weights obtained through training. These multiple self-attention weights represent the degree of attention to the features of the same image areas and different image areas in multiple palm images, so that the features of the same image areas can be paid attention to, thereby extracting different detail features existing in the same image areas, thereby obtaining more feature information.

[0226] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0227] Based on the same inventive concept, the present application also provides a feature extraction model processing device for implementing the feature extraction model processing method involved above. The implementation solution provided by this device is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more feature extraction model processing device embodiments provided below can be found in the above limitations on the feature extraction model processing method and will not be repeated here.

[0228] In one embodiment, as shown in FIG13 , a feature extraction model processing device is provided, comprising:

[0229] The sample acquisition module 1302 is used to acquire multiple sample palm images of the sample palm, and determine a first sample image area and a second sample image area other than the first sample image area in each sample palm image; each first sample image area includes a preset part of the sample palm.

[0230] The key space mapping module 1304 is used to obtain the first sample embedding features of each first sample image region, map each first sample embedding feature to the key space, and obtain a key vector of each first sample embedding feature.

[0231] The query space mapping module 1306 is configured to obtain the second sample embedded features of each second sample image region, map each second sample embedded feature to the query space, and obtain a query vector for each second sample embedded feature.

[0232] The weight prediction module 1308 is used to perform self-attention prediction based on each key vector and each query vector to obtain the predicted self-attention weight corresponding to each first sample embedding feature and the predicted self-attention weight corresponding to each second sample embedding feature.

[0233] The sample feature fusion module 1310 is used to fuse each first sample embedded feature and each second sample embedded feature according to their respective predicted self-attention weights to obtain the predicted palm features corresponding to multiple sample palm images.

[0234] The training module 1312 is used to obtain the label palm features corresponding to multiple sample palm images, perform model training based on the predicted differences between the predicted palm features and the label palm features, and obtain a feature extraction model.

[0235] In this embodiment, multiple sample palm images of a sample palm are obtained, and a first sample image region and a second sample image region other than the first sample image region are determined in each sample palm image. Each first sample image region includes a preset portion of the sample palm, thereby determining the same image region in the multiple sample palm images and the different image regions in the multiple sample palm images. A first sample embedding feature is obtained for each first sample image region, and each first sample embedding feature is mapped to a key space. The feature representations of the same first sample image regions can be mapped from the vector space to the key space to obtain a key vector for each first sample embedding feature. A second sample embedding feature is obtained for each second sample image region, and each second sample embedding feature is mapped to a query space. The feature representations of different first sample image regions can be mapped from the vector space to the query space to obtain a query vector for each second sample embedding feature. Based on the key vectors of the same sample image region and the query vectors of different sample image regions, the predicted self-attention weight corresponding to each first sample embedded feature and the predicted self-attention weight corresponding to each second sample embedded feature can be predicted. The predicted self-attention weight represents the importance of each key vector and each query vector in the fusion, that is, the model's attention to the features in each of the same sample image regions, thereby simultaneously considering the features in the same sample image region and the features in different sample image regions. By fusing the first sample embedded features and the second sample embedded features according to the predicted self-attention weights, the same features and different features in multiple sample palm images can be fused to obtain predicted palm features corresponding to the multiple sample palm images. The model's different attention to the features in the same sample image region makes the features extracted from the same sample image region not completely the same, thereby being able to extract different detail features existing in the same sample image region, thereby obtaining more feature information. The labeled palm features corresponding to multiple sample palm images are obtained. The labeled palm features are used as the palm features expected to be obtained in model training. Model training is performed based on the difference between the predicted palm features obtained by the model prediction and the expected palm features. The difference between the predicted palm features and the expected palm features is gradually reduced during training, thereby gradually improving the extraction accuracy of the model and finally obtaining a feature extraction model.

[0236] The final feature extraction model contains the self-attention weights obtained through training. The self-attention weights represent the degree of attention paid to the features of the same image areas and different image areas in multiple palm images, so that the features of the same image areas can be paid attention to, thereby extracting different detail features existing in the same image areas, thereby obtaining more feature information.

[0237] In one embodiment, the weight prediction module 1308 is further used to determine the key similarity between each key vector and the query similarity between each query vector; for each key vector, determine the content similarity between the key vector and each query vector; perform self-attention prediction based on each key similarity, each query similarity and each content similarity to obtain the predicted self-attention weight corresponding to each first sample embedding feature and the predicted self-attention weight corresponding to each second sample embedding feature.

[0238] In this embodiment, the key similarity between each key vector and the query similarity between each query vector are determined. The key similarity represents the degree of correlation between the key vectors. The higher the key similarity, the higher the degree of correlation between the key vectors. The query similarity represents the degree of correlation between the query vectors. The higher the query similarity, the higher the degree of correlation between the query vectors. For each key vector, the content similarity between the targeted key vector and each query vector is determined. The content similarity represents the degree of correlation between the key vector and the query vector. The higher the content similarity, the higher the degree of correlation between the key vector and the query vector. Self-attention prediction is performed based on each key similarity, each query similarity, and each content similarity. The predicted self-attention weight corresponding to each first sample embedded feature and the predicted self-attention weight corresponding to each second sample embedded feature are determined. This allows the corresponding weights to be accurately calculated based on the degree of correlation between the vectors, thereby obtaining the predicted self-attention weights of all features in multiple sample palm images.

[0239] In one embodiment, the key space mapping module 1304 is further configured to obtain an initial key weight, and based on the initial key weight, map each first sample embedding feature to the key space to obtain a key vector for each first sample embedding feature;

[0240] The query space mapping module 1306 is further configured to obtain an initial query weight, and map each second sample embedding feature to the query space based on the initial query weight to obtain a query vector for each second sample embedding feature.

[0241] In this embodiment, initial key weights are obtained, and based on the initial key weights, each first sample embedding feature is mapped to the key space, thereby accurately projecting each first sample image region from the vector space to the key space, and obtaining a key vector for each first sample embedding feature. Initial query weights are obtained, and based on the initial query weights, each second sample embedding feature is mapped to the query space, thereby accurately projecting each second sample image region from the vector space to the query space, and obtaining a query vector for each second sample embedding feature.

[0242] In one embodiment, the training module 1312 is further used to determine the predicted difference between the predicted palm feature and the labeled palm feature; when the predicted difference meets the preset iteration condition, determine the key similarity between each key vector, and adjust the initial key weight according to each key similarity to obtain an updated key weight; determine the query similarity between each query vector, adjust the initial query weight according to each query similarity, and obtain an updated query weight; use the updated key weight as the initial key weight of the next iteration, use the updated query weight as the initial query weight of the next iteration, return to the step of mapping each first sample embedding feature to the key space based on the initial key weight and continue executing until the determined predicted difference does not meet the preset iteration condition, and stop, thereby obtaining a feature extraction model.

[0243] In this embodiment, the predicted difference between the predicted palm features and the labeled palm features is determined, and the difference between the predicted palm features obtained by the model prediction and the expected palm features can be determined. When the predicted difference meets the preset iteration conditions, indicating that the next iteration is required, the key weight and query weight are first updated. That is, the key similarity between each key vector is determined. The key similarity represents the correlation between the key vectors. The higher the similarity, the higher the correlation. Therefore, the initial key weight can be adjusted according to each key similarity to obtain an updated key weight. The query similarity represents the correlation between the query vectors. Therefore, the initial query weight can be adjusted according to each query similarity to obtain an updated query weight. The updated key weight is used as the initial key weight of the next iteration, and the updated query weight is used as the initial query weight of the next iteration, so as to enter the next iteration and continue execution, so that the key weight and query weight are continuously updated in the iteration, and the predicted self-attention weight determined based on the key weight and query weight is also continuously updated, so that the difference between the predicted palm features and the expected palm features is gradually reduced in the iteration, until the determined prediction difference does not meet the preset iteration condition and stops. At this time, the final key weight, query weight and self-attention weight are obtained, thereby obtaining a feature extraction model.

[0244] In one embodiment, the apparatus further comprises:

[0245] a value space mapping module, configured to obtain an initial content weight and, based on the initial content weight, map each first sample embedding feature and each second sample embedding feature to the value space to obtain a content vector for each first sample embedding feature and a content vector for each second sample embedding feature;

[0246] The sample feature fusion module 1310 is further configured to fuse the content vectors of each first sample embedded feature and the content vectors of each second sample embedded feature according to their respective predicted self-attention weights to obtain predicted palm features corresponding to multiple sample palm images.

[0247] In this embodiment, initial content weights are obtained, and based on these initial content weights, each first sample embedded feature and each second sample embedded feature are mapped to a value space. This allows all embedded features of multiple sample palm images to be accurately mapped to the value space, thereby obtaining a content vector for each first sample embedded feature and a content vector for each second sample embedded feature. Based on the predicted self-attention weights, the content vectors of each first sample embedded feature and each second sample embedded feature are fused. This allows features of the same region in multiple sample palm images to be fused to varying degrees, resulting in different levels of attention to features of the same region in each sample palm image. This allows attention to be paid to the differences between features of the same region in multiple sample palm images, and the acquisition of these detailed features. Combined with features of other image regions that do not contain the preset region, the predicted palm features corresponding to the multiple sample palm images obtained contain more information, helping to improve the accuracy of palm feature recognition and verification in different scenarios.

[0248] In one embodiment, the training module 1312 is further used to determine, for each key vector, the content similarity between the key vector and each query vector when the predicted difference meets the preset iteration condition; adjust the initial content weight according to each content similarity to obtain an updated content weight; use the updated content weight as the initial content weight for the next iteration, and execute the step of returning to the initial key weight to map each first sample embedding feature to the key space.

[0249] In this embodiment, when the predicted difference meets the preset iteration condition, for each key vector, the content similarity between the key vector and each query vector is determined, and the content similarity characterizes the degree of correlation between the key vector and the query vector, so that the initial content weight is adjusted according to each content similarity, and an updated content weight is obtained. The updated content weight is used as the initial content weight for the next iteration, and the return is executed based on the initial key weight to continuously update the content weight in the iteration, thereby gradually reducing the difference between the predicted palm features and the expected palm features in the iteration, thereby gradually improving the extraction accuracy of the model, and finally obtaining a feature extraction model.

[0250] In one embodiment, the sample acquisition module 1302 is further configured to extract feature points of each sample palm image respectively; identify a preset part in each sample palm image based on the feature points of each sample palm image, and determine a first sample image region in which the preset part is located in each sample palm image; and determine, for each sample palm image, a second sample image region in the sample palm image other than the corresponding first sample image region.

[0251] In this embodiment, feature points are extracted from each sample palm image. Based on the feature points of each sample palm image, a predetermined portion of each sample palm image is identified, and a first sample image region within each sample palm image where the predetermined portion is located is determined. This allows for the identification of image regions containing the same predetermined portion across multiple sample palm images. These image regions contain identical or very similar features, facilitating further extraction of detailed features based on the identical or similar features, thereby improving the accuracy of detailed feature acquisition and identification. For each sample palm image, a second sample image region, excluding the corresponding first sample image region, is determined within the sample palm image, facilitating subsequent different processing for sample image regions containing the predetermined portion and sample image regions not containing the predetermined portion.

[0252] Based on the same inventive concept, the present application also provides a feature extraction device for implementing the feature extraction method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations in one or more feature extraction device embodiments provided below can be found in the limitations of the feature extraction method described above and will not be repeated here.

[0253] In one embodiment, as shown in FIG14 , a feature extraction device 1400 is provided, comprising:

[0254] The image acquisition module 1402 is configured to acquire multiple palm images.

[0255] The region determination module 1404 is used to obtain the feature extraction model trained by the feature extraction model processing device, and determine the first image region and the second image region other than the first image region in each palm image, where each first image region includes a preset part of the palm.

[0256] The feature acquisition module 1406 is used to acquire the first embedded features of each first image region and the second embedded features of each second image region through the feature extraction model.

[0257] The feature fusion module 1408 is used to obtain the palm features corresponding to multiple palm images through the feature extraction model, and fuse each first embedded feature with each second embedded feature according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model, so as to obtain the palm features corresponding to multiple palm images. The self-attention weight is obtained through training of the feature extraction model.

[0258] In this embodiment, a feature extraction model trained through a feature extraction model processing method includes multiple self-attention weights, each of which represents the degree to which the feature extraction model focuses on features of identical image regions and different image regions in multiple palm images. Multiple palm images are obtained, and a first image region and a second image region other than the first image region are determined within each palm image using a feature extraction model. Each first image region includes a predetermined portion of the palm, thereby determining identical image regions and different image regions within the multiple palm images. The feature extraction model obtains a first embedded feature for each first image region and a second embedded feature for each second image region, thereby transforming each image region from image space to vector space to obtain a corresponding embedded feature. The feature extraction model fuses the first and second embedded features according to their self-attention weights within the feature extraction model, thereby focusing not only on features of different image regions but also on features of identical image regions. This allows for the extraction of different detail features within the same image region, resulting in a greater number of feature quantities in the obtained palm features. Moreover, the extracted palm features contain more detailed features, which can improve the recognition accuracy when using palm features for feature recognition in different scenarios.

[0259] In one embodiment, the feature extraction model further includes content weights obtained through training; and the apparatus further includes:

[0260] a mapping module, configured to map each first embedding feature and each second embedding feature to a value space based on a content weight, to obtain a content vector for each first embedding feature and a content vector for each second embedding feature;

[0261] The feature fusion module is also used to fuse the content vector of each first embedded feature and the content vector of each second embedded feature through the feature extraction model according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model, so as to obtain the palm features corresponding to multiple palm images.

[0262] In this embodiment, through a feature extraction model, each first embedded feature and each second embedded feature is mapped to a value space based on content weights. This allows all embedded features of multiple palm images to be accurately mapped to the value space to obtain a content vector for each first embedded feature and a content vector for each second embedded feature. Based on their respective attention weights, the content vectors of each first embedded feature and each second embedded feature are fused, thereby enabling the features of the same part in multiple palm images to be fused to varying degrees. This results in different levels of attention being paid to the features of the same part in each palm image, thereby focusing on the differences between the features of the same part in multiple palm images and obtaining these detailed features. Combined with the features of other image regions that do not contain the preset part, the palm features corresponding to the multiple palm images obtained contain more information, helping to improve the accuracy of palm feature recognition and verification in different scenarios.

[0263] In one embodiment, the image acquisition module is further configured to, in response to a resource transfer trigger operation, call a camera to shoot the palm of the object to be verified at different angles to obtain multiple palm images;

[0264] The device also includes:

[0265] A verification module, used to authenticate the identity of the object to be verified based on palm features and obtain a verification result;

[0266] The resource transfer module is used to obtain resource transfer information when the verification result indicates that the identity authentication of the object to be verified is successful, and perform a resource transfer operation based on the resource transfer information.

[0267] In this embodiment, the feature extraction method is applied to the resource transfer scenario, so that in response to the resource transfer trigger operation, the camera is called to shoot the palm of the object to be verified at different angles to obtain multiple palm images, and then the palm features of the multiple palm images are obtained through the feature extraction model. The obtained palm features include features with detailed differences in the same parts of the multiple palm images, so that the identity authentication of the object to be verified based on the palm features can be more accurate, and the resource transfer operation is performed based on the resource transfer information, which can effectively improve the security of the resource transfer.

[0268] The feature extraction model processing device and each module in the feature extraction device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0269] In one embodiment, a computer device is provided, which may be a terminal or a server. Taking a terminal as an example, its internal structure diagram may be shown in Figure 15. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via wired or wireless communication, where the wireless communication may be achieved via Wi-Fi, a mobile cellular network, NFC (near field communication), or other technologies. When executed by the processor, the computer program implements a feature extraction model processing method and a feature extraction method. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0270] Those skilled in the art will understand that the structure shown in FIG15 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0271] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0272] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0273] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0274] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0275] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0276] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0277] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A feature extraction model processing method, executed by a computer device, the method comprising: Acquire multiple sample palm images of a sample palm, determine a first sample image area in each of the sample palm images and a second sample image area other than the first sample image area, each of the first sample image areas includes a preset part of the sample palm; Obtaining first sample embedding features of respective first sample image regions, and mapping the first sample embedding features to key spaces to obtain key vectors of each first sample embedding feature; Obtaining respective second sample embedding features of respective second sample image regions, and mapping respective second sample embedding features to a query space to obtain a query vector of each second sample embedding feature; Performing self-attention prediction according to each of the key vectors and each of the query vectors to obtain a predicted self-attention weight corresponding to each of the first sample embedding features and a predicted self-attention weight corresponding to each of the second sample embedding features; fusing each of the first sample embedding features and each of the second sample embedding features according to their respective predicted self-attention weights to obtain predicted palm features commonly corresponding to the multiple sample palm images; and The labeled palm features commonly corresponding to the multiple sample palm images are obtained, and model training is performed based on the predicted difference between the predicted palm features and the labeled palm features to obtain a feature extraction model.

2. The method according to claim 1, wherein the step of performing self-attention prediction according to each of the key vectors and each of the query vectors to obtain a predicted self-attention weight corresponding to each of the first sample embedding features and a predicted self-attention weight corresponding to each of the second sample embedding features comprises: Determining key similarities between the key vectors and query similarities between the query vectors; For each of the key vectors, determining the content similarity between the key vector and each of the query vectors; and Self-attention prediction is performed according to each of the key similarities, each of the query similarities, and each of the content similarities to obtain a predicted self-attention weight corresponding to each of the first sample embedding features and a predicted self-attention weight corresponding to each of the second sample embedding features.

3. The method according to claim 1 or 2, wherein mapping each of the first sample embedding features to a key space to obtain a key vector of each of the first sample embedding features comprises: Acquire initial key weights, and based on the initial key weights, map each of the first sample embedding features to a key space to obtain a key vector of each of the first sample embedding features; and The mapping each of the second sample embedding features to the query space to obtain a query vector of each of the second sample embedding features includes: An initial query weight is obtained, and each of the second sample embedding features is respectively mapped to a query space based on the initial query weight to obtain a query vector of each of the second sample embedding features.

4. The method according to claim 3, wherein the model training is performed based on the predicted difference between the predicted palm features and the labeled palm features to obtain a feature extraction model, comprising: determining a predicted difference of the predicted palm feature relative to the labeled palm feature; When the predicted difference satisfies a preset iteration condition, determining the key similarity between each of the key vectors, and adjusting the initial key weight according to each of the key similarities to obtain an updated key weight; Determining query similarities between the query vectors, adjusting the initial query weight according to the query similarities, and obtaining an updated query weight; and The updated key weight is used as the initial key weight of the next iteration, and the updated query weight is used as the initial query weight of the next iteration. Return to the step of mapping each of the first sample embedding features to the key space based on the initial key weight and continue to execute until the determined prediction difference does not meet the preset iteration condition, and stop to obtain a feature extraction model.

5. The method according to claim 4, further comprising: Obtain an initial content weight, and embed each of the first samples into features and each of the first samples into features based on the initial content weight. The two sample embedding features are respectively mapped to the value space to obtain a content vector of each first sample embedding feature and a content vector of each second sample embedding feature; and The step of fusing each of the first sample embedding features and each of the second sample embedding features according to their respective predicted self-attention weights to obtain predicted palm features corresponding to the multiple sample palm images includes: The content vectors of each of the first sample embedded features and the content vectors of each of the second sample embedded features are fused according to their respective predicted self-attention weights to obtain predicted palm features that correspond to the multiple sample palm images.

6. The method according to claim 5, further comprising: When the predicted difference satisfies a preset iteration condition, for each of the key vectors, determining the content similarity between the key vector and each of the query vectors; Adjust the initial content weight according to each content similarity to obtain an updated content weight; and The updated content weight is used as the initial content weight of the next iteration, and the step of returning to the step of mapping each of the first sample embedding features to the key space based on the initial key weight is executed.

7. The method according to any one of claims 1 to 6, wherein determining a first sample image region and a second sample image region other than the first sample image region in each of the sample palm images comprises: Extracting feature points of each sample palm image respectively; Based on the characteristic points of each of the sample palm images, a preset part in each of the sample palm images is identified, and a first sample image region in which the preset part in each of the sample palm images is located is determined; and For each of the sample palm images, a second sample image region other than the first sample image region in the sample palm image is determined.

8. A feature extraction method, performed by a computer device, comprising: acquiring a plurality of palm images of the palm; The feature extraction model obtained by training the feature extraction model processing method according to any one of claims 1 to 7 is used to determine a first image area and a second image area other than the first image area in each of the palm images, wherein each of the first image areas includes a preset part of the palm; Acquire, by means of the feature extraction model, a first embedding feature of each of the first image regions and a second embedding feature of each of the second image regions; and Through the feature extraction model, each of the first embedded features and each of the second embedded features are fused according to the self-attention weight for each of the first embedded features and the self-attention weight for each of the second embedded features in the feature extraction model to obtain the palm features corresponding to the multiple palm images, and the self-attention weight is obtained through training of the feature extraction model.

9. The method according to claim 8, wherein the feature extraction model further comprises content weights obtained through training; and the method further comprises: Mapping each of the first embedded features and each of the second embedded features to a value space based on the content weight through the feature extraction model to obtain a content vector for each of the first embedded features and a content vector for each of the second embedded features; and The step of fusing each of the first embedded features and each of the second embedded features according to the self-attention weight for each of the first embedded features and the self-attention weight for each of the second embedded features in the feature extraction model to obtain a palm feature commonly corresponding to the multiple palm images includes: Through the feature extraction model, according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model, the content vector of each first embedded feature and the content vector of each second embedded feature are fused to obtain the palm features corresponding to the multiple palm images.

10. The method according to any one of claims 8 to 9, wherein the feature extraction method is applied to a resource transfer scenario; The step of acquiring a plurality of palm images of the palm comprises: In response to the resource transfer trigger operation, calling a camera to shoot the palm of the object to be verified at different angles to obtain multiple palm images; The method further comprises: Performing identity authentication on the object to be authenticated according to the palm feature to obtain a verification result; and When the verification result indicates that the identity authentication of the object to be verified is successful, resource transfer information is obtained, and a resource transfer operation is performed based on the resource transfer information.

11. A feature extraction model processing device, the device comprising: A sample acquisition module, used to acquire a plurality of sample palm images of a sample palm, and determine a first sample image region and a second sample image region other than the first sample image region in each of the sample palm images, wherein each of the first sample image regions includes a preset portion of the sample palm; A key space mapping module, used for obtaining the first sample embedding features of each of the first sample image regions, and mapping each of the first sample embedding features to the key space to obtain a key vector of each of the first sample embedding features; A query space mapping module, used to obtain the second sample embedding features of each second sample image region, and map each second sample embedding feature to the query space to obtain a query vector of each second sample embedding feature; A weight prediction module, configured to perform self-attention prediction according to each of the key vectors and each of the query vectors, and obtain a predicted self-attention weight corresponding to each of the first sample embedding features and a predicted self-attention weight corresponding to each of the second sample embedding features; A sample feature fusion module, used for fusing each of the first sample embedding features and each of the second sample embedding features according to their respective predicted self-attention weights, to obtain a predicted palm feature commonly corresponding to the multiple sample palm images; and The training module is used to obtain the label palm features commonly corresponding to the multiple sample palm images, perform model training based on the predicted difference between the predicted palm features and the label palm features, and obtain a feature extraction model.

12. A feature extraction device, comprising: An image acquisition module, used for acquiring a plurality of palm images of the palm; a region determination module, configured to obtain the feature extraction model trained by the feature extraction model processing device according to claim 11, and determine a first image region in each of the palm images and a second image region other than the first image region, wherein each of the first image regions includes a preset part of the palm; A feature acquisition module, used for acquiring a first embedded feature of each of the first image regions and a second embedded feature of each of the second image regions through the feature extraction model; and A feature fusion module is used to obtain the palm features corresponding to the multiple palm images through the feature extraction model, according to the self-attention weight for each first embedded feature and the self-attention weight for each second embedded feature in the feature extraction model, and fuse the first embedded features with the second embedded features to obtain the palm features corresponding to the multiple palm images, wherein the self-attention weight is obtained through training of the feature extraction model.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 10 when executing the computer program.

14. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

15. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images