Identity recognition model training and electronic resource transfer method, device and equipment

Abstract

The present application discloses a method, device and equipment for identity recognition model training and electronic resource transfer, which relates to the field of electronic resource transfer technology, and is used to improve the identity recognition rate during electronic resource transfer and reduce the probability of error during electronic resource transfer. The embodiments of the present application can be applied to various scenarios such as cloud technology, artificial intelligence and smart transportation. In this method, multimodal learning is performed on palm images and audio data, and the learned model can predict the palm image and audio data respectively, and fuse the prediction results to obtain the final identity recognition result to improve the accuracy of identity recognition. In addition, during training, the final identity recognition result is obtained through palm images and audio data, and the entire model is adjusted based on this identity recognition result. In this way, the palm recognition sub-model and the audio recognition sub-model can both be learned based on palm images and audio data, thereby further improving the identity recognition accuracy of the entire model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to the field of electronic resource transfer technology, and provides an identity recognition model training and electronic resource transfer method, device and equipment. Background Art

[0002] With the development of network technology, the way of electronic resource transfer can make it more convenient for the operation objects to exchange items, making it more widely used, and the types of electronic resource transfer are becoming more and more diverse.

[0003] Taking the transfer of electronic resources as an example, the ways to realize electronic payment are becoming more and more abundant, such as the relatively mature scanning code payment, face payment and fingerprint payment, etc., and new electronic payment methods are gradually emerging, such as the latest proposed palm payment technology. Palm payment refers to a mechanism that can complete the confirmation and payment of the payment object's identity by scanning the palm. It only needs to use the camera to take a photo of the palm in advance to complete the authorization of palm payment. When making a payment, the palm swiping device collects the palm image of the payment object, and identifies the palm features based on the palm image to identify the identity of the payment object. There is no need to use a credit card and a smart terminal when paying, which is more convenient.

[0004] However, when paying by palm swiping, the identity of the payee may be misidentified, resulting in an error in the transfer of electronic resources. For example, after the palm swiping device captures the palm image of object B, it misidentifies it as the palm of object A, resulting in the amount consumed by object B being deducted from the payment account of object A.

[0005] It can be seen that accurate identification of the payment object's identity is the basis of palm payment. Therefore, how to reduce the error rate of payment object identity recognition is an issue worth considering. Summary of the invention

[0006] The embodiments of the present application provide a method, device and equipment for identity recognition model training and electronic resource transfer, which are used to improve the identity recognition rate during electronic resource transfer and reduce the error probability during electronic resource transfer.

[0007] On the one hand, a method for training an identity recognition model is provided, wherein the identity recognition model includes a palm recognition sub-model, an audio recognition sub-model and a fusion sub-model, and the method includes:

[0008] Inputting each training sample into the palm recognition sub-model to obtain the predicted palm identity corresponding to the palm image included in each training sample;

[0009] Inputting each of the training samples into the audio recognition sub-model to obtain the audio prediction identity corresponding to the audio data included in each of the training samples;

[0010] Using the fusion sub-model, based on the palm predicted identity and the audio predicted identity corresponding to each of the training samples, respectively determine the identity recognition results corresponding to each of the training samples;

[0011] Based on the obtained identity recognition results and the total loss values ​​corresponding to the corresponding real identity labels, parameters of the palm recognition sub-model, the audio recognition sub-model and the fusion sub-model are adjusted.

[0012] On the one hand, a method for transferring electronic resources based on the identity recognition model trained by the above method is provided, the method comprising:

[0013] receiving an electronic resource transfer request, the electronic resource transfer request being used to transfer a specified amount of electronic resources owned by a resource transfer object to a resource receiving object, and the electronic resource transfer request carrying a palm image and audio data of the resource transfer object;

[0014] Inputting the palm image and audio data into the identity recognition model to obtain an identity recognition result of the resource transfer object;

[0015] Based on the identity recognition result, the specified amount of electronic resources is transferred to the resource recipient.

[0016] In one aspect, an identity recognition model training device is provided, wherein the identity recognition model includes a palm recognition sub-model, an audio recognition sub-model and a fusion sub-model, and the device includes:

[0017] An image recognition unit, used for inputting each training sample into the palm recognition sub-model to obtain the predicted palm identity corresponding to the palm image included in each training sample;

[0018] An audio recognition unit, used to input each of the training samples into the audio recognition sub-model, and obtain the audio prediction identity corresponding to the audio data respectively included in each of the training samples;

[0019] A fusion prediction unit, configured to use the fusion sub-model to determine the identity recognition results corresponding to each of the training samples based on the palm prediction identity and the audio prediction identity corresponding to each of the training samples;

[0020] The model adjustment unit is used to adjust the parameters of the palm recognition sub-model, the audio recognition sub-model and the fusion sub-model based on the obtained identity recognition results and the total loss value corresponding to the corresponding real identity label.

[0021] Optionally, each training sample is annotated with a real identity label corresponding to the palm image and audio data included therein; then the device further comprises a model loss determination unit, which is used to:

[0022] Based on the obtained predicted palm identity and the corresponding true identity label, obtaining a palm recognition loss value of the palm recognition sub-model;

[0023] Based on the obtained audio prediction identity and the corresponding true identity label, obtaining the audio recognition loss value of the audio recognition sub-model;

[0024] Based on the individual identity recognition results and the true identity labels corresponding to the corresponding training samples, a sample recognition loss value is obtained;

[0025] The total loss value is obtained based on the palm recognition loss value, the audio recognition loss value and the sample recognition loss value.

[0026] Optionally, each training sample includes a positive sample and a negative sample, the palm image and the audio data included in each positive sample have the same real identity label, and the palm image and the audio data included in each negative sample have different real identity labels; then the model loss determination unit is further used to:

[0027] Compare the identity recognition results corresponding to each positive sample with the true identity label to obtain corresponding first comparison results respectively;

[0028] Compare the identity recognition results corresponding to each negative sample with the set recognition failure results to obtain corresponding second comparison results respectively; wherein the recognition failure results are used to indicate identity recognition failure;

[0029] Based on the first comparison results and the second comparison results, the sample identification loss value is obtained.

[0030] Optionally, the palm recognition sub-model includes two convolution sub-networks with different convolution kernel sizes, a region generation sub-network, a region pooling layer and a fully connected layer; then the image recognition unit is specifically used for:

[0031] For each training sample, perform the following operations respectively:

[0032] For a training sample, the palm image is convolved by the two convolution sub-networks to obtain two corresponding convolution feature maps, and the region of interest is extracted by the region generation sub-network to obtain at least one region feature map;

[0033] Performing pooling processing on the at least one regional feature map through the regional pooling layer to obtain a pooling feature map;

[0034] The obtained pooled feature map is subjected to feature mapping processing through the fully connected layer, and a corresponding palm prediction identity is determined based on the obtained palm prediction vector.

[0035] Optionally, the image recognition unit is further used to:

[0036] For each palm image included in each training sample, the following operations are performed respectively:

[0037] For a palm image, the palm image is divided into regions to obtain a plurality of corresponding palm sub-regions.

[0038] image;

[0039] Respectively extracting directional gradient histogram features corresponding to each of the plurality of palm sub-images;

[0040] Performing splicing processing on the obtained multiple directional gradient histogram features to obtain a palm basic feature vector of the palm image;

[0041] The palm basic feature vectors corresponding to the respective palm images are input into the palm recognition sub-model.

[0042] Optionally, the audio recognition unit is further used for:

[0043] For each audio data included in each training sample, the following operations are performed respectively:

[0044] For a piece of audio data, dividing the piece of audio data into frames to obtain a corresponding plurality of audio frames;

[0045] Extracting Mel-frequency cepstrum features corresponding to each of the multiple audio frames respectively;

[0046] Performing splicing processing on the obtained multiple Mel-frequency cepstrum features to obtain an audio basic feature vector of the one audio data;

[0047] Then, each training sample is input into the audio recognition sub-model, including:

[0048] The audio basic feature vectors corresponding to each of the audio data are input into the audio recognition sub-model.

[0049] Optionally, the audio recognition unit is specifically used to:

[0050] The audio data is divided into frames in a cross-division manner, so that adjacent audio frames in the multiple audio frames have overlapping parts.

[0051] Optionally, the audio recognition unit is further used for:

[0052] Based on the respective audio data, an audio frame filtering model is trained to obtain a trained audio frame filtering model; wherein the audio frame filtering model has the same model structure as the audio recognition sub-model;

[0053] For each of the audio data, perform the following operations respectively:

[0054] For an audio data, a plurality of audio frames included in the audio data are input into the trained audio

[0055] A frame filtering model is used to obtain an audio frame identification identity corresponding to each of the plurality of audio frames;

[0056] If there is an audio frame whose corresponding audio frame identification identity is inconsistent with the real identity tag of the one audio data, filtering the one audio frame from the one audio data;

[0057] The filtered audio data are input into the audio recognition sub-model.

[0058] Optionally, the device further comprises a pre-training unit, configured to:

[0059] Pre-training the palm recognition sub-model using a plurality of palm images to obtain a first model parameter of the palm recognition sub-model when a convergence condition is satisfied; and pre-training the audio recognition sub-model using a plurality of audio data to obtain a second model parameter of the audio recognition sub-model when a convergence condition is satisfied;

[0060] Then the image recognition unit is specifically used to initialize the palm recognition sub-model based on the first model parameters, and input each training sample into the initialized palm recognition sub-model;

[0061] The audio recognition unit is specifically used to initialize the audio recognition sub-model based on the second model parameters, and input each training sample into the initialized audio recognition sub-model.

[0062] In one aspect, an electronic resource transfer device is provided, the device comprising:

[0063] A receiving unit, configured to receive an electronic resource transfer request, wherein the electronic resource transfer request is used to transfer a specified number of electronic resources owned by a resource transfer object to a resource receiving object, and the electronic resource transfer request carries a palm image and audio data of the resource transfer object;

[0064] An identity recognition unit, used for inputting the palm image and audio data into an identity recognition model trained based on any one of the methods of claims 1 to 9 to obtain an identity recognition result of the resource transfer object;

[0065] The resource transfer unit is used to transfer the specified number of electronic resources to the resource receiving object based on the identity recognition result.

[0066] On the one hand, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.

[0067] In one aspect, a computer storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the steps of any of the above methods are implemented.

[0068] In one aspect, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above methods.

[0069] In the embodiment of the present application, the identity recognition model includes a palm recognition sub-model and an audio recognition sub-model. During training, the palm recognition sub-model and the audio recognition sub-model respectively recognize the palm image and audio data in the training sample, obtain the corresponding palm predicted identity and audio predicted identity, and fuse the palm predicted identity and audio predicted identity through the fusion sub-model to determine the identity recognition result of each training sample, and each training sample is also marked with a corresponding real identity label, so that the model can be adjusted based on these identity recognition results and the real identity labels of each training sample, so that the predetermined accuracy can be achieved when the model converges. It can be seen that the embodiment of the present application uses palm images and audio data as training data for multimodal learning. The learned model can predict palm images and audio data respectively, and fuses the prediction results of the two to obtain the final identity recognition result to improve the accuracy of identity recognition. In addition, during training, the final identity recognition result is obtained through palm images and audio data, and the entire model is adjusted based on this identity recognition result. In this way, both the palm recognition sub-model and the audio recognition sub-model can learn based on palm images and audio data, thereby further improving the identity recognition accuracy of the entire model. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0071] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0072] Figure 2 A schematic diagram of the architecture of the identity recognition model provided in the embodiment of the present application;

[0073] Figure 3 A possible structural diagram of a palm recognition sub-model provided in an embodiment of the present application;

[0074] Figure 4 A possible structural diagram of an audio recognition sub-model provided in an embodiment of the present application;

[0075] Figure 5 A flowchart of the palm image training data processing process provided in the embodiment of the present application;

[0076] Figure 6 A schematic diagram of an image feature extraction process provided in an embodiment of the present application;

[0077] Figure 7 A flowchart of the audio training data processing process provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of dividing an audio data frame provided in an embodiment of the present application;

[0079] Fig. 9 A schematic diagram of a process for extracting mel spectrum features of an audio frame provided in an embodiment of the present application;

[0080] Fig.10 A flowchart of an identity recognition model training method provided in an embodiment of the present application;

[0081] Fig.11 A schematic diagram of the process of obtaining a predicted palm identity by a palm recognition sub-model provided in an embodiment of the present application;

[0082] Fig.12 A schematic diagram of a flow chart of an electronic resource transfer method provided in an embodiment of the present application;

[0083] Fig.13 A schematic diagram of the structure of an identity recognition model training device provided in an embodiment of the present application;

[0084] Fig.14 A schematic diagram of a structure of an electronic resource transfer device provided in an embodiment of the present application;

[0085] Fig.15 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0087] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0088] Electronic resource transfer: refers to the transfer of electronic resources to complete a specific transaction. For example, payee A uses his own account to log in to device a at 13:48 on March 21, 2021, and pays 5 yuan to the merchant to buy a pound of apples. In this process, the transfer of electronic resources is involved. Among them, device a is an electronic device used by payee A. The electronic device can be a personal computer, mobile phone, tablet computer, notebook, e-book reader, smart home, and other computer devices with certain computing capabilities and supporting electronic payment.

[0089] Supervised learning: is a machine learning task that infers a function from a labeled training data set, which consists of a set of training examples. In supervised learning, each example is a pair consisting of an input object (usually a vector) and a desired output value.

[0090] The embodiments of the present application relate to artificial intelligence and machine learning technologies, and are mainly designed based on machine learning in artificial intelligence.

[0091] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0092] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0093] Computer vision technology (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform image processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, autonomous driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0094] The key technologies of speech technology include automatic speech recognition technology, speech synthesis technology and voiceprint recognition technology. Enabling computers to listen, see, speak and feel is the development direction of human-computer interaction in the future, among which speech has become one of the most promising human-computer interaction methods in the future.

[0095] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0096] Machine learning is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0097] Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning and other technologies. Artificial neural networks (ANN) abstract the human brain neural network from the perspective of information processing, establish a simple model, and form different networks according to different connection methods. A neural network is a computational model composed of a large number of nodes (or neurons) connected to each other. Each node represents a specific output function, called an activation function. The connection between each two nodes represents a weighted value for the signal passing through the connection, called a weight, which is equivalent to the memory of the artificial neural network. The output of the network varies depending on the network connection method, weight value and activation function. The network itself is usually an approximation of a certain algorithm or function in nature, or it may be an expression of a logical strategy.

[0098] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0099] The solution provided by the embodiment of the present application involves artificial intelligence machine learning, voice technology, computer vision technology and other technologies. The embodiment of the present application uses voice technology and computer vision technology to extract palm images and voice data from the operation object, and uses these palm images and voice data to learn the identity recognition model using machine learning technology, and then in the actual process of electronic resource transfer, the learned identity recognition model can be used to identify the palm image and voice data of the resource transfer object collected, so as to obtain the identity recognition result of the resource transfer object, such as a payment account, and use the identity recognition result to complete the electronic resource transfer, such as using the recognized payment account to pay the merchant.

[0100] Specifically, the artificial neural network model based on deep learning is adopted in the embodiment of the present application, which can be divided into two parts, including a training part and an application part. Among them, the training part involves the technical field of machine learning. In the training part, the artificial neural network model (such as the identity recognition model) is trained by the machine learning technology, so that the artificial neural network model is trained based on the multiple training sample pairs and training methods given in the embodiment of the present application, and the model parameters are continuously adjusted through the optimization algorithm until the model converges; the application part is used to use the artificial neural network model trained in the training part to identify the palm image and voice data of the resource transfer object, obtain the corresponding identity recognition result, and use the identity recognition result to complete the process of electronic resource transfer. In addition, it should be noted that the artificial neural network model in the embodiment of the present application can be either online training or offline training, which is not specifically limited here. In this article, offline training is used as an example for illustration.

[0101] The following is a brief introduction to the design concept of the embodiments of the present application.

[0102] At present, the application of electronic resource transfer is becoming increasingly popular.

[0103] Taking the palm-swiping payment technology in the electronic resource transfer as an example, it refers to a mechanism that can complete the identity confirmation and payment of the payment object by scanning the palm.

[0104] However, currently when performing palm swiping payment, if identity misrecognition occurs, it will lead to errors in the transfer of electronic resources, causing economic losses to the payee, so there is a need to further improve the accuracy of payment object identity recognition.

[0105] In view of this, an embodiment of the present application provides an identity recognition model training method, in which the identity recognition model trained includes a palm recognition sub-model and an audio recognition sub-model, so that when training, the palm recognition sub-model and the audio recognition sub-model respectively recognize the palm image and audio data in the training sample, obtain the corresponding palm predicted identity and audio predicted identity, and fuse the palm predicted identity and audio predicted identity through the fusion sub-model to determine the identity recognition result of each training sample, and each training sample is also marked with a corresponding real identity label, so that the model can be adjusted based on these identity recognition results and the real identity labels of each training sample until the model converges. Among them, by using palm images and audio data as training data for multimodal learning, the learned model can predict the palm image and audio data respectively, and fuse the prediction results of the two to obtain the final identity recognition result, so as to improve the accuracy of identity recognition. In addition, during training, the final identity recognition result is obtained through palm images and audio data, and the entire model is adjusted based on this identity recognition result. In this way, both the palm recognition sub-model and the audio recognition sub-model can learn based on palm images and audio data, thereby further improving the identity recognition accuracy of the entire model.

[0106] Furthermore, in the actual process of electronic resource transfer, the identity recognition model obtained through the above training can be used to make predictions based on the palm image and audio data of the resource transfer object respectively, and the prediction results of the two can be integrated to obtain the final identity recognition result, thereby accurately identifying the identity of the resource transfer object, improving the accuracy of the electronic resource transfer process, and ensuring the security of the electronic resources of each payment object.

[0107] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0108] The solution provided in the embodiments of the present application can be applied to various scenarios such as cloud technology, artificial intelligence, and smart transportation, for example, electronic resource transfer scenarios, such as electronic payment scenarios, etc. Figure 1 As shown, it is a schematic diagram of an application scenario provided by an embodiment of the present application. In this scenario, a resource transfer terminal device 101 and a resource transfer server 102 may be included.

[0109] In one implementation, the resource transfer terminal device 101 may be, for example, a device owned by the resource transfer object itself, such as a mobile phone, a tablet computer (PAD), a laptop computer, a desktop computer, a smart TV, a smart vehicle-mounted device, and a smart wearable device, etc. The resource transfer terminal device 101 may be installed with an electronic resource transfer application, and the application involved in the embodiment of the present application may be a software client, or a client such as a web page or a small program, and the server is a background server corresponding to the software or web page, small program, etc., without limiting the specific type of the client.

[0110] In another embodiment, the resource transfer terminal device 101 may also be a device owned by the resource recipient, such as a palm payment device in a store, and may also be installed with an electronic resource transfer application.

[0111] Regardless of the type of terminal device the resource transfer terminal device 101 is, it includes a data collection device, such as a camera and a microphone, which are used to collect palm images and audio data respectively.

[0112] The resource transfer server 102 may be a background server corresponding to the electronic resource transfer application installed on the resource transfer terminal device 101, which may provide background service functions of the electronic resource transfer system, such as implementing the training method of the identity recognition model and the steps of the electronic resource transfer method provided in the embodiment of the present application. The resource transfer server 102 may be, for example, an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, i.e., content delivery networks (CDNs), as well as big data and artificial intelligence platforms, but is not limited thereto.

[0113] In one embodiment, the resource transfer server 102 may include one or more processors 1021, a memory 1022, and an I / O interface 1023 for interacting with a terminal. In addition, the resource transfer server 102 may also be configured with a database 1024, which may be used to store identity feature vectors of each resource transfer object and trained model parameters. Among them, the memory 1022 of the resource transfer server 102 may also store program instructions for the training method of the identity recognition model and the electronic resource transfer method provided in the embodiment of the present application, which, when executed by the processor 1021, can be used to implement the steps of the training method of the identity recognition model and the electronic resource transfer method provided in the embodiment of the present application, so as to implement the training process of the identity recognition model and execute the electronic resource transfer process based on the trained identity recognition model.

[0114] In one embodiment, the training process of the identity recognition model can be performed by the resource transfer server 102 to quickly implement the training of the identity recognition model by using the computing resources of the resource transfer server 102, and the electronic resource transfer method can also be performed with the participation of the resource transfer terminal device 101. For example, the resource transfer terminal device 101 uses the feature extraction part in the trained identity recognition model to extract features from the collected palm image and audio data, so as to transmit the extracted feature vectors to the resource transfer server 102, and then the resource transfer server 102 can perform retrieval based on these feature vectors to obtain the identity recognition result. Alternatively, after the resource transfer terminal device 101 performs feature extraction to obtain the feature vectors of the palm image and audio data, the original data of the palm image and audio data and the feature vectors of the palm image and audio data can also be encrypted and transmitted to the resource transfer server 102, and then the resource transfer server 102 can use the feature extraction part in the trained identity recognition model to extract features based on the original data, and compare the extracted feature vectors with the feature vectors sent by the resource transfer terminal device 101 to determine whether the original data has been tampered with, thereby ensuring the security of the electronic resource transfer data transmission.

[0115] The resource transfer terminal device 101 and the resource transfer server 102 may be directly or indirectly connected to each other through one or more networks 103. The network 103 may be a wired network or a wireless network, for example, a mobile cellular network or a wireless fidelity (Wireless-Fidelity, WIFI) network, or other possible networks, which are not limited in the embodiment of the present invention.

[0116] It should be noted that in the embodiment of the present application, the number of resource transfer terminal devices 101 can be one or more, and similarly, the number of resource transfer servers 102 can be one or more, that is, there is no restriction on the number of resource transfer terminal devices 101 or resource transfer servers 102.

[0117] In a possible application scenario, the relevant data (such as feature vectors, etc.) and model parameters involved in the embodiments of the present application can be stored using cloud storage technology. Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (or storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0118] In a possible application scenario, in order to reduce the communication delay of retrieval, resource transfer servers 102 can be deployed in various regions, or for load balancing, different resource transfer servers 102 can serve resource transfer terminal devices 101 in different regions respectively. For example, the resource transfer terminal device 101 is located at location a and establishes a communication connection with the resource transfer server 102 serving location a. The resource transfer terminal device 101 is located at location b and establishes a communication connection with the resource transfer server 102 serving location b. Multiple resource transfer servers 102 form a data sharing system, and data sharing is achieved through blockchain.

[0119] Each resource transfer server 102 in the data sharing system has a node identifier corresponding to the resource transfer server 102. Each resource transfer server 102 in the data sharing system can store the node identifiers of other resource transfer servers 102 in the data sharing system, so that the generated blocks can be broadcast to other resource transfer servers 102 in the data sharing system according to the node identifiers of other resource transfer servers 102. A node identifier list can be maintained in each resource transfer server 102, and the resource transfer server 102 name and the node identifier are stored in the node identifier list. The node identifier can be an Internet Protocol (IP) address for interconnection between networks and any other information that can be used to identify the node.

[0120] Of course, the method provided in the embodiment of the present application is not limited to Figure 1 The application scenario shown or Figure 2 The architecture can also be used in other possible application scenarios, which are not limited by the embodiments of the present application. Figure 1 The functions that can be implemented by each device in the application scenario shown will be described in the subsequent method embodiments, and will not be described in detail here.

[0121] The method flow provided in each embodiment of the present application can be Figure 1 The resource transfer server 102 or the resource transfer terminal device 101 may be executed, or the resource transfer server 102 and the resource transfer terminal device 101 may be executed together. Here, the resource transfer server 102 is mainly used as an example for introduction.

[0122] In the embodiment of the present application, the training process of the identity recognition model is a process of performing multiple cycles of iterative training using training samples, which can mainly include a model design phase, a data preparation phase and an iterative training phase, which will be introduced separately below.

[0123] (1) Model design stage

[0124] See also Figure 2 The figure is a schematic diagram of the architecture of the identity recognition model of the embodiment of the present application. The identity recognition model includes a palm recognition sub-model, an audio recognition sub-model and a fusion sub-model. The palm recognition sub-model performs identity recognition on the collected palm image to obtain the corresponding palm prediction identity. The audio recognition sub-model performs identity recognition on the collected audio data to obtain the corresponding audio prediction identity. The palm prediction identity and the audio prediction identity are then fused by the fusion sub-model to obtain the final identity recognition result.

[0125] like Figure 2 As shown, the palm recognition sub-model extracts features from the collected palm images, performs model prediction based on the extracted feature vectors, obtains the corresponding palm prediction vectors, and then performs embedding retrieval based on the palm prediction vectors (embedding) to obtain the palm prediction identity. Similarly, the audio sub-model extracts features from the collected audio data, performs model prediction based on the extracted feature vectors, obtains the corresponding audio prediction vectors, and then performs embedding retrieval based on the audio prediction vectors (embedding) to obtain the audio prediction identity.

[0126] In one implementation, a binary classification method may be used to perform the final classification of the model, and the model may meet the following conditions to improve the quality of model training:

[0127] (1) Contains a residual network substructure to prevent gradient vanishing.

[0128] (2) Include Batch Normalization (BN) layers to prevent oscillation during model training.

[0129] It should be noted that any network model that meets the following conditions can be used as a backbone network model. For example, ResNet can be selected as the backbone network model of the palm recognition sub-model. ResNet is a series of multiple residual blocks. The ResNet structure is very easy to modify and expand. By adjusting the number of channels in a block and the number of stacked blocks, the width and depth of the network can be easily adjusted to obtain networks with different expressive capabilities without worrying too much about the "degeneration" of the network. As long as there is enough training data and the network is gradually deepened, better performance can be obtained.

[0130] See also Figure 3FIG. 1 is a schematic diagram of a possible structure of a palm recognition sub-model in an embodiment of the present application. The palm recognition sub-model may include at least two branch networks, such as Figure 3 The two branch networks shown include the teacher backbone subnetwork and the student backbone subnetwork. Both subnetworks can be implemented using a convolutional neural network (CNN). In order to extract features of different granularities, the convolution kernel sizes used by the teacher backbone subnetwork and the student backbone subnetwork can be different. The palm recognition submodel can also include a region proposal network (RPN), a region of interest pooling (ROI) layer, and fully connected layers (FC).

[0131] In an embodiment of the present application, a densely connected convolutional network (Densnet) can be selected as the backbone network model of the audio recognition sub-model. Densnet is composed of multiple densely connected blocks. The input of each layer comes from the output of all previous layers, thereby establishing a close connection relationship between different layers. Under the premise of ensuring the maximum information transmission between layers in the network, all layers are directly connected. This connection method makes the transmission of features and gradients more effective, and the network is easier to train.

[0132] See also Figure 4 As shown in Figure 1, a possible structural diagram of the audio recognition sub-model is shown. Each square represents a dense block. For example, each dense block can be a stem operation, a reduction operation, multiple inception-resnet operations, a mean pooling operation, or a dropout operation, etc. Figure 4 In practical applications, the operations performed by each dense block can be designed according to actual needs, and the connection relationship between layers can also be designed based on actual conditions.

[0133] The data processing process performed by the above model will be introduced in detail in the subsequent process, so it will not be described in detail here.

[0134] (II) Data preparation stage

[0135] Data collection is of paramount importance in machine learning, and can be said to be the most important link. The data preparation stage of the embodiment of the present application mainly includes the following preparation of training data.

[0136] 1. Preparation of palm image training data

[0137] See also Figure 5 The figure shows a flowchart of the palm image training data processing process.

[0138] Step 501: Acquire multiple palm image data.

[0139] In one implementation, the palm red-green-blue (RGB) three-color image data collected by a shooting device can be obtained. The image resolution is related to the camera of the actual shooting device. Usually, the palm image is generally shot with the maximum resolution supported by the camera.

[0140] Of course, in actual scenarios, the amount of palm image data obtained is usually limited, so the existing palm images can also be subjected to certain sample expansion processing to obtain more palm images, such as distorting, cropping, Gaussian blurring, or stretching the original palm images.

[0141] In the embodiment of the present application, signal data processing technology is used to obtain, process and extract meaningful image features and attributes from the collected palm image data, so as to provide them to the model for training. Steps 502 to 504 are a possible image feature extraction process. Of course, in practical applications, other possible image feature extraction methods can still be used, such as CNN network.

[0142] Step 502: For each palm image, divide the palm image into regions to obtain a corresponding plurality of palm sub-images.

[0143] Specifically, considering that the resolutions of the collected palm images are different, the resolutions of the palm images can be unified before processing so that the dimensions of the features extracted subsequently are the same, which is convenient for training. For example, the resolution can be unified by cropping or image compression.

[0144] In one implementation, each palm image may be divided into blocks according to a set image resolution size (eg, 60x60), and the original palm image may be divided into a plurality of blocks.

[0145] Step 503: extracting directional gradient histogram features corresponding to each of the plurality of palm sub-images.

[0146] For each palm sub-image corresponding to the palm image, the corresponding Histogram of Directed Gradient (HOG) feature is extracted until all the palm sub-images are processed. The HOG feature performs extremely well in palm detection and other aspects, and is therefore very suitable for detection using palm features.

[0147] Specifically, Sobel can be used to extract HOG features. The calculation method of the Sobel algorithm is as follows:

[0148]

[0149]

[0150] Among them, M(x, y) represents the magnitude of the gradient, θ(x, y) represents the direction of the gradient, and I x and I y Represents the gradient values ​​in the horizontal and vertical directions respectively.

[0151] Step 504: performing concatenation processing on the obtained multiple HOG features to obtain a palm basic feature vector of a palm image.

[0152] Through the above steps, each palm sub-image of each palm image can obtain a HOG feature. By connecting the HOG of each palm image end to end, a large one-dimensional vector can be spliced. This one-dimensional vector is the palm basic feature vector of the palm image.

[0153] See also Figure 6 As shown in FIG. 1 , a schematic diagram of the image feature extraction process is shown. Here, a palm image A is taken as an example. After the palm image A is divided into multiple palm sub-images according to a resolution of 60x60, the corresponding HOG feature is extracted for each palm sub-image. Figure 6 As shown, each square represents a palm sub-image, and a small HOG feature is extracted from each palm sub-image ( Figure 6 Connect all the small HOG features end to end to get a large one-dimensional vector, that is, the basic palm feature vector of the palm image A. The basic palm feature vector can be sent to the classifier for training.

[0154] It should be noted that when collecting palm images, the source of each palm image will also be recorded, that is, the resource transfer object from which it comes, to serve as label data for subsequent model training.

[0155] 2. Preparation of audio training data

[0156] See also Figure 7 , which is a flowchart of the audio training data processing process.

[0157] Step 701: Acquire multiple audio data.

[0158] In an embodiment of the present application, audio data of the resource transfer object collected by the audio collection device can be obtained. For example, taking the resource transfer process as electronic payment, the payee can say "confirm payment" when making a payment, so that the audio collection device collects the audio data of "confirm payment" at one end.

[0159] In one implementation, the audio capture device may use a sampling rate of 16k, a sampling depth of 16 bits, and a mono channel for audio capture. Of course, other sampling rates or sampling depths may also be used, and this embodiment of the application does not impose any limitation on this.

[0160] In the embodiment of the present application, signal data processing technology is used to obtain, process and extract meaningful audio features and attributes from the collected audio data, so as to provide them to the model for training. Steps 702 to 704 are a possible audio feature extraction process. Of course, in practical applications, other possible audio feature extraction methods can still be used, and the embodiment of the present application does not limit this.

[0161] Step 702: Divide each audio data into frames to obtain multiple audio frames corresponding to each audio data.

[0162] Specifically, considering that the lengths of the collected audio data may be different, the lengths of the audio data may be unified before processing so that the dimensions of the features extracted subsequently are the same, which is convenient for training. For example, the lengths may be unified by cutting. Of course, the original collected audio data may also be directly used for subsequent processing, and the dimensions of the audio features may be unified after the audio features are obtained.

[0163] In one implementation, each audio data may be processed in blocks according to a set frame length, and the original audio data may be divided into a number of audio frames. For example, the frames may be divided in a manner of 80 sampling points and 10 milliseconds per audio frame.

[0164] In one embodiment, in order to make the correlation between the extracted audio frames higher and enable the final audio features to more accurately express the audio data, each audio data can be divided into frames in a cross-division manner so that adjacent audio frames in the audio frame have overlapping parts. For example, the frame division can be performed in a manner of 80 sampling points per audio frame, 10 milliseconds, and an overlap of 40 sampling points. Of course, the specific number of sampling points, duration, and overlap number can also be flexibly adjusted.

[0165] See also Figure 8 As shown in the figure, it is a schematic diagram of dividing an audio data frame. The division is performed every 10ms, for example, the first audio frame is from 0ms to 10ms, and each overlap is 40 sampling points, which is half of an audio frame, that is, 5ms, so the second audio frame is from 5ms to 15ms, and the subsequent audio frames are deduced by analogy until all audio frames are divided. It can be seen that there is an overlap between every two adjacent audio frames, so the correlation between every two adjacent audio frames extracted is closer. For example, the extracted features of the first audio frame include some features of the second audio frame, so that the final extracted audio features can more accurately express the audio data, which is helpful for subsequent identity recognition.

[0166] Step 703: for each audio data, extract the mel-frequency cepstrum features corresponding to the corresponding multiple audio frames.

[0167] For each audio frame corresponding to each audio data, the corresponding mel spectrum feature is extracted until all audio frames are processed. Here, an audio frame of an audio data is taken as an example to introduce the mel spectrum feature extraction process of the audio frame. Here, the sampling rate is 8k, hop_length=512, and windows is a Hamming window to extract the mel spectrum feature.

[0168] See also Fig. 9 , which is a flow chart of the mel spectrum feature extraction process of an audio frame, including steps 7031 to 7034.

[0169] Step 7031: Perform Fourier transform on the audio frame signal to obtain the frequency spectrum corresponding to the audio frame.

[0170] Step 7032: Map the spectrum corresponding to the audio frame to the Mel scale using a triangular overlapping window function.

[0171] Step 7033: Take the logarithm of the mapped spectrum.

[0172] Step 7034: Perform discrete cosine transformation to obtain the transformed Mel-Frequency Cepstral Coefficients (MFCC), i.e., the coefficients of the mel spectrum. The mel spectrum is derived from the cepstrum of the audio data, and its frequency band division is equally spaced on the mel scale. Compared with the linearly spaced frequency bands in the normal logarithmic cepstrum, it can better approximate the human auditory system. Such nonlinear representation can make the sound signal have a better representation and is more suitable for speech recognition function to improve the accuracy of audio recognition.

[0173] Step 704: for each audio data, concatenate the multiple mel spectrum features obtained to obtain an audio basic feature vector of the audio data.

[0174] Through the above steps, each audio frame of each audio data can obtain a mel spectrum feature. By connecting the mel spectrum features of each audio frame end to end, a large one-dimensional vector can be spliced. This one-dimensional vector is the audio basic feature vector of the audio data. The palm basic feature vector can be sent to the classifier for training.

[0175] It should be noted that when collecting audio data, the source of each audio data will also be recorded, that is, the resource transfer object it comes from, to serve as label data for subsequent model training.

[0176] 3. Iterative training phase

[0177] In the embodiment of the present application, after the training data is prepared, the constructed model can be trained using the training data.

[0178] In one embodiment, the parameters and data required for the identity recognition model can be set according to the structure of the above-mentioned model, including the parameters that need to be trained, and the TensorFlow (a programming framework) framework can be selected as the training framework. After setting the hyperparameters such as batch, number of iterations (epoch) and learning rate, the training is started, and the identity recognition model is finally obtained and saved as a file in pb format. The pb format is a saving mode of TensorFlow, which only includes one file and takes up little space.

[0179] For example, the batch size of the palm recognition sub-model is set to 128, the epoch size is set to 1000, and the learning rate is set to 0.0001, that is, the training is iterated 1000 times, and each iteration divides the training samples into 128 batches for learning; in addition, the batch size of the audio recognition sub-model can also be set to 128, the epoch size is set to 5000), and the learning rate is set to 0.0001. Of course, the training parameters here are just a possible example, and they can be adjusted according to needs in actual situations.

[0180] See also Fig.10 As shown, it is a flow chart of the identity recognition model training method provided in the embodiment of the present application. In the iterative training process, all training samples are divided into specified batches, and training is performed based on the training samples of each sub-batch. Since the steps performed for each batch in each iteration are similar, the training for one batch is used as an example for explanation.

[0181] Step 1001: Input each training sample into the palm recognition sub-model to obtain the predicted palm identity corresponding to the palm image included in each training sample.

[0182] In the embodiment of the present application, each training sample includes a palm image and a piece of audio data, as well as a corresponding real identity label. In actual situations, the content of the real identity label can be adjusted according to training requirements.

[0183] In one embodiment, the real identity label may be the real identity corresponding to the palm image and audio data included in each training sample. For example, if a training sample includes palm image A and audio data B, the real identity label may indicate the real identity C of palm image A and the real identity D of audio data B. The real identity C and the real identity D may be the same or different.

[0184] In one embodiment, the real identity label can indicate whether the real identities corresponding to the palm image and audio data included in each training sample are the same. For example, if a training sample includes palm image A and audio data B, the real identity label can indicate whether the real identity C of palm image A and the real identity D of audio data B are the same.

[0185] Specifically, here Figure 3 Taking the palm recognition sub-model shown in the figure as an example, the process of obtaining the predicted palm identity by the palm recognition sub-model is introduced. Fig.11 As shown, it is a flowchart of the process of obtaining the predicted identity of the palm by the palm recognition sub-model. Here, a palm image A in a training sample is taken as an example for introduction.

[0186] Step 10011: Perform convolution processing on the palm image A through two convolution sub-networks respectively to obtain two corresponding convolution feature maps.

[0187] Among them, the two convolutional sub-networks are Figure 3 The teacher backbone sub-network and the student backbone sub-network shown in the figure can respectively extract the convolution feature map of the palm image A through these two backbone networks.

[0188] In one implementation, the palm image A may refer to the original palm image collected, and then the basic image features are extracted through the two convolutional sub-networks. In this method, the process of extracting features by the two convolutional sub-networks can be as described above. Figure 5 The processing process shown.

[0189] In one embodiment, the palm image A may refer to Figure 5 The basic feature vector of the palm obtained by the processing process shown is further extracted through these two convolutional sub-networks.

[0190] Step 10012: Extract the regions of interest in the two convolutional feature maps through the region generation subnetwork to obtain a regional feature map.

[0191] Specifically, Figure 3 As shown, the region generation subnetwork can be used to extract the region of interest from the convolutional feature map obtained by the student backbone subnetwork and the convolutional feature map obtained by the teacher backbone subnetwork to obtain the corresponding regional feature map.

[0192] Step 10013: Perform pooling processing on the obtained regional feature map through the regional pooling layer to obtain a corresponding pooling feature map.

[0193] Specifically, since the scales and lengths of the obtained regional feature maps may be different, it is necessary to perform pooling processing on the regional feature maps through a regional pooling layer to obtain a uniform size.

[0194] Specifically, the pooling processing here can adopt pooling processing methods such as mean pooling or maximum pooling.

[0195] Step 10014: Perform feature mapping processing on the obtained pooled feature map through the fully connected layer to obtain a palm prediction vector.

[0196] Step 10015: Determine the corresponding predicted palm identity based on the obtained palm prediction vector.

[0197] Among them, the fully connected layer includes multiple weight parameters, which can be initially assigned through an initialization method and continuously adjusted through a training process. Based on the pooled feature map obtained above and these weight parameters, a palm prediction vector can be mapped accordingly, and then the palm prediction identity corresponding to the palm image A can be determined based on the palm prediction vector.

[0198] Please continue to see Fig.10 .

[0199] Step 1002: Input each training sample into the audio recognition sub-model to obtain the audio prediction identity corresponding to the audio data included in each training sample.

[0200] In the embodiment of the present application, after the audio data of each training sample is input, the audio recognition sub-model performs model forward processing to determine the corresponding audio prediction identity. The audio data input here can be the audio basic feature vector of the audio data extracted above.

[0201] by Figure 4 Taking the audio recognition sub-model shown in the figure as an example to recognize audio data B, the audio data B inputs a 299x299x3 audio basic feature vector through the input layer, and a 35x35x256 vector is obtained after processing by the stem layer, and a 35x35x256 vector is obtained after processing by 5 Inception-resnet layers, which is input to the next reducetion layer and its output vector of 17x17x896 is obtained, and so on, until the 1x1000 audio prediction vector output by the softmax layer is obtained, and then the audio prediction identity corresponding to the audio data B is determined based on the audio prediction vector.

[0202] It should be noted that there is no substantial order between the above-mentioned step 1001 and step 1002. They can be executed simultaneously or in sequence, and the embodiments of the present application do not limit this.

[0203] Step 1003: using the fusion sub-model, based on the palm predicted identity and the audio predicted identity corresponding to each training sample, respectively determine the identity recognition results corresponding to each training sample.

[0204] In one implementation, the fusion sub-model may include weight values ​​corresponding to the palm predicted identity and the audio predicted identity, and then obtain corresponding identity recognition results based on the respective weight values.

[0205] For example, the obtained palm prediction identity and audio prediction identity can be the confidence levels of multiple object identities, and the weight value of the palm prediction identity can be set to 70%, and the weight value of the audio prediction identity can be set to 30, that is, the palm prediction identity is mainly based on the palm prediction identity and the audio prediction identity is supplemented. Then, a weighted sum can be performed to obtain the final confidence level of each object identity, and the confidence level determines the final identity recognition result.

[0206] In one embodiment, the fusion sub-model can also adopt a neural network, which includes multiple network parameters. These network parameters can be initially assigned values ​​through an initialization method and continuously adjusted through a training process, so as to obtain corresponding identity recognition results based on the multiple network parameters included in the neural network and the above-mentioned palm predicted identity and audio predicted identity.

[0207] Step 1004: Based on each identity recognition result obtained and the corresponding true identity label, determine the total loss value corresponding to the identity recognition model.

[0208] In the embodiments of the present application, the total loss value may be obtained in the following ways, which are introduced one by one below.

[0209] (1) The first method

[0210] The palm recognition loss value of the palm recognition sub-model can be obtained based on the predicted palm identity and the real identity label corresponding to each palm image input this time, and the audio recognition loss value of the audio recognition sub-model can be obtained based on the predicted audio identity and the real identity label corresponding to each audio data input this time, and then the total loss value can be obtained based on the palm recognition loss value and the audio recognition loss value.

[0211] Among them, the above-mentioned palm recognition loss value and audio recognition loss value can be calculated using loss functions such as cross entropy loss function, quadratic loss function and absolute loss function.

[0212] In one implementation, the total loss value may be the sum of the palm recognition loss value and the audio recognition loss value, or different weight values ​​may be set for the palm recognition loss value and the audio recognition loss value, and then a weighted sum is performed to obtain the total loss value.

[0213] (2) Second method

[0214] In the embodiment of the present application, the loss value of the sample can also be measured as a whole from the perspective of the sample.

[0215] Specifically, the palm recognition loss value of the palm recognition sub-model can be obtained based on the predicted palm identity and the real identity label corresponding to each palm image input this time, and the audio recognition loss value of the audio recognition sub-model can be obtained based on the predicted audio identity and the real identity label corresponding to each audio data input this time. In addition, the sample recognition loss value can be obtained based on each identity recognition result and the real identity label corresponding to the corresponding training sample, and then the total loss value can be obtained based on the palm recognition loss value, the audio recognition loss value and the sample recognition loss value.

[0216] Specifically, according to whether the identities of the objects to which the palm images and audio data in the training samples belong are the same, the training samples can be divided into positive samples and negative samples. The real identity labels corresponding to the palm images and audio data included in each positive sample are the same, and the real identity labels corresponding to the palm images and audio data included in each negative sample are different. Then, the identity recognition results and real identity labels corresponding to each positive sample are compared to obtain the corresponding first comparison results respectively, and the identity recognition results and the set recognition failure results corresponding to each negative sample are compared to obtain the corresponding second comparison results respectively; wherein the recognition failure result is used to indicate the failure of identity recognition, and then based on each first comparison result and each second comparison result, the sample recognition loss value is obtained.

[0217] For any positive sample, the palm image and audio data included in it may have the same object identity, so the final predicted identity recognition result should theoretically be consistent with the labeled object identity, so that the identity recognition result can be compared with the labeled object identity to obtain the sample recognition loss value of each positive sample; and for any negative sample, the palm image and audio data included in it may have different object identities, so the final predicted identity recognition result should theoretically indicate recognition failure, that is, the palm image and audio data are inconsistent, and the identity cannot be correctly identified, so the identity recognition result can also be compared with the theoretical result of the negative sample to obtain the sample recognition loss value of each negative sample, and then the sample recognition loss values ​​of all training samples can be summarized for the calculation of the total loss value of the identity recognition model.

[0218] In one embodiment, when the real identity label indicates the real identities corresponding to the palm images and audio data in the training samples, it can be determined whether the real identities corresponding to the palm images and audio data in each training sample are the same. If they are the same, the sample recognition loss value is obtained according to the calculation method of the above-mentioned positive sample; if they are not the same, the sample recognition loss value is obtained according to the calculation method of the above-mentioned negative sample.

[0219] In one embodiment, when the real identity label indicates whether the palm image and the audio data in the training sample have the same object identity, for example, when the real identity label is 0, it indicates that the palm image and the audio data have different object identities, and when the real identity label is 1, it indicates that the palm image and the audio data have the same object identity. Then, the identity recognition result may include indication information indicating whether the palm predicted identity and the audio predicted identity are the same, and then, a judgment may be made based on the palm predicted identity and the audio predicted identity. If they are the same, the indication information may be 1, and if they are not the same, the indication information may be 0, so that the real identity label and the indication information in the identity recognition result may be compared to determine the sample recognition loss value of each training sample.

[0220] Step 1005: Determine whether the identity recognition model meets the convergence condition.

[0221] In the embodiment of the present application, the convergence condition may include at least one of the following conditions:

[0222] (1) The total loss value is not greater than the preset loss value threshold.

[0223] (2) The number of iterations reaches the preset upper limit.

[0224] Step 1006: If the judgment result of step 1005 is no, parameters of the palm recognition sub-model, the audio recognition sub-model and the fusion sub-model are adjusted based on the total loss value.

[0225] If the above conditions are met, it is determined that the identity recognition model has met the convergence conditions and the training ends. Otherwise, it is determined that the identity recognition model has not met the convergence conditions and the model parameters need to be further adjusted. The adjusted identity recognition model is used to enter the next training process, that is, jump to steps 1001 and 1002 to execute.

[0226] In an embodiment of the present application, the final identity recognition result is obtained through palm images and audio data, and the entire model is adjusted based on the total loss value obtained based on this identity recognition result. In this way, both the palm recognition sub-model and the audio recognition sub-model can be learned based on the palm image and audio data, thereby further improving the identity recognition accuracy of the entire model.

[0227] In the embodiment of the present application, in order to improve the training efficiency, Fig.10 Before the training process, the palm recognition sub-model and the audio recognition sub-model can be pre-trained respectively, and the palm recognition sub-model and the audio recognition sub-model after the pre-training are used to participate in Fig.10 The training process, since the two models have converged after training separately, we can use Fig.10The training process is fine-tuned so that the palm recognition sub-model and the audio recognition sub-model can learn the features of both images and audio at the same time, making the identity recognition model more accurate.

[0228] Specifically, a palm recognition sub-model may be pre-trained using multiple palm images to obtain a first model parameter of the palm recognition sub-model when a convergence condition is satisfied, and a plurality of audio data may be pre-trained using multiple audio data to obtain a second model parameter of the audio recognition sub-model when a convergence condition is satisfied. Fig.10 When the training process starts, the palm recognition sub-model can be initialized based on the first model parameters, and the audio recognition sub-model can be initialized based on the second model parameters, and the initialized audio recognition sub-model and palm recognition sub-model are trained through various training samples.

[0229] In the embodiment of the present application, for audio data, considering that in actual scenarios, when a user pays to collect audio data, the voices of other nearby users may also be collected, so the extracted audio features may be mixed with the voice features of other users, thereby affecting the accuracy of model training. Based on this, the embodiment of the present application can filter out the noise in the audio data, and then use the filtered audio data for training.

[0230] In one implementation, each audio data may be used in advance to train an audio frame filtering model to obtain a trained audio frame filtering model, and then the audio frame filtering model may be used to reversely identify the audio frames of each audio data to assist in filtering out noise in the audio data.

[0231] Specifically, taking an audio data as an example, multiple audio frames included in the audio data are input into a trained audio frame filtering model to obtain audio frame identification identities corresponding to the multiple audio frames. If there is an audio frame corresponding to an audio frame that is inconsistent with the real identity label of the audio data to which it belongs, it means that the audio frame belongs to the voice of other users. The audio frame can be filtered out from the audio data to which it belongs, so that each audio data can belong to the same user. Subsequently, the filtered audio data are input into the audio recognition sub-model for training, and the accuracy of the obtained audio recognition sub-model is improved.

[0232] Specifically, the audio frame filtering model may adopt the same model structure as the above-mentioned audio recognition sub-model.

[0233] In an embodiment of the present application, after the identity recognition model training is completed, the identity recognition model can also be evaluated. If the evaluation accuracy meets the requirements, it can be put into actual use. When performing model evaluation, evaluation samples can be used.

[0234] Specifically, the palm image and audio data in each evaluation sample are divided into blocks, and the corresponding HOG features and mel spectrum features are extracted, and then these extracted HOG features and mel spectrum features are sent to the identity recognition model for prediction, and the prediction results are statistically calculated by voting. For an evaluation sample, if the prediction results of the palm recognition sub-model and the audio recognition sub-model need to be for the same user, otherwise the prediction fails, and the weights of the palm prediction results and audio prediction results can be adjusted, for example, the weight of the palm image is 70%, and the weight of the audio data is 30%.

[0235] In the embodiment of the present application, a method for transferring electronic resources is also provided. Fig.12 The figure shows a flow chart of the electronic resource transfer method.

[0236] Step 1201: receiving an electronic resource transfer request, where the electronic resource transfer request is used to transfer a specified number of electronic resources owned by a resource transfer object to a resource receiving object, and the electronic resource transfer request carries a palm image and audio data of the resource transfer object.

[0237] In an embodiment of the present application, taking electronic resource transfer as electronic payment as an example, when a user purchases an item at a merchant, the user can pay by swiping his palm. The resource transfer terminal device can then use the user's palm image and audio data, and then upload it to the background server, so that the background server can identify the user's identity based on the palm image and audio data and pay the merchant.

[0238] Specifically, the resource transfer terminal device can perform feature extraction on the collected data to obtain feature vectors of the palm image and audio data, and can also encrypt and transmit the original data of the palm image and audio data and the feature vectors of the palm image and audio data to the resource transfer server. After the resource transfer server receives the electronic resource transfer request, it can use the feature extraction part of the trained identity recognition model to perform feature extraction based on the original data, and compare the extracted feature vector with the feature vector sent by the resource transfer terminal device to determine whether the original data has been tampered with, thereby ensuring the security of electronic resource transfer data transmission.

[0239] Step 1202: Input the palm image and audio data into the identity recognition model to obtain the identity recognition result of the resource transfer object.

[0240] The identity recognition model may be the identity recognition model obtained through the above training, and thus the process of obtaining the identity recognition result of the resource transfer object may refer to the description of the corresponding part above, which will not be repeated here.

[0241] Step 1203: Based on the identity recognition result, transfer the specified amount of electronic resources to the resource recipient.

[0242] For example, the identity recognition result may indicate the payment account of the user corresponding to the collected palm image and audio data, so that the electronic resources owned by the payment account may be transferred to the merchant in exchange for corresponding items.

[0243] It can be understood that in the specific implementation of the present application, user information such as the above-mentioned palm images, audio data, and payment account related data is involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0244] See also Fig.13 Based on the same inventive concept, the embodiment of the present application further provides an identity recognition model training device 130, the identity recognition model includes a palm recognition sub-model, an audio recognition sub-model and a fusion sub-model, and the device includes:

[0245] The image recognition unit 1301 is used to input each training sample into the palm recognition sub-model to obtain the predicted palm identity corresponding to the palm image included in each training sample;

[0246] The audio recognition unit 1302 is used to input each training sample into the audio recognition sub-model to obtain the audio prediction identity corresponding to the audio data included in each training sample;

[0247] A fusion prediction unit 1303 is used to use a fusion sub-model to determine the identity recognition results corresponding to each training sample based on the palm prediction identity and the audio prediction identity corresponding to each training sample;

[0248] The model adjustment unit 1304 is used to adjust the parameters of the palm recognition sub-model, the audio recognition sub-model and the fusion sub-model based on the obtained identity recognition results and the total loss values ​​corresponding to the corresponding real identity labels.

[0249] Optionally, each training sample is annotated with a real identity label corresponding to the palm image and audio data included therein; then the device further includes a model loss determination unit 1305, which is used to:

[0250] Based on the obtained palm prediction identity and the corresponding true identity label, obtain the palm recognition loss value of the palm recognition sub-model;

[0251] Based on the obtained audio predicted identity and the corresponding true identity label, obtain the audio recognition loss value of the audio recognition sub-model;

[0252] Based on each identity recognition result and the true identity label corresponding to the corresponding training sample, the sample recognition loss value is obtained;

[0253] Based on the palm recognition loss value, the audio recognition loss value, and the sample recognition loss value, a total loss value is obtained.

[0254] Optionally, each training sample includes a positive sample and a negative sample, the palm image and the audio data included in each positive sample have the same true identity label, and the palm image and the audio data included in each negative sample have different true identity labels; then the model loss determination unit 1305 is further used to:

[0255] Compare the identity recognition results corresponding to each positive sample with the true identity label to obtain corresponding first comparison results respectively;

[0256] Compare the identity recognition results corresponding to each negative sample with the set recognition failure results to obtain corresponding second comparison results respectively; wherein the recognition failure results are used to indicate identity recognition failure;

[0257] Based on each first comparison result and each second comparison result, a sample recognition loss value is obtained.

[0258] Optionally, the palm recognition sub-model includes two convolution sub-networks with different convolution kernel sizes, a region generation sub-network, a region pooling layer and a fully connected layer; then the image recognition unit 1301 is specifically used for:

[0259] For each training sample, perform the following operations:

[0260] For a training sample, two convolution sub-networks are used to perform convolution processing on the palm image respectively to obtain two corresponding convolution feature maps, and the region of interest is extracted through the region generation sub-network to obtain at least one region feature map;

[0261] Performing pooling processing on at least one regional feature map through a regional pooling layer to obtain a pooled feature map;

[0262] Through the fully connected layer, the obtained pooled feature map is feature mapped, and the corresponding palm prediction identity is determined based on the obtained palm prediction vector.

[0263] Optionally, the image recognition unit 1301 is further used for:

[0264] For each palm image included in each training sample, perform the following operations respectively:

[0265] For a palm image, the palm image is divided into regions to obtain a corresponding plurality of palm sub-images;

[0266] Extracting directional gradient histogram features corresponding to each of the multiple palm sub-images respectively;

[0267] The obtained multiple directional gradient histogram features are spliced ​​to obtain a palm basic feature vector of a palm image;

[0268] The palm basic feature vectors corresponding to each palm image are input into the palm recognition sub-model.

[0269] Optionally, the audio recognition unit 1302 is further configured to:

[0270] For each audio data included in each training sample, perform the following operations respectively:

[0271] For a piece of audio data, the piece of audio data is divided into frames to obtain a corresponding plurality of audio frames;

[0272] Extract the Mel-frequency cepstrum features corresponding to multiple audio frames respectively;

[0273] The obtained multiple Mel-frequency cepstrum features are concatenated to obtain an audio basic feature vector of the audio data;

[0274] Then each training sample is input into the audio recognition sub-model, including:

[0275] The audio basic feature vectors corresponding to each audio data are input into the audio recognition sub-model.

[0276] Optionally, the audio recognition unit 1302 is specifically configured to:

[0277] An audio data is divided into frames by cross division, so that adjacent audio frames in a plurality of audio frames have overlapping parts.

[0278] Optionally, the audio recognition unit 1302 is further configured to:

[0279] Based on each audio data, the audio frame filtering model is trained to obtain a trained audio frame filtering model; wherein the audio frame filtering model has the same model structure as the audio recognition sub-model;

[0280] For each audio data, perform the following operations respectively:

[0281] For an audio data, multiple audio frames included in the audio data are input into the trained audio frame process.

[0282] Filter the model to obtain audio frame identification identities corresponding to each of the multiple audio frames;

[0283] If there is an audio frame whose corresponding audio frame identification identity is inconsistent with a real identity label of an audio data, then an audio frame is filtered out from an audio data;

[0284] The filtered audio data are input into the audio recognition sub-model.

[0285] Optionally, the apparatus further includes a pre-training unit 1306, configured to:

[0286] Pre-training a palm recognition sub-model using a plurality of palm images to obtain a first model parameter of the palm recognition sub-model when a convergence condition is satisfied; and pre-training an audio recognition sub-model using a plurality of audio data to obtain a second model parameter of the audio recognition sub-model when a convergence condition is satisfied;

[0287] The image recognition unit is specifically used to initialize the palm recognition sub-model based on the first model parameter, and input each training sample into the initialized palm recognition sub-model;

[0288] The audio recognition unit is specifically used to initialize the audio recognition sub-model based on the second model parameters, and input each training sample into the initialized audio recognition sub-model.

[0289] Through the above device, multimodal learning can be performed by using palm images and audio data as training data. The learned model can predict the palm image and audio data separately, and fuse the prediction results of the two to obtain the final identity recognition result, so as to improve the accuracy of identity recognition. In addition, during training, the final identity recognition result is obtained through palm images and audio data, and the entire model is adjusted based on this identity recognition result. In this way, the palm recognition sub-model and the audio recognition sub-model can both learn based on palm images and audio data, thereby further improving the identity recognition accuracy of the entire model.

[0290] The device can be used to execute the methods shown in the various embodiments of the present application. Therefore, for the functions that can be implemented by the various functional modules of the device, reference can be made to the description of the aforementioned embodiments and no further details will be given.

[0291] See also Fig.14 Based on the same inventive concept, the embodiment of the present application further provides an electronic resource transfer device 140, which includes:

[0292] The receiving unit 1401 is used to receive an electronic resource transfer request, where the electronic resource transfer request is used to transfer a specified number of electronic resources owned by a resource transfer object to a resource receiving object, and the electronic resource transfer request carries a palm image and audio data of the resource transfer object;

[0293] An identity recognition unit 1402, configured to input the palm image and audio data into an identity recognition model trained based on any one of the methods of claims 1 to 9, to obtain an identity recognition result of the resource transfer object;

[0294] The resource transfer unit 1403 is used to transfer a specified amount of electronic resources to a resource receiving object based on the identity recognition result.

[0295] The device can be used to execute the methods shown in the various embodiments of the present application. Therefore, for the functions that can be implemented by the various functional modules of the device, reference can be made to the description of the aforementioned embodiments and no further details will be given.

[0296] See also Fig.15 Based on the same technical concept, the embodiment of the present application further provides a computer device 150, which can be Figure 1 As shown in the terminal device or server, the computer device 150 may include a memory 1501 and a processor 1502 .

[0297] The memory 1501 is used to store computer programs executed by the processor 1502. The memory 1501 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the computer device, etc. The processor 1502 may be a central processing unit (CPU), or a digital processing unit, etc. The specific connection medium between the above-mentioned memory 1501 and the processor 1502 is not limited in the embodiments of the present application. The embodiments of the present application are Fig.15 In the embodiment, the memory 1501 and the processor 1502 are connected via a bus 1503. The bus 1503 is connected to the processor 1502 via a bus 1503. Fig.15 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 1503 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0298] The memory 1501 may be a volatile memory, such as a random-access memory (RAM); the memory 1501 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 1501 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1501 may be a combination of the above memories.

[0299] The processor 1502 is used to execute the method executed by the device in each embodiment of the present application when calling the computer program stored in the memory 1501.

[0300] In some possible implementations, various aspects of the method provided in the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the method according to the various exemplary implementations of the present application described above in this specification. For example, the computer device can execute the method executed by the device in each embodiment of the present application.

[0301] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0302] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0303] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for training an identity recognition model, characterized in that: The identity recognition model includes a palm recognition sub-model, an audio recognition sub-model and a fusion sub-model, and the method includes: Inputting each training sample into the palm recognition sub-model to obtain the predicted palm identity corresponding to the palm image included in each training sample; Inputting each of the training samples into the audio recognition sub-model to obtain the audio prediction identity corresponding to the audio data included in each of the training samples; Using the fusion sub-model, based on the palm predicted identity and the audio predicted identity corresponding to each of the training samples, respectively determine the identity recognition results corresponding to each of the training samples; Determine the total loss value corresponding to the identity recognition model based on each identity recognition result obtained and the corresponding real identity label; Parameters of the palm recognition sub-model, the audio recognition sub-model, and the fusion sub-model are adjusted based on the total loss value.

2. The method according to claim 1, characterized in that Each training sample is annotated with the real identity label corresponding to the palm image and audio data included; Before adjusting the parameters of the palm recognition sub-model, the audio recognition sub-model and the fusion sub-model based on the obtained identity recognition results and the total loss values ​​corresponding to the corresponding real identity tags, the method further includes: Based on the obtained predicted palm identity and the corresponding true identity label, obtaining a palm recognition loss value of the palm recognition sub-model; Based on the obtained audio prediction identity and the corresponding true identity label, obtaining the audio recognition loss value of the audio recognition sub-model; Based on the individual identity recognition results and the true identity labels corresponding to the corresponding training samples, a sample recognition loss value is obtained; The total loss value is obtained based on the palm recognition loss value, the audio recognition loss value and the sample recognition loss value.

3. The method according to claim 2, characterized in that Each training sample includes a positive sample and a negative sample, the palm image and the audio data included in each positive sample have the same real identity label, and the palm image and the audio data included in each negative sample have different real identity labels; Based on the individual identity recognition results and the real identity labels corresponding to the corresponding training samples, a sample recognition loss value is obtained, including: Compare the identity recognition results corresponding to each positive sample with the true identity label to obtain corresponding first comparison results respectively; Compare the identity recognition results corresponding to each negative sample with the set recognition failure results to obtain corresponding second comparison results respectively; wherein the recognition failure results are used to indicate identity recognition failure; Based on the first comparison results and the second comparison results, the sample identification loss value is obtained.

4. The method according to any one of claims 1 to 3, characterized in that: The palm recognition sub-model includes two convolution sub-networks with different convolution kernel sizes, a region generation sub-network, a region pooling layer and a fully connected layer; each training sample is input into the palm recognition sub-model to obtain the predicted palm identity corresponding to the palm image included in each training sample, including: For each training sample, perform the following operations respectively: For a training sample, the palm image is convolved by the two convolution sub-networks to obtain two corresponding convolution feature maps, and the region of interest is extracted by the region generation sub-network to obtain at least one region feature map; Performing pooling processing on the at least one regional feature map through the regional pooling layer to obtain a pooling feature map; The obtained pooled feature map is subjected to feature mapping processing through the fully connected layer, and a corresponding palm prediction identity is determined based on the obtained palm prediction vector.

5. The method according to any one of claims 1 to 3, characterized in that: Before inputting each training sample into the palm recognition sub-model, the method further includes: For each palm image included in each training sample, the following operations are performed respectively: For a palm image, dividing the palm image into regions to obtain a corresponding plurality of palm sub-images; Respectively extracting directional gradient histogram features corresponding to each of the plurality of palm sub-images; Performing splicing processing on the obtained multiple directional gradient histogram features to obtain a palm basic feature vector of the palm image; Then, each training sample is input into the palm recognition sub-model, including: The palm basic feature vectors corresponding to the respective palm images are input into the palm recognition sub-model.

6. The method according to any one of claims 1 to 3, characterized in that: Before inputting each training sample into the audio recognition sub-model, the method further includes: For each audio data included in each training sample, the following operations are performed respectively: For a piece of audio data, dividing the piece of audio data into frames to obtain a corresponding plurality of audio frames; Extracting Mel-frequency cepstrum features corresponding to each of the multiple audio frames respectively; Performing splicing processing on the obtained multiple Mel-frequency cepstrum features to obtain an audio basic feature vector of the one audio data; Then, each training sample is input into the audio recognition sub-model, including: The audio basic feature vectors corresponding to each of the audio data are input into the audio recognition sub-model.

7. The method according to claim 6, characterized in that The one audio data is divided into frames to obtain a corresponding plurality of audio frames, including: The audio data is divided into frames in a cross-division manner, so that adjacent audio frames in the multiple audio frames have overlapping parts.

8. The method according to claim 6, characterized in that The method further comprises: Based on the respective audio data, an audio frame filtering model is trained to obtain a trained audio frame filtering model; wherein the audio frame filtering model has the same model structure as the audio recognition sub-model; For each of the audio data, perform the following operations respectively: For one audio data, multiple audio frames included in the one audio data are input into a trained audio frame filtering model to obtain audio frame identification identities corresponding to each of the multiple audio frames; If there is an audio frame whose corresponding audio frame identification identity is inconsistent with the real identity tag of the one audio data, filtering the one audio frame from the one audio data; Then, each training sample is input into the audio recognition sub-model, including: The filtered audio data are input into the audio recognition sub-model.

9. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Pre-training the palm recognition sub-model using a plurality of palm images to obtain a first model parameter of the palm recognition sub-model when a convergence condition is satisfied; Pre-training the audio recognition sub-model using a plurality of audio data to obtain a second model parameter of the audio recognition sub-model when a convergence condition is satisfied; Then, inputting each training sample into the palm recognition sub-model includes: Initializing the palm recognition sub-model based on the first model parameters, and inputting each training sample into the initialized palm recognition sub-model; Then, inputting each training sample into the audio recognition sub-model comprises: The audio recognition sub-model is initialized based on the second model parameters, and each training sample is input into the initialized audio recognition sub-model.

10. An electronic resource transfer method based on an identity recognition model trained by any one of claims 1 to 9, characterized in that: The method comprises: receiving an electronic resource transfer request, the electronic resource transfer request being used to transfer a specified amount of electronic resources owned by a resource transfer object to a resource receiving object, and the electronic resource transfer request carrying a palm image and audio data of the resource transfer object; Inputting the palm image and audio data into the identity recognition model to obtain an identity recognition result of the resource transfer object; Based on the identity recognition result, the specified amount of electronic resources is transferred to the resource recipient.

11. An identity recognition model training device, characterized in that: The identity recognition model includes a palm recognition sub-model, an audio recognition sub-model and a fusion sub-model, and the device includes: An image recognition unit, used for inputting each training sample into the palm recognition sub-model to obtain the predicted palm identity corresponding to the palm image included in each training sample; An audio recognition unit, used to input each of the training samples into the audio recognition sub-model, and obtain the audio prediction identity corresponding to the audio data respectively included in each of the training samples; A fusion prediction unit, configured to use the fusion sub-model to determine the identity recognition results corresponding to each of the training samples based on the palm prediction identity and the audio prediction identity corresponding to each of the training samples; A model adjustment unit is used to determine the total loss value corresponding to the identity recognition model based on the obtained identity recognition results and the corresponding real identity labels, and adjust the parameters of the palm recognition sub-model, the audio recognition sub-model and the fusion sub-model based on the total loss value.

12. An electronic resource transfer device, characterized in that: The device comprises: A receiving unit, configured to receive an electronic resource transfer request, wherein the electronic resource transfer request is used to transfer a specified number of electronic resources owned by a resource transfer object to a resource receiving object, and the electronic resource transfer request carries a palm image and audio data of the resource transfer object; An identity recognition unit, used for inputting the palm image and audio data into an identity recognition model trained based on any one of the methods of claims 1 to 9 to obtain an identity recognition result of the resource transfer object; The resource transfer unit is used to transfer the specified number of electronic resources to the resource receiving object based on the identity recognition result.

13. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 or 10 are implemented.

14. A computer storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 or 10 are implemented.

15. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 or 10 are implemented.

Images