Palm image processing method, palm detection model training method, palm image processing device, palm detection model training device, computer device, and computer program

Abstract

A palm image processing method executed by a palm recognition device includes the steps of: acquiring a palm image and extracting features from the palm image to acquire image features at multiple scales (302); fusing the image features at multiple scales to acquire image fusion features (304); determining a palm frame that identifies a palm in the palm image based on the image fusion features (306); cropping a palm image from the palm image based on the palm frame (308); and performing image encryption on the palm image to acquire palm image encrypted data (310), which is transmitted to a server and used by the server to perform identity recognition based on the palm image encrypted data (310).

Description

Technical Field

[0001] This application claims priority based on a Chinese patent application filed on November 21, 2022, with an application number of No. 2022114733356 and an invention title of "Method, Apparatus, Device, Storage Medium, and Program Product for Palm Image Recognition", the entire content of which is incorporated herein by reference.

[0002] Embodiments of this application relate to the technical field of computers, and further relate to the technical field of computer vision, and in particular, relate to a method, apparatus, device, storage medium, and program product for palm image processing.

Background Art

[0003] With the development of computer technology, palm recognition technology has been increasingly widely applied and can be applied to various scenarios. For example, in scenarios such as payment scenarios or punching a commuter time card, the user's identity can be verified by palm recognition.

[0004] In related technologies, the user provides a physical palm, the palm recognition device collects a palm image from the user's palm, the palm image recognition device encrypts the palm image, and the encrypted palm image is used for identity verification.

[0005] However, since the time taken for encryption and decryption of palm images is long, the efficiency of palm recognition decreases.

Summary of the Invention

Problems to be Solved by the Invention

[0006] According to various embodiments of this application, a method, apparatus, device, storage medium, and program product for palm image processing are provided.

Means for Solving the Problems

[0007] In one embodiment, a palm image processing method is provided that is performed by a palm recognition device. This method is The steps include acquiring a palm image and extracting features from the palm image to obtain image features of multiple scales, The steps include obtaining image fusion features by fusing image features of the aforementioned multiple scales, The steps include determining a palm frame that identifies the palm area in the palm image based on the aforementioned image fusion features, The steps include: cutting out the palm image from the palm image based on the palm frame, The method includes the step of obtaining encrypted palm image data by encrypting the palm image, wherein the encrypted palm image data is transmitted to a server and used by the server to perform identity recognition based on the encrypted palm image data.

[0008] In another aspect, the present invention provides a method for training a palm detection model performed by a computer device. This method is Steps include obtaining a sample palm image and a sample palm image, The steps include obtaining sample image features of multiple scales by extracting features from the aforementioned sample palm image, The steps include obtaining sample image fusion features by fusing the sample image features of the aforementioned multiple scales, The steps include determining a sample predicted palm frame that identifies the palm area in the sample palm image based on the sample image fusion features, The steps include: cutting out the sample palm image from the sample palm image based on the sample palm frame, The steps include: calculating a loss function value based on the sample palm image and the sample predicted palm image; The process includes updating the model parameters of the palm detection model based on the loss function value.

[0009] In another embodiment, the present invention provides a palm image processing device. This device is A feature extraction module that acquires a palm image and extracts features from the palm image to acquire image features at multiple scales, A feature fusion module that obtains image fusion features by fusing image features of the aforementioned multiple scales, A prediction module that determines a palm frame for identifying the palm area in the palm image based on the aforementioned image fusion features, A cropping module that crops the palm image from the palm image based on the palm frame, An encryption module that obtains encrypted palm image data by encrypting the palm image, wherein the encrypted palm image data is transmitted to a server and used by the server to perform identity verification based on the encrypted palm image data.

[0010] In another aspect, the present invention provides a training device for a palm detection model. The device includes a palm detection model, further including an acquisition module, a clipping module, a calculation module, and an update module, wherein the palm detection model includes a feature extraction module, a feature fusion module, and a prediction module. The acquisition module acquires a sample palm image and a sample palm image. The feature extraction module obtains sample image features of multiple scales by extracting features from the sample palm image. The feature fusion module obtains sample image fusion features by fusing the sample image features of the multiple scales, The prediction module determines a sample predicted palm frame that identifies the palm area in the sample palm image based on the sample image fusion features, The cropping module crops the sample palm image from the sample palm image based on the sample palm frame. The calculation module calculates a loss function value based on the sample palm image and the sample predicted palm image, The update module updates the model parameters of the palm detection model based on the loss function value.

[0011] In another embodiment, the present invention further provides a computer device comprising memory and a processor. The memory stores computer-readable instructions, and the processor, upon executing the computer-readable instructions, performs the steps of each embodiment of the present invention.

[0012] In another embodiment, the present invention provides a computer-readable storage medium storing computer-readable instructions. When the computer-readable instructions are executed by a processor, the processor causes the processor to perform the steps of each embodiment of the present invention.

[0013] In another embodiment, the present application provides a computer program product that includes computer-readable instructions. When the computer-readable instructions are executed by a processor, the processor causes the processor to perform the steps of each embodiment of the present invention.

[0014] Details of one or more embodiments of the present application are described in the following drawings and description. Other features, purposes, and advantages of the present application will become apparent from the specification, drawings, and claims. [Brief explanation of the drawing]

[0015] To more clearly explain the configuration of the embodiments of this application, the following is a brief introduction of the drawings necessary for describing the embodiments. Clearly, the drawings in the following description only show some embodiments of this application, and those skilled in the art can obtain other drawings from these without any creative work. [Figure 1] This is a schematic diagram of a palm image processing method provided in one exemplary embodiment of the present application. [Figure 2]It is a schematic diagram of the architecture of a computer system provided in one exemplary embodiment of the present application. [Figure 3] It is a flowchart of a palm image processing method provided in one exemplary embodiment of the present application. [Figure 4] It is a flowchart of a palm image processing method provided in one exemplary embodiment of the present application. [Figure 5] It is a schematic diagram of image feature fusion provided in one exemplary embodiment of the present application. [Figure 6] It is a schematic diagram of water scooping in a palm provided in one exemplary embodiment of the present application. [Figure 7] It is a schematic diagram of a palm image processing method provided in one exemplary embodiment of the present application. [Figure 8] It is a schematic diagram of a flow based on a scenario where the palm image processing method provided in one exemplary embodiment of the present application is applied to cross-device payment. [Figure 9] It is a schematic diagram of a flow based on a scenario where the palm image processing method provided in one exemplary embodiment of the present application is applied to cross-device identity verification. [Figure 10] It is a schematic diagram of a flow of a palm image processing method provided in one exemplary embodiment of the present application. [Figure 11] It is a framework diagram of the generation stage and the prediction stage of a palm detection model provided in one exemplary embodiment of the present application. [Figure 12] It is a flowchart of a training method of a palm detection model provided in one exemplary embodiment of the present application. [Figure 13] It is a flowchart of a training method of a palm detection model provided in one exemplary embodiment of the present application. [Figure 14] It is a block diagram of a palm image processing device provided in one exemplary embodiment of the present application. [Figure 15] It is a block diagram of a training device of a palm detection model provided in one exemplary embodiment of the present application. [Figure 16]This is a schematic diagram of the configuration of a computer device provided in one exemplary embodiment of the present application. [Modes for carrying out the invention]

[0016] The configuration of the embodiments of this application will be described clearly and completely below with reference to the drawings of the embodiments. It will be clear that the embodiments described are only a selection of embodiments of this application, not all embodiments. All other embodiments that a person skilled in the art can obtain from the embodiments of this application without creative work are within the scope of protection of this application.

[0017] Computer vision (CV) is the science that studies how to make things "visible" to machines. More specifically, it uses cameras and computers in place of human eyes to perform machine vision such as target recognition and measurement, and further performs graphic processing. Through computer processing, it acquires suitable images that can be observed by the human eye or transmitted to machines for detection. As a science discipline, computer vision studies related theories and technologies and aims to build artificial palm image recognition systems that can acquire information from images or multidimensional data. Computer vision technologies typically include technologies such as image processing, image recognition, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / action recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning, and map building, and further includes general bio-feature recognition technologies.

[0018] In embodiments of the present application, a schematic diagram of a palm image processing method is provided. As shown in Figure 1, this method may be performed by a palm recognition device 10. This palm recognition device 10 may have an image encryption function.

[0019] Exemplary, the palm recognition device 10 includes a camera. The palm recognition device 10 acquires a palm image 11 using the camera. The palm recognition device 10 may be connected to an image collector. The image collector has a camera, and the palm recognition device 10 can acquire the palm image 11 collected by this image collector. The palm recognition device 10 acquires a palm image 12 corresponding to the palm in the palm image 11 by performing palm prediction on the palm image 11. The palm recognition device 10 acquires encrypted palm image data 13 by encrypting the palm image 12. The palm recognition device 10 acquires a user identifier 14 corresponding to the palm image 11 by sending the encrypted palm image data 13 to a palm recognition server for decryption and palm comparison recognition.

[0020] For example, the palm recognition device 10 acquires a palm image 11 and extracts features from the palm image 11 to obtain image features of multiple scales. The palm recognition device 10 obtains image fusion features by performing feature fusion on the image features of multiple scales. Based on the image fusion features, the palm recognition device 10 determines the palm area frame in the palm image 11. Based on the palm area frame, the palm recognition device 10 crops the palm image 12 from the palm image 11. The palm recognition device 10 obtains encrypted palm image data 13 by encrypting the palm image 12.

[0021] The encrypted palm image data 13 is transmitted to the palm recognition server and used to obtain a user identifier 14 corresponding to the palm image 11 by decrypting and performing palm comparison recognition.

[0022] The palm image 12 is the effective recognition area in the palm image 11, or the palm image 12 refers to the area in the palm image 11 where the palm is located, or the palm image 12 refers to the area in the palm image 11 that can be used for palm recognition.

[0023] The method by which the palm recognition device 10 transmits the palm image encrypted data 13 to the palm recognition server may optionally include, but is not limited to, at least one of network transmission, data line transmission, or Bluetooth® transmission. The embodiments of this application are not specifically limited thereto.

[0024] For example, the palm recognition device 10 acquires image features by extracting features from the palm image 11. The palm recognition device 10 acquires image fused features by performing feature fusion on the image features. The palm recognition device 10 determines the palm area frame in the palm image and crops the palm image 12 from the palm image based on the palm area frame.

[0025] For example, the palm frame may be predicted by a palm detection model, which includes a backbone network, a neck network, and a prediction network.

[0026] The palm recognition device 10 inputs the palm image 11 into the backbone network, and the backbone network performs a slicing operation on the palm image to obtain sliced ​​images of multiple scales. The palm recognition device 10 then performs feature extraction on the sliced ​​images of multiple scales to obtain image features of multiple scales corresponding to the palm image 11. The scale represents the size of the feature, and multiple scales refer to multiple sizes.

[0027] For example, the size of the input palm image 11 is 640*640, and the sizes of the output image features at multiple scales are 80*80, 40*40, and 20*20. The scale and size of an image or feature may also be the dimensions of the matrix representing the image or feature.

[0028] For example, the palm recognition device 10 obtains an image fusion feature by inputting image features of multiple scales into a neck network and performing feature fusion. The palm recognition device 10 obtains the palm frame in the palm image 11 by inputting the image fusion feature into a prediction network and performing prediction. The palm recognition device 10 crops the image region outside the palm frame in the palm image 11, and the remaining image becomes the palm image 12.

[0029] As described above, the method provided in this embodiment acquires a palm image and performs feature extraction to obtain image features of multiple scales corresponding to the palm image, performs feature fusion on the image features of multiple scales to obtain an image fusion feature, determines the palm frame corresponding to the palm area in the palm image based on the image fusion feature, cuts out the palm image corresponding to the palm area from the palm image based on the palm frame, obtains encrypted palm image data by encrypting the palm image, and performs identity recognition by sending the encrypted palm image data to a palm recognition server for decryption and palm comparison recognition. In this application, by cutting out the palm image corresponding to the palm area from the palm image and encrypting, transmitting, and decrypting only the palm image, the recognition time for the palm image is reduced and the recognition efficiency of palm recognition is improved.

[0030] Figure 2 shows a schematic diagram of the architecture of a computer system provided in one embodiment of the present invention. The computer system may include a palm recognition device 100 and a palm recognition server 200.

[0031] The palm recognition device 100 may be an electronic device such as a mobile phone, tablet computer, in-vehicle terminal (in-vehicle device), wearable device, personal computer (PC), voice interaction device with palm image recognition function, home appliance with palm image recognition function, aircraft, or unmanned sales terminal. The palm recognition device 100 may have a client installed on it that runs an application, and this application may be an application that specializes in palm image recognition, or it may be another application that provides palm image recognition function, but this application is not limited thereto. Furthermore, in this application, the form of the application is not limited and includes, but is not limited to, an application (App) installed on the palm recognition device 100, a child application, a web page program, etc. Here, a child application is a program that runs in the execution environment provided by the parent application, the parent application is an independent native application, and the child application runs depending on the parent application.

[0032] The palm recognition server 200 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing base cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data or artificial palm image recognition platforms. The palm recognition server 200 may also be a backend server for the above application, providing backend services to the application's clients.

[0033] Here, Cloud Technology is a hosting technology that integrates a set of resources such as hardware, software, and networks within a wide-area or local network to enable data computation, storage, processing, and sharing. Cloud Technology is a general term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to business modes based on cloud computing. It is flexible and convenient, as it can configure a resource pool and be utilized on demand. Cloud computing technology is a crucial support. Backend services of technological network systems require a large amount of computing resources, storage resources, such as video sites, picture sites, and many portal sites. With the advanced development and application of the internet industry, in the future, individual items may have their own identification marks, and all of these will need to be transmitted to the backend system for logic processing. Different levels of data will be processed separately, and each type of industry data will require strong system support as a backing, which can only be achieved with cloud computing.

[0034] In some embodiments, the above server may be implemented as a node in a blockchain system. Blockchain is a new application mode of computer technology, including distributed data storage, point-to-point transmission, consensus mechanisms, and cryptographic algorithms. A blockchain is essentially a decentralized database, a series of data blocks generated and linked together using cryptographic methods, each data block containing information for one lot of network transactions, verifying the validity (counterfeit prevention) of that information, and generating the next block. A blockchain may include a blockchain base-level platform, a platform product and service layer, and an application service layer.

[0035] Communication between the palm recognition device 100 and the palm recognition server 200 is possible, for example, via a wired or wireless network.

[0036] Figure 3 is a flowchart of a palm image processing method provided in one exemplary embodiment of the present application. The method is performed by a palm recognition device. To make it clear, the method may also be performed by a server. The method includes the following steps:

[0037] In step 302, a palm image is acquired, and image features across multiple scales are obtained by extracting features from the palm image.

[0038] Here, a palm image is an image of a real person's palm. The real person's palm includes the palm and fingers, with the palm being the organic part connecting the arm and fingers of the real person. A palm image contains footage of all or part of the real person's palm. For the sake of explanation, the footage of the real person's palm in a palm image will hereafter be referred to as the palm. The palm included in a palm image may be the palm of the real person of the user whose identity is being recognized. The palm in a palm image may include at least the palm, and may also include fingers or part of the arm, and may also include footage of the environment in which the real person's palm was located at the time the palm was photographed.

[0039] The scale is the size of the image features and can be expressed in terms of the matrix dimensions, for example, 100*100, 80*80, 60*60, or 40*40. Multiple scales refer to two or more scales, each being different. The largest of the multiple scales may be the same as the scale of the palm image, but is generally smaller than the palm image scale.

[0040] The palm image may be a photograph taken by the palm recognition device of the palm of the actual user being verified, or it may be transmitted from another device. The palm recognition device may acquire the palm image by directly collecting an image of the actual palm. The palm recognition device may acquire the palm image by collecting an image of the actual palm using an image collector connected to it. The palm recognition device may acquire image features of multiple scales corresponding to the palm image by performing feature extraction on the acquired palm image.

[0041] For example, the palm recognition device is a store register device, and the store register device acquires an image of the user's palm by taking a picture of the user's actual palm with a camera. Alternatively, the palm recognition device is a palm image recognition server, and after the store register device takes an image of the user's palm, it transmits this palm image to the palm image recognition server.

[0042] For example, image features across multiple scales may be extracted from each of the low-resolution images after downsampling a high-resolution palm image into low-resolution images of multiple scales. Here, high resolution and low resolution are relative concepts, with high resolution being higher than low resolution.

[0043] For example, the input palm image size is 640*640, and the output image features of multiple scales are 80*80, 40*40, and 20*20.

[0044] In step 304, image fusion features are obtained by fusing image features from multiple scales.

[0045] Palm recognition devices can acquire image fusion features by performing feature fusion on image features of multiple scales.

[0046] Image features across multiple scales refer to features obtained by performing feature extraction on images across multiple scales. Image features across different scales can represent different information. In this case, among image features across multiple scales, the larger the scale, the more spatial information is represented, and the smaller the scale, the more semantic information is represented.

[0047] For example, a palm recognition device extracts different classification features from different receptive fields across multiple scales and fuses image features from multiple scales to obtain image fusion features corresponding to a palm image.

[0048] In step 306, a palm frame is determined to identify the palm area in the palm image based on image fusion features.

[0049] The palm frame refers to a mark used to indicate the location of the palm in a palm image. The palm frame includes the palm in the palm image. The boundary of the palm frame surrounds the palm in the palm image.

[0050] Optionally, the shape of the palm frame may be a regular shape, for example, at least one of a rectangle, square, rhombus, hexagon, circle, or triangle. The shape of the palm frame may also be irregular. A regular shape is a shape that can be drawn based on fixed rules with fixed parameters. An irregular shape cannot be drawn based on fixed rules with fixed parameters and can generally only be represented by a set of points that make up the irregular shape.

[0051] The palm recognition device may use an artificial intelligence method to make predictions based on image fusion features and determine the palm frame corresponding to the palm area in the palm image.

[0052] In some embodiments, a computer device may determine the position in a palm image of a palm frame that identifies the palm in the palm image based on image fusion features. If the palm frame has a regular shape, its position can be represented by the values ​​of fixed parameters necessary to draw the regular shape, for example, the positions of the fixed points of the palm frame and the size parameter of the palm frame. The fixed points are points fixed relative to the position of the palm frame. The size parameter is a parameter that represents the size of the palm frame. For example, if the palm frame is rectangular, the fixed points may be any corner points or the center point of the rectangle, and the size parameter is the side lengths of adjacent sides of the palm frame.

[0053] In step 308, the palm image is cropped from the palm image based on the palm frame.

[0054] The palm image is the effective recognition area in the palm image, or the palm image refers to the area in the palm image where the palm is located, or the palm image refers to the area in the palm image that can be used for palm recognition.

[0055] For example, a palm recognition device crops out the image region outside the palm frame in the palm image based on the position of the palm frame, and the remaining image becomes the palm image.

[0056] In step 310, the palm image is encrypted to obtain encrypted palm image data, which is then transmitted to the server, where it is used to perform identity recognition based on the encrypted palm image data.

[0057] The palm image encryption data is data obtained by encrypting a palm image, and the palm image can be obtained by decrypting the palm image encryption data. Here, the palm image encryption data may be a bitstream or a string of characters, or it may be an image whose content is difficult to identify with the naked eye. The palm image is transmitted to a server, which uses the palm image encryption data to perform identity recognition.

[0058] The palm recognition device may access a network and transmit encrypted palm image data to a server via the network. The palm recognition device may transmit encrypted palm image data to a terminal via a point-to-point connection, and the terminal, having accessed the network, may transmit the encrypted palm image data to the server. If the palm recognition device itself is the server, it may transmit encrypted palm image data to other servers via a direct cable (also called a straight cable) or a network.

[0059] In some embodiments, the server may obtain a palm image by decrypting encrypted palm image data and perform identity recognition based on the decrypted palm image.

[0060] In some embodiments, the server may obtain a palm image by decrypting encrypted palm image data, search a database for a palm image that matches the decrypted palm image, and if a matching palm image is found, determine the user identifier associated with the matching palm image and perform identity recognition. If no matching palm image is found, it may be determined that identity recognition has failed.

[0061] In some embodiments, the server may decrypt encrypted palm image data to obtain a palm image, extract palm features from the palm image, search a database for palm features that match the extracted palm features, and if a matching palm feature is found, determine the user identifier associated with the matching palm feature and perform identity recognition.

[0062] In some embodiments, the server may perform identity recognition based on encrypted palm image data to determine a user identifier, and then further verify the permissions corresponding to the user identifier. If it is verified that the user identifier has the permissions for a predetermined function, the server may further execute or trigger this predetermined function. This predetermined function may be, for example, access control or other functions that require verification of user permissions.

[0063] The method by which the palm recognition device transmits encrypted palm image data to the palm recognition server may optionally include, but is not limited to, at least one of network transmission, data line transmission, or Bluetooth transmission. The embodiments of this application are not specifically limited thereto.

[0064] The palm recognition device may perform decryption and palm comparison recognition based on encrypted palm image data, and then perform identity recognition. Specifically, palm comparison recognition may involve comparing palm features in the palm image with predetermined palm features in a database.

[0065] A predetermined palm feature is a palm feature of the palm of a stored user identifier. Each predetermined palm feature has a corresponding user identifier. This indicates that the predetermined palm feature belongs to the user identifier and is a palm feature of the user's palm. The user identifier may be any user identifier. For example, the user identifier may be a user identifier registered in a payment application. Alternatively, the user identifier may be a user identifier registered with a company.

[0066] In the embodiments of the present invention, the palm recognition server includes a database containing a plurality of predetermined palm features and a user identifier corresponding to each predetermined palm feature. In this database, there may be a one-to-one correspondence between predetermined palm features and user identifiers, or one user identifier may correspond to at least two predetermined palm features.

[0067] For example, multiple users register with a payment application, and each user's predetermined palm features are linked to a corresponding user identifier. This allows the palm features of multiple users to be associated with their corresponding user identifiers and stored in a database. Subsequently, when a user uses the payment application, palm comparison recognition is performed between the first palm region of the infrared image, the second palm region of the color image, and the predetermined palm features in the database. This determines the target user identifier and enables user identity verification.

[0068] For example, a palm recognition device sends encrypted palm image data to a palm recognition server for decryption and palm comparison recognition, thereby obtaining a user identifier corresponding to the palm image.

[0069] As described above, the method provided in this embodiment acquires a palm image and performs feature extraction to obtain image features of multiple scales corresponding to the palm image, performs feature fusion on the image features of multiple scales to obtain an image fusion feature, performs prediction based on the image fusion feature to obtain a palm frame corresponding to the palm in the palm image, cuts out the palm image corresponding to the palm from the palm image based on the palm frame, encrypts the palm image to obtain encrypted palm image data, and sends the encrypted palm image data to a palm recognition server to decrypt and perform palm comparison recognition to obtain a user identifier corresponding to the palm image. In this application, by cutting out the palm image corresponding to the palm from the palm image and encrypting, transmitting, and decrypting only the palm image, the recognition time for the palm image is reduced and the recognition efficiency of palm recognition is improved.

[0070] Figure 4 is a flowchart of a palm image processing method provided in one exemplary embodiment of the present application. The method is performed by a palm recognition device. The method includes the following steps:

[0071] In step 402, a palm image is acquired, and features are extracted from the palm image using the backbone network in the palm detection model, thereby obtaining image features across multiple scales.

[0072] The palm image is the palm image used to determine the user identifier.

[0073] Exemplary, a palm recognition device acquires a palm image by photographing the palm of a user's physical hand. Here, the palm image includes the palm, which may be the user's left palm or the user's right palm.

[0074] For example, the palm recognition device is an Internet of Things (IoT) device, and the IoT device acquires a palm image by photographing the user's left palm with a camera. The IoT device may also be a store register device. Alternatively, for example, when a user makes a purchase and conducts a transaction at a store, the user extends their palm towards the camera of the store's payment terminal, and the store's payment terminal acquires a palm image by photographing the user's actual palm with its camera.

[0075] In some embodiments, a palm recognition device establishes a communication connection with another device and receives a palm image transmitted from the other device via this connection. For example, the palm recognition device may be a payment application server, and the other device may be a payment terminal. The payment terminal acquires a palm image by capturing the palm of a user's body, and then transmits the palm image to the payment application server via the communication connection between the payment terminal and the payment application server. This allows the payment application server to determine the user identifier of the palm image.

[0076] In some embodiments, the palm frame is predicted by a palm detection model, which includes a backbone network, and the step of obtaining image features of multiple scales by extracting features from a palm image includes the step of obtaining sliced ​​images of multiple scales by performing a slicing operation on the palm image using the backbone network, and the step of obtaining image features of multiple scales by extracting features from each of the sliced ​​images of multiple scales using the backbone network.

[0077] Here, the palm detection model is an artificial intelligence model, which may be a deep learning model or a neural network model. The palm detection model has at least the function of determining a palm frame that identifies the palm in a palm image based on image fusion features. The backbone network is a configuration in the palm detection model for extracting image features of multiple scales from a palm image. The slicing operation is an operation that divides the palm image to generate images of even smaller scales. Even smaller scales refer to the scale of the generated image being smaller than the scale of the palm image. The pixel points of the sliced ​​image originate from the palm image.

[0078] The palm recognition device may acquire sliced ​​images of multiple scales by performing slicing operations using a backbone network. The palm recognition device may also acquire image features of multiple scales corresponding to the palm image by performing feature extraction on the sliced ​​images of multiple scales.

[0079] In some embodiments, the step of obtaining slice images of multiple scales by performing a slicing operation on a palm image using a backbone network includes the steps of: determining the largest scale slice image among the multiple scales based on the palm image using the backbone network; and starting layer-by-layer downsampling with the largest scale slice image as the first layer using the backbone network, thereby obtaining slice images of multiple scales that include the slice image of the first layer.

[0080] The computer device may first determine the largest-scale slice image among several scales based on the palm image. The scale of this largest-scale slice image may be less than or equal to the scale of the palm image. The computer device may generate the largest-scale slice image among several scales by downsampling the palm image once.

[0081] Downsampling a palm image is the process of sampling every other pixel in the palm image. By stitching together the downsampled pixel points in the same order as in the palm image, a slice image with a smaller scale can be obtained.

[0082] If there are multiple scales and the number of scales is N, then there are N scales. Since each of the N scales is distinct, there is a maximum scale among the N scales. The slice image of the maximum scale is used as the first layer. Downsampling is performed layer by layer from the first layer, downsampling the (N-1)th layer until the slice image of the Nth layer is obtained, and so on, downsampling the nth layer until the slice image of the (n+1)th layer is obtained. n is a positive integer from 1 to N-1. Finally, a total of N slice images from the first to the Nth layer can be obtained.

[0083] In this embodiment, slice images of multiple scales are obtained by layer-by-layer downsampling, and a new means of separating slice images of multiple scales from a palm image is provided.

[0084] In some embodiments, the step of obtaining slice images of multiple scales by performing a slicing operation on a palm image using a backbone network includes the step of obtaining slice images of multiple scales by sampling and stitching together pixel points in the palm image at multiple scales using a backbone network. Here, two adjacent pixel points in slice images of different scales have different numbers of pixel points between them at the sampling location of the palm image.

[0085] In this embodiment, the palm recognition device may obtain a slice image for each scale by sampling pixel points from the palm image at each scale using a sampling interval for that scale, and then stitching together the sampled pixel points according to their relative positional relationships in the palm image. Here, the sampling interval is the number of pixel points in the interval, and includes the spacing between pixel points in both row and column directions. There is a negative correlation between the scale and the sampling interval, where a larger scale results in a smaller sampling interval, and a smaller scale results in a larger sampling interval. Pixel points in the palm image that were not sampled may be concentrated in the channels of the image features.

[0086] For example, by sampling every other row in the palm image using the largest of several scales, that is, by selecting a pixel point every other pixel in the row direction and joining the selected pixel points, a slice image can be obtained. While no information is lost in the joined slice image, by concentrating the size information of the palm image into the channel and performing a convolution operation on the resulting new picture, a downsampled feature map can be obtained without any loss of information.

[0087] Furthermore, for example, by sampling every two columns in the palm image using the second largest of several scales, that is, by selecting pixel points every two pixels and joining the selected pixel points, a slice image can be obtained. While no information is lost in the joined slice image, by concentrating the size information of the palm image into the channels and performing a convolution operation on the resulting new picture, it is possible to obtain a downsampled feature map that does not lose any information.

[0088] For example, if the original image has a size of 640*640*3, a slicing operation will yield a feature map of 320*320*12. Here, the 640*640 in the original image represents the width * height, and the 3 in the original image represents the length of the feature vector corresponding to each pixel (also called the number of channels).

[0089] In step 404, the neck network of the palm detection model is used to perform feature fusion on image features of multiple scales, thereby obtaining image fusion features.

[0090] The palm frame is predicted by a palm detection model, which includes a neck network, and the step of obtaining an image fusion feature by fusing image features of multiple scales includes the step of obtaining an image fusion feature by performing feature fusion on image features of multiple scales using the neck network.

[0091] Here, the neck network belongs to the palm detection model and is configured to obtain image fusion features by performing feature fusion on image features of multiple scales. The neck network is connected to the backbone network and receives image features of multiple scales output from the backbone network.

[0092] Image features across multiple scales refer to features obtained by performing feature extraction on images across multiple scales. Image features across different scales can represent different information. In this case, among image features across multiple scales, the larger the scale, the more spatial information is represented, and the smaller the scale, the more semantic information is represented.

[0093] For example, a palm recognition device extracts different classification features from different receptive fields across multiple scales and fuses image features from multiple scales to obtain image fusion features corresponding to a palm image.

[0094] In some embodiments, the step of obtaining an image fusion feature by performing feature fusion on image features of multiple scales using a neck network includes the step of obtaining an image fusion feature by fusing image features of multiple scales on the same scale using a neck network. Here, fusing image features of the same scale may involve calculating the sum or the mean.

[0095] By fusing image features from multiple scales, palm recognition devices can increase the detection accuracy of small targets, more accurately detect the palm frame, more precisely crop palm images, further reduce the amount of encrypted palm image data, improve the efficiency of identity recognition, and ensure the security of identity recognition.

[0096] In some embodiments, the step of obtaining an image fusion feature by performing feature fusion on image features of multiple scales using a neck network includes the steps of obtaining first intermediate features of multiple scales by performing feature fusion based on image features of multiple scales using a neck network, obtaining second intermediate features of multiple scales by performing feature fusion on multiple scales based on the first intermediate features of multiple scales using a neck network, and obtaining an image fusion feature by performing feature fusion on the second intermediate features of multiple scales using a neck network.

[0097] Here, feature fusion of multiple scales refers to performing feature fusion on different scales. The first intermediate features of multiple scales may correspond one-to-one with the image features of multiple scales. The first intermediate features of multiple scales may correspond one-to-one with the second intermediate features of multiple scales. The multiple scales of the image features may coincide with the multiple scales of the first intermediate features. The multiple scales of the first intermediate features may coincide with the multiple scales of the second intermediate features. Performing feature fusion on the second intermediate features of multiple scales may also mean unifying the second intermediate features of multiple scales to the same scale and then fusing them. This fusion may involve calculating the sum or the mean.

[0098] In this embodiment, by performing feature fusion at different scales and finally fusing them into image fusion features, it is possible to fully represent the fine features in the palm image, making it easier to detect the palm frame more accurately, allowing for more accurate cropping of the palm image, further reducing the amount of encrypted palm image data, improving the efficiency of identity recognition, and ensuring the security of identity recognition.

[0099] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the step of extracting the first intermediate features of the multiple scales based on the image features of the multiple scales includes the step of determining the first-ranked first intermediate feature in the correct order of the scales based on the first-ranked image feature in the correct order of the scales; the step of obtaining the (n+1)-ranked first intermediate feature in the correct order of the scales by upsampling layer by layer from the first-ranked first intermediate feature in the correct order of the scales and fusing the upsampled result of the nth-ranked first intermediate feature in the correct order of the scales with the (n+1)-ranked image feature in the correct order of the scales; and the step of determining the first intermediate features of the multiple scales from the first to the Nth layer after obtaining the first intermediate feature of the Nth layer, where n is a positive integer from 1 to N-1, and the correct order of the scales is from smallest to largest.

[0100] Here, the order from smallest to largest is called the true order, and the true order of the scale is from smallest to largest. Upsampling is the process of expanding the scale. Upsampling may also be performed by targeting the scale of the next layer and interpolating adjacent feature elements in the features of each layer to obtain the features of the next layer.

[0101] In some embodiments, the step of determining the first intermediate feature in the first order of the scale based on the first image feature in the first order of the scale includes the step of obtaining the first intermediate feature in the first order of the scale by performing at least one convolution on the first image feature in the first order of the scale. In addition to convolution, channel adjustment may be performed. After channel adjustment, the number of channels in the first intermediate feature can be matched.

[0102] In some embodiments, the scale of the upsampled result of the nth-rank first intermediate feature in the correct order of the scale may be the same as the scale of the (n+1)th-rank image feature in the correct order of the scale. Merging the upsampled result of the nth-rank first intermediate feature in the correct order of the scale with the (n+1)th-rank image feature in the correct order of the scale may involve joining the two, or calculating a sum or mean using the same scale.

[0103] As an example, as shown in the schematic diagram of image feature fusion in Figure 5, the palm recognition device obtains image features 502 of multiple scales corresponding to the palm image 501 by performing feature extraction on the palm image 501. The image features 502 include feature layers T1, T2, T3, and T4. After obtaining the image features 502, feature layer P4 is obtained by convolving feature layer T4 once and adjusting the channels. Feature layer P3 is obtained by upsampling feature layer P4 and combining it with feature layer T3, then performing convolution and channel adjustment. Feature layer P2 is obtained by upsampling feature layer P3 and combining it with feature layer T2, then performing convolution and channel adjustment. Feature layer P1 is obtained by upsampling feature layer P2 and combining it with feature layer T1, then performing 1*1 convolution and channel adjustment.

[0104] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the step of obtaining the second intermediate features of the multiple scales by merging the features of the multiple scales based on the first intermediate features of the multiple scales includes the step of determining the first-rank second intermediate feature in reverse order of the scales based on the first-rank first intermediate feature in reverse order of the scales (reverse order of scales is from largest to smallest scale), the step of downsampling from the first-rank second intermediate feature in reverse order of the scales layer by layer, and merging the downsampled result of the second intermediate feature at the m-th rank (m is an integer from 1 to N-1) in reverse order of the scales with the first intermediate feature at the (m+1)th rank in reverse order of the scales to obtain the second intermediate feature at the (m+1)th rank in reverse order of the scales, and the step of determining the second intermediate features of the multiple scales from the 1st to the Nth layer after obtaining the second intermediate features of the Nth layer.

[0105] Here, the order from largest to smallest is called the reverse order, and the reverse order of the scale is the order from largest to smallest. Downsampling is the process of reducing the scale. Downsampling may also be performed by targeting the scale of the next layer and intermittently adopting feature elements from the features of each layer to obtain the features of the next layer. Here, the first intermediate feature can be called a semantic image feature. The second intermediate feature can be called a spatial image feature. Semantic image features represent more semantic information, and spatial image features represent more spatial information.

[0106] In some embodiments, the step of determining the second intermediate feature, which is the first in the scale order, based on the first intermediate feature, which is the first in the scale order, includes the step of obtaining the second intermediate feature, which is the first in the scale order, by performing at least one convolution on the first intermediate feature, which is the first in the scale order. In addition to convolution, channel adjustment may be performed. After channel adjustment, the number of channels in the first intermediate feature can be matched.

[0107] The scale of the downsampled second intermediate feature at the m-th position in reverse scale order may be the same as the scale of the first intermediate feature at the (m+1)-th position in reverse scale order. Merging the downsampled second intermediate feature at the m-th position in reverse scale order with the first intermediate feature at the (m+1)-th position in reverse scale order may involve joining the two together, or calculating a sum or mean for both using the same scale.

[0108] For example, a palm recognition device may obtain feature layer N1 by convolving feature layer P1 once and adjusting the channels. Feature layer N2 is obtained by downsampling feature layer N1 and combining it with feature layer P2, then performing convolution and channel adjustment. Feature layer N3 is obtained by downsampling feature layer N2 and combining it with feature layer P3, then performing convolution and channel adjustment. Feature layer N4 is obtained by downsampling feature layer N3 and combining it with feature layer P4, then performing convolution and channel adjustment. The palm recognition device obtains image fusion feature 503 by performing feature fusion on feature layers N1 to N4.

[0109] Optionally, the activation function in convolution is a logistic activation function.

[0110] In some embodiments, the palm recognition device obtains a first intermediate feature by upsampling image features of multiple scales from a small scale to a large scale, obtains a second intermediate feature by downsampling image features of multiple scales from a large scale to a small scale, and obtains an image fused feature by performing feature fusion of the first and second intermediate features.

[0111] In step 406, the palm detection model's predictive network determines a palm frame that identifies the palm in the palm image based on image fusion features.

[0112] The palm frame refers to a mark used to indicate the position of the palm in a palm image.

[0113] In some embodiments, the palm frame is determined by a palm detection model, which includes a prediction network, and the step of determining a palm frame that identifies the palm in a palm image based on image fusion features includes the steps of obtaining multiple mesh features of the image fusion features by meshing the image fusion features, and obtaining the palm frame by inputting the multiple mesh features into a prediction network and making predictions for the multiple mesh features.

[0114] Here, the mesh feature is a meshed image fusion feature obtained by meshing the image fusion feature. For example, a palm recognition device may obtain multiple mesh features by meshing the image fusion feature, and the palm recognition device may obtain prediction results corresponding to each mesh feature by performing predictions based on each mesh feature using a prediction network. Furthermore, the position of the palm frame in the palm image may be determined by combining the prediction results corresponding to each of the multiple mesh features.

[0115] In some embodiments, the palm recognition device is equipped with an infrared camera, and the palm image processing method further includes the steps of: acquiring an infrared image that captures the same palm as the palm image, wherein the infrared image refers to an image of the palm captured by the infrared camera based on infrared light; and recognizing the palm region from the infrared image. The step of cropping the palm image from the palm image based on the palm frame includes the steps of: determining the common set of palm frames in the palm image and the palm region in the infrared image; and cropping the palm image from the palm image based on the common set.

[0116] In this embodiment, the palm recognition device is further equipped with an infrared camera. The palm recognition device acquires an infrared image corresponding to the palm of the same entity, and performs region recognition on the infrared image to determine the palm region in the infrared image. The infrared image refers to an image of the palm captured by the infrared camera based on infrared light.

[0117] In some embodiments, the step of recognizing a palm region from an infrared image includes the steps of detecting webbing (also called interdigital webbing) in the infrared image and determining the palm region in the infrared image based on the webbing.

[0118] For example, a palm recognition device detects webbing in an infrared image and, based on the webbing, determines the palm region in the infrared image.

[0119] As shown in the schematic diagram of the webbing in the palm of the hand in Figure 6, the webbing is the first webbing 601 between the index finger and the middle finger, or the second webbing 602 between the middle finger and the ring finger, or the third webbing 603 between the ring finger and the little finger.

[0120] Since the palm region in a palm image can exist in any region of the palm image, webbing detection is performed on the palm image to obtain at least one webbing in the palm image, enabling the determination of the position of the palm region in the palm image. This makes it possible to subsequently determine the palm region based on the at least one webbing.

[0121] In step 408, the palm image is cropped from the palm image based on the palm frame.

[0122] The palm image is the effective recognition area in the palm image, or the palm image refers to the area in the palm image where the palm is located, or the palm image refers to the area in the palm image that can be used for palm recognition.

[0123] In some embodiments, a palm recognition device acquires at least three webbed feet in an infrared image, connects the webbed feet in sequence to obtain webbed foot-connection lines, and the palm recognition device cuts out the palm region from the infrared image based on the webbed foot-connection lines.

[0124] In some embodiments, the step of determining the common set of the palm frame in the palm image and the palm region in the infrared image includes the steps of obtaining the coordinate parameters of the palm frame in the palm image and obtaining the coordinate parameters of the palm region in the infrared image, and determining the common set of the palm frame and the palm region based on the coordinate parameters of the palm frame and the coordinate parameters of the palm region.

[0125] The palm recognition device obtains coordinate parameters of the palm frame in the palm image and coordinate parameters of the palm region in the infrared image. Based on the intersection of the coordinate parameters of the palm frame and the coordinate parameters of the palm region, the palm recognition device extracts the palm image corresponding to the palm from the palm image.

[0126] In step 410, the palm image is encrypted to obtain encrypted palm image data. This encrypted palm image data is transmitted to the server, which uses it to perform identity recognition.

[0127] Here, the encrypted palm image data is sent to a palm recognition server, where it is decrypted and palm comparison recognition is performed to obtain a user identifier corresponding to the palm image. Palm comparison recognition refers to the process of comparing palm features in the palm image with predetermined palm features in the database.

[0128] Furthermore, the palm, as one of its biological features, possesses uniqueness and distinctiveness. Compared to facial recognition, which is currently widely applied in fields such as identity verification, payment, access control, and transportation, the palm is unaffected by makeup, masks, sunglasses, etc., and can improve the accuracy of user identity verification. In some scenarios, for example, in infection prevention scenarios, it is necessary to wear a mask to cover the mouth and nose, and in this case, identity verification using palm images can be a better option.

[0129] Cross-device registration recognition is a crucial capability for user experience. In the case of two associated device types, a user can register on one type of device, linking their user identifier to their palm features. The user can then perform identity verification on the other type of device. Because mobile phones and Internet of Things (IoT) devices have significant differences in image style and quality, cross-device registration recognition allows users to register on their mobile phone and then use it directly on an IoT device, eliminating the need for the user to register on both types of devices. For example, a user can register on their mobile phone and then perform identity verification directly on a store's device, eliminating the need for the user to register on the store's device and preventing user information leakage.

[0130] As described above, the method provided in this embodiment acquires a palm image and performs feature extraction to obtain image features of multiple scales corresponding to the palm image, performs feature fusion on the image features of multiple scales to obtain an image fusion feature, performs prediction based on the image fusion feature to obtain a palm frame corresponding to the palm in the palm image, cuts out the palm image corresponding to the palm from the palm image based on the palm frame, encrypts the palm image to obtain encrypted palm image data, and sends the encrypted palm image data to a palm recognition server to decrypt and perform palm comparison recognition to obtain a user identifier corresponding to the palm image. In this application, by cutting out the palm image corresponding to the palm from the palm image and encrypting, transmitting, and decrypting only the palm image, the recognition time for the palm image is reduced and the recognition efficiency of palm recognition is improved.

[0131] The method provided in this embodiment obtains image features of multiple scales by performing slicing operations and feature extraction on palm images using a backbone network. According to this method, features of different scales, different receptive fields, and different classifications are obtained, improving the accuracy of palm frame prediction.

[0132] In the method provided in this embodiment, a neck network is used to obtain a first intermediate feature by upsampling image features of multiple scales from a small scale to a large scale, a second intermediate feature by downsampling image features of multiple scales from a large scale to a small scale, and an image fused feature by merging the first and second intermediate features. According to the above method, image features of multiple scales can be merged, increasing the detection accuracy of small targets in palm images.

[0133] In the method provided in this embodiment, meshed image fusion features are obtained by meshing the image fusion features, and a palm frame is obtained by making predictions based on the meshed image fusion features using a prediction network. According to the above method, the accuracy of palm frame prediction is improved and the security of identity recognition by the palm is further guaranteed.

[0134] In the method provided in this embodiment, an infrared image corresponding to the same palm area is acquired, and the palm area in the infrared image is determined by performing region recognition on the infrared image. Based on the common set of the palm frame and the palm area in the infrared image, the palm image corresponding to the palm is cropped from the palm image. According to the above method, the palm image corresponding to the palm is acquired using both the palm area and the palm frame in the infrared image, improving the accuracy of the palm image cropping and making the acquired palm image more precise.

[0135] Figure 7 is a schematic diagram of a palm image processing method provided in one exemplary embodiment of the present application. The method is performed by a palm recognition device. The method includes the following steps:

[0136] The palm recognition device acquires a palm image 701 corresponding to the palm area, and inputs the acquired palm image 701 into a backbone network 702 to perform feature extraction, thereby acquiring image features of multiple scales corresponding to the palm image 701.

[0137] The palm recognition device acquires image fusion features by performing feature fusion on image features of multiple scales using the neck network 703.

[0138] The palm recognition device uses a prediction network 704 to make predictions based on image fusion features, thereby obtaining a palm frame corresponding to the palm area in the palm image 701. Based on the palm frame, the palm recognition device extracts a palm image 705 corresponding to the palm area from the palm image 701.

[0139] The palm recognition device inputs the palm image 705 into the encryption network 706 to encrypt the image and obtains encrypted palm image data 707.

[0140] The palm recognition device transmits encrypted palm image data 707 to the decryption network 709 at the palm recognition server via network transmission 708 to decrypt the image, and then performs palm comparison recognition via the verification network 710 to obtain a user identifier 711 corresponding to the palm image 701.

[0141] As described above, the method provided in this embodiment extracts a palm image corresponding to the palm area from the palm image, and performs image encryption, transmission, and decryption only on the palm image, thereby reducing the time required for palm image recognition and improving the recognition efficiency of palm recognition.

[0142] Figure 8 is a schematic diagram of a cross-device payment using a palm image processing method provided in one exemplary embodiment of the present application. The method relates to a user terminal 801, a store terminal 803, and a payment application server 802.

[0143] Here, the user terminal 801 has a payment application installed. The user terminal 801 logs into the payment application based on its user identifier and establishes a communication connection with the payment application server 802. Through this communication connection, the user terminal 801 and the payment application server 802 can communicate with each other. The store terminal 803 also has a payment application installed. The store terminal 803 logs into the payment application based on its store identifier and establishes a communication connection with the payment application server 802. Through this communication connection, the store terminal 803 and the payment application server 802 can communicate with each other.

[0144] This cross-device payment process includes the following steps:

[0145] In step 1, the user holds the user terminal 801 in their hand at home and takes a picture of their own palm using the user terminal 801 to obtain an image of the user's palm. Based on the user identifier, the user logs into the payment application and sends a palm image registration request containing the user identifier and the palm image to the payment application server 802.

[0146] In step 2, when the payment application server 802 receives a palm image registration request sent from the user terminal 801, it performs feature extraction on the palm image to obtain the palm features of the palm image, stores the palm features in association with the user identifier, and sends a palm image linking success notification to the user terminal 801.

[0147] Here, the payment application server 802 stores the palm feature and the user identifier in association, and then designates the palm feature as a predetermined palm feature. Subsequently, the corresponding user identifier can be determined using the stored predetermined palm feature.

[0148] In step 3, when the user terminal 801 receives a notification of successful palm image linking, it displays the notification and informs the user that the palm image and the user identifier have been linked.

[0149] Here, the user completes the registration of their palm image through communication between their user terminal 801 and the payment application server 802. Subsequently, automatic payment can be achieved using the palm image.

[0150] In step 4, when a user purchases goods at a store and completes a transaction, the store terminal 803 takes a picture of the user's palm to obtain a palm image, logs into the payment application based on the store identifier, and sends a payment request to the payment application server 802. The payment request includes the store identifier, the amount to be spent, and the palm image.

[0151] In step 5, when the payment application server 802 receives a payment request, it performs palm comparison recognition on the palm image, determines the user identifier of the palm image, determines the account for that user identifier in the payment application, completes the transfer using that account, and once the transfer is complete, sends a payment completion notification to the store terminal 803.

[0152] Here, after registering a palm image using the user terminal 801, the user can directly make a payment using their palm at the store terminal 803, eliminating the need for the user to register a palm image at the store terminal 803. This realizes the effectiveness of cross-device palm image recognition and improves convenience.

[0153] In step 6, when the store terminal 803 receives a payment completion notification, it displays the payment completion notification and informs the user that the transaction between the user and the store is complete and the user can take the goods home.

[0154] Furthermore, in the above embodiment, the user terminal 801 and the store terminal 803 realize the cross-device payment process, but the store terminal 803 may be replaced with a payment device on the bus, and the configuration for cross-device boarding payment may be realized according to the above steps.

[0155] Figure 9 is a schematic diagram of cross-device identity verification using a palm image processing method provided in one exemplary embodiment of the present application. The method relates to a user terminal 901, an access control device 903, and an access control server 902.

[0156] Here, the user terminal 901 establishes a communication connection with the access control server 902. Through this communication connection, the user terminal 901 and the access control server 902 can communicate with each other. The access control device 903 establishes a communication connection with the access control server 902. Through this communication connection, the access control device 903 and the access control server 902 can communicate with each other.

[0157] This cross-device identity verification process includes the following steps:

[0158] In step 1, the user holds the user terminal 901 in their hand at home and uses the user terminal 901 to take a picture of their own palm, thereby obtaining an image of the user's palm. The user then sends a palm registration request containing the user identifier and the palm image to the access control server 902.

[0159] In step 2, when the access control server 902 receives a palm registration request sent from the user terminal 901, it performs feature extraction on the palm image to obtain the palm features of the palm image, stores the palm features in association with the user identifier, and sends a palm linking success notification to the user terminal 901.

[0160] Here, the access control server 902 may store the palm feature and the user identifier in association, and then designate the palm feature as a predetermined palm feature. Subsequently, the corresponding user identifier can be determined based on the stored predetermined palm feature.

[0161] In step 3, when the user terminal 901 receives a palm linking success notification, it displays the palm linking success notification and informs the user that the palm image and the user identifier have been linked.

[0162] Here, the user completes the registration of their palm image through communication between their user terminal 901 and the access control server. Subsequently, it becomes possible to automatically open the door based on the palm image.

[0163] In step 4, when the user returns home from going out, the access control device 903 takes a picture of the user's palm to obtain an image of the user's palm, and sends an identity verification request to the access control server 902 with this palm image attached.

[0164] In step 5, the access control server 902 receives an identity verification request sent from the access control device 903, recognizes the palm image to be verified, obtains the user identifier of the palm image, determines that the user is a registered user, and sends a verification success notification to the access control device 903.

[0165] In step 6, the access control device 903 receives a verification pass notification sent from the access control server 902 and controls the door to open and allow the user to enter the room based on this verification pass notification.

[0166] In the above embodiment, the user terminal 901 and the access control device 903 realize a cross-device identity verification process.

[0167] As can be seen from the cross-device identity verification scenario described above, whether it is the palm registration stage where communication takes place between the user terminal 901 and the access control server 902, or the palm image processing stage where communication takes place with the server via another terminal device, after the user terminal 901 or the other terminal device acquires a palm image, the region where the palm is located in the palm image is cropped and encrypted, and the palm image of the region where the palm is located is sent to the server, where the server performs palm comparison recognition. Furthermore, in the palm comparison recognition stage, the access control server 902 obtains the recognition result of the current user by comparing the palm features with predetermined palm features.

[0168] Figure 10 is a schematic diagram of a palm image processing method provided in one exemplary embodiment of the present application. The method is performed by a palm recognition device. The method includes the following steps:

[0169] The palm recognition device acquires a palm image 1001 corresponding to the palm area, inputs the palm image 1001 to the focus adjustment layer 1002 in the backbone network, and the focus adjustment layer 1002 acquires image features of multiple scales by performing a slicing operation on the palm image 1001. The image features are input to the convolutional layer 1003 for convolution and channel adjustment, and the result is input to the bottleneck layer 1004 for feature extraction. The pooling layer 1005 performs pooling on the image features.

[0170] For example, when a palm image 1001 is input to the focus adjustment layer 1002, the image features of multiple scales obtained by the focus adjustment layer 1002 performing a slicing operation on the palm image 1001 are represented as (3,64,1,1), the image features after convolution and channel adjustment in the convolution layer 1003 are represented as (64,128,3,2), the image features after passing through the bottleneck layer 1004 are represented as (128,128)*3, the image features after convolution and channel adjustment in the convolution layer 1003 are represented as (128,256,3,2), and the image features after passing through the bottleneck layer 1004 are represented as (256,256)*9. The image features (256,256)*9 are convolved in the convolutional layer 1003 to adjust the channels, resulting in the image features represented as (256,512,3,2). The image features after passing through the bottleneck layer 1004 are represented as (512,512)*9. The image features after convolution again in the convolutional layer 1003 to adjust the channels are represented as (512,1024,3,2). The image features after pooling by the pooling layer 1005 and passing through the bottleneck layer 1004 are represented as (1024,1024)*3. The image feature obtained by convolution of image feature (1024,1024)*3 in convolutional layer 1003 and adjusting the channels is represented as (1024,512,1,1). This is then upsampled 1006 and combined with image feature (512,512)*9 in 1007, and the resulting image feature after passing through bottleneck layer 1004 is represented as (1024,512)*3. The image feature obtained by convolution of image feature (1024,512)*3 in convolutional layer 1003 and adjusting the channels is represented as (512,256,1,1). This is then upsampled 1006 and combined with image feature (256,256)*9 in 1007, and the resulting image feature after passing through bottleneck layer 1004 is represented as (512,256)*3. The image feature obtained by convolution on the image feature (512,256)*3 in the convolutional layer 1003 and adjusting the channels is represented as (512,1024,3,2). The image feature (512,1024,3,2) and the image feature (512,256,1,1) are joined together in layer 1007, and the image feature obtained after passing through the bottleneck layer 1004 is represented as (512,512)*3.In the convolutional layer 1003, the image feature (512,512)*3 is convolved and the channels are adjusted, resulting in an image feature represented as (512,512,3,2). The image feature represented as (512,512,3,2) and the image feature represented as (1024,512,1,1) are joined together in 1007, and the image feature that passes through the bottleneck layer 1004 is represented as (1024,1024)*3. The final output image features 1008 for multiple scales are (512,256), (512,512), and (1024,1024). Here, for (x,y,a,b), x and y can represent the two dimensions of the feature matrix, respectively, and a and b can represent the number of channels and the scale, respectively. In (x,y)*b, x and y can represent the two dimensions of the feature matrix, and b can represent the number of channels.

[0171] The palm recognition device obtains image fusion features by performing feature fusion on multiple scale image features 1008. The palm recognition device obtains a palm frame corresponding to the palm area in the palm image 1001 by making predictions on the image fusion features. Based on the palm frame, the palm recognition device cuts out the palm image corresponding to the palm area from the palm image 1001. The palm recognition device encrypts the palm image and transmits it to the palm recognition server via the network, where it decrypts the image and performs palm comparison recognition to obtain a user identifier corresponding to the palm image 1001.

[0172] As described above, the method provided in this embodiment acquires image features of multiple scales by performing slicing operations and feature extraction on palm images using a backbone network. According to this method, different classification features from different receptive fields of multiple scales are acquired, improving the accuracy of palm frame prediction.

[0173] The palm frame prediction according to the present application may be realized based on a palm detection model. This configuration includes a palm detection model generation step and a palm frame prediction step. Figure 11 is a framework diagram of the palm detection model generation step and the palm frame prediction step shown in one exemplary embodiment of the present application. As shown in Figure 11, in the palm detection model generation step, the palm detection model generation device 1110 acquires a palm detection model using a pre-set training sample dataset (including sample palm images and sample palm images), and then generates a palm frame based on the palm detection model. In the palm frame prediction step, the palm frame prediction processing device 1120 obtains a prediction result of the palm image by processing the input palm image based on the palm detection model, and determines, for example, the area and position of the palm in the palm image.

[0174] Here, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may be computer devices. For example, the computer device may be a stationary computer device such as a personal computer or a server. Alternatively, the computer device may be a portable computer device such as a tablet computer or an e-reader.

[0175] Optionally, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may be the same device. Alternatively, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may be different devices. Furthermore, if the palm detection model generation device 1110 and the palm frame prediction processing device 1120 are different devices, they may be devices of the same type. For example, both the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may be servers. Alternatively, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may be different types of devices. For example, the palm frame prediction processing device 1120 may be a personal computer or terminal, while the palm detection model generation device 1110 may be a server or the like. The embodiments of this application do not limit the specific types of the palm detection model generation device 1110 and the palm frame prediction processing device 1120.

[0176] In the above embodiment, a palm image processing method was described. Next, a method for training a palm detection model will be described.

[0177] Figure 12 is a flowchart of a method for training a palm detection model provided in one exemplary embodiment of the present application. The method is performed by a palm recognition device. The method includes the following steps:

[0178] Step 1201 involves acquiring a sample palm image and a sample palm image.

[0179] The sample palm image is the image used to determine the user identifier. This sample palm image includes a palm, which is an image of the palm of the user whose identity is being verified. This sample palm image may also include other information, such as the user's fingers and the location of the user's palm when it was photographed. This sample palm image may be a photograph of the palm of the user whose identity is being verified taken by the palm recognition device, or it may be transmitted from another device.

[0180] For example, the palm recognition device is a store register device, and the store register device acquires a sample palm image by taking a picture of the user's actual palm with a camera. Alternatively, the palm recognition device is a palm image recognition server, and after the store register device takes a sample palm image of the user, it transmits this sample palm image to the palm image recognition server.

[0181] The sample palm image refers to the annotated effective recognition area in the sample palm image, or the area in the sample palm image where the annotated palm is located, or the sample palm image refers to the annotated area in the sample palm image that can be used for recognition.

[0182] In step 1202, sample image features for multiple scales are obtained by extracting features from the sample palm image.

[0183] Here, the sample palm image is an image of a real person's palm. The sample palm image includes footage of all or part of the real person's palm. For the sake of explanation, the footage of the real person's palm in the sample palm image will be referred to as the palm below. The palm included in the sample palm image may be the real person's palm of the user whose identity is being recognized. The palm in the sample palm image includes at least the palm area, may include fingers or part of the arm, and may include footage of the environment in which the real person's palm was located when it was photographed.

[0184] The scale is the size of the sample image features and can be expressed in terms of the matrix dimensions, for example, 100*100, 80*80, 60*60, or 40*40. Multiple scales refer to two or more scales, each distinct from the others. The largest of the multiple scales may be the same as the scale of the sample palm image, but is generally smaller than the scale of the sample palm image.

[0185] This sample palm image may be a photograph of the palm of the actual user being verified by the palm recognition device, or it may be transmitted from another device. The palm recognition device may obtain the sample palm image by directly collecting an image of the actual palm. The palm recognition device may obtain the sample palm image by collecting an image of the actual palm using an image collector connected to it. The palm recognition device may obtain sample image features of multiple scales corresponding to the sample palm image by performing feature extraction on the acquired sample palm image.

[0186] For example, the palm recognition device is a store register device, and the store register device acquires a sample palm image by taking a picture of the user's actual palm with a camera. Alternatively, the palm recognition device is a sample palm image recognition server, and after the store register device takes a sample palm image of the user, it transmits this sample palm image to the sample palm image recognition server.

[0187] For example, sample image features at multiple scales may be extracted from each of the low-resolution images at multiple scales after downsampling a high-resolution sample palm image to low-resolution images at multiple scales. Here, high resolution and low resolution are relative concepts, with high resolution being higher than low resolution.

[0188] For example, the input sample palm image size is 640*640, and the output sample image features for multiple scales are 80*80, 40*40, and 20*20.

[0189] In step 1203, sample image fusion features are obtained by fusing sample image features from multiple scales.

[0190] A palm recognition device can acquire sample image fusion features by performing feature fusion on sample image features of multiple scales.

[0191] Sample image features across multiple scales refer to features obtained by performing feature extraction on images across multiple scales. Sample image features across different scales can represent different information. In this case, among the sample image features across multiple scales, larger scales represent more spatial information, while smaller scales represent more semantic information.

[0192] For example, a palm recognition device extracts different classification features from different receptive fields across multiple scales and fuses the sample image features from multiple scales to obtain a sample image fused feature corresponding to a sample palm image.

[0193] In step 1204, a sample predicted palm frame is determined to identify the palm area in the sample palm image based on the sample image fusion features.

[0194] The sample prediction palm frame refers to a mark indicating the location of the palm in the sample palm image. The sample prediction palm frame includes the palm in the sample palm image. The boundary of the sample prediction palm frame surrounds the palm in the sample palm image.

[0195] Optionally, the shape of the sample prediction palm frame may be a regular shape, for example, at least one of a rectangle, square, rhombus, hexagon, circle, or triangle. The shape of the sample prediction palm frame may also be irregular. A regular shape is a shape that can be drawn based on fixed rules with fixed parameters. An irregular shape cannot be drawn based on fixed rules with fixed parameters and can generally only be represented by a set of points that make up the irregular shape.

[0196] The palm recognition device may use an artificial intelligence method to make predictions based on sample image fusion features and determine a sample predicted palm frame that identifies the palm area in the sample palm image.

[0197] In some embodiments, a computer device may determine the position in a sample palm image of a sample palm prediction frame that identifies the palm area in the sample palm image based on sample image fusion features. If the sample prediction frame has a regular shape, its position can be represented by the values ​​of fixed parameters necessary to draw the regular shape, for example, by the positions of the fixed points of the sample prediction frame and the size parameter of the sample prediction frame. The fixed points are points fixed relative to the position of the sample prediction frame. The size parameter is a parameter that represents the size of the sample prediction frame. For example, if the sample prediction frame is rectangular, the fixed points may be any corner points or the center point of the rectangle, and the size parameter is the side lengths of adjacent sides of the sample prediction frame.

[0198] In step 1205, the sample predicted palm image is cropped from the sample palm image based on the sample predicted palm frame.

[0199] The sample predicted palm image is the effective recognition area in the sample palm image, or the sample predicted palm image refers to the area in the sample palm image where the palm is located, or the sample predicted palm image refers to the area in the sample palm image that can be used for palm recognition.

[0200] For example, the palm recognition device crops out the image region located outside the sample predicted palm frame in the sample palm image, based on the position of the sample predicted palm frame, and uses the remaining image as the sample predicted palm image.

[0201] In step 1206, the loss function value is calculated based on the sample palm image and the sample predicted palm image.

[0202] For example, a palm recognition device calculates a loss function value based on a sample palm image and a sample predicted palm image.

[0203] In step 1207, the model parameters of the palm detection model are updated based on the loss function value.

[0204] For example, a palm recognition device updates the model parameters of its palm detection model based on the loss function value.

[0205] Model parameter updates refer to, but are not limited to, updating network parameters within the palm detection model, updating network parameters for each network module within the model, or updating network parameters for each network layer within the model. The embodiments of this application are not limited to these.

[0206] The model parameters of the palm detection model include at least one of the network parameters of the backbone network, the neck network, and the prediction network in the palm detection model.

[0207] As described above, the method provided in this embodiment involves performing feature extraction on the acquired sample palm image to obtain sample image features of multiple scales corresponding to the sample palm image, performing feature fusion on the sample image features of multiple scales to obtain sample image fusion features, performing prediction based on the sample image fusion features to obtain a sample predicted palm frame that identifies the palm in the sample palm image, cropping the sample predicted palm image corresponding to the palm from the sample palm image based on the sample predicted palm frame, calculating a loss function value based on the sample palm image and the sample predicted palm image, and updating the model parameters of the palm detection model based on the loss function value. As a result, the trained palm detection model has higher prediction accuracy for the sample predicted palm frame and can obtain a more accurate sample predicted palm frame.

[0208] Figure 13 is a flowchart of a method for training a palm detection model provided in one exemplary embodiment of the present application. The method is performed by a palm recognition device. The method includes the following steps:

[0209] In step 1301, a sample palm image and a sample palm image are obtained.

[0210] The sample palm image is the image used to determine the user identifier. This sample palm image includes a palm, which is an image of the palm of the actual user whose identity is being verified. This sample palm image may also include other information, such as the user's fingers and the location of the user's palm when it was photographed. This sample palm image may be a photograph of the palm of the actual user whose identity is being verified taken by the palm recognition device, or it may be transmitted from another device.

[0211] The sample palm image refers to the annotated effective recognition area in the sample palm image, or the area in the sample palm image where the annotated palm is located, or the sample palm image refers to the annotated area in the sample palm image that can be used for recognition.

[0212] As an example, a sample palm image collection method involves acquiring 50 to 100 sample palm images. The sample palm images are then cleaned up by searching for duplicates, missing values, and outliers, resulting in the acquisition of valid sample palm images. The code for cleaning up the sample palm images can be expressed as follows:

[0213] [Table 1] Sample palm image acquisition distance requirement: Place the palm at 5cm, 8cm, 10cm, 12cm, and 15cm from the palm recognition device and acquire images for 10-15 seconds at each distance.

[0214] Method for acquiring sample palm images: The palm region in the sample palm image is annotated using the annotation tool Labeling. The code for annotating the palm region can be expressed as follows.

[0215] [Table 2] In the code above, 45 represents the minimum value of the palm area on the x-axis, 104 represents the maximum value of the palm area on the x-axis, 34 represents the minimum value of the palm area on the y-axis, and 84 represents the maximum value of the palm area on the y-axis.

[0216] The annotation tool Labeling applies the uniform distribution theorem (also known as the uniform distribution theorem) to the palm region of a sample palm image, thereby creating a label vector y from the conventional one-hot encoding. hot Instead, the updated label vector

number

[0217]

number

[0218] Label vector

number

[0219]

number

number

[0220] In step 1302, the backbone network in the palm detection model is used to extract features from the sample palm image, thereby obtaining sample image features at multiple scales.

[0221] Exemplary, a palm recognition device acquires a sample palm image by photographing the palm of a user's actual hand. Here, the sample palm image includes the palm, which may be the user's left palm or the user's right palm.

[0222] For example, the palm recognition device is an Internet of Things (IoT) device, and the IoT device acquires a sample palm image by photographing the user's left palm with a camera. The IoT device may also be a store register device. Alternatively, for example, when a user makes a purchase and conducts a transaction at a store, the user extends their palm towards the camera of the store's payment terminal, and the store's payment terminal acquires a sample palm image by photographing the user's actual palm with the camera.

[0223] In some embodiments, a palm recognition device establishes a communication connection with another device and receives a sample palm image transmitted from the other device via this connection. For example, the palm recognition device may be a payment application server, and the other device may be a payment terminal. The payment terminal acquires a sample palm image by capturing the palm of a user's body, and then transmits the sample palm image to the payment application server via the communication connection between the payment terminal and the payment application server. This allows the payment application server to determine the user identifier of the sample palm image.

[0224] In some embodiments, the sample predicted palm frame is predicted by a palm detection model, which includes a backbone network, and the step of obtaining sample image features of multiple scales by extracting features from a sample palm image includes inputting the sample palm image into the backbone network, obtaining sliced ​​images of multiple scales by performing a slicing operation on the sample palm image using the backbone network, and obtaining sample image features of multiple scales by extracting features from each of the sliced ​​images of multiple scales using the backbone network.

[0225] Here, the palm detection model is an artificial intelligence model, which may be a deep learning model or a neural network model. The palm detection model has the function of determining a sample predicted palm frame that identifies the palm in the sample palm image based on sample image fusion features. The backbone network is a configuration in the palm detection model for extracting sample image features of multiple scales from the sample palm image. The slicing operation is an operation that divides the sample palm image to generate images of even smaller scales. Even smaller scales refer to the scale of the generated image being smaller than the scale of the sample palm image. The pixel points of the sliced ​​image originate from the sample palm image.

[0226] The palm recognition device may input a sample palm image into a backbone network and perform a slicing operation using the backbone network to obtain sliced ​​images at multiple scales. The palm recognition device may also perform feature extraction on the sliced ​​images at multiple scales to obtain sample image features at multiple scales corresponding to the sample palm image.

[0227] In some embodiments, the step of obtaining slice images of multiple scales by performing a slicing operation on a sample palm image using a backbone network includes the step of determining the largest scale slice image among the multiple scales based on the sample palm image using the backbone network, and the step of starting layer-by-layer downsampling with the largest scale slice image as the first layer using the backbone network, thereby obtaining slice images of multiple scales that include the slice image of the first layer.

[0228] The computer device may first determine the largest-scale slice image among several scales based on the sample palm image. The scale of this largest-scale slice image may be less than or equal to the scale of the sample palm image. The computer device may generate the largest-scale slice image among several scales by downsampling the sample palm image once.

[0229] Downsampling a sample palm image is the process of sampling every other pixel in the sample palm image. By stitching together the downsampled pixel points in the same order as in the original sample palm image, a slice image with a smaller scale can be obtained.

[0230] If there are multiple scales and the number of scales is N, then there are N scales. Since each of the N scales is distinct, there is a maximum scale among the N scales. The slice image of the maximum scale is used as the first layer. Downsampling is performed layer by layer from the first layer, downsampling the (N-1)th layer until the slice image of the Nth layer is obtained, and so on, downsampling the nth layer until the slice image of the (n+1)th layer is obtained. n is a positive integer from 1 to N-1. Finally, a total of N slice images from the first to the Nth layer can be obtained.

[0231] In this embodiment, slice images of multiple scales are obtained by layer-by-layer downsampling, and a new method is provided for separating slice images of multiple scales from a sample palm image.

[0232] In some embodiments, the step of obtaining slice images of multiple scales by performing a slicing operation on a sample palm image using a backbone network includes the step of obtaining slice images of multiple scales by sampling and stitching together pixel points in the sample palm image at multiple scales using a backbone network. Here, two adjacent pixel points in slice images of different scales have different numbers of pixel points between them at the sampling location of the sample palm image.

[0233] In this embodiment, the palm recognition device may obtain slice images for each scale by sampling pixel points from a sample palm image at each scale using a sampling interval for that scale, and then stitching together the sampled pixel points according to their relative positional relationships in the sample palm image. Here, the sampling interval is the number of pixel points in the interval, and includes the spacing between pixel points in both row and column directions. There is a negative correlation between the scale and the sampling interval, where a larger scale results in a smaller sampling interval, and a smaller scale results in a larger sampling interval. Pixel points in the sample palm image that were not sampled may be concentrated in the channels of the sample image features.

[0234] For example, by sampling every other column of a sample palm image using the largest of several scales, that is, by selecting a pixel point every other pixel in the row direction and joining the selected pixel points, a slice image can be obtained. While no information is lost in the joined slice image, by concentrating the size information of the sample palm image into the channels and performing a convolution operation on the resulting new picture, a downsampled feature map can be obtained without any loss of information.

[0235] Furthermore, for example, by sampling every two columns in the sample palm image using the second largest of several scales, that is, by selecting pixel points every two pixels and joining the selected pixel points, a slice image can be obtained. While no information is lost in the joined slice image, by concentrating the size information of the sample palm image into the channels and performing a convolution operation on the resulting new picture, a downsampled feature map can be obtained without any loss of information.

[0236] For example, if the original image has a size of 640*640*3, a slicing operation will yield a feature map of 320*320*12. Here, the 640*640 in the original image represents the width * height, and the 3 in the original image represents the length of the feature vector corresponding to each pixel (also called the number of channels).

[0237] In step 1303, the neck network of the palm detection model is used to perform feature fusion on sample image features of multiple scales, thereby obtaining sample image fusion features.

[0238] The sample predicted palm frame is predicted by a palm detection model, which includes a neck network, and the step of obtaining a sample image fusion feature is obtained by fusing sample image features of multiple scales using the neck network.

[0239] Here, the neck network belongs to the palm detection model and is configured to obtain sample image fusion features by performing feature fusion on sample image features of multiple scales. The neck network is connected to the backbone network and receives sample image features of multiple scales output from the backbone network.

[0240] Sample image features across multiple scales refer to features obtained by performing feature extraction on images across multiple scales. Sample image features across different scales can represent different information. In this case, among the sample image features across multiple scales, larger scales represent more spatial information, while smaller scales represent more semantic information.

[0241] For example, a palm recognition device extracts different classification features from different receptive fields across multiple scales and fuses the sample image features from multiple scales to obtain a sample image fused feature corresponding to a sample palm image.

[0242] In some embodiments, the step of obtaining a sample image fusion feature by performing feature fusion on sample image features of multiple scales using a neck network includes the step of obtaining a sample image fusion feature by fusing sample image features of multiple scales on the same scale using a neck network. Here, fusing sample image features on the same scale may involve calculating the sum or the mean.

[0243] By fusing sample image features from multiple scales, the palm recognition device can increase the detection accuracy of small targets, more accurately detect the sample prediction palm frame, more precisely crop the sample prediction palm image, further reduce the amount of encrypted data in the sample prediction palm image, improve the efficiency of identity recognition, and ensure the security of identity recognition.

[0244] In some embodiments, the step of obtaining a sample image fusion feature by performing feature fusion on sample image features of multiple scales using a neck network includes the steps of obtaining a first intermediate feature of multiple scales by performing feature fusion based on sample image features of multiple scales using a neck network, obtaining a second intermediate feature of multiple scales by performing feature fusion on multiple scales based on the first intermediate features of multiple scales using a neck network, and obtaining a sample image fusion feature by performing feature fusion on the second intermediate features of multiple scales using a neck network.

[0245] Here, feature fusion across multiple scales refers to performing feature fusion on different scales. The first intermediate features of multiple scales may correspond one-to-one with the sample image features of multiple scales. The first intermediate features of multiple scales may correspond one-to-one with the second intermediate features of multiple scales. The multiple scales of the sample image features may coincide with the multiple scales of the first intermediate features. The multiple scales of the first intermediate features may coincide with the multiple scales of the second intermediate features. Performing feature fusion on the second intermediate features of multiple scales may involve unifying the second intermediate features of multiple scales to the same scale before fusion. This fusion may involve calculating the sum or the mean.

[0246] In this embodiment, by performing feature fusion at different scales and finally fusing them with the sample image fusion features, it is possible to fully represent the fine features of the sample palm image, making it easier to detect the sample predicted palm frame more accurately, allowing for more accurate cropping of the sample predicted palm image, further reducing the amount of encrypted data of the sample predicted palm image, improving the efficiency of identity recognition, and ensuring the security of identity recognition.

[0247] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the step of extracting the first intermediate features of the multiple scales based on the sample image features of the multiple scales includes the step of determining the first-ranked first intermediate feature in the correct order of the scales based on the first-ranked sample image feature in the correct order of the scales; the step of obtaining the (n+1)-ranked first intermediate feature in the correct order of the scales by upsampling layer by layer from the first-ranked first intermediate feature in the correct order of the scales and fusing the upsampled result of the nth-ranked first intermediate feature in the correct order of the scales with the (n+1)-ranked sample image feature in the correct order of the scales; and the step of determining the first intermediate features of the multiple scales from the first to the Nth layer after obtaining the first intermediate feature of the Nth layer, where n is a positive integer from 1 to N-1, and the correct order of the scales is from smallest to largest.

[0248] Here, the order from smallest to largest is called the true order, and the true order of the scale is from smallest to largest. Upsampling is the process of expanding the scale. Upsampling may also be performed by targeting the scale of the next layer and interpolating adjacent feature elements in the features of each layer to obtain the features of the next layer.

[0249] In some embodiments, the step of determining the first intermediate feature in the first order of the scale based on the first-ranked sample image feature in the first order of the scale includes the step of obtaining the first intermediate feature in the first order of the scale by performing at least one convolution on the first-ranked sample image feature in the first order of the scale. In addition to convolution, channel adjustment may be performed. After channel adjustment, the number of channels in the first intermediate feature can be matched.

[0250] In some embodiments, the scale of the upsampled first intermediate feature at the nth position in the scale may be the same as the scale of the (n+1)th position sample image feature at the scale in the scale. Merging the upsampled first intermediate feature at the nth position in the scale with the (n+1)th position sample image feature at the scale in the scale may involve joining the two together, or calculating a sum or mean of the two using the same scale.

[0251] As an example, as shown in the schematic diagram of sample image feature fusion in Figure 5, the palm recognition device obtains sample image features 502 at multiple scales corresponding to the sample palm image 501 by performing feature extraction on the sample palm image 501. The sample image features 502 include feature layers T1, T2, T3, and T4. After obtaining the sample image features 502, feature layer P4 is obtained by convolving feature layer T4 once and adjusting the channels. Feature layer P3 is obtained by upsampling feature layer P4 and combining it with feature layer T3, then performing convolution and channel adjustment. Feature layer P2 is obtained by upsampling feature layer P3 and combining it with feature layer T2, then performing convolution and channel adjustment. Feature layer P1 is obtained by upsampling feature layer P2 and combining it with feature layer T1, then performing 1*1 convolution and channel adjustment.

[0252] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the step of obtaining the second intermediate features of the multiple scales by merging the features of the multiple scales based on the first intermediate features of the multiple scales includes the step of determining the first-rank second intermediate feature in reverse order of the scales based on the first-rank first intermediate feature in reverse order of the scales (reverse order of scales is from largest to smallest scale), the step of downsampling from the first-rank second intermediate feature in reverse order of the scales layer by layer, and merging the downsampled result of the second intermediate feature at the m-th rank (m is an integer from 1 to N-1) in reverse order of the scales with the first intermediate feature at the (m+1)th rank in reverse order of the scales to obtain the second intermediate feature at the (m+1)th rank in reverse order of the scales, and the step of determining the second intermediate features of the multiple scales from the 1st to the Nth layer after obtaining the second intermediate features of the Nth layer.

[0253] Here, the order from largest to smallest is called the reverse order, and the reverse order of the scale is the order from largest to smallest. Downsampling is the process of reducing the scale. Downsampling may also be performed by targeting the scale of the next layer and intermittently adopting feature elements from the features of each layer to obtain the features of the next layer. Here, the first intermediate feature can be called the semantic sample image feature. The second intermediate feature can be called the spatial sample image feature. The semantic sample image feature represents more semantic information, and the spatial sample image feature represents more spatial information.

[0254] In some embodiments, the step of determining the second intermediate feature, which is the first in the scale order, based on the first intermediate feature, which is the first in the scale order, includes the step of obtaining the second intermediate feature, which is the first in the scale order, by performing at least one convolution on the first intermediate feature, which is the first in the scale order. In addition to convolution, channel adjustment may be performed. After channel adjustment, the number of channels in the first intermediate feature can be matched.

[0255] The scale of the downsampled second intermediate feature at the m-th position in reverse scale order may be the same as the scale of the first intermediate feature at the (m+1)-th position in reverse scale order. Merging the downsampled second intermediate feature at the m-th position in reverse scale order with the first intermediate feature at the (m+1)-th position in reverse scale order may involve joining the two together, or calculating a sum or mean for both using the same scale.

[0256] For example, a palm recognition device may obtain feature layer N1 by convolving feature layer P1 once and adjusting the channels. Feature layer N2 is obtained by downsampling feature layer N1 and combining it with feature layer P2, then performing convolution and channel adjustment. Feature layer N3 is obtained by downsampling feature layer N2 and combining it with feature layer P3, then performing convolution and channel adjustment. Feature layer N4 is obtained by downsampling feature layer N3 and combining it with feature layer P4, then performing convolution and channel adjustment. The palm recognition device obtains sample image fusion feature 503 by performing feature fusion on feature layers N1 to N4.

[0257] Optionally, the activation function in convolution is a logistic activation function.

[0258] In some embodiments, the palm recognition device obtains a first intermediate feature by upsampling sample image features of multiple scales from a small scale to a large scale, obtains a second intermediate feature by downsampling sample image features of multiple scales from a large scale to a small scale, and obtains a sample image fused feature by performing feature fusion of the first and second intermediate features.

[0259] In step 1304, the palm detection model's prediction network determines a sample predicted palm frame that identifies the palm in the sample palm image based on the sample image fusion features.

[0260] The sample prediction palm frame refers to a mark used to indicate the position of the palm in the sample palm image.

[0261] In some embodiments, the sample predicted palm frame is determined by a palm detection model, which includes a prediction network, and the step of determining the sample predicted palm frame that identifies the palm in a sample palm image based on the sample image fusion features includes the steps of obtaining multiple mesh features of the sample image fusion features by meshing the sample image fusion features, and obtaining the sample predicted palm frame by inputting the multiple mesh features into a prediction network and making predictions for the multiple mesh features.

[0262] For example, a palm recognition device obtains meshed sample image fusion features by meshing the sample image fusion features, inputs the meshed sample image fusion features into a prediction network, and obtains a sample predicted palm frame by having the prediction network make predictions for each mesh. Specifically, the step of obtaining a sample predicted palm frame by making predictions for multiple mesh features may involve determining the position of the sample predicted palm frame by combining the prediction results of multiple mesh features.

[0263] In some embodiments, the palm recognition device is equipped with an infrared camera, and the sample palm image processing method further includes the steps of: acquiring an infrared image that captures the same palm as the sample palm image, wherein the infrared image refers to an image of the palm captured by the infrared camera based on infrared light; and recognizing the palm region from the infrared image. The step of cropping a sample predicted palm image from a sample palm image based on a sample predicted palm frame includes the steps of: determining the common set of sample predicted palm frames in the sample palm image and the palm region in the infrared image; and cropping a sample predicted palm image from the sample palm image based on the common set.

[0264] In this embodiment, an infrared camera is provided in the palm recognition device. The palm recognition device acquires an infrared image corresponding to the palm of the same entity, and the palm recognition device performs region recognition on the infrared image to determine the palm region in the infrared image. The infrared image refers to an image obtained by the infrared camera taking a picture of the palm based on infrared light.

[0265] In some embodiments, the step of recognizing the palm region from the infrared image includes the step of detecting the crease in the infrared image and the step of determining the palm region in the infrared image based on the crease.

[0266] Exemplarily, the palm recognition device detects the crease in the infrared image and determines the palm region in the infrared image based on the crease.

[0267] As shown in the schematic diagram of the crease in the palm shown in FIG. 6, the crease is the first crease 601 between the index finger and the middle finger, or the crease is the second crease 602 between the middle finger and the ring finger, or the crease is the third crease 603 between the ring finger and the little finger.

[0268] Since the palm region in the sample palm image may exist in any region in the sample palm image, in order to enable the determination of the position of the palm region in the sample palm image, crease detection is performed on the sample palm image to obtain at least one crease of the sample palm image. Thereby, it becomes possible to determine the palm region based on the at least one crease subsequently.

[0269] In step 1305, based on the sample predicted palm frame, a sample predicted palm image is cut out from the sample palm image.

[0270] The sample predicted palm image is the predicted effective recognition area in the sample palm image, or the sample predicted palm image refers to the area in the sample palm image where the predicted palm is located, or the sample predicted palm image refers to the predicted area in the sample palm image that can be used for recognition.

[0271] In some embodiments, a palm recognition device acquires at least three webbed feet in an infrared image, connects the webbed feet in sequence to obtain webbed foot connections, and the palm recognition device cuts out the palm region from the infrared image based on the webbed foot connections.

[0272] In some embodiments, the step of determining the common set of the sample predicted palm frame in the sample palm image and the palm region in the infrared image includes the steps of obtaining the coordinate parameters of the sample predicted palm frame in the sample palm image and obtaining the coordinate parameters of the palm region in the infrared image, and determining the common set of the sample predicted palm frame and the palm region based on the coordinate parameters of the sample predicted palm frame and the coordinate parameters of the palm region.

[0273] The palm recognition device obtains coordinate parameters of the sample predicted palm frame in the sample palm image and coordinate parameters of the palm region in the infrared image. Based on the intersection of the coordinate parameters of the sample predicted palm frame and the coordinate parameters of the palm region, the palm recognition device extracts the sample predicted palm image corresponding to the palm from the sample palm image.

[0274] In step 1306, the loss function value is calculated based on the sample palm image and the sample predicted palm image.

[0275] For example, a palm recognition device calculates a loss function value based on a sample palm image and a sample predicted palm image.

[0276] In step 1307, the model parameters of the palm detection model are updated based on the loss function value.

[0277] Model parameter updates refer to, but are not limited to, updating network parameters within the palm detection model, updating network parameters for each network module within the model, or updating network parameters for each network layer within the model. The embodiments of this application are not limited to these.

[0278] The model parameters of the palm detection model include at least one of the network parameters of the backbone network, the network parameters of the neck network, and the network parameters of the prediction network.

[0279] In some embodiments, updating the model parameters of a palm detection model includes updating the network parameters of all network modules within the palm detection model, or fixing the network parameters of some network modules within the palm detection model and updating only the network parameters of the remaining network modules. For example, when updating the model parameters of a palm detection model, the network parameters of the backbone network in the palm detection model are fixed, and only the network parameters of the neck network and the prediction network are updated.

[0280] Based on the loss function value, a trained palm detection model is obtained by updating the network parameters of the backbone network, the neck network, and the prediction network in the palm detection model until the loss function value converges, using the loss function value as the training metric.

[0281] Convergence of the loss function value means, but is not limited to, at least one of the following: the loss function value no longer changes; the difference in error between two adjacent iterations during training of the palm detection model is less than a predetermined value; or the number of training iterations of the palm detection model reaches a predetermined number. The embodiments of this application are not limited to these.

[0282] Optionally, the target condition to be met by training may be that the number of training iterations of the initial model reaches the target number. The engineer can pre-set the number of training iterations. Alternatively, the target condition to be met by training may be, but is not limited to, that the loss value satisfies the target threshold condition. Embodiments of the present application are not limited to these.

[0283] As described above, the method provided in this embodiment involves performing feature extraction on the acquired sample palm image to obtain sample image features of multiple scales corresponding to the sample palm image, performing feature fusion on the sample image features of multiple scales to obtain sample image fusion features, performing prediction based on the sample image fusion features to obtain a sample predicted palm frame that identifies the palm in the sample palm image, cropping the sample predicted palm image corresponding to the palm from the sample palm image based on the sample predicted palm frame, calculating a loss function value based on the sample palm image and the sample predicted palm image, and updating the model parameters of the palm detection model based on the loss function value. As a result, the trained palm detection model has higher prediction accuracy for the sample predicted palm frame and can obtain a more accurate sample predicted palm frame.

[0284] Schematically, the application scenarios for the palm image processing method provided in the embodiments of this application include, but are not limited to, the following scenarios.

[0285] For example, in a palm image recognition payment scenario, The palm recognition device of the store captures the physical palm of the user to obtain the palm image of the user. By the palm image processing method provided in the embodiments of the present application, the target user identifier of the palm image is determined, and a part of the resources in the resource account corresponding to the target user identifier is transferred to the store resource account to realize automatic payment by the palm.

[0286] Also, for example, in a cross-device payment scenario, the user can complete identity registration using their personal mobile phone at home or other private locations, and associate the user's account with the user's palm image. Then, the device in the store recognizes the user's palm image to determine the user's account and directly make a payment through the account.

[0287] Also, for example, in a scenario of pressing a commuting time card, the palm recognition device captures the physical palm of the user to obtain the palm image of the user. By the palm image processing method provided in the embodiments of the present application, the target user identifier of the palm image is determined, a time card pressing mark is created for the target user identifier, and it is determined that the target user identifier has completed pressing the commuting time card at the current time.

[0288] Of course, in addition to the above scenarios, the method provided in the embodiments of the present application can also be applied to other scenarios where palm image processing is required. The embodiments of the present application do not limit specific application scenarios.

[0289] FIG. 14 shows a schematic diagram of the configuration of a palm image processing device provided in one exemplary embodiment of the present application. This device may be implemented as all or part of the palm recognition device by software, hardware, or a combination of both. This device A feature extraction module 1401 acquires image features at multiple scales by acquiring palm images and extracting features from those palm images. A feature fusion module 1402 acquires image fusion features by fusing image features from multiple scales, A prediction module 1403 determines a palm frame that identifies the palm area in a palm image based on image fusion features, A cropping module 1404 that crops the palm image from the palm image based on the palm frame, The encryption module 1405 obtains encrypted palm image data by encrypting a palm image, wherein the encrypted palm image data is transmitted to a server and used by the server to perform identity verification based on the encrypted palm image data.

[0290] Here, the feature extraction module 1401 may be the backbone network of the palm detection model, the feature fusion module 1402 may be the neck network of the palm detection model, the prediction module 1403 may be at least a part of the prediction network of the palm detection model, and the clipping module 1404 may be part of the prediction network, an independent module within the palm detection model, or an independent module outside the palm detection model.

[0291] In some embodiments, the feature extraction module 1401 is the backbone network of the palm detection model, and by performing a slicing operation on the palm image, it obtains sliced ​​images of multiple scales, and by extracting features from each of the sliced ​​images of multiple scales, it obtains image features of multiple scales.

[0292] In some embodiments, the feature extraction module 1401 is the backbone network of the palm detection model, which determines the largest scale slice image among multiple scales based on the palm image, uses the largest scale slice image as the first layer, starts layer-by-layer downsampling, and obtains slice images of multiple scales that include the slice image of the first layer.

[0293] In some embodiments, the feature extraction module 1401 is the backbone network of the palm detection model, and obtains slice images of multiple scales by sampling and stitching together pixel points in the palm image at multiple scales. Here, two adjacent pixel points in slice images of different scales have different numbers of pixel points between them at the sampling location in the palm image.

[0294] In some embodiments, the feature fusion module 1402 is the neck network of the palm detection model, and it obtains image fusion features by performing feature fusion on image features of multiple scales.

[0295] In some embodiments, the feature fusion module 1402 is a neck network of the palm detection model, and obtains first intermediate features of multiple scales by performing feature fusion based on image features of multiple scales, obtains second intermediate features of multiple scales by performing feature fusion based on the first intermediate features of multiple scales, and obtains image fusion features by performing feature fusion on the second intermediate features of multiple scales.

[0296] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1. The feature fusion module 1402 is a neck network of the palm detection model. Based on the first-ranked image feature in ascending order of scales, it determines the first intermediate feature in ascending order of scales. From the first intermediate feature in ascending order of scales, it upsamples layer by layer. By fusing the upsampled result of the first intermediate feature in the nth rank (where n is a positive integer from 1 to N-1) in ascending order of scales with the (n+1)th-ranked image feature in ascending order of scales, it obtains the (n+1)th-ranked first intermediate feature in ascending order of scales. Once the first intermediate feature of the Nth layer is obtained, the first intermediate features of multiple scales from the first to the Nth layer are determined.

[0297] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1. The feature fusion module 1402 is a neck network of the palm detection model. Based on the first intermediate feature of the first rank in reverse order of the scales (reverse order of scales is from largest to smallest scale), it determines the second intermediate feature of the first rank in reverse order of the scales. It then downsamples from the second intermediate feature of the first rank in reverse order of the scales layer by layer. By fusing the downsampled result of the second intermediate feature of the mth rank (where m is a positive integer from 1 to N-1) in reverse order of the scales with the first intermediate feature of the (m+1)th rank in reverse order of the scales, it obtains the second intermediate feature of the (m+1)th rank in reverse order of the scales. Once the second intermediate feature of the Nth layer is obtained, the second intermediate features of multiple scales from the first to the Nth layer are determined.

[0298] In some embodiments, the prediction module 1403 is at least part of the prediction network of the palm detection model, and obtains multiple mesh features of the image fusion features by meshing the image fusion features, and obtains the palm frame by making predictions based on the multiple mesh features.

[0299] In some embodiments, the palm recognition device is equipped with an infrared camera, and the acquisition module 1406 acquires an infrared image of the same palm area as the palm image, where the infrared image refers to an image of the palm captured by the infrared camera based on infrared light.

[0300] The region recognition module 1407 recognizes the palm region from the infrared image.

[0301] In some embodiments, the cropping module 1404 determines the common set of the palm frame in the palm image and the palm region in the infrared image, and crops the palm image from the palm image based on the common set.

[0302] In some embodiments, the region recognition module 1407 detects webbing in the infrared image and determines the palm region in the infrared image based on the webbing.

[0303] In some embodiments, the region recognition module 1407 acquires at least three webbed feet in an infrared image, connects the webbed feet in sequence to obtain a webbed foot connection line, and cuts out the palm region from the infrared image based on the webbed foot connection line.

[0304] In some embodiments, the cropping module 1404 obtains coordinate parameters of the palm frame in the palm image, obtains coordinate parameters of the palm region in the infrared image, and determines the common set of the palm frame and the palm region based on the coordinate parameters of the palm frame and the palm region.

[0305] Figure 15 shows a schematic diagram of the configuration of a training device for a palm detection model provided in one exemplary embodiment of the present application. This device may be implemented as all or part of a palm recognition device by software, hardware, or a combination thereof. The device includes a palm detection model, an acquisition module 1501, a computation module 1506, and an update module 1507. The palm detection model includes a feature extraction module 1502, a feature fusion module 1503, a prediction module 1504, and may further include a clipping module 1505. Here, the feature extraction module 1502 may be the backbone network of the palm detection model, the feature fusion module 1503 may be the neck network of the palm detection model, the prediction module 1504 may be at least part of the prediction network, and the clipping module 1505 may be part of the prediction network, an independent module within the palm detection model, or an independent module outside the palm detection model.

[0306] The acquisition module 1501 acquires a sample palm image and a sample palm image.

[0307] The feature extraction module 1502 obtains sample image features at multiple scales by extracting features from sample palm images.

[0308] The feature fusion module 1503 obtains sample image fusion features by fusing sample image features of multiple scales.

[0309] The prediction module 1504 determines a sample predicted palm frame that identifies the palm area in the sample palm image based on the sample image fusion features.

[0310] The cropping module 1505 crops the sample predicted palm image from the sample palm image based on the sample predicted palm frame.

[0311] The calculation module 1506 calculates the loss function value based on the sample palm image and the sample predicted palm image.

[0312] The update module 1507 updates the model parameters of the palm detection model based on the loss function value.

[0313] In some embodiments, the feature extraction module 1502 is the backbone network of the palm detection model, and by performing a slicing operation on a sample palm image, it obtains sliced ​​images of multiple scales, and by extracting features from each of the sliced ​​images of multiple scales, it obtains sample image features of multiple scales.

[0314] In some embodiments, the feature extraction module 1502 is the backbone network of the palm detection model, which determines the largest scale slice image among multiple scales based on a sample palm image, uses the largest scale slice image as the first layer, starts layer-by-layer downsampling, and obtains slice images of multiple scales that include the slice image of the first layer.

[0315] In some embodiments, the feature extraction module 1502 is the backbone network of the palm detection model, and obtains slice images of multiple scales by sampling and stitching together pixel points in a sample palm image at multiple scales. Here, two adjacent pixel points in slice images of different scales have different numbers of pixel points between them at the sampling location in the sample palm image.

[0316] In some embodiments, the feature fusion module 1503 is the neck network of the palm detection model, and it obtains sample image fusion features by performing feature fusion on sample image features of multiple scales.

[0317] In some embodiments, the feature fusion module 1503 is a neck network of the palm detection model. Sample image features of multiple scales are input to the neck network, and feature fusion is performed based on the sample image features of multiple scales to obtain first intermediate features of multiple scales. Feature fusion is then performed based on the first intermediate features of multiple scales to obtain second intermediate features of multiple scales, and feature fusion is performed on the second intermediate features of multiple scales to obtain sample image fusion features.

[0318] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1. The feature fusion module 1503 is a neck network of the palm detection model. Based on the first-rank sample image feature in ascending order of scales, it determines the first intermediate feature in ascending order of scales. From the first intermediate feature in ascending order of scales, it upsamples layer by layer. By fusing the upsampled result of the nth-rank first intermediate feature in ascending order of scales (where n is a positive integer from 1 to N-1) with the (n+1)th-rank sample image feature in ascending order of scales, it obtains the (n+1)th-rank first intermediate feature in ascending order of scales. Once the first intermediate feature of the Nth layer is obtained, the first intermediate features of multiple scales from the first to the Nth layer are determined.

[0319] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1. The feature fusion module 1503 determines the second intermediate feature, which is the first in the reverse order of the scales, based on the first intermediate feature, which is the first in the reverse order of the scales (in descending order of scales). It then downsamples each layer from the second intermediate feature, which is the first in the reverse order of the scales, and fuses the downsampled result of the second intermediate feature, which is the mth in the reverse order of the scales (where m is a positive integer from 1 to N-1), with the first intermediate feature, which is the (m+1)th in the reverse order of the scales, to obtain the second intermediate feature, which is the (m+1)th in the reverse order of the scales. Once the second intermediate feature of the Nth layer is obtained, the second intermediate features of the multiple scales from the first to the Nth layer are determined.

[0320] In some embodiments, the prediction module 1504 is at least part of the prediction network of the palm detection model, and obtains multiple mesh features of the sample image fusion features by meshing the sample image fusion features, and obtains a sample predicted palm frame by making predictions on the multiple mesh features.

[0321] In some embodiments, the palm recognition device is equipped with an infrared camera, and the acquisition module 1501 acquires an infrared image of the same palm as the sample palm image, where the infrared image refers to an image of the palm captured by the infrared camera based on infrared light.

[0322] The region recognition module 1508 recognizes the palm region from an infrared image.

[0323] The cropping module 1505 determines the common set of the sample palm area in the sample palm image and the palm area in the infrared image, and crops the palm image from the sample palm image based on the common set.

[0324] In some embodiments, the region recognition module 1508 detects webbing in the infrared image and determines the palm region in the infrared image based on the webbing.

[0325] In some embodiments, the cropping module 1505 obtains coordinate parameters of the sample predicted palm frame in the sample palm image, obtains coordinate parameters of the palm region in the infrared image, and determines the common set of the sample predicted palm frame and the palm region based on the coordinate parameters of the sample predicted palm frame and the coordinate parameters of the palm region.

[0326] In some embodiments, the region recognition module 1508 detects webbing in the infrared image and determines the palm region in the infrared image based on the webbing.

[0327] In some embodiments, the region recognition module 1508 acquires at least three webbed feet in an infrared image, connects the webbed feet in sequence to obtain webbed foot connecting lines, and cuts out the palm region from the infrared image based on the webbed foot connecting lines.

[0328] Figure 16 shows a block diagram of a computer device 1600 as shown in one exemplary embodiment of the present invention. This computer device can be implemented as a server in the above configuration of the present invention. The computer device 1600 includes a central processing unit (CPU) 1601, system memory 1604 including random access memory (RAM) 1602 and read-only memory (ROM) 1603, and a system bus 1605 connecting the system memory 1604 and the central processing unit 1601. The computer device 1600 further includes a mass storage device 1606 for storing an operating system 1609, application programs 1610, and other program modules 1611.

[0329] The mass storage device 1606 is connected to the central processing unit 1601 via a mass storage controller (not shown) connected to the system bus 1605. The mass storage device 1606 and its associated computer-readable media provide non-volatile storage to the computer device 1600. In other words, the mass storage device 1606 may include computer-readable media (not shown), such as a hard disk or a Compact Disc Read-Only Memory (CD-ROM) drive.

[0330] Without loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, movable and immovable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, Erasable Programmable Read Only Memory (EPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), flash memory or other solid-state storage devices, CD-ROM, Digital Versatile Disc (DVD) or other optical storage devices, tape cartridges, magnetic tapes, magnetic disk storage devices or other magnetic storage devices. Of course, those skilled in the art will see that computer storage media are not limited to these. The system memory 1604 and mass storage devices 1606 described above may be collectively referred to as memory.

[0331] According to various embodiments of this disclosure, the computer device 1600 may be connected to and run on a remote computer on a network, such as the Internet. That is, the computer device 1600 may be connected to a network 1608 via a network interface unit 1607 connected to a system bus 1605, or in other words, it may be connected to other types of networks or remote computer systems (not shown) using the network interface unit 1607.

[0332] The memory further includes at least one computer-readable instruction, and at least one computer-readable instruction is stored in the memory, and the central processing unit 1601 implements all or part of the steps of the palm image processing method or palm detection model training method shown in each of the above embodiments by executing the at least one computer-readable instruction.

[0333] In embodiments of the present invention, a computer device is provided comprising a processor and memory, the memory storing at least one program, which, when loaded and executed by the processor, implements a palm image processing method or a palm detection model training method provided in each embodiment of the above-described method.

[0334] In embodiments of the present invention, a computer-readable storage medium is provided, which stores at least one computer-readable instruction, and when loaded and executed by a processor, the at least one computer-readable instruction enables the implementation of the palm image processing method or the palm detection model training method provided in the embodiments of each of the above methods.

[0335] In embodiments of the present invention, a computer program product including computer-readable instructions is provided, the computer-readable instructions are stored in a computer-readable storage medium, and the processor of a computer device reads and executes the computer-readable instructions from the computer-readable storage medium, thereby causing the computer device to execute the palm image processing method or the palm detection model training method provided in each embodiment of the above method.

[0336] To ensure understanding, in the specific embodiments of this application, with regard to data related to user data processing concerning user identity or characteristics, such as palm images, historical data, and images, user permission or consent is required to apply the above embodiments of this application to specific products or technologies, and the collection, use, and processing of the relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0337] It should be understood that "plural" as used herein refers to two or more. "And / or" describes the relationship between related objects and indicates that there are three possible relationships. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, or B existing alone. The letter " / " generally indicates that the preceding and succeeding related objects are in an "or" relationship.

[0338] As those skilled in the art will understand, all or part of the steps for realizing the above embodiment may be performed by hardware or by instructing the relevant hardware via a program. The program may be stored on a computer-readable storage medium. The storage medium mentioned above may be read-only memory, a magnetic disk, or an optical disk, etc.

[0339] The constituent elements of the embodiments described above may be combined in any way. For the sake of brevity, not all possible combinations of the constituent elements of the embodiments described above are described. However, these combinations of constituent elements should be considered to fall within the scope described herein, as long as they do not contradict each other.

[0340] The embodiments described above are merely examples of some embodiments of the present application, and although the descriptions are specific and detailed, they should not be understood as limiting the scope of the patent. It should be noted that, for those skilled in the art, some modifications and improvements are possible without departing from the concept of the present application. All of these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application should be in accordance with the attached claims.

Claims

1. A palm image processing method performed by a palm recognition device, The steps include acquiring a palm image and extracting features from the palm image to obtain image features of multiple scales, The steps include obtaining image fusion features by fusing image features of the aforementioned multiple scales, The steps include determining a palm frame that identifies the palm area in the palm image based on the aforementioned image fusion features, The steps include: cutting out the palm image from the palm image based on the palm frame, A step of obtaining encrypted palm image data by encrypting the palm image, wherein the encrypted palm image data is transmitted to a server and used by the server to perform identity recognition based on the encrypted palm image data, Palm image processing method.

2. The palm frame is predicted by a palm detection model, and the palm detection model includes a backbone network. The step of obtaining image features of multiple scales by extracting features from the aforementioned palm image is as follows: The steps include obtaining slice images of multiple scales by performing a slicing operation on the palm image using the backbone network, The process includes the step of obtaining image features for multiple scales by extracting features from slice images of multiple scales using the backbone network, The palm image processing method according to claim 1.

3. The step of obtaining slice images of multiple scales by performing a slicing operation on the palm image using the backbone network is as follows: The backbone network determines the largest slice image among the multiple scales based on the palm image, The backbone network includes the step of starting layer-by-layer downsampling with the largest scale slice image as the first layer, and obtaining slice images of multiple scales including the slice image of the first layer. The palm image processing method according to claim 2.

4. The step of obtaining slice images of multiple scales by performing a slicing operation on the palm image using the backbone network is as follows: The process includes the step of obtaining slice images of the multiple scales by sampling and stitching together pixel points in the palm image at multiple scales using the backbone network, Two adjacent pixel points in slice images of different scales have different numbers of pixel points between them at the sampling location of the palm image. The palm image processing method according to claim 2.

5. The palm frame is predicted by a palm detection model, and the palm detection model includes a neck network. The step of obtaining an image fusion feature by fusing the image features of the multiple scales is as follows: The process includes the step of inputting the image features of the multiple scales into the neck network and performing feature fusion to obtain the image fusion features, The palm image processing method according to claim 1.

6. The step of obtaining the image fusion feature by inputting the image features of the multiple scales into the neck network and performing feature fusion is as follows: The steps include inputting the image features of the multiple scales into the neck network, The steps include obtaining first intermediate features of multiple scales by performing feature fusion based on the image features of the multiple scales using the aforementioned neck network, The steps include obtaining a second intermediate feature of the multiple scales by performing feature fusion of the multiple scales based on the first intermediate features of the multiple scales using the aforementioned neck network, The process includes the step of obtaining the image fusion feature by performing feature fusion on the second intermediate features of the plurality of scales using the neck network, The palm image processing method according to claim 5.

7. The aforementioned plurality of scales are N scales, where N is a positive integer greater than 1, and the step of obtaining a first intermediate feature of the plurality of scales by performing feature fusion based on the image features of the plurality of scales is as follows: The first intermediate feature is determined based on the image feature that ranks first in the scale in ascending order (the scale is ordered from smallest to largest). The first intermediate feature is obtained by upsampling each layer from the first intermediate feature of the first rank in the correct order of the scale, and fusing the upsampled result of the first intermediate feature of the nth rank (where n is a positive integer from 1 to N-1) in the correct order of the scale with the (n+1)th rank image feature in the correct order of the scale. The steps include obtaining the first intermediate feature of the Nth layer, and determining the first intermediate features of multiple scales from the first to the Nth layer. The palm image processing method according to claim 6.

8. The aforementioned plurality of scales are N scales, where N is a positive integer greater than 1, and the step of obtaining a second intermediate feature of the plurality of scales by performing feature fusion of the plurality of scales based on the first intermediate feature of the plurality of scales is as follows: The first intermediate feature, ranked first in the reverse order of the scale (reverse order of the scale is from largest to smallest), is used to determine the second intermediate feature, ranked first in the reverse order of the scale. The process involves downsampling layer by layer from the first-rank second intermediate feature in reverse order of the scale, fusing the downsampled result of the m-th-rank second intermediate feature (where m is a positive integer from 1 to N-1) in reverse order of the scale with the (m+1)-th-rank first intermediate feature in reverse order of the scale to obtain the (m+1)-th-rank second intermediate feature in reverse order of the scale. The process includes obtaining the second intermediate feature of the Nth layer, and determining the second intermediate features of multiple scales from the first to the Nth layer. The palm image processing method according to claim 6.

9. The palm frame is predicted by a palm detection model, and the palm detection model includes a prediction network. The step of determining a palm frame that identifies the palm area in the palm image based on the image fusion features is: The steps include obtaining a plurality of mesh features of the image fusion feature by mesh division of the image fusion feature, The process includes the step of inputting the plurality of mesh features into the prediction network and performing predictions on the plurality of mesh features to obtain the palm frame, The palm image processing method according to claim 1.

10. The aforementioned palm recognition device is equipped with an infrared camera, and the palm image processing method is as follows: A step of acquiring an infrared image of the same palm as the aforementioned palm image, wherein the infrared image refers to an image of the palm captured by the infrared camera based on infrared light. The method further includes the step of recognizing the palm region from the infrared image, The step of cropping the palm image from the palm image based on the palm frame is: The steps include determining the common set of the palm frame in the palm image and the palm region in the infrared image, The step of cutting out a palm image from the palm image based on the aforementioned common set, The palm image processing method according to claim 1.

11. The step of recognizing the palm region from the infrared image is as follows: The steps include detecting the webbing in the infrared image, The step of determining the palm region in the infrared image based on the webbed feet, The palm image processing method according to claim 10.

12. The step of determining the common set of the palm frame in the palm image and the palm region in the infrared image is: The steps include obtaining the coordinate parameters of the palm frame in the palm image and obtaining the coordinate parameters of the palm region in the infrared image, The process includes the step of determining the common set of the palm frame and the palm region based on the coordinate parameters of the palm frame and the coordinate parameters of the palm region. The palm image processing method according to claim 10.

13. A method for training a palm detection model performed by a computer device, Steps include obtaining a sample palm image and a sample palm image, The steps include obtaining sample image features of multiple scales by extracting features from the aforementioned sample palm image, The steps include obtaining sample image fusion features by fusing the sample image features of the aforementioned multiple scales, The steps include determining a sample predicted palm frame that identifies the palm area in the sample palm image based on the sample image fusion features, The steps include: cutting out the sample palm image from the sample palm image based on the sample palm frame, The steps include: calculating a loss function value based on the sample palm image and the sample predicted palm image; The steps include updating the model parameters of the palm detection model based on the loss function value, Training method for palm detection model.

14. A palm image processing device, A feature extraction module that acquires a palm image and extracts features from the palm image to acquire image features at multiple scales, A feature fusion module that obtains image fusion features by fusing image features of the aforementioned multiple scales, A prediction module that determines a palm frame for identifying the palm area in the palm image based on the aforementioned image fusion features, A cropping module that crops the palm image from the palm image based on the palm frame, An encryption module that obtains encrypted palm image data by encrypting the palm image, wherein the encrypted palm image data is transmitted to a server and used by the server to perform identity verification based on the encrypted palm image data, includes: Palm image processing device.

15. A training device for a palm detection model, comprising a palm detection model, further comprising an acquisition module, a cropping module, a calculation module, and an update module, wherein the palm detection model comprises a feature extraction module, a feature fusion module, and a prediction module. The acquisition module acquires a sample palm image and a sample palm image. The feature extraction module obtains sample image features of multiple scales by extracting features from the sample palm image. The feature fusion module obtains sample image fusion features by fusing the sample image features of the multiple scales, The prediction module determines a sample predicted palm frame that identifies the palm area in the sample palm image based on the sample image fusion features, The cropping module crops the sample palm image from the sample palm image based on the sample palm frame. The calculation module calculates a loss function value based on the sample palm image and the sample predicted palm image, The update module updates the model parameters of the palm detection model based on the loss function value. A training device for palm detection models.

16. A computer device comprising a processor and memory, wherein the memory stores at least one computer-readable instruction, and when the at least one computer-readable instruction is loaded and executed by the processor, the computer device causes the processor to implement the palm image processing method according to any one of claims 1 to 12, or the palm detection model training method according to claim 13.

17. A computer program that causes a computer to perform the palm image processing method described in any one of claims 1 to 12, or the palm detection model training method described in claim 13.

Images