Hand image processing method and apparatus, device, storage medium, and program product

MYPI2025000906A0Undetermined Publication Date: 2024-05-21TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
MYPI2025000906
Authority / Receiving Office
MY · MY
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-11-21
Filing Date
2023-09-13
Publication Date
2024-05-21

AI Technical Summary

Technical Problem

In existing palm recognition technology, the palm image encryption and decryption process takes a long time, resulting in low palm recognition efficiency.

Method used

By extracting multiple scale features of the palm image, performing feature fusion, determining the palm frame, cropping the palm image, and performing image encryption, only the palm image is encrypted and transmitted, and the palm detection model training and prediction network are used to improve recognition. efficiency.

Benefits of technology

It significantly saves the recognition time of palm recognition, improves the efficiency of palm recognition, and ensures the security of identity recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure PI2025000906_DRAWING_001
    Figure PI2025000906_DRAWING_001
  • Figure PI2025000906_DRAWING_002
    Figure PI2025000906_DRAWING_002
  • Figure PI2025000906_DRAWING_003
    Figure PI2025000906_DRAWING_003
Patent Text Reader

Abstract

ABSTRACT HAND IMAGE PROCESSING METHOD AND APPARATUS, DEVICE, STORAGE MEDIUM, AND PROGRAM PRODUCT A hand image processing method, executed by a hand identification device, and comprising: acquiring a hand image, and performing feature extraction on the hand image to obtain image features of a plurality of sizes (302); fusing said image features to obtain a fused image feature (304); on the basis of the fused image feature, determining a palm bounding box for identifying the palm in the hand image (306); on the basis of the palm bounding box, extracting a palm image from the hand image (308); and performing image encryption on the palm image to obtain palm image encrypted data, wherein the palm image encrypted data is to be transmitted to a server, and the server performs identity identification on the basis of the palm image encrypted data (310). FIG. 3
Need to check novelty before this filing date? Find Prior Art

Description

Palm image processing method, device, equipment, storage medium and program product

[0001] Related applications

[0002] This application claims priority to Chinese patent application number 2022114733356, filed on November 21, 2022, entitled “Method, device, equipment, storage medium and program product for palm image recognition,” the entire text of which is hereby incorporated by reference. Technical Field

[0003] The embodiments of the present application relate to the field of computer technology, further to the field of computer vision technology, and in particular to a palm image processing method, apparatus, device, storage medium, and program product. Background Art

[0004] With the development of computer technology, palm recognition technology is becoming more and more widely used and can be applied in various scenarios. For example, palm recognition can be used to verify user identity in payment scenarios or clocking in at work.

[0005] In related technologies, a user provides a physical palm, and a palm recognition device collects a palm image of the user's palm, and then encrypts the palm image based on the palm image recognition device, and the encrypted palm image is used for identity verification.

[0006] However, it takes a lot of time to encrypt and decrypt the palm image, which reduces the efficiency of palm recognition.

[0007] Summary of the Invention

[0008] According to various embodiments of the present application, a palm image processing method, apparatus, device, storage medium, and program product are provided.

[0009] In one aspect, a palm image processing method is provided, which is performed by a palm recognition device and includes:

[0010] Acquire a palm image, extract features from the palm image, and obtain image features at multiple scales;

[0011] fusing the image features at the multiple scales to obtain an image fusion feature;

[0012] determining a palm frame identifying a palm in the palm image based on the image fusion feature;

[0013] Based on the palm frame, cropping a palm image from the palm image; and

[0014] The palm image is encrypted to obtain palm image encrypted data, which is transmitted to a server, and the server performs identity recognition based on the palm image encrypted data.

[0015] In another aspect, the present application provides a palm detection model training method, performed by a computer device, comprising:

[0016] Acquire a sample palm image and a sample palm image;

[0017] Extracting features from the sample palm image to obtain sample image features at multiple scales;

[0018] Fusing the sample image features at the multiple scales to obtain a sample image fusion feature;

[0019] determining a sample predicted palm frame for identifying a palm in the sample palm image based on the sample image fusion feature;

[0020] Based on the sample predicted palm frame, cropping a sample predicted palm image from the sample palm image;

[0021] Calculating a loss function value according to the sample palm image and the sample predicted palm image; and

[0022] Model parameters of the palm detection model are updated based on the loss function value.

[0023] In another aspect, the present application provides a palm image processing device, comprising:

[0024] A feature extraction module is used to obtain a palm image, extract features from the palm image, and obtain image features at multiple scales;

[0025] A feature fusion module, configured to fuse the image features at multiple scales to obtain image fusion features;

[0026] a prediction module, configured to determine a palm frame identifying a palm in the palm image based on the image fusion feature;

[0027] a cropping module, configured to crop a palm image from the palm image based on the palm frame; and

[0028] The encryption module is used to encrypt the palm image to obtain palm image encrypted data, and the palm image encrypted data is used to transmit to the server, and the server performs identity authentication based on the palm image encrypted data.

[0029] On the other hand, the present application provides a training device for a palm detection model, comprising a palm detection model, an acquisition module, a cropping module, a calculation module, and an update module, wherein the palm detection model comprises a feature extraction module, a feature fusion module, and a prediction module;

[0030] The acquisition module is used to acquire a sample palm image and a sample palm image;

[0031] The feature extraction module is used to extract features from the sample palm image to obtain sample image features at multiple scales;

[0032] The feature fusion module is used to fuse the sample image features of the multiple scales to obtain sample image fusion features;

[0033] The prediction module is configured to determine a sample predicted palm frame identifying the palm in the sample palm image based on the sample image fusion features;

[0034] The cropping module is configured to crop a sample predicted palm image from the sample palm image based on the sample predicted palm frame;

[0035] The calculation module is used to calculate the loss function value according to the sample palm image and the sample predicted palm image;

[0036] The updating module is used to update the model parameters of the palm detection model based on the loss function value.

[0037] On the other hand, the present application also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the steps of the various method embodiments of the present application when executing the computer-readable instructions.

[0038] On the other hand, the present application also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, perform the steps of the various method embodiments of the present application.

[0039] On the other hand, the present application also provides a computer program product, comprising computer-readable instructions, which, when executed by a processor, perform the steps of the method embodiments of the present application.

[0040] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] FIG1 is a schematic diagram of a palm image processing method provided by an exemplary embodiment of the present application;

[0043] FIG2 is a schematic diagram of the architecture of a computer system provided by an exemplary embodiment of the present application;

[0044] FIG3 is a flowchart of a palm image processing method provided by an exemplary embodiment of the present application;

[0045] FIG4 is a flowchart of a palm image processing method provided by an exemplary embodiment of the present application;

[0046] FIG5 is a schematic diagram of image feature fusion provided by an exemplary embodiment of the present application;

[0047] FIG6 is a schematic diagram of finger gaps in a palm provided by an exemplary embodiment of the present application;

[0048] FIG7 is a schematic diagram of a palm image processing method provided by an exemplary embodiment of the present application;

[0049] FIG8 is a flowchart of an exemplary embodiment of the present application, which shows a palm image processing method applied to a cross-device payment scenario;

[0050] FIG9 is a flowchart of a palm image processing method applied to a cross-device identity authentication scenario according to an exemplary embodiment of the present application;

[0051] FIG10 is a flowchart of a palm image processing method provided by an exemplary embodiment of the present application;

[0052] FIG11 is a framework diagram of a palm detection model generation phase and a palm frame prediction phase provided by an exemplary embodiment of the present application;

[0053] FIG12 is a flowchart of a palm detection model training method provided by an exemplary embodiment of the present application;

[0054] FIG13 is a flowchart of a palm detection model training method provided by an exemplary embodiment of the present application;

[0055] FIG14 is a block diagram of a palm image processing apparatus provided by an exemplary embodiment of the present application;

[0056] FIG15 is a block diagram of a training device for a palm detection model provided by an exemplary embodiment of the present application;

[0057] FIG16 is a schematic diagram of the structure of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0059] Computer vision (CV) is the science of making machines "see." Specifically, it refers to machine vision, where cameras and computers replace the human eye in identifying and measuring objects, performing further image processing to create images more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, aiming to develop artificial hand image recognition systems capable of extracting information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies.

[0060] An embodiment of the present application provides a schematic diagram of a palm image processing method, as shown in FIG1 . The method may be executed by a palm recognition device 10 , which may have an image encryption function.

[0061] Illustratively, the palm recognition device 10 includes a camera, and the palm recognition device 10 uses the camera to capture a palm image 11. The palm recognition device 10 may also be connected to an image collector having a camera, and the palm recognition device 10 may capture the palm image 11 captured by the image collector. The palm recognition device 10 performs palm prediction on the palm image 11 to obtain a palm image 12 corresponding to the palm in the palm image 11. The palm recognition device 10 encrypts the palm image 12 to obtain encrypted palm image data 13. The palm recognition device 10 sends the encrypted palm image data 13 to the palm recognition server for decryption and palm comparison recognition, thereby obtaining a user identifier 14 corresponding to the palm image 11.

[0062] Exemplarily, the palm recognition device 10 acquires a palm image 11 and extracts features from the palm image 11 to obtain image features at multiple scales. The palm recognition device 10 then fuses the image features at multiple scales to obtain fused image features. Based on the fused image features, the palm recognition device 10 determines a palm frame within the palm image 11. Based on the palm frame, the palm recognition device 10 crops a palm image 12 from the palm image 11. The palm recognition device 10 encrypts the palm image 12 to obtain encrypted palm image data 13.

[0063] The palm image encrypted data 13 is sent to a palm recognition server for decryption and palm comparison recognition to obtain a user identification 14 corresponding to the palm image 11 .

[0064] The palm image 12 is an effective recognition area in the palm image 11 , or the palm image 12 refers to the area in the palm image 11 where the palm is located, or the palm image 12 refers to the area in the palm image 11 that can be used for palm recognition.

[0065] Optionally, the palm recognition device 10 sends the palm image encrypted data 13 to the palm recognition server by at least one of network transmission, data line transmission, and Bluetooth transmission, but is not limited thereto. This application does not impose any specific limitation on this.

[0066] For example, the palm recognition device 10 extracts features from the palm image 11 to obtain image features. The palm recognition device 10 performs feature fusion on the image features to obtain image fusion features. The palm recognition device 10 determines a palm frame in the palm image and, based on the palm frame, crops a palm image 12 from the palm image.

[0067] Exemplarily, the palm frame can be predicted by a palm detection model, which includes a backbone network, a neck network, and a prediction network.

[0068] The palm recognition device 10 inputs a palm image 11 into the backbone network. The backbone network slices the palm image to obtain sliced ​​images at multiple scales. The palm recognition device 10 then extracts features from the sliced ​​images at multiple scales to obtain image features corresponding to the palm image 11 at multiple scales. Scale refers to the size of a feature, and multiple scales refer to multiple sizes.

[0069] For example, the size of the input palm image 11 is 640*640, and the sizes of the output image features at multiple scales are: 80*80, 40*40, and 20*20. The scale or size of an image or feature can be the dimension of the matrix representing the image or feature.

[0070] For example, palm recognition device 10 inputs image features at multiple scales into the neck network for feature fusion, obtaining fused image features. Palm recognition device 10 inputs the fused image features into the prediction network for prediction, obtaining a palm frame in palm image 11. Palm recognition device 10 crops the image area outside the palm frame from palm image 11, and uses the remaining cropped image as palm image 12.

[0071] In summary, the method provided in this embodiment obtains image features at multiple scales corresponding to the palm image by acquiring a palm image and performing feature extraction; fuses the image features at multiple scales to obtain image fusion features; determines the palm frame corresponding to the palm in the palm image based on the image fusion features; crops the palm image corresponding to the palm in the palm image based on the palm frame; encrypts the palm image to obtain encrypted palm image data, and sends the encrypted palm image data to a palm recognition server for decryption and palm comparison recognition to perform identity recognition. This application saves palm image recognition time and improves palm recognition efficiency by cropping the palm image corresponding to the palm from the palm image and only encrypting, transmitting, and decrypting the palm image.

[0072] 2 shows a schematic diagram of the architecture of a computer system provided by an embodiment of the present application. The computer system may include: a palm recognition device 100 and a palm recognition server 200.

[0073] The palm recognition device 100 can be an electronic device such as a mobile phone, a tablet computer, a vehicle-mounted terminal (vehicle computer), a wearable device, a personal computer (PC), a voice interaction device with a palm image recognition function, a home appliance with a palm image recognition function, a vehicle-mounted terminal, an aircraft, an unmanned vending terminal, etc. The palm recognition device 100 can be installed with a client that runs an application. The application can be an application specifically for palm image recognition or other application that provides a palm image recognition function. This application is not limited to this. In addition, this application does not limit the form of the application, including but not limited to an application (Application, App) installed in the palm recognition device 100, a sub-application, a web application, etc. Among them, the sub-application is a program that runs in the operating environment provided by the parent application, and the parent application is an independent native application. The sub-application depends on the parent application to run.

[0074] The palm recognition server 200 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud computing services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial palm image recognition platforms. The palm recognition server 200 can be the backend server of the aforementioned application, providing backend services to the application's client.

[0075] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to enable data computing, storage, processing, and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology, all based on the cloud computing business model. It can form a resource pool that can be used on demand with flexibility and convenience. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as video websites, image websites, and more portals, require significant computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identification mark and will need to be transmitted to backend systems for logical processing. Data of varying levels will be processed separately, and data from all industries will require a strong system backend, which can only be achieved through cloud computing.

[0076] In some embodiments, the aforementioned servers can also be implemented as nodes in a blockchain system. Blockchain is a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of network transactions, used to verify the validity of the information (to prevent counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product and service layer, and an application service layer.

[0077] The palm recognition device 100 and the palm recognition server 200 may communicate with each other via a network, such as a wired or wireless network.

[0078] FIG3 is a flow chart of a palm image processing method provided by an exemplary embodiment of the present application. The method is executed by a palm recognition device. It is understood that the method can also be executed by a server. The method includes:

[0079] Step 302: Acquire a palm image, extract features from the palm image, and obtain image features at multiple scales.

[0080] The palm image is an image captured of a physical palm. A physical palm includes the palm and fingers, with the palm being the portion of the body connecting the physical arm and fingers. A palm image includes all or part of the physical palm. For ease of description, the image of the physical palm in the palm image will be referred to as the palm. The palm included in the palm image can be the physical palm of the user to be identified. The palm in the palm image includes at least the palm and may also include fingers or part of the arm. It may also include an image of the environment in which the physical palm was captured.

[0081] Scale refers to the size of the image features and can be expressed as a matrix dimension, such as 100*100, 80*80, 60*60, or 40*40. Multiple scales refer to two or more scales, each of which is different. The largest scale among the multiple scales can be the same as the scale of the palm image, but is generally smaller than the scale of the palm image.

[0082] The palm image can be obtained by the palm recognition device by capturing the physical palm of the user to be authenticated, or it can be sent by another device. The palm recognition device can directly capture the physical palm image to obtain the palm image. The palm recognition device can also capture the physical palm image using an image collector connected to the palm image to obtain the palm image. The palm recognition device can perform feature extraction on the acquired palm image to obtain image features corresponding to the palm image at multiple scales.

[0083] For example, the palm recognition device is a store cash register device, which uses a camera to capture the user's physical palm to obtain the palm image; or, the palm recognition device is a palm image recognition server, which captures the user's palm image and sends the palm image to the palm image recognition server.

[0084] For example, image features at multiple scales can be extracted from the multiple low-resolution images after downsampling the high-resolution palm image into multiple low-resolution images. High resolution and low resolution are relative terms, and high resolution is a higher resolution than low resolution.

[0085] For example, the size of the input palm image is 640*640, and the sizes of the output image features at multiple scales are: 80*80, 40*40, and 20*20.

[0086] Step 304: Fuse the image features at multiple scales to obtain image fusion features.

[0087] The palm recognition device can fuse image features at multiple scales to obtain image fusion features.

[0088] Multi-scale image features are features extracted from images at multiple scales. Image features at different scales can convey different information. Among multi-scale image features, features with larger scales convey more positional information, while features with smaller scales convey more semantic information.

[0089] Exemplarily, the palm recognition device extracts features of multiple scales, different receptive fields, and different categories, and fuses the image features of multiple scales to obtain image fusion features corresponding to the palm image.

[0090] Step 306: Determine a palm frame for identifying the palm in the palm image based on the image fusion features.

[0091] A palm frame is a marker used to indicate the location of the palm in a palm image. The palm frame encompasses the palm in the palm image. The palm frame's boundaries enclose the palm in the palm image.

[0092] Optionally, the palm frame may be a regular shape, such as at least one of a rectangle, square, rhombus, hexagon, circle, or triangle. The palm frame may also be an irregular shape. A regular shape is one that can be drawn using fixed parameters and fixed rules. Irregular shapes cannot be drawn using fixed parameters and fixed rules and can generally only be represented by a collection of points that form the irregular shape.

[0093] The palm recognition device can use artificial intelligence to make predictions based on image fusion features and determine the palm frame corresponding to the palm in the palm image.

[0094] In some embodiments, the computer device may determine the position of a palm frame identifying the palm in the palm image based on the image fusion features. When the palm frame is a regular shape, the position can be represented by the values ​​of fixed parameters required to draw the regular shape, such as the position of the palm frame's fixed points and the palm frame's size parameters. The fixed points are points fixed relative to the palm frame's position. The size parameters are parameters indicating the size of the palm frame. For example, when the palm frame is a rectangle, the fixed points can be any corner points or the center point of the rectangle, and the size parameters are the lengths of adjacent sides of the palm frame.

[0095] Step 308: Based on the palm frame, crop the palm image from the palm image.

[0096] The palm image is an effective recognition area in the palm image, or the palm image refers to the area in the palm image where the palm is located, or the palm image refers to the area in the palm image that can be used for palm recognition.

[0097] Exemplarily, the palm recognition device cuts out the image area outside the palm frame in the palm image based on the position of the palm frame, and uses the remaining image after cutting out as the palm image.

[0098] Step 310: Encrypt the palm image to obtain encrypted palm image data. The encrypted palm image data is transmitted to a server, and the server performs identity recognition based on the encrypted palm image data.

[0099] Encrypted palm image data is the encrypted palm image data. Decrypting the encrypted palm image data yields the palm image. The encrypted palm image data can be a bit stream, a string of characters, or an image whose content is difficult to discern with the naked eye. The palm image is then sent to a server, which performs identity verification based on the encrypted palm image data.

[0100] The palm recognition device can be connected to a network and transmit the encrypted palm image data to a server via the network. Alternatively, the palm recognition device can transmit the encrypted palm image data to a terminal via a point-to-point connection, which then transmits the encrypted palm image data to the server. If the palm recognition device itself is a server, the encrypted palm image data can be transmitted to another server via a direct cable or network.

[0101] In some embodiments, the server may decrypt the palm image encrypted data to obtain the palm image, and then perform identity recognition based on the decrypted palm image.

[0102] In some embodiments, the server may decrypt the encrypted palm image data to obtain a palm image, and search a database for a palm image that matches the decrypted palm image. When a matching palm image is found, the server determines the user ID associated with the matching palm image for identity verification. If no matching palm image is found, the server may determine that the identity verification has failed.

[0103] In some embodiments, the server may decrypt the palm image encrypted data to obtain the palm image, extract palm features from the palm image, and then search the database for palm features that match the extracted palm features. When matching palm features are found, the server determines the user identifier associated with the matching palm features for identity recognition.

[0104] In some embodiments, after the server determines the user ID based on the encrypted palm image data, it may further verify the permissions corresponding to the user ID. If the verified user ID has permissions for a preset function, the server may further execute or trigger the preset function. The preset function may include access control or other functions that require verification of user permissions.

[0105] Optionally, the palm recognition device sends the palm image encrypted data to the palm recognition server by at least one of network transmission, data line transmission, and Bluetooth transmission, but is not limited thereto. This application does not impose specific limitations on this.

[0106] The palm recognition device can decrypt the encrypted data based on the palm image and perform palm comparison and recognition for identity verification. Palm comparison and recognition can specifically compare the palm features in the palm image with pre-set palm features in the database.

[0107] The preset palm features are palm features of a stored user ID. Each preset palm feature has a corresponding user ID, indicating that the preset palm feature belongs to the user ID and is the palm feature of the user's palm. The user ID can be any user ID, such as a user ID registered in a payment application or a user ID registered in an enterprise.

[0108] In an embodiment of the present application, the palm recognition server includes a database containing a plurality of preset palm features and a user ID corresponding to each preset palm feature. In the database, the preset palm features and user IDs may correspond one-to-one, or one user ID may correspond to at least two preset palm features.

[0109] For example, multiple users register in a payment application. By binding the preset palm features of each user with the corresponding user identification, the palm features of multiple users and the corresponding user identification are stored in a database. When the user subsequently uses the payment application, the target user identification is determined by comparing the first palm area of ​​the infrared image, the second palm area of ​​the color image and the preset palm features in the database, thereby realizing the identity authentication of the user.

[0110] Exemplarily, the palm recognition device sends the encrypted palm image data to the palm recognition server for decryption and palm comparison recognition to obtain a user identifier corresponding to the palm image.

[0111] In summary, the method provided in this embodiment obtains image features at multiple scales corresponding to the palm image by acquiring a palm image and performing feature extraction; fuses the image features at multiple scales to obtain image fusion features; predicts based on the image fusion features to obtain a palm frame corresponding to the palm in the palm image; based on the palm frame, crops the palm image corresponding to the palm in the palm image; encrypts the palm image to obtain encrypted palm image data, and sends the encrypted palm image data to a palm recognition server for decryption and palm comparison recognition to obtain a user identifier corresponding to the palm image. This application saves palm image recognition time and improves palm recognition efficiency by cropping the palm image corresponding to the palm from the palm image and only encrypting, transmitting, and decrypting the palm image.

[0112] FIG4 is a flow chart of a palm image processing method provided by an exemplary embodiment of the present application. The method is executed by a palm recognition device. The method includes:

[0113] Step 402: Acquire a palm image, extract features from the palm image through the backbone network in the palm detection model, and obtain image features at multiple scales.

[0114] The palm image is a palm image of a user whose identity is to be determined.

[0115] For example, the palm recognition device captures the user's physical palm to obtain a palm image, wherein the palm image includes the user's left palm or the user's right palm.

[0116] For example, the palm recognition device is an IoT device that uses a camera to capture the user's left palm to obtain a palm image. The IoT device can also be a store cash register. For another example, when a user is shopping in a store, they extend their palm toward the camera of the store's payment terminal, which then captures the user's physical palm through the camera to obtain a palm image.

[0117] In some embodiments, the palm recognition device establishes a communication connection with another device and receives a palm image sent by the other device through the communication connection. For example, the palm recognition device may be a payment application server, and the other device may be a payment terminal. The payment terminal captures the user's physical palm, obtains a palm image, and then transmits the palm image to the payment application server through the communication connection between the payment terminal and the payment application server, so that the payment application server can determine the user identification of the palm image.

[0118] In some embodiments, the palm frame is predicted by a palm detection model, which includes a backbone network; features are extracted from the palm image to obtain image features at multiple scales, including: slicing the palm image through the backbone network to obtain sliced ​​images at multiple scales; and features are extracted from the sliced ​​images at multiple scales through the backbone network to obtain image features at multiple scales.

[0119] The palm detection model is an artificial intelligence model, which can be a deep learning model or a neural network model. The palm detection model at least has the function of determining a palm frame that identifies the palm in a palm image based on image fusion features. The backbone network is the structure in the palm detection model used to extract image features at multiple scales from the palm image. The slicing operation is the operation of dividing the palm image into smaller-scale images. Smaller-scale means that the scale of the generated image is smaller than that of the palm image. The pixels of the sliced ​​image are derived from the palm image.

[0120] The palm recognition device can perform slicing operations through the backbone network to obtain slice images at multiple scales. The palm recognition device can also extract features from the slice images at multiple scales to obtain image features at multiple scales corresponding to the palm image.

[0121] In some embodiments, a palm image is sliced ​​through a backbone network to obtain slice images of multiple scales, including: determining a slice image of the largest scale among multiple scales based on the palm image through the backbone network; using the slice image of the largest scale as the first layer through the backbone network, and starting downsampling layer by layer to obtain slice images of multiple scales including the slice image of the first layer.

[0122] The computer device may first determine a slice image of the largest scale among multiple scales based on the palm image. The scale of the image feature at the largest scale may be smaller than or equal to the scale of the palm image. The computer device may perform a downsampling on the palm image to generate a slice image of the largest scale among the multiple scales.

[0123] Downsampling the palm image is a process of sampling at least one pixel in the palm image, and splicing the downsampled pixels in the order of the palm image to obtain a slice image with a smaller scale.

[0124] If the number of scales in the multiple scales is N, then the multiple scales are N scales. Each of the N scales is a different scale, so there is a maximum scale among the N scales. The slice image with the maximum scale is taken as the first layer. Starting from the first layer, downsampling is performed layer by layer. Layer n is downsampled to obtain the slice image of layer n+1, and so on until layer N-1 is downsampled to obtain the slice image of layer N. n is a positive integer from 1 to N-1. Finally, N slice images are obtained, from layers 1 to N.

[0125] In this embodiment, multi-scale slice images are obtained by downsampling layer by layer, which provides a new approach to segmenting multi-scale slice images from a palm image.

[0126] In some embodiments, slicing a palm image using a backbone network to obtain sliced ​​images at multiple scales includes: sampling and splicing pixels in the palm image at multiple scales using the backbone network to obtain sliced ​​images at multiple scales. Two adjacent pixels in sliced ​​images of different scales have different numbers of pixels separating them at the sampling locations in the palm image.

[0127] In this embodiment, the palm recognition device can sample pixels from the palm image at each scale using the sampling interval at that scale, and then splice the sampled pixels according to their relative positions in the palm image, thereby generating a slice image at each scale. The sampling interval is the number of pixels spaced, including the spacing between pixels in both row and column directions. The scale and sampling interval are negatively correlated: the larger the scale, the smaller the sampling interval. The smaller the scale, the larger the sampling interval. Pixels not sampled in the palm image can be focused on the image feature channel.

[0128] For example, at the maximum scale among multiple scales, every other column in the palm image is sampled, that is, every other pixel in the row direction is selected, and these selected pixels are spliced ​​together to obtain a slice image. The information in the spliced ​​slice image is not lost, but the size information of the palm image is concentrated in the channel. The resulting new image is then subjected to a convolution operation to obtain a downsampled feature map without information loss.

[0129] For example, at the second largest scale among multiple scales, in a palm image, every two columns are sampled, that is, every two pixels are selected, and these selected pixels are spliced ​​together to obtain a slice image. Information in the spliced ​​slice image is not lost, but the size information in the palm image is concentrated in the channel. The resulting new image is then convolved to obtain a downsampled feature map without information loss.

[0130] For example, the original image size is 640*640*3, and through slicing, a feature map of 320*320*12 is obtained. Among them, 640*640 in the original image is used to represent width*height, and 3 in the original image is used to represent the length of the feature vector corresponding to each pixel (also known as the number of channels).

[0131] Step 404: The image features at multiple scales are fused through the neck network of the palm detection model to obtain image fusion features.

[0132] The palm frame is predicted by a palm detection model, which includes a neck network. Image features at multiple scales are fused to obtain image fusion features, including: fusing image features at multiple scales through the neck network to obtain image fusion features.

[0133] The neck network, part of the palm detection model, is used to fuse image features at multiple scales to generate fused image features. The neck network is connected to the backbone network and receives the multi-scale image features output by the backbone network.

[0134] Multi-scale image features are features extracted from images at multiple scales. Image features at different scales can convey different information. Among multi-scale image features, features with larger scales convey more positional information, while features with smaller scales convey more semantic information.

[0135] Exemplarily, the palm recognition device extracts features of multiple scales, different receptive fields, and different categories, and fuses the image features of multiple scales to obtain image fusion features corresponding to the palm image.

[0136] In some embodiments, fusing image features at multiple scales through a neck network to obtain a fused image feature includes fusing image features at multiple scales at the same scale through the neck network to obtain the fused image feature. The fusion of image features at the same scale may be summing or averaging.

[0137] The palm recognition device fuses image features at multiple scales, increasing the detection accuracy of small targets, thereby more accurately detecting the palm frame and cropping the palm image, further reducing the amount of palm image encryption data, improving identity recognition efficiency, and ensuring identity recognition security.

[0138] In some embodiments, image features at multiple scales are fused through a neck network to obtain image fusion features, including: performing feature fusion based on image features at multiple scales through a neck network to obtain first intermediate features at multiple scales; performing feature fusion based on the first intermediate features at multiple scales through a neck network to obtain second intermediate features at multiple scales; and performing feature fusion on the second intermediate features at multiple scales through a neck network to obtain image fusion features.

[0139] Among them, feature fusion at multiple scales refers to fusing features at different scales. First intermediate features at multiple scales can correspond one-to-one with image features at multiple scales. First intermediate features at multiple scales can correspond one-to-one with second intermediate features at multiple scales. The multiple scales of image features can be consistent with the multiple scales of first intermediate features. The multiple scales of first intermediate features can be consistent with the multiple scales of second intermediate features. Fusion of second intermediate features at multiple scales can be performed by unifying the second intermediate features at the same scale and fusing them. This fusion can be performed by summing or averaging.

[0140] In this embodiment, features at different scales are fused and finally fused into image fusion features, which can fully express the detailed features in the palm image, thereby facilitating more accurate detection of the palm frame, thereby more accurately cropping the palm image, further reducing the amount of palm image encryption data, improving identity recognition efficiency, and ensuring identity recognition security.

[0141] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1. Extracting first intermediate features at multiple scales based on image features at the multiple scales includes: determining the first intermediate feature at the first position in the positive order of the scale based on the image feature at the first position in the positive order of the scale; upsampling layer by layer starting from the first intermediate feature at the first position in the positive order of the scale, fusing the upsampling result of the first intermediate feature at the nth position in the positive order of the scale with the image feature at the n+1th position in the positive order of the scale to obtain the first intermediate feature at the n+1th position in the positive order of the scale; and once the first intermediate feature at the Nth layer is obtained, determining first intermediate features at multiple scales from the 1st layer to the Nth layer. Where n is a positive integer from 1 to N-1; and the positive order of the scale is an ascending scale order.

[0142] The order from smallest to largest is called positive order. Positive order by scale is the process of scaling up. Upsampling is the process of scaling up. Upsampling can be performed to the scale of the next layer, interpolating adjacent feature elements in each layer to obtain the features of the next layer.

[0143] In some embodiments, determining the first intermediate feature at the first position in the scaled positive order based on the first image feature at the first position in the scaled positive order includes performing at least one convolution on the first image feature at the first position in the scaled positive order to obtain the first intermediate feature at the first position in the scaled positive order. In addition to performing the convolution, channel adjustment may also be performed. After the channel adjustment, the number of channels of the first intermediate feature may be consistent.

[0144] In some embodiments, the scale of the upsampling result of the first intermediate feature at the nth position in the positive scale order can be the same as the scale of the image feature at the n+1th position in the positive scale order. The upsampling result of the first intermediate feature at the nth position in the positive scale order and the image feature at the n+1th position in the positive scale order can be fused by concatenating the two, or summing or averaging them at the same scale.

[0145] For example, a schematic diagram of image feature fusion is shown in Figure 5. The palm recognition device performs feature extraction on a palm image 501 to obtain image features 502 at multiple scales corresponding to the palm image 501. Image features 502 include feature layer T1, feature layer T2, feature layer T3, and feature layer T4. After obtaining image features 502, feature layer T4 is convolved once and the channels are adjusted to obtain feature layer P4. Feature layer P4 is upsampled and combined with feature layer T3, and then convolved and the channels are adjusted to obtain feature layer P3. Feature layer P3 is upsampled and combined with feature layer T2, and then convolved and the channels are adjusted to obtain feature layer P2. Feature layer P2 is upsampled and combined with feature layer T1, and then 1*1 convolution is performed to adjust the channels to obtain feature layer P1.

[0146] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the features of the multiple scales are fused based on the first intermediate features of the multiple scales to obtain the second intermediate features of the multiple scales, including: determining the second intermediate feature of the first position in the positive order of the scale based on the first intermediate feature at the first position in the reverse order of the scale; starting from the second intermediate feature at the first position in the reverse order of the scale, downsampling layer by layer, fusing the result of downsampling the second intermediate feature at the mth position in the reverse order of the scale with the first intermediate feature at the m+1th position in the reverse order of the scale to obtain the second intermediate feature at the n+1th position in the reverse order of the scale; m is an integer from 1 to N-1; the reverse order of the scale is the order from large to small; when the first intermediate feature of the Nth layer is obtained, determining the first intermediate features of multiple scales from the 1st layer to the Nth layer.

[0147] The order from largest to smallest is called reverse order, and reverse order by scale refers to the order from largest to smallest. Downsampling is the process of reducing the scale. Downsampling can target the scale of the next layer by taking intervals from the feature elements of each layer to obtain the features of the next layer. The first intermediate feature can be called a semantic image feature. The second intermediate feature can be called a positional image feature. Semantic image features reflect more semantic information, while semantic image features reflect more positional information.

[0148] In some embodiments, determining the first-ranked second intermediate feature in reverse order of scale based on the first intermediate feature in reverse order of scale includes performing at least one convolution on the first intermediate feature in reverse order of scale to obtain the first-ranked second intermediate feature in reverse order of scale. In addition to performing the convolution, channel adjustment may also be performed. After the channel adjustment, the number of channels of the first intermediate feature may be consistent.

[0149] The scale of the down-sampled result of the second intermediate feature at the mth position in reverse scale order can be consistent with the scale of the first intermediate feature at the m+1th position in reverse scale order. The down-sampled result of the second intermediate feature at the mth position in reverse scale order and the first intermediate feature at the m+1th position in reverse scale order can be fused by concatenating the two, or summing or averaging the two at the same scale.

[0150] For example, the palm recognition device may perform one convolution on feature layer P1 and adjust the channels to obtain feature layer N1. Feature layer N1 is downsampled and combined with feature layer P2, and then convolved and adjusted for channels to obtain feature layer N2. Feature layer N2 is downsampled and combined with feature layer P3, and then convolved and adjusted for channels to obtain feature layer N3. Feature layer N3 is downsampled and combined with feature layer P4, and then convolved and adjusted for channels to obtain feature layer N4. The palm recognition device performs feature fusion on feature layers N1-4 to obtain image fusion features 503.

[0151] Optionally, the activation function in the convolution is a logistic activation function.

[0152] In some embodiments, the palm recognition device upsamples the image features of multiple scales from a small scale to a large scale to obtain a first intermediate feature; the palm recognition device downsamples the image features of multiple scales from a large scale to a small scale to obtain a second intermediate feature; the palm recognition device fuses the first intermediate feature and the second intermediate feature to obtain an image fusion feature.

[0153] Step 406: Determine a palm frame that identifies the palm in the palm image based on the image fusion features through the prediction network of the palm detection model.

[0154] The palm frame is a marker used to indicate the location of the palm in a palm image.

[0155] In some embodiments, the palm frame is determined by a palm detection model, which includes a prediction network; based on the image fusion features, a palm frame that identifies the palm in the palm image is determined, including: gridding the image fusion features to obtain multiple grid features of the image fusion features; inputting the multiple grid features into the prediction network, predicting the multiple grid features, and obtaining the palm frame.

[0156] The grid features are gridded image fusion features obtained by dividing the image fusion features into grids. For example, the palm recognition device divides the image fusion features into grids to obtain multiple grid features. The palm recognition device then uses a prediction network to perform a prediction based on each grid feature, obtaining a prediction result corresponding to each grid feature. The prediction results corresponding to the multiple grid features can then be combined to determine the position of the palm frame in the palm image.

[0157] In some embodiments, the palm recognition device is equipped with an infrared camera, and the palm image processing method further includes: obtaining an infrared image captured of the same palm as the palm image, where the infrared image is an image obtained by the infrared camera imaging the palm using infrared light; identifying the palm region from the infrared image; and cropping the palm image from the palm image based on the palm frame, including: determining the intersection of the palm frame in the palm image and the palm region in the infrared image; and cropping the palm image from the palm image based on the intersection.

[0158] In this embodiment, the palm recognition device is further provided with an infrared camera. The palm recognition device obtains an infrared image corresponding to the same physical palm and performs region recognition on the infrared image to determine the palm region in the infrared image. The infrared image is an image obtained by the infrared camera using infrared light to image the palm.

[0159] In some embodiments, identifying a palm area from an infrared image includes: detecting finger gaps in the infrared image; and determining the palm area in the infrared image based on the finger gaps.

[0160] Exemplarily, the palm recognition device detects finger gaps in the infrared image and determines a palm area on the infrared image based on the finger gaps.

[0161] As shown in FIG6 , the schematic diagram of the finger seam points in the palm of the hand, the finger seam point is the first finger seam point 601 between the index finger and the middle finger, or the finger seam point is the second finger seam point 602 between the middle finger and the ring finger, or the finger seam point is the third finger seam point 603 between the ring finger and the little finger.

[0162] Since the palm area in a palm image may exist in any area of ​​the palm image, in order to determine the position of the palm area in the palm image, finger seam point detection is performed on the palm image to obtain at least one finger seam point of the palm image, so that the palm area can be subsequently determined based on the at least one finger seam point.

[0163] Step 408: Based on the palm frame, crop the palm image from the palm image.

[0164] The palm image is an effective recognition area in the palm image, or the palm image refers to the area in the palm image where the palm is located, or the palm image refers to the area in the palm image that can be used for palm recognition.

[0165] In some embodiments, the palm recognition device obtains at least three finger seam points in the infrared image; sequentially connects the finger seam points to obtain a finger seam point connection line; and the palm recognition device crops the palm area in the infrared image based on the finger seam point connection line.

[0166] In some embodiments, determining the intersection of a palm frame in a palm image and a palm area in an infrared image includes: obtaining coordinate parameters of the palm frame in the palm image and obtaining coordinate parameters of the palm area in the infrared image; and determining the intersection of the palm frame and the palm area based on the coordinate parameters of the palm frame and the coordinate parameters of the palm area.

[0167] The palm recognition device obtains the coordinate parameters of the palm frame in the palm image and the coordinate parameters of the palm area in the infrared image; the palm recognition device crops the palm image corresponding to the palm based on the intersection of the coordinate parameters of the palm frame and the coordinate parameters of the palm area.

[0168] Step 410: Encrypt the palm image to obtain encrypted palm image data. The encrypted palm image data is transmitted to a server, and the server performs identity recognition based on the encrypted palm image data.

[0169] The encrypted palm image data is sent to a palm recognition server for decryption and palm comparison, which yields the user ID corresponding to the palm image. Palm comparison involves comparing the palm features in the palm image with pre-set palm features in a database.

[0170] It's important to note that the palm, as a biometric, is biologically unique and distinguishable. Compared to facial recognition, which is currently widely used in areas like identity verification, payment, access control, and ride-hailing, the palm is unaffected by makeup, masks, sunglasses, and other factors, improving the accuracy of user authentication. In certain scenarios, such as epidemic prevention and control, where masks covering the mouth and nose are required, palm images can be a better option for authentication.

[0171] Cross-device registration and identification is a capability that is crucial to user experience. For two associated device types, a user can register on one type of device, bind their user ID to their palm features, and then authenticate themselves on the other type of device. Because mobile phones and IoT devices differ significantly in image style and quality, cross-device registration and identification allows users to register on their mobile phone and use the IoT device directly, eliminating the need for users to register on both types of devices. For example, after registering on their mobile phone, users can authenticate themselves directly on a store's device, eliminating the need for users to register on the store's device, thus preventing user information leakage.

[0172] In summary, the method provided in this embodiment obtains image features at multiple scales corresponding to the palm image by acquiring a palm image and performing feature extraction; fuses the image features at multiple scales to obtain image fusion features; predicts based on the image fusion features to obtain a palm frame corresponding to the palm in the palm image; based on the palm frame, crops the palm image corresponding to the palm in the palm image; encrypts the palm image to obtain encrypted palm image data, and sends the encrypted palm image data to a palm recognition server for decryption and palm comparison recognition to obtain a user identifier corresponding to the palm image. This application saves palm image recognition time and improves palm recognition efficiency by cropping the palm image corresponding to the palm from the palm image and only encrypting, transmitting, and decrypting the palm image.

[0173] The method provided in this embodiment slices the palm image and performs feature extraction through a backbone network to obtain image features at multiple scales. Based on the above method, features of multiple scales, different receptive fields, and different categories are obtained, thereby improving the accuracy of palm frame prediction.

[0174] The method provided in this embodiment uses a neck network to upsample image features of multiple scales from a small scale to a large scale to obtain a first intermediate feature; downsample image features of multiple scales from a large scale to a small scale to obtain a second intermediate feature; and fuse the first intermediate feature and the second intermediate feature to obtain an image fusion feature. Based on the above method, image features of multiple scales can be fused, thereby improving the detection accuracy of small targets in palm images.

[0175] The method provided in this embodiment obtains gridded image fusion features by dividing the image fusion features into grids; and obtains a palm frame based on the prediction of the gridded image fusion features through a prediction network. Based on the above method, the accuracy of palm frame prediction is improved, and the security of palm-based identity recognition is further guaranteed.

[0176] The method provided in this embodiment obtains an infrared image corresponding to the same palm, performs region recognition on the infrared image, determines the palm area on the infrared image, and crops the palm image corresponding to the palm based on the intersection of the palm frame and the palm area on the infrared image. Based on the above method, the palm image corresponding to the palm is obtained through the palm area and the palm frame on the infrared image, which improves the cropping accuracy of the palm image and makes the obtained palm image more accurate.

[0177] FIG7 is a schematic diagram of a palm image processing method provided by an exemplary embodiment of the present application. The method is executed by a palm recognition device. The method includes:

[0178] The palm recognition device obtains a palm image 701 corresponding to the palm, and inputs the obtained palm image 701 into the backbone network 702 for feature extraction to obtain image features of multiple scales corresponding to the palm image 701.

[0179] The palm recognition device fuses the image features at multiple scales through the neck network 703 to obtain image fusion features.

[0180] The palm recognition device performs prediction based on the image fusion features through the prediction network 704 to obtain a palm frame corresponding to the palm in the palm image 701. Based on the palm frame, the palm recognition device crops the palm image 701 to obtain a palm image 705 corresponding to the palm.

[0181] The palm recognition device inputs the palm image 705 into the encryption network 706 for image encryption to obtain palm image encrypted data 707 .

[0182] The palm recognition device transmits the palm image encrypted data 707 through the network 708 to the decryption network 709 in the palm recognition server for image decryption and performs palm comparison and recognition through the verification network 710 to obtain the user identification 711 corresponding to the palm image 701.

[0183] In summary, the method provided in this embodiment obtains the palm image corresponding to the palm from the palm image by cropping it, and only encrypts, transmits and decrypts the palm image, thereby saving the palm image recognition time and improving the recognition efficiency of palm recognition.

[0184] FIG8 is a schematic diagram of a cross-device payment based on a palm image processing method provided by an exemplary embodiment of the present application. The method involves a user terminal 801 , a merchant terminal 803 , and a payment application server 802 .

[0185] Among them, the user terminal 801 is installed with a payment application, and the user terminal 801 logs in to the payment application based on the user identification and establishes a communication connection with the payment application server 802. Through this communication connection, the user terminal 801 and the payment application server 802 can interact; the merchant terminal 803 is installed with a payment application, and the merchant terminal 803 logs in to the payment application based on the merchant identification and establishes a communication connection with the payment application server 802. Through this communication connection, the merchant terminal 803 and the payment application server 802 can interact.

[0186] The cross-device payment process includes:

[0187] 1. The user holds the user terminal 801 at home and uses the user terminal 801 to capture the user's physical palm to obtain the user's palm image. The user logs into the payment application based on the user ID and sends a palm image registration request to the payment application server 802. The palm image registration request carries the user ID and palm image.

[0188] 2. The payment application server 802 receives the palm image registration request sent by the user terminal 801, performs feature extraction on the palm image, obtains the palm features of the palm image, stores the palm features in correspondence with the user ID, and sends a palm image binding success notification to the user terminal 801.

[0189] Among them, after the payment application server 802 stores the palm feature and the user identification in correspondence, it uses the palm feature as a preset palm feature, and can subsequently determine the corresponding user identification through the stored preset palm feature.

[0190] 3. The user terminal 801 receives the palm image binding success notification, displays the palm image binding success notification, and prompts the user to bind the palm image with the user ID.

[0191] The user completes palm image registration through interaction between his / her user terminal 801 and the payment application server 802, and can subsequently realize automatic payment through the palm image.

[0192] 4. When a user purchases goods in a store, the merchant terminal 803 captures the user's physical palm to obtain a palm image. Based on the payment application logged in with the merchant ID, the merchant terminal sends a payment request to the payment application server 802. The payment request carries the merchant ID, the consumption amount, and the palm image.

[0193] 5. After receiving the payment request, the payment application server 802 performs palm comparison and recognition on the palm image, determines the user ID of the palm image, determines the account number of the user ID in the payment application, completes the transfer through the account number, and sends a payment completion notification to the merchant terminal 803 after the transfer is completed.

[0194] Among them, after the user uses the user terminal 801 to register the palm image, he can directly pay through the palm at the merchant terminal 803 without the user having to register the palm image on the merchant terminal 803, thereby achieving the effect of cross-device palm image recognition and improving convenience.

[0195] 6. The merchant terminal 803 receives the payment completion notification, displays the payment completion notification, and prompts the user that the payment is completed, so that the user and the merchant complete the transaction of the item, and the user can take the item away.

[0196] In addition, the above embodiment uses the user terminal 801 and the merchant terminal 803 to realize the cross-device payment process. The above merchant terminal 803 can also be replaced with a payment device on the bus. According to the above steps, a cross-device bus payment solution can be realized.

[0197] FIG9 is a schematic diagram of a cross-device identity authentication method based on a palm image processing method provided by an exemplary embodiment of the present application. The method involves a user terminal 901, an access control device 903, and an access control server 902.

[0198] Among them, the user terminal 901 establishes a communication connection with the access control server 902, through which the user terminal 901 and the access control server 902 can interact; the access control device 903 establishes a communication connection with the access control server 902, through which the access control device 903 and the access control server 902 can interact.

[0199] The cross-device authentication process includes:

[0200] 1. The user holds the user terminal 901 at home and uses the user terminal 901 to capture the user's physical palm to obtain the user's palm image. The user then sends a palm registration request to the access control server 902. The palm registration request carries the user ID and the palm image.

[0201] 2. The access control server 902 receives the palm registration request sent by the user terminal 901, extracts features from the palm image, obtains the palm features of the palm image, stores the palm features in correspondence with the user ID, and sends a palm binding success notification to the user terminal 901.

[0202] Among them, after the access control server 902 stores the palm feature and the user identification in correspondence, the palm feature can be used as a preset palm feature, and the corresponding user identification can be determined later through the stored preset palm feature.

[0203] 3. The user terminal 901 receives the palm binding success notification, displays the palm binding success notification, and prompts the user to bind the palm image with the user ID.

[0204] The user completes palm image registration through the interaction between his / her user terminal 901 and the access control server, and the door can be opened automatically later by the palm image.

[0205] 4. When the user goes out and comes home, the access control device 903 takes a picture of the user's physical palm, obtains the user's palm image, and sends an identity authentication request to the access control server 902. The identity authentication request carries the verification palm image.

[0206] 5. The access control server 902 receives the identity authentication request sent by the access control device 903, identifies the verification palm image, obtains the user ID of the palm image, determines that the user is a registered user, and sends a verification success notification to the access control device 903.

[0207] 6. The access control device 903 receives the verification pass notification sent by the access control server 902, and controls the door to open according to the verification pass notification, so that the user can enter the room.

[0208] The above embodiment is a process of implementing cross-device identity authentication through the user terminal 901 and the access control device 903.

[0209] As can be seen from the above cross-device authentication scenario, whether in the palm registration phase between the user terminal 901 and the access control server 902, or in the palm image processing phase between the user terminal 901 and the server via other terminal devices, after obtaining the palm image, the user terminal 901 or other terminal device crops and encrypts the palm area in the palm image, and then sends the palm image of the palm area to the server, which performs palm comparison and recognition. In the palm comparison and recognition phase, the access control server 902 compares the palm features with the preset palm features to obtain the current user's identification result.

[0210] FIG10 is a schematic diagram of a palm image processing method provided by an exemplary embodiment of the present application. The method is executed by a palm recognition device. The method includes:

[0211] The palm recognition device obtains a palm image 1001 corresponding to the palm and inputs it into the focusing layer 1002 in the backbone network. This layer slices the palm image 1001 to obtain image features at multiple scales. These features are then fed into the convolution layer 1003 for convolution channel adjustment and then into the bottleneck layer 1004 for feature extraction. The pooling layer 1005 pools the image features.

[0212] For example, the palm image 1001 is input to the focusing layer 1002, and the focusing layer 1002 slices the palm image 1001 to obtain image features of multiple scales represented as (3, 64, 1, 1). After convolution and channel adjustment by the convolution layer 1003, the image features are represented as (64, 128, 3, 2). After passing through the bottleneck layer 1004, the image features are represented as (128, 128)*3. After convolution and channel adjustment by the convolution layer 1003, the image features are represented as (128, 256, 3, 2). After passing through the bottleneck layer 1004, the image features are represented as (256, 256)*3. After the image feature (256, 256)*3 undergoes convolution layer 1003 and adjusts its channel, it is represented as (256, 512, 3, 2). After passing through bottleneck layer 1004, it is represented as (512, 512)*9. After passing through convolution layer 1003 again and adjusting its channel, it is represented as (512, 1024, 3, 2). After pooling in pooling layer 1005 and passing through bottleneck layer 1004, it is represented as (1024, 1024)*3. After upsampling 1006 and concatenating 1007 with the image feature (512, 512)*9, the image feature (1024, 1024)*3 after passing through bottleneck layer 1004 is represented as (1024, 512)*3. After the image feature (1024, 512)*3 undergoes convolution at convolution layer 1003 and adjusts its channel, the image feature is represented as (512, 256, 1, 1). After the image feature (512, 256, 1, 1) is upsampled 1006 and concatenated 1007 with the image feature (256, 512, 3, 2), the image feature after passing through bottleneck layer 1004 is represented as (512, 256)*3. After the image feature (512, 256)*3 undergoes convolution at convolution layer 1003 and adjusts its channel, the image feature is represented as (256, 256, 3, 2). After the image feature (256, 256, 3, 2) is concatenated 1007 with the image feature (512, 256, 1, 1), the image feature after passing through bottleneck layer 1004 is represented as (512, 512)*3. After the image feature (512, 512)*3 undergoes convolutional channel adjustment at convolution layer 1003, the image feature representation is (512, 512, 3, 2). After the image feature representation (512, 512, 3, 2) is concatenated 1007 with the image feature representation (1024, 512, 1, 1), the image feature representation after passing through bottleneck layer 1004 is (1024, 1024)*3. The final output image features at multiple scales 1008 are: (512, 256), (512, 512), and (1024, 1024). (x, y, a, b) represents the two dimensions of the feature matrix, a represents the number of channels, and b represents the scale.(x, y)*b, x, y can represent the two dimensions of the feature matrix respectively, and b can represent the number of channels. (x, y), x, y can represent the two dimensions of the feature matrix respectively.

[0213] The palm recognition device fuses the image features 1008 at multiple scales to obtain a fused image feature. The palm recognition device predicts the fused image feature to obtain a palm frame corresponding to the palm in palm image 1001. Based on the palm frame, the palm recognition device crops the palm image 1001 to obtain a palm image corresponding to the palm. The palm recognition device encrypts the palm image and transmits it via a network to a palm recognition server for image decryption and palm comparison recognition, thereby obtaining a user ID corresponding to palm image 1001.

[0214] In summary, the method provided in this embodiment slices the palm image and performs feature extraction through the backbone network to obtain image features at multiple scales. Based on the above method, features of multiple scales, different receptive fields, and different categories are obtained, thereby improving the accuracy of palm frame prediction.

[0215] The prediction of the palm frame involved in this application can be implemented based on the palm detection model. The scheme includes a palm detection model generation stage and a palm frame prediction stage. Figure 11 is a framework diagram of a palm detection model generation stage and a palm frame prediction stage shown in an exemplary embodiment of the present application. As shown in Figure 11, in the palm detection model generation stage, the palm detection model generation device 1110 obtains the palm detection model through a pre-set training sample data set (including sample palm images and sample palm images), and then generates a palm frame based on the palm detection model. In the palm frame prediction stage, the palm frame prediction processing device 1120 processes the input palm image based on the palm detection model to obtain a prediction result of the palm image, such as determining the area and position of the palm in the palm image, etc.

[0216] Among them, the above-mentioned palm detection model generation device 1110 and palm frame prediction processing device 1120 can be computer devices. For example, the computer device can be a fixed computer device such as a personal computer or a server, or the computer device can also be a mobile computer device such as a tablet computer or an e-book reader.

[0217] Optionally, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 can be the same device, or they can be different devices. Furthermore, when the palm detection model generation device 1110 and the palm frame prediction processing device 1120 are different devices, they can be the same type of device, such as both being servers. Alternatively, they can be different types of devices, such as a personal computer or terminal, while the palm detection model generation device 1110 can be a server. This embodiment of the present application does not limit the specific types of the palm detection model generation device 1110 and the palm frame prediction processing device 1120.

[0218] The above embodiment describes the palm image processing method. Next, the training method of the palm detection model will be described.

[0219] FIG12 is a flow chart of a palm detection model training method provided by an exemplary embodiment of the present application. The method is executed by a palm recognition device. The method includes:

[0220] Step 1201: Acquire a sample palm image and a sample palm image.

[0221] The sample palm image is an image of the user whose identity is to be determined. The sample palm image contains a palm, which is an image of the physical palm of the user whose identity is to be verified. The sample palm image may also contain other information, such as the user's fingers and the scene in which the user's palm was captured. The sample palm image may be obtained by the palm recognition device by capturing the physical palm of the user whose identity is to be verified, or it may be sent by another device.

[0222] For example, the palm recognition device is a store cash register device, which uses a camera to capture the user's physical palm to obtain the sample palm image; or, the palm recognition device is a palm image recognition server, which captures the user's sample palm image and sends the sample palm image to the palm image recognition server.

[0223] The sample palm image refers to the effective identification area marked in the sample palm image, or the sample palm image refers to the area where the palm is located marked in the sample palm image, or the sample palm image refers to the area marked in the sample palm image that can be used for identification.

[0224] Step 1202: Extract features from the sample palm image to obtain sample image features at multiple scales.

[0225] The sample palm image is an image obtained by capturing a physical palm. The sample palm image includes all or part of the physical palm. For ease of description, the image of the physical palm in the sample palm image will be referred to as the palm. The palm included in the sample palm image can be the physical palm of the user to be identified. The palm in the sample palm image includes at least the palm, and may also include fingers or part of the arm. It may also include an image of the environment in which the physical palm was captured.

[0226] Scale is the size of the sample image features and can be expressed as a matrix dimension, such as 100*100, 80*80, 60*60, or 40*40. Multiple scales refer to two or more scales, each of which is different. The largest scale among the multiple scales can be the same as the scale of the sample palm image, but is generally smaller than the scale of the sample palm image.

[0227] The sample palm image can be obtained by the palm recognition device by photographing the physical palm of the user whose identity is to be verified, or it can be sent by another device. The palm recognition device can directly capture the image of the physical palm to obtain the sample palm image. The palm recognition device can also capture the image of the physical palm through an image collector connected to the sample palm image to obtain the sample palm image. The palm recognition device can perform feature extraction on the obtained sample palm image to obtain sample image features at multiple scales corresponding to the sample palm image.

[0228] For example, the palm recognition device is a store cash register device, which uses a camera to capture the user's physical palm to obtain the sample palm image; or, the palm recognition device is a sample palm image recognition server, which captures the user's sample palm image and sends the sample palm image to the sample palm image recognition server.

[0229] For example, features of sample images at multiple scales can be extracted from the low-resolution images at multiple scales after downsampling a high-resolution sample palm image into low-resolution images at multiple scales. High resolution and low resolution are relative terms, and high resolution is a higher resolution than low resolution.

[0230] For example, the size of the input sample palm image is 640*640, and the sizes of the output sample image features at multiple scales are: 80*80, 40*40, and 20*20.

[0231] Step 1203: Fusing sample image features at multiple scales to obtain sample image fusion features.

[0232] The palm recognition device can fuse sample image features at multiple scales to obtain sample image fusion features.

[0233] Multi-scale sample image features refer to features extracted from images at multiple scales. Sample image features at different scales can convey different information. Among sample image features at multiple scales, features at larger scales convey more positional information, while features at smaller scales convey more semantic information.

[0234] Exemplarily, the palm recognition device extracts features of multiple scales, different receptive fields, and different categories, and fuses sample image features of multiple scales to obtain sample image fusion features corresponding to the sample palm image.

[0235] Step 1204: Determine a sample predicted palm frame that identifies the palm in the sample palm image based on the sample image fusion features.

[0236] The sample predicted palm frame is a marker used to indicate the location of the palm in the sample palm image. The sample predicted palm frame contains the palm in the sample palm image. The boundary of the sample predicted palm frame encloses the palm in the sample palm image.

[0237] Optionally, the shape of the sample predicted palm frame can be a regular shape, such as at least one of a rectangle, square, rhombus, hexagon, circle, or triangle. The shape of the sample predicted palm frame can also be an irregular shape. A regular shape is a shape that can be drawn using fixed parameters and fixed rules. Irregular shapes cannot be drawn using fixed parameters and fixed rules and can generally only be represented by a collection of points that form the irregular shape.

[0238] The palm recognition device can use artificial intelligence to make predictions based on the fusion features of the sample image to determine a sample predicted palm frame that identifies the palm in the sample palm image.

[0239] In some embodiments, the computer device may determine the position of the sample predicted palm frame that identifies the palm in the sample palm image based on the sample image fusion features. When the sample predicted palm frame is a regular shape, the position can be represented by the value of the fixed parameter required to draw the regular shape, such as the position of the fixed point of the sample predicted palm frame and the size parameter of the sample predicted palm frame. The fixed point is a point fixed relative to the position of the sample predicted palm frame. The size parameter is a parameter that represents the size of the sample predicted palm frame. For example, when the sample predicted palm frame is a rectangle, the fixed point can be any corner point or the center point of the rectangle, and the size parameter is the length of the adjacent sides of the sample predicted palm frame.

[0240] Step 1205: Based on the sample predicted palm frame, a sample predicted palm image is cropped from the sample palm image.

[0241] The sample predicted palm image is an effective recognition area in the sample palm image, or the sample predicted palm image refers to the area where the palm is located in the sample palm image, or the sample predicted palm image refers to the area in the sample palm image that can be used for palm recognition.

[0242] Exemplarily, the palm recognition device crops the image area outside the sample predicted palm frame in the sample palm image based on the position of the sample predicted palm frame, and uses the remaining image after cropping as the sample predicted palm image.

[0243] Step 1206: Calculate a loss function value based on the sample palm image and the sample predicted palm image.

[0244] Exemplarily, the palm recognition device calculates a loss function value based on the sample palm image and the sample predicted palm image.

[0245] Step 1207: Update the model parameters of the palm detection model based on the loss function value.

[0246] Exemplarily, the palm recognition device updates the model parameters of the palm detection model based on the loss function value.

[0247] Model parameter updating refers to updating the network parameters in the palm detection model, or updating the network parameters of each network module in the model, or updating the network parameters of each network layer in the model, but is not limited to this, and the embodiments of the present application do not limit this.

[0248] The model parameters of the palm detection model include at least one of the network parameters of the backbone network, the network parameters of the neck network, and the network parameters of the prediction network in the palm detection model.

[0249] In summary, the method provided in this embodiment obtains sample image features of multiple scales corresponding to the sample palm image by performing feature extraction on the acquired sample palm image; performs feature fusion on the sample image features of multiple scales to obtain sample image fusion features; performs prediction based on the sample image fusion features to obtain a sample predicted palm frame that identifies the palm in the sample palm image; based on the sample predicted palm frame, a sample predicted palm image corresponding to the palm is cropped in the sample palm image; based on the sample predicted palm image and the sample predicted palm image, a loss function value is calculated; based on the loss function value, the model parameters of the palm detection model are updated, so that the trained palm detection model can have a higher sample predicted palm frame prediction accuracy, thereby obtaining a more accurate sample predicted palm frame.

[0250] FIG13 is a flow chart of a palm detection model training method provided by an exemplary embodiment of the present application. The method is executed by a palm recognition device. The method includes:

[0251] Step 1301: Acquire a sample palm image and a sample palm image.

[0252] The sample palm image is an image of the user whose identity is to be determined. The sample palm image contains a palm, which is an image of the physical palm of the user whose identity is to be verified. The sample palm image may also contain other information, such as the user's fingers and the scene in which the user's palm was captured. The sample palm image may be obtained by the palm recognition device by capturing the physical palm of the user whose identity is to be verified, or it may be sent by another device.

[0253] The sample palm image refers to the effective identification area marked in the sample palm image, or the sample palm image refers to the area where the palm is located marked in the sample palm image, or the sample palm image refers to the area marked in the sample palm image that can be used for identification.

[0254] For example, the sample palm images are collected by obtaining 50-100 sample palm images. The sample palm images are cleaned by finding duplicate values, missing values, and outliers, thereby obtaining valid sample palm images. The code for cleaning the sample palm images can be expressed as:

[0255] The collection distance requirements for sample palm images are as follows: Place the palm at 5cm, 8cm, 10cm, 12cm, and 15cm from the palm recognition device for collection, and collect for 10-15 seconds at each distance.

[0256] The sample palm image is obtained by labeling the palm area in the sample palm image using the labeling tool. The code for labeling the palm area can be expressed as:

[0257] In the above code, 45 represents the minimum value of the palm area on the x-axis, 104 represents the maximum value of the palm area on the x-axis, 34 represents the minimum value of the palm area on the y-axis, and 84 represents the minimum value of the palm area on the y-axis.

[0258] The labeling tool Labeling combines the uniform distribution theorem with the updated label vector for the palm area in the sample palm image. Instead of the traditional one-hot encoded label vector y hot .

[0259] In the formula, k is the total number of multi-classification categories, and α is a small hyperparameter (usually 0.1).

[0260] That is, the label vector It can be expressed as:

[0261] Label vector This is equivalent to adding noise to the true distribution, preventing the model from being overconfident about the correct labels, making the difference between the output values ​​of the predicted positive and negative samples less significant, thereby avoiding overfitting and improving the generalization ability of the model.

[0262] Step 1302: Extract features from the sample palm image through the backbone network in the palm detection model to obtain sample image features at multiple scales.

[0263] For example, the palm recognition device captures the user's physical palm to obtain a sample palm image, wherein the sample palm image includes the palm, which may be the user's left palm or the user's right palm.

[0264] For example, the palm recognition device is an IoT device that uses a camera to capture the user's left palm to obtain a sample palm image. The IoT device can also be a store cash register. For another example, when a user is shopping in a store, they extend their palm toward the camera of the store's payment terminal. The store's payment terminal uses the camera to capture the user's physical palm and obtain a sample palm image.

[0265] In some embodiments, the palm recognition device establishes a communication connection with another device and receives a sample palm image sent by the other device through the communication connection. For example, the palm recognition device may be a payment application server, and the other device may be a payment terminal. The payment terminal captures the user's physical palm to obtain a sample palm image. The sample palm image is then sent to the payment application server through the communication connection between the payment terminal and the payment application server, so that the payment application server can determine the user identification of the sample palm image.

[0266] In some embodiments, the sample predicted palm frame is predicted by a palm detection model, which includes a backbone network; features are extracted from the sample palm image to obtain sample image features of multiple scales, including: inputting the sample palm image into the backbone network; slicing the sample palm image through the backbone network to obtain sliced ​​images of multiple scales; and extracting features from the sliced ​​images of multiple scales through the backbone network to obtain sample image features of multiple scales.

[0267] The palm detection model is an artificial intelligence model, which can be a deep learning model or a neural network model. The palm detection model has at least the function of determining a sample predicted palm frame that identifies the palm in the sample palm image based on the fusion features of the sample image. The backbone network is the structure in the palm detection model used to extract sample image features at multiple scales from the sample palm image. The slicing operation is an operation that divides the sample palm image into smaller-scale images. Smaller scale means that the scale of the generated image is smaller than the scale of the sample palm image. The pixels of the sliced ​​image are derived from the sample palm image.

[0268] The palm recognition device can input a sample palm image into the backbone network, which then performs a slicing operation to obtain slice images at multiple scales. The palm recognition device can then perform feature extraction on the slice images at multiple scales to obtain sample image features at multiple scales corresponding to the sample palm image.

[0269] In some embodiments, a slicing operation is performed on the sample palm image through the backbone network to obtain slice images of multiple scales, including: determining a slice image of the largest scale among multiple scales based on the sample palm image through the backbone network; using the slice image of the largest scale as the first layer through the backbone network, and starting downsampling layer by layer to obtain slice images of multiple scales including the slice image of the first layer.

[0270] The computer device may first determine a slice image of the largest scale among multiple scales based on the sample palm image. The scale of the sample image feature of the largest scale may be smaller than or equal to the scale of the sample palm image. The computer device may perform a downsampling on the sample palm image to generate a slice image of the largest scale among the multiple scales.

[0271] Downsampling the sample palm image is a process of sampling at least one pixel in the sample palm image, and splicing the downsampled pixels in the order of the sample palm image to obtain a slice image with a smaller scale.

[0272] If the number of scales in the multiple scales is N, then the multiple scales are N scales. Each of the N scales is a different scale, so there is a maximum scale among the N scales. The slice image with the maximum scale is taken as the first layer. Starting from the first layer, downsampling is performed layer by layer. Layer n is downsampled to obtain the slice image of layer n+1, and so on until layer N-1 is downsampled to obtain the slice image of layer N. n is a positive integer from 1 to N-1. Finally, N slice images are obtained, from layers 1 to N.

[0273] In this embodiment, multi-scale slice images are obtained by downsampling layer by layer, which provides a new approach to segmenting multi-scale slice images from a sample palm image.

[0274] In some embodiments, slicing the sample palm image through the backbone network to obtain sliced ​​images at multiple scales includes: sampling and splicing pixels in the sample palm image at multiple scales through the backbone network to obtain sliced ​​images at multiple scales. Two adjacent pixels in the sliced ​​images of different scales have different numbers of pixels separating them at the sampling locations in the sample palm image.

[0275] In this embodiment, the palm recognition device can sample pixels from the sample palm image at each scale using the sampling interval at that scale, and then splice the sampled pixels according to their relative positions in the sample palm image, thereby obtaining a slice image at each scale. The sampling interval is the number of pixels spaced, including the spacing between pixels in both row and column directions. The scale and sampling interval are negatively correlated: the larger the scale, the smaller the sampling interval. The smaller the scale, the larger the sampling interval. Pixels not sampled in the sample palm image can be concentrated on the channel of the sample image feature.

[0276] For example, at the maximum scale among multiple scales, every other column in the sample palm image is sampled, that is, every other pixel in the row direction is selected, and these selected pixels are spliced ​​together to obtain a slice image. The information in the spliced ​​slice image is not lost, but the size information of the sample palm image is concentrated in the channel. The resulting new image is then subjected to a convolution operation to obtain a downsampled feature map without information loss.

[0277] For example, at the second largest scale among multiple scales, every two columns of the sample palm image are sampled, that is, every two pixels are selected, and these selected pixels are spliced ​​together to obtain a slice image. Information in the spliced ​​slice image is not lost, but the size information of the sample palm image is concentrated in the channel. The resulting new image is then convolved to obtain a downsampled feature map without information loss.

[0278] For example, the original image size is 640*640*3, and through slicing, a feature map of 320*320*12 is obtained. Among them, 640*640 in the original image is used to represent width*height, and 3 in the original image is used to represent the length of the feature vector corresponding to each pixel (also known as the number of channels).

[0279] Step 1303: Through the neck network of the palm detection model, feature fusion is performed on the sample image features of multiple scales to obtain sample image fusion features.

[0280] The sample predicted palm frame is predicted by a palm detection model, which includes a neck network; sample image features at multiple scales are fused to obtain sample image fusion features, including: feature fusing sample image features at multiple scales through the neck network to obtain sample image fusion features.

[0281] The neck network, part of the palm detection model, is used to fuse features from sample images at multiple scales to generate fused features. The neck network is connected to the backbone network and receives the multiple-scale sample image features output by the backbone network.

[0282] Multi-scale sample image features refer to features extracted from images at multiple scales. Sample image features at different scales can convey different information. Among sample image features at multiple scales, features at larger scales convey more positional information, while features at smaller scales convey more semantic information.

[0283] Exemplarily, the palm recognition device extracts features of multiple scales, different receptive fields, and different categories, and fuses sample image features of multiple scales to obtain sample image fusion features corresponding to the sample palm image.

[0284] In some embodiments, the neck network is used to fuse sample image features at multiple scales to obtain a fused sample image feature, including: fusing sample image features at multiple scales at the same scale at the neck network to obtain a fused sample image feature. The fusion of sample image features at the same scale may be performed by summing or averaging.

[0285] The palm recognition device fuses sample image features at multiple scales, increasing the detection accuracy of small targets, thereby more accurately detecting the sample predicted palm frame, and more accurately cropping the sample predicted palm image, further reducing the amount of encrypted data of the sample predicted palm image, improving identity recognition efficiency, and ensuring identity recognition security.

[0286] In some embodiments, feature fusion of sample image features at multiple scales is performed through a neck network to obtain sample image fusion features, including: feature fusion based on sample image features at multiple scales is performed through a neck network to obtain first intermediate features at multiple scales; feature fusion of multiple scales based on the first intermediate features at multiple scales is performed through a neck network to obtain second intermediate features at multiple scales; and feature fusion of the second intermediate features at multiple scales is performed through a neck network to obtain sample image fusion features.

[0287] Among them, feature fusion at multiple scales refers to fusing features at different scales. The first intermediate features at multiple scales can correspond one-to-one with sample image features at multiple scales. The first intermediate features at multiple scales can correspond one-to-one with second intermediate features at multiple scales. The multiple scales of the sample image features can be consistent with the multiple scales of the first intermediate features. The multiple scales of the first intermediate features can be consistent with the multiple scales of the second intermediate features. Feature fusion of the second intermediate features at multiple scales can be performed by unifying the second intermediate features at multiple scales to the same scale for fusion, and the fusion can be performed by summing or averaging.

[0288] In this embodiment, features at different scales are fused and finally fused into sample image fusion features, which can fully express the detailed features in the sample palm image, thereby facilitating more accurate detection of the sample predicted palm frame, thereby more accurately cropping the sample predicted palm image, further reducing the amount of encrypted data of the sample predicted palm image, improving identity recognition efficiency, and ensuring identity recognition security.

[0289] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1. Extracting first intermediate features at multiple scales based on sample image features at the multiple scales includes: determining the first intermediate feature at the first position in the positive order of the scale based on the sample image feature at the first position in the positive order of the scale; upsampling layer by layer starting from the first intermediate feature at the first position in the positive order of the scale, fusing the upsampling result of the first intermediate feature at the nth position in the positive order of the scale with the sample image feature at the n+1th position in the positive order of the scale to obtain the first intermediate feature at the n+1th position in the positive order of the scale; and once the first intermediate feature at the Nth layer is obtained, determining first intermediate features at multiple scales from the 1st layer to the Nth layer. Here, n is a positive integer from 1 to N-1; and the positive order of the scale is an ascending order of scales.

[0290] The order from smallest to largest is called positive order. Positive order by scale is the process of scaling up. Upsampling is the process of scaling up. Upsampling can be performed to the scale of the next layer, interpolating adjacent feature elements in each layer to obtain the features of the next layer.

[0291] In some embodiments, determining the first intermediate feature at the first position in the positive scale order based on the sample image feature at the first position in the positive scale order includes: performing at least one convolution on the sample image feature at the first position in the positive scale order to obtain the first intermediate feature at the first position in the positive scale order. In addition to performing the convolution, channel adjustment may also be performed. After the channel adjustment, the number of channels of the first intermediate feature may be consistent.

[0292] In some embodiments, the scale of the upsampling result of the first intermediate feature at the nth position in the positive scale order can be the same as the scale of the sample image feature at the n+1th position in the positive scale order. The upsampling result of the first intermediate feature at the nth position in the positive scale order and the sample image feature at the n+1th position in the positive scale order can be fused by concatenating the two, or summing or averaging the two at the same scale.

[0293] For example, a schematic diagram of sample image feature fusion is shown in Figure 5. The palm recognition device performs feature extraction on the sample palm image 501 to obtain sample image features 502 at multiple scales corresponding to the sample palm image 501. The sample image features 502 include feature layer T1, feature layer T2, feature layer T3, and feature layer T4. After obtaining the sample image features 502, feature layer T4 is convolved once and the channels are adjusted to obtain feature layer P4. Feature layer P4 is upsampled and combined with feature layer T3, and then convolved and the channels are adjusted to obtain feature layer P3. Feature layer P3 is upsampled and combined with feature layer T2, and then convolved and the channels are adjusted to obtain feature layer P2. Feature layer P2 is upsampled and combined with feature layer T1, and then 1*1 convolution is performed to adjust the channels to obtain feature layer P1.

[0294] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the features of the multiple scales are fused based on the first intermediate features of the multiple scales to obtain the second intermediate features of the multiple scales, including: determining the second intermediate feature of the first position in the positive order of the scale based on the first intermediate feature at the first position in the reverse order of the scale; starting from the second intermediate feature at the first position in the reverse order of the scale, downsampling layer by layer, fusing the result of downsampling the second intermediate feature at the mth position in the reverse order of the scale with the first intermediate feature at the m+1th position in the reverse order of the scale to obtain the second intermediate feature at the n+1th position in the reverse order of the scale; m is an integer from 1 to N-1; the reverse order of the scale is the order from large to small; when the first intermediate feature of the Nth layer is obtained, determining the first intermediate features of multiple scales from the 1st layer to the Nth layer.

[0295] The order from largest to smallest is called reverse order, and reverse order by scale refers to the order from largest to smallest. Downsampling is the process of reducing the scale. Downsampling can target the scale of the next layer by taking intervals from the feature elements of each layer to obtain the features of the next layer. The first intermediate feature can be called the semantic sample image feature. The second intermediate feature can be called the position sample image feature. The semantic sample image feature reflects more semantic information, while the semantic sample image feature reflects more position information.

[0296] In some embodiments, determining the first-ranked second intermediate feature in reverse order of scale based on the first intermediate feature in reverse order of scale includes performing at least one convolution on the first intermediate feature in reverse order of scale to obtain the first-ranked second intermediate feature in reverse order of scale. In addition to performing the convolution, channel adjustment may also be performed. After the channel adjustment, the number of channels of the first intermediate feature may be consistent.

[0297] The scale of the down-sampled result of the second intermediate feature at the mth position in reverse scale order can be consistent with the scale of the first intermediate feature at the m+1th position in reverse scale order. The down-sampled result of the second intermediate feature at the mth position in reverse scale order and the first intermediate feature at the m+1th position in reverse scale order can be fused by concatenating the two, or summing or averaging the two at the same scale.

[0298] For example, the palm recognition device may perform one convolution on feature layer P1 and adjust the channels to obtain feature layer N1. Feature layer N1 is downsampled and combined with feature layer P2, and then convolved and adjusted for channels to obtain feature layer N2. Feature layer N2 is downsampled and combined with feature layer P3, and then convolved and adjusted for channels to obtain feature layer N3. Feature layer N3 is downsampled and combined with feature layer P4, and then convolved and adjusted for channels to obtain feature layer N4. The palm recognition device performs feature fusion on feature layers N1-4 to obtain sample image fusion features 503.

[0299] Optionally, the activation function in the convolution is a logistic activation function.

[0300] In some embodiments, the palm recognition device upsamples sample image features of multiple scales from a small scale to a large scale to obtain a first intermediate feature; the palm recognition device downsamples sample image features of multiple scales from a large scale to a small scale to obtain a second intermediate feature; the palm recognition device fuses the first intermediate feature and the second intermediate feature to obtain a sample image fusion feature.

[0301] Step 1304: Determine a sample predicted palm frame that identifies the palm in the sample palm image based on the sample image fusion features through the prediction network of the palm detection model.

[0302] The sample predicted palm frame refers to the identifier used to indicate the location of the palm in the sample palm image.

[0303] In some embodiments, the sample predicted palm frame is determined by a palm detection model, which includes a prediction network; based on the sample image fusion features, the sample predicted palm frame that identifies the palm in the sample palm image is determined, including: gridding the sample image fusion features to obtain multiple grid features of the sample image fusion features; inputting the multiple grid features into the prediction network, predicting the multiple grid features, and obtaining the sample predicted palm frame.

[0304] For example, the palm recognition device divides the sample image fusion features into grids to obtain gridded sample image fusion features. The palm recognition device then inputs the gridded fusion features into a prediction network, which predicts each grid to obtain a sample predicted palm frame. Predictions are performed on multiple grid features to obtain a sample predicted palm frame. Specifically, the prediction results of the multiple grid features can be combined to determine the position of the sample predicted palm frame.

[0305] In some embodiments, the palm recognition device is equipped with an infrared camera, and the sample palm image processing method further includes: obtaining an infrared image captured of the same palm as the sample palm image, where the infrared image is an image obtained by imaging the palm using infrared light by the infrared camera; and identifying a palm region from the infrared image. Based on the sample predicted palm frame, a sample predicted palm image is cropped from the sample palm image, including: determining the intersection of the sample predicted palm frame in the sample palm image and the palm region in the infrared image; and cropping the sample predicted palm image from the sample palm image based on the intersection.

[0306] In this embodiment, the palm recognition device is further provided with an infrared camera. The palm recognition device obtains an infrared image corresponding to the same physical palm and performs region recognition on the infrared image to determine the palm region in the infrared image. The infrared image is an image obtained by the infrared camera using infrared light to image the palm.

[0307] In some embodiments, identifying a palm area from an infrared image includes: detecting finger gaps in the infrared image; and determining the palm area in the infrared image based on the finger gaps.

[0308] Exemplarily, the palm recognition device detects finger gaps in the infrared image and determines a palm area on the infrared image based on the finger gaps.

[0309] As shown in FIG6 , the schematic diagram of the finger seam points in the palm of the hand, the finger seam point is the first finger seam point 601 between the index finger and the middle finger, or the finger seam point is the second finger seam point 602 between the middle finger and the ring finger, or the finger seam point is the third finger seam point 603 between the ring finger and the little finger.

[0310] Since the palm area in the sample palm image may exist in any area of ​​the sample palm image, in order to determine the position of the palm area in the sample palm image, finger seam point detection is performed on the sample palm image to obtain at least one finger seam point of the sample palm image, so that the palm area can be subsequently determined based on the at least one finger seam point.

[0311] Step 1305: Based on the sample predicted palm frame, a sample predicted palm image is cropped from the sample palm image.

[0312] The sample predicted palm image refers to the predicted effective recognition area in the sample palm image, or the sample predicted palm image refers to the area in the sample palm image where the palm is predicted to be located, or the sample predicted palm image refers to the area in the sample palm image that is predicted to be able to be used for recognition.

[0313] In some embodiments, the palm recognition device obtains at least three finger seam points in the infrared image; sequentially connects the finger seam points to obtain a finger seam point connection line; and the palm recognition device crops the palm area in the infrared image based on the finger seam point connection line.

[0314] In some embodiments, determining the intersection of a sample predicted palm frame in a sample palm image and a palm area in an infrared image includes: obtaining coordinate parameters of the sample predicted palm frame in the sample palm image and obtaining coordinate parameters of the palm area in the infrared image; and determining the intersection of the sample predicted palm frame and the palm area based on the coordinate parameters of the sample predicted palm frame and the coordinate parameters of the palm area.

[0315] The palm recognition device obtains the coordinate parameters of the sample predicted palm frame in the sample palm image and the coordinate parameters of the palm area in the infrared image; the palm recognition device crops the sample predicted palm image corresponding to the palm in the sample palm image based on the intersection of the coordinate parameters of the sample predicted palm frame and the coordinate parameters of the palm area.

[0316] Step 1306: Calculate the loss function value based on the sample palm image and the sample predicted palm image.

[0317] Exemplarily, the palm recognition device calculates a loss function value based on the sample palm image and the sample predicted palm image.

[0318] Step 1307: Update the model parameters of the palm detection model based on the loss function value.

[0319] Model parameter updating refers to updating the network parameters in the palm detection model, or updating the network parameters of each network module in the model, or updating the network parameters of each network layer in the model, but is not limited to this, and the embodiments of the present application do not limit this.

[0320] The model parameters of the palm detection model include at least one of network parameters of the backbone network, network parameters of the neck network, and network parameters of the prediction network.

[0321] In some embodiments, updating the model parameters of the palm detection model includes updating the network parameters of all network modules in the palm detection model, or fixing the network parameters of some network modules in the palm detection model and updating only the network parameters of the remaining network modules. For example, when updating the model parameters of the palm detection model, the network parameters of the backbone network in the palm detection model are fixed, and only the network parameters of the neck network and the prediction network are updated.

[0322] Based on the loss function value, the network parameters of the backbone network, the network parameters of the neck network and the network parameters of the prediction network in the palm detection model are updated using the loss function value as a training indicator until the loss function value converges, thereby obtaining a trained palm detection model.

[0323] The convergence of the loss function value means that the loss function value no longer changes, or the error difference between two adjacent iterations during the training of the palm detection model is less than a preset value, or the number of training times of the palm detection model reaches at least one of the preset times, but is not limited to this, and the embodiments of the present application are not limited to this.

[0324] Optionally, the target condition satisfied by training may be that the number of training iterations of the initial model reaches a target number, and the technician may pre-set the number of training iterations. Alternatively, the target condition satisfied by training may be that the loss value meets a target threshold condition, but is not limited thereto and is not limited to this embodiment of the present application.

[0325] In summary, the method provided in this embodiment obtains sample image features of multiple scales corresponding to the sample palm image by performing feature extraction on the acquired sample palm image; performs feature fusion on the sample image features of multiple scales to obtain sample image fusion features; performs prediction based on the sample image fusion features to obtain a sample predicted palm frame that identifies the palm in the sample palm image; based on the sample predicted palm frame, crops the sample palm image corresponding to the palm in the sample palm image; calculates the loss function value based on the sample palm image and the sample predicted palm image; and updates the model parameters of the palm detection model based on the loss function value, so that the trained palm detection model can have higher palm frame prediction accuracy, thereby obtaining a more accurate palm frame.

[0326] Illustratively, the application scenarios of the palm image processing method provided in the embodiments of the present application include but are not limited to the following scenarios:

[0327] For example, in the palm image recognition payment scenario:

[0328] The merchant's palm recognition device obtains the user's palm image by photographing the user's physical palm, and uses the palm image processing method provided in the embodiment of the present application to determine the target user identifier of the palm image, and transfers part of the resources in the resource account corresponding to the target user identifier to the merchant's resource account, thereby realizing automatic payment through the palm.

[0329] For example, in a cross-device payment scenario:

[0330] Users can complete identity registration using their personal mobile phones at home or other private spaces, bind their account to the user's palm image, and then use the in-store device to identify the user's palm image, determine the user's account, and pay directly through the account.

[0331] For example, in the clock-in scenario:

[0332] The palm recognition device obtains the user's palm image by photographing the user's physical palm, and uses the palm image processing method provided in the embodiment of the present application to determine the target user identifier of the palm image, establish a check-in mark for the target user identifier, and determine that the target user identifier has completed the work check-in at the current time.

[0333] Of course, in addition to being applied to the above-mentioned scenarios, the method provided in the embodiments of the present application can also be applied to other scenarios requiring palm image processing, and the embodiments of the present application are not limited to specific application scenarios.

[0334] FIG14 shows a schematic diagram of the structure of a palm image processing device provided by an exemplary embodiment of the present application. The device can be implemented as all or part of a palm recognition device through software, hardware, or a combination of both. The device includes:

[0335] The feature extraction module 1401 is used to obtain a palm image, extract features from the palm image, and obtain image features at multiple scales.

[0336] The feature fusion module 1402 is used to fuse image features at multiple scales to obtain image fusion features.

[0337] The prediction module 1403 is configured to determine a palm frame for identifying the palm in the palm image based on the image fusion features.

[0338] The cropping module 1404 is configured to crop a palm image from the palm image based on the palm frame.

[0339] The encryption module 1405 is used to encrypt the palm image to obtain palm image encrypted data. The palm image encrypted data is used to transmit to the server, and the server performs identity authentication based on the palm image encrypted data.

[0340] Among them, the feature extraction module 1401 can be the backbone network of the palm detection model, the feature fusion module 1402 can be the neck network of the palm detection model, the prediction module 1403 can be at least a part of the prediction network of the palm detection model, and the cropping module 1404 can be a part of the prediction network, or an independent module in the palm detection model, or an independent module outside the palm detection model.

[0341] In some embodiments, the feature extraction module 1401 is the backbone network of the palm detection model, which is used to slice the palm image to obtain slice images of multiple scales; and extract features from the slice images of multiple scales to obtain image features of multiple scales.

[0342] In some embodiments, the feature extraction module 1401 is the backbone network of the palm detection model, which is used to determine the slice image of the largest scale among multiple scales based on the palm image; the slice image of the largest scale is used as the first layer, and downsampling is started layer by layer to obtain slice images of multiple scales including the slice image of the first layer.

[0343] In some embodiments, the feature extraction module 1401 is the backbone network of the palm detection model, which is used to sample and splice the pixels in the palm image at multiple scales to obtain slice images of multiple scales; wherein, the number of pixels between two adjacent pixels in slice images of different scales is different at the sampling point in the palm image.

[0344] In some embodiments, the feature fusion module 1402 is a neck network of the palm detection model, which is used to fuse image features at multiple scales to obtain image fusion features.

[0345] In some embodiments, the feature fusion module 1402 is a neck network of the palm detection model, which is used to perform feature fusion based on image features at multiple scales to obtain first intermediate features at multiple scales; perform feature fusion at multiple scales based on the first intermediate features at multiple scales to obtain second intermediate features at multiple scales; and perform feature fusion on the second intermediate features at multiple scales to obtain image fusion features.

[0346] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1. The feature fusion module 1402 is a neck network of the palm detection model, which is used to determine the first intermediate feature at the first position in the positive order of the scale based on the image feature at the first position in the positive order of the scale; starting from the first intermediate feature at the first position in the positive order of the scale, upsampling layer by layer is performed, and the result of upsampling the first intermediate feature at the nth position in the positive order of the scale is fused with the image feature at the n+1th position in the positive order of the scale to obtain the first intermediate feature at the n+1th position in the positive order of the scale; n is a positive integer from 1 to N-1; the positive order of the scale is the order from small to large scale; when the first intermediate feature of the Nth layer is obtained, the first intermediate features of multiple scales from the 1st layer to the Nth layer are determined.

[0347] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1. The feature fusion module 1402 is a neck network of the palm detection model, which is used to determine the second intermediate feature at the first position in reverse order of the scale based on the first intermediate feature at the first position in reverse order of the scale; starting from the second intermediate feature at the first position in reverse order of the scale, downsampling layer by layer is performed, and the result of downsampling the second intermediate feature at the mth position in reverse order of the scale is fused with the first intermediate feature at the m+1th position in reverse order of the scale to obtain the second intermediate feature at the n+1th position in reverse order of the scale; m is an integer from 1 to N-1; reverse order of scale is an order from large to small scale; when the first intermediate feature of the Nth layer is obtained, the first intermediate features of multiple scales from the 1st layer to the Nth layer are determined.

[0348] In some embodiments, the prediction module 1403 is at least a part of the prediction network of the palm detection model, which is used to grid the image fusion features to obtain multiple grid features of the image fusion features; and perform prediction based on the multiple grid features to obtain a palm frame.

[0349] In some embodiments, the palm recognition device is equipped with an infrared camera, and the acquisition module 1406 is used to acquire an infrared image captured from the same palm as the palm image. The infrared image refers to an image obtained by the infrared camera imaging the palm based on infrared light.

[0350] The region recognition module 1407 is configured to recognize the palm region from the infrared image.

[0351] In some embodiments, the cropping module 1404 is configured to determine an intersection of a palm frame in the palm image and a palm region in the infrared image; and crop a palm image from the palm image based on the intersection.

[0352] In some embodiments, the region recognition module 1407 is configured to detect finger gaps in the infrared image and determine the palm region in the infrared image based on the finger gaps.

[0353] In some embodiments, the region identification module 1407 is used to obtain at least three finger seam points in the infrared image; sequentially connect the finger seam points to obtain a finger seam point connection line; and based on the finger seam point connection line, crop the palm region in the infrared image.

[0354] In some embodiments, the cropping module 1404 is used to obtain the coordinate parameters of the palm frame in the palm image and the coordinate parameters of the palm area in the infrared image; and determine the intersection of the palm frame and the palm area based on the coordinate parameters of the palm frame and the coordinate parameters of the palm area.

[0355] FIG15 shows a schematic diagram of the structure of a training device for a palm detection model provided by an exemplary embodiment of the present application. The device can be implemented as all or part of a palm recognition device through software, hardware, or a combination of both. The device includes a palm detection model, an acquisition module 1501, a calculation module 1506, and an update module 1507. The palm detection model includes a feature extraction module 1502, a feature fusion module 1503, and a prediction module 1504. It can also include a cropping module 1505. Among them, the feature extraction module 1502 can be the backbone network of the palm detection model, the feature fusion module 1503 can be the neck network of the palm detection model, the prediction module 1504 can be at least a part of the prediction network, and the cropping module 1505 can be a part of the prediction network, or an independent module in the palm detection model, or an independent module outside the palm detection model.

[0356] The acquisition module 1501 is used to acquire a sample palm image and a sample palm image.

[0357] The feature extraction module 1502 is used to extract features from the sample palm image to obtain sample image features at multiple scales.

[0358] The feature fusion module 1503 is used to fuse sample image features of multiple scales to obtain sample image fusion features.

[0359] The prediction module 1504 is configured to determine a sample predicted palm frame that identifies the palm in the sample palm image based on the sample image fusion features.

[0360] The cropping module 1505 is configured to crop a sample predicted palm image from the sample palm image based on the sample predicted palm frame.

[0361] The calculation module 1506 is configured to calculate a loss function value based on the sample palm image and the sample predicted palm image.

[0362] The updating module 1507 is configured to update the model parameters of the palm detection model based on the loss function value.

[0363] In some embodiments, the feature extraction module 1502 is the backbone network of the palm detection model, which is used to slice the sample palm image to obtain slice images of multiple scales; and extract features from the slice images of multiple scales to obtain sample image features of multiple scales.

[0364] In some embodiments, the feature extraction module 1502 is the backbone network of the palm detection model, which is used to determine the slice image of the largest scale among multiple scales based on the sample palm image; take the slice image of the largest scale as the first layer, and start downsampling layer by layer to obtain slice images of multiple scales including the slice image of the first layer.

[0365] In some embodiments, the feature extraction module 1502 is the backbone network of the palm detection model, which is used to sample and splice the pixels in the sample palm image at multiple scales to obtain slice images of multiple scales; wherein, the number of pixels between two adjacent pixels in the slice images of different scales is different at the sampling point in the sample palm image.

[0366] In some embodiments, the feature fusion module 1503 is a neck network of the palm detection model, which is used to fuse sample image features of multiple scales to obtain sample image fusion features.

[0367] In some embodiments, the feature fusion module 1503 is the neck network of the palm detection model, which is used to input sample image features of multiple scales into the neck network; perform feature fusion based on the sample image features of multiple scales to obtain first intermediate features of multiple scales; perform feature fusion of multiple scales based on the first intermediate features of multiple scales to obtain second intermediate features of multiple scales; perform feature fusion on the second intermediate features of multiple scales to obtain sample image fusion features.

[0368] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1. The feature fusion module 1503 is a neck network of the palm detection model, which is used to determine the first intermediate feature at the first position in the positive order of the scale based on the sample image feature at the first position in the positive order of the scale; starting from the first intermediate feature at the first position in the positive order of the scale, upsampling layer by layer is performed, and the result of upsampling the first intermediate feature at the nth position in the positive order of the scale is fused with the sample image feature at the n+1th position in the positive order of the scale to obtain the first intermediate feature at the n+1th position in the positive order of the scale; n is a positive integer from 1 to N-1; the positive order of the scale is the order from small to large scale; when the first intermediate feature of the Nth layer is obtained, the first intermediate features of multiple scales from the 1st layer to the Nth layer are determined.

[0369] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1. The feature fusion module 1503 is used to determine the second intermediate feature at the first position in reverse order of the scale based on the first intermediate feature at the first position in reverse order of the scale; starting from the second intermediate feature at the first position in reverse order of the scale, down-sampling layer by layer, and fusing the result of downsampling the second intermediate feature at the mth position in reverse order of the scale with the first intermediate feature at the m+1th position in reverse order of the scale to obtain the second intermediate feature at the n+1th position in reverse order of the scale; m is an integer from 1 to N-1; reverse order of scale is the order from large to small; when the first intermediate feature of the Nth layer is obtained, the first intermediate features of multiple scales from the 1st layer to the Nth layer are determined.

[0370] In some embodiments, the prediction module 1504 is at least a part of the prediction network of the palm detection model, which is used to grid the sample image fusion features to obtain multiple grid features of the sample image fusion features; and predict the multiple grid features to obtain a sample predicted palm frame.

[0371] In some embodiments, the palm recognition device is equipped with an infrared camera, and the acquisition module 1501 is used to acquire an infrared image captured from the same palm as the sample palm image. The infrared image refers to an image obtained by the infrared camera imaging the palm based on infrared light.

[0372] The region recognition module 1508 is configured to recognize the palm region from the infrared image.

[0373] The cropping module 1505 is configured to determine the intersection of the sample predicted palm frame in the sample palm image and the palm region in the infrared image; and crop the palm image from the sample palm image based on the intersection.

[0374] In some embodiments, the region identification module 1508 is configured to detect finger gaps in the infrared image; and based on the finger gaps, determine the palm region in the infrared image.

[0375] In some embodiments, the cropping module 1505 is used to obtain the coordinate parameters of the sample predicted palm frame in the sample palm image, and to obtain the coordinate parameters of the palm area in the infrared image; based on the coordinate parameters of the sample predicted palm frame and the coordinate parameters of the palm area, determine the intersection of the sample predicted palm frame and the palm area.

[0376] In some embodiments, the region recognition module 1508 is configured to detect finger seam points in the infrared image and determine the palm region on the infrared image based on the finger seam points.

[0377] In some embodiments, the region identification module 1508 is used to obtain at least three finger seam points in the infrared image; sequentially connect the finger seam points to obtain a finger seam point connection line; and based on the finger seam point connection line, crop the palm region in the infrared image.

[0378] FIG16 shows a block diagram of a computer device 1600 according to an exemplary embodiment of the present application. The computer device can be implemented as the server described in the aforementioned embodiment of the present application. The image computer device 1600 includes a central processing unit (CPU) 1601, a system memory 1604 including a random access memory (RAM) 1602 and a read-only memory (ROM) 1603, and a system bus 1605 connecting the system memory 1604 and the CPU 1601. The image computer device 1600 also includes a mass storage device 1606 for storing an operating system 1609, application programs 1610, and other program modules 1611.

[0379] The mass storage device 1606 is connected to the central processing unit 1601 via a mass storage controller (not shown) connected to the system bus 1605. The mass storage device 1606 and its associated computer-readable media provide non-volatile storage for the image computing device 1600. In other words, the mass storage device 1606 may include computer-readable media (not shown) such as a hard disk or a Compact Disc Read-Only Memory (CD-ROM) drive.

[0380] Without loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, Erasable Programmable Read Only Memory (EPROM), Electronically Erasable Programmable Read-Only Memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, tape cassettes, magnetic tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above-mentioned ones. The above-mentioned system memory 1604 and mass storage device 1606 can be collectively referred to as memory.

[0381] According to various embodiments of the present disclosure, the image computer device 1600 may also be connected to a remote computer on a network such as the Internet for operation. That is, the image computer device 1600 may be connected to a network 1608 via a network interface unit 1607 connected to the system bus 1605, or the network interface unit 1607 may be used to connect to other types of networks or remote computer systems (not shown).

[0382] The memory also includes at least one segment of computer-readable instructions, which are stored in the memory. The central processing unit 1601 implements the palm image processing method shown in the above-mentioned embodiments, or all or part of the steps in the palm detection model training method by executing the at least one segment of the program.

[0383] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the palm image processing method provided by the above-mentioned method embodiments, or the palm detection model training method.

[0384] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one computer-readable instruction. The at least one computer-readable instruction is loaded and executed by a processor to implement the palm image processing method provided in the above-mentioned method embodiments, or the palm detection model training method.

[0385] An embodiment of the present application also provides a computer program product, which includes computer-readable instructions stored in a computer-readable storage medium; the computer-readable instructions are read and executed by a processor of a computer device from the computer-readable storage medium, so that the computer device executes to implement the palm image processing method provided in the above-mentioned method embodiments, or the palm detection model training method.

[0386] It is understandable that in the specific implementation methods of this application, the data involved, such as palm images, historical data, and portraits, etc., related to user data processing related to user identity or characteristics, when the above embodiments of this application are applied to specific products or technologies, need to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0387] It should be understood that the term "plurality" used herein refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0388] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0389] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0390] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A palm image processing method, performed by a palm recognition device, comprising: Acquire a palm image, extract features from the palm image, and obtain image features at multiple scales; fusing the image features at the multiple scales to obtain an image fusion feature; determining a palm frame identifying a palm in the palm image based on the image fusion feature; Based on the palm frame, cropping a palm image from the palm image; and The palm image is encrypted to obtain palm image encrypted data, which is transmitted to a server, and the server performs identity recognition based on the palm image encrypted data.

2. The method according to claim 1, wherein the palm frame is predicted by a palm detection model, wherein the palm detection model comprises a backbone network; The feature extraction from the palm image to obtain image features at multiple scales includes: Slicing the palm image through the backbone network to obtain slice images of multiple scales; The backbone network is used to extract features from the slice images of the multiple scales respectively to obtain image features of the multiple scales.

3. The method according to claim 2, wherein slicing the palm image using the backbone network to obtain slice images at multiple scales comprises: Determining, by the backbone network, a slice image of the largest scale among the multiple scales based on the palm image; Through the backbone network, the slice image of the largest scale is taken as the first layer, and downsampling is started layer by layer to obtain slice images of multiple scales including the slice image of the first layer.

4. The method according to claim 2, wherein slicing the palm image using the backbone network to obtain slice images at multiple scales comprises: Through the backbone network, pixels in the palm image are sampled and spliced ​​at multiple scales to obtain slice images at multiple scales; Wherein, the number of pixel points between two adjacent pixel points in the slice images of different scales and the two pixel points at the sampling location in the palm image is different.

5. The method according to any one of claims 1 to 3, wherein the palm frame is predicted by a palm detection model, wherein the palm detection model includes a neck network; The fusing the image features at the multiple scales to obtain the image fusion feature includes: The image features at multiple scales are input into the neck network for feature fusion to obtain image fusion features.

6. The method according to claim 5, wherein inputting the image features at multiple scales into the neck network for feature fusion to obtain image fusion features comprises: inputting the image features at the multiple scales into the neck network; Performing feature fusion based on the image features at the multiple scales through the neck network to obtain first intermediate features at the multiple scales; Performing feature fusion of multiple scales based on the first intermediate features of the multiple scales through the neck network to obtain second intermediate features of the multiple scales; The second intermediate features of the multiple scales are fused through the neck network to obtain image fusion features.

7. The method according to claim 6, wherein the multiple scales are N scales, where N is a positive integer greater than 1, and extracting first intermediate features at multiple scales based on the image features at the multiple scales comprises: Determine, based on the image feature that ranks first in the positive order of scale, a first intermediate feature that ranks first in the positive order of scale; Starting from the first intermediate feature at the 1st position in the positive order of scale, upsample layer by layer, fuse the upsampling result of the first intermediate feature at the nth position in the positive order of scale with the image feature at the n+1th position in the positive order of scale, and obtain the first intermediate feature at the n+1th position in the positive order of scale; n is a positive integer from 1 to N-1; Positive order by scale is from small to large; When the first intermediate features of the Nth layer are obtained, first intermediate features of multiple scales from the 1st layer to the Nth layer are determined.

8. The method according to claim 6 or 7, wherein the multiple scales are N scales, where N is a positive integer greater than 1, and the performing feature fusion of multiple scales based on the first intermediate features of the multiple scales to obtain second intermediate features of the multiple scales comprises: Determine the second intermediate feature that is ranked first in reverse order of scale based on the first intermediate feature that is ranked first in reverse order of scale; Starting from the second intermediate feature at the 1st position in reverse scale order, downsample layer by layer, fuse the downsampled result of the second intermediate feature at the mth position in reverse scale order with the first intermediate feature at the m+1th position in reverse scale order to obtain the second intermediate feature at the n+1th position in reverse scale order; m is an integer from 1 to N-1; In reverse order of scale, the scale is arranged from largest to smallest; When the first intermediate features of the Nth layer are obtained, first intermediate features of multiple scales from the 1st layer to the Nth layer are determined.

9. The method according to any one of claims 1 to 8, wherein the palm frame is determined by a palm detection model, the palm detection model comprising a prediction network; The determining, based on the image fusion feature, a palm frame identifying the palm in the palm image includes: Dividing the image fusion feature into grids to obtain a plurality of grid features of the image fusion feature; The multiple grid features are input into the prediction network, and the multiple grid features are predicted to obtain the palm frame.

10. The method according to any one of claims 1 to 9, wherein the palm recognition device is equipped with an infrared camera, and the method further comprises: Acquire an infrared image captured from the same palm as the palm image, where the infrared image refers to an image obtained by imaging the palm with infrared light by the infrared camera; identifying a palm region from the infrared image; The step of cropping a palm image from the palm image based on the palm frame includes: determining an intersection of the palm frame in the palm image and the palm area in the infrared image; Based on the intersection, a palm image is cropped from the palm image.

11. The method according to claim 10, wherein identifying a palm area from the infrared image comprises: detecting finger gap points in the infrared image; The palm area in the infrared image is determined based on the finger seam points.

12. The method according to claim 10 or 11, wherein determining the intersection of the palm frame in the palm image and the palm area in the infrared image comprises: Acquiring coordinate parameters of the palm frame in the palm image, and acquiring coordinate parameters of the palm area in the infrared image; An intersection of the palm frame and the palm region is determined based on the coordinate parameters of the palm frame and the coordinate parameters of the palm region.

13. A palm detection model training method, executed by a computer device, comprising: Acquire a sample palm image and a sample palm image; Extracting features from the sample palm image to obtain sample image features at multiple scales; Fusing the sample image features at the multiple scales to obtain a sample image fusion feature; determining a sample predicted palm frame for identifying a palm in the sample palm image based on the sample image fusion feature; Based on the sample predicted palm frame, cropping a sample predicted palm image from the sample palm image; Calculating a loss function value according to the sample palm image and the sample predicted palm image; and Model parameters of the palm detection model are updated based on the loss function value.

14. A palm image processing device, comprising: A feature extraction module is used to obtain a palm image, extract features from the palm image, and obtain image features at multiple scales; A feature fusion module, configured to fuse the image features at multiple scales to obtain image fusion features; a prediction module, configured to determine a palm frame identifying a palm in the palm image based on the image fusion feature; a cropping module, configured to crop a palm image from the palm image based on the palm frame; and The encryption module is used to encrypt the palm image to obtain palm image encryption data. The palm image encrypted data is used to be transmitted to a server, and the server performs identity authentication based on the palm image encrypted data.

15. A palm detection model training device, comprising a palm detection model, an acquisition module, a cropping module, a calculation module, and an update module, wherein the palm detection model comprises a feature extraction module, a feature fusion module, and a prediction module; The acquisition module is used to acquire a sample palm image and a sample palm image; The feature extraction module is used to extract features from the sample palm image to obtain sample image features at multiple scales; The feature fusion module is used to fuse the sample image features of the multiple scales to obtain sample image fusion features; The prediction module is configured to determine a sample predicted palm frame identifying the palm in the sample palm image based on the sample image fusion features; The cropping module is configured to crop a sample predicted palm image from the sample palm image based on the sample predicted palm frame; The calculation module is used to calculate the loss function value according to the sample palm image and the sample predicted palm image; The updating module is used to update the model parameters of the palm detection model based on the loss function value.

16. A computer device comprising a processor and a memory, wherein the memory stores at least one computer-readable instruction, and the at least one computer-readable instruction is loaded and executed by the processor to implement the method according to any one of claims 1 to 13. 17 . A computer storage medium, wherein at least one computer-readable instruction is stored in the computer-readable storage medium, and the at least one computer-readable instruction is loaded and executed by a processor to implement the method according to claim 1 .

18. A computer program product, comprising computer-readable instructions, wherein the computer-readable instructions are loaded and executed by a processor to implement the method according to any one of claims 1 to 13.