Palm image processing method, palm detection model training method, palm image processing device, palm detection model training device, computer device, and computer program

Abstract

A palm image processing method executed by a palm recognition device includes the steps of: acquiring a palm image and extracting features from the palm image to acquire image features at multiple scales (302); fusing the image features at multiple scales to acquire image fusion features (304); determining a palm frame that identifies a palm in the palm image based on the image fusion features (306); cropping a palm image from the palm image based on the palm frame (308); and performing image encryption on the palm image to acquire palm image encrypted data (310), which is transmitted to a server and used by the server to perform identity recognition based on the palm image encrypted data (310).

Description

[Technical Field]

[0001] This application claims priority to a Chinese patent application filed on November 21, 2022, bearing application number 2022114733356 and entitled "Palm image recognition method, apparatus, device, storage medium, and program product," the entire contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD Embodiments of the present application relate to the technical field of computers, and further to the technical field of computer vision, and in particular to a palm image processing method, apparatus, device, storage medium, and program product. [Background technology]

[0003] With the development of computer technology, palm recognition technology has become more and more applicable and can be applied to various scenarios, such as payment scenarios and punching a commuter time card, where palm recognition can be used to verify the user's identity.

[0004] In the related technology, a user provides a physical palm, a palm recognition device collects a palm image from the user's palm, the palm image recognition device encrypts the palm image, and the encrypted palm image is used for identity verification.

[0005] However, the time required to encrypt and decrypt palm images is long, which reduces the efficiency of palm recognition. Summary of the Invention [Problem to be solved by the invention]

[0006] According to various embodiments of the present application, a palm image processing method, apparatus, device, storage medium, and program product are provided. [Means for solving the problem]

[0007] In one aspect, there is provided a palm image processing method performed by a palm recognition device, the method comprising: acquiring a palm image and extracting features from the palm image to obtain image features at multiple scales; fusing the image features of the multiple scales to obtain an image fusion feature; determining a palm frame that identifies a palm in the palm image based on the image fusion features; cutting out a palm image from the palm image based on the palm frame; and a step of obtaining palm image encrypted data by performing image encryption on the palm image, the palm image encrypted data being transmitted to a server and used by the server to perform identity recognition based on the palm image encrypted data.

[0008] According to another aspect, the present application provides a method for training a palm detection model, the method being executed by a computing device, the method comprising: acquiring a sample palm image and a sample palm region image; extracting features from the sample palm image to obtain sample image features at multiple scales; fusing the sample image features at the multiple scales to obtain a sample image fusion feature; determining a sample predicted palm outline based on the sample image fusion features, the sample predicted palm outline identifying a palm in the sample palm image; cropping a sample predicted palm image from the sample palm image based on the sample predicted palm frame; calculating a loss function value based on the sample palm image and the sample predicted palm image; and updating model parameters of the palm detection model based on the loss function value.

[0009] According to another aspect, the present application provides a palm image processing device, the device comprising: a feature extraction module that acquires a palm image and extracts features from the palm image to acquire image features at multiple scales; a feature fusion module for fusing the image features of the multiple scales to obtain an image fusion feature; a prediction module that determines a palm frame that identifies a palm in the palm image based on the image fusion features; a cutout module that cuts out a palm image from the palm image based on the palm frame; an encryption module that obtains palm image encrypted data by performing image encryption on the palm image, the palm image encrypted data being transmitted to a server and used by the server to perform identity verification based on the palm image encrypted data.

[0010] According to another aspect, the present application provides an apparatus for training a palm detection model, the apparatus including a palm detection model, further including an acquisition module, a cropping module, a calculation module, and an update module, the palm detection model including a feature extraction module, a feature fusion module, and a prediction module; The acquisition module acquires a sample palm image and a sample palm portion image; the feature extraction module extracts features from the sample palm image to obtain sample image features at multiple scales; the feature fusion module fuses the sample image features at the multiple scales to obtain sample image fusion features; the prediction module determines a sample predicted palm outline that identifies a palm in the sample palm image based on the sample image fusion features; the cropping module crops a sample predicted palm image from the sample palm image based on the sample predicted palm frame; the calculation module calculates a loss function value based on the sample palm image and the sample predicted palm image; The update module updates model parameters of a palm detection model based on the loss function value.

[0011] According to another aspect, the present application further provides a computing device comprising a memory and a processor, the memory storing computer-readable instructions that, when executed by the processor, perform steps of each method embodiment of the present application.

[0012] According to another aspect, the present application provides a computer-readable storage medium having stored thereon computer-readable instructions that, when executed by a processor, cause the processor to perform steps of each method embodiment of the present application.

[0013] According to another aspect, the present application provides a computer program product including computer-readable instructions that, when executed by a processor, cause the processor to perform the steps of each method embodiment of the present application.

[0014] The details of one or more embodiments of the present application are set forth in the drawings and description below. Other features, objects, and advantages of the present application will become apparent from the description, drawings, and claims. [Brief explanation of the drawings]

[0015] In order to more clearly explain the configuration of the embodiments of the present application, the following briefly introduces the drawings necessary for the description of the embodiments. Obviously, the drawings in the following description only show some embodiments of the present application, and those skilled in the art can derive other drawings from these drawings without creative work. [Figure 1] FIG. 1 is a schematic diagram of a palm image processing method provided in one exemplary embodiment of the present application. [Figure 2]1 is a schematic diagram of the architecture of a computer system provided in one exemplary embodiment of the present application. [Figure 3] 1 is a flowchart of a palm image processing method provided in one exemplary embodiment of the present application. [Figure 4] 1 is a flowchart of a palm image processing method provided in one exemplary embodiment of the present application. [Figure 5] FIG. 1 is a schematic diagram of image feature fusion provided in one exemplary embodiment of the present application. [Figure 6] 1 is a schematic diagram of webbing in the palm of a hand provided in one exemplary embodiment of the present application. [Figure 7] FIG. 1 is a schematic diagram of a palm image processing method provided in one exemplary embodiment of the present application. [Figure 8] FIG. 1 is a flow diagram based on a scenario in which the palm image processing method provided in one exemplary embodiment of the present application is applied to cross-device payment. [Figure 9] FIG. 1 is a flow diagram based on a scenario in which the palm image processing method provided in one exemplary embodiment of the present application is applied to cross-device identity verification. [Figure 10] FIG. 2 is a schematic diagram of the flow of a palm image processing method provided in one exemplary embodiment of the present application. [Figure 11] FIG. 1 is a framework diagram of the palm detection model generation stage and palm frame prediction stage provided in one exemplary embodiment of the present application. [Figure 12] 1 is a flowchart of a method for training a palm detection model provided in one exemplary embodiment of the present application. [Figure 13] 1 is a flowchart of a method for training a palm detection model provided in one exemplary embodiment of the present application. [Figure 14] 1 is a block diagram of a palm image processing device provided in one exemplary embodiment of the present application; [Figure 15] FIG. 1 is a block diagram of a training device for a palm detection model provided in one exemplary embodiment of the present application. [Figure 16]1 is a schematic diagram of the configuration of a computing device provided in one exemplary embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, the configuration of the embodiments of the present application will be described clearly and completely with reference to the drawings of the embodiments of the present application. It is clear that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments that a person skilled in the art can obtain from the embodiments of the present application without creative work fall within the scope of protection of the present application.

[0017] Computer vision (CV) is the science that studies how machines "see"—using cameras and computers instead of human eyes to perform machine vision tasks such as target recognition and measurement, and then using graphics processing and computer processing to obtain images suitable for human observation or transmission to a machine for detection. As a branch of science, CV studies related theories and technologies, aiming to build artificial palm image recognition systems that can extract information from images or multidimensional data. CV technology typically includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / action recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning, and map building, as well as general biometric feature recognition technology.

[0018] In the embodiment of the present application, a schematic diagram of a palm image processing method is provided. As shown in Figure 1, the method may be performed by a palm recognition device 10. The palm recognition device 10 may have an image encryption function.

[0019] For example, the palm recognition device 10 includes a camera. The palm recognition device 10 acquires a palm image 11 using the camera. The palm recognition device 10 may be connected to an image collector. The image collector has a camera, and the palm recognition device 10 can acquire the palm image 11 collected by the image collector. The palm recognition device 10 performs palm region prediction on the palm image 11 to acquire a palm image 12 corresponding to the palm region in the palm image 11. The palm recognition device 10 performs image encryption on the palm image 12 to acquire palm image encrypted data 13. The palm recognition device 10 transmits the palm image encrypted data 13 to a palm recognition server for decryption and palm region comparison recognition, thereby acquiring a user identifier 14 corresponding to the palm image 11.

[0020] For example, the palm recognition device 10 acquires a palm image 11 and extracts features from the palm image 11 to obtain image features at multiple scales. The palm recognition device 10 performs feature fusion on the image features at multiple scales to obtain image fusion features. The palm recognition device 10 determines a palm frame in the palm image 11 based on the image fusion features. The palm recognition device 10 cuts out a palm image 12 from the palm image 11 based on the palm frame. The palm recognition device 10 performs image encryption on the palm image 12 to obtain palm image encrypted data 13.

[0021] The palm image encrypted data 13 is sent to a palm recognition server, where it is used to obtain a user identifier 14 corresponding to the palm image 11 by performing decryption and palm comparison recognition.

[0022] The palm image 12 is the valid recognition area in the palm image 11, or the palm image 12 refers to the area in the palm image 11 where the palm is located, or the palm image 12 refers to the area in the palm image 11 that can be used for palm recognition.

[0023] Optionally, the manner in which the palm recognition device 10 transmits the palm image encrypted data 13 to the palm recognition server includes, but is not limited to, at least one of network transmission, data line transmission, and Bluetooth transmission, although the embodiments of the present application do not specifically limit this.

[0024] For example, the palm recognition device 10 obtains image features by extracting features from a palm image 11. The palm recognition device 10 obtains image fusion features by performing feature fusion on the image features. The palm recognition device 10 determines a palm frame in the palm image and cuts out a palm image 12 from the palm image based on the palm frame.

[0025] Illustratively, the palm frame may be predicted by a palm detection model, which includes a backbone network, a neck network, and a prediction network.

[0026] The palm recognition device 10 inputs a palm image 11 to the backbone network, which performs a slicing operation on the palm image to obtain slice images at multiple scales, and the palm recognition device 10 performs feature extraction on the slice images at multiple scales to obtain image features at multiple scales corresponding to the palm image 11. The scale represents the magnitude of the feature size, and multiple scales refer to multiple sizes.

[0027] For example, the size of the input palm image 11 is 640*640, and the sizes of the output image features at multiple scales are 80*80, 40*40, and 20*20. The scale and size of the image or feature may be the dimensions of a matrix representing the image or feature.

[0028] For example, the palm recognition device 10 acquires image fusion features by inputting image features at multiple scales into a network and performing feature fusion. The palm recognition device 10 acquires a palm frame in the palm image 11 by inputting the image fusion features into a prediction network and performing prediction. The palm recognition device 10 cuts out the image area outside the palm frame in the palm image 11, and defines the remaining image as the palm image 12.

[0029] As described above, the method provided in this embodiment acquires a palm image and performs feature extraction to obtain image features of multiple scales corresponding to the palm image, performs feature fusion on the image features of multiple scales to obtain image fusion features, determines a palm frame corresponding to the palm portion in the palm image based on the image fusion features, cuts out a palm image corresponding to the palm portion from the palm image based on the palm frame, encrypts the palm image to obtain encrypted palm image data, and transmits the encrypted palm image data to a palm recognition server for decryption and palm comparison recognition, thereby performing identity recognition. In this application, by cutting out a palm image corresponding to the palm portion from the palm image and encrypting, transmitting, and decrypting only the palm image, the time required for palm image recognition is reduced and the recognition efficiency is improved.

[0030] 2 shows a schematic diagram of the architecture of a computer system provided in one embodiment of the present application. The computer system may include a palm recognition device 100 and a palm recognition server 200.

[0031] The palm recognition device 100 may be an electronic device such as a mobile phone, a tablet computer, an in-vehicle terminal (in-vehicle device), a wearable device, a personal computer (PC), a voice interaction device with palm image recognition function, a home appliance with palm image recognition function, an in-vehicle terminal, an aircraft, or an unmanned vending machine. A client that executes an application may be installed in the palm recognition device 100. The application may be an application that specializes in palm image recognition or another application that provides palm image recognition function, but this application is not limited to this. Furthermore, in this application, the form of the application is not limited, and includes, but is not limited to, an application (App), a child application, a web page program, etc. installed in the palm recognition device 100. Here, a child application is a program that runs in an execution environment provided by a parent application, and a parent application is an independent native application that runs depending on the parent application.

[0032] The palm recognition server 200 may be an independent physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server that provides base cloud computing services such as cloud database, cloud computing, cloud function, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial palm image recognition platforms. The palm recognition server 200 may be a back-end server for the above applications and provide back-end services to clients of the applications.

[0033] Cloud technology is a hosting technology that integrates a set of resources, such as hardware, software, and networks, within a wide-area network or local network to realize data calculation, storage, processing, and sharing. Cloud technology is a collective term for network technology, information technology, integration technology, management platform technology, and application technology applied to cloud computing-based business models, which form resource pools and enable on-demand utilization, providing flexibility and convenience. Cloud computing technology is an important support. The back-end services of technology network systems require large amounts of computing and storage resources, such as video sites, picture sites, and more portal sites. With the rapid development and application of the Internet industry, in the future, each individual item may have its own identifying mark, which will all need to be transmitted to a back-end system for logic processing. Different levels of data will be processed separately, and various types of industry data will require powerful system support, which can only be achieved through cloud computing.

[0034] In some embodiments, the server may be implemented as a node in a blockchain system. Blockchain is a new application mode of computer technology, including distributed data storage, point-to-point transmission, consensus mechanisms, and encryption algorithms. A blockchain is essentially a decentralized database, a series of data blocks associated and generated using cryptographic methods, each containing information about a single network transaction, verifying the validity of the information (anti-counterfeiting), and generating the next block. A blockchain may include a blockchain bottom-layer platform, a platform product service layer, and an application service layer.

[0035] Communication between the palm recognition device 100 and the palm recognition server 200 is possible via, for example, a wired or wireless network.

[0036] 3 is a flowchart of a palm image processing method provided in one exemplary embodiment of the present application. The method is performed by a palm recognition device. As can be understood, the method may also be performed by a server. The method includes the following steps:

[0037] In step 302, a palm image is acquired and features are extracted from the palm image to obtain image features at multiple scales.

[0038] Here, a palm image is an image of a physical palm. The physical palm includes the palm and fingers, and the palm is the organic part connecting the physical arm and the physical fingers. The palm image includes an image of all or part of the physical palm. For ease of explanation, the image of the physical palm in the palm image will be referred to as the palm hereinafter. The palm included in the palm image may be the physical palm of a user whose identity is to be recognized. The palm in the palm image includes at least the palm, and may also include fingers or a portion of the arm, and may include an image of the environment in which the physical palm is located when the physical palm is captured.

[0039] The scale is the size of the image feature and can be expressed as the number of dimensions of a matrix, for example, 100*100, 80*80, 60*60, or 40*40. The multiple scales refer to two or more scales, each different from the others. The maximum scale among the multiple scales may be the same as the scale of the palm image, but is generally smaller than the scale of the palm image.

[0040] The palm image may be an image of the physical palm of the user whose identity is to be verified captured by the palm recognition device, or may be transmitted from another device. The palm recognition device may acquire the palm image by directly collecting an image of the physical palm. The palm recognition device may acquire the palm image by collecting an image of the physical palm using an image collector connected to the palm recognition device. The palm recognition device may perform feature extraction on the acquired palm image to acquire image features at multiple scales corresponding to the palm image.

[0041] For example, the palm recognition device may be a store register device that captures an image of the user's palm using a camera, or may be a palm image recognition server that captures an image of the user's palm and then transmits the image to the palm image recognition server.

[0042] For example, the image features at multiple scales may be extracted from the low-resolution images at multiple scales after downsampling a high-resolution palm image to low-resolution images at multiple scales, where high resolution and low resolution are relative concepts, and high resolution is higher than low resolution.

[0043] For example, the size of the input palm image is 640*640, and the sizes of the output multi-scale image features are 80*80, 40*40, and 20*20.

[0044] In step 304, image fusion features are obtained by fusing image features at multiple scales.

[0045] The palm recognition device can obtain image fusion features by performing feature fusion on image features at multiple scales.

[0046] Multi-scale image features refer to features obtained by performing feature extraction on images at multiple scales. Image features at different scales can express different information. Here, among multi-scale image features, the larger the scale of the image feature, the more positional information it expresses, and the smaller the scale of the image feature, the more semantic information it expresses.

[0047] Illustratively, the palm recognition device extracts features of different categories at multiple scales and different receptive fields, and fuses the image features at multiple scales to obtain image fusion features corresponding to the palm image.

[0048] In step 306, a palm frame is determined based on the image fusion features to identify the palm in the palm image.

[0049] The palm frame refers to a mark that indicates the position of the palm in the palm image. The palm frame includes the palm in the palm image. The boundary of the palm frame surrounds the palm in the palm image.

[0050] Optionally, the shape of the palm frame may be a regular shape, for example, at least one of a rectangle, a square, a diamond, a hexagon, a circle, or a triangle. The shape of the palm frame may be an irregular shape. A regular shape is a shape that can be drawn with fixed parameters and based on fixed rules. An irregular shape cannot be drawn with fixed parameters and based on fixed rules, and generally can only be represented by a set of points that make up the irregular shape.

[0051] The palm recognition device may use an artificial intelligence method to make predictions based on image fusion features to determine a palm frame corresponding to the palm in the palm image.

[0052] In some embodiments, a computing device may determine the position of a palm frame in a palm image, which identifies the palm in the palm image, based on image fusion features. If the palm frame is a regular shape, its position can be represented by values ​​of fixed parameters required to draw the regular shape, such as the position of a fixed point of the palm frame and a size parameter of the palm frame. The fixed point is a point that is fixed relative to the position of the palm frame. The size parameter is a parameter that represents the size of the palm frame. For example, if the palm frame is a rectangle, the fixed point may be any corner point or the center point of the rectangle, and the size parameter is the side length of the adjacent side of the palm frame.

[0053] In step 308, the palm image is cut out from the palm image based on the palm frame.

[0054] The palm image is the effective recognition area in the palm image, or the palm image refers to the area in the palm image where the palm is located, or the palm image refers to the area in the palm image that can be used for palm recognition.

[0055] For example, the palm recognition device cuts out an image area of ​​the palm image that is outside the palm frame based on the position of the palm frame, and regards the remaining image as the palm image.

[0056] In step 310, image encryption is performed on the palm image to obtain palm image encrypted data, which is transmitted to a server and used by the server to perform identity recognition based on the palm image encrypted data.

[0057] The palm image encrypted data is data obtained by encrypting a palm image, and the palm image can be obtained by decrypting the palm image encrypted data. Here, the palm image encrypted data may be a bit stream or a character string, or may be an image whose contents are difficult to discern with the naked eye. The palm image is transmitted to a server, and the server uses the palm image encrypted data to perform identity recognition.

[0058] The palm recognition device may access a network and transmit the encrypted palm image data to a server via the network. The palm recognition device may transmit the encrypted palm image data to the terminal via a point-to-point connection, and the terminal that has accessed the network may transmit the encrypted palm image data to the server. If the palm recognition device itself is a server, it may transmit the encrypted palm image data to another server via a direct cable (also called a straight cable) or a network.

[0059] In some embodiments, the server may obtain the palm image by decrypting the palm image encryption data and perform identity recognition based on the decrypted palm image.

[0060] In some embodiments, the server may obtain the palm image by decrypting the palm image encryption data, search a database for a palm image that matches the decrypted palm image, and if a match is found, determine a user identifier associated with the matched palm image and perform identity recognition. If a match is not found, it may determine that identity recognition has failed.

[0061] In some embodiments, the server may obtain a palm image by decrypting the palm image encrypted data, extract palm features from the palm image, search a database for palm features that match the extracted palm features, and if a match is found, determine a user identifier associated with the matched palm feature and perform identity recognition.

[0062] In some embodiments, after the server performs identity recognition based on the palm image encryption data to determine the user identifier, the server may further verify the authorization corresponding to the user identifier. If the user identifier is verified to have authorization for a predetermined function, the server may further execute or trigger the predetermined function. The predetermined function may be, for example, access control or other functions that require user authorization verification.

[0063] Optionally, the manner in which the palm recognition device transmits the encrypted palm image data to the palm recognition server includes, but is not limited to, at least one of network transmission, data line transmission, and Bluetooth transmission, although the embodiments of the present application do not specifically limit this.

[0064] The palm recognition device may perform identity recognition by performing decryption and palm comparison recognition based on the encrypted palm image data. Specifically, the palm comparison recognition may be performed by comparing palm features in the palm image with predetermined palm features in a database.

[0065] The predetermined palm feature is a palm feature of the palm of a stored user identifier. Each predetermined palm feature has a corresponding user identifier, which indicates that the predetermined palm feature belongs to the user identifier and is a palm feature of the user's palm. The user identifier may be any user identifier. For example, the user identifier may be a user identifier registered with a payment application. Alternatively, the user identifier may be a user identifier registered with a company.

[0066] In an embodiment of the present application, the palm recognition server includes a database containing a plurality of predetermined palm-part features and user identifiers corresponding to each predetermined palm-part feature. In this database, the predetermined palm-part features and user identifiers may correspond one-to-one, or one user identifier may correspond to at least two predetermined palm-part features.

[0067] For example, multiple users can register with a payment application and associate each user's specific palm features with the corresponding user identifier, thereby storing the corresponding user identifiers in a database. When a user then uses the payment application, the application performs palm comparison recognition between the first palm region in the infrared image and the second palm region in the color image and the specific palm features in the database, thereby determining a target user identifier and verifying the user's identity.

[0068] For example, the palm recognition device transmits the palm image encrypted data to the palm recognition server for decryption and palm comparison recognition, thereby obtaining a user identifier corresponding to the palm image.

[0069] As described above, the method provided in this embodiment acquires a palm image and performs feature extraction to obtain image features at multiple scales corresponding to the palm image, performs feature fusion on the image features at multiple scales to obtain image fusion features, performs prediction based on the image fusion features to obtain a palm frame corresponding to the palm portion in the palm image, cuts out a palm image corresponding to the palm portion from the palm image based on the palm frame, encrypts the palm image to obtain encrypted palm image data, and transmits the encrypted palm image data to a palm recognition server for decryption and palm comparison recognition to obtain a user identifier corresponding to the palm image. In this application, by cutting out a palm image corresponding to the palm portion from the palm image and encrypting, transmitting, and decrypting only the palm image, the palm image recognition time is reduced and the recognition efficiency is improved.

[0070] 4 is a flowchart of a palm image processing method provided in one exemplary embodiment of the present application, which is executed by a palm recognition device. The method includes the following steps:

[0071] In step 402, a palm image is acquired, and image features at multiple scales are obtained by extracting features from the palm image using a backbone network in the palm detection model.

[0072] The palm image is a palm image for which a user identifier is to be determined.

[0073] For example, the palm recognition device captures a palm image by capturing an image of the user's actual palm, where the palm image includes the user's palm, and the palm may be the user's left palm or the user's right palm.

[0074] For example, the palm recognition device may be an Internet of Things (IoT) device that captures a palm image by photographing a user's left palm with a camera. The Internet of Things device may also be a store cash register device. For example, when a user makes a purchase at a store and makes a transaction, the user extends their palm toward the camera of a store payment terminal, and the store payment terminal captures a palm image by photographing the user's actual palm with the camera.

[0075] In some embodiments, the palm recognition device establishes a communication connection with another device and receives a palm image transmitted from the other device via the communication connection. For example, the palm recognition device may be a payment application server, and the other device may be a payment terminal. The payment terminal captures a palm image by photographing the user's physical palm, and then transmits the palm image to the payment application server via the communication connection between the payment terminal and the payment application server. This allows the payment application server to determine a user identifier for the palm image.

[0076] In some embodiments, the palm frame is predicted by a palm detection model, the palm detection model including a backbone network, and the step of obtaining image features at multiple scales by extracting features from the palm image includes: performing a slicing operation on the palm image using the backbone network to obtain slice images at multiple scales; and obtaining image features at multiple scales by extracting features from the slice images at multiple scales using the backbone network.

[0077] Here, the palm detection model is an artificial intelligence model, and may be a deep learning model or a neural network model. The palm detection model has at least the function of determining a palm frame that identifies the palm in a palm image based on image fusion features. The backbone network is a component in the palm detection model for extracting image features at multiple scales from the palm image. The slicing operation is an operation of dividing the palm image to generate an image at a smaller scale. The smaller scale means that the scale of the generated image is smaller than the scale of the palm image. The pixel points of the sliced ​​image are derived from the palm image.

[0078] The palm recognition device may acquire slice images at multiple scales by performing a slicing operation using the backbone network. The palm recognition device may acquire image features at multiple scales corresponding to the palm images by performing feature extraction on the slice images at multiple scales.

[0079] In some embodiments, the step of obtaining slice images at multiple scales by performing a slicing operation on the palm image using a backbone network includes the steps of: determining, using the backbone network, a slice image at a maximum scale among the multiple scales based on the palm image; and starting layer-by-layer downsampling using the backbone network, with the slice image at the maximum scale as a first layer, to obtain slice images at multiple scales including the slice images of the first layer.

[0080] The computer device may first determine a slice image at a maximum scale among the plurality of scales based on the palm image. The scale of the slice image at a maximum scale may be equal to or smaller than the scale of the palm image. The computer device may generate the slice image at a maximum scale among the plurality of scales by downsampling the palm image once.

[0081] Downsampling a palm image is the process of sampling at least every other pixel point in the palm image. By stitching the downsampled pixel points together in their order in the palm image, slice images of decreasing scale can be obtained.

[0082] If the number of scales among the multiple scales is N, the multiple scales are N scales. Since the N scales are all different, there is a maximum scale among the N scales. The slice image with the maximum scale is set as the first layer, and downsampling is performed layer by layer from the first layer, until the N-1th layer is downsampled to obtain the Nth layer slice image, and the nth layer is downsampled to obtain the n+1th layer slice image. n is a positive integer between 1 and N-1. Ultimately, a total of N slice images can be obtained from the first layer to the Nth layer.

[0083] In this embodiment, slice images at multiple scales are obtained by layer-by-layer downsampling, providing a new means for separating slice images at multiple scales from a palm image.

[0084] In some embodiments, obtaining slice images at multiple scales by performing a slicing operation on the palm image using a backbone network includes obtaining slice images at multiple scales by sampling pixel points in the palm image at multiple scales and stitching them together using the backbone network, where two adjacent pixel points in the slice images at different scales have different numbers of pixel points between them at the sampling locations of the palm image.

[0085] In this embodiment, the palm recognition device may sample pixel points from the palm image at each scale using a sampling interval at that scale, and stitch the sampled pixel points together according to their relative positions in the palm image to obtain a slice image at each scale. Here, the sampling interval is the number of pixel points at the interval, including the interval between pixel points in both row and column directions. Here, there is a negative correlation between the scale and the sampling interval, and the larger the scale, the smaller the sampling interval. The smaller the scale, the larger the sampling interval. The unsampled pixel points in the palm image may be concentrated in the image feature channel.

[0086] For example, a slice image can be obtained by sampling every other column in the palm image at the maximum scale among multiple scales, i.e., selecting every other pixel in the row direction and stitching the selected pixel points. The stitched slice image does not lose information, but by concentrating the size information of the palm image into a channel and performing a convolution operation on the obtained new picture, a downsampled feature map can be obtained without losing information.

[0087] For example, we can sample every third column of the palm image at the second largest scale among multiple scales, i.e., select every third pixel and stitch the selected pixel points to obtain a slice image. Although the stitched slice image does not lose information, we can obtain a downsampled feature map without losing information by concentrating the size information of the palm image into a channel and performing a convolution operation on the resulting new picture.

[0088] For example, the size of the original image is 640*640*3, and a feature map of 320*320*12 is obtained through the slicing operation, where 640*640 of the original image represents the width*height, and 3 of the original image represents the length of the feature vector corresponding to each pixel (also called the number of channels).

[0089] In step 404, the neck network of the palm detection model performs feature fusion on the image features at multiple scales to obtain image fusion features.

[0090] The palm frame is predicted by a palm detection model, the palm detection model including a neck network, and the step of fusing image features at multiple scales to obtain image fusion features includes the step of performing feature fusion on the image features at multiple scales using the neck network to obtain image fusion features.

[0091] Here, the neck network belongs to the palm detection model and is configured to acquire image fusion features by performing feature fusion on image features at multiple scales. The neck network is connected to the backbone network and receives the image features at multiple scales output from the backbone network.

[0092] Multi-scale image features refer to features obtained by performing feature extraction on images at multiple scales. Image features at different scales can express different information. Here, among multi-scale image features, the larger the scale of the image feature, the more positional information it expresses, and the smaller the scale of the image feature, the more semantic information it expresses.

[0093] Illustratively, the palm recognition device extracts features of different categories at multiple scales and different receptive fields, and fuses the image features at multiple scales to obtain image fusion features corresponding to the palm image.

[0094] In some embodiments, obtaining image fusion features by performing feature fusion on image features at multiple scales using a neck network includes obtaining image fusion features by fusing image features at multiple scales at the same scale using a neck network, where the fusion of image features at the same scale may be by calculating a sum or an average value.

[0095] By fusing image features of multiple scales, the palm recognition device can increase the detection accuracy of small targets, more accurately detect the palm frame, more accurately crop the palm image, further reduce the amount of palm image encrypted data, improve the efficiency of identity recognition, and ensure the security of identity recognition.

[0096] In some embodiments, the step of obtaining image fusion features by performing feature fusion on image features at multiple scales using a neck network includes the steps of: obtaining first intermediate features at multiple scales by performing feature fusion based on image features at multiple scales using a neck network; obtaining second intermediate features at multiple scales by performing feature fusion on the first intermediate features at multiple scales using a neck network; and obtaining image fusion features by performing feature fusion on the second intermediate features at multiple scales using a neck network.

[0097] Here, feature fusion at multiple scales refers to performing feature fusion at different scales. The first intermediate features at multiple scales may correspond one-to-one with the image features at multiple scales. The first intermediate features at multiple scales may correspond one-to-one with the second intermediate features at multiple scales. The multiple scales of the image features may match the multiple scales of the first intermediate features. The multiple scales of the first intermediate features may match the multiple scales of the second intermediate features. Performing feature fusion on the second intermediate features at multiple scales may involve unifying the second intermediate features at multiple scales to the same scale and performing fusion. This fusion may be performed by calculating a sum or an average value.

[0098] In this embodiment, feature fusion is performed at different scales and then finally fused into image fusion features, which can fully represent the fine features in the palm image, facilitate more accurate detection of the palm frame, more accurately crop the palm image, further reduce the amount of data in the encrypted palm image data, improve the efficiency of identity recognition, and ensure the security of identity recognition.

[0099] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the step of extracting first intermediate features at the multiple scales based on the image features at the multiple scales includes: determining a first intermediate feature at the first order in the normal order of the scales based on the image feature at the first order in the normal order of the scales; upsampling the first intermediate feature at the first order in the normal order of the scales, layer by layer, and fusing the upsampling result of the first intermediate feature at the nth order in the normal order of the scales with the image feature at the n+1th order in the normal order of the scales to obtain a first intermediate feature at the n+1th order in the normal order of the scales; and, once the first intermediate feature at the Nth layer is obtained, determining first intermediate features at the multiple scales from the first layer to the Nth layer, where n is a positive integer from 1 to N-1, and the normal order of the scales is from smallest to largest.

[0100] Here, the ascending order is called the normal order, and the normal order of scales is the ascending order of scales. Upsampling is a process of increasing the scale. Upsampling may target the scale of the next layer and obtain the features of the next layer by interpolating the neighboring feature elements in the features of each layer.

[0101] In some embodiments, determining the first intermediate feature ranked first in order of scale based on the image feature ranked first in order of scale includes performing at least one convolution on the image feature ranked first in order of scale to obtain the first intermediate feature ranked first in order of scale. In addition to the convolution, channel adjustment may be performed. After the channel adjustment, the number of channels of the first intermediate feature may be consistent.

[0102] In some embodiments, the scale of the upsampled result of the first intermediate feature at the nth position in the normal order of scale may be the same as the scale of the image feature at the n+1th position in the normal order of scale. Fusing the upsampled result of the first intermediate feature at the nth position in the normal order of scale with the image feature at the n+1th position in the normal order of scale may be performed by joining them together, or by calculating the sum or average value on the same scale.

[0103] For example, as shown in the schematic diagram of image feature fusion in FIG. 5, a palm recognition device performs feature extraction on a palm image 501 to obtain image features 502 at multiple scales corresponding to the palm image 501. The image features 502 include feature layers T1, T2, T3, and T4. After obtaining the image features 502, feature layer T4 is convolved once and channel-adjusted to obtain feature layer P4. Feature layer P4 is upsampled and combined with feature layer T3, followed by convolution and channel adjustment to obtain feature layer P3. Feature layer P3 is upsampled and combined with feature layer T2, followed by convolution and channel adjustment to obtain feature layer P2. Feature layer P2 is upsampled and combined with feature layer T1, followed by 1*1 convolution and channel adjustment to obtain feature layer P1.

[0104] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the step of obtaining second intermediate features of the multiple scales by performing feature fusion of the multiple scales based on the first intermediate features of the multiple scales includes: determining a first-ranked second intermediate feature in reverse order of scale based on a first-ranked first intermediate feature in reverse order of scale (the reverse order of scales is from largest to smallest); downsampling the first-ranked second intermediate feature in reverse order of scale, layer by layer, and fusing the downsampled result of the m-th-ranked second intermediate feature in reverse order of scale (m is an integer from 1 to N-1) with the m+1-th intermediate feature in reverse order of scale to obtain the m+1-th second intermediate feature in reverse order of scale; and determining second intermediate features of the multiple scales from the first layer to the Nth layer after obtaining the N-th layer second intermediate feature.

[0105] Here, the order of increasing is called the reverse order, and the reverse order of scale is the order of increasing scale. Downsampling is the process of reducing the scale. Downsampling may target the scale of the next layer and acquire the features of the next layer by adopting feature elements in the features of each layer at intervals. Here, the first intermediate feature can be called a semantic image feature. The second intermediate feature can be called a positional image feature. The semantic image feature expresses more semantic information, and the positional image feature expresses more positional information.

[0106] In some embodiments, determining the second intermediate feature ranked first in reverse scale based on the first intermediate feature ranked first in reverse scale includes performing at least one convolution on the first intermediate feature ranked first in reverse scale to obtain the second intermediate feature ranked first in reverse scale. In addition to the convolution, channel adjustment may be performed. After the channel adjustment, the number of channels of the first intermediate feature may be consistent.

[0107] The scale of the downsampled result of the second intermediate feature at the m-th rank in the reverse order of the scale may be the same as the scale of the first intermediate feature at the (m+1)-th rank in the reverse order of the scale. The fusion of the downsampled result of the second intermediate feature at the m-th rank in the reverse order of the scale and the first intermediate feature at the (m+1)-th rank in the reverse order of the scale may be achieved by joining them together, or by calculating the sum or average of both of them on the same scale.

[0108] For example, the palm recognition device may obtain feature layer N1 by convolving feature layer P1 once and adjusting the channels. The feature layer N1 is downsampled and combined with feature layer P2, followed by convolution and channel adjustment to obtain feature layer N2. The feature layer N2 is downsampled and combined with feature layer P3, followed by convolution and channel adjustment to obtain feature layer N3. The feature layer N3 is downsampled and combined with feature layer P4, followed by convolution and channel adjustment to obtain feature layer N4. The palm recognition device obtains image fusion features 503 by performing feature fusion on feature layers N1 to N4.

[0109] Optionally, the activation function in the convolution is a logistic activation function.

[0110] In some embodiments, the palm recognition device obtains first intermediate features by upsampling image features at multiple scales from a smaller scale to a larger scale, the palm recognition device obtains second intermediate features by downsampling image features at multiple scales from a larger scale to a smaller scale, and the palm recognition device obtains image fusion features by performing feature fusion of the first intermediate features and the second intermediate features.

[0111] In step 406, a palm detection model predictive network determines a palm frame that identifies the palm in the palm image based on image fusion features.

[0112] The palm frame refers to a mark that indicates the position of the palm in the palm image.

[0113] In some embodiments, the palm frame is determined by a palm detection model, which includes a predictive network, and the step of determining a palm frame for identifying the palm in the palm image based on the image fusion features includes the steps of: obtaining a plurality of mesh features of the image fusion features by mesh dividing the image fusion features; and obtaining the palm frame by inputting the plurality of mesh features into the predictive network and making predictions on the plurality of mesh features.

[0114] Here, the mesh features are meshed image fusion features obtained by dividing the image fusion features into meshes. For example, the palm recognition device may obtain multiple mesh features by dividing the image fusion features into meshes, and the palm recognition device may perform predictions based on each mesh feature using a prediction network to obtain prediction results corresponding to each mesh feature. The prediction results corresponding to each of the multiple mesh features may then be combined to determine the position of the palm frame in the palm image.

[0115] In some embodiments, the palm recognition device is equipped with an infrared camera, and the palm image processing method further includes the steps of acquiring an infrared image capturing the same palm as the palm image, where the infrared image refers to an image of the palm captured by the infrared camera based on infrared light, and recognizing a palm region from the infrared image. The step of cutting out the palm image from the palm image based on the palm frame includes the steps of determining an intersection between the palm frame in the palm image and the palm region in the infrared image, and cutting out the palm image from the palm image based on the intersection.

[0116] In this embodiment, the palm recognition device further includes an infrared camera, which captures an infrared image corresponding to the palm of the same physical hand, and performs region recognition on the infrared image to determine the palm region in the infrared image. The infrared image refers to an image of the palm captured by the infrared camera using infrared light.

[0117] In some embodiments, recognizing the palm region from the infrared image includes detecting webbing (also called inter-finger webbing) in the infrared image, and determining the palm region in the infrared image based on the webbing.

[0118] Illustratively, the palm recognition device detects webs in the infrared image and determines a palm region in the infrared image based on the webs.

[0119] As shown in the schematic diagram of webbing in the palm of the hand in FIG. 6, the webbing is the first webbing 601 between the index finger and the middle finger, or the second webbing 602 between the middle finger and the ring finger, or the third webbing 603 between the ring finger and the little finger.

[0120] Since the palm region in a palm image can be located in any region of the palm image, to enable determination of the location of the palm region in the palm image, flipper detection is performed on the palm image to obtain at least one flipper of the palm image, which subsequently enables determination of the palm region based on the at least one flipper.

[0121] In step 408, the palm image is cut out from the palm image based on the palm frame.

[0122] The palm image is the effective recognition area in the palm image, or the palm image refers to the area in the palm image where the palm is located, or the palm image refers to the area in the palm image that can be used for palm recognition.

[0123] In some embodiments, the palm recognition device acquires at least three webbings in the infrared image, connects the webbings in order to acquire a webbing connection line, and cuts out a palm region from the infrared image based on the webbing connection line.

[0124] In some embodiments, determining the intersection of the palm frame in the palm image and the palm region in the infrared image includes obtaining coordinate parameters of the palm frame in the palm image and obtaining coordinate parameters of the palm region in the infrared image; and determining the intersection of the palm frame and the palm region based on the coordinate parameters of the palm frame and the coordinate parameters of the palm region.

[0125] The palm recognition device acquires coordinate parameters of a palm frame in the palm image, acquires coordinate parameters of a palm region in the infrared image, and cuts out a palm image corresponding to the palm from the palm image based on the intersection of the coordinate parameters of the palm frame and the coordinate parameters of the palm region.

[0126] In step 410, image encryption is performed on the palm image to obtain palm image encrypted data, which is transmitted to a server and used by the server to perform identity recognition based on the palm image encrypted data.

[0127] The encrypted palm image data is sent to a palm recognition server, where it is decrypted and used to obtain a user identifier corresponding to the palm image by performing palm comparison and recognition. Palm comparison and recognition refers to the comparison of palm features in the palm image with predetermined palm features in a database.

[0128] As a biometric feature, palm images are unique and distinctive. Compared to face recognition, which is currently widely applied in fields such as identity verification, payment, access control, and boarding, palm images are unaffected by makeup, masks, sunglasses, etc., and can improve the accuracy of user identity verification. In some scenarios, such as infection prevention scenarios, where people need to wear masks to cover their mouths and noses, palm images can be a better option for identity verification.

[0129] Cross-device registration recognition is a very important capability for user experience. For two types of associated devices, a user can register on one type of device to associate the user's user identifier with the user's palm characteristics. The user can then perform identity verification on another type of device. Because there are significant differences in image style and image quality between mobile phones and Internet of Things devices, cross-device registration recognition allows a user to register on the mobile phone and then directly use the Internet of Things device, eliminating the need for the user to register on two types of devices. For example, a user can register on the mobile phone and then directly perform identity verification on a store's device, eliminating the need for the user to register on the store's device, thereby avoiding user information leakage.

[0130] As described above, the method provided in this embodiment acquires a palm image and performs feature extraction to obtain image features at multiple scales corresponding to the palm image, performs feature fusion on the image features at multiple scales to obtain image fusion features, performs prediction based on the image fusion features to obtain a palm frame corresponding to the palm portion in the palm image, cuts out a palm image corresponding to the palm portion from the palm image based on the palm frame, encrypts the palm image to obtain encrypted palm image data, and transmits the encrypted palm image data to a palm recognition server for decryption and palm comparison recognition to obtain a user identifier corresponding to the palm image. In this application, by cutting out a palm image corresponding to the palm portion from the palm image and encrypting, transmitting, and decrypting only the palm image, the palm image recognition time is reduced and the recognition efficiency is improved.

[0131] In the method provided in this embodiment, the palm image is sliced ​​and feature extracted using a backbone network to obtain image features at multiple scales, which allows obtaining features of different categories at multiple scales and with different receptive fields, thereby improving the accuracy of palm frame prediction.

[0132] In the method provided in this embodiment, a neck network is used to upsample image features at multiple scales from a small scale to a large scale to obtain first intermediate features, downsample image features at multiple scales from a large scale to a small scale to obtain second intermediate features, and perform feature fusion of the first and second intermediate features to obtain image fusion features. This method enables the fusion of image features at multiple scales, thereby increasing the accuracy of detecting small targets in palm images.

[0133] In the method provided in this embodiment, the image fusion features are divided into meshes to obtain meshed image fusion features, and a prediction network is used to make predictions based on the meshed image fusion features to obtain the palm frame, which improves the accuracy of palm frame prediction and further ensures the security of palm-based identity recognition.

[0134] The method provided in this embodiment acquires an infrared image corresponding to the same palm, performs region recognition on the infrared image to determine the palm region in the infrared image, and then cuts out the palm image corresponding to the palm from the palm image based on the intersection of the palm frame and the palm region in the infrared image. This method acquires the palm image corresponding to the palm using both the palm region and the palm frame in the infrared image, improving the accuracy of the palm image cutout and making the acquired palm image more accurate.

[0135] 7 is a schematic diagram of a palm image processing method provided in one exemplary embodiment of the present application, which is executed by a palm recognition device, and includes the following steps:

[0136] The palm recognition device acquires a palm image 701 corresponding to the palm of the hand, inputs the acquired palm image 701 into a backbone network 702, and performs feature extraction to acquire image features of multiple scales corresponding to the palm image 701.

[0137] The palm recognition device obtains image fusion features by performing feature fusion on image features at multiple scales using the neck network 703.

[0138] The palm recognition device obtains a palm frame corresponding to the palm in the palm image 701 by making a prediction based on the image fusion features using a prediction network 704. The palm recognition device cuts out a palm image 705 corresponding to the palm from the palm image 701 based on the palm frame.

[0139] The palm recognition device inputs a palm image 705 into an encryption network 706 to encrypt the image, thereby obtaining encrypted palm image data 707 .

[0140] The palm recognition device transmits encrypted palm image data 707 to a decryption network 709 in the palm recognition server via network transmission 708 to decrypt the image, and then performs palm comparison and recognition via a verification network 710 to obtain a user identifier 711 corresponding to the palm image 701.

[0141] As described above, the method provided in this embodiment cuts out a palm image corresponding to the palm of the hand from a palm image, and performs image encryption, transmission, and decryption on only the palm image, thereby reducing the time required to recognize the palm image and improving the efficiency of palm recognition.

[0142] 8 is a schematic diagram of a cross-device payment by palm image processing method provided in one exemplary embodiment of the present application, which involves a user terminal 801, a store terminal 803, and a payment application server 802.

[0143] Here, a payment application is installed on user terminal 801. User terminal 801 logs in to the payment application based on a user identifier and establishes a communication connection with payment application server 802. Via this communication connection, user terminal 801 and payment application server 802 can communicate with each other. A payment application is also installed on each of store terminals 803. Store terminal 803 logs in to the payment application based on a store identifier and establishes a communication connection with payment application server 802. Via this communication connection, store terminal 803 and payment application server 802 can communicate with each other.

[0144] This cross-device payment flow includes the following steps:

[0145] In step 1, the user holds the user terminal 801 at home and photographs the user's physical palm with the user terminal 801 to obtain an image of the user's palm, logs in to the payment application based on the user identifier, and sends a palm image registration request accompanied by the user identifier and palm image to the payment application server 802.

[0146] In step 2, when the payment application server 802 receives the palm image registration request sent from the user terminal 801, it performs feature extraction on the palm image to obtain palm features of the palm image, stores the palm features in association with the user identifier, and sends a palm image linking success notification to the user terminal 801.

[0147] Here, the payment application server 802 stores the palm feature and the user identifier in association with each other, and then sets the palm feature as a predetermined palm feature. Subsequently, the corresponding user identifier can be determined based on the stored predetermined palm feature.

[0148] In step 3, when the user terminal 801 receives the palm image linking success notification, it displays the palm image linking success notification to notify the user that the palm image and the user identifier have been linked.

[0149] Here, the user completes palm image registration through communication between his / her user terminal 801 and the payment application server 802. Subsequently, automatic payment can be realized using the palm image.

[0150] In step 4, when a user purchases an item at a store and makes a transaction, the store terminal 803 takes a photograph of the user's actual palm to obtain a palm image, logs in to the payment application based on the store identifier, and sends a payment request to the payment application server 802, which includes the store identifier, the amount spent, and the palm image.

[0151] In step 5, when the payment application server 802 receives the payment request, it performs palm comparison recognition on the palm image, determines the user identifier for the palm image, determines the account for that user identifier in the payment application, completes the transfer using that account, and once the transfer is complete, sends a payment completion notification to the store terminal 803.

[0152] Here, after the user registers a palm image using the user terminal 801, the user can directly make a payment with the palm of their hand at the store terminal 803, eliminating the need for the user to register a palm image at the store terminal 803. This achieves the effect of cross-device palm image recognition and improves convenience.

[0153] In step 6, when the store terminal 803 receives the payment completion notification, it displays the payment completion notification and notifies the user that the transaction of the item between the user and the store has been completed so that the user can take the item home.

[0154] In addition, in the above embodiment, the cross-device payment process is realized by the user terminal 801 and the store terminal 803, but the store terminal 803 may be replaced with a payment device on the bus, and the cross-device boarding payment configuration may be realized by following the above steps.

[0155] 9 is a schematic diagram of cross-device identity verification by palm image processing method provided in one exemplary embodiment of the present application, the method involving a user terminal 901, an access control device 903, and an access control server 902.

[0156] Here, the user terminal 901 establishes a communication connection with the access control server 902. Through this communication connection, the user terminal 901 and the access control server 902 can communicate with each other. The access control device 903 establishes a communication connection with the access control server 902. Through this communication connection, the access control device 903 and the access control server 902 can communicate with each other.

[0157] This cross-device identity verification flow includes the following steps:

[0158] In step 1, the user holds the user terminal 901 at home and photographs the palm of the user's own physical hand using the user terminal 901 to obtain an image of the user's palm, and then sends a palm registration request together with the user identifier and palm image to the access control server 902.

[0159] In step 2, when the access control server 902 receives the palm registration request sent from the user terminal 901, it extracts features from the palm image to acquire palm features from the palm image, stores the palm features in association with the user identifier, and sends a palm linking success notification to the user terminal 901.

[0160] Here, the access control server 902 may store the palm features and the user identifiers in association with each other, and then set the palm features as predetermined palm features. Subsequently, the corresponding user identifier can be determined based on the stored predetermined palm features.

[0161] In step 3, when the user terminal 901 receives the palm linking success notification, it displays the palm linking success notification to notify the user that the palm image and the user identifier have been linked.

[0162] Here, the user completes palm image registration through communication between his / her user terminal 901 and the access control server, and then the door can be automatically opened by the palm image.

[0163] In step 4, when the user returns home from an outing, the access control device 903 takes a photograph of the user's actual palm to obtain an image of the user's palm, and sends an identity verification request accompanied by the image of the palm to be verified to the access control server 902.

[0164] In step 5, the access control server 902 receives the identity verification request sent from the access control device 903, recognizes the palm image of the verification target, obtains the user identifier of the palm image, determines that the user is a registered user, and sends a verification success notification to the access control device 903.

[0165] In step 6, the access control device 903 receives the verification pass notification sent from the access control server 902, and controls the door to open and allow the user to enter the room based on the verification pass notification.

[0166] In the above embodiment, the user terminal 901 and the access control device 903 realize the process of cross-device identity verification.

[0167] As can be seen from the above cross-device identity verification scenario, whether it is the palm registration stage involving communication between the user terminal 901 and the access control server 902 or the palm image processing stage involving communication with the server via another terminal device, the user terminal 901 or another terminal device acquires a palm image, then cuts out and encrypts the palm region of the palm image, and transmits the palm image containing the palm region to the server, which then performs palm comparison and recognition. Furthermore, in the palm comparison and recognition stage, the access control server 902 compares the palm features with predetermined palm features to obtain a recognition result for the current user.

[0168] 10 is a schematic diagram of a palm image processing method provided in one exemplary embodiment of the present application, which is executed by a palm recognition device, and includes the following steps:

[0169] The palm recognition device acquires a palm image 1001 corresponding to the palm of the hand, and inputs the palm image 1001 to a focus adjustment layer 1002 in the backbone network. The focus adjustment layer 1002 performs a slicing operation on the palm image 1001 to acquire image features at multiple scales. The image features are input to a convolution layer 1003, where they are subjected to convolution and channel adjustment, and then input to a bottleneck layer 1004 for feature extraction. A pooling layer 1005 pools the image features.

[0170] For example, a palm image 1001 is input to the focus adjustment layer 1002, and the focus adjustment layer 1002 performs a slicing operation on the palm image 1001 to obtain image features at multiple scales, which are represented as (3, 64, 1, 1), the image features obtained by performing convolution in the convolution layer 1003 to adjust the channels are represented as (64, 128, 3, 2), the image features that have passed through the bottleneck layer 1004 are represented as (128, 128) * 3, the image features obtained by performing convolution in the convolution layer 1003 to adjust the channels are represented as (128, 256, 3, 2), and the image features that have passed through the bottleneck layer 1004 are represented as (256, 256) * 9. The image feature (256,256)*9 is convolved in the convolution layer 1003 to adjust the channel, and the resulting image feature is expressed as (256,512,3,2). The image feature after passing through the bottleneck layer 1004 is expressed as (512,512)*9. The image feature after passing through the bottleneck layer 1004 to adjust the channel is expressed as (512,1024,3,2) after convolution again in the convolution layer 1003. The image feature after image pooling in the pooling layer 1005 and then passing through the bottleneck layer 1004 is expressed as (1024,1024)*3. The image feature (1024,1024)*3 is convolved in the convolution layer 1003 to adjust the channel, resulting in an image feature (1024,512,1,1). This is then upsampled 1006 and spliced ​​1007 with the image feature (512,512)*9 to produce an image feature that passes through the bottleneck layer 1004, resulting in an image feature (1024,512)*3. The image feature (1024,512)*3 is convolved in the convolution layer 1003 to adjust the channel, resulting in an image feature (512,256,1,1). The image feature (512,256,1,1) is upsampled 1006 and spliced ​​1007 with the image feature (256,256)*9 to produce an image feature that passes through the bottleneck layer 1004, resulting in an image feature (512,256)*3. The image feature (512,256)*3 is convolved in the convolution layer 1003 to adjust the channel, resulting in an image feature (512,1024,3,2). The image feature (512,1024,3,2) and the image feature (512,256,1,1) are spliced ​​1007, and the image feature after passing through the bottleneck layer 1004 is represented as (512,512)*3.The image feature (512,512)*3 is convolved in the convolution layer 1003 to adjust the channel, resulting in an image feature represented as (512,512,3,2). The image feature represented as (512,512,3,2) is spliced ​​1007 with the image feature represented as (1024,512,1,1), and the image feature passed through the bottleneck layer 1004 is represented as (1024,1024)*3. The final output image features 1008 at multiple scales are (512,256), (512,512), and (1024,1024). Here, in (x,y,a,b), x and y can represent two dimensions of the feature matrix, respectively, and a and b can represent the number of channels and scale, respectively. For (x,y)*b, x and y can represent the two dimensions of the feature matrix, respectively, and b can represent the number of channels.For (x,y), x and y can represent the two dimensions of the feature matrix, respectively.

[0171] The palm recognition device acquires image fusion features by performing feature fusion on image features 1008 of multiple scales. The palm recognition device acquires a palm frame corresponding to the palm in palm image 1001 by making predictions on the image fusion features. The palm recognition device cuts out a palm image corresponding to the palm from palm image 1001 based on the palm frame. The palm recognition device transmits the encrypted palm image to a palm recognition server via a network, decrypts the image, and performs palm comparison recognition to acquire a user identifier corresponding to palm image 1001.

[0172] As described above, the method provided in this embodiment uses a backbone network to perform slicing and feature extraction on the palm image to obtain image features at multiple scales. This method obtains features of different categories at multiple scales and with different receptive fields, thereby improving the accuracy of palm frame prediction.

[0173] The palm frame prediction according to the present application may be realized based on a palm detection model. This configuration includes a palm detection model generation stage and a palm frame prediction stage. FIG. 11 is a framework diagram of the palm detection model generation stage and palm frame prediction stage shown in one exemplary embodiment of the present application. As shown in FIG. 11, in the palm detection model generation stage, a palm detection model generation device 1110 obtains a palm detection model using a preset training sample dataset (including a sample palm image and a sample palm image), and then generates a palm frame based on the palm detection model. In the palm frame prediction stage, a palm frame prediction processing device 1120 processes the input palm image based on the palm detection model to obtain a prediction result of the palm image, and determines, for example, the area and position of the palm in the palm image.

[0174] Here, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may be computer devices. For example, the computer devices may be stationary computer devices such as personal computers and servers. Alternatively, the computer devices may be portable computer devices such as tablet computers and e-book readers.

[0175] Optionally, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may be the same device. Alternatively, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may be different devices. Furthermore, when the palm detection model generation device 1110 and the palm frame prediction processing device 1120 are different devices, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may be the same type of device. For example, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may both be servers. Alternatively, the palm detection model generation device 1110 and the palm frame prediction processing device 1120 may be different types of devices. For example, the palm frame prediction processing device 1120 may be a personal computer or a terminal, while the palm detection model generation device 1110 may be a server or the like. The embodiments of the present application do not limit the specific types of the palm detection model generation device 1110 and the palm frame prediction processing device 1120.

[0176] In the above embodiment, a palm image processing method has been described. Next, a method for training a palm detection model will be described.

[0177] 12 is a flowchart of a method for training a palm detection model provided in one exemplary embodiment of the present application, which is executed by a palm recognition device. The method includes the following steps:

[0178] In step 1201, a sample palm image and a sample palm region image are acquired.

[0179] The sample palm image is an image for which a user identifier is to be determined. The sample palm image includes a palm, which is an image of the actual palm of the user whose identity is to be verified. The sample palm image may further include other information, such as the user's fingers and the location scenario of the user's palm when it was captured. The sample palm image may be captured by the palm recognition device of the actual palm of the user whose identity is to be verified, or may be transmitted from another device.

[0180] For example, the palm recognition device may be a store register device that captures a physical palm image of a user using a camera to obtain the sample palm image, or the palm recognition device may be a palm image recognition server that captures a sample palm image of a user and then transmits the sample palm image to the palm image recognition server.

[0181] The sample palm image points to an annotated valid recognition area in the sample palm image, or the sample palm image points to an annotated area in the sample palm image where the palm is located, or the sample palm image points to an annotated area in the sample palm image that can be used for recognition.

[0182] In step 1202, sample image features at multiple scales are obtained by extracting features for the sample palm image.

[0183] Here, the sample palm image is an image of a physical palm. The sample palm image includes an image of all or part of the physical palm. For ease of explanation, the image of the physical palm in the sample palm image will hereinafter be referred to as the palm. The palm included in the sample palm image may be the physical palm of a user whose identity is to be recognized. The palm in the sample palm image includes at least the palm region, and may also include fingers or a portion of the arm, and may include an image of the environment in which the physical palm is located when the physical palm is captured.

[0184] The scale is the size of the sample image feature and can be expressed as the number of dimensions of a matrix, for example, 100*100, 80*80, 60*60, or 40*40. The multiple scales refer to two or more scales, each different from the others. The maximum scale among the multiple scales may be the same as the scale of the sample palm image, but is generally smaller than the scale of the sample palm image.

[0185] The sample palm image may be an image of the physical palm of the user whose identity is to be verified captured by the palm recognition device, or may be transmitted from another device. The palm recognition device may acquire the sample palm image by directly collecting an image of the physical palm. The palm recognition device may acquire the sample palm image by collecting an image of the physical palm using an image collector connected to the palm recognition device. The palm recognition device may perform feature extraction on the acquired sample palm image to acquire sample image features at multiple scales corresponding to the sample palm image.

[0186] For example, the palm recognition device may be a store register device that captures a physical palm image of a user using a camera to obtain the sample palm image, or the palm recognition device may be a sample palm image recognition server that captures a sample palm image of a user and then transmits the sample palm image to the sample palm image recognition server.

[0187] For example, the sample image features at multiple scales may be extracted from the low-resolution images at multiple scales after downsampling the high-resolution sample palm image to the low-resolution images at multiple scales, respectively, where high resolution and low resolution are relative concepts, and high resolution is higher than low resolution.

[0188] For example, the size of the input sample palm image is 640*640, and the sizes of the output sample image features at multiple scales are 80*80, 40*40, and 20*20.

[0189] In step 1203, sample image fusion features are obtained by fusing sample image features at multiple scales.

[0190] The palm recognition device can obtain sample image fusion features by performing feature fusion on sample image features at multiple scales.

[0191] The multi-scale sample image features refer to features obtained by performing feature extraction on images at multiple scales. The sample image features at different scales can express different information. Here, among the multi-scale sample image features, the larger the scale of the sample image features, the more positional information they express, and the smaller the scale of the sample image features, the more semantic information they express.

[0192] Illustratively, the palm recognition device extracts features of different categories at multiple scales and different receptive fields, and fuses the sample image features at multiple scales to obtain sample image fusion features corresponding to the sample palm image.

[0193] In step 1204, a sample predicted palm outline is determined based on the sample image fusion features to identify the palm in the sample palm image.

[0194] The sample predicted palm frame refers to a mark for indicating the location of the palm in the sample palm image. The sample predicted palm frame includes the palm in the sample palm image. The boundary of the sample predicted palm frame surrounds the palm in the sample palm image.

[0195] Optionally, the shape of the sample predicted palm frame may be a regular shape, for example, at least one of a rectangle, a square, a diamond, a hexagon, a circle, or a triangle. The shape of the sample predicted palm frame may be an irregular shape. A regular shape is a shape that can be drawn with fixed parameters and based on fixed rules. An irregular shape cannot be drawn with fixed parameters and based on fixed rules, and generally can only be represented by a set of points that make up the irregular shape.

[0196] The palm recognition device may perform prediction based on the sample image fusion features in an artificial intelligence manner to determine a sample predicted palm outline that identifies the palm in the sample palm image.

[0197] In some embodiments, a computing device may determine, based on the sample image fusion features, the position of a sample predicted palm frame in the sample palm image, which identifies the palm in the sample palm image. If the sample predicted palm frame is a regular shape, its position can be represented by the values ​​of fixed parameters required to draw the regular shape, such as the position of a fixed point of the sample predicted palm frame and a size parameter of the sample predicted palm frame. The fixed point is a point that is fixed relative to the position of the sample predicted palm frame. The size parameter is a parameter that represents the size of the sample predicted palm frame. For example, if the sample predicted palm frame is a rectangle, the fixed point may be any corner point or the center point of the rectangle, and the size parameter is the side length of the adjacent side of the sample predicted palm frame.

[0198] In step 1205, a sample predicted palm image is cut out from the sample palm image based on the sample predicted palm frame.

[0199] The sample predicted palm image is a valid recognition area in the sample palm image, or the sample predicted palm image refers to an area in the sample palm image where the palm is located, or the sample predicted palm image refers to an area in the sample palm image that can be used for palm recognition.

[0200] For example, the palm recognition device cuts out an image area in the sample palm image that is outside the sample predicted palm frame based on the position of the sample predicted palm frame, and sets the remaining image as the sample predicted palm image.

[0201] In step 1206, a loss function value is calculated based on the sample palm image and the sample predicted palm image.

[0202] Illustratively, the palm recognition device calculates a loss function value based on the sample palm image and the sample predicted palm image.

[0203] In step 1207, the model parameters of the palm detection model are updated based on the loss function value.

[0204] Illustratively, the palm recognition device updates the model parameters of the palm detection model based on the loss function value.

[0205] The model parameter update refers to, but is not limited to, updating the network parameters in the palm detection model, updating the network parameters of each network module in the model, or updating the network parameters of each network layer in the model, although the embodiments of the present application are not limited thereto.

[0206] The model parameters of the palm detection model include at least one of a network parameter of a backbone network, a network parameter of a neck network, and a network parameter of a prediction network in the palm detection model.

[0207] As described above, the method provided in this embodiment performs feature extraction on an acquired sample palm image to obtain sample image features at multiple scales corresponding to the sample palm image, performs feature fusion on the sample image features at multiple scales to obtain sample image fusion features, performs prediction based on the sample image fusion features to obtain a sample predicted palm frame for identifying the palm area in the sample palm image, cuts out a sample predicted palm image corresponding to the palm area from the sample palm image based on the sample predicted palm frame, calculates a loss function value based on the sample palm image and the sample predicted palm image, and updates the model parameters of the palm detection model based on the loss function value. As a result, the trained palm detection model can achieve higher prediction accuracy for the sample predicted palm frame and obtain a more accurate sample predicted palm frame.

[0208] 13 is a flowchart of a method for training a palm detection model provided in one exemplary embodiment of the present application. The method is performed by a palm recognition device. The method includes the following steps:

[0209] In step 1301, a sample palm image and a sample palm region image are acquired.

[0210] The sample palm image is an image for which a user identifier is to be determined. The sample palm image includes a palm, which is an image of the actual palm of the user whose identity is to be verified. The sample palm image may also include other information, such as the user's fingers and the location scenario of the user's palm when it is captured. The sample palm image may be captured by the palm recognition device of the actual palm of the user whose identity is to be verified, or may be transmitted from another device.

[0211] The sample palm image points to an annotated valid recognition area in the sample palm image, or the sample palm image points to an annotated area in the sample palm image where the palm is located, or the sample palm image points to an annotated area in the sample palm image that can be used for recognition.

[0212] For example, a sample palm image collection method involves acquiring 50 to 100 sample palm images. The sample palm images are then cleaned up by searching for duplicates, missing values, and outliers to obtain valid sample palm images. The code for cleaning up the sample palm images can be expressed as follows:

[0213] [Table 1] Sample palm image collection distance requirements: Place the palm at 5cm, 8cm, 10cm, 12cm, and 15cm from the palm recognition device to collect images, and collect images for 10 to 15 seconds at each distance.

[0214] Acquisition method of sample palm image: Annotate the palm area in the sample palm image using the annotation tool Labeling. The code for annotating the palm area can be expressed as follows:

[0215] [Table 2] In the above code, 45 represents the minimum value of the palm area on the x-axis, 104 represents the maximum value of the palm area on the x-axis, 34 represents the minimum value of the palm area on the y-axis, and 84 represents the maximum value of the palm area on the y-axis.

[0216] The annotation tool, Labeling, combines the uniform distribution theorem (also known as the uniform distribution theorem) with the label vector y hot Instead of

number

[0217]

number

[0218] Label Vector

number

[0219]

number

number

[0220] In step 1302, sample image features at multiple scales are obtained by extracting features from the sample palm image using the backbone network in the palm detection model.

[0221] Illustratively, the palm recognition device acquires a sample palm image by capturing an image of a user's actual palm, where the sample palm image includes the palm, which may be the user's left palm or the user's right palm.

[0222] For example, the palm recognition device may be an Internet of Things device that captures a sample palm image by using a camera to capture a user's left palm. The Internet of Things device may also be a store cash register. For example, when a user makes a purchase at a store and makes a transaction, the user holds out their palm to the camera of a store payment terminal, and the store payment terminal captures a physical image of the user's palm using the camera to capture the sample palm image.

[0223] In some embodiments, the palm recognition device establishes a communication connection with another device and receives a sample palm image transmitted from the other device via the communication connection. For example, the palm recognition device may be a payment application server, and the other device may be a payment terminal. The payment terminal acquires the sample palm image by photographing the user's physical palm, and then transmits the sample palm image to the payment application server via the communication connection between the payment terminal and the payment application server. This allows the payment application server to determine a user identifier for the sample palm image.

[0224] In some embodiments, the sample predicted palm frame is predicted by a palm detection model, and the palm detection model includes a backbone network. The step of obtaining sample image features at multiple scales by extracting features from the sample palm image includes: inputting the sample palm image into the backbone network; performing a slicing operation on the sample palm image using the backbone network to obtain slice images at multiple scales; and obtaining sample image features at multiple scales by extracting features from the slice images at the multiple scales using the backbone network.

[0225] Here, the palm detection model is an artificial intelligence model, and may be a deep learning model or a neural network model. The palm detection model has at least the function of determining a sample predicted palm frame that identifies the palm in the sample palm image based on the sample image fusion features. The backbone network is a component in the palm detection model for extracting sample image features at multiple scales from the sample palm image. The slicing operation is an operation of dividing the sample palm image to generate an image at a smaller scale. The smaller scale means that the scale of the generated image is smaller than that of the sample palm image. The pixel points of the sliced ​​image are derived from the sample palm image.

[0226] The palm recognition device may input a sample palm image to a backbone network and perform a slicing operation using the backbone network to obtain slice images at multiple scales. The palm recognition device may also perform feature extraction on the slice images at multiple scales to obtain sample image features at multiple scales corresponding to the sample palm image.

[0227] In some embodiments, the step of obtaining slice images at multiple scales by performing a slicing operation on the sample palm image using a backbone network includes the steps of: determining, using the backbone network, a slice image at a maximum scale among the multiple scales based on the sample palm image; and starting layer-by-layer downsampling using the backbone network with the slice image at the maximum scale as a first layer to obtain slice images at multiple scales including the slice images of the first layer.

[0228] The computing device may first determine a slice image at a maximum scale among the plurality of scales based on the sample palm image. The scale of the slice image at a maximum scale may be equal to or less than the scale of the sample palm image. The computing device may generate the slice image at a maximum scale among the plurality of scales by downsampling the sample palm image once.

[0229] Downsampling a sample palm image is the process of sampling at least every other pixel point in the sample palm image. By stitching the downsampled pixel points together in their order in the sample palm image, slice images of decreasing scale can be obtained.

[0230] If the number of scales among the multiple scales is N, the multiple scales are N scales. Since the N scales are all different, there is a maximum scale among the N scales. The slice image with the maximum scale is set as the first layer, and downsampling is performed layer by layer from the first layer, until the N-1th layer is downsampled to obtain the Nth layer slice image, and the nth layer is downsampled to obtain the n+1th layer slice image. n is a positive integer between 1 and N-1. Ultimately, a total of N slice images can be obtained from the first layer to the Nth layer.

[0231] In this embodiment, multi-scale slice images are obtained by layer-wise downsampling, and a new means is provided to segment multi-scale slice images from a sample palm image.

[0232] In some embodiments, obtaining slice images at multiple scales by performing a slicing operation on the sample palm image using a backbone network includes obtaining slice images at multiple scales by sampling pixel points in the sample palm image at multiple scales and stitching them together using a backbone network, wherein two adjacent pixel points in the slice images at different scales have different numbers of pixel points between them at the sampling locations of the sample palm image.

[0233] In this embodiment, the palm recognition device may sample pixel points from the sample palm image at each scale using a sampling interval at that scale, and stitch the sampled pixel points together according to their relative positions in the sample palm image to obtain a slice image at each scale. Here, the sampling interval is the number of pixel points in the interval, including the interval between pixel points in both row and column directions. Here, there is a negative correlation between the scale and the sampling interval, and the larger the scale, the smaller the sampling interval. The smaller the scale, the larger the sampling interval. The unsampled pixel points in the sample palm image may be concentrated in a channel of the sample image feature.

[0234] For example, we can obtain a slice image by sampling every other column in the sample palm image at the maximum scale among multiple scales, i.e., selecting every other pixel in the row direction and stitching the selected pixel points. The stitched slice image does not lose information, but by concentrating the size information of the sample palm image into a channel and performing a convolution operation on the obtained new picture, we can obtain a downsampled feature map without losing information.

[0235] Alternatively, for example, every third column of the sample palm image can be sampled at the second largest scale among the multiple scales, i.e., every third pixel can be selected and stitched together to obtain a slice image. The stitched slice image does not lose any information, but by concentrating the size information of the sample palm image into a channel and performing a convolution operation on the resulting new picture, a downsampled feature map can be obtained without losing any information.

[0236] For example, the size of the original image is 640*640*3, and a feature map of 320*320*12 is obtained through the slicing operation, where 640*640 of the original image represents the width*height, and 3 of the original image represents the length of the feature vector corresponding to each pixel (also called the number of channels).

[0237] In step 1303, the neck network of the palm detection model performs feature fusion on the sample image features at multiple scales to obtain sample image fusion features.

[0238] The sample predicted palm frame is predicted by a palm detection model, the palm detection model including a neck network, and the step of fusing sample image features at multiple scales to obtain sample image fusion features includes the step of performing feature fusion on the sample image features at multiple scales using the neck network to obtain sample image fusion features.

[0239] Here, the neck network belongs to the palm detection model and is configured to acquire sample image fusion features by performing feature fusion on sample image features at multiple scales. The neck network is connected to the backbone network and receives the sample image features at multiple scales output from the backbone network.

[0240] The multi-scale sample image features refer to features obtained by performing feature extraction on images at multiple scales. The sample image features at different scales can express different information. Here, among the multi-scale sample image features, the larger the scale of the sample image features, the more positional information they express, and the smaller the scale of the sample image features, the more semantic information they express.

[0241] Illustratively, the palm recognition device extracts features of different categories at multiple scales and different receptive fields, and fuses the sample image features at multiple scales to obtain sample image fusion features corresponding to the sample palm image.

[0242] In some embodiments, the step of obtaining sample image fusion features by performing feature fusion on sample image features at multiple scales using a neck network includes the step of obtaining sample image fusion features by fusing sample image features at multiple scales at the same scale using a neck network, where the fusion of sample image features at the same scale may be by calculating a sum or an average value.

[0243] By fusing sample image features of multiple scales, the palm recognition device can increase the detection accuracy of small targets, more accurately detect the sample predicted palm frame, more accurately crop the sample predicted palm image, further reduce the amount of encrypted data of the sample predicted palm image, improve the efficiency of identity recognition, and ensure the security of identity recognition.

[0244] In some embodiments, the step of obtaining sample image fusion features by performing feature fusion on sample image features at multiple scales using a neck network includes the steps of: obtaining first intermediate features at multiple scales by performing feature fusion based on the sample image features at multiple scales using a neck network; obtaining second intermediate features at multiple scales by performing feature fusion on the first intermediate features at multiple scales using a neck network; and obtaining sample image fusion features by performing feature fusion on the second intermediate features at multiple scales using a neck network.

[0245] Here, feature fusion at multiple scales refers to performing feature fusion at different scales. The first intermediate features at multiple scales may correspond one-to-one with the sample image features at multiple scales. The first intermediate features at multiple scales may correspond one-to-one with the second intermediate features at multiple scales. The multiple scales of the sample image features may match the multiple scales of the first intermediate features. The multiple scales of the first intermediate features may match the multiple scales of the second intermediate features. Performing feature fusion on the second intermediate features at multiple scales may involve unifying the second intermediate features at multiple scales to the same scale and performing fusion. This fusion may be performed by calculating a sum or an average value.

[0246] In this embodiment, feature fusion is performed at different scales and then finally fused into the sample image fusion feature, which can fully express the detailed features in the sample palm image, facilitate more accurate detection of the sample predicted palm frame, more accurately crop the sample predicted palm image, further reduce the data volume of the sample predicted palm image encrypted data, improve the efficiency of identity recognition, and ensure the security of identity recognition.

[0247] In some embodiments, the plurality of scales is N scales, where N is a positive integer greater than 1, and the step of extracting first intermediate features at the plurality of scales based on the sample image features at the plurality of scales includes: determining a first intermediate feature at the first order in normal order of scales based on the sample image feature at the first order in normal order of scales; upsampling the first intermediate feature at the first order in normal order of scales layer by layer, and fusing the upsampling result of the first intermediate feature at the nth order in normal order of scales with the sample image feature at the n+1th order in normal order of scales to obtain a first intermediate feature at the n+1th order in normal order of scales; and, once the first intermediate feature at the Nth layer is obtained, determining first intermediate features at the plurality of scales from the first layer to the Nth layer, where n is a positive integer from 1 to N-1, and the normal order of scales is from smallest to largest.

[0248] Here, the ascending order is called the normal order, and the normal order of scales is the ascending order of scales. Upsampling is a process of increasing the scale. Upsampling may target the scale of the next layer and obtain the features of the next layer by interpolating the neighboring feature elements in the features of each layer.

[0249] In some embodiments, determining the first intermediate feature ranked first in order of scale based on the sample image feature ranked first in order of scale includes performing at least one convolution on the sample image feature ranked first in order of scale to obtain the first intermediate feature ranked first in order of scale. In addition to the convolution, channel adjustment may be performed. After the channel adjustment, the number of channels of the first intermediate feature may be consistent.

[0250] In some embodiments, the scale of the upsampled result of the first intermediate feature at the nth position in the normal order of scale may be the same as the scale of the sample image feature at the (n+1)th position in the normal order of scale. Fusing the upsampled result of the first intermediate feature at the nth position in the normal order of scale with the sample image feature at the (n+1)th position in the normal order of scale may involve joining them together, or calculating the sum or average of both at the same scale.

[0251] For example, as shown in the schematic diagram of sample image feature fusion in FIG. 5, a palm recognition device performs feature extraction on a sample palm image 501 to obtain sample image features 502 at multiple scales corresponding to the sample palm image 501. The sample image features 502 include feature layers T1, T2, T3, and T4. After obtaining the sample image features 502, feature layer T4 is convolved once and channel-adjusted to obtain feature layer P4. Feature layer P4 is upsampled and combined with feature layer T3, followed by convolution and channel adjustment to obtain feature layer P3. Feature layer P3 is upsampled and combined with feature layer T2, followed by convolution and channel adjustment to obtain feature layer P2. Feature layer P2 is upsampled and combined with feature layer T1, followed by 1*1 convolution and channel adjustment to obtain feature layer P1.

[0252] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the step of obtaining second intermediate features of the multiple scales by performing feature fusion of the multiple scales based on the first intermediate features of the multiple scales includes: determining a first-ranked second intermediate feature in reverse order of scale based on a first-ranked first intermediate feature in reverse order of scale (the reverse order of scales is from largest to smallest); downsampling the first-ranked second intermediate feature in reverse order of scale, layer by layer, and fusing the downsampled result of the m-th-ranked second intermediate feature in reverse order of scale (m is an integer from 1 to N-1) with the m+1-th intermediate feature in reverse order of scale to obtain the m+1-th second intermediate feature in reverse order of scale; and determining second intermediate features of the multiple scales from the first layer to the Nth layer after obtaining the N-th layer second intermediate feature.

[0253] Here, the order of increasing magnitude is called the reverse order, and the reverse order of scales is the order of increasing magnitude. Downsampling is the process of reducing the scale. Downsampling may target the scale of the next layer and acquire the features of the next layer by adopting feature elements in the features of each layer at intervals. Here, the first intermediate features can be called semantic sample image features. The second intermediate features can be called positional sample image features. The semantic sample image features express more semantic information, and the positional sample image features express more positional information.

[0254] In some embodiments, determining the second intermediate feature ranked first in reverse scale based on the first intermediate feature ranked first in reverse scale includes performing at least one convolution on the first intermediate feature ranked first in reverse scale to obtain the second intermediate feature ranked first in reverse scale. In addition to the convolution, channel adjustment may be performed. After the channel adjustment, the number of channels of the first intermediate feature may be consistent.

[0255] The scale of the downsampled result of the second intermediate feature at the m-th rank in the reverse order of the scale may be the same as the scale of the first intermediate feature at the (m+1)-th rank in the reverse order of the scale. The fusion of the downsampled result of the second intermediate feature at the m-th rank in the reverse order of the scale and the first intermediate feature at the (m+1)-th rank in the reverse order of the scale may be achieved by joining them together, or by calculating the sum or average of both of them on the same scale.

[0256] For example, the palm recognition device may obtain feature layer N1 by convolving feature layer P1 once and adjusting the channels. The feature layer N1 is downsampled and combined with feature layer P2, followed by convolution and channel adjustment to obtain feature layer N2. The feature layer N2 is downsampled and combined with feature layer P3, followed by convolution and channel adjustment to obtain feature layer N3. The feature layer N3 is downsampled and combined with feature layer P4, followed by convolution and channel adjustment to obtain feature layer N4. The palm recognition device obtains sample image fusion features 503 by performing feature fusion on feature layers N1 to N4.

[0257] Optionally, the activation function in the convolution is a logistic activation function.

[0258] In some embodiments, the palm recognition device obtains first intermediate features by upsampling sample image features at multiple scales from a smaller scale to a larger scale, the palm recognition device obtains second intermediate features by downsampling sample image features at multiple scales from a larger scale to a smaller scale, and the palm recognition device obtains sample image fusion features by performing feature fusion of the first intermediate features and the second intermediate features.

[0259] In step 1304, a sample predicted palm frame is determined by the predictive network of the palm detection model based on the sample image fusion features to identify the palm in the sample palm image.

[0260] The sample predicted palm frame refers to a mark that indicates the position where the palm is located in the sample palm image.

[0261] In some embodiments, the sample predicted palm outline is determined by a palm detection model, which includes a prediction network, and determining the sample predicted palm outline for identifying the palm in the sample palm image based on the sample image fusion features includes: mesh-dividing the sample image fusion features to obtain a plurality of mesh features of the sample image fusion features; and inputting the plurality of mesh features into the prediction network and performing predictions on the plurality of mesh features to obtain the sample predicted palm outline.

[0262] For example, the palm recognition device divides the sample image fusion features into meshes to obtain meshed sample image fusion features, and the palm recognition device inputs the meshed sample image fusion features into a prediction network, which then performs prediction for each mesh to obtain a sample predicted palm frame. The step of obtaining a sample predicted palm frame by performing prediction for multiple mesh features may specifically involve combining prediction results of multiple mesh features to determine the position of the sample predicted palm frame.

[0263] In some embodiments, the palm recognition device is equipped with an infrared camera, and the sample palm image processing method further includes acquiring an infrared image capturing the same palm area as the sample palm image, where the infrared image refers to an image of the palm captured by the infrared camera based on infrared light, and recognizing a palm area from the infrared image.Cropping the sample predicted palm image from the sample palm image based on the sample predicted palm frame includes determining an intersection between the sample predicted palm frame in the sample palm image and the palm area in the infrared image, and cropping the sample predicted palm image from the sample palm image based on the intersection.

[0264] In this embodiment, the palm recognition device is equipped with an infrared camera, and the palm recognition device acquires an infrared image corresponding to the palm of the same physical hand, and performs region recognition on the infrared image to determine the palm region in the infrared image. The infrared image refers to an image of the palm captured by the infrared camera using infrared light.

[0265] In some embodiments, recognizing the palm region from the infrared image includes detecting flippers in the infrared image, and determining the palm region in the infrared image based on the flippers.

[0266] Illustratively, the palm recognition device detects webs in the infrared image and determines a palm region in the infrared image based on the webs.

[0267] As shown in the schematic diagram of webbing in the palm of the hand in FIG. 6, the webbing is the first webbing 601 between the index finger and the middle finger, or the second webbing 602 between the middle finger and the ring finger, or the third webbing 603 between the ring finger and the little finger.

[0268] Since the palm region in the sample palm image may be located in any region in the sample palm image, to enable determination of the location of the palm region in the sample palm image, web detection is performed on the sample palm image to obtain at least one web of the sample palm image, which then enables determination of the palm region based on the at least one web.

[0269] In step 1305, a sample predicted palm image is cut out from the sample palm image based on the sample predicted palm frame.

[0270] The sample predicted palm image is a predicted valid recognition area in the sample palm image, or the sample predicted palm image refers to a region in the sample palm image where a predicted palm is located, or the sample predicted palm image refers to a predicted region in the sample palm image that can be used for recognition.

[0271] In some embodiments, the palm recognition device acquires at least three webbings in the infrared image, connects the webbings in order to acquire a webbing connection line, and cuts out a palm region from the infrared image based on the webbing connection line.

[0272] In some embodiments, determining the intersection of the sample predicted palm frame in the sample palm image and the palm region in the infrared image includes obtaining coordinate parameters of the sample predicted palm frame in the sample palm image and obtaining coordinate parameters of the palm region in the infrared image; and determining the intersection of the sample predicted palm frame and the palm region based on the coordinate parameters of the sample predicted palm frame and the coordinate parameters of the palm region.

[0273] The palm recognition device obtains coordinate parameters of a sample predicted palm frame in the sample palm image, obtains coordinate parameters of a palm region in the infrared image, and cuts out a sample predicted palm image corresponding to the palm from the sample palm image based on the intersection of the coordinate parameters of the sample predicted palm frame and the coordinate parameters of the palm region.

[0274] In step 1306, a loss function value is calculated based on the sample palm image and the sample predicted palm image.

[0275] Illustratively, the palm recognition device calculates a loss function value based on the sample palm image and the sample predicted palm image.

[0276] In step 1307, the model parameters of the palm detection model are updated based on the loss function value.

[0277] The model parameter update refers to, but is not limited to, updating the network parameters in the palm detection model, updating the network parameters of each network module in the model, or updating the network parameters of each network layer in the model, although the embodiments of the present application are not limited thereto.

[0278] The model parameters of the palm detection model include at least one of a network parameter of a backbone network, a network parameter of a neck network, and a network parameter of a prediction network.

[0279] In some embodiments, updating the model parameters of the palm detection model includes updating the network parameters of all network modules in the palm detection model, or fixing the network parameters of some network modules in the palm detection model and updating only the network parameters of the remaining network modules. For example, when updating the model parameters of the palm detection model, fixing the network parameters of the backbone network in the palm detection model and updating only the network parameters of the neck network and the prediction network.

[0280] Based on the loss function value, the network parameters of the backbone network, the network parameters of the neck network, and the network parameters of the prediction network in the palm detection model are updated using the loss function value as a training index until the loss function value converges, thereby obtaining a trained palm detection model.

[0281] The loss function value converges when the loss function value becomes constant, or when the difference in error between two adjacent iterations during training of the palm detection model is smaller than a predetermined value, or when the palm detection model has been trained a predetermined number of times, but this does not limit the scope of the present application.

[0282] Optionally, the target condition to be met by training may be that the number of training iterations of the initial model reaches a target number. The number of training iterations can be preset by an engineer. Alternatively, the target condition to be met by training may be, but is not limited to, that the loss value meets a target threshold condition. The embodiments of the present application are not limited thereto.

[0283] As described above, the method provided in this embodiment performs feature extraction on an acquired sample palm image to obtain sample image features at multiple scales corresponding to the sample palm image, performs feature fusion on the sample image features at multiple scales to obtain sample image fusion features, performs prediction based on the sample image fusion features to obtain a sample predicted palm frame for identifying the palm area in the sample palm image, cuts out a sample predicted palm image corresponding to the palm area from the sample palm image based on the sample predicted palm frame, calculates a loss function value based on the sample palm image and the sample predicted palm image, and updates the model parameters of the palm detection model based on the loss function value. As a result, the trained palm detection model can achieve higher prediction accuracy for the sample predicted palm frame and obtain a more accurate sample predicted palm frame.

[0284] Schematically, application scenarios of the palm image processing method provided in the embodiments of the present application include, but are not limited to, the following scenarios:

[0285] For example, in a palm-image recognition payment scenario, The store's palm recognition device captures a palm image of the user's actual palm, and the palm image processing method provided in the embodiments of the present application determines a target user identifier for the palm image. A portion of the resources in the resource account corresponding to the target user identifier are transferred to the store's resource account, thereby realizing automatic payment via the palm.

[0286] Also, for example, in cross-device payment scenarios, A user can complete identity registration using their personal mobile phone at home or other private location, linking their account to their palm image, after which an in-store device can recognize the user's palm image to determine their account and make payments directly through that account.

[0287] Also, for example, in a scenario where you punch in your commuter time card, The palm recognition device captures a palm image of the user by photographing the user's actual palm, and the palm image processing method provided in the embodiments of the present application determines a target user identifier from the palm image, creates a time card punch mark for the target user identifier, and determines that the target user identifier has completed punching in the commute time card at the current time.

[0288] Of course, in addition to the above scenarios, the method provided in the embodiments of the present application can also be applied to other scenarios where palm image processing is required, and the embodiments of the present application do not limit the specific application scenarios.

[0289] 14 shows a schematic diagram of the configuration of a palm image processing device provided in one exemplary embodiment of the present application. This device may be realized as all or part of a palm recognition device by software, hardware, or a combination of both. a feature extraction module 1401 that acquires a palm image and extracts features from the palm image to acquire image features at multiple scales; a feature fusion module 1402 for fusing image features at multiple scales to obtain image fusion features; a prediction module 1403 for determining a palm frame for identifying the palm in the palm image based on the image fusion features; a cropping module 1404 that crops a palm image from a palm image based on a palm frame; an encryption module 1405 that performs image encryption on the palm image to obtain palm image encrypted data, the palm image encrypted data being transmitted to a server and used by the server to perform identity verification based on the palm image encrypted data.

[0290] Here, the feature extraction module 1401 may be a backbone network of the palm detection model, the feature fusion module 1402 may be a neck network of the palm detection model, the prediction module 1403 may be at least a part of the prediction network of the palm detection model, and the cutout module 1404 may be a part of the prediction network, an independent module within the palm detection model, or an independent module outside the palm detection model.

[0291] In some embodiments, the feature extraction module 1401 is a backbone network of a palm detection model, which performs a slicing operation on the palm image to obtain slice images at multiple scales, and extracts features from the slice images at multiple scales to obtain image features at multiple scales.

[0292] In some embodiments, the feature extraction module 1401 is a backbone network of a palm detection model, which determines a slice image at a maximum scale among multiple scales based on the palm image, and starts layer-by-layer downsampling with the slice image at the maximum scale as the first layer to obtain slice images at multiple scales including the slice image at the first layer.

[0293] In some embodiments, the feature extraction module 1401 is a backbone network of a palm detection model, which samples pixel points in the palm image at multiple scales and stitches them together to obtain slice images at multiple scales, where two adjacent pixel points in the slice images at different scales have different numbers of pixel points between them at the sampling locations of the palm image.

[0294] In some embodiments, the feature fusion module 1402 is a neck network of palm detection models, which performs feature fusion on image features at multiple scales to obtain image fusion features.

[0295] In some embodiments, the feature fusion module 1402 is a neck network of a palm detection model, and performs feature fusion based on image features at multiple scales to obtain first intermediate features at multiple scales, performs feature fusion based on the first intermediate features at multiple scales to obtain second intermediate features at multiple scales, and performs feature fusion on the second intermediate features at multiple scales to obtain image fusion features.

[0296] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the feature fusion module 1402 is a neck network of the palm detection model, which determines a first intermediate feature ranked first in the normal order of scale based on the image feature ranked first in the normal order of scale (the normal order of scales is from smallest to largest), upsamples the first intermediate feature ranked first in the normal order of scale layer by layer, and fuses the upsampled result of the first intermediate feature ranked nth (n is a positive integer from 1 to N-1) in the normal order of scale with the image feature ranked n+1 in the normal order of scale to obtain a first intermediate feature ranked n+1 in the normal order of scale, and determines first intermediate features of multiple scales from the first layer to the Nth layer after obtaining the first intermediate feature of the Nth layer.

[0297] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the feature fusion module 1402 is a neck network of the palm detection model, and determines a first-ranked second intermediate feature in reverse order of scale based on a first-ranked first intermediate feature in reverse order of scale (the reverse order of scale is from largest to smallest), downsamples the first-ranked second intermediate feature in reverse order of scale layer by layer, and fuses the downsampled result of the second intermediate feature in reverse order of scale m (m is a positive integer from 1 to N-1) with the first intermediate feature in reverse order of scale to obtain an m+1-th intermediate feature in reverse order of scale, and then determines second intermediate features of multiple scales from the first layer to the Nth layer after obtaining the N-th layer second intermediate feature.

[0298] In some embodiments, the prediction module 1403 is at least a part of a prediction network of a palm detection model, and obtains a plurality of mesh features of the image fusion features by mesh dividing the image fusion features, and obtains a palm frame by making predictions based on the plurality of mesh features.

[0299] In some embodiments, the palm recognition device is equipped with an infrared camera, and the acquisition module 1406 acquires an infrared image that captures the same palm as the palm image, where the infrared image refers to an image of the palm captured by the infrared camera based on infrared light.

[0300] The region recognition module 1407 recognizes the palm region from the infrared image.

[0301] In some embodiments, the cropping module 1404 determines the intersection of the palm frame in the palm image and the palm region in the infrared image, and crops the palm image from the palm image based on the intersection.

[0302] In some embodiments, the region recognition module 1407 detects flippers in the infrared image and determines a palm region in the infrared image based on the flippers.

[0303] In some embodiments, the region recognition module 1407 acquires at least three webbings in the infrared image, connects the webbings in order to acquire webbing connecting lines, and cuts out the palm region from the infrared image based on the webbing connecting lines.

[0304] In some embodiments, the cropping module 1404 obtains coordinate parameters of a palm frame in the palm image, obtains coordinate parameters of a palm region in the infrared image, and determines an intersection of the palm frame and the palm region based on the coordinate parameters of the palm frame and the coordinate parameters of the palm region.

[0305] FIG. 15 is a schematic diagram illustrating the configuration of a training device for a palm detection model provided in one exemplary embodiment of the present application. This device may be implemented as all or part of a palm recognition device using software, hardware, or a combination of both. The device includes a palm detection model, an acquisition module 1501, a calculation module 1506, and an update module 1507. The palm detection model includes a feature extraction module 1502, a feature fusion module 1503, and a prediction module 1504, and may further include a cropping module 1505. Here, the feature extraction module 1502 may be a backbone network of the palm detection model, the feature fusion module 1503 may be a neck network of the palm detection model, the prediction module 1504 may be at least a part of the prediction network, and the cropping module 1505 may be part of the prediction network, an independent module within the palm detection model, or an independent module outside the palm detection model.

[0306] The acquisition module 1501 acquires a sample palm image and a sample palm portion image.

[0307] The feature extraction module 1502 extracts features from the sample palm image to obtain sample image features at multiple scales.

[0308] The feature fusion module 1503 obtains sample image fusion features by fusing sample image features at multiple scales.

[0309] The prediction module 1504 determines a sample predicted palm outline that identifies the palm in the sample palm image based on the sample image fusion features.

[0310] The cropping module 1505 crops the sample predicted palm image from the sample palm image based on the sample predicted palm frame.

[0311] The calculation module 1506 calculates a loss function value based on the sample palm image and the sample predicted palm image.

[0312] The update module 1507 updates the model parameters of the palm detection model based on the loss function value.

[0313] In some embodiments, the feature extraction module 1502 is a backbone network of a palm detection model, which performs a slicing operation on a sample palm image to obtain slice images at multiple scales, and extracts features from the slice images at multiple scales to obtain sample image features at multiple scales.

[0314] In some embodiments, the feature extraction module 1502 is a backbone network of a palm detection model, and determines a slice image at a maximum scale among multiple scales based on a sample palm image, and starts layer-by-layer downsampling with the slice image at the maximum scale as a first layer to obtain slice images at multiple scales including the slice image at the first layer.

[0315] In some embodiments, the feature extraction module 1502 is a backbone network of a palm detection model, which samples pixel points in the sample palm image at multiple scales and stitches them together to obtain slice images at multiple scales, where two adjacent pixel points in the slice images at different scales have different numbers of pixel points between them at the sampling locations of the sample palm image.

[0316] In some embodiments, the feature fusion module 1503 is a neck network of palm detection models, and performs feature fusion on sample image features at multiple scales to obtain sample image fusion features.

[0317] In some embodiments, the feature fusion module 1503 is a neck network of the palm detection model, which inputs sample image features at multiple scales into the neck network, performs feature fusion based on the sample image features at multiple scales to obtain first intermediate features at multiple scales, performs feature fusion at multiple scales based on the first intermediate features at multiple scales to obtain second intermediate features at multiple scales, and performs feature fusion on the second intermediate features at multiple scales to obtain sample image fusion features.

[0318] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the feature fusion module 1503 is a neck network of the palm detection model, which determines a first intermediate feature ranked first in the normal order of scale based on the sample image feature ranked first in the normal order of scale (the normal order of scales is from smallest to largest), upsamples the first intermediate feature ranked first in the normal order of scale layer by layer, and fuses the upsampling result of the first intermediate feature ranked nth (n is a positive integer from 1 to N-1) in the normal order of scale with the sample image feature ranked n+1 in the normal order of scale to obtain a first intermediate feature ranked n+1 in the normal order of scale, and determines first intermediate features of multiple scales from the first layer to the Nth layer after obtaining the first intermediate feature of the Nth layer.

[0319] In some embodiments, the multiple scales are N scales, where N is a positive integer greater than 1, and the feature fusion module 1503 determines a second intermediate feature of the first order in reverse order of scale based on a first intermediate feature of the first order in reverse order of scale (the reverse order of scale is from largest to smallest), downsamples the second intermediate feature of the first order in reverse order of scale, layer by layer, and fuses the downsampled result of the second intermediate feature of the mth order in reverse order of scale (m is a positive integer from 1 to N-1) with the first intermediate feature of the m+1th order in reverse order of scale to obtain a second intermediate feature of the m+1th order in reverse order of scale, and once the second intermediate feature of the Nth layer is obtained, determines second intermediate features of the multiple scales from the first layer to the Nth layer.

[0320] In some embodiments, the prediction module 1504 is at least a part of a prediction network of a palm detection model, and obtains a plurality of mesh features of the sample image fusion features by mesh dividing the sample image fusion features, and obtains a sample predicted palm frame by performing prediction on the plurality of mesh features.

[0321] In some embodiments, the palm recognition device is equipped with an infrared camera, and the acquisition module 1501 acquires an infrared image capturing the same palm as the sample palm image, where the infrared image refers to an image of the palm captured by the infrared camera based on infrared light.

[0322] The region recognition module 1508 recognizes the palm region from the infrared image.

[0323] The cropping module 1505 determines the intersection of the sample predicted palm frame in the sample palm image and the palm region in the infrared image, and crops the palm image from the sample palm image based on the intersection.

[0324] In some embodiments, the region recognition module 1508 detects flippers in the infrared image and determines a palm region in the infrared image based on the flippers.

[0325] In some embodiments, the cropping module 1505 obtains coordinate parameters of a sample predicted palm frame in the sample palm image, obtains coordinate parameters of a palm region in the infrared image, and determines an intersection of the sample predicted palm frame and the palm region based on the coordinate parameters of the sample predicted palm frame and the coordinate parameters of the palm region.

[0326] In some embodiments, the region recognition module 1508 detects flippers in the infrared image and determines a palm region in the infrared image based on the flippers.

[0327] In some embodiments, the region recognition module 1508 acquires at least three webbings in the infrared image, connects the webbings in order to acquire webbing connecting lines, and cuts out the palm region from the infrared image based on the webbing connecting lines.

[0328] 16 is a block diagram of a computer device 1600 according to an exemplary embodiment of the present application. This computer device can be implemented as a server in the above-described configuration of the present application. The computer device 1600 includes a central processing unit (CPU) 1601, a system memory 1604 including a random access memory (RAM) 1602 and a read-only memory (ROM) 1603, and a system bus 1605 connecting the system memory 1604 and the CPU 1601. The computer device 1600 further includes a mass storage device 1606 for storing an operating system 1609, application programs 1610, and other program modules 1611.

[0329] Mass storage device 1606 is connected to central processing unit 1601 via a mass storage controller (not shown) connected to system bus 1605. Mass storage device 1606 and its associated computer-readable media provide non-volatile storage for computing device 1600. That is, mass storage device 1606 may include a computer-readable medium (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0330] Without loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented by any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage, CD-ROM, digital versatile disk (DVD) or other optical storage, tape cartridges, magnetic tape, magnetic disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media is not limited thereto. The system memory 1604 and mass storage device 1606 described above may be collectively referred to as memory.

[0331] According to various embodiments of the present disclosure, computing device 1600 may be implemented via a network, such as the Internet, connected to a remote computer on the network, i.e., computing device 1600 may be connected to network 1608 via network interface unit 1607 connected to system bus 1605, or may in turn use network interface unit 1607 to connect to other types of networks or remote computer systems (not shown).

[0332] The memory further includes at least one computer-readable instruction, and the at least one computer-readable instruction is stored in the memory. The central processing unit 1601 executes the at least one computer-readable instruction to realize all or part of the steps of the palm image processing method or palm detection model training method described in each of the above embodiments.

[0333] In an embodiment of the present application, a computer device is provided that includes a processor and a memory. The memory stores at least one program, which, when loaded and executed by the processor, realizes the palm image processing method or the palm detection model training method provided in each of the above-described embodiment of the method.

[0334] In an embodiment of the present application, a computer-readable storage medium is provided, which stores at least one computer-readable instruction, which, when loaded and executed by a processor, realizes the palm image processing method or the palm detection model training method provided in each of the above method embodiments.

[0335] In an embodiment of the present application, a computer program product including computer-readable instructions is provided. The computer-readable instructions are stored in a computer-readable storage medium. A processor of a computer device reads and executes the computer-readable instructions from the computer-readable storage medium, thereby causing the computer device to execute and realize the palm image processing method or the palm detection model training method provided in each of the above method embodiments.

[0336] As can be appreciated, in specific embodiments of the present application, with respect to data related to user data processing relating to user identity or characteristics, such as palm images, history data, and images, application of the above examples of the present application to specific products and technologies requires user permission or consent, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0337] It should be understood that "plurality" as referred to herein refers to two or more. "And / or" describes an association relationship between related objects and indicates that three types of relationships are possible. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0338] As can be understood by those skilled in the art, all or part of the steps for realizing the above embodiments may be executed by hardware, or may be executed by instructing relevant hardware through a program. The program may be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0339] The components of the above-described embodiments may be combined in any manner. For the sake of brevity, not all possible combinations of the components of the above-described embodiments are described. However, combinations of these components should be considered within the scope of the present specification unless there is a contradiction.

[0340] The above examples merely illustrate some embodiments of the present application, and although the explanations are specific and detailed, they should not be understood as limiting the patent scope of the present application. It should be noted that those skilled in the art may make some modifications and improvements without departing from the concept of the present application. All of these modifications and improvements fall within the scope of protection of the present application. Therefore, the patent scope of the present application should be governed by the appended claims.

Claims

1. A palm image processing method executed by a palm recognition device, comprising: acquiring a palm image and extracting features from the palm image to obtain image features at multiple scales; fusing the image features of the multiple scales to obtain an image fusion feature; determining a palm frame that identifies a palm in the palm image based on the image fusion features; cutting out a palm image from the palm image based on the palm frame; and a step of obtaining palm image encrypted data by performing image encryption on the palm image, the palm image encrypted data being transmitted to a server and used by the server to perform identity recognition based on the palm image encrypted data. Palm image processing method.

2. the palm frame is predicted by a palm detection model, the palm detection model including a backbone network; The step of extracting features from the palm image to obtain image features at multiple scales includes: performing a slicing operation on the palm image using the backbone network to obtain slice images at multiple scales; and extracting features from the slice images at the plurality of scales using the backbone network to obtain image features at the plurality of scales. The palm image processing method according to claim 1 .

3. The step of obtaining slice images at a plurality of scales by performing a slicing operation on the palm image using the backbone network includes: determining a slice image at a maximum scale among the plurality of scales based on the palm image by the backbone network; and starting layer-by-layer downsampling using the backbone network, with the slice image at the maximum scale as a first layer, to obtain slice images at multiple scales including the slice image at the first layer. The palm image processing method according to claim 2 .

4. The step of obtaining slice images at a plurality of scales by performing a slicing operation on the palm image using the backbone network includes: obtaining slice images at multiple scales by sampling pixel points in the palm image at multiple scales and stitching them together using the backbone network; Two adjacent pixel points in the slice images at different scales have different numbers of pixel points between them at the sampling points of the palm image. The palm image processing method according to claim 2 .

5. the palm frame is predicted by a palm detection model, the palm detection model including a neck network; The step of fusing the image features of the multiple scales to obtain an image fusion feature includes: inputting the image features of the multiple scales into the neck network to perform feature fusion, thereby obtaining the image fusion features; The palm image processing method according to claim 1 .

6. The step of inputting the image features of the multiple scales into the neck network to perform feature fusion to obtain the image fusion features includes: inputting the image features at multiple scales into the neck network; obtaining first intermediate features of multiple scales by performing feature fusion based on the image features of the multiple scales using the neck network; obtaining second intermediate features of multiple scales by performing feature fusion of multiple scales based on the first intermediate features of multiple scales using the neck network; and performing feature fusion on the second intermediate features of the multiple scales by the neck network to obtain the image fusion features. The palm image processing method according to claim 5 .

7. The plurality of scales are N scales, where N is a positive integer greater than 1, and the step of obtaining first intermediate features of the plurality of scales by performing feature fusion based on the image features of the plurality of scales includes: determining a first intermediate feature that is first in order of scale based on the image feature that is first in order of scale (the order of scale is from smallest to largest); upsampling the first intermediate feature ranked first in normal order of scale layer by layer, and fusing the upsampling result of the first intermediate feature ranked nth (n is a positive integer from 1 to N-1) in normal order of scale with the image feature ranked n+1 in normal order of scale to obtain the first intermediate feature ranked n+1 in normal order of scale; Upon obtaining the first intermediate features of the Nth layer, determining first intermediate features of a plurality of scales from the first layer to the Nth layer; The palm image processing method according to claim 6.

8. The plurality of scales is N scales, where N is a positive integer greater than 1, and the step of obtaining second intermediate features of the plurality of scales by performing feature fusion of the plurality of scales based on first intermediate features of the plurality of scales includes: determining a second intermediate feature ranked first in reverse order of scale based on a first intermediate feature ranked first in reverse order of scale (reverse order of scale is from largest to smallest); downsampling the second intermediate feature of the first rank in reverse order of scale layer by layer, and fusing the downsampling result of the second intermediate feature of the mth rank (m is a positive integer from 1 to N-1) in reverse order of scale with the first intermediate feature of the m+1th rank in reverse order of scale to obtain the second intermediate feature of the m+1th rank in reverse order of scale; and when the second intermediate features of the Nth layer are obtained, determining second intermediate features of a plurality of scales from the first layer to the Nth layer. The palm image processing method according to claim 6.

9. the palm frame is predicted by a palm detection model, the palm detection model including a prediction network; The step of determining a palm frame for identifying a palm in the palm image based on the image fusion features includes: obtaining a plurality of mesh features of the image fusion feature by mesh dividing the image fusion feature; inputting the plurality of mesh features into the prediction network and performing prediction on the plurality of mesh features to obtain the palm frame. The palm image processing method according to claim 1 .

10. The palm recognition device is equipped with an infrared camera, and the palm image processing method includes: acquiring an infrared image of the same palm as the palm image, the infrared image being an image of the palm captured by the infrared camera using infrared light; and recognizing a palm region from the infrared image. The step of cutting out a palm image from the palm image based on the palm frame includes: determining an intersection of the palm frame in the palm image and the palm region in the infrared image; and cropping a palm image from the palm image based on the intersection. The palm image processing method according to claim 1 .

11. The step of recognizing a palm region from the infrared image includes: detecting flippers in the infrared image; and determining the palm region in the infrared image based on the flippers. The palm image processing method according to claim 10.

12. The step of determining an intersection of the palm frame in the palm image and the palm region in the infrared image includes: acquiring coordinate parameters of the palm frame in the palm image and acquiring coordinate parameters of the palm region in the infrared image; determining an intersection of the palm frame and the palm region based on coordinate parameters of the palm frame and coordinate parameters of the palm region; The palm image processing method according to claim 10.

13. 1. A method for training a palm detection model executed by a computing device, comprising: acquiring a sample palm image and a sample palm region image; extracting features from the sample palm image to obtain sample image features at multiple scales; fusing the sample image features at the multiple scales to obtain a sample image fusion feature; determining a sample predicted palm outline based on the sample image fusion features, the sample predicted palm outline identifying a palm in the sample palm image; cropping a sample predicted palm image from the sample palm image based on the sample predicted palm frame; calculating a loss function value based on the sample palm image and the sample predicted palm image; updating model parameters of the palm detection model based on the loss function value. Training method for palm detection model.

14. A palm image processing device, a feature extraction module that acquires a palm image and extracts features from the palm image to acquire image features at multiple scales; a feature fusion module for fusing the image features of the multiple scales to obtain an image fusion feature; a prediction module that determines a palm frame that identifies a palm in the palm image based on the image fusion features; a cutout module that cuts out a palm image from the palm image based on the palm frame; an encryption module that performs image encryption on the palm image to obtain palm image encrypted data, the palm image encrypted data being transmitted to a server and used by the server to perform identity verification based on the palm image encrypted data; Palm image processing device.

15. A training device for a palm detection model, comprising: a palm detection model, further comprising: an acquisition module, a clipping module, a calculation module, and an update module; the palm detection model comprising: a feature extraction module, a feature fusion module, and a prediction module; The acquisition module acquires a sample palm image and a sample palm portion image; the feature extraction module extracts features from the sample palm image to obtain sample image features at multiple scales; the feature fusion module fuses the sample image features at the multiple scales to obtain sample image fusion features; the prediction module determines a sample predicted palm outline that identifies a palm in the sample palm image based on the sample image fusion features; the cropping module crops a sample predicted palm image from the sample palm image based on the sample predicted palm frame; the calculation module calculates a loss function value based on the sample palm image and the sample predicted palm image; the update module updates model parameters of a palm detection model based on the loss function value. Training device for palm detection model.

16. 14. A computer device comprising a processor and a memory, wherein the memory stores at least one computer-readable instruction, which, when loaded and executed by the processor, causes the processor to implement the palm image processing method of claim 1 or the palm detection model training method of claim 13.

17. A computer program causing a computer to execute the palm image processing method according to any one of claims 1 to 12 or the palm detection model training method according to claim 13.

Images