A palm print full region of interest image extraction method, device, equipment and medium

Abstract

The application discloses a palmprint full region of interest image extraction method, device and equipment and a medium, which comprises the following steps: acquiring a hand image, inputting the hand image into a preset palmprint extraction model; performing a feature extraction operation on the hand image by using a backbone network to generate a plurality of multi-layer feature maps; performing branch sampling splicing and maximum pooling on each multi-layer feature map by using a feature pyramid network to determine a plurality of feature maps; inputting the feature maps into a region proposal network for convolution operation to output a plurality of candidate boxes; performing a pooling classification operation on the plurality of candidate boxes and the plurality of feature maps by using a region generation predictor to determine a prediction box; and cutting the hand image by using the prediction box to generate a palmprint full region of interest image. The application solves the technical problem that the prior art only extracts a local region of a palm center as a ROI, which makes the palmprint ROI extraction difficult and leads to an unsatisfactory palmprint ROI extraction precision in a complex environment.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for extracting the region of interest (ROI) image of a palm print. Background Technology

[0002] With the rapid development of the digital age, biometric identification technology has become an indispensable part of the information society. Currently, biometric identification technologies mainly include fingerprint recognition, iris recognition, facial recognition, gait recognition, and palmprint recognition. Compared with other biometric features, the palm, as a region of the human body rich in biometric features, contains abundant biometric information. Its features are stable and unique, therefore, palmprint recognition technology can improve the convenience and security of accurate personal identification and personal information security, and has strong application value.

[0003] Traditional methods for extracting palm print ROI (Region of Interest) images mainly involve first finding two key points between the index and middle fingers and between the ring and little fingers through edge detection; then setting the perpendicular bisector of the line segment between the two key points as the center line of the ROI and locating the ROI region; finally, cropping and resizing the located region, for example, to 128×128 pixels, as the traditional ROI of the palm print.

[0004] In real-world user scenarios, this method is often hampered by complex backgrounds and diverse hand poses, making the extracted palm area easily confused with the background. Furthermore, traditional palmprint ROI extraction methods only extract a local area in the center of the palm as the ROI, requiring the discovery of hidden finger key points before finding the central area of ​​the palm, resulting in unsatisfactory accuracy in palmprint ROI extraction under complex environments. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for palmprint region of interest (ROI) image extraction. It solves the technical problem that in real-world user scenarios, existing technologies are often affected by complex backgrounds and diverse hand postures, making the extracted palmprint ROI easily confused with the background. Furthermore, traditional palmprint ROI extraction methods only extract a local area in the center of the palm as the ROI, requiring the location of hidden finger key points before finding the central palm area, resulting in unsatisfactory accuracy in palmprint ROI extraction under complex environments.

[0006] This invention provides a method for extracting the region of interest (ROI) image of a palm print, comprising:

[0007] A hand image to be identified is acquired, and the hand image to be identified is input into a preset palm print extraction model. The preset palm print extraction model includes a backbone network, a feature pyramid network, a region proposal network, and a region generation predictor.

[0008] The backbone network is used to perform feature extraction on the hand image to be identified, generating multiple multi-layer feature maps;

[0009] The feature pyramid network is used to perform branch sampling, concatenation, and max pooling on each of the multi-layer feature maps to determine multiple multi-scale feature maps and coarse feature maps.

[0010] The coarse feature map is input into the region proposal network for convolution operation, and multiple candidate boxes are output.

[0011] The region generation predictor performs pooling classification operations on multiple candidate boxes and multiple multi-scale feature maps to determine the target prediction box.

[0012] The target prediction bounding box is used to crop the hand image to be identified, generating a palm print region of interest image.

[0013] Optionally, the backbone network includes depthwise separable convolutional layers and an input convolutional layer; the step of using the backbone network to perform feature extraction on the hand image to be identified and generating multiple multi-layer feature maps includes:

[0014] The input hand image to be identified is convolved by an input convolutional layer to generate a shallow feature map;

[0015] The shallow feature map is input into four cascaded depth-separable convolutional layers to perform feature extraction operations, and the first multi-layer feature map, the second multi-layer feature map, the third multi-layer feature map and the fourth multi-layer feature map are output in sequence.

[0016] Optionally, the feature pyramid network includes downsampling layers, 1×1 two-dimensional convolutional layers, 3×3 two-dimensional convolutional layers, and max pooling layers; the step of determining multiple multi-scale feature maps and coarse feature maps by performing branch sampling concatenation and max pooling on each of the multi-layer feature maps through the feature pyramid network includes:

[0017] The first multi-layer feature map is input into the 1×1 two-dimensional convolutional layer to perform two-dimensional convolution operation to determine the first feature map;

[0018] After concatenating the first feature map and the second multi-layer feature map, a two-dimensional convolution operation is performed through the 1×1 two-dimensional convolution layer to generate the second feature map;

[0019] After concatenating the second feature map and the third multi-layer feature map, a two-dimensional convolution operation is performed through the 1×1 two-dimensional convolution layer to output the third feature map;

[0020] After concatenating the third feature map and the fourth multi-layer feature map, a two-dimensional convolution operation is performed through the 1×1 two-dimensional convolution layer to determine the fourth feature map;

[0021] Based on the downsampling layer, the 3×3 two-dimensional convolutional layer, and the max pooling layer, sampling convolution and max pooling are performed on the first feature map, the second feature map, the third feature map, and the fourth feature map to determine multiple multi-scale feature maps and coarse feature maps.

[0022] Optionally, the step of performing sampling convolution and max pooling on the first feature map, the second feature map, the third feature map, and the fourth feature map based on the downsampling layer, the 3×3 two-dimensional convolutional layer, and the max pooling layer to determine multiple multi-scale feature maps and coarse feature maps includes:

[0023] The fourth feature map is input into the downsampling layer to perform a downsampling operation, thereby determining the first sampled feature map;

[0024] The first sampled feature map and the third feature map are concatenated to generate a first concatenated feature map. Then, the first concatenated feature map is downsampled through the downsampling layer to output a second sampled feature map.

[0025] The second sampled feature map and the second feature map are concatenated to generate a second concatenated feature map. Then, the downsampling layer is used to downsample the second concatenated feature map to output a third sampled feature map.

[0026] After concatenating the third sampled feature map and the first feature map, feature extraction is performed through the 3×3 two-dimensional convolutional layer to determine the first multi-scale feature map;

[0027] After the first multi-scale feature map and the second stitched feature map are stitched together, feature extraction is performed through the 3×3 two-dimensional convolutional layer to determine the second multi-scale feature map.

[0028] After the second multi-scale feature map and the first stitched feature map are stitched together, feature extraction is performed through the 3×3 two-dimensional convolutional layer to generate a third multi-scale feature map.

[0029] After concatenating the third multi-scale feature map and the fourth feature map, feature extraction is performed through the 3×3 two-dimensional convolutional layer to output the fourth multi-scale feature map.

[0030] The max pooling layer is used to perform a max pooling operation on the fourth multi-scale feature map to determine a coarse feature map.

[0031] Optionally, the region proposal network includes a 1×1 convolutional layer, a 3×3 convolutional layer, an anchor box generator, and a candidate box filter; the step of inputting the coarse feature map into the region proposal network for convolutional operation and outputting multiple candidate boxes includes:

[0032] The coarse feature map is input into the 3×3 convolutional layer and the anchor box generator respectively to perform 3×3 convolution operation and pixel mapping translation to generate the first convolutional feature map and multiple initial anchor boxes;

[0033] The first convolutional feature map is input into the 1×1 convolutional layer to perform 1×1 convolution operation to determine the prediction category and the initial prediction box;

[0034] The candidate box filter is used to filter multiple initial anchor boxes, prediction categories, and initial prediction boxes, and outputs multiple candidate boxes.

[0035] Optionally, the region generation predictor includes a region of interest pooling layer and a fully connected layer; the step of performing pooling classification operations on multiple candidate boxes and multiple multi-scale feature maps through the region generation predictor to determine the target prediction box includes:

[0036] The region of interest pooling layer performs pooling operations on multiple candidate boxes and multiple multi-scale feature maps to generate multiple pooled feature maps.

[0037] After performing feature flattening operation on each of the pooled feature maps, feature mapping and classification are performed through three cascaded fully connected layers to output multiple corresponding intermediate prediction boxes.

[0038] The non-maximum suppression algorithm is used to filter the multiple intermediate prediction boxes to determine the target prediction box.

[0039] Optionally, before the steps of acquiring the hand image to be recognized and inputting the hand image to be recognized into a preset palmprint extraction model, the following steps are included:

[0040] Obtain a set of hand images to be trained, and annotate the regions of interest in the set of hand images to be trained to determine the annotated palm print image set;

[0041] The labeled palmprint image set was used to train the feature pyramid network, region proposal network, and region generation predictor of the initial palmprint extraction model locally, and the number of training iterations was counted in real time.

[0042] When the number of training iterations reaches the preset number of training iterations, the trained initial palmprint extraction model is obtained;

[0043] The trained initial palmprint extraction model is used as the preset palmprint extraction model.

[0044] A second aspect of the present invention provides a palm print region of interest image extraction device, comprising:

[0045] An image acquisition module is used to acquire an image of the hand to be identified and input the image of the hand to be identified into a preset palm print extraction model. The preset palm print extraction model includes a backbone network, a feature pyramid network, a region proposal network, and a region generation predictor.

[0046] The feature extraction module is used to perform feature extraction operations on the hand image to be identified using the backbone network, and generate multiple multi-layer feature maps;

[0047] The feature splicing module is used to perform branch sampling splicing and max pooling on each of the multi-layer feature maps through the feature pyramid network to determine multiple multi-scale feature maps and coarse feature maps.

[0048] The convolutional feature module is used to input the coarse feature map into the region proposal network for convolution operation and output multiple candidate boxes;

[0049] The pooling feature module is used to perform pooling classification operations on multiple candidate boxes and multiple multi-scale feature maps through the region generation predictor to determine the target prediction box;

[0050] The image cropping module is used to crop the hand image to be identified using the target prediction bounding box to generate a palmprint region of interest image.

[0051] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the palm print region of interest image extraction method as described in any of the preceding claims.

[0052] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the palm print region of interest image extraction method as described in any of the preceding claims.

[0053] As can be seen from the above technical solutions, the present invention has the following advantages:

[0054] This invention first replaces the original feature extraction module with a lightweight feature extraction backbone network (MobileNet), which significantly reduces network parameters while still efficiently and accurately extracting feature information from hand images, ensuring the accuracy of palmprint region of interest (ROI) image extraction. Secondly, the multi-layer features extracted by the backbone network are input into a Feature Pyramid Network (FPN) for image processing to obtain multi-scale features, greatly avoiding non-discriminatory loss of feature information. Simultaneously, high-quality candidate boxes output by a Region Proposal Network (RPN) are input into a region generation predictor (Faster R-CNN predictor) for mapping processing to obtain predicted boxes. This effectively resists the effects of lighting and hand tilt, exhibiting good robustness to complex scenes and hand poses, thus accurately determining whether the image to be identified contains a hand image and accurately locating the region to which the hand belongs. Finally, the palmprint ROI image is obtained by locating and cropping the image using the predicted boxes. This image contains the complete palm region and more comprehensive palmprint information, providing information support for large-scale palmprint recognition and high-precision recognition, thereby improving the accuracy of palmprint ROI extraction and achieving better detection results. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating the steps of a palm print region of interest image extraction method provided in this embodiment of the invention;

[0057] Figure 2 This is a schematic diagram of the structure of the preset palmprint extraction model provided in an embodiment of the present invention;

[0058] Figure 3 This diagram illustrates the extraction process of a palmprint CROI (region of total interest) image provided in an embodiment of the present invention.

[0059] Figure 4 A diagram illustrating the extraction process of a traditional palmprint ROI image provided in an embodiment of the present invention;

[0060] Figure 5 A flowchart illustrating the steps of another palmprint region of interest image extraction method provided in this embodiment of the invention;

[0061] Figure 6 This is an image showing the extraction effect of the preset palmprint extraction model provided in this embodiment of the invention on the XJTU-UP database;

[0062] Figure 7 A comparison image of a traditional palm print image and a palm print region of interest image extracted from the original image in the XJTU-UP database, provided in an embodiment of the present invention;

[0063] Figure 8 This is a structural block diagram of a palm print region of interest image extraction device provided in an embodiment of the present invention. Detailed Implementation

[0064] This invention provides a method, apparatus, device, and medium for palmprint region of interest (ROI) image extraction. It addresses the technical problem that existing technologies often suffer from interference from complex backgrounds and diverse hand postures in real-world user scenarios, making the extracted palmprint ROI easily confused with the background. Furthermore, traditional palmprint ROI extraction methods only extract a local area at the center of the palm as the ROI, requiring the location of hidden finger key points before finding the central palm region, resulting in unsatisfactory accuracy in complex environments.

[0065] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0066] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a palm print region of interest (ROI) image extraction method provided in this embodiment of the invention.

[0067] This invention provides a method for extracting the region of interest (ROI) image of a palm print, comprising:

[0068] Step 101: Obtain the hand image to be identified. Input the hand image to be identified into the preset palm print extraction model. The preset palm print extraction model includes a backbone network, a feature pyramid network, a region proposal network, and a region generation predictor.

[0069] It is worth mentioning that the hand image to be identified is obtained by taking a picture of the hand using the camera of a mobile phone or other camera device.

[0070] Please see Figure 2The pre-built palmprint extraction model provided in this embodiment of the invention is a lightweight palmprint CROI (Complete Region of Interest) extraction network model based on Faster R-CNN. It is a lightweight palmprint CROI extraction model based on Faster R-CNN (Faster Region-CNN, a region generation model), with the feature extraction backbone network improved to MobileNet (a lightweight convolutional neural network). This model includes a backbone network, a feature pyramid network, a region proposal network, and a region generation predictor. The backbone network is MobileNet, which can efficiently and accurately extract feature information. The feature pyramid network is an FPN (Feature Pyramid Networks) module, which can combine features across scales to obtain features of different scales, avoiding feature loss and overstretching. The region proposal network is an RPN (Region Proposal Network) module, used to extract high-quality candidate boxes. The region generation predictor is a Faster R-CNN module, which can improve detection speed and accuracy.

[0071] In this embodiment, an image of the hand to be identified is acquired and input into a preset palmprint extraction model. The preset palmprint extraction model includes a backbone network, a feature pyramid network, a region proposal network, and a region generation predictor.

[0072] Step 102: Use a backbone network to perform feature extraction on the hand image to be identified, generating multiple multi-layer feature maps.

[0073] In this step, the multi-layer feature map includes a first multi-layer feature map, a second multi-layer feature map, a third multi-layer feature map, and a fourth multi-layer feature map.

[0074] In this embodiment, a backbone network is used to perform feature extraction on the hand image to be identified, generating multiple multi-layer feature maps.

[0075] Step 103: Perform branch sampling, splicing, and max pooling on each multi-layer feature map using the feature pyramid network to determine multiple multi-scale feature maps and coarse feature maps.

[0076] In this step, the multi-scale feature map includes a first multi-scale feature map, a second multi-scale feature map, a third multi-scale feature map, and a fourth multi-scale feature map.

[0077] In this embodiment, branch sampling and splicing and max pooling are performed on each multi-layer feature map through a feature pyramid network to determine multiple multi-scale feature maps and coarse feature maps.

[0078] Step 104: Input the coarse feature map into the region proposal network for convolution operation and output multiple candidate boxes.

[0079] In this embodiment, the coarse feature map is input into the region proposal network for convolution operation, and multiple candidate boxes are output.

[0080] Step 105: Perform pooling classification on multiple candidate boxes and multiple multi-scale feature maps using the region generation predictor to determine the target prediction box.

[0081] In this embodiment, a region generation predictor performs pooling classification on multiple candidate boxes and multiple multi-scale feature maps to determine the target prediction box.

[0082] Step 106: Use the target prediction bounding box to crop the hand image to be identified and generate a palm print region of interest image.

[0083] Please see Figure 3 This embodiment designs a palmprint CROI (Complete Region of Interest) image extraction process as follows: Based on a given original hand image (a); the palmprint region of the hand is identified (b); based on this palmprint region, a rectangular frame tangent to it is determined (c); finally, the palmprint image cropped from the rectangular frame (d) is the palmprint CROI image. The palmprint CROI image extracted by this embodiment using a pre-set palmprint extraction model contains the complete palm region, possessing the characteristics of including comprehensive palmprint information and facilitating palmprint extraction, providing information support for large-scale palmprint recognition and high-precision recognition.

[0084] Please see Figure 4 The traditional palmprint ROI image extraction process involves: (e) obtaining a given original hand image; (f) detecting two key points between the index and middle fingers and between the ring and little fingers; (g) locating the traditional palmprint ROI position based on the key points; and (h) cropping the image to obtain the traditional palmprint ROI. This image needs to be extracted against a simple, solid-color background, and carries too little palmprint information and is greatly affected by palmprint shrinkage and deformation, making it difficult to meet the needs of a large number of users. Therefore, the palmprint region of interest (CROI) image extracted in this embodiment carries more comprehensive palmprint information, which can improve the accuracy and stability of the palmprint recognition system and bring better detection results.

[0085] In this embodiment, a target prediction bounding box is used to crop the image of the hand to be identified, generating a palm print region of interest image.

[0086] In this embodiment of the invention, by replacing the original feature extraction module with a lightweight feature extraction backbone network (MobileNet), the network parameters are significantly reduced while still efficiently and accurately extracting feature information from hand images, ensuring the accuracy of palmprint region of interest (ROI) image extraction. Secondly, the multi-layer features extracted by the backbone network are input into the Feature Pyramid Network (FPN) for image processing to obtain multi-scale features, greatly avoiding non-discriminatory loss of feature information. Simultaneously, high-quality candidate boxes output by the Region Proposal Network (RPN) are input into the region generation predictor (Faster R-CNN predictor) for mapping processing to obtain predicted boxes. This effectively resists the effects of lighting and hand tilt, exhibiting good robustness to complex scenes and hand poses, thus accurately determining whether the image to be identified contains a hand image and accurately locating the region to which the hand belongs. Finally, the palmprint ROI image obtained by locating and cropping the image using the predicted boxes contains the complete palm region and more comprehensive palmprint information, providing information assurance for large-scale palmprint recognition and high-precision recognition, thereby improving the accuracy of palmprint ROI extraction and achieving better detection results.

[0087] Please see Figure 5 , Figure 5 The flowchart illustrates the steps of a palm print region of interest (ROI) image extraction method provided in this embodiment of the invention.

[0088] This invention provides a method for extracting the region of interest (ROI) image of a palm print, comprising:

[0089] Step 501: Obtain the set of hand images to be trained, and annotate the region of interest in the set of hand images to be trained to determine the annotated palm print image set.

[0090] It is worth mentioning that 2,000 different hand images were first collected using the camera of a mobile phone or other camera device as the hand image set to be trained, and the palm print CROI was labeled using the LabelImg label creation tool to obtain the labeled palm print image set.

[0091] In this embodiment, a set of hand images to be trained is obtained, and regions of interest are labeled on the set of hand images to be trained to determine the labeled palm print image set.

[0092] Step 502: Use the labeled palmprint image set to train the feature pyramid network, region proposal network and region generation predictor of the initial palmprint extraction model locally, and count the number of training iterations in real time.

[0093] It is worth mentioning that the pre-trained parameters on the ImageNet dataset were used as the parameters of the backbone network and the backbone network was frozen. The RPN, FPN and Faster R-CNN predictors in the initial palmprint extraction model were locally trained using a set of labeled palmprint images, and the number of training iterations was counted in real time. The initial palmprint extraction model refers to the untrained Faster R-CNN lightweight palmprint CROI extraction network model.

[0094] In this embodiment, the feature pyramid network, region proposal network, and region generation predictor of the initial palmprint extraction model are trained locally using a set of labeled palmprint images, and the number of training iterations is counted in real time.

[0095] Step 503: When the number of training iterations reaches the preset number of training iterations, the trained initial palmprint extraction model is obtained.

[0096] For example, when the number of training iterations reaches 30, the parameters of the last iteration are used as the test parameters of the model, and the model is used as the trained initial palmprint extraction model to extract palmprint CROI images from newly input hand images.

[0097] In this embodiment, when the number of training iterations reaches the preset number of training iterations, a trained initial palmprint extraction model is obtained.

[0098] Step 504: Use the trained initial palmprint extraction model as the preset palmprint extraction model.

[0099] It is worth mentioning that after using the trained initial palmprint extraction model as the preset palmprint extraction model, please refer to [link / reference needed]. Figure 6 Next, the XJTU-UP database, a palmprint database composed of multiple hand images captured by an iPhone 6s under natural light and flash, was used as a test set to evaluate the palmprint CROI extraction performance. The test set consisted of 6000 original palmprint hand images against complex real-world backgrounds (some of which are shown below). Figure 6 The first row shows the extraction effect of the corresponding palm print CROI as follows: Figure 6 As shown in the second row, the pre-defined palmprint extraction model accurately locates the palmprint CROI and contains comprehensive palmprint information, demonstrating its efficient and accurate extraction of palmprint CROI images. Furthermore, it shows that the lightweight palmprint CROI extraction network can accurately extract CROIs regardless of hand pose or background complexity, indicating that the pre-defined palmprint extraction model in this embodiment has good robustness to complex scenes and hand poses.

[0100] In this embodiment, the trained initial palmprint extraction model is used as the preset palmprint extraction model.

[0101] Step 505: Obtain the hand image to be identified. Input the hand image to be identified into the preset palm print extraction model. The preset palm print extraction model includes a backbone network, a feature pyramid network, a region proposal network, and a region generation predictor.

[0102] In this embodiment, an image of the hand to be identified is acquired and input into a preset palmprint extraction model. The preset palmprint extraction model includes a backbone network, a feature pyramid network, a region proposal network, and a region generation predictor.

[0103] Step 506: Use the backbone network to perform feature extraction on the hand image to be identified, and generate multiple multi-layer feature maps.

[0104] Multilayer feature maps refer to the feature maps output by the backbone network after image processing. In other words, they can correspond to any feature map output by the backbone network after image processing during image detection.

[0105] The backbone network consists of depthwise separable convolution (DSC) layers and input convolutional layers (Conv1).

[0106] For example, the hand image to be identified is input into the backbone network MobileNet. First, the image is convolved with a stride of 2 to generate a shallow feature map of 320*320*32. Then, the feature is extracted through the DSC block in MobileNet. In this embodiment, the feature maps of the first four DSC blocks are taken. It can be seen that the resolution of these four feature maps ranges from 160*160 to 20*20, covering features of multiple scales. Therefore, they are the best multi-feature input for FPN.

[0107] Further, step 506 may include the following sub-steps:

[0108] S61. The input convolutional layer is used to perform convolution processing on the input hand image to be recognized to generate a shallow feature map.

[0109] S62. Input the shallow feature map into the four cascaded depth separable convolutional layers to perform feature extraction operations, and output the first multi-layer feature map, the second multi-layer feature map, the third multi-layer feature map and the fourth multi-layer feature map in sequence.

[0110] It is worth mentioning that feature extraction is performed on the shallow feature map by a depthwise separable convolutional layer to output the first multi-layer feature map; the first multi-layer feature map is input into the depthwise separable convolutional layer to perform feature extraction to obtain the second multi-layer feature map; the second multi-layer feature map is then extracted using a depthwise separable convolutional layer to generate the third multi-layer feature map; and the third multi-layer feature map is then extracted using a depthwise separable convolutional layer to output the fourth multi-layer feature map; where the shallow feature map refers to the intermediate map generated in the backbone network.

[0111] In this embodiment, a backbone network is used to perform feature extraction on the hand image to be identified, generating multiple multi-layer feature maps.

[0112] Step 507: Perform branch sampling, splicing, and max pooling on each multi-layer feature map using the feature pyramid network to determine multiple multi-scale feature maps and coarse feature maps.

[0113] The feature pyramid network consists of downsampling layers, 1×1 two-dimensional convolutional layers (Conv_2D 1×1), 3×3 two-dimensional convolutional layers (Conv_2D 3×3), and max pooling layers (Maxpool 1×1).

[0114] Multi-scale feature maps and coarse feature maps refer to the feature maps output by the feature pyramid network after image processing.

[0115] For example, after multiple features are input into the FPN, each feature is downsampled, added to the features below it, and then convolved to obtain four different features, which are the multi-scale features output by the FPN. Then, a max pooling layer is added to the 20*20*256 output features to output a coarse feature, which is then fed into the feature input of the RPN.

[0116] Furthermore, step 507 may include the following sub-steps:

[0117] S71. Input the first multi-layer feature map into a 1×1 two-dimensional convolutional layer to perform two-dimensional convolution operation to determine the first feature map.

[0118] S72. After concatenating the first feature map and the second multi-layer feature map, perform a two-dimensional convolution operation through a 1×1 two-dimensional convolution layer to generate the second feature map.

[0119] S73. After concatenating the second feature map and the third multi-layer feature map, perform two-dimensional convolution operation through a 1×1 two-dimensional convolution layer to output the third feature map.

[0120] S74. After concatenating the third feature map and the fourth multi-layer feature map, perform a two-dimensional convolution operation through a 1×1 two-dimensional convolution layer to determine the fourth feature map.

[0121] S75. Based on the downsampling layer, the 3×3 two-dimensional convolutional layer and the max pooling layer, sampling convolution and max pooling are performed on the first feature map, the second feature map, the third feature map and the fourth feature map to determine multiple multi-scale feature maps and coarse feature maps.

[0122] Furthermore, S75 may include the following sub-steps:

[0123] The fourth feature map is input to a downsampling layer to perform a downsampling operation, determining the first sampled feature map. The first sampled feature map and the third feature map are concatenated to generate the first concatenated feature map. Then, the first concatenated feature map is downsampled by the downsampling layer to output the second sampled feature map. The second sampled feature map and the second feature map are concatenated to generate the second concatenated feature map. Then, the second concatenated feature map is downsampled by the downsampling layer to output the third sampled feature map. After concatenating the third sampled feature map and the first feature map, feature extraction is performed through a 3×3 two-dimensional convolutional layer to determine the first multi-scale feature map. After concatenating the first multi-scale feature map and the second concatenated feature map, feature extraction is performed through a 3×3 two-dimensional convolutional layer to determine the second multi-scale feature map. After concatenating the second multi-scale feature map and the first concatenated feature map, feature extraction is performed through a 3×3 two-dimensional convolutional layer to generate the third multi-scale feature map. After concatenating the third and fourth multi-scale feature maps, a 3×3 two-dimensional convolutional layer is used to perform feature extraction, outputting the fourth multi-scale feature map. A max-pooling layer is then applied to the fourth multi-scale feature map to determine the coarse feature map.

[0124] In this embodiment, branch sampling and splicing and max pooling are performed on each multi-layer feature map through a feature pyramid network to determine multiple multi-scale feature maps and coarse feature maps.

[0125] Step 508: Input the coarse feature map into the region proposal network for convolution operation and output multiple candidate boxes.

[0126] The region proposal network consists of 1×1 convolutional layers (Conv 1×1), 3×3 convolutional layers (Conv 3×3), an anchor box generator, and a candidate box filter.

[0127] For example, after obtaining the coarse features from the FPN output, the RPN passes them through a 3x3 convolution and then through 1x1 convolutional layers to obtain the predicted class and predicted bounding box. At the same time, the anchor map generator in the RPN generates multiple anchor boxes based on the input feature map, and then sends them together with the obtained predicted class and predicted bounding box to the candidate box filter to filter out candidate boxes containing the target. The candidate boxes are then sent to the Faster R-CNN predictor.

[0128] Further, step 508 may include the following sub-steps:

[0129] S81. Input the coarse feature map into the 3×3 convolutional layer and the anchor box generator respectively to perform 3×3 convolution operation and pixel mapping translation to generate the first convolutional feature map and multiple initial anchor boxes.

[0130] S82. Input the first convolutional feature map into the 1×1 convolutional layer to perform 1×1 convolution operation to determine the prediction category and the initial prediction box.

[0131] S83. Use a candidate box filter to filter multiple initial anchor boxes, prediction categories, and initial prediction boxes, and output multiple candidate boxes.

[0132] In this embodiment, the coarse feature map is input into the region proposal network for convolution operation, and multiple candidate boxes are output.

[0133] Step 509: Perform pooling classification on multiple candidate boxes and multiple multi-scale feature maps using the region generation predictor to determine the target prediction box.

[0134] The region generation predictor includes a Region of Interest (ROI) pooling layer and a fully connected layer (FC layer).

[0135] For example, the multi-scale features generated by FPN and the candidate boxes generated by RPN are input into the ROI pooling layer. The pooling is performed on the input multi-scale feature map according to the candidate boxes, and the corresponding IoU is calculated to match the appropriate feature map. Then, the feature map of the candidate box is flattened and passed through multiple fully connected layers (FC) (feature map classification) to obtain the preliminary predicted box. Then, it is fed into the filtering process, which includes nonmaximum suppression and other processing, and finally, the most suitable predicted box is output.

[0136] Further, step 509 may include the following sub-steps:

[0137] S91. Perform pooling operations on multiple candidate boxes and multiple multi-scale feature maps through the region of interest pooling layer to generate multiple pooled feature maps.

[0138] S92. After performing feature flattening operation on each pooling feature map, feature mapping and classification are performed through three cascaded fully connected layers to output multiple corresponding intermediate prediction boxes.

[0139] S93. The non-maximum suppression algorithm is used to filter multiple intermediate prediction boxes to determine the target prediction box.

[0140] It is worth mentioning that after multiple candidate boxes and multiple multi-scale feature maps are input into the region of interest pooling layer, each candidate box is pooled onto each multi-scale feature map through the region of interest pooling layer, resulting in multiple pooled feature maps. The IOU value corresponding to the multiple pooled feature maps is calculated. Then, each pooled feature map is flattened and passed through multiple fully connected layers for feature mapping and classification, thereby obtaining multiple intermediate predicted boxes corresponding to the multiple pooled feature maps. Next, the multiple intermediate predicted boxes are filtered, and finally, non-maximum suppression (NMS) is used to select the intermediate predicted box with the largest IOU value as the target predicted box. Here, the pooled feature map is an intermediate map generated by the region generation predictor.

[0141] Step 510: Use the target prediction bounding box to crop the hand image to be identified and generate a palm print region of interest image.

[0142] It is worth mentioning that, to demonstrate the effectiveness of palmprint CROI in palmprint recognition, this embodiment compares its palmprint recognition performance with that of traditional palmprint ROI. Please refer to [link / reference]. Figure 7 This embodiment uses the XJTU-UP database for comparative experiments, extracting both traditional palmprint ROI and palmprint CROI. Three classic deep learning classification models are used: VGG16, ResNet50, and MobileNet. Each experiment is iterated 300 times, and the identification accuracy and error rate are calculated. The results are shown in Table 1.

[0143] As shown in Table 1, the palmprint CROI performs better than the traditional palmprint ROI. This is because the palmprint information in the CROI is more comprehensive, making the palmprint recognition system more accurate and stable. It's worth noting that the palmprint CROI does not perform rotation correction on the palmprint image, yet its recognition effect is still better than that of the traditional palmprint ROI, which does perform rotation correction. Therefore, in the future, the palmprint CROI will be more effective than the traditional palmprint ROI in palmprint recognition systems for large-scale users.

[0144] Table 1 Comparison of experimental data between traditional palmprint ROI and palmprint CROI

[0145]

[0146] In this embodiment of the invention, by replacing the original feature extraction module with a lightweight feature extraction backbone network (MobileNet), the network parameters are significantly reduced while still efficiently and accurately extracting feature information from hand images, ensuring the accuracy of palmprint region of interest (ROI) image extraction. Secondly, the multi-layer features extracted by the backbone network are input into the Feature Pyramid Network (FPN) for image processing to obtain multi-scale features, greatly avoiding non-discriminatory loss of feature information. Simultaneously, high-quality candidate boxes output by the Region Proposal Network (RPN) are input into the region generation predictor (Faster R-CNN predictor) for mapping processing to obtain predicted boxes. This effectively resists the effects of lighting and hand tilt, exhibiting good robustness to complex scenes and hand poses, thus accurately determining whether the image to be identified contains a hand image and accurately locating the region to which the hand belongs. Finally, the palmprint ROI image obtained by locating and cropping the image using the predicted boxes contains the complete palm region and more comprehensive palmprint information, providing information assurance for large-scale palmprint recognition and high-precision recognition, thereby improving the accuracy of palmprint ROI extraction and achieving better detection results.

[0147] Please see Figure 8 , Figure 8 This is a structural block diagram of a palm print region of interest image extraction device provided in an embodiment of the present invention.

[0148] Image acquisition module 801 is used to acquire the hand image to be identified and input the hand image to be identified into a preset palm print extraction model. The preset palm print extraction model includes a backbone network, a feature pyramid network, a region proposal network, and a region generation predictor.

[0149] The feature extraction module 802 is used to perform feature extraction operations on the hand image to be identified using the backbone network, generating multiple multi-layer feature maps.

[0150] The feature splicing module 803 is used to perform branch sampling splicing and max pooling on the feature pyramid network to determine multiple multi-scale feature maps and coarse feature maps.

[0151] The convolutional feature module 804 is used to input coarse feature maps into the region proposal network for convolution operations and output multiple candidate boxes.

[0152] The pooling feature module 805 is used to perform pooling classification operations on multiple candidate boxes and multiple multi-scale feature maps through the region generation predictor to determine the target prediction box.

[0153] The image cropping module 806 is used to crop the hand image to be recognized using the target prediction bounding box to generate a palm print region of interest image.

[0154] Furthermore, the feature extraction module 802 includes:

[0155] The input convolution submodule is used to perform convolution processing on the input hand image to be recognized using the input convolutional layer to generate a shallow feature map.

[0156] The output multi-layer feature map submodule is used to input shallow feature maps into four cascaded depthwise separable convolutional layers to perform feature extraction operations, and outputs the first multi-layer feature map, the second multi-layer feature map, the third multi-layer feature map and the fourth multi-layer feature map in sequence.

[0157] Furthermore, the splicing feature module 803 includes:

[0158] The first feature submodule is used to input the first multi-layer feature map into a 1×1 two-dimensional convolutional layer to perform two-dimensional convolution operations and determine the first feature map.

[0159] The second feature submodule is used to concatenate the first feature map and the second multi-layer feature map, and then perform a two-dimensional convolution operation through a 1×1 two-dimensional convolution layer to generate the second feature map.

[0160] The third feature submodule is used to concatenate the second feature map and the third multi-layer feature map, and then perform two-dimensional convolution operation through a 1×1 two-dimensional convolution layer to output the third feature map.

[0161] The fourth feature submodule is used to concatenate the third feature map and the fourth multi-layer feature map, and then perform a two-dimensional convolution operation through a 1×1 two-dimensional convolution layer to determine the fourth feature map.

[0162] The multi-feature submodule is used to perform sampling convolution and max pooling on the first feature map, the second feature map, the third feature map, and the fourth feature map based on the downsampling layer, the 3×3 two-dimensional convolutional layer, and the max pooling layer to determine multiple multi-scale feature maps and coarse feature maps.

[0163] Furthermore, the multi-feature submodule includes:

[0164] The first sampling unit is used to input the fourth feature map into the downsampling layer to perform downsampling operations and determine the first sampled feature map.

[0165] The first sampling and splicing unit is used to splice the first sampled feature map and the third feature map to generate the first spliced ​​feature map. Then, the first spliced ​​feature map is downsampled by the downsampling layer to output the second sampled feature map.

[0166] The second sampling and splicing unit is used to splice the second sampled feature map and the second feature map to generate a second spliced ​​feature map. Then, a downsampling layer is used to downsample the second spliced ​​feature map to output a third sampled feature map.

[0167] The first multi-scale unit is used to concatenate the third sampled feature map and the first feature map, and then extract features through a 3×3 two-dimensional convolutional layer to determine the first multi-scale feature map.

[0168] The second multi-scale unit is used to concatenate the first multi-scale feature map and the second concatenated feature map, and then perform feature extraction operations through a 3×3 two-dimensional convolutional layer to determine the second multi-scale feature map.

[0169] The third multi-scale unit is used to concatenate the second multi-scale feature map and the first concatenated feature map, and then extract features through a 3×3 two-dimensional convolutional layer to generate the third multi-scale feature map.

[0170] The fourth multi-scale unit is used to concatenate the third and fourth multi-scale feature maps and then perform feature extraction operations through a 3×3 two-dimensional convolutional layer to output the fourth multi-scale feature map.

[0171] The coarse feature unit is used to perform max pooling on the fourth multi-scale feature map using a max pooling layer to determine the coarse feature map.

[0172] Furthermore, the convolutional feature module 804 includes:

[0173] The convolutional submodule is used to input the coarse feature map into a 3×3 convolutional layer and an anchor box generator to perform 3×3 convolution operations and pixel mapping translation, generating the first convolutional feature map and multiple initial anchor boxes.

[0174] The prediction submodule is used to input the first convolutional feature map into a 1×1 convolutional layer for 1×1 convolution operation to determine the prediction category and the initial prediction box.

[0175] The output submodule is used to filter multiple initial anchor boxes, prediction categories, and initial prediction boxes using a candidate box filter, and output multiple candidate boxes.

[0176] Furthermore, the pooling feature module 805 includes:

[0177] The pooling submodule is used to perform pooling operations on multiple candidate boxes and multiple multi-scale feature maps through the region of interest pooling layer to generate multiple pooled feature maps.

[0178] The mapping submodule is used to perform feature flattening operations on each pooling feature map, and then perform feature mapping and classification through three cascaded fully connected layers to output multiple corresponding intermediate prediction boxes.

[0179] The filtering submodule is used to filter multiple intermediate prediction boxes using a non-maximum suppression algorithm to determine the target prediction box.

[0180] In one alternative embodiment, the apparatus further includes:

[0181] The data acquisition module is used to acquire a set of hand images to be trained, and to annotate the regions of interest in the set of hand images to be trained, thereby determining the annotated palmprint image set.

[0182] The training module is used to train the feature pyramid network, region proposal network, and region generation predictor of the initial palmprint extraction model using a set of labeled palmprint images, and to count the number of training iterations in real time.

[0183] The judgment module is used to obtain the trained initial palmprint extraction model when the number of training iterations reaches the preset number of training iterations.

[0184] The model module is used to use the trained initial palmprint extraction model as a preset palmprint extraction model.

[0185] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, sub-modules and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0186] This invention also provides an electronic device, which includes a processor and a memory:

[0187] The memory is used to store program code and transfer the program code to the processor;

[0188] The processor is used to execute the palmprint region of interest image extraction method of the above embodiments of the present invention according to the instructions in the program code.

[0189] This invention also provides a computer-readable storage medium for storing program code for executing the palmprint region of interest image extraction method described above.

[0190] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0191] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0192] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0193] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0194] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting the region of interest (ROI) image of a palm print, characterized in that, include: A hand image to be identified is acquired, and the hand image to be identified is input into a preset palm print extraction model. The preset palm print extraction model includes a backbone network, a feature pyramid network, a region proposal network, and a region generation predictor. The backbone network is used to perform feature extraction on the hand image to be identified, generating multiple multi-layer feature maps; The feature pyramid network is used to perform branch sampling, concatenation, and max pooling on each of the multi-layer feature maps to determine multiple multi-scale feature maps and coarse feature maps. The coarse feature map is input into the region proposal network for convolution operation, and multiple candidate boxes are output. The region generation predictor performs pooling classification operations on multiple candidate boxes and multiple multi-scale feature maps to determine the target prediction box. The target prediction bounding box is used to crop the hand image to be identified, generating a palm print region of interest image; The backbone network includes depthwise separable convolutional layers and input convolutional layers with a stride of 2; The step of performing feature extraction on the hand image to be identified using the backbone network to generate multiple multi-layer feature maps includes: The input hand image to be identified is convolved using an input convolutional layer with a stride of 2 to generate a shallow feature map; The shallow feature map is input into four cascaded depth-separable convolutional layers to perform feature extraction operations, and the first multi-layer feature map, the second multi-layer feature map, the third multi-layer feature map and the fourth multi-layer feature map are output in sequence.

2. The palmprint region of interest image extraction method according to claim 1, characterized in that, The feature pyramid network includes downsampling layers, 1×1 two-dimensional convolutional layers, 3×3 two-dimensional convolutional layers, and max pooling layers; the step of determining multiple multi-scale feature maps and coarse feature maps by performing branch sampling, concatenation, and max pooling on each of the multi-layer feature maps through the feature pyramid network includes: The first multi-layer feature map is input into the 1×1 two-dimensional convolutional layer to perform two-dimensional convolution operation to determine the first feature map; After concatenating the first feature map and the second multi-layer feature map, a two-dimensional convolution operation is performed through the 1×1 two-dimensional convolution layer to generate the second feature map; After concatenating the second feature map and the third multi-layer feature map, a two-dimensional convolution operation is performed through the 1×1 two-dimensional convolution layer to output the third feature map; After concatenating the third feature map and the fourth multi-layer feature map, a two-dimensional convolution operation is performed through the 1×1 two-dimensional convolution layer to determine the fourth feature map; Based on the downsampling layer, the 3×3 two-dimensional convolutional layer, and the max pooling layer, sampling convolution and max pooling are performed on the first feature map, the second feature map, the third feature map, and the fourth feature map to determine multiple multi-scale feature maps and coarse feature maps.

3. The palmprint region of interest image extraction method according to claim 2, characterized in that, The step of performing sampling convolution and max pooling on the first feature map, the second feature map, the third feature map, and the fourth feature map based on the downsampling layer, the 3×3 two-dimensional convolutional layer, and the max pooling layer to determine multiple multi-scale feature maps and coarse feature maps includes: The fourth feature map is input into the downsampling layer to perform a downsampling operation, thereby determining the first sampled feature map; The first sampled feature map and the third feature map are concatenated to generate a first concatenated feature map. Then, the first concatenated feature map is downsampled through the downsampling layer to output a second sampled feature map. The second sampled feature map and the second feature map are concatenated to generate a second concatenated feature map. Then, the downsampling layer is used to downsample the second concatenated feature map to output a third sampled feature map. After concatenating the third sampled feature map and the first feature map, feature extraction is performed through the 3×3 two-dimensional convolutional layer to determine the first multi-scale feature map; After the first multi-scale feature map and the second stitched feature map are stitched together, feature extraction is performed through the 3×3 two-dimensional convolutional layer to determine the second multi-scale feature map. After the second multi-scale feature map and the first stitched feature map are stitched together, feature extraction is performed through the 3×3 two-dimensional convolutional layer to generate a third multi-scale feature map. After concatenating the third multi-scale feature map and the fourth feature map, feature extraction is performed through the 3×3 two-dimensional convolutional layer to output the fourth multi-scale feature map. The max pooling layer is used to perform a max pooling operation on the fourth multi-scale feature map to determine a coarse feature map.

4. The palmprint region of interest image extraction method according to claim 1, characterized in that, The region proposal network includes a 1×1 convolutional layer, a 3×3 convolutional layer, an anchor box generator, and a candidate box filter; the step of inputting the coarse feature map into the region proposal network for convolution operation and outputting multiple candidate boxes includes: The coarse feature map is input into the 3×3 convolutional layer and the anchor box generator respectively to perform 3×3 convolution operation and pixel mapping translation to generate the first convolutional feature map and multiple initial anchor boxes; The first convolutional feature map is input into the 1×1 convolutional layer to perform 1×1 convolution operation to determine the prediction category and the initial prediction box; The candidate box filter is used to filter multiple initial anchor boxes, prediction categories, and initial prediction boxes, and outputs multiple candidate boxes.

5. The palmprint region of interest image extraction method according to claim 1, characterized in that, The region generation predictor includes a region of interest pooling layer and a fully connected layer; the step of performing pooling classification operations on multiple candidate boxes and multiple multi-scale feature maps through the region generation predictor to determine the target prediction box includes: The region of interest pooling layer performs pooling operations on multiple candidate boxes and multiple multi-scale feature maps to generate multiple pooled feature maps. After performing feature flattening operation on each of the pooled feature maps, feature mapping and classification are performed through three cascaded fully connected layers to output multiple corresponding intermediate prediction boxes. The non-maximum suppression algorithm is used to filter the multiple intermediate prediction boxes to determine the target prediction box.

6. The palmprint region of interest image extraction method according to claim 1, characterized in that, Before the steps of acquiring the hand image to be identified and inputting the hand image to be identified into a preset palmprint extraction model, the following steps are included: Obtain a set of hand images to be trained, and annotate the regions of interest in the set of hand images to be trained to determine the annotated palm print image set; The labeled palmprint image set was used to train the feature pyramid network, region proposal network, and region generation predictor of the initial palmprint extraction model locally, and the number of training iterations was counted in real time. When the number of training iterations reaches the preset number of training iterations, the trained initial palmprint extraction model is obtained; The trained initial palmprint extraction model is used as the preset palmprint extraction model.

7. A palmprint region of interest (ROI) image extraction device, applied to the palmprint ROI image extraction method of claim 1, characterized in that, include: An image acquisition module is used to acquire an image of the hand to be identified and input the image of the hand to be identified into a preset palm print extraction model. The preset palm print extraction model includes a backbone network, a feature pyramid network, a region proposal network, and a region generation predictor. The feature extraction module is used to perform feature extraction operations on the hand image to be identified using the backbone network, and generate multiple multi-layer feature maps; The feature splicing module is used to perform branch sampling splicing and max pooling on each of the multi-layer feature maps through the feature pyramid network to determine multiple multi-scale feature maps and coarse feature maps. The convolutional feature module is used to input the coarse feature map into the region proposal network for convolution operation and output multiple candidate boxes; The pooling feature module is used to perform pooling classification operations on multiple candidate boxes and multiple multi-scale feature maps through the region generation predictor to determine the target prediction box; The image cropping module is used to crop the hand image to be identified using the target prediction bounding box to generate a palmprint region of interest image.

8. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the palm print region of interest image extraction method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the palmprint region of interest image extraction method as described in any one of claims 1-6.

Images