A palm vein edge refinement detection method integrating cross self-attention

Abstract

The present invention provides a palm vein edge refinement detection method that incorporates cross-self-attention. The method comprises: obtaining a palm vein image to be processed; inputting the image into a trained palm vein edge refinement detection network to obtain edge information. The palm vein edge refinement detection network comprises a feature extraction unit for extracting local details and global structure, a feature fusion unit for fusing features of different scales in space and channels, and a feature refinement unit for segmenting the fused features. The method first extracts local details and global features using the feature extraction unit, then uses the feature fusion unit to aggregate features of different scales in space and channels, and finally uses the feature refinement unit to specifically refine uncertain points in the initial prediction results, thereby achieving accurate segmentation of palm vein edges.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a palm vein edge refinement detection method integrating cross self-attention. Background Art

[0002] As time goes by, biometric technology has gradually replaced traditional authentication methods such as magnetic cards and passwords, effectively eliminating the risks of loss and theft of traditional identity authentication methods. While biometric technology has achieved significant improvements in security and convenience, it is also susceptible to privacy breaches and brute force attacks. For example, fingerprint recognition, due to its practicality and cost-effectiveness, has been widely adopted in various scenarios, such as time clocks and electronic payments. However, it has high requirements for skin surface condition, and excessively dry or humid environments can reduce fingerprint recognition accuracy. Furthermore, as a contact-based biometric technology, fingerprint recognition also poses hygiene risks and increases the potential for disease transmission. Facial recognition, as a contactless recognition technology, is simple to use and provides a good user experience, but its information content is limited and is easily affected by facial expressions, makeup, age, lighting, and facial occlusion. Palm vein recognition technology, which uses near-infrared cameras to capture images, has the advantages of being unrepeatable and contactless, and has gained high recognition in the field of public security authentication. However, in practical applications, palm vein images acquired under near-infrared light are prone to poor quality due to uneven lighting, changes in palm posture, or differences in individual vein structure, resulting in poor edge detection and low palm vein recognition accuracy. Summary of the Invention

[0003] In view of the above defects of the prior art, the present invention provides a palm vein edge refinement detection method integrating cross self-attention to solve the technical problem of poor palm vein edge detection effect.

[0004] To achieve the above-mentioned and other related purposes, the present invention provides a palm vein edge thinning detection method integrating cross self-attention, comprising: obtaining a palm vein image to be processed; inputting the palm vein image to be processed into a trained palm vein edge thinning detection network to obtain edge information, wherein the palm vein edge thinning detection network is expressed as follows: out =H FRU (H FFU (H FEU (I in ))), where H FEU is a feature extraction unit used to extract local details and global structures, H FFU It is a feature fusion unit used to fuse features of different scales in space and channels. FRU It is a feature refinement unit used to segment the fused features.

[0005] In one embodiment of the present invention, the feature extraction unit is composed of a cascade of an overlapping patch extraction module and four dual cross-attention Transformer modules.

[0006] In one embodiment of the present invention, the input features of the first dual cross-attention Transformer module are the output features of the overlapping patch extraction module, and the input features of the j-th dual cross-attention Transformer module are the output features of the j-1-th dual cross-attention Transformer module, where j∈{2,3,4}.

[0007] In one embodiment of the present invention, the expression of the overlapping patch extraction module is as follows: i =DConv1D 1×1 (Reshape(AvgPool2D(E i ))), where E i To divide the overlapping patch area obtained by the palm vein image to be processed, AvgPool2D is a two-dimensional average pooling, Reshape is reshaping, and DConv1D 1×1 is a one-dimensional depth-wise separable convolution with a convolution kernel size of 1×1, P i is the feature after the i-th mapping.

[0008] In one embodiment of the present invention, the dual cross attention Transformer module is composed of a cascade of a dual cross attention layer and a hybrid feedforward layer. The expressions of the dual cross attention layer and the hybrid feedforward layer are as follows: X2=H SCA (H CCA (X1)+X1)+H CCA (X1)+X1,X3=GELU(Conv 3×3 (MLP(X2)))+X2, where X1 is the input feature of the double cross attention Transformer module, H CCA is the channel cross attention, H SCA is the spatial cross attention, X2 is the output feature of the double cross attention layer, MLP is the multi-layer perceptron, Conv 3×3 is a convolution with a kernel size of 1×1, GELU is the activation function, and X3 is the output feature of the dual cross attention Transformer module.

[0009] In one embodiment of the present invention, the channel cross attention is processed as follows: for each mapped feature P i Perform layer normalization to obtain feature P i ';For feature P i 'Splice along the channel dimension to obtain feature PC ; Calculate the key, value, and query according to the following formula: K T =DConv1D K (P C ), V=DConv1D V (P C ), Q i =DConv1D Q (P i '); Then, based on the key, value, and query, the output feature of the channel cross attention is calculated according to the following formula:

[0010] ,

[0011] Among them, Q i is the i-th query matrix, K is the key, K T is its transpose, V is the value, C C is the dimension of the key, DConv1D is the depthwise separable convolution, and Softmax is the normalization.

[0012] In one embodiment of the present invention, the spatial cross attention is processed as follows: the output feature P of the channel cross attention is processed as follows: i Perform layer normalization to obtain feature P i ';For feature P i 'Splice along the channel dimension to obtain feature P C ; Calculate the key, value, and query according to the following formula: K T =DConv1D K (P C ), V i =DConv1D V (P i '), Q = DConv1D Q (P C ); then, based on the key, value, and query, the output feature of the channel cross attention is calculated according to the following formula:

[0013] ,

[0014] Among them, Q is the query, K is the key, K T is its transpose, V i is the i-th value matrix, C C is the dimension of the key, DConv1D is the depthwise separable convolution, and Softmax is the normalization.

[0015] In one embodiment of the present invention, the feature fusion unit includes a cascade of: a multi-scale feature fusion module for spatially enhancing the output features of the four dual cross-attention Transformer modules and integrating features of different receptive fields through feature splicing; and a channel feature fusion module for performing channel feature fusion.

[0016] In one embodiment of the present invention, the expression of the multi-scale feature fusion module is as follows: j =Conv 1×1 X 3j +Conv 3×3 X 3j +…+Conv n×n X 3j , X4=Y1+Upsample(Y2,Y3,Y4), where j∈{1,2,3,4}, X 3j is the output feature of the j-th double cross attention Transformer module, Conv n×n is a convolution with a kernel size of n×n, Y j is the j-th multi-scale feature output, Upsample is upsampling, and X4 is the output feature of the multi-scale feature fusion module.

[0017] In one embodiment of the present invention, the expression of the channel feature fusion module is as follows: j =Softmax(Sigmoid(GAP(X 4j ))), X5=Conv 1×1 (Channel Shuffle(w1·X 41 +w2·X 42 +w3·X 43 +w4·X 44 ), where j∈{1,2,3,4}, X 4j is obtained by dividing the output features of the multi-scale feature fusion module into channels, GAP is global average pooling, Sigmoid is the Sigmoid function, Softmax is normalization, and w j is the weight vector, ChannelShuffle is channel shuffle, and X5 is the output feature of the channel feature fusion module.

[0018] In one embodiment of the present invention, the feature refinement unit includes multiple groups of cascaded coarse prediction modules and fine prediction modules; the coarse prediction module is used to make a rough prediction of the fused features, and the fine prediction module is used to perform targeted refinement on uncertain points in the coarse prediction results.

[0019] In one embodiment of the present invention, the expression of the coarse prediction module is as follows: X6=Sigmoid(Conv 3×3 (BilinearUpsampling(X5))), where X5 is the output feature of the feature fusion unit, BilinearUpsampling is bilinear interpolation upsampling, Conv 3×3 is a convolution with a convolution kernel size of 3×3, Sigmoid is a Sigmoid function, and X6 is the output feature of the coarse prediction module.

[0020] In one embodiment of the present invention, the fine prediction module includes a cascade of dynamic convolution layers, bilinear interpolation upsampling, and a multi-layer perceptron; the dynamic convolution layer is processed as follows: k×N points p are randomly generated in the rough prediction result. i , i=1,2,…,k*N; extract each point p i The local feature map around the input position is obtained by using a 1×1 convolution layer to generate a set of dynamic convolution kernels K for the local information of each input position. pi ; The dynamic convolution kernel K pi Convolve with the preliminary prediction map to obtain feature Y pi ; Calculate feature Y pi The uncertainty of each point in , and then sort and select the top β×N most uncertain true value points.

[0021] Beneficial effects of the present invention: The present invention proposes a palm vein edge refinement detection method that integrates cross-self-attention. This method first extracts local details and global features through a feature extraction unit, then uses a feature fusion unit to aggregate features of different scales in space and channels, and finally uses a feature refinement unit to specifically refine the uncertain points in the preliminary prediction results, thereby achieving accurate segmentation of the palm vein edges. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 A structural diagram of a palm vein edge thinning detection network provided by an embodiment of the present invention;

[0024] Figure 2 A structural diagram of a dual-cross attention Transformer module provided by one embodiment of the present invention;

[0025] Figure 3 A structural diagram of channel cross attention provided by one embodiment of the present invention;

[0026] Figure 4 A structural diagram of spatial cross attention provided by one embodiment of the present invention;

[0027] Figure 5 A structural diagram of a multi-scale feature fusion module provided in one embodiment of the present invention;

[0028] Figure 6 A structural diagram of a channel feature fusion module provided in one embodiment of the present invention;

[0029] Figure 7 A structural diagram of a feature refinement unit provided in one embodiment of the present invention;

[0030] Figure 8 A comparison diagram of the original image and edge information provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following describes the embodiments of the present invention through specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. It should be noted that the following embodiments and the features in the embodiments can be combined with each other unless they conflict. In addition to the specific methods, equipment, and materials used in the embodiments, based on the understanding of the prior art by those skilled in the art and the description of the present invention, any methods, equipment, and materials of the prior art that are similar or equivalent to the methods, equipment, and materials described in the embodiments of the present invention can also be used to implement the present invention.

[0032] It should be understood that the terms used in the examples of the present invention are for describing specific embodiments rather than for limiting the scope of protection of the present invention. Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those generally understood by those skilled in the art.

[0033] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In some of the embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0034] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions and operations that may be implemented by the methods and computer program products of various embodiments disclosed in the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0035] It should be noted that the subscripts such as i, j and some count values ​​N used in this application have different meanings in different paragraphs. They only serve as subscript identification or counting functions, and unless otherwise specified, they do not represent the same physical quantity.

[0036] It should be noted that in this application, the same feature is recorded as both the output feature of module A and the input feature of module B. This is because the feature output by module A is input into module B, so these two features refer to the same feature. Furthermore, for module A, which includes two submodules A1 and A2, the output feature of module A and the output feature of module A2 also refer to the same feature.

[0037] See Figure 1 , Figure 1 A palm vein edge thinning detection method integrating cross self-attention is provided in one embodiment of the present invention, comprising the following steps: obtaining a palm vein image to be processed; inputting the palm vein image to be processed into a trained palm vein edge thinning detection network to obtain edge information.

[0038] In the above steps, the first step is to obtain the palm vein image to be processed. This image can be a raw image captured by the palm vein detection device. Before inputting it into the trained palm vein edge thinning detection network, some preprocessing is required to make it match the size supported by the detection network.

[0039] In the above steps, the palm vein edge refinement detection network needs to be built in advance. The expression of the palm vein edge refinement detection network is as follows: out =H FRU (H FFU (H FEU (Iin ))), where H FEU is a feature extraction unit used to extract local details and global structures, H FFU It is a feature fusion unit used to fuse features of different scales in space and channels. FRU It is a feature refinement unit used to segment the fused features.

[0040] In the above steps, the original image and its corresponding edge information can be obtained as training samples. The edge information is obtained by manually annotating the original image. In order to increase the number of data sets, operations such as cropping and rotation can be used. The original image and its corresponding edge information are converted into a tensor format and encapsulated into a training data set. By adjusting the program parameters, the total number of image pairs and the number of each batch can be flexibly controlled to ensure that the original image is used as the network input, and the edge information is used to supervise the training process. The prepared data set can effectively support the training of the palm vein edge refinement detection network, thereby obtaining a trained palm vein edge refinement detection network.

[0041] See Figure 1 In one embodiment of the present invention, the feature extraction unit is composed of a cascade of an overlapping patch extraction module and four dual-cross-attention Transformer modules. The overlapping patch extraction module is mainly used to divide patches and reduce the computational complexity of image processing; the dual-cross-attention Transformer module is mainly used to extract local details and global structure.

[0042] The cascade mentioned in this embodiment can also be understood as follows: the input features of the first double cross-attention Transformer module are the output features of the overlapping patch extraction module, and the input features of the j-th double cross-attention Transformer module are the output features of the j-1-th double cross-attention Transformer module, where j∈{2,3,4}. The output features of the four double cross-attention Transformer modules are simultaneously output to the feature fusion unit for fusion.

[0043] In a specific embodiment of the present invention, the expression of the overlapping patch extraction module is as follows: i =DConv1D 1×1 (Reshape(AvgPool2D(E i ))), where E iThe overlapping patch area obtained by dividing the palm vein image to be processed. AvgPool2D is a two-dimensional average pooling, which is a simple but effective downsampling operation that compresses the spatial size of the feature map by taking the average of the local area while retaining the main feature information. Reshape is reshaping, which usually refers to reshaping the dimensions of a tensor or array without changing its data content. DConv1D 1×1 is a one-dimensional depth-wise separable convolution with a convolution kernel size of 1×1, P i is the feature after the i-th mapping. In this step, the input image is divided into overlapping patch areas E in the overlapping patch extraction module. i , each patch is divided from different parts of the input image, and its pixel values ​​and feature distribution are different. The mapped feature P i ∈R p×Ci , p represents the total number of features extracted from the patch, C i Represents the i-th channel, each feature P i The total number of feature markers p is the same.

[0044] Each feature P output by the overlapping patch extraction module i They are processed by a series of double cross-attention Transformer modules to extract and cross-feature multiple overlapping patches.

[0045] See Figure 2 In a specific embodiment of the present invention, the dual cross attention Transformer module is composed of a cascade of a dual cross attention layer and a hybrid feedforward layer. The expression of the dual cross attention layer is as follows: X2=H SCA (H CCA (X1)+X1)+H CCA (X1)+X1, where X1 is the input feature of the double cross attention Transformer module. For the first double cross attention Transformer module, its input feature is the output feature of the overlapping patch extraction module. For other double cross attention Transformer modules, its input feature is the output feature of the previous double cross attention Transformer module. H CCA is the channel cross attention, H SCA is the spatial cross attention, and X2 is the output feature of the double cross attention layer.

[0046] See Figure 3 In a specific embodiment of the present invention, the channel cross attention is processed as follows: (1) for each mapped feature P i Perform layer normalization to obtain feature P i', layer normalization processing is Figure 3 LayerNorm in; (2) for feature P i 'Splice along the channel dimension to obtain feature P C , the splicing operation corresponds to Figure 3 Concat in; (3) Calculate the key, value, and query according to the following formula: K T =DConv1D K (P C ), V=DConv1D V (P C ), Q i =DConv1D Q (P i '); (4) Based on the key, value and query, the output features of the channel cross attention are calculated according to the following formula:

[0047] ,

[0048] Among them, Q i is the i-th query matrix, K is the key, K T is its transpose, V is the value, C C is the dimension of the key, DConv1D is the depthwise separable convolution, and Softmax is the normalization.

[0049] See Figure 4 In a specific embodiment of the present invention, spatial cross attention is processed as follows: (1) the output feature P of channel cross attention is processed. i Perform layer normalization to obtain feature P i '; (2) for feature P i 'Splice along the channel dimension to obtain feature P C ; (3) Calculate the key, value, and query according to the following formula: K T =DConv1D K (P C ), V i =DConv1D V (P i '), Q = DConv1D Q (P C ); (4) Based on the key, value, and query, the output features of the channel cross attention are calculated according to the following formula:

[0050] ,

[0051] Among them, Q is the query, K is the key, K T is its transpose, V i is the i-th value matrix, C Cis the dimension of the key, DConv1D is the depthwise separable convolution, and Softmax is the normalization.

[0052] See Figure 2 In a specific embodiment of the present invention, the expression of the mixed feedforward layer is as follows: X3=GELU(Conv 3×3 (MLP(X2)))+X2, where X2 is the output feature of the double cross attention layer, MLP is the multi-layer perceptron, Conv 3×3 is a convolution with a kernel size of 1×1, GELU is the activation function, and X3 is the output feature of the dual cross-attention Transformer module. Since the present invention involves four dual cross-attention Transformer modules, there are four corresponding outputs, which can be recorded as X 31 ~X 34 .

[0053] See Figure 1 In a specific embodiment of the present invention, the feature fusion unit includes a cascaded multi-scale feature fusion module and a channel feature fusion module, wherein the multi-scale feature fusion module is used to perform spatial feature enhancement on the output features of the four dual cross-attention Transformer modules and integrate features of different receptive fields through feature splicing; the channel feature fusion module is used to perform feature fusion on the channel.

[0054] See Figure 5 In a specific embodiment of the present invention, the expression of the multi-scale feature fusion module is as follows: j =Conv 1×1 X 3j +Conv 3×3 X 3j +…+Conv n×n X 3j , X4=Y1+Upsample(Y2,Y3,Y4), where j∈{1,2,3,4}, X 3j is the output feature of the j-th double cross attention Transformer module, Conv n×n is a convolution with a kernel size of n×n, Y j is the j-th multi-scale feature output, Upsample is upsampling, and X4 is the output feature of the multi-scale feature fusion module.

[0055] In this embodiment, the multi-scale feature fusion module performs spatial feature enhancement on the input features. The module uses four dilated convolutions of different scales to capture the multi-scale features Y j, to better capture the diversity of the target. The above n is the convolution scale, an odd number, up to 7. After the dilated convolution, upsampling is used to unify the feature maps of different scales. In the formula in the previous paragraph, Upsample(Y2,Y3,Y4) indicates that features Y2, Y3, and Y4 are upsampled, respectively, to H / 4 × W / 4. The complete formula for X4 is: X4 = Y1 + Upsample(Y2) + Upsample(Y3) + Upsample(Y4).

[0056] See Figure 6 In a specific embodiment of the present invention, the expression of the channel feature fusion module is as follows: j =Softmax(Sigmoid(GAP(X 4j ))), X5=Conv 1×1 (Channel Shuffle(w1·X 41 +w2·X 42 +w3·X 43 +w4·X 44 ), where j∈{1,2,3,4}, X 4j is obtained by dividing the output features of the multi-scale feature fusion module into channels, GAP is global average pooling, Sigmoid is the Sigmoid function, Softmax is normalization, and w j is the weight vector, ChannelShuffle is the channel shuffle, and X5 is the output feature of the channel feature fusion module.

[0057] In this embodiment, the feature X4 obtained in the previous step is first divided into four channels, so that the feature X4 is divided into four different channel features. 41 ~X 44 Then calculate the weight vector w according to the first formula j GAP (Global Average Pooling) is a pooling operation commonly used in convolutional neural networks (CNNs). Its main function is to reduce the spatial dimension of the feature map while retaining the channel dimension information. Through the global average pooling operation, the feature X 41 ~X 44 Compressed to 1×1×C. The Sigmoid function maps input values ​​to the range (0, 1) and is commonly used in binary classification problems or as an activation function for probabilistic output. Softmax converts a set of input values ​​into a probability distribution, where the sum of the output values ​​is 1, i.e., w1+w2+w3+w4=1.

[0058] After getting the weight vector w jAfter that, multiply the weight vector by the corresponding original feature, that is, w j With X 4j Multiplication and channel shuffling are performed to interweave and fuse features from different channels. Channel shuffling is an operation used in convolutional neural networks (CNNs) to enhance feature interaction. Its core idea is to rearrange the order of channels in feature maps to facilitate information exchange between different channels, thereby improving the model's expressive power. To reduce the scale of the feature map after channel shuffling, 1×1 convolution is used for compression. Finally, the resulting fused feature map is X5.

[0059] See Figure 7 In a specific embodiment of the present invention, the feature refinement unit includes multiple groups of cascaded coarse prediction modules and fine prediction modules; the coarse prediction module is used to make a rough prediction of the fused features, and the fine prediction module is used to perform targeted refinement on the uncertain points in the coarse prediction results. It should be noted that a coarse prediction module and a fine prediction module constitute a group, and the feature refinement unit is composed of multiple groups. Figure 7 “×N” is added to indicate the concept of multiple groups.

[0060] In a specific embodiment of the present invention, the expression of the rough prediction module is as follows: X6=Sigmoid(Conv 3×3 (BilinearUpsampling(X5))), where X5 is the output feature of the feature fusion unit, BilinearUpsampling is bilinear interpolation upsampling, Conv 3×3 is a convolution with a kernel size of 3×3, Sigmoid is the sigmoid function, and X6 is the output feature of the coarse prediction module. Bilinear upsampling is a commonly used upsampling method, mainly used to amplify low-resolution feature maps to high resolution. Its core idea is to interpolate the input feature map through bilinear interpolation to produce a smooth, high-resolution output.

[0061] In this embodiment, the input features are first scaled up using bilinear interpolation upsampling, and then a 3×3 convolution kernel is used to perform a convolution operation on the scaled features. Finally, a Sigmoid function is used to generate an edge prediction graph X6, which is a preliminary prediction result.

[0062] In a specific embodiment of the present invention, the fine prediction module includes a cascade of dynamic convolution layers, bilinear interpolation upsampling and a multi-layer perceptron, which is used to select the first β×N most uncertain true value points and perform local optimization on these points.

[0063] In a specific embodiment of the present invention, the dynamic convolution layer is processed as follows: (1) randomly generating k×N points p in the rough prediction result i , i=1,2,…,k*N, each point p i is a two-dimensional coordinate (x i ,y i ), for example, k can be set to 3 and N can be set to 142. (2) Extract each point p i The surrounding local feature map X pi , and use a 1×1 convolution layer to generate a set of dynamic convolution kernels K for the local information of each input position pi , which enhances activation in high uncertainty areas and suppresses contribution in low uncertainty areas. Its expression is: K pi =Conv 1×1 (X pi ). (3) Dynamic convolution kernel K pi Convolve with the preliminary prediction map to obtain feature Y pi (4)Calculate feature Y pi The uncertainty of each point in the equation is u i , its expression is as follows: i =|0.5-p(c|Y pi )|, where p(c|Y pi ) is at position p i The predicted probability of the true label (0 or 1) on . (5) After sorting, select the top β×N most uncertain true value points. i By sorting all the points, we can select the first β×N most uncertain true value points, and the remaining (1-β)×N points are selected according to a uniform distribution. Here, β can be 0.75, for example.

[0064] Subsequently, bilinear interpolation upsampling is used to perform bilinear interpolation on the four nearest neighboring points of the first β×N points on the f regular grid in turn to calculate the features of the true value points. Then, sub-pixel information is encoded along the channel dimension to predict more accurate segmentation results.

[0065] In MLP, these points are input into a trained MLP model, and the output points are used to replace the uncertain points in the rough prediction results to obtain the output after the first round of feature refinement.

[0066] The training process of this MLP model is as follows: the input point passes through 3 hidden layers, and K-class predictions are made at the selected points to obtain the predicted labels. Then, the loss is calculated using the binary cross entropy loss function, which is expressed as follows:

[0067] , where N is the number of samples, y iis the true label of the i-th pixel (0 or 1), is the predicted probability of the i-th pixel. Gradients are calculated based on the loss and propagated back to the network using the backpropagation algorithm. The weights in the network are adjusted iteratively until the loss is minimized.

[0068] Because the feature map size output by the feature fusion unit is H / 4×W / 4×C1, which is much smaller than the original input image size H×W×C1, in order to make the size consistent, the feature refinement unit will iteratively run until the final output size is completely consistent with the input image.

[0069] After the first pass through the fine prediction module, we get the detection results of this time, and use the loss function to calculate the measurement of the detection results I out With real data I real The error between .

[0070] Subsequently, we use a non-iterative strategy based on random sampling to randomly modify the parameters used in the training process, generating a total of N sets of candidate parameters. For each set of candidate parameters, we use the loss function to recalculate the error between the test results of that set and the real data, and record the error value of each set. The set of parameters with the smallest error value will be used as the final model training parameters, and the test results of that set I out This is the final edge detection result.

[0071] The loss function we use is composed of the sum of two parts, and the specific expression formula is as follows: L loss =L coarse +L point , where the first part is to use the binary cross entropy loss function to calculate the loss of the coarse prediction module to ensure that it can generate preliminary low-resolution mask predictions. The specific expression is as follows:

[0072] , where N is the number of pixels in the target box, is the true label of the i-th pixel (0 or 1), is the corresponding predicted probability.

[0073] The second part also uses the binary cross entropy loss function to calculate the loss of the fine prediction module to ensure that it can generate high-resolution fine segmentation predictions. Its expression is as follows:

[0074] ,

[0075] Where M is the number of pixels in the target box, is the true label of the j-th pixel (0 or 1), is the corresponding predicted probability.

[0076] It should be noted that the step division of the various methods above is only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they contain the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0077] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A palm vein edge refinement detection method integrating cross self-attention, characterized in that: include: Acquiring a palm vein image to be processed; The palm vein image to be processed is input into the trained palm vein edge thinning detection network to obtain edge information. The expression of the palm vein edge thinning detection network is as follows: I out =H FRU (H FFU (H FEU (I in ))), Among them, H FEU is a feature extraction unit used to extract local details and global structures, H FFU It is a feature fusion unit used to fuse features of different scales in space and channels. FRU It is a feature refinement unit used to segment the fused features; The feature refinement unit includes multiple groups of cascaded rough prediction modules and fine prediction modules; the rough prediction module is used to make a rough prediction of the fused features, and the fine prediction module is used to perform targeted refinement on uncertain points in the rough prediction results; The expression of the rough prediction module is as follows: X6=Sigmoid(Conv 3×3 (BilinearUpsampling(X5))), Among them, X5 is the output feature of the feature fusion unit, BilinearUpsampling is bilinear interpolation upsampling, Conv 3×3 is a convolution with a convolution kernel size of 3×3, Sigmoid is a Sigmoid function, and X6 is the output feature of the coarse prediction module; The fine prediction module includes a cascade of dynamic convolution layers, bilinear interpolation upsampling and a multi-layer perceptron; The dynamic convolution layer is processed as follows: Randomly generate k×N points p in the rough prediction result i , i=1,2,…,kN; Extract each point p i The local feature map around the input position is obtained by using a 1×1 convolution layer to generate a set of dynamic convolution kernels K for the local information of each input position. pi ; The dynamic convolution kernel K pi Convolve with the preliminary prediction map to obtain feature Y pi ; Calculate feature Y pi The uncertainty of each point in , and after sorting, select the top β×N most uncertain true value points.

2. The palm vein edge refinement detection method integrating cross self-attention according to claim 1 is characterized in that: The feature extraction unit is composed of an overlapping patch extraction module and four double cross-attention Transformer modules cascaded.

3. The palm vein edge refinement detection method integrating cross self-attention according to claim 2 is characterized in that: The expression of the overlapping patch extraction module is as follows: P i =DConv1D 1×1 (Reshape(AvgPool2D(E i ))), Among them, E i To divide the overlapping patch area obtained by the palm vein image to be processed, AvgPool2D is a two-dimensional average pooling, Reshape is reshaping, and DConv1D 1×1 is a one-dimensional depth-wise separable convolution with a convolution kernel size of 1×1, P i is the feature after the i-th mapping.

4. The palm vein edge refinement detection method integrating cross self-attention according to claim 2, characterized in that: The dual cross-attention Transformer module is composed of a cascade of a dual cross-attention layer and a hybrid feed-forward layer. The expressions of the dual cross-attention layer and the hybrid feed-forward layer are as follows: X2=H SCA (H CCA (X1)+X1)+H CCA (X1)+X1, X3=GELU(Conv 3×3 (MLP(X2)))+X2, Among them, X1 is the input feature of the dual cross attention Transformer module, H CCA is the channel cross attention, H SCA is the spatial cross attention, X2 is the output feature of the double cross attention layer, MLP is the multi-layer perceptron, Conv 3×3 is a convolution with a kernel size of 1×1, GELU is the activation function, and X3 is the output feature of the dual cross attention Transformer module.

5. The palm vein edge refinement detection method integrating cross self-attention according to claim 2, characterized in that: The feature fusion unit includes a cascade of: A multi-scale feature fusion module is used to perform spatial feature enhancement on the output features of the four dual-cross attention Transformer modules and integrate features of different receptive fields through feature splicing; Channel feature fusion module is used to perform feature fusion on channels.

6. The palm vein edge refinement detection method integrating cross self-attention according to claim 5, characterized in that: The expression of the multi-scale feature fusion module is as follows: Y j =Conv 1×1 X 3j +Conv 3×3 X 3j +…+Conv n×n X 3j , X4=Y1+Upsample(Y2,Y3,Y4), Among them, j∈{1,2,3,4}, X 3j is the output feature of the j-th double cross attention Transformer module, Conv n×n is a convolution with a kernel size of n×n, Y j is the j-th multi-scale feature output, Upsample is upsampling, and X4 is the output feature of the multi-scale feature fusion module.

7. The palm vein edge refinement detection method integrating cross self-attention according to claim 5, characterized in that: The expression of the channel feature fusion module is as follows: w j =Softmax(Sigmoid(GAP(X 4j ))), X5=Conv 1×1 (Channel Shuffle(w1·X 41 +w2·X 42 +w3·X 43 +w4·X 44 )), Among them, j∈{1,2,3,4}, X 4j is obtained by dividing the output features of the multi-scale feature fusion module into channels, GAP is global average pooling, Sigmoid is the Sigmoid function, Softmax is normalization, and w j is the weight vector, ChannelShuffle is channel shuffle, and X5 is the output feature of the channel feature fusion module.

Images