Image segmentation method and image segmentation system

Abstract

The present application relates to the field of image segmentation technology, and discloses an image segmentation method and an image segmentation system. The method comprises: collecting a user's venous 3D vascular image through a preset smart door lock, and performing standardized filtering and contrast enhancement processing on the venous 3D vascular image to obtain a preprocessed 3D vascular image; performing multi-scale convolution and layer attention mechanism analysis on the preprocessed 3D vascular image to obtain a 3D feature representation; performing an adaptive 3D to 2D projection algorithm on the 3D feature representation to obtain a 2D vascular feature map; using a cross-scale cascaded hierarchical transformer network to perform image segmentation on the 2D vascular feature map to obtain an initial 2D segmentation result; performing reverse boundary attention optimization on the initial 2D segmentation result to obtain an optimized 2D segmentation map; performing cross-entropy loss function evaluation and optimization on the optimized 2D segmentation map, and outputting a target 2D segmentation result. The present application improves the accuracy of image segmentation.

Description

Technical Field

[0001] The present application relates to the technical field of image segmentation, and in particular to an image segmentation method and an image segmentation system. Background Art

[0002] As an important method of biometric identification, vein recognition technology has been widely used in the security field. With the popularity of smart door locks, combining vein recognition technology with smart door locks has become a new research hotspot. However, traditional vein recognition methods are mainly based on 2D images, which makes it difficult to capture the three-dimensional structure information of blood vessels, resulting in limited recognition accuracy.

[0003] In recent years, the development of 3D vein image acquisition technology has provided new possibilities for improving recognition accuracy. However, the processing and analysis of 3D vein images face multiple challenges. The amount of 3D image data is large, and the direct processing computational complexity is high; secondly, the complexity of vascular structure and individual differences make accurate segmentation difficult; how to effectively convert 3D information into 2D features to adapt to existing recognition algorithms is also a problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides an image segmentation method and an image segmentation system, which are used to improve the accuracy of image segmentation.

[0005] In a first aspect, the present application provides an image segmentation method, the image segmentation method comprising:

[0006] The user's venous 3D blood vessel image is collected through a preset smart door lock, and the venous 3D blood vessel image is subjected to standardization filtering and contrast enhancement processing to obtain a preprocessed 3D blood vessel image;

[0007] Performing multi-scale convolution and layer attention mechanism analysis on the preprocessed 3D blood vessel image to obtain a 3D feature representation;

[0008] performing an adaptive 3D to 2D projection algorithm on the 3D feature representation to obtain a 2D blood vessel feature map;

[0009] Using a cross-scale cascaded hierarchical transformer network to perform image segmentation on the 2D blood vessel feature map to obtain an initial 2D segmentation result;

[0010] Performing reverse boundary attention optimization on the initial 2D segmentation result to obtain an optimized 2D segmentation map;

[0011] The optimized 2D segmentation map is evaluated and optimized using a cross entropy loss function, and a target 2D segmentation result is output.

[0012] In a second aspect, the present application provides an image segmentation system, the image segmentation system comprising:

[0013] An acquisition module is used to acquire a 3D venous blood vessel image of a user through a preset smart door lock, and perform standardization filtering and contrast enhancement processing on the 3D venous blood vessel image to obtain a preprocessed 3D blood vessel image;

[0014] An analysis module, configured to perform multi-scale convolution and layer attention mechanism analysis on the pre-processed 3D vascular image to obtain a 3D feature representation;

[0015] An execution module, configured to execute an adaptive 3D to 2D projection algorithm on the 3D feature representation to obtain a 2D blood vessel feature map;

[0016] A segmentation module, used for performing image segmentation on the 2D blood vessel feature map by using a cross-scale cascaded hierarchical transformer network to obtain an initial 2D segmentation result;

[0017] An optimization module, configured to perform reverse boundary attention optimization on the initial 2D segmentation result to obtain an optimized 2D segmentation map;

[0018] The output module is used to evaluate and optimize the cross entropy loss function of the optimized 2D segmentation map and output the target 2D segmentation result.

[0019] The third aspect of the present application provides an image segmentation device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the image segmentation device performs the above-mentioned image segmentation method.

[0020] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned image segmentation method.

[0021] In the technical solution provided by this application, the combination of multi-scale convolution and layer attention mechanism improves the feature extraction capability of 3D vascular structure and can better capture vascular information of different scales. The adaptive 3D to 2D projection algorithm achieves dimensionality reduction while retaining 3D structural information, reducing the computational complexity of subsequent processing. The application of cross-scale cascaded hierarchical transformer network enhances the model's learning and fusion capabilities for multi-scale features and improves the accuracy of segmentation. The reverse boundary attention optimization mechanism pays special attention to the vascular boundary area and improves the boundary accuracy of the segmentation results. The improved cross entropy loss function better guides the optimization process of the model by introducing deterministic observation samples and boundary sensitive items. The full process from 3D image acquisition to 2D segmentation result output is realized, which improves the overall efficiency and accuracy of vein recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0023] Figure 1 A schematic diagram of an embodiment of an image segmentation method in an embodiment of the present application;

[0024] Figure 2 Schematic diagram of an embodiment of an image segmentation system in an embodiment of the present application. DETAILED DESCRIPTION

[0025] Embodiments of the present application provide an image segmentation method and an image segmentation system. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of the image segmentation method in the embodiment of the present application includes:

[0027] Step S101, collecting a 3D vascular image of a user's veins through a preset smart door lock, and performing standardization filtering and contrast enhancement processing on the 3D vascular image of the veins to obtain a preprocessed 3D vascular image;

[0028] It is understandable that the execution subject of the present application may be an image segmentation system, or a terminal or a server, which is not limited here. The present application embodiment is described by taking a server as the execution subject as an example.

[0029] Specifically, the user's venous 3D vascular image is collected through a preset smart door lock. The smart door lock is equipped with a high-precision imaging device that can obtain vascular image data. The pixel value of the venous 3D vascular image is normalized to ensure the consistency of the image data and reduce the deviation under different acquisition conditions. A three-dimensional Gaussian filtering operation is performed to smooth the image, reduce the influence of noise, and retain the main structural information. The three-dimensional histogram equalization technology is applied to enhance the contrast of the image, making the details of the blood vessels more obvious and improving the effect of subsequent processing. The optimal segmentation threshold of the contrast-enhanced 3D vascular image is calculated, and a preliminary segmented 3D vascular image is generated based on it. The vascular area is preliminarily extracted through an automated segmentation algorithm. A three-dimensional morphological opening operation is performed on the preliminary segmentation result to remove small noise points and obtain a denoised 3D vascular image. A three-dimensional morphological closing operation is performed on the denoised image to fill in the small holes that may exist in the vascular area, generate a filled 3D vascular image, and make the vascular area more complete. The filled 3D vascular image is used as a mask to perform a mask operation on the contrast-enhanced 3D vascular image to obtain a 3D image of the vascular area. The 3D image of the vascular region is subjected to edge-preserving filtering to generate an edge-enhanced 3D vascular image. The local contrast map of the edge-enhanced 3D vascular image is calculated and adaptively enhanced to make the details of the image more prominent, thereby obtaining a detail-enhanced 3D vascular image. The detail-enhanced 3D vascular image is standardized to ensure the consistency and reliability of the image data in subsequent analysis, thereby obtaining a preprocessed 3D vascular image.

[0030] Step S102, performing multi-scale convolution and layer attention mechanism analysis on the pre-processed 3D blood vessel image to obtain a 3D feature representation;

[0031] Specifically, 3D convolution kernels of 3x3x3, 5x5x5 and 7x7x7 are applied to the preprocessed 3D vascular image to perform convolution operations and obtain multi-scale initial feature maps. Through convolution kernels of different sizes, features of different scales are captured, and detailed information and overall structure in the image are effectively extracted. The multi-scale initial feature map is processed by batch normalization and ReLU activation function to obtain the activated multi-scale feature map, and the hierarchical feature weight matrix is ​​constructed according to the activated multi-scale feature map. Batch normalization helps to improve the stability and training speed of the model, while the ReLU activation function introduces nonlinearity, so that the model can better fit the complex data distribution. The hierarchical feature weight matrix and the activated multi-scale feature map are multiplied element by element to obtain the weighted multi-scale feature map. The weighted multi-scale feature map is spliced ​​in the channel dimension to obtain the multi-scale fusion feature map. Through the splicing operation, the feature maps of different scales are integrated together. The 1x1x1 convolution kernel is applied to the multi-scale fusion feature map to perform feature compression processing to obtain the compressed 3D feature map. The 1x1x1 convolution kernel is used to adjust the number of channels of the feature map, which plays a role in feature compression and dimensionality reduction, and retains important information. The product of the spatial attention map and the channel attention map is calculated based on the compressed 3D feature map to obtain the comprehensive attention map. The spatial attention map and the channel attention map focus on important positions and important channels in the image respectively. Through their product, the information of space and channels is comprehensively considered to improve the accuracy of feature representation. The comprehensive attention map and the compressed 3D feature map are element-by-element multiplication to obtain the attention-enhanced 3D feature map. The attention-enhanced 3D feature map is residually connected with the multi-scale fusion feature map to obtain the residual 3D feature map. The residual connection solves the problem of gradient disappearance in the deep network by retaining the information of the original feature map and enhances the training effect of the model. The residual 3D feature map is processed by adaptive global average pooling and fully connected layer to obtain the 3D feature representation. Adaptive global average pooling extracts the global features of the image by performing global average operation on the feature map, reduces the number of parameters, and prevents overfitting; the fully connected layer maps the extracted features to the output space to obtain the final 3D feature representation.

[0032] Step S103, performing an adaptive 3D to 2D projection algorithm on the 3D feature representation to obtain a 2D blood vessel feature map;

[0033] Specifically, principal component analysis is performed on the 3D feature representation to obtain the main direction vector of the blood vessel, and the 3D feature representation is spatially aligned according to the main direction vector of the blood vessel to obtain an aligned 3D feature representation. Principal component analysis can effectively extract the main direction in the data, and the main direction of the blood vessel is unified through the alignment operation. 3D sliding window processing is performed on the aligned 3D feature representation to obtain a local feature block set, and a projection weight function is generated based on the local feature block set. 3D sliding window processing can extract detailed information of the local area, and the projection weight function determines the importance of each area during projection based on the statistical characteristics of these local feature block sets. The aligned 3D feature representation is processed by the projection weight function to obtain a weighted 3D feature body. The maximum intensity projection is calculated for the weighted 3D feature body along the optimal projection direction to obtain a MIP feature map. Maximum intensity projection (MIP) can select the pixel with the highest intensity in each projection direction, thereby retaining the significant features in the image. At the same time, the weighted average projection is calculated for the weighted 3D feature body along the optimal projection direction to obtain a WAP feature map. Weighted average projection (WAP) calculates the weighted average value and comprehensively considers the intensity distribution of different regions to make the projection result smoother and more balanced. The MIP feature map and the WAP feature map are fused by channel attention to obtain a fused 2D feature map. The fused 2D feature map is encoded with depth information to obtain a depth-aware 2D feature map. Depth information encoding can capture the deep hierarchical structure in the image, making the feature map more three-dimensional and hierarchical. The depth-aware 2D feature map is subjected to residual enhancement and nonlinear transformation to obtain a 2D vascular feature map. Residual enhancement retains the information of the original features and avoids information loss by introducing residual connections, while nonlinear transformation increases the expressiveness of the model, allowing the feature map to better adapt to complex vascular structures.

[0034] Step S104, using a cross-scale cascaded hierarchical transformer network to perform image segmentation on the 2D blood vessel feature map to obtain an initial 2D segmentation result;

[0035] Specifically, a multi-scale convolution operation is performed on the 2D vascular feature map to obtain a multi-scale feature pyramid. Different levels of information of the image are captured by convolution kernels of different scales to obtain a multi-scale feature pyramid. A local-global cross-scale transformation is performed on the multi-scale feature pyramid to obtain a mixed-scale feature, ensuring that global structural information is retained while extracting local details. A multi-head hierarchical cross-scale adaptive embedding analysis is performed on the mixed-scale feature to obtain a cross-scale embedded feature. The multi-head hierarchical cross-scale adaptive embedding analysis captures important information in the feature map through multiple self-attention heads, enhancing the model's ability to represent complex structures. Local context processing is performed on the cross-scale embedded feature to obtain a context-enhanced feature. The local context processing can combine local and global information of the image to improve the semantic expression ability of the feature map. The context-enhanced feature is subjected to a cross-scale transformer cascade process to obtain a multi-level transformer feature. Through the cascade operation of the transformer network, the multi-level information in the feature map is captured, and the ability to represent complex structures is improved. A jump connection block is constructed based on the multi-level transformer feature to obtain a multi-scale fusion feature. The jump connection block ensures the integrity and continuity of information transmission by connecting features of different scales across layers, thereby improving the effect of feature fusion. The multi-scale fusion features are interactively fused by channel tokens to obtain interactive enhancement features. Channel token interactive fusion makes the expression of feature maps more comprehensive and balanced through information exchange between different channels. A decoding module is constructed based on the interactive enhancement features to obtain a preliminary decoding feature map. The decoding module generates a preliminary decoding feature map by gradually restoring the spatial resolution of the feature map. The preliminary decoding feature map is subjected to cross-scale position transformation to obtain a position-aware feature map. The cross-scale position transformation adjusts the positional relationship of the feature map so that the model can better understand the spatial structure of the image. The position-aware feature map is subjected to a classification convolution operation to obtain the initial 2D segmentation result. The classification convolution operation generates the final segmentation result by pixel-by-pixel classification.

[0036] Step S105, performing reverse boundary attention optimization on the initial 2D segmentation result to obtain an optimized 2D segmentation map;

[0037] Specifically, edge detection is performed on the initial 2D segmentation result to obtain an initial boundary map. Edge detection outlines the edges of blood vessels in the image by detecting the parts of the image where the pixel intensity changes significantly. The pixel-level significance score is calculated according to the initial boundary map to obtain a significance distribution map. The significance distribution map is reverse mapped to obtain a reverse attention weight map. The reverse mapping converts the significance distribution map into a weight map, so that different regions can be treated differently by these weights in the subsequent processing. A boundary attention network is constructed according to the reverse attention weight map to generate a boundary enhancement weight matrix. The boundary attention network generates a weight matrix for enhancing boundary features by performing deep learning analysis on the significance weight map, so that the boundary area receives higher attention in the subsequent processing. The initial 2D segmentation result and the 2D blood vessel feature map are channel-joined to obtain a multimodal fusion feature map. The multimodal fusion feature map is weighted according to the boundary enhancement weight matrix to obtain a boundary enhancement feature map. The boundary enhancement feature map is subjected to boundary-aware convolution to obtain a refined boundary feature map. Boundary-aware convolution refines and strengthens boundary features by performing convolution operations on feature maps, making the boundaries of image segmentation clearer and more accurate. An iterative boundary optimization module is constructed based on the refined boundary feature map to generate an optimized boundary feature map. The iterative boundary optimization module gradually optimizes boundary features through multiple iterative processes. A residual connection is performed on the optimized boundary feature map and the initial 2D segmentation result to obtain a residual fusion segmentation map. The residual connection combines the optimized boundary feature map with the initial segmentation result to retain the effective information in the initial result, while introducing the optimized boundary features to improve the overall segmentation effect. A boundary correction convolution operation is performed on the residual fusion segmentation map to obtain an optimized 2D segmentation map.

[0038] Step S106: Evaluate and optimize the cross entropy loss function on the optimized 2D segmentation map, and output the target 2D segmentation result.

[0039] Specifically, the pixel-level cross entropy is calculated for the optimized 2D segmentation map and the true label image to obtain the basic cross entropy loss value. The pixel-level cross entropy evaluates the accuracy of the segmentation result by comparing the predicted value of each pixel with the true value. The deterministic observation samples are calculated according to the true label image to obtain the sample weight matrix. The deterministic observation samples identify the samples with the most reference value for model training by analyzing the true label image, and construct a sample weight matrix so that important samples occupy a larger weight in the loss calculation. The basic cross entropy loss value and the sample weight matrix are weighted to obtain the weighted cross entropy loss value. The weighting operation enables the model to pay more attention to important samples by assigning different weights to different samples, thereby improving the efficiency and effect of training. The distance transformation map is calculated according to the optimized 2D segmentation map to obtain the boundary distance weight map. The distance transformation map generates a weight map reflecting the boundary information by calculating the distance from each pixel to the nearest boundary, so that the boundary area receives more attention in the loss calculation. The boundary distance weight map and the weighted cross entropy loss value are fused to obtain the boundary sensitive loss value. The weights of the loss components are dynamically adjusted according to the training stage and verification performance to obtain the adaptive comprehensive loss function. The adaptive comprehensive loss function dynamically adjusts the weights of different loss components according to the performance during the training process, so that the model can optimize different goals at different training stages. The back propagation algorithm is executed on the adaptive comprehensive loss function to obtain the gradient update value. The back propagation algorithm guides the update direction and amplitude of the model parameters by calculating the gradient of the loss function, and gradually optimizes the model performance. According to the gradient update value, the network parameters of the cross-scale cascaded hierarchical transformer network are optimized and adjusted to obtain the updated network model. Through repeated iterations, the parameter optimization enables the model to gradually approach the optimal solution and improve the segmentation effect. The 2D vascular feature map is inferred through the updated network model to obtain the target 2D segmentation result. The inference process generates the final segmentation result by applying the optimized model to the actual data, ensuring the accuracy and reliability of the segmentation result.

[0040] In the embodiment of the present application, the combination of multi-scale convolution and layer attention mechanism improves the feature extraction capability of 3D vascular structure and can better capture vascular information of different scales. The adaptive 3D to 2D projection algorithm achieves dimensionality reduction while retaining 3D structural information, reducing the computational complexity of subsequent processing. The application of cross-scale cascaded hierarchical transformer network enhances the model's learning and fusion capabilities for multi-scale features and improves the accuracy of segmentation. The reverse boundary attention optimization mechanism pays special attention to the vascular boundary area and improves the boundary accuracy of the segmentation result. The improved cross entropy loss function better guides the optimization process of the model by introducing deterministic observation samples and boundary sensitive items. The full process processing from 3D image acquisition to 2D segmentation result output is realized, which improves the overall efficiency and accuracy of vein recognition.

[0041] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0042] (1) The user's venous 3D vascular image is collected through a preset smart door lock, and the venous 3D vascular image is normalized by pixel values ​​to obtain a normalized 3D vascular image;

[0043] (2) performing three-dimensional Gaussian filtering on the normalized 3D vascular image to obtain a smoothed 3D vascular image, and performing three-dimensional histogram equalization on the smoothed 3D vascular image to obtain a contrast-enhanced 3D vascular image;

[0044] (3) calculating the optimal segmentation threshold of the contrast-enhanced 3D vascular image and generating a preliminary segmented 3D vascular image;

[0045] (4) performing a three-dimensional morphological opening operation on the preliminary segmented 3D vascular image to obtain a denoised 3D vascular image, and performing a three-dimensional morphological closing operation on the denoised 3D vascular image to obtain a filled 3D vascular image;

[0046] (5) performing a mask operation on the contrast-enhanced 3D blood vessel image according to the filled 3D blood vessel image to obtain a 3D blood vessel region image, and performing an edge-preserving filter on the 3D blood vessel region image to obtain an edge-enhanced 3D blood vessel image;

[0047] (6) calculating a local contrast map based on the edge-enhanced 3D vascular image, and adaptively enhancing the local contrast map to obtain a detail-enhanced 3D vascular image;

[0048] (7) The detail-enhanced 3D vascular image is standardized to obtain a preprocessed 3D vascular image.

[0049] Specifically, the user's venous 3D vascular image is collected through a preset smart door lock. The smart door lock has built-in high-precision imaging equipment and infrared sensors, which can collect venous vascular images in the user's fingers when the user's fingers touch the door lock. The collected 3D vascular image may have large pixel value fluctuations due to different ambient light, skin color and other factors. The image needs to be normalized in pixel value. The normalized 3D vascular image can reduce the influence of ambient light and other interference factors. The normalized 3D vascular image is subjected to three-dimensional Gaussian filtering to smooth the image and reduce noise interference, while retaining the edge features of the image to obtain a smooth 3D vascular image. The mathematical expression of Gaussian filtering is:

[0050]

[0051] Where G(x,y,z) represents the Gaussian kernel function, and σ is the standard deviation of the Gaussian function, which is used to control the degree of smoothing. The smoothed 3D vascular image is subjected to three-dimensional histogram equalization to improve the contrast of the image, making the details of the image more obvious, and obtaining a contrast-enhanced 3D vascular image. By calculating the cumulative distribution function of the image, the pixel values ​​are redistributed so that the enhanced image has a more uniform grayscale distribution. The optimal segmentation threshold of the contrast-enhanced 3D vascular image is calculated, and a preliminary segmented 3D vascular image is generated. The optimal segmentation threshold can be determined by the Otsu method, which is an adaptive threshold segmentation method that can automatically find the threshold that can maximize the inter-class variance. By dividing the image pixels into foreground and background, calculating the intra-class variance and inter-class variance, and selecting the threshold that can maximize the inter-class variance. The 3D vascular image after the preliminary segmentation is subjected to three-dimensional morphological opening and closing operations. The three-dimensional morphological opening operation can remove small noise points to obtain a denoised 3D vascular image, while the closing operation can fill small holes in the vascular area to obtain a filled 3D vascular image. The basic operations of morphological opening and closing operations include corrosion and dilation. By selecting appropriate structural elements, the image can be refined and repaired. According to the filled 3D vascular image, the contrast-enhanced 3D vascular image is masked to obtain the vascular region 3D image. The mask operation uses the filled image as a mask to retain only the corresponding vascular region in the contrast-enhanced image and remove other irrelevant background information. The vascular region 3D image is edge-preserving filtered to obtain the edge-enhanced 3D vascular image. The edge-preserving filter can smooth the image while retaining important edge features in the image, making the vascular boundary clearer. According to the edge-enhanced 3D vascular image, a local contrast map is calculated, and the local contrast map is adaptively enhanced to obtain a detail-enhanced 3D vascular image. The local contrast map reflects the detail changes of the image by calculating the contrast of the pixel values ​​in each local area. Adaptive enhancement makes the details more prominent by adjusting the contrast of the local area. The detail-enhanced 3D vascular image is standardized to obtain a preprocessed 3D vascular image. Standardization ensures that different images have consistent pixel value distribution by mapping the image pixel values ​​to a uniform range, which is convenient for subsequent image processing and analysis.

[0052] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0053] (1) Apply 3x3x3, 5x5x5, and 7x7x7 3D convolution kernels to perform convolution operations on the preprocessed 3D vascular image to obtain multi-scale initial feature maps;

[0054] (2) performing batch normalization and ReLU activation function processing on the multi-scale initial feature maps respectively to obtain the activated multi-scale feature maps, and constructing the hierarchical feature weight matrix according to the activated multi-scale feature maps;

[0055] (3) performing element-by-element multiplication on the hierarchical feature weight matrix and the activated multi-scale feature map to obtain a weighted multi-scale feature map, and performing a splicing operation on the weighted multi-scale feature map in the channel dimension to obtain a multi-scale fusion feature map;

[0056] (4) Apply a 1x1x1 convolution kernel to perform feature compression processing on the multi-scale fusion feature map to obtain a compressed 3D feature map, and calculate the product of the spatial attention map and the channel attention map based on the compressed 3D feature map to obtain a comprehensive attention map;

[0057] (5) performing element-by-element multiplication on the comprehensive attention map and the compressed 3D feature map to obtain the attention-enhanced 3D feature map, and performing residual connection on the attention-enhanced 3D feature map and the multi-scale fusion feature map to obtain the residual 3D feature map;

[0058] (6) Perform adaptive global average pooling and fully connected layer processing on the residual 3D feature map to obtain a 3D feature representation.

[0059] Specifically, 3x3x3, 5x5x5, and 7x7x7 3D convolution kernels are applied to the preprocessed 3D vascular image for convolution operations. Different scale features in the image are captured by convolution kernels of different sizes. The 3x3x3 convolution kernel can extract small features in the image, the 5x5x5 convolution kernel can extract medium-scale features, and the 7x7x7 convolution kernel can extract features in a larger range. Assuming that the input preprocessed 3D vascular image is I(x, y, z), the operation of applying the 3x3x3 convolution kernel can be expressed as:

[0060]

[0061] Among them, F 3x3x3 (x, y, z) represents the feature map after convolution, w i,j,k Represents the weight of the convolution kernel, and I(x+i,y+j,z+k) represents the pixel value of the input image at the corresponding position. Similarly, the feature maps processed by 5x5x5 and 7x7x7 convolution kernels are obtained. The multi-scale initial feature maps are processed by batch normalization and ReLU activation function respectively. Batch normalization can speed up the training process and stabilize the model performance, while the ReLU activation function introduces nonlinearity, enabling the model to better fit complex data distribution. The formula for batch normalization is:

[0062]

[0063] Among them, x represents the input feature, μ and σ 2 Respectively represent the mean and variance of the mini-batch data, ∈ is a small constant used to prevent division by zero errors. The formula of the ReLU activation function is:

[0064] f(x)=max(0,x);

[0065] After batch normalization and ReLU activation function processing, the activated multi-scale feature map is obtained. The hierarchical feature weight matrix is ​​constructed based on the activated multi-scale feature map. The hierarchical feature weight matrix weights the feature maps of different scales so that important features receive greater attention in subsequent processing. The hierarchical feature weight matrix and the activated multi-scale feature map are element-by-element multiplied to obtain a weighted multi-scale feature map. The element-by-element multiplication operation can be expressed as:

[0066] F weighted (x,y,z)=F normalized (x,y,z)·W(x,y,z);

[0067] Among them, F weighted (x, y, z) represents the weighted feature map, F normalized (x, y, z) represents the normalized feature map, and W(x, y, z) represents the hierarchical feature weight matrix. The weighted multi-scale feature map is concatenated in the channel dimension to obtain a multi-scale fused feature map. The concatenation operation integrates feature maps of different scales to form a feature map containing more information. A 1x1x1 convolution kernel is applied to the multi-scale fused feature map to perform feature compression processing to obtain a compressed 3D feature map. The 1x1x1 convolution kernel is used to adjust the number of channels of the feature map, which plays a role in feature compression and dimensionality reduction, and retains important information. The compression process can be expressed as:

[0068]

[0069] Among them, F compressed (x, y, z) represents the compressed feature map, F fused (x, y, z, c) represents the eigenvalue of the multi-scale fusion feature map in channel c, w c Represents the weight of the 1x1x1 convolution kernel. The product of the spatial attention map and the channel attention map is calculated based on the compressed 3D feature map to obtain the comprehensive attention map. The spatial attention map calculates the importance of each spatial position, and the channel attention map calculates the importance of each channel. The comprehensive attention map can be expressed as:

[0070] A comprehensive (x,y,z)=A spatial (x,y,z)·A channel (c);

[0071] Among them, A comprehensive (x, y, z) represents the comprehensive attention map, A spatial (x,y,z) represents the spatial attention map A channel (c) represents the channel attention map. The comprehensive attention map and the compressed 3D feature map are element-wise multiplied to obtain the attention-enhanced 3D feature map, and the attention-enhanced 3D feature map is residually connected with the multi-scale fusion feature map to obtain the residual 3D feature map. The residual connection avoids information loss by retaining the information of the original feature map, and at the same time introduces enhanced features to make the model more stable. The residual 3D feature map is processed by adaptive global average pooling and fully connected layer to obtain a 3D feature representation. Global average pooling extracts the global features of the image by performing a global average operation on the feature map, reduces the number of parameters, and prevents overfitting; the fully connected layer maps the extracted features to the output space to obtain the final 3D feature representation.

[0072] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0073] (1) performing principal component analysis on the 3D feature representation to obtain the main direction vector of the blood vessel, and spatially aligning the 3D feature representation according to the main direction vector of the blood vessel to obtain an aligned 3D feature representation;

[0074] (2) performing 3D sliding window processing on the aligned 3D feature representation to obtain a local feature block set, and generating a projection weight function based on the local feature block set;

[0075] (3) The aligned 3D feature representation is processed by a projection weight function to obtain a weighted 3D feature body, and the maximum intensity projection of the weighted 3D feature body is calculated along the optimal projection direction to obtain a MIP feature map;

[0076] (4) Calculate the weighted average projection of the weighted 3D feature volume along the optimal projection direction to obtain the WAP feature map, and perform channel attention fusion on the MIP feature map and the WAP feature map to obtain a fused 2D feature map;

[0077] (5) Encode the depth information of the fused 2D feature map to obtain a depth-aware 2D feature map, and perform residual enhancement and nonlinear transformation on the depth-aware 2D feature map to obtain a 2D vascular feature map.

[0078] Specifically, principal component analysis is performed on the 3D feature representation to extract the main direction vector of the blood vessel. Principal component analysis is a statistical method used to project high-dimensional data into a low-dimensional space while retaining the main direction of change of the data. The 3D feature representation matrix is ​​centered, that is, the mean is subtracted from each data point, and then the covariance matrix is ​​calculated. The eigenvectors and eigenvalues ​​are obtained by performing eigenvalue decomposition on the covariance matrix. The eigenvectors represent the main direction of change of the data, while the eigenvalues ​​represent the variance in these directions. The eigenvector with the largest eigenvalue is selected as the main direction vector of the blood vessel.

[0079]

[0080] Where C represents the covariance matrix, x i Represents the i-th data point, μ represents the mean of the data, and N represents the number of data points. Through eigenvalue decomposition, the eigenvector v and eigenvalue λ are obtained. The 3D feature representation is spatially aligned according to the main direction vector of the blood vessel. The alignment process includes rotation and flipping, so that the feature representation is spatially consistent with the main direction vector. The aligned 3D feature representation is processed by 3D sliding window, and a fixed-size window is used in 3D space to gradually scan the entire image, extract the features of the local area, and obtain a set of local feature blocks. A projection weight function is generated based on the local feature block set. The projection weight function highlights important feature areas by weighting local features. The local feature blocks are processed using the weighted average method to obtain a weighted 3D feature body. The weighted operation can be expressed as:

[0081] F weighted (x,y,z)=∑ i,j,k w(i,j,k)·F local (x+i,y+j,z+k);

[0082] Among them, F weighted (x, y, z) represents the weighted feature volume, w(i, j, k) represents the projection weight function, and F local (x+i, y+j, z+k) represents the eigenvalue in the local feature block set. The aligned 3D feature representation is processed by the projection weight function to obtain a weighted 3D feature volume. The maximum intensity projection (MIP) is calculated along the optimal projection direction of the weighted 3D feature volume to obtain the MIP feature map. The maximum intensity projection retains the salient features in the image by selecting the pixel with the largest intensity in each projection direction.

[0083]

[0084] Among them, MIP(x,y) represents the maximum intensity projection map, max zIndicates the maximum value operation along the z-axis. At the same time, the weighted average projection (WAP) is calculated along the optimal projection direction of the weighted 3D feature volume to obtain the WAP feature map. The weighted average projection comprehensively considers the intensity distribution of different regions by calculating the weighted average value in each projection direction.

[0085]

[0086] Where WAP(x,y) represents the weighted average projection map, w z Represents the weight in the z direction. Channel attention fusion is performed on the MIP feature map and the WAP feature map to obtain a fused 2D feature map. The channel attention mechanism assigns different weights to different channels, so that the fused feature map retains both significant features and global information. Depth information encoding is performed on the fused 2D feature map to obtain a depth-aware 2D feature map. Depth information encoding can capture the deep hierarchical structure in the image, making the feature map more three-dimensional and hierarchical. Residual enhancement and nonlinear transformation are performed on the depth-aware 2D feature map to obtain a 2D vascular feature map. Residual enhancement retains the information of the original features and avoids information loss by introducing residual connections; nonlinear transformation increases the expressiveness of the model, so that the feature map can better adapt to complex vascular structures.

[0087] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0088] (1) Perform multi-scale convolution operations on the 2D vascular feature map to obtain a multi-scale feature pyramid, and perform local-global cross-scale transformation on the multi-scale feature pyramid to obtain mixed-scale features;

[0089] (2) Perform multi-head hierarchical cross-scale adaptive embedding analysis on the mixed-scale features to obtain cross-scale embedded features, and perform local context processing on the cross-scale embedded features to obtain context-enhanced features;

[0090] (3) Performing cross-scale transformer cascade processing on the context enhancement features to obtain multi-level transformer features, and constructing skip connection blocks based on the multi-level transformer features to obtain multi-scale fusion features;

[0091] (4) Perform channel token interactive fusion on the multi-scale fusion features to obtain interactive enhancement features, and construct a decoding module based on the interactive enhancement features to obtain a preliminary decoding feature map;

[0092] (5) Perform cross-scale position transformation on the preliminary decoded feature map to obtain a position-aware feature map, and perform classification convolution operation on the position-aware feature map to obtain the initial 2D segmentation result.

[0093] Specifically, a multi-scale convolution operation is performed on the 2D vascular feature map to obtain a multi-scale feature pyramid. Features of different scales in the image are captured by convolution kernels of different sizes. Using convolution kernels of different sizes (for example, 3x3, 5x5, 7x7), the 2D vascular feature map is convolved layer by layer to extract multi-scale features. These feature maps are stacked to form a pyramid structure, in which each layer represents information of different scales. The multi-scale convolution operation can be expressed as:

[0094]

[0095] Among them, F k (x, y) represents the feature map obtained using the (2k+1)×(2k+1) convolution kernel, w ij Represents the convolution kernel weight, and I(x+i,y+j) represents the pixel value of the input image at the corresponding position. Perform local-global cross-scale transformation on the multi-scale feature pyramid to obtain mixed-scale features. Local-global cross-scale transformation generates feature representations containing more information by combining local features and global features. This is achieved by splicing or weighted fusion of feature maps of different scales in the spatial dimension, ensuring that the features at each position contain not only local details but also global contextual information. Perform multi-head hierarchical cross-scale adaptive embedding analysis on the mixed-scale features to obtain cross-scale embedded features. Multi-head hierarchical cross-scale adaptive embedding analysis uses a multi-head attention mechanism to embed features of different scales into a high-dimensional feature space and extract the correlation between features. The multi-head attention mechanism can be expressed as:

[0096]

[0097] Where Q represents the query matrix, K represents the key matrix, V represents the value matrix, and d k Represents the dimension of the key. Local context processing is performed on the cross-scale embedded features to obtain context-enhanced features. Local context processing captures detail information and enhances the contextual expression ability of the features by applying convolution operations in local areas of the feature map. The context-enhanced features are processed by cross-scale transformer cascade to obtain multi-level transformer features. Cross-scale transformer cascade processing applies the multi-level structure of the transformer network to the feature map, and each layer can capture higher-level feature relationships. The core of the transformer network is a combination of self-attention mechanism and feedforward neural network, which can be expressed as:

[0098] TransformerLayer(X)=FFN(Attention(X,X,X));

[0099] Among them, Attention(X,X,X) represents the self-attention mechanism, and FFN represents the feedforward neural network. A skip connection block is constructed based on the multi-layer transformer features to obtain multi-scale fusion features. The skip connection block connects features of different levels to ensure the transmission of information between different levels and enhance the expressiveness of features. The multi-scale fusion features are interactively fused by channel tokens to obtain interactively enhanced features. Channel token interactive fusion exchanges information between different channels of the feature map, making the expression of the feature map more comprehensive and balanced. The formula can be expressed as:

[0100]

[0101] Among them, F interaction (x, y, c) represents the interactive enhancement feature, F fused (x,y,i) represents the feature value w of the fused feature map in channel i i Represents the channel weight. The decoding module is constructed based on the interactive enhancement features to obtain a preliminary decoding feature map. The decoding module generates a preliminary decoding feature map by gradually restoring the spatial resolution of the feature map, retaining rich semantic information and detail features. The preliminary decoding feature map is subjected to cross-scale position transformation to obtain a position-aware feature map. The cross-scale position transformation adjusts the position relationship of the feature map so that the model can better understand the spatial structure of the image. The position transformation can be expressed as:

[0102] P(x ′ ,y ′ ) = f(P(x,y));

[0103] Among them, P(x ′ ,y ′ ) represents the transformed position, and f represents the position transformation function. The position-aware feature map is subjected to a classification convolution operation to obtain the initial 2D segmentation result. The classification convolution operation generates the final segmentation result by pixel-by-pixel classification to ensure the accuracy and precision of the segmentation result. The formula can be expressed as:

[0104] S(x,y)=softmax(W·P(x,y)+b);

[0105] Among them, S(x,y) represents the segmentation result, W represents the convolution weight, P(x,y) represents the position-aware feature map, and b represents the bias term.

[0106] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0107] (1) Perform edge detection on the initial 2D segmentation result to obtain an initial boundary map, and calculate the pixel-level significance score based on the initial boundary map to obtain a significance distribution map;

[0108] (2) Reverse mapping the saliency distribution map to obtain a reverse attention weight map, and construct a boundary attention network based on the reverse attention weight map to generate a boundary enhancement weight matrix;

[0109] (3) performing channel splicing on the initial 2D segmentation result and the 2D vascular feature map to obtain a multimodal fusion feature map, and performing a weighted operation on the multimodal fusion feature map according to a boundary enhancement weight matrix to obtain a boundary enhancement feature map;

[0110] (4) performing a boundary-aware convolution operation on the boundary enhancement feature map to obtain a refined boundary feature map, and constructing an iterative boundary optimization module based on the refined boundary feature map to generate an optimized boundary feature map;

[0111] (5) Perform residual connection on the optimized boundary feature map and the initial 2D segmentation result to obtain a residual fusion segmentation map, and perform a boundary correction convolution operation on the residual fusion segmentation map to obtain an optimized 2D segmentation map.

[0112] Specifically, edge detection is performed on the initial 2D segmentation result, and the boundary information in the segmentation result is extracted to obtain the initial boundary map. For example, the Sobel operator is used to implement edge detection. The specific operation is as follows: Calculate the gradient of the image in the x direction and the y direction. This can be done by convolving the image with the Sobel operator. The convolution kernels of the Sobel operator in the x direction and the y direction are:

[0113]

[0114] The initial 2D segmentation result is convolved with the two convolution kernels to obtain the gradient image G in the x and y directions. x and G y By calculating the magnitude of the gradient The initial boundary map is obtained. The pixel-level saliency score is calculated based on the initial boundary map to obtain a saliency distribution map. The saliency score can be obtained by calculating the normalized value of the gradient intensity of each pixel, and the formula is:

[0115]

[0116] Among them, S(x,y) represents the saliency score of pixel (x,y), and G(x,y) represents the gradient strength of the initial boundary map. The saliency distribution map is reversed to obtain the reverse attention weight map. The reverse mapping increases the weight of the non-salient area by inverting the saliency distribution map. The formula is:

[0117] W reverse (x,y)=1-S(x,y);

[0118] Among them, W reverse(x,y) represents the reverse attention weight map. A boundary attention network is constructed based on the reverse attention weight map to generate a boundary enhancement weight matrix. The boundary attention network learns the importance of the boundary area by performing convolution operations on the reverse attention weight map and generates a weight matrix for subsequent weighting. The initial 2D segmentation result and the 2D vascular feature map are channel-joined to obtain a multimodal fusion feature map. Channel joining merges the two feature maps in the channel dimension so that the fused feature map contains more information, which is convenient for subsequent processing. The multimodal fusion feature map is weighted according to the boundary enhancement weight matrix to obtain a boundary enhancement feature map. The weighting operation highlights the features of important areas by multiplying the boundary enhancement weight matrix element by element with the multimodal fusion feature map. The boundary-aware convolution is performed on the boundary enhancement feature map to obtain a refined boundary feature map. The boundary-aware convolution further refines and enhances the features of the boundary area by performing convolution operations on the feature map. The iterative boundary optimization module is constructed based on the refined boundary feature map to generate an optimized boundary feature map. The iterative boundary optimization module gradually optimizes the boundary features through multiple iterative processes to ensure the fineness and accuracy of the segmentation results. The optimized boundary feature map and the initial 2D segmentation result are residually connected to obtain the residual fusion segmentation map. The residual connection avoids information loss by retaining the information of the original feature map, and at the same time introduces the optimized boundary features to improve the overall segmentation effect. The boundary correction convolution operation is performed on the residual fusion segmentation map to obtain the optimized 2D segmentation map. The boundary correction convolution makes the boundary in the segmentation map more accurate and smooth by making the final correction and adjustment to the residual fusion segmentation map.

[0119] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0120] (1) Calculate the pixel-level cross entropy of the optimized 2D segmentation map and the true label image to obtain the basic cross entropy loss value, and calculate the deterministic observation samples based on the true label image to obtain the sample weight matrix;

[0121] (2) Weight the basic cross entropy loss value and the sample weight matrix to obtain the weighted cross entropy loss value;

[0122] (3) Calculate the distance transformation map based on the optimized 2D segmentation map to obtain the boundary distance weight map, and fuse the boundary distance weight map and the weighted cross entropy loss value to obtain the boundary sensitive loss value;

[0123] (4) Dynamically adjust the weights of the loss components according to the training stage and verification performance to obtain an adaptive comprehensive loss function, and perform a back propagation algorithm on the adaptive comprehensive loss function to obtain a gradient update value;

[0124] (5) Optimize and adjust the network parameters of the cross-scale cascaded hierarchical transformer network according to the gradient update value to obtain an updated network model;

[0125] (6) The 2D vascular feature map is inferred through the updated network model to obtain the target 2D segmentation result.

[0126] Specifically, the pixel-level cross entropy is calculated for the optimized 2D segmentation map and the true label image to obtain the basic cross entropy loss value. The cross entropy loss function is used to measure the difference between the predicted result and the true label.

[0127] For each pixel, the cross entropy loss value can be calculated by the following formula:

[0128] L CE (x,y)=-[Y(x,y)log(P(x,y))+(1-Y(x,y))log(1-P(x,y))];

[0129] Among them, L CE (x,y) represents the cross entropy loss value of pixel (x,y), Y(x,y) represents the true label value, and P(x,y) represents the predicted probability. Calculate the deterministic observation samples based on the true label image to obtain the sample weight matrix. Deterministic observation samples identify those samples with the most reference value for model training by analyzing the true label image. Calculate the certainty score of each pixel in the true label image. The certainty score can be obtained by calculating the average label value of each pixel in the neighborhood. Normalize the certainty score to obtain the sample weight matrix.

[0130]

[0131] Among them, W sample (x,y) represents the weight in the sample weight matrix, and C(x,y) represents the deterministic score of the pixel (x,y). The basic cross entropy loss value and the sample weight matrix are weighted to obtain the weighted cross entropy loss value. The weighted operation multiplies the sample weight matrix by the basic cross entropy loss value element by element, so that important samples have a greater weight in the loss calculation. The specific operation is as follows:

[0132] L weighted (x,y)=L CE (x,y)·W sample (x,y);

[0133] Among them, L weighted(x,y) represents the weighted cross entropy loss value. The distance transformation map is calculated based on the optimized 2D segmentation map to obtain the boundary distance weight map. The distance transformation map generates a weight map reflecting the boundary information by calculating the distance from each pixel to the nearest boundary. The distance transformation algorithm is used to calculate the distance from each pixel to the nearest boundary in the optimized 2D segmentation map. The distance values ​​are normalized to obtain the boundary distance weight map.

[0134]

[0135] Among them, W boundary (x,y) represents the weight in the boundary distance weight map, and D(x,y) represents the distance from the pixel (x,y) to the nearest boundary. The boundary distance weight map and the weighted cross entropy loss value are fused to obtain the boundary sensitive loss value. The fusion operation multiplies the boundary distance weight map and the weighted cross entropy loss value element by element, so that the boundary area receives higher attention in the loss calculation. The specific operation is as follows:

[0136] L boundary (x,y)=L weighted (x,y)·W boundary (x,y);

[0137] Among them, L boundary (x,y) represents the boundary-sensitive loss value. Dynamically adjust the weights of the loss components according to the training stage and verification performance to obtain an adaptive comprehensive loss function. The adaptive comprehensive loss function dynamically adjusts the weights of different loss components according to the performance during the training process, so that the model can optimize different objectives at different training stages. Define a weight adjustment function to adjust the weights of the loss components according to the training stage and verification performance. The basic cross entropy loss value and the boundary-sensitive loss value are weighted and summed according to the weight adjustment function to obtain the adaptive comprehensive loss function.

[0138] L adaptive =α(t)·L CE +β(t)·L boundary ;

[0139] Among them, L adaptive Represents the adaptive comprehensive loss function, α(t) and β(t) represent the values ​​of the weight adjustment function at time step t. Perform the back propagation algorithm on the adaptive comprehensive loss function to obtain the gradient update value. The back propagation algorithm guides the update direction and amplitude of the model parameters by calculating the gradient of the loss function, thereby gradually optimizing the model performance. Calculate the gradient of the adaptive comprehensive loss function to the model parameters.

[0140] Update the model parameters according to the gradient descent method.

[0141]

[0142] Among them, θ t represents the value of the model parameter at time step t, η represents the learning rate, Represents the gradient of the adaptive comprehensive loss function to the model parameters. According to the gradient update value, the network parameters of the cross-scale cascaded hierarchical transformer network are optimized and adjusted to obtain the updated network model. The gradient update value is applied to each layer of the cross-scale cascaded hierarchical transformer network, and the parameters of each layer are updated so that the model gradually approaches the optimal solution. The 2D vascular feature map is inferred through the updated network model to obtain the target 2D segmentation result. The inference process generates the final segmentation result by applying the optimized model to the actual data, ensuring the accuracy and reliability of the segmentation result.

[0143] The image segmentation method in the embodiment of the present application is described above. The image segmentation system in the embodiment of the present application is described below. Figure 2 In the embodiment of the present application, an image segmentation system includes:

[0144] The acquisition module 201 is used to acquire the user's venous 3D blood vessel image through a preset smart door lock, and perform standardization filtering and contrast enhancement processing on the venous 3D blood vessel image to obtain a pre-processed 3D blood vessel image;

[0145] An analysis module 202 is used to perform multi-scale convolution and layer attention mechanism analysis on the pre-processed 3D blood vessel image to obtain a 3D feature representation;

[0146] An execution module 203 is used to execute an adaptive 3D to 2D projection algorithm on the 3D feature representation to obtain a 2D blood vessel feature map;

[0147] A segmentation module 204 is used to perform image segmentation on the 2D blood vessel feature map using a cross-scale cascaded hierarchical transformer network to obtain an initial 2D segmentation result;

[0148] An optimization module 205 is used to perform reverse boundary attention optimization on the initial 2D segmentation result to obtain an optimized 2D segmentation map;

[0149] The output module 206 is used to evaluate and optimize the cross entropy loss function on the optimized 2D segmentation map and output the target 2D segmentation result.

[0150] Through the collaborative cooperation of the above components, the combination of multi-scale convolution and layer attention mechanism improves the feature extraction ability of 3D vascular structure and can better capture vascular information of different scales. The adaptive 3D to 2D projection algorithm achieves dimensionality reduction while retaining 3D structural information, reducing the computational complexity of subsequent processing. The application of cross-scale cascaded hierarchical transformer network enhances the model's learning and fusion capabilities of multi-scale features and improves the accuracy of segmentation. The reverse boundary attention optimization mechanism pays special attention to the vascular boundary area and improves the boundary accuracy of the segmentation result. The improved cross entropy loss function better guides the optimization process of the model by introducing deterministic observation samples and boundary sensitive items. The full process from 3D image acquisition to 2D segmentation result output is realized, improving the overall efficiency and accuracy of vein recognition.

[0151] The present application also provides an image segmentation device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the image segmentation method in the above embodiments.

[0152] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the image segmentation method.

[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0155] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image segmentation method, characterized in that: The image segmentation method comprises: The user's venous 3D blood vessel image is collected through a preset smart door lock, and the venous 3D blood vessel image is subjected to standardization filtering and contrast enhancement processing to obtain a preprocessed 3D blood vessel image; Performing multi-scale convolution and layer attention mechanism analysis on the preprocessed 3D blood vessel image to obtain a 3D feature representation; Executing an adaptive 3D to 2D projection algorithm on the 3D feature representation to obtain a 2D vascular feature map; specifically comprising: performing principal component analysis on the 3D feature representation to obtain a vascular main direction vector, and spatially aligning the 3D feature representation according to the vascular main direction vector to obtain an aligned 3D feature representation; performing 3D sliding window processing on the aligned 3D feature representation to obtain a local feature block set, and generating a projection weight function according to the local feature block set; processing the aligned 3D feature representation by the projection weight function to obtain a weighted 3D feature body, and calculating a maximum intensity projection of the weighted 3D feature body along an optimal projection direction to obtain a MIP feature map; calculating a weighted average projection of the weighted 3D feature body along an optimal projection direction to obtain a WAP feature map, and performing channel attention fusion on the MIP feature map and the WAP feature map to obtain a fused 2D feature map; performing depth information encoding on the fused 2D feature map to obtain a depth-aware 2D feature map, and performing residual enhancement and nonlinear transformation on the depth-aware 2D feature map to obtain a 2D vascular feature map; Using a cross-scale cascaded hierarchical transformer network to perform image segmentation on the 2D blood vessel feature map to obtain an initial 2D segmentation result; Performing reverse boundary attention optimization on the initial 2D segmentation result to obtain an optimized 2D segmentation map; The optimized 2D segmentation map is evaluated and optimized using a cross entropy loss function, and a target 2D segmentation result is output.

2. The image segmentation method according to claim 1, characterized in that: The method of collecting the user's venous 3D blood vessel image through the preset smart door lock, and performing standardization filtering and contrast enhancement processing on the venous 3D blood vessel image to obtain a preprocessed 3D blood vessel image includes: The user's venous 3D blood vessel image is collected through a preset smart door lock, and pixel value normalization processing is performed on the venous 3D blood vessel image to obtain a normalized 3D blood vessel image; performing three-dimensional Gaussian filtering on the normalized 3D vascular image to obtain a smoothed 3D vascular image, and performing three-dimensional histogram equalization on the smoothed 3D vascular image to obtain a contrast-enhanced 3D vascular image; Calculating an optimal segmentation threshold of the contrast-enhanced 3D vascular image and generating a preliminary segmented 3D vascular image; performing a three-dimensional morphological opening operation on the preliminary segmented 3D vascular image to obtain a denoised 3D vascular image, and performing a three-dimensional morphological closing operation on the denoised 3D vascular image to obtain a filled 3D vascular image; performing a mask operation on the contrast-enhanced 3D blood vessel image according to the filled 3D blood vessel image to obtain a blood vessel region 3D image, and performing edge-preserving filtering on the blood vessel region 3D image to obtain an edge-enhanced 3D blood vessel image; Calculating a local contrast map according to the edge-enhanced 3D blood vessel image, and adaptively enhancing the local contrast map to obtain a detail-enhanced 3D blood vessel image; The detail-enhanced 3D blood vessel image is subjected to standardization processing to obtain a preprocessed 3D blood vessel image.

3. The image segmentation method according to claim 1, characterized in that: The performing multi-scale convolution and layer attention mechanism analysis on the pre-processed 3D blood vessel image to obtain a 3D feature representation includes: Applying 3x3x3, 5x5x5 and 7x7x7 3D convolution kernels to perform convolution operations on the preprocessed 3D blood vessel image to obtain a multi-scale initial feature map; Performing batch normalization and ReLU activation function processing on the multi-scale initial feature map to obtain an activated multi-scale feature map, and constructing a hierarchical feature weight matrix according to the activated multi-scale feature map; Performing element-by-element multiplication operation on the hierarchical feature weight matrix and the activated multi-scale feature map to obtain a weighted multi-scale feature map, and performing a splicing operation on the weighted multi-scale feature map in the channel dimension to obtain a multi-scale fusion feature map; Applying a 1x1x1 convolution kernel to perform feature compression processing on the multi-scale fusion feature map to obtain a compressed 3D feature map, and calculating the product of the spatial attention map and the channel attention map according to the compressed 3D feature map to obtain a comprehensive attention map; Performing element-by-element multiplication on the comprehensive attention map and the compressed 3D feature map to obtain an attention-enhanced 3D feature map, and performing a residual connection on the attention-enhanced 3D feature map and the multi-scale fusion feature map to obtain a residual 3D feature map; The residual 3D feature map is subjected to adaptive global average pooling and full connection layer processing to obtain a 3D feature representation.

4. The image segmentation method according to claim 1, characterized in that: The cross-scale cascaded hierarchical transformer network is used to perform image segmentation on the 2D blood vessel feature map to obtain an initial 2D segmentation result, including: Performing a multi-scale convolution operation on the 2D blood vessel feature map to obtain a multi-scale feature pyramid, and performing a local-global cross-scale transformation on the multi-scale feature pyramid to obtain a mixed-scale feature; Performing multi-head hierarchical cross-scale adaptive embedding analysis on the mixed-scale features to obtain cross-scale embedding features, and performing local context processing on the cross-scale embedding features to obtain context enhanced features; Performing cross-scale transformer cascade processing on the context enhancement feature to obtain a multi-level transformer feature, and constructing a skip connection block based on the multi-level transformer feature to obtain a multi-scale fusion feature; Performing channel token interactive fusion on the multi-scale fusion features to obtain interactive enhancement features, and constructing a decoding module according to the interactive enhancement features to obtain a preliminary decoding feature map; The preliminary decoded feature map is subjected to a cross-scale position transformation to obtain a position-aware feature map, and a classification convolution operation is performed on the position-aware feature map to obtain an initial 2D segmentation result.

5. The image segmentation method according to claim 1, characterized in that: The performing reverse boundary attention optimization on the initial 2D segmentation result to obtain an optimized 2D segmentation map includes: Performing edge detection on the initial 2D segmentation result to obtain an initial boundary map, and calculating pixel-level significance scores according to the initial boundary map to obtain a significance distribution map; Reverse mapping the saliency distribution map to obtain a reverse attention weight map, and constructing a boundary attention network according to the reverse attention weight map to generate a boundary enhancement weight matrix; Perform channel splicing on the initial 2D segmentation result and the 2D blood vessel feature map to obtain a multimodal fusion feature map, and perform a weighted operation on the multimodal fusion feature map according to the boundary enhancement weight matrix to obtain a boundary enhancement feature map; Performing a boundary-aware convolution operation on the boundary enhancement feature map to obtain a refined boundary feature map, and constructing an iterative boundary optimization module based on the refined boundary feature map to generate an optimized boundary feature map; A residual connection is performed on the optimized boundary feature map and the initial 2D segmentation result to obtain a residual fusion segmentation map, and a boundary correction convolution operation is performed on the residual fusion segmentation map to obtain an optimized 2D segmentation map.

6. The image segmentation method according to claim 1, characterized in that: The step of evaluating and optimizing the cross entropy loss function on the optimized 2D segmentation map and outputting a target 2D segmentation result includes: Calculating pixel-level cross entropy for the optimized 2D segmentation map and the true label image to obtain a basic cross entropy loss value, and calculating deterministic observation samples based on the true label image to obtain a sample weight matrix; Weighting the basic cross entropy loss value and the sample weight matrix to obtain a weighted cross entropy loss value; Calculating a distance transformation map according to the optimized 2D segmentation map to obtain a boundary distance weight map, and fusing the boundary distance weight map with the weighted cross entropy loss value to obtain a boundary sensitive loss value; Dynamically adjust the weights of the loss components according to the training stage and the verification performance to obtain an adaptive comprehensive loss function, and perform a back propagation algorithm on the adaptive comprehensive loss function to obtain a gradient update value; Optimizing and adjusting the network parameters of the cross-scale cascaded hierarchical transformer network according to the gradient update value to obtain an updated network model; The 2D blood vessel feature map is inferred through the updated network model to obtain a target 2D segmentation result.

7. An image segmentation system, characterized in that: Used to perform the image segmentation method according to any one of claims 1 to 6, the image segmentation system comprises: An acquisition module is used to acquire a 3D venous blood vessel image of a user through a preset smart door lock, and perform standardization filtering and contrast enhancement processing on the 3D venous blood vessel image to obtain a preprocessed 3D blood vessel image; An analysis module, configured to perform multi-scale convolution and layer attention mechanism analysis on the pre-processed 3D vascular image to obtain a 3D feature representation; An execution module is used to execute an adaptive 3D to 2D projection algorithm on the 3D feature representation to obtain a 2D vascular feature map; specifically comprising: performing principal component analysis on the 3D feature representation to obtain a vascular main direction vector, and spatially aligning the 3D feature representation according to the vascular main direction vector to obtain an aligned 3D feature representation; performing 3D sliding window processing on the aligned 3D feature representation to obtain a local feature block set, and generating a projection weight function according to the local feature block set; processing the aligned 3D feature representation by the projection weight function to obtain a weighted 3D feature body, and calculating a maximum intensity projection of the weighted 3D feature body along an optimal projection direction to obtain a MIP feature map; calculating a weighted average projection of the weighted 3D feature body along an optimal projection direction to obtain a WAP feature map, and performing channel attention fusion on the MIP feature map and the WAP feature map to obtain a fused 2D feature map; performing depth information encoding on the fused 2D feature map to obtain a depth-aware 2D feature map, and performing residual enhancement and nonlinear transformation on the depth-aware 2D feature map to obtain a 2D vascular feature map; A segmentation module, used for performing image segmentation on the 2D blood vessel feature map by using a cross-scale cascaded hierarchical transformer network to obtain an initial 2D segmentation result; An optimization module, configured to perform reverse boundary attention optimization on the initial 2D segmentation result to obtain an optimized 2D segmentation map; The output module is used to evaluate and optimize the cross entropy loss function of the optimized 2D segmentation map and output the target 2D segmentation result.

8. An image segmentation device, characterized in that: The image segmentation device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the image segmentation device to perform the image segmentation method according to any one of claims 1 to 6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the image segmentation method according to any one of claims 1 to 6 is implemented.

Images