An ultrasound-guided radial artery puncture-assisted method based on deep learning

Abstract

The present invention discloses an ultrasound-guided radial artery assisted puncture method based on deep learning, including S1: constructing a data set; S2: enhancing the features of the radial artery ultrasound image sample data in S1 to obtain an enhanced radial artery ultrasound image; S3: constructing a training set and a test set; step S4: constructing a block feature extraction module to obtain an extracted feature map; step S5: constructing a semantic segmentation model, the model performs semantic segmentation on the extracted feature map in S4, and obtains a segmented image to calibrate the position of the radial artery. The present invention constructs a semantic segmentation model, uses an encoder and decoder structure with a small number of layers to highlight the position of the radial artery in the segmented image, and uses an average pooling layer to reduce model parameters to prevent overfitting; further, an attention gating unit is added between the encoder and the decoder to filter irrelevant environmental features output by the encoder to the decoder, further highlighting the position of the radial artery in the segmented image.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an ultrasound-guided radial artery assisted puncture method based on deep learning. Background Art

[0002] In medical practice, radial artery puncture is a common and important operation, such as invasive pressure measurement under anesthesia, coronary angiography, etc. However, the traditional manual puncture method relies on the doctor's experience and feel, and there is a high failure rate and risk of complications, such as hematoma, arterial spasm, etc. Therefore, a more accurate and efficient guidance method is needed. In addition, the position, morphology and depth of the radial artery vary greatly among different patients, and may be affected by factors such as surrounding tissues and pathological changes. This increases the difficulty and uncertainty of puncture and places higher demands on the doctor's skills. Ultrasound imaging technology has the advantages of real-time, radiation-free, and portability. It can display vascular structures and surrounding tissues in real time, providing intuitive and dynamic guidance for puncture. However, traditional ultrasound imaging technology mainly relies on the doctor's visual judgment, which has certain subjectivity and limitations.

[0003] With the development of computer image processing technology, the resolution, contrast and clarity of ultrasound images have been significantly improved. Through edge detection, image segmentation and other algorithms, the target blood vessels can be identified and located more accurately, providing accurate basis for puncture. Deep learning technology, especially convolutional neural network (CNN), can automatically learn feature representations in images and has powerful pattern recognition capabilities. By training a CNN model, the radial artery in ultrasound images can be automatically identified and segmented, improving the accuracy and robustness of segmentation. Combining ultrasound imaging technology with deep learning technology can give full play to the advantages of both and achieve more accurate and efficient radial artery puncture guidance. Automatic analysis and processing of ultrasound images through deep learning models can provide real-time information on the location, morphology and depth of target blood vessels, providing decision support for doctors.

[0004] Considering the performance of the radial artery at different scales (such as thickness, curvature, etc.), a multi-scale feature fusion mechanism can be incorporated into the network design to improve the segmentation accuracy of radial arteries at different scales. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides an ultrasound-guided radial artery assisted puncture method based on deep learning, comprising the following steps:

[0006] Step S1: Collect radial artery ultrasound image sample data and construct a data set based on it;

[0007] Step S2: constructing an image enhancement module, importing the radial artery ultrasound image sample data in step S1 into the image enhancement module, enhancing the features of the radial artery ultrasound image sample data in step S1, and obtaining an enhanced radial artery ultrasound image;

[0008] Step S3: constructing a training set and a test set based on the radial artery ultrasound image enhanced in step S2;

[0009] Step S4: constructing a block feature extraction module, which cuts the radial artery ultrasound image into several blocks through image segmentation and then extracts features from each small block of the image, thereby capturing local features in the radial artery ultrasound image to obtain an extracted feature map;

[0010] Step S5: constructing a radial artery ultrasound image semantic segmentation model based on a U-net architecture, the model performs semantic segmentation on the extracted feature map in step S4, and obtains a segmented image to calibrate the position of the radial artery;

[0011] The model consists of 4 encoders, 4 decoders, a Softmax activation function layer, and an attention gating unit; the encoder consists of a depthwise separable convolutional layer, a global average pooling layer, and a downsampling pooling layer, and the decoder structure is symmetrical to the encoder;

[0012] During the semantic segmentation process, the encoder and decoder can obtain high-level semantic features of the extracted feature map to highlight the position of the radial artery in the segmented image; the average pooling layer of the encoder and decoder can reduce the number of parameters and suppress model overfitting; the attention gating unit can filter out irrelevant environmental features output by the encoder to the decoder, further highlighting the position of the radial artery in the segmented image.

[0013] Furthermore, the radial artery ultrasound image sample data collected in step S1 is specifically:

[0014] Step S11: applying coupling agent to the radial artery of the patient to reduce the influence of air on the ultrasonic signal, scanning the radial artery with a probe, and collecting a number of radial artery ultrasonic images at different angles and depths;

[0015] Step S12: Open the acquired radial artery ultrasound image in LabelImg annotation software, manually identify the radial artery area in the image, distinguish the boundary between the radial artery area and the surrounding tissue or background, and use the annotation tool to draw a closed curve along the boundary of the radial artery;

[0016] Step S13: labeling the closed curve drawn in step S12 with a semantic label.

[0017] Furthermore, the image enhancement module in step S2 includes a multi-scale pooling layer, a channel convolution layer, a channel fusion layer, a multi-scale convolution layer and a fully connected layer connected in series;

[0018] Step S2 is specifically as follows:

[0019] Step S21: Multi-scale pooling layer for radial artery ultrasound image Perform pooling processing; expressed as:

[0020] ;

[0021] ;

[0022] in, and They represent the radial artery ultrasound images when the pooling kernel is i. The output after average pooling and maximum pooling, represents the convolution kernel size, represents the average pooling operation, represents the maximum pooling operation, Indicates the image boundary filling operation, i={3, 5, 7};

[0023] By using boundary padding to ensure that the output length is consistent after processing with different pooling kernels, the output of the multi-scale pooling layer is finally obtained. ;

[0024] Step S22: The channel convolution layer applies convolution operations independently on each channel, so that the network can better capture the feature information of different channels in the image, thereby enhancing the perception of the radial artery structure. At the same time, reducing the number of parameters improves the efficiency of the model, allowing the network to perform feature extraction and image processing faster, accelerating the process of radial artery ultrasound image enhancement. In addition, the channel-by-channel convolution layer can also achieve feature fusion and combination to produce richer and more diverse feature representations, thereby improving the clarity and recognition accuracy of radial artery ultrasound imaging. The channel convolution layer captures the feature information of different channels in the radial artery ultrasound image by applying convolution operations independently on each channel, thereby enhancing the perception of the radial artery structure; each channel in the channel convolution layer is provided with an independent convolution kernel, which performs convolution operations with the corresponding channels of the input image respectively; the calculation process is as follows:

[0025] ;

[0026] in, represents the output of the channel convolution layer. Its number of channels is consistent with the input of the corresponding layer of the channel convolution layer, which is 6. represents the convolution kernel, Represents the RELU activation function;

[0027] Step S23: the channel fusion layer adaptively assigns weights to each channel;

[0028] The output of step S22 All perform independent full connection calculations, and the calculation process is as follows:

[0029] ;

[0030] in, represents the calculated weight score of the i-th channel, and Represent the corresponding weight matrix and bias vector respectively;

[0031] Then calculate the weight score Multiply and add the corresponding outputs of step S22 respectively to obtain the output of the channel fusion layer, which is expressed as:

[0032] ;

[0033] in, Represents the output of the channel fusion layer;

[0034] Step S24: Use the multi-scale convolution layer to fusion the output of the channel fusion layer Convolution kernels of size 3, 5, and 7 are used in parallel. The multi-scale convolution layer consists of 3 convolution kernels in parallel. 3 convolutional layers, 5 5 convolutional layers and 7 7 convolutional layers;

[0035] The calculation process is expressed as:

[0036] ;

[0037] in, Indicates i The output of the i convolutional layer, represents the convolution kernel of size i, Represents the Sigmoid activation function;

[0038] Step S25: The fully connected layer is used for the 3 3. Output of convolutional layer , 5 5. Output of convolutional layer and 7 7 Output of the convolutional layer Perform feature fusion, specifically:

[0039] 3 3. Output of convolutional layer , 5 5. Output of convolutional layer and 7 7 Output of the convolutional layer Flatten them into corresponding one-dimensional vectors, that is, 3 3 Convolution of 1D vector , 5 5 Convolution of 1D vector and 7 7 Convolution of 1D vector , and then forward calculation is performed through the fully connected layer to obtain the corresponding weights, expressed as:

[0040] ;

[0041] ;

[0042] ;

[0043] in, 3 respectively 3 Convolution of 1D vector , 5 5 Convolution of 1D vector and 7 7 Convolution of 1D vector The corresponding weights, are respectively the learnable weight matrices;

[0044] 3 3 Convolution of 1D vector , 5 5 Convolution of 1D vector and 7 7 Convolution of 1D vector Multiplying and adding the corresponding weights respectively gives the final output, which is the enhanced radial artery ultrasound image:

[0045] ;

[0046] in, Represents enhanced ultrasound image of the radial artery.

[0047] Furthermore, step S3 is specifically as follows:

[0048] The semantic labels of the radial artery ultrasound image sample data collected in step S1 are paired with the radial artery ultrasound image enhanced in step S2, and are randomly divided into a training set and a test set in a ratio of 8:2 for training and testing a block feature extraction module and a radial artery ultrasound image semantic segmentation model based on a U-net architecture;

[0049] The radial artery ultrasound image sample data collected in step S1 is paired with the radial artery ultrasound image enhanced in step S2, wherein the enhanced radial artery ultrasound image is used as a label and is randomly divided into a training set and a test set in a ratio of 8:2 for training and testing of the image enhancement module.

[0050] Furthermore, the block feature extraction module in step S4 includes a preprocessing module, a feature extraction module and an image extraction module connected in series;

[0051] Among them, the feature extraction module includes upsampling layers of different scales, lightweight channel attention layers, spatial attention layers, gating units and feature splicing layers; the lightweight channel attention layer adopts the ECA-NET module;

[0052] Step S4 is specifically as follows:

[0053] Step S41: The preprocessing module divides the enhanced radial artery ultrasound image into equal intervals using an image block method. small blocks, and obtain the radial artery ultrasound image after N blocks;

[0054] Step S42: The feature extraction module extracts features from the divided radial artery ultrasound images to obtain extracted radial artery ultrasound images; specifically:

[0055] Step S421: Input the radial artery ultrasound image after the block division in step S41 into upsampling layers of different scales. The upsampling layers of different scales upsample the input images respectively to obtain feature maps of different scales. The calculation process is as follows:

[0056] ;

[0057] in, Represents the image input The feature map obtained after upsampling layers, Indicates The upsampling operation performed by the upsampling layer, This is the ultrasound image of the radial artery after segmentation. ;

[0058] Step S422: The feature map Input lightweight channel attention layer to obtain channel attention weight , the calculation process is as follows:

[0059] ;

[0060] in, is the Sigmoid activation function, and Represent the maximum pooling and average pooling operations respectively, is a multi-layer perceptron;

[0061] Will and Perform product operation to obtain channel attention features ;

[0062] Step S423: Channel attention features Input spatial attention layer to obtain channel attention weights , the calculation process is as follows:

[0063] ;

[0064] in, and Represent the maximum pooling and average pooling operations respectively, is the convolutional layer;

[0065] Step S424: Set the channel attention weight Input the gated unit GRU to obtain the weight after adjusting the weight ,Will and Perform product operation to obtain gated attention features ;

[0066] Step S425: Gated attention features at three different scales Perform feature splicing to obtain fusion features , that is, extract the radial artery ultrasound image, expressed as:

[0067] ;

[0068] in, Represents a splicing and fusion operation;

[0069] Step S43: The image extraction module performs feature extraction on the radial artery ultrasound image extracted in step S42, and obtains the final extracted feature map through weighted fusion calculation; specifically:

[0070] Step S431: sequentially obtain the extracted radial artery ultrasound image of each small image block and mark it as , expressed as:

[0071] ;

[0072] in, represents the operation of extracting the radial artery ultrasound image from each small image. The first segment of the radial artery ultrasound image Small pieces of images, Indicates the extracted Extraction of small pieces of images from radial artery ultrasound images, ;

[0073] Step S432: Feature matrix conversion: The extracted radial artery ultrasound image of each small block image obtained in step S431 is sequentially converted to the linear transformation matrix Multiply them together to get Extraction of small blocks of images The radial artery ultrasound image is converted into the corresponding one-dimensional feature vector, expressed as:

[0074] ;

[0075] in, Indicates Extraction of small image blocks: One-dimensional feature vector corresponding to the radial artery ultrasound image;

[0076] Step S433: Multi-head attention weight calculation: Calculate the Extraction of small blocks of images The attention weight corresponding to the radial artery ultrasound image is expressed as:

[0077] ;

[0078] in, represents the calculated attention weight corresponding to the extracted radial artery ultrasound image of the i-th small image, Indicates Extract the transpose of the one-dimensional feature vector corresponding to the radial artery ultrasound image. represents the normalization operation function, ;

[0079] Step S434: weighted image feature calculation: using the calculated attention weight corresponding to the nth small image block and its corresponding The feature matrix of a small image Multiply them together to get the weighted image feature matrix ,and , expressed as:

[0080] ;

[0081] Step S435: Obtain the weighted feature matrix of all small blocks of images Combine to obtain the weighted extracted feature map , expressed as:

[0082] .

[0083] Furthermore, step S5 is specifically as follows:

[0084] Step S51: constructing a radial artery ultrasound image semantic segmentation model based on a U-net architecture, the model consisting of 4 encoders, 4 decoders, a Softmax activation function layer and an attention gating unit;

[0085] Step S52: The encoder is composed of a depth-separable convolutional layer, a global average pooling layer and a downsampling pooling layer, and the decoder structure is symmetrical to the encoder, both of which are composed of a depth-separable convolutional layer, a global average pooling layer and an upsampling pooling layer;

[0086] For The calculation formula is as follows:

[0087] ;

[0088] in, For the The output features of the encoder are It is the feature map extracted Representation in latent feature space, It is The output features of the encoder; It is a downsampling pooling operation; is the trainable weight parameter of the encoder; is a depth-separable convolution operation; the input of the first encoder is the extracted feature map ;

[0089] Step S53: The output features of the encoder Input the attention gate unit, the calculation formula is as follows:

[0090] ;

[0091] in, represents the output of the attention gating unit, Represents the attention gating mechanism;

[0092] Step S54: The output of the attention gating unit Enter the The first encoder jump connection The restored features are obtained in the decoder , expressed as:

[0093] ;

[0094] in, is the upsampling pooling operation, Represents the trainable weight parameters of the decoder;

[0095] Step S55: The output of the last layer of decoder Input the Softmax layer to obtain the semantic segmentation result, which is expressed as:

[0096] ;

[0097] in, is the probability that each pixel in the feature map corresponds to the radial artery;

[0098] The pixels in P whose probability is greater than a preset probability threshold are classified as pixels constituting the radial artery to obtain a segmented image.

[0099] The positive and progressive effects of the present invention are:

[0100] 1) Improving the robustness of deep learning models: By constructing an image enhancement module, the feature representation capability of radial artery ultrasound images at different scales is enhanced, which helps the model better capture and identify radial arteries of different thicknesses and shapes, thereby improving segmentation accuracy. Multi-scale feature processing enables the model to show higher robustness and adaptability when facing complex and changeable image inputs.

[0101] 2) Improve the computational efficiency of feature extraction and deeply mine local features: The block feature extraction module cuts the image into multiple image blocks for feature extraction through image segmentation technology, effectively capturing the local detail features in the image, which is crucial to improving segmentation accuracy; block processing can reduce the amount of calculation to a certain extent and improve the efficiency of feature extraction, especially when processing high-resolution images.

[0102] 3) Achieve accurate semantic segmentation: The model based on the U-net architecture integrates design elements such as attention mechanism, skip connection, coordinated upsampling and downsampling, which significantly improves the modeling ability of the model, enabling the model to more accurately capture the contextual information and structural features in the image. The combination of these design elements enables the model to perform well in semantic segmentation tasks and can more accurately identify the position of the radial artery in the image, providing doctors with reliable auxiliary diagnostic information. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] Figure 1 This is a flowchart of the steps of an ultrasound-guided radial artery assisted puncture method based on deep learning of the present invention.

[0104] Figure 2 This is a structural principle diagram of an image enhancement module of an ultrasound-guided radial artery assisted puncture method based on deep learning of the present invention.

[0105] Figure 3This is a structural principle diagram of a block feature extraction module of an image enhancement module of an ultrasound-guided radial artery assisted puncture method based on deep learning of the present invention.

[0106] Figure 4 This is a structural schematic diagram of a model based on a U-net architecture of an image enhancement module of an ultrasound-guided radial artery assisted puncture method based on deep learning of the present invention. DETAILED DESCRIPTION

[0107] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0108] Reference Figure 1~Figure 4 The present invention provides an ultrasound-guided radial artery assisted puncture method based on deep learning. In one example, the method comprises the following steps:

[0109] Step S1: Collect radial artery ultrasound image sample data and construct a data set based on it;

[0110] Step S2: constructing an image enhancement module, importing the radial artery ultrasound image sample data in step S1 into the image enhancement module, enhancing the features of the radial artery ultrasound image sample data in step S1, and obtaining an enhanced radial artery ultrasound image;

[0111] Step S3: constructing a training set and a test set based on the radial artery ultrasound image enhanced in step S2;

[0112] Step S4: constructing a block feature extraction module, which cuts the radial artery ultrasound image into several blocks through image segmentation and then extracts features from each small block of the image, thereby capturing local features in the radial artery ultrasound image to obtain an extracted feature map;

[0113] Step S5: constructing a radial artery ultrasound image semantic segmentation model based on a U-net architecture, the model performs semantic segmentation on the extracted feature map in step S4, and obtains a segmented image to calibrate the position of the radial artery;

[0114] The model consists of 4 encoders, 4 decoders, a Softmax activation function layer, and an attention gating unit; the encoder consists of a depthwise separable convolutional layer, a global average pooling layer, and a downsampling pooling layer, and the decoder structure is symmetrical to the encoder;

[0115] During the semantic segmentation process, the encoder and decoder can obtain high-level semantic features of the extracted feature map to highlight the position of the radial artery in the segmented image; the average pooling layer of the encoder and decoder can reduce the number of parameters and suppress model overfitting; the attention gating unit can filter out irrelevant environmental features output by the encoder to the decoder, further highlighting the position of the radial artery in the segmented image.

[0116] Further, in one example, the radial artery ultrasound image sample data collected in step S1 is specifically:

[0117] Step S11: applying coupling agent to the radial artery of the patient to reduce the influence of air on the ultrasonic signal, scanning the radial artery with a probe, and collecting a number of radial artery ultrasonic images at different angles and depths;

[0118] Step S12: Open the acquired radial artery ultrasound image in LabelImg annotation software, manually identify the radial artery area in the image, distinguish the boundary between the radial artery area and the surrounding tissue or background, and use the annotation tool to draw a closed curve along the boundary of the radial artery;

[0119] Step S13: labeling the closed curve drawn in step S12 with a semantic label.

[0120] Further, in one example, the image enhancement module in step S2 includes a multi-scale pooling layer, a channel convolution layer, a channel fusion layer, a multi-scale convolution layer and a fully connected layer connected in series;

[0121] Step S2 is specifically as follows:

[0122] Step S21: Multi-scale pooling layer for radial artery ultrasound image Perform pooling processing; expressed as:

[0123] ;

[0124] ;

[0125] in, and They represent the radial artery ultrasound images when the pooling kernel is i. The output after average pooling and maximum pooling, represents the convolution kernel size, represents the average pooling operation, represents the maximum pooling operation, Indicates the image boundary filling operation, i={3, 5, 7};

[0126] By using boundary padding to ensure that the output length is consistent after processing with different pooling kernels, the output of the multi-scale pooling layer is finally obtained. ;

[0127] Step S22: The channel convolution layer captures the feature information of different channels in the radial artery ultrasound image by applying convolution operations independently on each channel, thereby enhancing the perception of the radial artery structure; each channel in the channel convolution layer is provided with an independent convolution kernel, and these convolution kernels are convolved with the corresponding channels of the input image respectively; the calculation process is as follows:

[0128] ;

[0129] in, represents the output of the channel convolution layer. Its number of channels is consistent with the input of the corresponding layer of the channel convolution layer, which is 6. represents the convolution kernel, Represents the RELU activation function;

[0130] Step S23: the channel fusion layer adaptively assigns weights to each channel;

[0131] The output of step S22 All perform independent full connection calculations, and the calculation process is as follows:

[0132] ;

[0133] in, represents the calculated weight score of the i-th channel, and Represent the corresponding weight matrix and bias vector respectively;

[0134] Then calculate the weight score Multiply and add the corresponding outputs of step S22 respectively to obtain the output of the channel fusion layer, which is expressed as:

[0135] ;

[0136] in, Represents the output of the channel fusion layer;

[0137] Step S24: Use the multi-scale convolution layer to fusion the output of the channel fusion layer Convolution kernels of size 3, 5, and 7 are used in parallel. The multi-scale convolution layer consists of 3 convolution kernels in parallel. 3 convolutional layers, 5 5 convolutional layers and 7 7 convolutional layers;

[0138] The calculation process is expressed as:

[0139] ;

[0140] in, Indicates i The output of the i convolutional layer, represents the convolution kernel of size i, Represents the Sigmoid activation function;

[0141] Step S25: The fully connected layer is used for the 3 3. Output of convolutional layer , 5 5. Output of the convolutional layer and 7 7 Output of the convolutional layer Perform feature fusion, specifically:

[0142] 3 3. Output of convolutional layer , 5 5. Output of convolutional layer and 7 7 Output of the convolutional layer Flatten them into corresponding one-dimensional vectors, that is, 3 3 Convolution of 1D vector , 5 5 Convolution of 1D vector and 7 7 Convolution of 1D vector , and then forward calculation is performed through the fully connected layer to obtain the corresponding weights, expressed as:

[0143] ;

[0144] ;

[0145] ;

[0146] in, 3 respectively 3 Convolution of 1D vector , 5 5 Convolution of 1D vector and 7 7 Convolution of 1D vector The corresponding weights, are respectively the learnable weight matrices;

[0147] 3 3 Convolution of 1D vector , 5 5 Convolution of 1D vector and 7 7 Convolution of 1D vector Multiplying and adding the corresponding weights respectively gives the final output, which is the enhanced radial artery ultrasound image:

[0148] ;

[0149] in, Represents enhanced ultrasound image of the radial artery.

[0150] Compared with the traditional single-scale CNN, the use of image enhancement module has significant advantages in radial artery ultrasound images. The traditional single-scale CNN usually only focuses on the features of a specific scale when processing images, and it is difficult to fully capture the vascular structures of different scales in the image. The multi-scale image enhancement module can process features of different scales at the same time, from local to global. By extracting features at different levels and performing feature fusion, it enhances the perception of the edge and contrast of the radial artery, allowing the network to more comprehensively understand and capture the radial artery information in the image. Such a multi-scale processing mechanism enables the network to better adapt to vascular structures of different scales and morphologies, and improves the clarity and recognition accuracy of radial artery ultrasound images. In addition, the multi-scale image enhancement module can also enhance the robustness of the network, reduce the risk of overfitting, and improve the adaptability of the model under different image transformations. In summary, the image enhancement module has obvious advantages in the process of radial artery semantic segmentation, which can improve image quality, recognition accuracy and model robustness.

[0151] Furthermore, in one example, step S3 is specifically:

[0152] The semantic labels of the radial artery ultrasound image sample data collected in step S1 are paired with the radial artery ultrasound image enhanced in step S2, and are randomly divided into a training set and a test set in a ratio of 8:2 for training and testing a block feature extraction module and a radial artery ultrasound image semantic segmentation model based on a U-net architecture;

[0153] The radial artery ultrasound image sample data collected in step S1 is paired with the radial artery ultrasound image enhanced in step S2, wherein the enhanced radial artery ultrasound image is used as a label and is randomly divided into a training set and a test set in a ratio of 8:2 for training and testing of the image enhancement module.

[0154] Further, in one example, the block feature extraction module in step S4 includes a preprocessing module, a feature extraction module and an image extraction module connected in series;

[0155] Among them, the feature extraction module includes upsampling layers of different scales, lightweight channel attention layers, spatial attention layers, gating units and feature splicing layers; the lightweight channel attention layer adopts the ECA-NET module;

[0156] Step S4 is specifically as follows:

[0157] Step S41: The preprocessing module divides the enhanced radial artery ultrasound image into equal intervals using an image block method. small blocks, and obtain the radial artery ultrasound image after N blocks;

[0158] Step S42: The feature extraction module extracts features from the divided radial artery ultrasound images to obtain extracted radial artery ultrasound images; specifically:

[0159] Step S421: Input the radial artery ultrasound image after the block division in step S41 into upsampling layers of different scales. The upsampling layers of different scales upsample the input images respectively to obtain feature maps of different scales. The calculation process is as follows:

[0160] ;

[0161] in, Represents the image input The feature map obtained after upsampling layers, Indicates The upsampling operation performed by the upsampling layer, This is the ultrasound image of the radial artery after segmentation. ;

[0162] Step S422: The feature map Input lightweight channel attention layer to obtain channel attention weight , the calculation process is as follows:

[0163] ;

[0164] in, is the Sigmoid activation function, and Represent the maximum pooling and average pooling operations respectively, is a multi-layer perceptron;

[0165] Will and Perform product operation to obtain channel attention features ;

[0166] Step S423: Channel attention features Input spatial attention layer to obtain channel attention weights , the calculation process is as follows:

[0167] ;

[0168] in, and Represent the maximum pooling and average pooling operations respectively, is the convolutional layer;

[0169] Step S424: Set the channel attention weight Input the gated unit GRU to obtain the weight after adjusting the weight ,Will and Perform product operation to obtain gated attention features ;

[0170] Step S425: Gated attention features at three different scales Perform feature splicing to obtain fusion features , that is, extract the radial artery ultrasound image, expressed as:

[0171] ;

[0172] in, Represents a splicing and fusion operation;

[0173] Step S43: The image extraction module performs feature extraction on the radial artery ultrasound image extracted in step S42, and obtains the final extracted feature map through weighted fusion calculation; specifically:

[0174] Step S431: sequentially obtain the extracted radial artery ultrasound image of each small image block and mark it as , expressed as:

[0175] ;

[0176] in, represents the operation of extracting the radial artery ultrasound image from each small image. The first segment of the radial artery ultrasound image Small pieces of images, Indicates the extracted Extraction of small pieces of images from radial artery ultrasound images, ;

[0177] Step S432: Feature matrix conversion: The extracted radial artery ultrasound image of each small block image obtained in step S431 is sequentially converted to the linear transformation matrix Multiply them together to get Extraction of small blocks of images The radial artery ultrasound image is converted into the corresponding one-dimensional feature vector, expressed as:

[0178] ;

[0179] in, Indicates Extraction of small image blocks: One-dimensional feature vector corresponding to the radial artery ultrasound image;

[0180] Step S433: Multi-head attention weight calculation: Calculate the Extraction of small blocks of images The attention weight corresponding to the radial artery ultrasound image is expressed as:

[0181] ;

[0182] in, represents the calculated attention weight corresponding to the extracted radial artery ultrasound image of the i-th small image, Indicates Extract the transpose of the one-dimensional feature vector corresponding to the radial artery ultrasound image. represents the normalization operation function, ;

[0183] Step S434: weighted image feature calculation: using the calculated attention weight corresponding to the nth small image block and its corresponding The feature matrix of a small image Multiply them together to get the weighted image feature matrix ,and , expressed as:

[0184] ;

[0185] Step S435: Obtain the weighted feature matrix of all small blocks of images Combine to obtain the weighted extracted feature map , expressed as:

[0186] .

[0187] Furthermore, in one example, step S5 is specifically as follows:

[0188] Step S51: constructing a radial artery ultrasound image semantic segmentation model based on a U-net architecture, the model consisting of 4 encoders, 4 decoders, a Softmax activation function layer and an attention gating unit;

[0189] The encoder and decoder are connected by skip connection, and the attention gating unit is embedded in the skip connection path of the encoder and decoder. The attention gating unit is used as a filter to selectively highlight the radial artery features in the feature map output by the encoder, while suppressing irrelevant environmental features. The practice of embedding the attention gating unit between the skip connection path of the encoder and the decoder can improve the accuracy of the semantic segmentation of the model. The output of the last layer of decoder is input into the fully connected layer to obtain the semantic segmentation result of the radial artery ultrasound image.

[0190] Step S52: The encoder is composed of a depth-separable convolutional layer, a global average pooling layer and a downsampling pooling layer, and the decoder structure is symmetrical to the encoder, both of which are composed of a depth-separable convolutional layer, a global average pooling layer and an upsampling pooling layer;

[0191] For The calculation formula is as follows:

[0192] ;

[0193] in, For the The output features of the encoder are It is the feature map extracted Representation in latent feature space, It is The output features of the encoder; It is a downsampling pooling operation; is the trainable weight parameter of the encoder; is a depth-separable convolution operation; the input of the first encoder is the extracted feature map ;

[0194] Step S53: The output features of the encoder Input the attention gate unit, the calculation formula is as follows:

[0195] ;

[0196] in, represents the output of the attention gating unit, Represents the attention gating mechanism;

[0197] Step S54: The output of the attention gating unit Enter the The first encoder jump connection The restored features are obtained in the decoder , expressed as:

[0198] ;

[0199] in, is the upsampling pooling operation, Represents the trainable weight parameters of the decoder;

[0200] Step S55: The output of the last layer of decoder Input the Softmax layer to obtain the semantic segmentation result, which is expressed as:

[0201] ;

[0202] in, is the probability that each pixel in the feature map corresponds to the radial artery;

[0203] The pixels in P whose probability is greater than a preset probability threshold are classified as pixels constituting the radial artery to obtain a segmented image.

[0204] The present invention is described in detail above in conjunction with the embodiments of the accompanying drawings. A person skilled in the art can make various variations of the present invention according to the above description. Therefore, some details in the embodiments should not be construed as limiting the present invention, and the present invention shall be protected by the scope defined by the attached claims.

Claims

1. A deep learning-based ultrasound-guided radial artery assisted puncture method, characterized in that: The following steps are involved: Step S1: Collect radial artery ultrasound image sample data and construct a data set based on it; Step S2: constructing an image enhancement module, importing the radial artery ultrasound image sample data in step S1 into the image enhancement module, enhancing the features of the radial artery ultrasound image sample data in step S1, and obtaining an enhanced radial artery ultrasound image; Step S3: constructing a training set and a test set based on the radial artery ultrasound image enhanced in step S2; Step S4: constructing a block feature extraction module, which cuts the radial artery ultrasound image into several blocks through image segmentation and then extracts features from each small block of the image, thereby capturing local features in the radial artery ultrasound image to obtain an extracted feature map; Step S5: constructing a radial artery ultrasound image semantic segmentation model based on a U-net architecture, the model performs semantic segmentation on the extracted feature map in step S4, and obtains a segmented image to calibrate the position of the radial artery; The model consists of 4 encoders, 4 decoders, a Softmax activation function layer, and an attention gating unit; the encoder consists of a depthwise separable convolutional layer, a global average pooling layer, and a downsampling pooling layer, and the decoder structure is symmetrical to the encoder; During the semantic segmentation process, the encoder and decoder obtain high-level semantic features of the extracted feature map to highlight the position of the radial artery in the segmented image; the average pooling layer of the encoder and decoder reduces the number of parameters and suppresses model overfitting; the attention gating unit filters irrelevant environmental features output by the encoder to the decoder to further highlight the position of the radial artery in the segmented image.

2. The method for ultrasound-guided radial artery puncture based on deep learning as claimed in claim 1, characterized in that: The collected radial artery ultrasound image sample data in step S1 is specifically: Step S11: applying coupling agent to the radial artery of the patient to reduce the influence of air on the ultrasonic signal, scanning the radial artery with a probe, and collecting a number of radial artery ultrasonic images at different angles and depths; Step S12: Open the acquired radial artery ultrasound image in LabelImg annotation software, manually identify the radial artery area in the image, distinguish the boundary between the radial artery area and the surrounding tissue or background, and use the annotation tool to draw a closed curve along the boundary of the radial artery; Step S13: labeling the closed curve drawn in step S12 with a semantic label.

3. The method for ultrasound-guided radial artery puncture based on deep learning as claimed in claim 2, characterized in that: The image enhancement module in step S2 includes a multi-scale pooling layer, a channel convolution layer, a channel fusion layer, a multi-scale convolution layer and a fully connected layer in series; Step S2 is specifically as follows: Step S21: Multi-scale pooling layer for radial artery ultrasound image Perform pooling processing; expressed as: ； ； in, and They represent the radial artery ultrasound images when the pooling kernel is i. The output after average pooling and maximum pooling, represents the convolution kernel size, represents the average pooling operation, represents the maximum pooling operation, Indicates the image boundary filling operation, i={3, 5, 7}; By using boundary padding to ensure that the output length is consistent after processing with different pooling kernels, the output of the multi-scale pooling layer is finally obtained. ; Step S22: The channel convolution layer captures the feature information of different channels in the radial artery ultrasound image by applying convolution operations independently on each channel; each channel in the channel convolution layer is provided with an independent convolution kernel, and these convolution kernels are convolved with the corresponding channels of the input image respectively; the calculation process is as follows: ； in, It represents the output of the channel convolution layer. The number of channels is consistent with the input of the corresponding layer of the channel convolution layer, which is 6. represents the convolution kernel, Represents the RELU activation function; Step S23: the channel fusion layer adaptively assigns weights to each channel; The output of step S22 All perform independent full connection calculations, and the calculation process is as follows: ； in, represents the calculated weight score of the i-th channel, represents the output of the i-th channel of the channel convolution layer, and Represent the corresponding weight matrix and bias vector respectively; Then calculate the weight score Multiply and add the corresponding outputs of step S22 respectively to obtain the output of the channel fusion layer, which is expressed as: ； in, Represents the output of the channel fusion layer; Step S24: Use the multi-scale convolution layer to fusion the output of the channel fusion layer Convolution kernels of size 3, 5, and 7 are used in parallel. The multi-scale convolution layer consists of 3 convolution kernels in parallel. 3 convolutional layers, 5 5 convolutional layers and 7 7 convolutional layers; The calculation process is expressed as: ； in, Indicates i The output of the i convolutional layer, represents the convolution kernel of size i, Represents the Sigmoid activation function; Step S25: The fully connected layer is used for the 3 3. Output of convolutional layer , 5 5. Output of the convolutional layer and 7 7 Output of the convolutional layer Perform feature fusion, specifically: 3 3. Output of convolutional layer , 5 5. Output of the convolutional layer and 7 7 Output of the convolutional layer Flatten them into corresponding one-dimensional vectors, that is, 3 3 Convolution of 1D vector , 5 5 Convolution of 1D vector and 7 7 Convolution of 1D vector , and then forward calculation is performed through the fully connected layer to obtain the corresponding weights, expressed as: ； ； ； in, 3 respectively 3 Convolution of 1D vector , 5 5 Convolution of 1D vector and 7 7 Convolution of 1D vector The corresponding weights, are respectively the learnable weight matrices; 3 3 Convolution of 1D vector , 5 5 Convolution of 1D vector and 7 7 Convolution of 1D vector Multiplying and adding the corresponding weights respectively gives the final output, which is the enhanced radial artery ultrasound image: ； in, Represents enhanced ultrasound image of the radial artery.

4. The method for ultrasound-guided radial artery puncture based on deep learning as claimed in claim 3, characterized in that: Step S3 is specifically as follows: The semantic labels of the radial artery ultrasound image sample data collected in step S1 are paired with the radial artery ultrasound image enhanced in step S2, and are randomly divided into a training set and a test set in a ratio of 8:2; The radial artery ultrasound image sample data collected in step S1 is paired with the radial artery ultrasound image enhanced in step S2, wherein the enhanced radial artery ultrasound image is used as a label and is randomly divided into a training set and a test set in a ratio of 8:

2.

5. The method for ultrasound-guided radial artery assisted puncture based on deep learning as claimed in claim 4, characterized in that: The block feature extraction module in step S4 includes a preprocessing module, a feature extraction module and an image extraction module connected in series; Among them, the feature extraction module includes upsampling layers of different scales, lightweight channel attention layers, spatial attention layers, gating units and feature splicing layers; the lightweight channel attention layer adopts the ECA-NET module; Step S4 is specifically as follows: Step S41: The preprocessing module divides the enhanced radial artery ultrasound image into equal intervals through the image block method. small blocks, and obtain the radial artery ultrasound image after N blocks; Step S42: The feature extraction module extracts features from the divided radial artery ultrasound images to obtain extracted radial artery ultrasound images; specifically: Step S421: Input the radial artery ultrasound image after the block division in step S41 into upsampling layers of different scales. The upsampling layers of different scales upsample the input images respectively to obtain feature maps of different scales. The calculation process is as follows: ； in, Represents the image input The feature map obtained after upsampling layers, Indicates The upsampling operation performed by the upsampling layer, This is the ultrasound image of the radial artery after segmentation. ; Step S422: The feature map Input lightweight channel attention layer to obtain channel attention weight , the calculation process is as follows: ； in, is the Sigmoid activation function, and Represent the maximum pooling and average pooling operations respectively, is a multi-layer perceptron; Will and Perform product operation to obtain channel attention features ; Step S423: Channel attention features Input spatial attention layer to obtain channel attention weights , the calculation process is as follows: ； in, and Represent the maximum pooling and average pooling operations respectively, is the convolutional layer; Step S424: Set the channel attention weight Input the gated unit GRU to obtain the weight after adjusting the weight ,Will and Perform product operation to obtain gated attention features ; Step S425: Gated attention features at three different scales Perform feature splicing to obtain fusion features , that is, extract the radial artery ultrasound image, expressed as: ； in, Represents a splicing and fusion operation; Step S43: The image extraction module performs feature extraction on the radial artery ultrasound image extracted in step S42, and obtains the final extracted feature map through weighted fusion calculation; specifically: Step S431: sequentially obtain the extracted radial artery ultrasound image of each small image block and mark it as , expressed as: ； in, represents the operation of extracting the radial artery ultrasound image from each small image. The first segment of the radial artery ultrasound image Small pieces of images, Indicates the extracted Extraction of small pieces of images from radial artery ultrasound images, ; Step S432: Feature matrix conversion: The extracted radial artery ultrasound image of each small block image obtained in step S431 is sequentially converted to the linear transformation matrix Multiply them together to get Extraction of small blocks of images The radial artery ultrasound image is converted into the corresponding one-dimensional feature vector, expressed as: ； in, Indicates Extraction of small image blocks: One-dimensional feature vector corresponding to the radial artery ultrasound image; Step S433: Multi-head attention weight calculation: Calculate the Extraction of small blocks of images The attention weight corresponding to the radial artery ultrasound image is expressed as: ； in, represents the calculated attention weight corresponding to the extracted radial artery ultrasound image of the i-th small image, Indicates Extract the transpose of the one-dimensional feature vector corresponding to the radial artery ultrasound image. represents the normalization operation function, ; Step S434: weighted image feature calculation: using the calculated attention weight corresponding to the nth small image block and the corresponding The feature matrix of a small image Multiply them together to get the weighted image feature matrix ,and , expressed as: ； Step S435: Obtain the weighted feature matrix of all small blocks of images Combine to obtain the weighted extracted feature map , expressed as: 。 6. The method for ultrasound-guided radial artery puncture based on deep learning as claimed in claim 5, characterized in that: Step S5 is specifically as follows: Step S51: constructing a radial artery ultrasound image semantic segmentation model based on a U-net architecture, the model consisting of 4 encoders, 4 decoders, a Softmax activation function layer and an attention gating unit; Step S52: The encoder is composed of a depth-separable convolutional layer, a global average pooling layer and a downsampling pooling layer, and the decoder structure is symmetrical to the encoder, both of which are composed of a depth-separable convolutional layer, a global average pooling layer and an upsampling pooling layer; For The calculation formula is as follows: ； in, For the The output features of the encoder are It is the feature map extracted Representation in latent feature space, It is The output features of the encoder; It is a downsampling pooling operation; is the trainable weight parameter of the encoder; is a depth-separable convolution operation; the input of the first encoder is the extracted feature map ; Step S53: The output features of the encoder Input the attention gate unit, the calculation formula is as follows: ； in, represents the output of the attention gating unit, Represents the attention gating mechanism; Step S54: The output of the attention gating unit Enter the The first encoder jump connection The restored features are obtained in the decoder , expressed as: ； in, is the upsampling pooling operation, Represents the trainable weight parameters of the decoder; Step S55: Input the output of the last layer of decoder into the Softmax layer to obtain the semantic segmentation result, which is expressed as: ； in, is the probability that each pixel in the feature map corresponds to the radial artery; The pixels in P whose probability is greater than a preset probability threshold are classified as pixels constituting the radial artery to obtain a segmented image.

Images