Improved method and application of U-Net++ model based on attention mechanism

Abstract

The application provides an attention mechanism-based U-Net++ model improvement method and application, and relates to the technical field of image processing.The method comprises the following steps: collecting a medical image by using a medical diagnosis device; and performing segmentation and identification on the medical image by using a trained U-Net++ model.The U-Net++ model comprises a down-sampling structure, a dense skip connection structure and an up-sampling structure; wherein a boundary-aware spatial attention mechanism module (BASA) is arranged in at least one of the down-sampling structure, the dense skip connection structure and the up-sampling structure; the BASA module uses three branch paths to respectively perform channel attention mechanism, spatial attention mechanism and boundary-aware attention mechanism.The application can improve the segmentation performance of a medical image, and can effectively process the medical image in the case of image boundary blur.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an improved method of U-Net++ model based on attention mechanism. BACKGROUND

[0002] Image segmentation technology has a wide range of applications in the field of medical image analysis. The goal of image segmentation is to divide an image into multiple meaningful regions, where each region represents a different object or structure. In many medical image processing tasks, accurately segmenting the boundaries of objects is crucial.

[0003] Currently, the U-Net++ model is considered one of the more effective methods for processing medical image segmentation due to its unique model structure. However, in actual operation, the original U-Net++ model still has certain limitations when processing image boundaries. Specifically, the following situations exist:

[0004] In the first situation, the image boundary has pixel-level differences. Although the dense skip connection of the U-Net++ model can help enhance information flow, when dealing with subtle boundaries, the low-resolution information near the boundary may not be perfectly captured. Therefore, the local features of the edge may not be fully focused.

[0005] In the second situation, during multi-level feature extraction, for example, in cases where the contrast between the background and the target is weak, the U-Net++ model may not be able to capture very subtle local contrasts.

[0006] In the third situation, the U-Net++ model may cause over-smoothing at the image boundary when fusing information from different layers through dense skip connections. This smoothing may affect small objects or areas with very delicate boundaries, causing the U-Net++ model to fail to preserve the details of these areas.

[0007] Therefore, the present application provides an improved method of U-Net++ model based on attention mechanism and its application to solve the problem of low image recognition accuracy when the image boundary is fuzzy, which is a technical problem that needs to be solved urgently. SUMMARY

[0008] The present application aims to overcome the shortcomings of the prior art and provide an improved method of U-Net++ model based on attention mechanism and its application. The present application aims to improve the segmentation performance of medical images, so that medical images can be effectively processed even when the image boundary is fuzzy.

[0009] To solve the existing technical problems, the present application provides the following technical solutions:

[0010] An improved method of U-Net++ model based on attention mechanism, comprising the steps of:

[0011] Acquiring a medical image using a medical diagnostic device;

[0012] Segmenting and identifying the aforementioned medical image using the trained U-Net++ model; the U-Net++ model comprises a down-sampling structure, a dense skip connection structure and an up-sampling structure; wherein a boundary-aware spatial attention mechanism module (BASA) is configured in at least one of the aforementioned down-sampling structure, dense skip connection structure and up-sampling structure, the BASA module uses three path branches to respectively perform channel attention mechanism, spatial attention mechanism and boundary-aware attention mechanism.

[0013] Further, the medical diagnostic device comprises at least one of an X-ray machine, a CT scanner, an MRI scanner, an ultrasonic diagnostic instrument, a PET scanner, a SPECT scanner, a radioisotope scanner, a mammography machine, an endoscope, an angiography machine and a functional magnetic resonance imaging device;

[0014] The medical image includes at least one of an X-ray image, a CT image, an MRI image, an ultrasound image, a PET image, a SPECT image, a radioisotope scan image, a mammography image, an endoscopic image, an angiographic image and a functional MRI image;

[0015] The U-Net++ model is trained and tested using a preset medical image dataset; wherein 80% of the medical images constitute a training set for training the aforementioned U-Net++ model; the remaining 20% of the medical images constitute a test set for verifying the aforementioned U-Net++ model.

[0016] Further, the three branch paths of the BASA module are respectively a channel attention path branch, a spatial attention path branch and a boundary-aware attention path branch; wherein,

[0017] When performing the channel attention path branch, first pass through a 1x1 convolution to compress the channel number, then calculate the size k of the convolution kernel according to the channel number, after calculating the channel attention weight through 1D convolution, use the Sigmoid function to generate the final channel attention weight, and multiply the output feature map obtained after using the Sigmoid function with the input feature map element by element to obtain the output feature map of the channel attention path branch;

[0018] When the spatial attention path branch is executed, the input feature map is first generated into a feature map with spatial weights through k×k convolution, k being a positive integer, and then the feature map with spatial weights is multiplied element by element with the output feature map of the aforementioned channel attention path branch after being activated by a Sigmoid function, to obtain the output feature map of the spatial attention path branch;

[0019] When the boundary-aware attention path branch is executed, the boundary information in the input feature map is extracted through 3×3 convolution and ReLU activation function, and the output feature map through 3×3 convolution and ReLU activation function is added element by element with the output feature map of the aforementioned spatial attention path branch, to obtain the output feature map of the boundary-aware attention path branch.

[0020] After obtaining the output feature map of the boundary-aware attention path branch, the output feature map of the aforementioned boundary-aware attention path branch is added element by element with the input feature map, to obtain the output feature map of the BASA module.

[0021] Further, at least one BASA module is configured in the down-sampling structure; for any down-sampling layer using the BASA module in the down-sampling structure, the BASA module is executed after the down-sampling layer sequentially executes convolution operation and pooling operation.

[0022] Further, at least one dense attention residual module DAR is used in the dense skip connection structure; any layer in the dense skip connection structure can receive the feature map obtained after the up-sampling operation of the previous layer and the feature map obtained after the skip connection operation, and the feature map obtained after the up-sampling operation of the previous layer and the feature map obtained after the skip connection operation are spliced; after splicing, the aforementioned DAR module is executed, and the feature map processed by the aforementioned DAR module is skip-connected to the next layer.

[0023] Further, the DAR module is configured as:

[0024] Different levels of features of the input feature map are extracted through different size of convolution respectively, to form a plurality of feature maps;

[0025] Two branch paths are generated; in one of the branch paths, the aforementioned plurality of feature maps are spliced, and after splicing, the BASA module and the 1×1 convolution are sequentially executed; in the other branch path, the aforementioned plurality of feature maps are added element by element, and then the attention mechanism module SA is executed.

[0026] The output feature maps of the two branch paths are element-wise added, and then processed by batch normalization and a ReLU activation function; and the processed feature map is element-wise added to the input feature map after 1x1 convolution, thereby obtaining the output feature map of the DAR module.

[0027] Further, at least one BASA module is configured in the upsampling structure; for any upsampling layer in the upsampling structure that uses a BASA module, the BASA module is executed after the convolution operation and the pooling operation are sequentially performed.

[0028] An attention mechanism-based U-Net++ model improvement device, comprising:

[0029] An image acquisition unit is configured to acquire medical images using a medical diagnostic device.

[0030] An image processing unit is configured to perform segmentation and recognition on the medical images using a trained U-Net++ model; the U-Net++ model comprises a downsampling structure, a dense skip connection structure, and an upsampling structure; at least one of the downsampling structure, the dense skip connection structure, and the upsampling structure is configured with a boundary-aware spatial attention mechanism module (BASA), which uses three path branches to respectively perform channel attention mechanism, spatial attention mechanism, and boundary-aware attention mechanism.

[0031] An attention mechanism-based U-Net++ model improvement system, comprising:

[0032] A medical diagnostic device is configured to acquire medical images.

[0033] A model configuration module is configured to configure a U-Net++ model.

[0034] A system server is connected to the medical diagnostic device and the model configuration module.

[0035] The system server is configured to acquire medical images using a medical diagnostic device; and perform segmentation and recognition on the medical images using a trained U-Net++ model; the U-Net++ model comprises a downsampling structure, a dense skip connection structure, and an upsampling structure; at least one of the downsampling structure, the dense skip connection structure, and the upsampling structure is configured with a boundary-aware spatial attention mechanism module (BASA), which uses three path branches to respectively perform channel attention mechanism, spatial attention mechanism, and boundary-aware attention mechanism.

[0036] A computer readable storage medium, the computer readable storage medium has a computer program stored therein, the computer program is executed by a processor to realize the implementation steps of the method of any one of the above.

[0037] Based on the above advantages and positive effects, the advantages of the present application are that a boundary-aware spatial attention mechanism module BASA is proposed, the BASA module combines channel attention, spatial attention and boundary-aware mechanism, so as to better focus on the key area in the image.

[0038] Further, the BASA module is applied to the U-Net++ model, the down-sampling structure, the dense skip connection structure and / or the up-sampling structure, to enhance the processing ability of the U-Net++ model to the image boundary, and improve the recognition accuracy of the U-Net++ model.

[0039] Further, a dense attention residual module DAR is proposed, which combines multi-scale feature fusion, BASA mechanism and residual connection to improve the recognition ability of the unclear boundary area in image segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The method flowchart provided for the embodiments of the present application.

[0041] Figure 2 The structure diagram of the U-Net++ model provided for the embodiments of the present application.

[0042] Figure 3 The structure diagram of the BASA module provided for the embodiments of the present application.

[0043] Figure 4 The structure diagram of the DAR module provided for the embodiments of the present application.

[0044] Figure 5 The structure diagram of the device provided for the embodiments of the present application.

[0045] Figure 6 The structure diagram of the system provided for the embodiments of the present application.

[0046] Explanation of reference signs:

[0047] The device 200, the image acquisition unit 201, the image processing unit 202;

[0048] The system 300, the medical diagnosis device 301, the model configuration module 302, the system server 303. DETAILED DESCRIPTION

[0049] The improved method of U-Net++ model based on attention mechanism and application are further described in detail below in combination with the accompanying drawings and specific embodiments. It should be noted that the technical features described in the following embodiments or combinations of technical features should not be considered in isolation, and they can be combined with each other to achieve better technical effects. In the drawings of the following embodiments, the same reference numerals appearing in different drawings represent the same features or components, which can be applied to different embodiments. Therefore, once a feature is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0050] It should be noted that the structures, proportions, sizes, etc. shown in the drawings attached to the present specification are only used to cooperate with the content disclosed in the specification, so that those skilled in the art can understand and read, and are not used to limit the conditions for implementing the invention. Any modification of structure, change of proportion relationship or adjustment of size, which does not affect the effect and purpose that the invention can produce, should fall within the scope of the technical content disclosed by the invention. The scope of the preferred embodiments of the present invention includes additional implementations, in which the functions can be performed in a substantially simultaneous manner or in reverse order according to the functions involved, without being performed in the order described or discussed, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0051] Techniques, methods, and equipment known to those of ordinary skill in the relevant art can not be discussed in detail, but in appropriate cases, the techniques, methods, and equipment should be considered as part of the authorized specification. In all examples shown and discussed here, any specific value should be interpreted as merely exemplary, not as a limitation. Therefore, other examples of exemplary embodiments can have different values. Embodiments

[0052] Referring to Figure 1 A flowchart is shown for providing the present invention. The implementation steps S100 of the method are as follows:

[0053] S101, collecting medical images using a medical diagnostic device.

[0054] The medical diagnostic device includes but is not limited to at least one of an X-ray machine, a CT scanner, an MRI scanner, an ultrasonic diagnostic instrument, a PET scanner, a SPECT scanner, a radioisotope scanner, a mammography machine, an endoscope, an angiography machine, and a functional magnetic resonance imaging device.

[0055] Correspondingly, the types of the medical images include, but are not limited to, at least one of an X-ray image, a CT image, an MRI image, an ultrasound image, a PET image, a SPECT image, a radioisotope scanning image, a mammogram image, an endoscopic image, an angiogram image, and a functional MRI image.

[0056] The medical diagnosis device and the types of the medical images are all conventional medical diagnosis devices and conventional image types in the art, and thus will not be described in detail herein.

[0057] S102, segmenting and identifying the aforementioned medical image using the trained U-Net++ model.

[0058] In the embodiment, the U-Net++ model is trained and tested using a preset medical image dataset; 80% of the medical images constitute a training set for training the U-Net++ model; and the remaining 20% of the medical images constitute a test set for verifying the U-Net++ model.

[0059] The medical image dataset can be a self-set medical image dataset or a public medical image dataset.

[0060] In combination with Figure 2 As shown, the U-Net++ model includes a down-sampling structure, a dense skip-connection structure, and an up-sampling structure; and at least one of the down-sampling structure, the dense skip-connection structure, and the up-sampling structure is configured with a boundary-aware spatial attention mechanism module BASA. The BASA module is an English full name.

[0061] The BASA module uses three path branches to respectively perform a channel attention mechanism, a spatial attention mechanism, and a boundary-aware attention mechanism.

[0062] Specifically, referring to Figure 3 As shown, the BASA module provided in the embodiment is a structural schematic diagram. The three branch paths of the BASA module are respectively a channel attention path branch, a spatial attention path branch, and a boundary-aware attention path branch.

[0063] Figure 3 In the formula, the symbols B, C, H, and W respectively represent batch size, channel number of a feature map, height, and width.

[0064] wherein, when the channel attention path branch is executed, a 1x1 convolution is first used to compress the number of channels to strengthen the correlation between channel features, and then the size k of the convolution kernel is adaptively calculated according to the number of channels, after the channel attention weight is calculated through 1D convolution, the final channel attention weight is generated using a Sigmoid function, and the output feature map obtained after using the Sigmoid function is multiplied element by element with the input feature map to obtain the output feature map of the channel attention path branch.

[0065] wherein, it is worth noting that in the channel attention path branch, the size k of the convolution kernel is adaptively calculated according to the number of channels, after the channel attention weight is calculated through 1D convolution, the final channel attention weight is generated using a Sigmoid function, and the output feature map obtained after using the Sigmoid function is multiplied element by element with the input feature map, which is the structure of the existing efficient channel attention mechanism ECA (Efficient Channel Attention, referred to as ECA), that is, the channel attention path branch is based on the ECA mechanism, and on the path of adaptively calculating the size k of the convolution kernel in the ECA mechanism, a 1x1 convolution is first used to compress the number of channels, which helps to strengthen the correlation between channel features in the process of executing the channel attention path branch.

[0066] Therefore, the operation advantage of the channel attention path branch is that it can automatically focus on the channel features most important to the current task and reduce the dependence on unimportant features.

[0067] The spatial attention path branch aims to capture the spatial information of the feature map and emphasize the importance of different spatial positions. When the spatial attention path branch is executed, the input feature map is first generated into a feature map with spatial weights through kxk convolution, where k is a positive integer, and k=3 is preferred here. Then, the feature map with spatial weights is multiplied element by element with the output feature map of the aforementioned channel attention path branch after being activated by a Sigmoid function to enhance the proportion weight of key features (such as target features to be recognized) in the input feature map, thereby obtaining the output feature map of the spatial attention path branch.

[0068] The operation advantage of the spatial attention path branch is that it can enhance the learning ability of the model for key features, reduce the interference of irrelevant regions (such as other features other than target features), and improve the overall performance and efficiency.

[0069] In addition, considering that the actual acquired image may have blurred image boundaries, it is very important to obtain accurate boundary information when processing the blurred image boundaries. Therefore, a boundary-aware attention path branch is introduced in the BASA module to enhance the feature expression of the boundary region.

[0070] Specifically, when performing the boundary-aware attention path branch, the boundary information in the input feature map is extracted by 3x3 convolution and ReLU activation function. The output feature map obtained by 3x3 convolution and ReLU activation function is element-wise added to the output feature map of the aforementioned spatial attention path branch to obtain the output feature map of the boundary-aware attention path branch.

[0071] After obtaining the output feature map of the boundary-aware attention path branch, the output feature map of the aforementioned boundary-aware attention path branch is element-wise added to the input feature map to obtain the output feature map of the BASA module. Under this operation, the BASA module combines the channel attention mechanism, the spatial attention mechanism and the boundary-aware attention mechanism, so that the module can focus on the local features of each position, can strengthen the boundary and structural features of the image, can effectively alleviate the gradient vanishing problem, and can retain the relevant information of the input feature map.

[0072] Based on this, it can be determined that the BASA module combines the channel attention, the spatial attention and the boundary-aware mechanism, and can perform weighted processing on the input features from multiple channel, spatial and boundary dimensions, so as to better focus on the key regions in the medical image.

[0073] In the down-sampling stage of the U-Net++ model mentioned in the embodiment, at least one BASA module is configured in the down-sampling structure; wherein for any down-sampling layer using the BASA module in the down-sampling structure, the down-sampling layer performs the BASA module after sequentially performing the convolution operation and the pooling operation.

[0074] In combination with Figure 2 As shown in the figure, the BASA module is added to at least one down-sampling layer of the down-sampling structure to ensure that the input features can still be weighted processed from multiple channel, spatial and boundary dimensions during the down-sampling process, so as to better focus on the key regions in the medical image.

[0075] As another preferred embodiment of the embodiment, in combination with Figure 2 As shown in the figure, at least one dense attention residual module DAR is used in the dense skip connection structure. The English full name of the dense attention residual module DAR is DenseAttention Residual Module.

[0076] InFigure 2 In the U-Net++ model, solid arrows and dashed arrows are used to represent down-layer sampling and skip connection, respectively. In the dense skip connection structure, any layer can receive the feature map obtained by the up-sampling operation of the previous layer and the feature map obtained after the skip connection operation, and then splice the two feature maps. After splicing, the aforementioned DAR module is executed, and the feature map processed by the DAR module is connected to the next layer.

[0077] Specifically, in combination with FIG. 1, the DAR module is configured to: Figure 4

[0078] First, different levels of features of the input feature map are extracted by different sizes of convolution to form a plurality of feature maps.

[0079] In this embodiment, it is preferred that three branches are generated when processing the input feature map, and different levels of features of the input feature map are extracted by 1x1, 3x3 and 5x5 convolution to form three feature maps.

[0080] Then, two branch paths are generated. In one of the branch paths, the aforementioned plurality of feature maps are spliced, and after splicing, the BASA module and the 1x1 convolution are sequentially executed to obtain a feature map with reduced channel dimension. In the other branch path, the aforementioned plurality of feature maps are element-wise added, and then the attention mechanism module SA is executed.

[0081] This operation helps the DAR module to focus on important feature regions and channels, thereby improving the recognition ability of the model for key regions.

[0082] Finally, the output feature maps of the two branch paths are obtained, and after element-wise addition of the output feature maps of the two branch paths, batch normalization and ReLU activation function are performed. Then, the feature map obtained after processing is element-wise added to the input feature map processed by the 1x1 convolution, thereby obtaining the output feature map of the DAR module. The DAR module is processed by batch normalization and ReLU activation function, and residual connection is added to the output, which can accelerate convergence and reduce the problem of gradient disappearance.

[0083] In this embodiment, the advantage of the DAR module is that it can combine multi-scale feature fusion, BASA mechanism and residual connection to improve the recognition ability of the model for unclear boundary regions in image segmentation, so that the U-Net++ model can adaptively select important feature regions and avoid the interference of redundant information, and ensure the efficiency and accuracy of recognition.

[0084] ​In the up-sampling stage, at least one BASA module is configured in the up-sampling structure; wherein for any up-sampling layer in the up-sampling structure using the BASA module, the up-sampling layer executes the BASA module after sequentially executing the convolution operation and the pooling operation.

[0085] In actual operation, mIOU (Mean Intersection over Union) is preferably used as an index for evaluating segmentation accuracy, i.e., an index for evaluating the segmentation performance of the U-Net++ model. The mIOU is a conventional technical term in the art, and therefore will not be described in more detail here.

[0086] Further, for the U-Net++ model provided in the present embodiment, two groups of ultrasound data sets are preferably used for training.

[0087] One group of ultrasound data sets is a lumbar plexus ultrasound data set constructed by the present embodiment, which focuses on segmenting the erector spinae muscle, the psoas muscle, the transverse process, the psoas major muscle and the lumbar plexus structure. The data set is collected from ultrasound images of 100 patients by professional technical personnel in the anesthesia department of a certain first-class hospital. When collecting, the professional technical personnel uses the convex array probe of the ultrasound equipment to collect data from the collected person, places the probe close to the patient's abdominal side, and when the video scans the abdominal external oblique muscle, the abdominal internal oblique muscle and the abdominal transverse muscle, gradually slides to the side of the iliac crest, the direction is inward, until the structure of the erector spinae muscle, the psoas muscle, the transverse process, the psoas major muscle and the lumbar plexus is seen. The ultrasound video is recorded for 30 seconds, and a total of 100 ultrasound videos are collected. Ultrasound images are captured from the 100 ultrasound videos, and manually labeled using the open source labeling tool Labelme. Preferably, 20 to 30 frames are extracted from each ultrasound video, and finally an ultrasound data set containing 1120 images is obtained.

[0088] In the preferred embodiment of the present embodiment, the U-Net++ model in the structural schematic diagram of Figure 2 is preferably used to perform the image segmentation task. Finally, the experimental results of the improved U-Net++ model compared with the conventional U-Net++ model are shown. For convenience of description, in the present experiment, the overall mIOU obtained by segmenting the psoas major muscle, the psoas muscle, the erector spinae muscle, the transverse process and the lumbar plexus by the conventional U-Net++ model is 0.820, while using the U-Net++ model in the present embodiment for recognition, the overall mIOU obtained is 0.824, which is improved by 0.04 compared with the conventional U-Net++ model.

[0089] In another experiment, to verify the performance of the improved U-Net++ model on the public dataset, the ultrasound dataset is preferably the brachial plexus dataset, which is from the reference: Ding Y, Member I, Yang Q, et al. MallesNet: A multi-object assistance based network for brachial plexus segmentation in ultrasound images[J]. Medical Image Analysis, 2022, 80: 102511. In the case of using the U-Net++ model in the structural diagram of Figure 2 , to perform the image segmentation task, the final experimental results show that using the conventional U-Net++ model for recognition, the overall mIOU obtained by segmenting the nerve, muscle, vein and artery is 0.813, while using the U-Net++ model in the embodiment for recognition, the overall mIOU obtained is 0.818, which is 0.05 higher than the mIOU of the conventional U-Net++ model.

[0090] Therefore, from the experimental results, it can be known that the U-Net++ model designed in the embodiment obtains good results in the above two datasets.

[0091] Other technical features refer to the previous embodiments, which will not be repeated here.

[0092] In addition, in combination with Figure 5 , the present application also gives an embodiment, which provides an improved device 200 of U-Net++ model based on attention mechanism, comprising:

[0093] The image acquisition unit 201 is used to acquire medical images using medical diagnostic equipment.

[0094] The image processing unit 202 is used to segment and identify the aforementioned medical images using the trained U-Net++ model; the U-Net++ model comprises a down-sampling structure, a dense skip connection structure and an up-sampling structure; wherein at least one of the down-sampling structure, the dense skip connection structure and the up-sampling structure is configured with a boundary-aware spatial attention mechanism module BASA, which uses three path branches to respectively execute channel attention mechanism, spatial attention mechanism and boundary-aware attention mechanism.

[0095] Other technical features refer to the previous embodiments, which will not be repeated here.

[0096] In addition, in combination with Figure 6As shown, the application also gives an embodiment, provides an attention mechanism-based U-Net++ model improvement system 300, which comprises:

[0097] A medical diagnosis device 301 is configured to collect medical images.

[0098] A model configuration module 302 is configured to configure a U-Net++ model.

[0099] A system server 303 is connected to the medical diagnosis device 301 and the model configuration module 302.

[0100] The system server 303 is configured to: collect medical images using the medical diagnosis device; and perform segmentation and identification on the medical images using the trained U-Net++ model; the U-Net++ model comprises a down-sampling structure, a dense skip connection structure and an up-sampling structure; and a boundary-aware spatial attention mechanism module BASA is configured in at least one of the down-sampling structure, the dense skip connection structure and the up-sampling structure, the BASA module uses three path branches to respectively perform channel attention mechanism, spatial attention mechanism and boundary-aware attention mechanism.

[0101] Other technical features are described in the preceding embodiments and will not be repeated here.

[0102] In addition, the application also provides a computer readable storage medium having a program stored thereon, which is used in the aforementioned attention mechanism-based U-Net++ model improvement system, and when the program is executed by a processor, the steps of the aforementioned any one of the attention mechanism-based U-Net++ model improvement methods can be implemented.

[0103] Other technical features are described in the preceding embodiments and will not be repeated here.

[0104] In the above description, within the target protection scope of the present disclosure, each component can be selectively and operatively combined in any number. In addition, the terms such as "include", "comprise" and "have" should be interpreted as inclusive or open, rather than exclusive or closed, unless they are explicitly defined as the opposite meaning. All technical, scientific or other terms are in accordance with the meaning understood by those skilled in the art, unless they are defined as the opposite meaning. The common terms found in the dictionary should not be interpreted too idealistically or too unrealistically in the context of relevant technical documents, unless the present disclosure explicitly limits them in this way.

[0105] Although example aspects of the present disclosure have been described with respect to the description above, it will be recognized by one of ordinary skill in the art that the description is merely illustrative of the preferred embodiments of the application and is not intended to limit the scope of the application, which is defined by the appended claims. The scope of the preferred embodiments of the application includes additional implementations that can not appear to be described or claimed in the description above. Any alterations, modifications, additions, or omissions made by one of ordinary skill in the art based on the disclosure above are intended to fall within the scope of the claims.

Claims

1. A method for improving the U-Net++ model based on an attention mechanism, characterized in that, Including the following steps: Medical images are acquired using medical diagnostic equipment; The aforementioned medical images were segmented and recognized using a trained U-Net++ model; the U-Net++ model includes a downsampling structure, a dense skip connection structure, and an upsampling structure. Specifically, at least one of the aforementioned downsampling structure, dense skip connection structure, and upsampling structure is configured with a boundary-aware spatial attention mechanism module (BASA). The BASA module uses three path branches to execute the channel attention mechanism, spatial attention mechanism, and boundary-aware attention mechanism, respectively. The three branch paths of the BASA module are the channel attention path branch, the spatial attention path branch, and the boundary-aware attention path branch, respectively. When executing the channel attention path branch, the number of channels is first compressed by 1×1 convolution, and then the size k of the convolution kernel is adaptively calculated according to the number of channels. After calculating the channel attention weights by 1D convolution, the final channel attention weights are generated by using the Sigmoid function. The output feature map obtained after using the Sigmoid function is multiplied element-wise with the input feature map to obtain the output feature map of the channel attention path branch. When executing the spatial attention path branch, the input feature map is first generated into a feature map with spatial weights through k×k convolution, where k is a positive integer. Then, the feature map with spatial weights is activated by the Sigmoid function and multiplied element-wise with the output feature map of the aforementioned channel attention path branch to obtain the output feature map of the spatial attention path branch. When executing the boundary-aware attention path branch, boundary information in the input feature map is extracted by using 3×3 convolution and ReLU activation function. The output feature map obtained by the 3×3 convolution and ReLU activation function is added element-wise to the output feature map of the aforementioned spatial attention path branch to obtain the output feature map of the boundary-aware attention path branch. After obtaining the output feature map of the boundary-aware attention path branch, the output feature map of the aforementioned boundary-aware attention path branch is added element by element to the input feature map to obtain the output feature map of the BASA module.

2. The method according to claim 1, characterized in that, The medical diagnostic equipment includes at least one of the following: X-ray machine, CT scanner, MRI scanner, ultrasound diagnostic instrument, PET scanner, SPECT scanner, radioisotope scanner, mammography machine, endoscope, angiography machine, and functional magnetic resonance imaging equipment. The types of medical images include at least one of the following: X-ray images, CT images, MRI images, ultrasound images, PET images, SPECT images, radioisotope scan images, mammogram images, endoscopic images, angiography images, and functional MRI images. The U-Net++ model is trained and tested using a pre-defined medical image dataset; 80% of the medical images are used as the training set for training the aforementioned U-Net++ model; and the remaining 20% ​​of the medical images are used as the test set for validating the aforementioned U-Net++ model.

3. The method according to claim 1, characterized in that, The downsampling structure is configured with at least one BASA module; wherein, for any downsampling layer in the downsampling structure that uses a BASA module, the BASA module is executed after the downsampling layer has sequentially performed convolution and pooling operations.

4. The method according to claim 1, characterized in that, The dense skip connection structure uses at least one dense attention residual module (DAR); wherein, any layer in the dense skip connection structure can receive a feature map from the previous layer's upsampling operation and a feature map obtained after the skip connection operation, and concatenate the feature map obtained after the previous layer's upsampling operation and the feature map obtained after the skip connection operation; after concatenation, the aforementioned DAR module is executed to skip-connect the feature map processed by the aforementioned DAR module to the next layer.

5. The method according to claim 4, characterized in that, The DAR module is configured as follows: Different features at different levels of the input feature map are extracted by convolutions of different sizes, forming multiple feature maps; Two branch paths are generated; in one branch path, the aforementioned multiple feature maps are concatenated, and after concatenation, the BASA module and 1×1 convolution are executed sequentially; in the other branch path, the aforementioned multiple feature maps are added element by element, and then the attention mechanism module SA is executed. The output feature maps of the two branch paths are obtained. After adding the output feature maps of the two branch paths element by element, they are processed by batch normalization and ReLU activation function. The processed feature map is then added element by element to the input feature map after 1x1 convolution to obtain the output feature map of the DAR module.

6. The method according to claim 1, characterized in that, The upsampling structure is configured with at least one BASA module; wherein, for any upsampling layer in the upsampling structure that uses a BASA module, the upsampling layer executes the BASA module after sequentially performing convolution and pooling operations.

7. A U-Net++ model improvement device based on an attention mechanism according to any one of claims 1-6, characterized in that... include: Image acquisition unit, used to acquire medical images using medical diagnostic equipment; An image processing unit is used to segment and recognize the aforementioned medical images using a trained U-Net++ model. The U-Net++ model includes a downsampling structure, a dense skip connection structure, and an upsampling structure. At least one of the downsampling, dense skip connection, and upsampling structures is configured with a boundary-aware spatial attention mechanism (BASA) module. The BASA module uses three path branches to execute channel attention, spatial attention, and boundary-aware attention mechanisms, respectively. The three branches of the BASA module are channel attention path branch, spatial attention path branch, and boundary-aware attention path branch. When executing the channel attention path branch, a 1×1 convolution is first used to compress the number of channels. Then, the kernel size k is adaptively calculated based on the number of channels. After calculating the channel attention weights through 1D convolution, the Sigmoid function is used to generate the final channel attention weights. The output feature map obtained after using the Sigmoid function is then compared with the input... The feature maps are multiplied element-wise to obtain the output feature map of the channel attention path branch. When executing the spatial attention path branch, the input feature map is first converted into a spatially weighted feature map by k×k convolution, where k is a positive integer. Then, the spatially weighted feature map is activated by the Sigmoid function and multiplied element-wise with the output feature map of the aforementioned channel attention path branch to obtain the output feature map of the spatial attention path branch. When executing the boundary-aware attention path branch, boundary information in the input feature map is extracted by 3×3 convolution and ReLU activation function. The output feature map of the 3×3 convolution and ReLU activation function is added element-wise with the output feature map of the aforementioned spatial attention path branch to obtain the output feature map of the boundary-aware attention path branch. After obtaining the output feature map of the boundary-aware attention path branch, the output feature map of the aforementioned boundary-aware attention path branch is added element-wise with the input feature map to obtain the output feature map of the BASA module.

8. A U-Net++ model improvement system based on an attention mechanism according to any one of claims 1-6, characterized in that... include: Medical diagnostic equipment used to acquire medical images; The model configuration module is used to configure U-Net++ models; System server, which connects to medical diagnostic equipment and model configuration module; The system server is configured to: acquire medical images using medical diagnostic equipment; segment and recognize the aforementioned medical images using a trained U-Net++ model; the U-Net++ model includes a downsampling structure, a dense skip connection structure, and an upsampling structure; wherein, at least one of the aforementioned downsampling structure, dense skip connection structure, and upsampling structure is configured with a boundary-aware spatial attention mechanism module (BASA), the BASA module using three path branches to execute the channel attention mechanism, spatial attention mechanism, and boundary-aware attention mechanism respectively; the three branch paths of the BASA module are the channel attention path branch, the spatial attention path branch, and the boundary-aware attention path branch; wherein, when executing the channel attention path branch, the number of channels is first compressed by a 1×1 convolution, then the size k of the convolution kernel is adaptively calculated according to the number of channels, after calculating the channel attention weights through 1D convolution, the final channel attention weights are generated using the Sigmoid function, and the weights obtained after using the Sigmoid function are... The output feature map is multiplied element-wise with the input feature map to obtain the output feature map of the channel attention path branch. When executing the spatial attention path branch, the input feature map is first generated into a spatially weighted feature map through a k×k convolution, where k is a positive integer. Then, the spatially weighted feature map is activated by the Sigmoid function and multiplied element-wise with the output feature map of the aforementioned channel attention path branch to obtain the output feature map of the spatial attention path branch. When executing the boundary-aware attention path branch, boundary information in the input feature map is extracted through a 3×3 convolution and a ReLU activation function. The output feature map obtained through the 3×3 convolution and ReLU activation function is added element-wise with the output feature map of the aforementioned spatial attention path branch to obtain the output feature map of the boundary-aware attention path branch. After obtaining the output feature map of the boundary-aware attention path branch, the output feature map of the aforementioned boundary-aware attention path branch is added element-wise with the input feature map to obtain the output feature map of the BASA module.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

Images