Abdominal transverse muscle plane tissue structure recognition method based on RMSD-Net and application

Abstract

The invention provides an RMSD-Net-based abdominal transverse muscle plane tissue structure identification method and application, and relates to the technical field of image processing. The method comprises the following steps: performing data preprocessing on an obtained ultrasonic image with an abdominal transverse muscle plane structure, and establishing an abdominal ultrasonic image data set; wherein the ultrasonic image with the abdominal transverse muscle plane structure comprises key abdominal tissue structure information which can be observed in the TAP block process; an RMSD-Net model of a U-shaped network structure is constructed; the RMSD-Net model comprises an encoder, a jump connection structure and a decoder; a multi-scale residual error feature extraction module RMDC module is correspondingly added in each layer of the encoder; the jump connection structure uses a feature fusion module SFF module layer by layer; adding an attention mechanism GAM module in the last layer of the decoder; executing a segmentation task aiming at the key abdominal tissue structure information, dividing the abdominal ultrasound image data set into a training set and a test set, and respectively training and testing by using the RMSD-Net model; and using the tested RMSD-Net model to perform segmentation identification of key abdominal tissue structure information on the newly obtained ultrasonic image with the abdominal transverse muscle plane structure. According to the method, a clinician can be helped to quickly and accurately interpret an ultrasonic image with an abdominal transverse muscle plane structure, so that the clinician is assisted to accurately position a retardation area and inject local anesthetics.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and application of transverse abdominal muscle plane tissue structure recognition based on RMSD-Net. Background Art

[0002] Nerve block (NB) is a technique that uses various physical and chemical methods to act on ganglia, roots, plexuses, trunks and nerve endings to temporarily or permanently block their conduction function. It is widely used in the clinical diagnosis and treatment of chronic pain diseases and regional anesthesia. Traditional peripheral nerve block technology usually relies on surface paresthesia, nerve stimulation and doctor's experience to locate nerves and block areas, which is inefficient and has many postoperative complications.

[0003] In recent years, with the development and popularization of ultrasound technology, ultrasound imaging provides a visual field, and ultrasound-guided nerve block has become the mainstream method for anesthesiologists and pain physicians to implement nerve block, greatly improving accuracy and safety.

[0004] Among them, ultrasound-guided transversus abdominis plane block (TAPB) is a widely used nerve block technique in clinical practice. It is to inject local anesthetics into the neurofascial plane between the internal oblique muscle and the transverse abdominal muscle, which can block the afferent nerves of T6-L1 and provide effective analgesia for various abdominal surgeries, such as gynecological laparoscopic surgery, laparoscopic colorectal surgery, pediatric hernia surgery, and cesarean section. TAP block includes four approaches: subcostal, anterior, posterior, and lateral. Different approaches have different blocking ranges and corresponding clinical applications. The appropriate blocking approach can be selected according to clinical needs.

[0005] With the increasing clinical application of TAP block, in order to meet the requirements of comfortable medical care and precise diagnosis and treatment, ultrasound-guided TAP block has gradually become a gold standard for regional block anesthesia.

[0006] However, whether it is traditional TAP block or ultrasound-guided TAP block, incorrect or inaccurate positioning of the block area may lead to poor anesthetic effect, and there is a risk of accidentally entering the nerves and blood vessels, resulting in complications such as local anesthetic poisoning, local hematoma, and nerve damage.

[0007] Due to the low spatial resolution and speckle noise of ultrasound imaging, subtle anatomical features cannot be clearly distinguished from the surrounding background by the naked eye. In particular, clinicians who have only received limited ultrasound image reading training face great challenges in interpreting ultrasound images.

[0008] At the same time, for blockade targets that are located deeper, poor ultrasound imaging quality may result in the puncture needle not being able to be clearly visualized. In this case, blind needle insertion may increase the incidence of complications.

[0009] In addition, the success of ultrasound-guided TAP block is highly dependent on the quality of ultrasound image acquisition and the level of the operator, especially in cases of anatomical variation or scanning difficulties (such as obesity). The entire process requires the doctor to have a strong ability to recognize images.

[0010] Based on this, in view of the complexity of the abdominal anatomical structure, the variability of ultrasound images and operator dependence, the present invention provides a transverse abdominal muscle plane tissue structure recognition method and application based on RMSD-Net to solve the problem of the inability to accurately and automatically segment and recognize the transverse abdominal muscle plane tissue structure information, which is a technical problem that needs to be solved urgently. Summary of the invention

[0011] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method and application for identifying the plane tissue structure of the transverse abdominal muscle based on RMSD-Net. The present invention realizes the automatic segmentation and recognition of the plane tissue structure information of the transverse abdominal muscle by constructing an efficient deep learning model, thereby helping clinicians to quickly and accurately interpret ultrasound images of the TAP area, and assist them in accurately locating the blocking area and injecting local anesthetics.

[0012] In order to solve the existing technical problems, the present invention provides the following technical solutions:

[0013] A method for identifying the plane tissue structure of the transverse abdominal muscle based on RMSD-Net, comprising the steps of:

[0014] Data preprocessing is performed on the acquired ultrasound images with the transverse abdominal muscle plane structure to establish an abdominal ultrasound image data set; wherein the ultrasound images with the transverse abdominal muscle plane structure include key abdominal tissue structure information that can be observed during TAP block; the key abdominal tissue structure information includes the external oblique muscle, the internal oblique muscle, the transverse abdominal muscle, the rectus abdominis muscle, and the anterior superior iliac spine;

[0015] Constructing an RMSD-Net model with a U-shaped network structure; the RMSD-Net model includes an encoder, a skip connection structure, and a decoder; the encoder uses the original double convolution structure in each layer, and accordingly adds a multi-scale residual feature extraction module RMDC module in each layer to extract features of the input ultrasound image with the transverse abdominal muscle plane structure layer by layer, thereby strengthening the important features of the aforementioned input ultrasound image with the transverse abdominal muscle plane structure layer by layer, and suppressing secondary information at the same time; the skip connection structure uses a feature fusion module SFF module layer by layer to enhance feature expression and promote the integration of contextual information; an attention mechanism GAM module is added to the last layer of the decoder to combine channel attention and spatial attention to integrate local information and global information in the feature map, and dynamically adjust the weight to provide feature representation for subsequent feature weighting;

[0016] Perform segmentation tasks for the aforementioned key abdominal tissue structure information, divide the aforementioned abdominal ultrasound image dataset into a training set and a test set, and use the aforementioned RMSD-Net model for training and testing respectively;

[0017] The tested RMSD-Net model is used to segment and identify key abdominal tissue structure information in newly acquired ultrasound images with transverse abdominal muscle plane structure.

[0018] Furthermore, the ultrasound image with the plane structure of the transverse abdominal muscle is a frame image in an ultrasound video capable of observing corresponding muscle tissues, which is recorded during the plane scanning of the transverse abdominal muscle;

[0019] The data preprocessing includes data annotation and data enhancement;

[0020] The data annotation can use Labelme annotation software to annotate the key abdominal tissue structure information in the aforementioned ultrasound image by drawing contour lines;

[0021] The data enhancement includes processing the ultrasound image having the transverse abdominal muscle plane structure by random rotation, random flipping and / or random mirroring;

[0022] 80% of the ultrasound images with the transverse abdominal muscle plane structure are extracted from the abdominal ultrasound image data to form a training set for training the aforementioned RMSD-Net model, and the remaining 20% ​​of the ultrasound images with the transverse abdominal muscle plane structure are composed of a test set for verifying the aforementioned RMSD-Net model.

[0023] Further, in the encoder, the execution of the RMDC module includes step S110:

[0024] S111, using three dilated convolutional layers with different dilation rates to capture spatial information at different scales, thereby effectively extracting global and local information of the image; when the dilation rates are 1, 2, and 4, the corresponding feature representations of different scales are {X1, X2, X3}, thereby obtaining X i =F i (X); where i = 1, 2 or 3, F i (X) represents the convolution operation with different expansion rates, and X represents the input feature map;

[0025] S112: After passing the output features of all the dilated convolutional layers through the BN layer, they are concatenated along the channel dimension, and then integrated through the voting convolutional layer voteConv layer, and the weight map W of the fusion feature is obtained through the Sigmoid function σ 11 , that is, W 11 =σ(F vote ([X1, X2, X3])); where F vote () indicates a voting convolution layer operation, and [X1,X2,X3] indicates a feature concatenation operation; the voteConv layer can strengthen key structural features and suppress secondary information by adjusting and filtering features extracted at different scales;

[0026] S113, using residual connection to combine the weighted feature map with the original input feature map X to obtain the final output feature map Y, that is, Y = X + X * W 11 .

[0027] Furthermore, the output feature map Y of each layer encoder corresponds to the kth layer and is marked as the output feature map E k ; In the skip connection structure, the SFF module converts the output feature map E of the encoder corresponding to each layer k After 3*3 convolution, the input feature map SFF corresponding to the SFF module k+1 After upsampling and element-by-element addition, features are extracted and outputted through a multi-scale deeply separable hybrid attention mechanism module MDSA module, where k is a positive integer; wherein the overall structure of the MDSA module includes channel attention and spatial attention; the spatial attention is composed of a multi-scale deeply separable convolution module to dynamically allocate attention weights in the channel dimension and the spatial dimension.

[0028] Further, the execution of the MDSA module includes step S120:

[0029] S121, obtaining an input feature map, and obtaining spatial information of the input feature map after channel attention aggregation, average pooling and maximum pooling operations;

[0030] S122, processing the aforementioned spatial information through a shared multi-layer perceptron MLP to generate a channel attention map; wherein the output feature map processed by the channel attention mechanism is obtained by multiplying the elements of the input feature;

[0031] S123, input the channel prior into the deep convolution module to generate a spatial attention map, receive the spatial attention feature map through a 1*1 convolution block and perform channel fusion;

[0032] S124, multiply the channel mixing result by the channel prior element by element to obtain the refined features as output.

[0033] Further, the execution of the GAM module includes step S130:

[0034] S131, the input is processed using a channel attention module; the channel attention module first adjusts the shape of the input tensor to H*W*C through a sizepermute operation, that is, the channel dimension is moved to the end, so as to perform global feature calculation in the spatial dimension; then the adjusted tensor is processed by a multi-layer perceptron MLP to generate global features; the MLP consists of two fully connected layers, and a reduction ratio r is used to reduce the computational complexity, so that the first fully connected layer reduces the number of channels from C to C / r, and the second fully connected layer restores the number of channels from C / r to C; the processed tensor dimension is restored to the original shape C*H*W through a size reverse operation for further operation, and finally a channel attention weight is generated through a Sigmoid activation function;

[0035] S132, the result of the processing of the aforementioned channel attention module is processed by the spatial attention module to obtain an output feature map; the execution of the spatial attention module includes: S1321, using two convolutional layers to fuse spatial information, S1322, using the same reduction ratio r as the channel attention module in the spatial attention module, S1323, using the Sigmoid activation function to weight the feature map of each position to improve the spatial expression ability of the feature map and reduce the influence of secondary information.

[0036] Furthermore, after the aforementioned tests, mIOU, mPA, Accuracy and Dice coefficient were used as evaluation indicators to measure the performance of the RMSD-Net model in the segmentation task to verify the final model effect; wherein, mIOU is an indicator for evaluating segmentation accuracy; mPA refers to the average value of pixel accuracy in each category; Accuracy refers to the proportion of correctly classified pixels in the segmentation result to the total number of pixels; and the Dice coefficient is an indicator for evaluating the degree of overlap between the segmentation result and the true annotation.

[0037] A transverse abdominal muscle plane tissue structure recognition device based on RMSD-Net, comprising:

[0038] A data construction unit, used for performing data preprocessing on the acquired ultrasound image with the plane structure of the transverse abdominal muscle to establish an abdominal ultrasound image data set; wherein the ultrasound image with the plane structure of the transverse abdominal muscle includes key abdominal tissue structure information that can be observed during the TAP block process; the key abdominal tissue structure information includes the external oblique muscle, the internal oblique muscle, the transverse abdominal muscle, the rectus abdominis muscle and the anterior superior iliac spine;

[0039] A model construction unit is used to construct an RMSD-Net model with a U-shaped network structure; the RMSD-Net model includes an encoder, a jump connection structure and a decoder; the encoder uses the original double convolution structure in each layer, and accordingly adds a multi-scale residual feature extraction module RMDC module in each layer to extract features of the input ultrasound image with the transverse abdominal muscle plane structure layer by layer, thereby strengthening the important features of the aforementioned input ultrasound image with the transverse abdominal muscle plane structure layer by layer, and suppressing secondary information at the same time; the jump connection structure uses a feature fusion module SFF module layer by layer to enhance feature expression and promote the integration of contextual information; an attention mechanism GAM module is added to the last layer of the decoder to combine channel attention and spatial attention to integrate local information and global information in the feature map, and dynamically adjust the weight to provide feature representation for subsequent feature weighting;

[0040] A model training and testing unit, used to perform segmentation tasks for the aforementioned key abdominal tissue structure information, divide the aforementioned abdominal ultrasound image dataset into a training set and a test set, and use the aforementioned RMSD-Net model for training and testing respectively;

[0041] The image recognition unit is used to segment and recognize key abdominal tissue structure information of a newly acquired ultrasound image with a transverse abdominal muscle plane structure using the tested RMSD-Net model.

[0042] A transverse abdominal muscle plane tissue structure recognition system based on RMSD-Net, comprising:

[0043] A network node, used for sending and receiving an ultrasound image with a plane structure of the transverse abdominal muscle acquired by an ultrasound device;

[0044] A model configuration module, for configuring an RMSD-Net model for segmenting and identifying key abdominal tissue structure information in an ultrasound image having a transverse abdominal muscle plane tissue structure; the key abdominal tissue structure information includes an external oblique muscle, an internal oblique muscle, a transverse abdominal muscle, a rectus abdominis muscle, and anterior superior iliac spine;

[0045] A system server, the system server connecting the network nodes and the model arrangement module;

[0046] The system server is configured to: perform data preprocessing on the acquired ultrasound images with the transverse abdominal muscle plane structure to establish an abdominal ultrasound image data set; wherein the ultrasound images with the transverse abdominal muscle plane structure include key abdominal tissue structure information that can be observed during TAP blocking; construct an RMSD-Net model with a U-shaped network structure; the RMSD-Net model includes an encoder, a jump connection structure and a decoder; the encoder uses the original double convolution structure in each layer, and adds a multi-scale residual feature extraction module RMDC module in each layer accordingly, so as to extract features of the input ultrasound image with the transverse abdominal muscle plane structure layer by layer, thereby strengthening the input ultrasound image with the transverse abdominal muscle plane structure layer by layer. important features and suppress secondary information at the same time; the skip connection structure uses the feature fusion module SFF module layer by layer to enhance feature expression and promote the integration of contextual information; an attention mechanism GAM module is added to the last layer of the decoder to combine channel attention and spatial attention to integrate local information and global information in the feature map, and dynamically adjust the weights to provide feature representation for subsequent feature weighting; a segmentation task is performed for the aforementioned key abdominal tissue structure information, the aforementioned abdominal ultrasound image dataset is divided into a training set and a test set, and the aforementioned RMSD-Net model is used for training and testing respectively; the tested RMSD-Net model is used to perform segmentation and recognition of key abdominal tissue structure information on the newly acquired ultrasound image with the transverse abdominal muscle plane structure.

[0047] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the implementation steps of any of the above methods.

[0048] Based on the above advantages and positive effects, the advantage of the present invention is that a data set of abdominal ultrasound images suitable for processing nerve block technology is constructed.

[0049] Furthermore, an RMSD-Net model suitable for segmentation and identification of key abdominal tissue structure information in nerve block technology was constructed, and the corresponding training and testing were completed.

[0050] Furthermore, an RMDC module is added to the encoder of the RMSD-Net model. The RMDC module can extract feature information of different scales by performing convolution operations on the input feature map under different receptive fields. This enables the model to capture image features of different levels and scales, which helps to improve the model's ability to understand complex scenes.

[0051] Furthermore, an SFF module is added to the skip connection structure of the RMSD-Net model. The SFF module extracts more fine-grained features through the multi-scale deeply separable hybrid attention mechanism module MDSA module and outputs them. The MDSA attention mechanism selects and weights the feature information of different channels by learning the importance weight of each channel in the feature map, which helps the model focus on the most relevant and discriminative features and improves the expressiveness of the features. At the same time, the attention mechanism can reduce the influence of irrelevant or unimportant feature channels by performing weighted summation on the feature map, thereby reducing the computational complexity of the model.

[0052] Furthermore, a GAM module is added to the decoder of the RMSD-Net model. The GAM module models the correlation between channels and spaces in the feature map globally, thereby better capturing the global dependencies of the feature map, improving the model's overall understanding and expression capabilities of the input features, and at the same time reducing feature loss during upsampling. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flow chart of a method provided by an embodiment of the present invention.

[0054] Figure 2 A schematic diagram of the structure of the RMSD-Net model provided in an embodiment of the present invention.

[0055] Figure 3 A schematic diagram of the structure of an RMDC module in an encoder provided in an embodiment of the present invention.

[0056] Figure 4 A schematic diagram of the structure of an SFF module in a jump connection structure provided in an embodiment of the present invention.

[0057] Figure 5 A schematic diagram of the structure of a GAM module in a decoder provided in an embodiment of the present invention.

[0058] Figure 6 This is a visualization result of an ablation experiment provided by an embodiment of the present invention.

[0059] Figure 7 The visualization result of the comparative experiment provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following is a further detailed description of a method and application of identifying the plane tissue structure of the transverse abdominal muscle based on RMSD-Net disclosed in the present invention in conjunction with the accompanying drawings and specific embodiments. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered isolated, and they can be combined with each other to achieve better technical effects. In the drawings of the following embodiments, the same reference numerals appearing in each drawing represent the same features or components, which can be applied to different embodiments. Therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0061] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the invention. Any modification of the structure, change of the proportion relationship or adjustment of the size should fall within the scope of the technical content disclosed by the invention without affecting the effects and purposes that can be achieved by the invention. The scope of the preferred embodiments of the present invention includes other implementations, in which the functions can be performed in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by those skilled in the art of the technical field to which the embodiments of the present invention belong.

[0062] Techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the techniques, methods, and devices should be considered part of the authorization specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0063] Example

[0064] See also Figure 1 FIG. 1 is a flow chart of the present invention. The implementation step S100 of the method is as follows:

[0065] S101, performing data preprocessing on the acquired ultrasound image with the transverse abdominal muscle plane structure to establish an abdominal ultrasound image data set.

[0066] The ultrasound image with the plane structure of the transverse abdominal muscle is a frame image in the ultrasound image video recorded during the plane scanning of the transverse abdominal muscle and capable of clearly observing the corresponding muscle tissue.

[0067] Considering the lack of publicly available data sets of relevant abdominal ultrasound images, this embodiment collects a large amount of clinical ultrasound data.

[0068] In actual operation, since the collected original data is in video format, in order to select frame images with clear image quality, clear structure and different content as data for subsequent training, technical personnel in this field (such as professional anesthesiologists) mark the areas to be segmented on the aforementioned frame images. The ultrasound image data set includes the corresponding key abdominal tissue structure information that can be clearly observed during the TAP block process.

[0069] It is also worth mentioning that the following targets to be segmented and labeled with clinical value are the transversus abdominis muscle (TA), the rectus abdominis muscle (RA), the external oblique muscle (EO), the internal oblique muscle (IO) and the anterior superior iliac spine (ASIS).

[0070] Corresponding to the aforementioned target to be segmented, it can be determined that the key abdominal tissue structure information includes the external oblique muscle, the internal oblique muscle, the transverse abdominal muscle, the rectus abdominis muscle and the anterior superior iliac spine.

[0071] The data preprocessing includes data labeling and data enhancement.

[0072] The data annotation can use Labelme annotation software to annotate the key abdominal tissue structure information in the aforementioned ultrasound image by drawing contour lines.

[0073] The data enhancement includes processing the ultrasound image with the transverse abdominal muscle plane structure by random rotation, random flipping and / or random mirroring.

[0074] Specifically, the ultrasound images used in this embodiment are from an authoritative comprehensive tertiary-level Class A hospital. Senior clinical physicians of the hospital collected and compiled about 300 videos of the plane tissue structure of the transverse abdominal muscles of about 100 patients.

[0075] These transverse abdominal muscle plane tissue structure videos were captured frame by frame to obtain about 8,000 original transverse abdominal muscle plane structure images.

[0076] In order to protect the privacy of patients, these original transverse abdominal muscle plane structure images were renumbered, and 1980 images with clear quality and identifiable were selected.

[0077] Finally, professionals confirmed that the image data marked in the above pictures were valid, generated in the json file format, and converted into the Pascal voc dataset format for input into the program.

[0078] In order to ensure that the data in the Pascal voc dataset is of high quality, data enhancement methods such as random rotation, random flipping, and random mirroring are used to ensure the quality of the input data, thereby improving the robustness of the trained RMSD-Net model.

[0079] S102, constructing a RMSD-Net model with a U-shaped network structure.

[0080] See also Figure 2 , which is a schematic diagram of the structure of the RMSD-Net model provided in this embodiment.

[0081] The RMSD-Net model includes an encoder, a skip connection structure and a decoder.

[0082] The encoder uses the U-Net model in each layer. On the basis of the original double convolution structure of the encoder part, a multi-scale residual feature extraction module RMDC module is added in each layer accordingly to extract features of the input ultrasound image with the transverse abdominal muscle plane structure layer by layer, thereby strengthening the important features of the input ultrasound image with the transverse abdominal muscle plane structure layer by layer and suppressing secondary information at the same time.

[0083] The skip connection structure uses a feature fusion module SFF module layer by layer to enhance feature expression and promote the integration of context information.

[0084] In addition, an attention mechanism GAM module is added to the last layer of the decoder to combine channel attention and spatial attention to integrate local and global information in the feature map, thereby dynamically adjusting the weights and providing feature representation for subsequent feature weighting.

[0085] It is worth noting that in the deep layer of the encoder, two 3×3 convolutional layers are preferably used to extract local detail features of the image. This is because in the deep layer of the encoder, the spatial size of the feature map has been reduced, which means that each pixel represents a larger area in the original image. At this scale, more context information can be captured. For this reason, the RMDC module is preferably used to further enhance the ability to capture this context by applying attention at multiple spatial scales.

[0086] For details, see Figure 3 FIG. 4 is a schematic diagram of the structure of the RMDC module in the encoder provided in this embodiment.

[0087] Correspondingly, the execution of the RMDC module includes step S110:

[0088] S111, uses three dilated convolutional layers with different dilation rates to capture spatial information at different scales, thereby effectively extracting the global and local information of the image.

[0089] It is worth emphasizing that in this embodiment, different dilation rates are used in the RMDC module mainly to capture information of different scales, so as to better identify and segment the target area, that is, to determine key abdominal tissue structure information, such as the external oblique muscle, internal oblique muscle, transverse abdominal muscle and rectus abdominis muscle. Specifically, using different dilation rates can help the RMSD-Net model extract features under different receptive fields, thereby improving the expression ability and segmentation accuracy of the RMSD-Net model.

[0090] Therefore, in actual operation, it is preferred to select the optimal configuration for the expansion rate parameter to ensure the expressiveness and segmentation accuracy of the RMSD-Net model.

[0091] This embodiment provides two combinations of different expansion rates for detailed description:

[0092] Combination 1: expansion rates are 1, 2, and 4.

[0093] When the expansion rate is 1, the receptive field is small, which can capture detailed information and local features, which helps to identify and segment small or subtle structures, such as muscle fibers or small muscle groups. In the segmentation process of EO, IO, TA, and RA, the identification of fascia is crucial. By setting the expansion rate to 1, the characteristics of fascia can be initially captured, laying the foundation for the subsequent use of attention mechanism to capture muscle characteristics.

[0094] When the expansion rate is 2, the medium receptive field balances the detail information and global features. It can capture slightly larger structures and medium-scale features, which helps to identify and segment medium-sized muscle groups. Since the sizes of EO, IO, and TA muscles are not much different in clinical practice, a expansion rate of 2 is more effective.

[0095] When the expansion rate is 4, the receptive field is larger, which can capture global information and large-scale features, which helps to identify and segment large structures and overall contours, such as the entire muscle group or the boundary between muscle and other tissues. Among them, the three muscles of EO, IO, and TA appear in the subcostal, anterior, posterior, and lateral approaches, but RA only appears in the subcostal approach, and ASIS only appears in the anterior approach. Therefore, after segmenting the three muscles of EO, IO, and TA, further identification of RA and ASIS can guide doctors to better determine which approach is used. The anesthesia method and anesthetic dosage of each approach will be different.

[0096] In addition, there is a situation where the back and the outside are assumed to be a complete scanning process, so these two sides can be processed together or separately.

[0097] Combination 2: Select experimental results with expansion rates of 2, 4, 8, or even larger expansion rates.

[0098] Among them, if the expansion rates of 2, 4, and 8 are used, the capture of detail information when the expansion rate is 1 will be missing, resulting in insufficient small-scale feature extraction.

[0099] Moreover, when the dilation rate is set to 2, 4, or 8, the setting has a large jump in scale change, which may lead to information loss or discontinuity during feature extraction, especially for complex structures that need to span multiple scales.

[0100] Furthermore, at a larger dilation rate (such as 8, 16), the area covered by the convolution kernel is larger, which can easily lead to a hole effect in the receptive field (i.e., the effective information area is reduced), resulting in some important features not being captured.

[0101] Finally, convolutions with larger dilation rates increase the computational complexity of the network because the convolution kernel covers a larger area and involves more computation.

[0102] To this end, this embodiment selects expansion rates of 1, 2, and 4, which have significant advantages in capturing information of different scales, smooth transition, and computational efficiency.

[0103] Therefore, when the expansion rates are 1, 2, and 4, the corresponding features of different scales are represented as {X1, X2, X3}, thus obtaining X i =F i (X); where i = 1, 2, 3, F i (X) represents the convolution operation with different dilation rates, and X represents the input feature map.

[0104] S112: After passing the output features of all the dilated convolutional layers through the BN layer, they are concatenated along the channel dimension, and then integrated through the voting convolutional layer voteConv layer, and the weight map W of the fusion feature is obtained through the Sigmoid function σ 11 , that is, W 11 =σ(F vote ([X1, X2, X3])); where F vote () indicates a voting convolution operation, and [X1,X2,X3] indicates a feature concatenation operation.

[0105] It is worth mentioning that the output features of all the dilated convolutional layers are concatenated along the channel dimension after passing through the BN layer, and combined through the voting convolutional layer, and then the Sigmoid activation function is used to play the role of spatial attention.

[0106] Moreover, the voteConv layer can strengthen key structural features and suppress secondary information by adjusting and filtering features extracted at different scales. The voting convolution operation is used as an effective feature selection and strengthening mechanism in actual operation, which helps improve the model's understanding of complex image content.

[0107] S113, using residual connection to combine the weighted feature map with the original input feature map X to obtain the final output feature map Y, that is, Y = X + X * W 11 .

[0108] Among them, when obtaining the final output feature map Y, it is preferably activated by a ReLU function to combine features extracted from different scales and contexts.

[0109] It is worth emphasizing that since the residual connection can connect the feature information of the low layer and the high layer through cross-layer connection to solve the problem of gradient disappearance and gradient explosion, the use of residual connection to combine the weighted feature map with the original input feature map X to obtain the final output feature map Y can make the features better transferred between layers, so as to improve the training efficiency, model performance and generalization ability.

[0110] Therefore, the above formulas can be combined to determine that the RMDC module can be expressed as: Y = X + X * σ (F vote ([F1(X), F2(X), F3(X)]))

[0111] Among them, F1(X), F2(X) and F3(X) represent the convolution operations of the dilated convolutional layer with dilation rates of 1, 2, and 4 respectively. vote () is the voting convolution layer operation, σ is the Sigmoid activation function, X is the input feature map, and Y is the final output feature map obtained after processing by the RMDC module. In this embodiment, the output feature map Y of each layer of encoder corresponds to the kth layer marked as the output feature map E k .

[0112] Based on this, the RMDC module can extract feature information of different scales by performing convolution operations on the input feature map under different receptive fields, which enables the model to capture image features of different levels and scales, which helps to improve the model's ability to understand complex scenes.

[0113] In order to better transfer features between layers, preferably, in the skip connection structure, considering that the role of the SFF module is to dynamically adjust the channel weight of the feature map according to the global information of the input feature map, thereby enhancing the network's attention to important features and suppressing the response to minor features. Therefore, the feature fusion module SFF module is used to replace the traditional U-Net skip connection structure, aiming to enhance feature expression and promote the integration of context information.

[0114] For details, see Figure 4 FIG. 1 is a schematic diagram of the structure of an SFF module in a jump connection structure provided in this embodiment.

[0115] In the skip connection structure, the SFF module converts the output feature map E of the encoder corresponding to each layer k After 3*3 convolution, the input feature map SFF corresponding to the SFF module k+1 After upsampling and element-by-element addition, features are extracted and outputted through the multi-scale deeply separable hybrid attention mechanism module MDSA module, where k is a positive integer.

[0116] Among them, the overall structure of the MDSA module includes channel attention and spatial attention; the spatial attention is composed of a multi-scale depthwise separable convolution module (Multi-Scale Depthwise Separable ConvolutionModule) to dynamically allocate attention weights in the channel dimension and the spatial dimension.

[0117] Therefore, it can be determined that the multi-scale deeply separable hybrid attention mechanism module MDSA module can extract more fine-grained features and output them.

[0118] As one of the preferred implementations of this embodiment, the execution of the MDSA module includes step S120:

[0119] S121, obtain the input feature map, and obtain the spatial information of the input feature map after channel attention aggregation average pooling and maximum pooling operations.

[0120] S122, the aforementioned spatial information is processed through a shared multi-layer perceptron MLP to generate a channel attention map; wherein, the output feature map processed by the channel attention mechanism is obtained by multiplying the elements of the input feature.

[0121] S123, input the channel prior into the deep convolution module to generate a spatial attention map, receive the spatial attention feature map through a 1*1 convolution block and perform channel fusion.

[0122] S124, multiply the channel mixing result by the channel prior element by element to obtain the refined features as output.

[0123] As another preferred implementation of this embodiment, specifically combined with Figure 4 , the attention mechanism of the MDSA module is described in detail. In this embodiment, the input feature map is F∈R C*H*W , where C is the number of channels, H and W are the height and width of the feature map respectively, and R represents a set of real numbers to indicate that all elements of the input feature map F are real numbers.

[0124] First, use the Channel Attention Module (CAM) to calculate a 1D channel attention map M c ∈R C*1*1 , then M c Multiply the input feature map F element by element to obtain the refined feature map F with channel attention c ∈R C*H*W . F c Represents the output feature map after the channel attention mechanism CA.

[0125] Secondly, the spatial attention module (SAM) is used to process F c Generate 3D spatial attention map M s ∈R C*H*W .

[0126] Finally, by element-by-element s With F c Multiply to get the final output feature map

[0127] Based on this, the entire attention process can be summarized as: and

[0128] It is worth noting that the generation of the channel attention map is the responsibility of the channel attention module, which generates the channel attention map by exploring the inter-channel relationship in the features. Specifically, average pooling and maximum pooling operations are used to aggregate spatial information from the feature map, and two independent spatial context descriptors are generated during the aggregation process.

[0129] The spatial context descriptors are then fed into a shared multi-layer perceptron (MLP), and the outputs of the shared MLP are combined via element-wise summation to obtain a channel attention map.

[0130] In order to reduce parameter overhead, it is preferred that in this embodiment, the shared MLP contains only one hidden layer, and the size of the hidden layer activation is set to R C / r*1*1 , where r represents the dimensionality reduction ratio.

[0131] To this end, the calculation of channel attention is expressed as:

[0132] CA(F)=σ(MLP(F avg )+MLP(F max )).

[0133] The MLP() represents processing the input vector through one or more fully connected layers (which contain nonlinear activation functions) and outputting a vector of the same size; F avg represents the feature map after average pooling, F max Represents the feature map after maximum pooling.

[0134] The spatial attention map is generated by extracting the spatial mapping relationship. Based on practical considerations, in this embodiment, it is considered that the spatial attention map of each channel should not be forced to remain consistent. In contrast, it is considered more realistic to dynamically allocate attention weights in the channel and spatial dimensions.

[0135] Correspondingly, Figure 4 We show how to use deep convolution to capture spatial relationships between features, thereby reducing computational complexity while preserving inter-channel relationships.

[0136] In order to enhance the ability of convolution operation to capture spatial relationships, this embodiment preferably adopts a multi-scale structure. That is, at the end of the spatial attention module, a 1×1 convolution is used to perform channel mixing to generate a more refined attention map; where DwConv represents a depth-separable convolution, Branch i represents the i-th branch, i∈{1,2,3};

[0137] The computation of the spatial attention is expressed as:

[0138]

[0139] Therefore, it is worth emphasizing that the MDSA attention mechanism in the SFF module selects and weights the feature information of different channels by learning the importance weight of each channel in the feature map, which helps the model focus on the most relevant and discriminative features and improves the expressiveness of the features. At the same time, the attention mechanism can reduce the influence of irrelevant or unimportant feature channels by performing weighted summation on the feature map, thereby reducing the computational complexity of the model.

[0140] In the decoder structure, combined with Figure 2As shown, the output features obtained by downsampling the last layer of the encoder and the output features obtained by the SFF module in the corresponding layer are used as the input features of the first upsampling operation in the decoder structure. The feature maps provided by the input are concatenated along the specified dimension through element-wise concatenation, and the corresponding feature maps are obtained after upsampling. Each subsequent upsampling operation uses the output features obtained by the SFF module in the corresponding layer and the output features of the upsampling operation of the previous layer as input information. The feature maps provided by the input are concatenated along the specified dimension through element-wise concatenation, and the corresponding feature maps are obtained through upsampling operations. After the last upsampling operation, the feature maps obtained by upsampling in all layers of the decoder structure are combined, and the local information and global information in the feature maps are integrated through the attention mechanism GAM module, so as to dynamically adjust the weights and provide a richer feature representation for subsequent feature weighting, and then the final output feature map is obtained through 1*1 convolution.

[0141] Specifically, a GAM module is added to the decoder. The GAM module performs channel attention processing on the input based on the combination of channel attention and spatial attention, and then performs spatial attention processing on the result of channel attention.

[0142] Preferably, the execution of the GAM module includes step S130:

[0143] S131, process the input by the channel attention module.

[0144] In the GAM module, the channel attention module first adjusts the shape of the input tensor to H*W*C through the size permute operation, that is, the channel dimension is moved to the end, so as to perform global feature calculations in the spatial dimension. The adjusted tensor is then processed by a multilayer perceptron (MLP) to generate global features. The MLP consists of two fully connected layers (FC layers) and uses a reduction ratio r to reduce computational complexity. The first fully connected layer reduces the number of channels from C to C / r, and the second fully connected layer restores the number of channels from C / r to C. The size reverse operation is used to restore the processed tensor dimension to the original shape C*H*W for further operations, and finally the channel attention weight is generated by the Sigmoid activation function.

[0145] S132, the result of processing by the aforementioned channel attention module is processed by the spatial attention module to obtain an output feature map.

[0146] The execution of the spatial attention module includes:

[0147] S1321, uses two convolutional layers for spatial information fusion.

[0148] S1322, use the same reduction ratio r in the spatial attention module as in the channel attention module.

[0149] S1323, use the Sigmoid activation function to weight the feature map at each position to improve the spatial expression ability of the feature map and reduce the influence of secondary information.

[0150] As another preferred implementation of this embodiment, specifically combined with Figure 2 and Figure 5 As shown, for the input feature map X∈R C*H*W , where C is the number of channels, H and W are the height and width of the feature map respectively.

[0151] First, the four upsampled features in the decoder are concatenated and output to the GAM module, outputting F cat ∈R 5C*H*W , that is, F cat =[X1,X2,X3,X4,X5],F cat Indicates that all output feature maps are concatenated.

[0152] For the channel attention module, assuming that the input feature map F1∈R C*H*W , where C is the number of channels, H and W are the height and width of the feature map respectively.

[0153] First, perform global average pooling and global maximum pooling operations to generate two feature vectors: F avg ∈R C*1*1 , F max ∈R C*1*1 .Right now:

[0154] and

[0155] Among them, F avg represents the feature map after global average pooling; F1c,i,j represents the feature value of position (i,j) on channel c; F max Represents the feature map after global maximum pooling.

[0156] The two pooled feature vectors are input into a shared multi-layer perceptron (MLP) to generate two new feature vectors F′ avg ,F′ max , to represent the feature map after global average pooling and the feature map after global maximum pooling in the multi-layer perceptron, that is:

[0157] F′ avg =σ(W1(W0(F avg ))) and F′ max =σ(W1(W0(F max ))).

[0158] Among them, W1 and W0 are shared weight matrices, and σ is the activation function.

[0159] The two feature vectors are summed element-wise to obtain the final channel attention map M c , that is, M c =σ(F′ avg +F′ max ).

[0160] Finally, the channel attention map is multiplied element-wise with the original feature map, and F2 is the output, that is,

[0161] In order to focus on spatial information, the spatial attention module first uses two convolutional layers to fuse spatial information, and then uses the same reduction ratio r as the proposed spatial attention module from the channel attention module. At the same time, since the maximum pooling operation reduces the use of information and has a negative impact, the pooling operation is deleted to further retain the feature map. Finally, the Sigmoid activation function is used to weight the feature map at each position, thereby improving the spatial expression ability of the feature map and reducing the influence of irrelevant information.

[0162] Therefore, for the spatial attention module, assuming that the input feature map F2∈R C*H*W , where C is the number of channels, H and W are the height and width of the feature map, respectively. First, global average pooling and global maximum pooling operations are performed to generate two feature vectors: F avg ∈R 1*H*W , F max ∈R 1*H*W .

[0163] in, and

[0164] The above two feature maps are concatenated in the channel dimension to generate a new feature map, F cat ∈R 2*H*W+ , that is, F cat =[F avg ; F max ].

[0165] The concatenated feature map undergoes two 7×7 convolution operations to generate the final spatial attention map M. s , that is: M s =σ(Conv(F cat)). Where Conv is a convolution operation, preferably using a 1×1 convolution kernel, and σ is an activation function.

[0166] Finally, the spatial attention map is element-wise multiplied with the input feature map. F3 is taken as the output, expressed as:

[0167] It can be seen that through the processing sequence of the above step S130, the GAM module can model the correlation between channels and spaces in the feature map on a global scale, so as to better capture the global dependencies of the feature map, and improve the model's overall understanding and expression capabilities of the input features. At the same time, it also reduces the loss of features during the upsampling process.

[0168] S103, performing a segmentation task for the aforementioned key abdominal tissue structure information, dividing the aforementioned abdominal ultrasound image dataset into a training set and a test set, and using the aforementioned RMSD-Net model for training and testing respectively.

[0169] In this embodiment, 80% of the ultrasound images with the transverse abdominal muscle plane structure are extracted from the abdominal ultrasound image dataset to form a training set for training the aforementioned RMSD-Net model, and the remaining 20% ​​of the ultrasound images with the transverse abdominal muscle plane structure are composed of a test set for verifying the aforementioned RMSD-Net model.

[0170] After testing, the RMSD-Net model can use mIOU, mPA, Accuracy and Dice coefficient as evaluation indicators to measure model performance in segmentation tasks to verify the final model effect.

[0171] Among them, the mIOU is an indicator for evaluating segmentation accuracy; the mPA refers to the average value of pixel accuracy in each category; the Accuracy refers to the proportion of correctly classified pixels in the segmentation result to the total number of pixels; and the Dice coefficient is an indicator for evaluating the degree of overlap between the segmentation result and the true annotation.

[0172] Specifically, in the process of segmenting the abdominal tissue structure in the aforementioned ultrasound image, the evaluation indicators mIOU, mPA, Accuracy and Dice coefficient can be used to measure the performance of the segmentation algorithm.

[0173] The mIOU (Mean Intersection over Union) is an indicator for evaluating segmentation accuracy and is defined as the ratio of the intersection over the union of the predicted area and the true area. The IOU is calculated for each category and then averaged over all categories.

[0174] The mIOU measures the segmentation accuracy of the model on different abdominal tissue structures. A high mIOU indicates that the model performs well in identifying and segmenting these structures.

[0175] The mPA (Mean Pixel Accuracy) refers to the average pixel accuracy of each category. It calculates the ratio of the number of correctly classified pixels in each category to the total number of pixels in that category, and then takes the average for all categories.

[0176] The mPA can reflect the classification accuracy of each tissue structure (such as liver, kidney, etc.) and help evaluate the performance balance of the model on different tissues.

[0177] The Accuracy (pixel accuracy) refers to the ratio of the number of correctly classified pixels to the total number of pixels in the segmentation result.

[0178] The Accuracy measures the segmentation accuracy of all pixels in the entire image, but because it is insensitive to category imbalance (ie, large-area background pixels may result in a high Accuracy value), it is usually used together with other indicators in ultrasound image segmentation.

[0179] The Dice coefficient is a commonly used indicator for evaluating the overlap between the segmentation result and the true annotation. It is defined as twice the intersection of the predicted area and the true area divided by their sum.

[0180] Application in abdominal tissue structure: The Dice coefficient is particularly suitable for evaluating ultrasound image segmentation because it is more sensitive to overlapping areas and can better reflect the segmentation performance of the model on small-scale tissues (such as small tumors or organ boundaries).

[0181] The above indicators are of great significance in the image segmentation of abdominal tissue structures. They can comprehensively evaluate the performance of the segmentation model from different angles and help optimize and improve the model.

[0182] S104, using the tested RMSD-Net model to segment and identify key abdominal tissue structure information of the newly acquired ultrasound image with the transverse abdominal muscle plane structure.

[0183] It is worth emphasizing here that the combination of TAP nerve block can realize the selection of four-side approach plans from the subcostal margin, anterior side, posterior side, and lateral side. Therefore, after the RMSD-Net model performs segmentation and recognition of key abdominal tissue structure information on the transverse abdominal muscle plane tissue structure image, the implementation plan of TAP nerve block can be determined at the same time.

[0184] By way of example and not limitation, the EO, IO, and TA muscles will appear during the subcostal, anterior, posterior, and lateral approaches, but the RA will only appear during the subcostal approach, and the ASIS will only appear during the anterior approach.

[0185] This means that the RMSD-Net model segments and identifies key abdominal tissue structure information in the transverse abdominal muscle plane tissue structure image. If RA is identified, the muscle group belongs to the subcostal approach; if ASIS is identified, the muscle group belongs to the anterior approach.

[0186] Therefore, after segmenting and identifying the three muscles EO, IO, and TA, RA and ASIS can be further identified, which helps guide doctors to determine which muscle to approach. At the same time, combined with the anesthesia method and dosage of each approach, it lays the foundation for subsequent ultrasound guidance of each approach.

[0187] See also Figure 6 The above is the visualization result of the ablation experiment provided by this embodiment. At the same time, Table 1 provides the numerical results of the evaluation indicators under different models in the ablation experiment.

[0188] Table 1 Numerical results of evaluation indicators under different models in ablation experiments

[0189]

[0190] Combination Figure 6 As shown in the figure, in the ablation experiment, the M, C, and G modules represent the RMDC module, the SFF module, and the GAM module, respectively. The baseline is the benchmark model U-Net, and the RMSD-Net is Baseline+RMDC+SFF+GAM.

[0191] Combination Figure 6 As shown in Table 1, the RMDC module, SFF module, and GAM module have a positive effect on the feature extraction and prediction capabilities of the model, and the numerical results of each evaluation index have been improved. Among them, the Dice coefficients reached 93.55%, 94.98%, and 95.45% respectively, and the IOU coefficients reached 88.12%, 90.57%, and 91.38% respectively. Although there is a great improvement in the numerical results, combined with Figure 6 It can be determined that the visualization effect of the test set is very poor, and the slightly blurred muscle images are not fully recognized or cannot be recognized at all (for example Figure 6 The visualization effect corresponding to the 2nd, 3rd, and 4th columns of image d).

[0192] Depend on Figure 6As shown in Table 1, the Dice score of Baseline+RMDC+GAM reached 94.51%, and the IOU reached 89.74%. The Dice score of Baseline+RMDC+SFF reached 95.01%, and the IOU reached 90.77%. The Dice score of Baseline+SFF+GAM reached 95.40%, and the IOU reached 91.29%. Therefore, it can be determined that the combination of RMDC module, SFF module, and GAM module added to the U-Net model has an improvement compared with the U-Net model adding a single module. Among them, the combination of Baseline+SFF+GAM reached the highest. However, the visualization results show that although all muscles can be recognized, some noises will be predicted incorrectly (for example, Figure 6 The corresponding visualization effect of the second-to-last column of image d in ), and the edge prediction of the fascia is also incomplete (e.g. Figure 6 The visualization effect corresponding to columns 5, 6, and 7 of image d).

[0193] Depend on Figure 6 As can be seen from Table 1, the recognition results of Baseline+RMDC+SFF+GAM, namely RMSD-Net, have reached the highest level, with a Dice score of 95.70% and an IOU of 91.83%. At the same time, the problems of false recognition, missed recognition, and noise recognition have been basically solved in the predicted results. This is because the RMSD-Net model integrates the three improvements, enabling it to extract and fuse features at different scales and in global contexts, and improves the segmentation accuracy and robustness of the network for complex backgrounds and low-contrast areas.

[0194] See also Figure 7 As shown, a comparative experiment is provided for this embodiment to demonstrate the visual comparison effect of the RMSD-Net model and different segmentation models.

[0195] The comparative experiment is a key step in evaluating the segmentation ability between the RMSD-Net network model and the current mainstream models, and can also verify the generalization performance of the model.

[0196] Tables 2 and 3 respectively show the evaluation values ​​given by the comparative experiment for the evaluation coefficient, and the specific numerical results of each part in the comparative experiment.

[0197] Table 2 shows the numerical results of the comparative experiment.

[0198]

[0199] Table 3 Specific numerical results of each part in the comparative experiment

[0200]

[0201] Depend on Figure 7 It can be seen that the segmentation models U-Net, U-Net++, Attention U-Net, etc. are not very effective in segmenting the transverse abdominal muscle tissue, mainly because they miss the recognition of RA or ASIS. Even if these two parts are excluded and only EO, IO, and TA are focused on, combined with Table 3, it can be seen that the recognition results of these models are relatively low compared to the RMSD-Net model, and the prediction results are poor. Although the recognition accuracy of models such as CMU-Net and MACU-Net is similar to that of RMSD-Net, they have greatly increased the computational complexity of the model.

[0202] Based on this, it is emphasized again that the advantages of the RMSD-Net model provided in this embodiment are:

[0203] First, a multi-scale residual feature extraction module (RMDC) is added to each layer of the encoder, so that convolution kernels with different dilation rates can capture information of different scales and suppress secondary information at the same time. Among them, the external oblique muscle, internal oblique muscle, transverse abdominal muscle and rectus abdominis muscle are different in size and shape, and the multi-scale dilated convolution can adaptively process these targets of different scales.

[0204] Secondly, the skip connection structure uses the feature fusion module SFF module layer by layer to enhance feature expression and promote the integration of contextual information. The MDSA attention mechanism in the module combines channel attention and spatial attention, and dynamically allocates attention weights through a multi-scale deeply separable convolution module and retains channel priors. This combination can help the network better capture important features and improve the feature representation capability.

[0205] Third, an attention mechanism GAM module is added to the last layer of the decoder to provide feature representation for subsequent feature weighting. In ultrasound images, the background is complex and noisy. The GAM attention mechanism can help the model focus on the target area and effectively filter out background noise and interference, so that the model can capture the details of different muscle boundaries more finely, which is very important for the identification of complex structures and interrelated muscle parts.

[0206] In addition, the RMSD-Net model preserves the original 3*3 convolution, which helps to extract local features in the image, which is crucial for distinguishing different muscle tissues (with large differences).

[0207] For other technical features, please refer to the previous embodiments and will not be described in detail here.

[0208] In addition, the present invention also provides an embodiment, which provides a transverse abdominal muscle plane tissue structure recognition device 200 based on RMSD-Net, comprising:

[0209] The data construction unit 201 is used to perform data preprocessing on the acquired ultrasound image with the transverse abdominal muscle plane structure to establish an abdominal ultrasound image data set; wherein the ultrasound image with the transverse abdominal muscle plane structure includes key abdominal tissue structure information that can be observed during the TAP block process; the key abdominal tissue structure information includes the external oblique muscle, the internal oblique muscle, the transverse abdominal muscle, the rectus abdominis muscle and the anterior superior iliac spine.

[0210] The model construction unit 202 is used to construct an RMSD-Net model with a U-shaped network structure; the RMSD-Net model includes an encoder, a skip connection structure and a decoder; the encoder uses the original double convolution structure in each layer, and adds a multi-scale residual feature extraction module RMDC module in each layer accordingly, so as to extract features of the input ultrasound image with the transverse abdominal muscle plane structure layer by layer, thereby strengthening the important features of the aforementioned input ultrasound image with the transverse abdominal muscle plane structure layer by layer, and suppressing secondary information at the same time; the skip connection structure uses the feature fusion module SFF module layer by layer to enhance feature expression and promote the integration of contextual information; the attention mechanism GAM module is added to the last layer of the decoder to combine channel attention and spatial attention to integrate local information and global information in the feature map, and dynamically adjust the weight to provide feature representation for subsequent feature weighting.

[0211] The model training and testing unit 203 is used to perform segmentation tasks for the aforementioned key abdominal tissue structure information, divide the aforementioned abdominal ultrasound image data set into a training set and a test set, and use the aforementioned RMSD-Net model for training and testing respectively.

[0212] The image recognition unit 204 is used to segment and recognize key abdominal tissue structure information of a newly acquired ultrasound image having a transverse abdominal muscle plane structure using the tested RMSD-Net model.

[0213] For other technical features, please refer to the previous embodiments and will not be repeated here.

[0214] In addition, the present invention also provides an embodiment, which provides a transverse abdominal muscle plane tissue structure recognition system 200 based on RMSD-Net, including:

[0215] The network node 301 is used to send and receive ultrasound images with the plane structure of the transverse abdominal muscle collected by ultrasound equipment.

[0216] The model arrangement module 302 is used to configure an RMSD-Net model for segmenting and identifying key abdominal tissue structure information in an ultrasound image having a transverse abdominal muscle plane tissue structure; the key abdominal tissue structure information includes the external oblique muscle, the internal oblique muscle, the transverse abdominal muscle, the rectus abdominis muscle and the anterior superior iliac spine.

[0217] The system server 303 is used to connect the network node 301 and the model arrangement module 302 .

[0218] The system server 303 is configured to: perform data preprocessing on the acquired ultrasound image with the transverse abdominal muscle plane structure to establish an abdominal ultrasound image data set; wherein the ultrasound image with the transverse abdominal muscle plane structure includes key abdominal tissue structure information that can be observed during TAP blocking; construct an RMSD-Net model with a U-shaped network structure; the RMSD-Net model includes an encoder, a jump connection structure and a decoder; the encoder uses the original double convolution structure in each layer, and adds a multi-scale residual feature extraction module RMDC module in each layer accordingly, so as to extract features of the input ultrasound image with the transverse abdominal muscle plane structure layer by layer, thereby strengthening the aforementioned input ultrasound image with the transverse abdominal muscle plane structure layer by layer. The invention discloses a novel method for segmenting the key abdominal tissue structure information of the present invention, which can identify the important features in the image and suppress the secondary information at the same time; the skip connection structure uses the feature fusion module SFF module layer by layer to enhance the feature expression and promote the integration of context information; the attention mechanism GAM module is added to the last layer of the decoder to combine the channel attention and the spatial attention to integrate the local information and the global information in the feature map, and dynamically adjust the weights to provide feature representation for the subsequent feature weighting; the segmentation task is performed for the aforementioned key abdominal tissue structure information, the aforementioned abdominal ultrasound image dataset is divided into a training set and a test set, and the aforementioned RMSD-Net model is used for training and testing respectively; the tested RMSD-Net model is used to perform segmentation and recognition of the key abdominal tissue structure information on the newly acquired ultrasound image with the transverse abdominal muscle plane structure.

[0219] For other technical features, please refer to the previous embodiments and will not be repeated here.

[0220] In addition, an embodiment of the present invention also provides a computer-readable storage medium on which a program is stored for use in the aforementioned transverse abdominal muscle plane tissue structure identification system based on RMSD-Net. When the program is executed by a processor, it can implement the steps of any of the above-mentioned methods for identifying the transverse abdominal muscle plane tissue structure based on RMSD-Net.

[0221] For other technical features, please refer to the previous embodiments and will not be repeated here.

[0222] In the above description, within the scope of the target protection of the present disclosure, the components can be selectively and operatively combined in any number. In addition, terms such as "including", "comprising" and "having" should be interpreted as inclusive or open by default, rather than exclusive or closed, unless they are explicitly defined to the contrary. All technical, scientific or other terms have the meaning understood by those skilled in the art unless they are defined to the contrary. Common terms found in dictionaries should not be interpreted too idealistically or too impractically in the context of relevant technical documents, unless the present disclosure explicitly defines them as such.

[0223] Although the exemplary aspects of the present disclosure have been described for illustrative purposes, those skilled in the art should appreciate that the above description is only a description of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. The scope of the preferred embodiments of the present invention includes other implementations in which functions may not be performed in the order in which they appear or are discussed. Any changes or modifications made by those skilled in the art based on the above disclosure are within the scope of the claims.

Claims

1. A method for identifying the plane tissue structure of the transverse abdominal muscle based on RMSD-Net, characterized in that: Specifically include: Data preprocessing is performed on the acquired ultrasound images with the transverse abdominal muscle plane structure to establish an abdominal ultrasound image data set; wherein the ultrasound images with the transverse abdominal muscle plane structure include key abdominal tissue structure information that can be observed during TAP block; the key abdominal tissue structure information includes the external oblique muscle, the internal oblique muscle, the transverse abdominal muscle, the rectus abdominis muscle, and the anterior superior iliac spine; Constructing an RMSD-Net model with a U-shaped network structure; the RMSD-Net model includes an encoder, a skip connection structure, and a decoder; the encoder uses the original double convolution structure in each layer, and accordingly adds a multi-scale residual feature extraction module RMDC module in each layer to extract features of the input ultrasound image with the transverse abdominal muscle plane structure layer by layer, thereby strengthening the important features of the aforementioned input ultrasound image with the transverse abdominal muscle plane structure layer by layer, and suppressing secondary information at the same time; the skip connection structure uses a feature fusion module SFF module layer by layer to enhance feature expression and promote the integration of contextual information; an attention mechanism GAM module is added to the last layer of the decoder to combine channel attention and spatial attention to integrate local information and global information in the feature map, and dynamically adjust the weight to provide feature representation for subsequent feature weighting; Perform segmentation tasks for the aforementioned key abdominal tissue structure information, divide the aforementioned abdominal ultrasound image dataset into a training set and a test set, and use the aforementioned RMSD-Net model for training and testing respectively; The tested RMSD-Net model is used to segment and identify key abdominal tissue structure information in newly acquired ultrasound images with transverse abdominal muscle plane structure.

2. The method according to claim 1, characterized in that: The ultrasound image with the plane structure of the transverse abdominal muscle is a frame image in the ultrasound image video that can observe the corresponding muscle tissue and is recorded during the transverse abdominal muscle plane scanning process; The data preprocessing includes data annotation and data enhancement; The data annotation can use Labelme annotation software to annotate the key abdominal tissue structure information in the aforementioned ultrasound image by drawing contour lines; The data enhancement includes processing the ultrasound image having the transverse abdominal muscle plane structure by random rotation, random flipping and / or random mirroring; 80% of the ultrasound images with the transverse abdominal muscle plane structure are extracted from the abdominal ultrasound image data to form a training set for training the aforementioned RMSD-Net model, and the remaining 20% ​​of the ultrasound images with the transverse abdominal muscle plane structure are composed of a test set for verifying the aforementioned RMSD-Net model.

3. The method according to claim 1, characterized in that In the encoder, the execution of the RMDC module includes step S110: S111, using three dilated convolutional layers with different dilation rates to capture spatial information at different scales, thereby effectively extracting global and local information of the image; when the dilation rates are 1, 2, and 4, the corresponding feature representations of different scales are {X1, X2, X3}, thereby obtaining X i =F i (X); where i = 1, 2 or 3, F i (X) represents the convolution operation with different expansion rates, and X represents the input feature map; S112: After passing the output features of all the dilated convolutional layers through the BN layer, they are concatenated along the channel dimension, and then integrated through the voting convolutional layer voteConv layer, and the weight map W of the fusion feature is obtained through the Sigmoid function σ 11 , that is, W 11 =σ(F vote ([X1, X2, X3])); where F vote () indicates a voting convolution layer operation, and [X1, X2, X3] indicates a feature concatenation operation; the voteConv layer can strengthen key structural features and suppress secondary information by adjusting and filtering features extracted at different scales; S113, using residual connection to combine the weighted feature map with the original input feature map X to obtain the final output feature map Y, that is, Y = X + X * W 11 .

4. The method according to claim 3, characterized in that: The output feature map Y of each layer encoder corresponds to the kth layer and is marked as the output feature map E k ; In the skip connection structure, the SFF module converts the output feature map E of the encoder corresponding to each layer k After 3*3 convolution, the input feature map SFF corresponding to the SFF module k+1 After upsampling and element-by-element addition, features are extracted and outputted through a multi-scale deeply separable hybrid attention mechanism module MDSA module, where k is a positive integer; wherein the overall structure of the MDSA module includes channel attention and spatial attention; the spatial attention is composed of a multi-scale deeply separable convolution module to dynamically allocate attention weights in the channel dimension and the spatial dimension.

5. The method according to claim 4, characterized in that The execution of the MDSA module includes step S120: S121, obtaining an input feature map, and obtaining spatial information of the input feature map after channel attention aggregation, average pooling and maximum pooling operations; S122, processing the aforementioned spatial information through a shared multi-layer perceptron MLP to generate a channel attention map; wherein the output feature map processed by the channel attention mechanism is obtained by multiplying the elements of the input feature; S123, input the channel prior into the deep convolution module to generate a spatial attention map, receive the spatial attention feature map through a 1*1 convolution block and perform channel fusion; S124, multiply the channel mixing result by the channel prior element by element to obtain the refined features as output.

6. The method according to claim 1, characterized in that The execution of the GAM module includes step S130: S131, the input is processed using a channel attention module; the channel attention module first adjusts the shape of the input tensor to H*W*C through a sizepermute operation, that is, the channel dimension is moved to the end, so as to perform global feature calculation in the spatial dimension; then the adjusted tensor is processed by a multi-layer perceptron MLP to generate global features; the MLP consists of two fully connected layers, and a reduction ratio r is used to reduce the computational complexity, so that the first fully connected layer reduces the number of channels from C to C / r, and the second fully connected layer restores the number of channels from C / r to C; the processed tensor dimension is restored to the original shape C*H*W through a size reverse operation for further operation, and finally a channel attention weight is generated through a Sigmoid activation function; S132, processing the result processed by the channel attention module through the spatial attention module to obtain an output feature map; The execution of the spatial attention module includes: S1321, using two convolutional layers to fuse spatial information, S1322, using the same reduction ratio r as the channel attention module in the spatial attention module, S1323, using the Sigmoid activation function to weight the feature map at each position to improve the spatial expression ability of the feature map and reduce the influence of secondary information.

7. The method according to claim 1, characterized in that After the above tests, mIOU, mPA, Accuracy and Dice coefficients were used as evaluation indicators to measure the performance of the RMSD-Net model in the segmentation task to verify the final model effect; Among them, the mIOU is an indicator for evaluating segmentation accuracy; The mPA refers to the average pixel accuracy of each category; Accuracy refers to the ratio of correctly classified pixels to the total number of pixels in the segmentation result; The Dice coefficient is an indicator for evaluating the degree of overlap between the segmentation result and the true annotation.

8. A device for identifying the plane tissue structure of the transverse abdominal muscle based on RMSD-Net according to any one of the methods described in claims 1 to 7, characterized in that include: A data construction unit, used for performing data preprocessing on the acquired ultrasound image with the plane structure of the transverse abdominal muscle to establish an abdominal ultrasound image data set; wherein the ultrasound image with the plane structure of the transverse abdominal muscle includes key abdominal tissue structure information that can be observed during the TAP block process; the key abdominal tissue structure information includes the external oblique muscle, the internal oblique muscle, the transverse abdominal muscle, the rectus abdominis muscle and the anterior superior iliac spine; A model construction unit is used to construct an RMSD-Net model with a U-shaped network structure; the RMSD-Net model includes an encoder, a jump connection structure and a decoder; the encoder uses the original double convolution structure in each layer, and accordingly adds a multi-scale residual feature extraction module RMDC module in each layer to extract features of the input ultrasound image with the transverse abdominal muscle plane structure layer by layer, thereby strengthening the important features of the aforementioned input ultrasound image with the transverse abdominal muscle plane structure layer by layer, and suppressing secondary information at the same time; the jump connection structure uses a feature fusion module SFF module layer by layer to enhance feature expression and promote the integration of contextual information; an attention mechanism GAM module is added to the last layer of the decoder to combine channel attention and spatial attention to integrate local information and global information in the feature map, and dynamically adjust the weight to provide feature representation for subsequent feature weighting; A model training and testing unit, used to perform segmentation tasks for the aforementioned key abdominal tissue structure information, divide the aforementioned abdominal ultrasound image dataset into a training set and a test set, and use the aforementioned RMSD-Net model for training and testing respectively; The image recognition unit is used to segment and recognize key abdominal tissue structure information of a newly acquired ultrasound image with a transverse abdominal muscle plane structure using the tested RMSD-Net model.

9. A transverse abdominal muscle plane tissue structure recognition system based on RMSD-Net according to any one of claims 1 to 7, characterized in that include: A network node, used for sending and receiving an ultrasound image with a plane structure of the transverse abdominal muscle acquired by an ultrasound device; A model configuration module, for configuring an RMSD-Net model for segmenting and identifying key abdominal tissue structure information in an ultrasound image having a transverse abdominal muscle plane tissue structure; the key abdominal tissue structure information includes an external oblique muscle, an internal oblique muscle, a transverse abdominal muscle, a rectus abdominis muscle, and anterior superior iliac spine; A system server, the system server connecting the network nodes and the model arrangement module; The system server is configured to: perform data preprocessing on the acquired ultrasound image with the transverse abdominal muscle plane structure to establish an abdominal ultrasound image data set; wherein the ultrasound image with the transverse abdominal muscle plane structure includes key abdominal tissue structure information that can be observed during the TAP block process; and construct an RMSD-Net model with a U-shaped network structure; The RMSD-Net model includes an encoder, a jump connection structure and a decoder; the encoder uses the original double convolution structure in each layer, and adds a multi-scale residual feature extraction module RMDC module in each layer accordingly, so as to extract features of the input ultrasound image with the transverse abdominal muscle plane structure layer by layer, thereby strengthening the important features of the aforementioned input ultrasound image with the transverse abdominal muscle plane structure layer by layer, and suppressing secondary information at the same time; the jump connection structure uses the feature fusion module SFF module layer by layer to enhance feature expression and promote the integration of contextual information; an attention mechanism GAM module is added to the last layer of the decoder to combine channel attention and spatial attention to integrate local information and global information in the feature map, and dynamically adjust the weights to provide feature representation for subsequent feature weighting; the segmentation task is performed for the aforementioned key abdominal tissue structure information, the aforementioned abdominal ultrasound image data set is divided into a training set and a test set, and the aforementioned RMSD-Net model is used for training and testing respectively; the tested RMSD-Net model is used to segment and identify the key abdominal tissue structure information of the newly acquired ultrasound image with the transverse abdominal muscle plane structure.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.

Images