Ultrasonic diaphragm parameter automatic measurement method and device based on multi-scale dilated convolution, medium and product
By constructing a multi-scale dilated convolutional segmentation network, the problem of relying on manual operation for measuring diaphragm thickness, mobility, and thickening fraction was solved, and automated and accurate measurement of diaphragm parameters was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2024-03-13
- Publication Date
- 2026-06-16
AI Technical Summary
In existing technologies, the measurement of diaphragm thickness, mobility, and thickening fraction relies on manual operation by professional sonographers, which has the problems of strong subjectivity, cumbersome operation, and poor consistency.
A multi-scale dilated convolutional segmentation network was constructed to replace the dual convolutional module in the encoder feature extraction stage of the traditional u-net network, and a convolutional module was set between the encoder and decoder to build a multi-scale dilated convolutional segmentation network for automatic segmentation and parameter measurement of ultrasound diaphragm images.
This improved the accuracy of diaphragm segmentation results, enabled automatic measurement of diaphragm parameters, reduced manual operations, and improved measurement efficiency and accuracy.
Smart Images
Figure CN118505596B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing, and in particular to an automatic measurement method, device, medium, and product for ultrasound diaphragm parameters based on multi-scale dilatational convolution. Background Technology
[0002] Diaphragmatic ultrasound assessment has become an important clinical tool for diaphragmatic evaluation, playing a crucial role in the diagnosis of stroke-related diaphragmatic issues, COPD diagnosis, and diaphragmatic electrical activity. Clinically, diaphragmatic thickness, mobility, and thickening fraction are commonly used as key indicators for assessing diaphragmatic function, and are significant for COPD diagnosis and weaning mechanically ventilated patients. However, current clinical measurements of diaphragmatic thickness, mobility, and thickening fraction are still performed by trained sonographers. Manual measurement suffers from drawbacks such as strong subjective bias, poor consistency, and cumbersome operation, necessitating a convenient automated measurement method. Summary of the Invention
[0003] The purpose of this invention is to provide an automatic measurement method, device, medium, and product for ultrasound diaphragm parameters based on multi-scale dilated convolution. Based on the traditional u-net network, the dual convolution modules in each feature extraction stage of the encoder in the traditional network are replaced with dilated convolution modules of different scales. Furthermore, a 1×1 convolution module is set between the encoder and decoder, forming a multi-scale dilated convolution segmentation network. Using this network for image segmentation greatly improves the accuracy of ultrasound diaphragm segmentation results, thereby improving the accuracy of ultrasound diaphragm parameter measurement.
[0004] To achieve the above objectives, the present invention provides the following solution:
[0005] An automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution, the method comprising:
[0006] Acquire ultrasound diaphragm videos and perform frame splitting on the ultrasound diaphragm videos to obtain an ultrasound diaphragm image frame sequence; the ultrasound diaphragm videos include a first ultrasound diaphragm video for calculating diaphragm thickness and a second ultrasound diaphragm video for calculating diaphragm movement.
[0007] A multi-scale dilated convolutional segmentation network is used to segment the diaphragm region in each frame of the ultrasound diaphragm video in the first and second ultrasound diaphragm videos, resulting in a first diaphragm segmentation mask image frame sequence and a second diaphragm segmentation mask image frame sequence. The multi-scale dilated convolutional segmentation network is obtained by replacing the dual convolutional modules in each feature extraction stage of the encoder in the traditional u-net network with dilated convolutional modules of different scales, and setting a convolutional module between the encoder and the decoder.
[0008] Calculate the diaphragm thickness in each first diaphragm segmentation mask image in the first diaphragm segmentation mask image frame sequence to obtain the diaphragm thickness sequence; determine the diaphragm thickening fraction based on the diaphragm thickness sequence;
[0009] The diaphragm position is calculated in each second diaphragm segmentation mask image frame sequence to obtain the diaphragm position sequence; the diaphragm movement is determined based on the diaphragm position sequence.
[0010] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the steps of the automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution as described above.
[0011] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilated convolution as described above.
[0012] A computer program product includes a computer program that, when executed by a processor, implements the steps of an automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilated convolution as described above.
[0013] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0014] This invention provides an automatic measurement method, device, medium, and product for ultrasound diaphragm parameters based on multi-scale dilated convolution. The invention replaces the dual convolutional modules in each feature extraction stage of the encoder in the traditional u-net network with dilated convolutional modules of different scales. Furthermore, a convolutional module is placed between the encoder and decoder, forming a multi-scale dilated convolutional segmentation network. The dilated convolutional modules effectively extract contextual information, enhancing the feature extraction capability of the backbone network. Simultaneously, a 1×1 convolutional module is designed between the encoder and decoder for channel mixing, improving the model's expressive power. Therefore, using this multi-scale dilated convolutional segmentation network for image segmentation significantly improves the accuracy of ultrasound diaphragm segmentation results, thereby improving the accuracy of ultrasound diaphragm parameter measurement. In addition, the measurement method designed in this invention enables automatic parameter measurement without manual operation, improving the efficiency of parameter measurement. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a schematic diagram of the process flow of an automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution provided in Embodiment 1 of the present invention;
[0017] Figure 2 This is a schematic diagram of the multi-scale dilated convolutional segmentation network structure provided in Embodiment 1 of the present invention;
[0018] Figure 3 This is a schematic diagram of the DilationConvBlock structure provided in Embodiment 1 of the present invention;
[0019] Figure 4 This is a schematic diagram of the automatic measurement of diaphragmatic thickening fraction provided in Embodiment 1 of the present invention;
[0020] Figure 5 This is a diagram of the internal structure of a computer device. Detailed Implementation
[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] The purpose of this invention is to provide an automatic measurement method, device, medium, and product for ultrasound diaphragm parameters based on multi-scale dilated convolution. Based on the traditional u-net network, the dual convolution modules in each feature extraction stage of the encoder in the traditional network are replaced with dilated convolution modules of different scales. Furthermore, a convolution module is placed between the encoder and decoder, forming a multi-scale dilated convolution segmentation network. Using this network for image segmentation greatly improves the accuracy of ultrasound diaphragm segmentation results, thereby improving the accuracy of ultrasound diaphragm parameter measurement.
[0023] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0024] Example 1
[0025] like Figure 1As shown, this embodiment provides an automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution, the method comprising:
[0026] S1: Acquire ultrasound diaphragm video and perform frame splitting on the ultrasound diaphragm video to obtain an ultrasound diaphragm image frame sequence; the ultrasound diaphragm video includes a first ultrasound diaphragm video for calculating diaphragm thickness and a second ultrasound diaphragm video for calculating diaphragm mobility.
[0027] For ease of distinction, the video recording of diaphragm thickness is referred to as the "8th-10th intercostal space" video, and the video recording of diaphragm movement is referred to as the "midclavicular line-anterior axillary line" video. The specific acquisition method is as follows: ① Diaphragm thickness: The target person is placed in a supine position. A high-frequency linear array probe (10-12MHz) is selected. The probe is placed perpendicular to the long axis of the ribs or parallel to the intercostal space at the 8th to 10th intercostal space along the mid-axillary line or anterior axillary line. The probe imaging mode is set to B-mode. Generally, the diaphragm can be seen at a depth of 2-4cm, presenting as the hyperechoic pleura and peritoneum on both sides and the anechoic diaphragm layer in the middle. After adjusting the probe parameters to make the diaphragm clearly visible, a fixed point is selected to place the sampling line. The B+M mode is selected, and the "cinema" function of the ultrasound machine is used to record ultrasound diaphragm videos for at least 2-3 respiratory cycles. Save the first ultrasound diaphragm video; ② Diaphragm movement: The target person is in a supine position. Select a low-frequency convex array probe (2-5MHz). Place the convex array probe on the lower edge of the costal arch between the midclavicular line and the anterior axillary line, with the direction as perpendicular as possible to the direction of movement of the dome. Select the probe imaging mode as B mode. The liver, inferior vena cava, and hyperechoic diaphragm line can be seen in the same section. Adjust the probe parameters to make the hyperechoic diaphragm clearly visible. Select B+M mode, and place the sampling line perpendicular to the diaphragm. Use the "cinema" function of the ultrasound machine to record at least 2-3 respiratory cycles of ultrasound diaphragm video and save it to obtain the second ultrasound diaphragm video.
[0028] The video captured using the OpenCV library is split into frames to obtain all the frames of the video, which are then arranged in order to obtain the frame sequence Image = {image1, image2, ..., imageN}.
[0029] S2: The diaphragm region of each frame of ultrasound diaphragm image in the first and second ultrasound diaphragm videos is segmented using a multi-scale dilated convolutional segmentation network to obtain the first and second diaphragm segmentation mask image frame sequences. The multi-scale dilated convolutional segmentation network is obtained by replacing the dual convolutional modules of each feature extraction stage of the encoder in the traditional u-net network with dilated convolutional modules of different scales, and setting a 1×1 convolutional module or a 3×3 convolutional module between the encoder and the decoder.
[0030] A multi-scale dilated convolutional segmentation network was designed to segment each frame of the split frame sequence Image, representing the ultrasound diaphragm image. The multi-scale dilated convolutional segmentation network can automatically identify and segment the diaphragm region in each frame, obtaining the diaphragm segmentation result for each frame, and finally obtaining the frame mask sequence Mask = {mask1, mask2, ..., maskN}.
[0031] Multi-scale dilated convolution segmentation networks are mainly designed based on dilated convolution modules and the u-net network, such as... Figure 2 As shown. The main improvements of the multi-scale dilated convolution segmentation network are based on the following three parts: 1) This model designs DilationConvBlock modules (dilated convolution modules) of different scales at different stages of the segmentation network based on dilated convolution, such as... Figure 3 As shown, the dual convolutional modules in the original u-net network are replaced with these modules. The multi-scale dilated convolutional modules effectively extract contextual information, enhancing the feature extraction capability of the backbone network. 2) Simultaneously, 1×1 convolutional modules are designed for channel mixing in the bottleneck region (between the encoder and decoder) to improve the model's expressive power. 3) The number of channels is reduced. Compared to the standard u-net network, the number of channels in each stage of the network is halved, reducing model complexity and the risk of overfitting while ensuring computational accuracy. Figure 2 In the diagram, the number of channels for each stage are C1=32, C2=64, C3=128, C4=256, and C5=512. By reducing the number of channels, the number of model parameters is reduced.
[0032] Among them, such as Figure 3 As shown, the dilated convolution module mainly consists of a 3×3 convolutional layer, a 3×3 dilated convolutional layer, two BatchNorm layers, and two ReLU layers. Each convolutional layer undergoes appropriate padding to ensure that the shapes of the input and output tensors remain unchanged. Assume the input tensor X is (B, C, H, W), where B is the batch size, C is the number of channels, and H and W represent the height and width, respectively. This tensor sequentially passes through the 3×3 convolutional layer, BatchNorm layer, ReLU layer, 3×3 dilated convolutional layer, BatchNorm layer, and ReLU layer. After residual connections, the output tensor retains its shape (B, C, H, W). The dilation coefficients of the dilated convolutional layers are designed according to the different stages they are embedded in.
[0033] like Figure 2As shown, the multi-scale dilated convolutional segmentation network adopts an "encoder-decoder" architecture, which can be divided into two stages: an encoder stage and a decoder stage. The encoding stage includes five feature extractions and four downsampling operations, while the decoding stage includes four upsampling operations and four feature fusion operations. Furthermore, the features from the encoder's final stage are processed by a 1×1 convolutional block for channel feature mixing before being passed to the decoder. The output of the decoder's final stage is then processed by a 1×1 convolutional block to fuse all channel features, yielding the final output. This output is then used by a sigmoid layer to predict the segmentation mask. All four downsampling operations utilize max-pooling layers to reduce resolution. All four upsampling operations employ bilinear interpolation to improve resolution. The encoder's five feature extraction modules all employ the designed dilated convolution module for feature extraction, but the dilation factor differs at each stage: In the first feature extraction, to capture local image information and avoid wasting pixel information, the dilation factor is set to 1, equivalent to using a standard 3×3 convolution kernel. The dilation factors for the second, third, fourth, and fifth feature extractions are 2, 3, 5, and 7 respectively. This results in a larger receptive field for each pixel in later stages, allowing for the acquisition of more global contextual information. In other words, the earlier stages of feature extraction capture more local information, while the later stages capture more global contextual information. The decoder's four feature fusion stages maintain the same design as the original u-net. In each stage of feature fusion, the upsampled feature map is first concatenated with the feature map of the same size as the encoder stage, and then fused using two consecutive 3×3 convolution modules.
[0034] Combination Figure 2 In step S2, a multi-scale dilated convolutional segmentation network is used to segment the diaphragm region of each frame of the ultrasound diaphragm image in the first and second ultrasound diaphragm videos, obtaining a first diaphragm segmentation mask image frame sequence and a second diaphragm segmentation mask image frame sequence, specifically including:
[0035] (1) Perform dilation convolution operation on the input image to obtain the first extracted feature image; the input image is each frame of ultrasound diaphragm image in the first ultrasound diaphragm video and the second ultrasound diaphragm video.
[0036] (2) Perform downsampling and dilation convolution operations on the first extracted feature image in sequence to obtain the second extracted feature image.
[0037] (3) Perform downsampling and dilation convolution operations on the second extracted feature image in sequence to obtain the third extracted feature image.
[0038] (4) Perform downsampling and dilation convolution operations sequentially on the third extracted feature image to obtain the fourth extracted feature image.
[0039] (5) Perform downsampling and dilation convolution operations on the fourth extracted feature image in sequence to obtain the fifth extracted feature image.
[0040] (6) Perform a 1×1 convolution operation or a 3×3 convolution operation on the fifth extracted feature image to obtain the convolved image.
[0041] (7) Perform an upsampling operation on the convolved image to obtain a first upsampled image; perform a concatenation operation on the first upsampled image and the fourth extracted feature image to obtain a first connected image; perform a double convolution operation on the first connected image to obtain a first feature fusion image.
[0042] (8) Perform an upsampling operation on the first feature fusion image to obtain a second upsampled image; perform a concatenation operation on the second upsampled image and the third extracted feature image to obtain a second connected image; perform a double convolution operation on the second connected image to obtain a second feature fusion image.
[0043] (9) Perform an upsampling operation on the second feature fusion image to obtain a third upsampling image; perform a concatenation operation on the third upsampling image and the second extracted feature image to obtain a third connected image; perform a double convolution operation on the third connected image to obtain a third feature fusion image.
[0044] (10) Perform an upsampling operation on the third feature fusion image to obtain a fourth upsampling image; perform a concatenation operation on the fourth upsampling image and the first extracted feature image to obtain a fourth connected image; perform a double convolution operation on the fourth connected image to obtain a fourth feature fusion image.
[0045] (11) Perform a 1×1 convolution operation on the fourth feature fusion image to obtain the diaphragm segmentation mask image corresponding to the input image.
[0046] Training process of multi-scale dilated convolutional segmentation network:
[0047] (1) Data Collection. Collect as many ultrasound images of the diaphragm as possible at the "8th-10th intercostal space" and the "midclavicular line-anterior axillary line" locations, and have a professional physician label the diaphragm regions. Here, we use 258 collected images of diaphragm thickness as an example.
[0048] (2) Divide the dataset into training set, validation set and test set in a ratio of 8:1:1.
[0049] (3) The designed AI segmentation model (multi-scale dilated convolutional segmentation network) was used for training.
[0050] (4) The loss function used is BCEWithLogitsLoss, which is calculated as follows:
[0051] loss(X, Y) = ∑ i∈[1，H]，j∈[1，W)]，n∈[1，batch_size] -(y n，i，j log(x n，i，j )+(1-y n，i，j log(1-x) n，i，j )).
[0052] This loss function is a batch loss function, where X represents the predicted mask, Y represents the manually annotated standard mask, batch_size represents the batch size, H represents the image height, and W represents the image width. n,i,j and Y n,i,j Let represent the predicted mask and the standard mask values in the i-th column and j-th row of the n-th image in a batch, respectively. Their values range from 0 to 1, where 1 indicates a positive prediction and 0 indicates a negative prediction. Here, ylog(x) + (1-y)log(1-x) is used to calculate the binary cross-entropy. This loss function calculates the sum of the losses from the predicted mask and the standard mask for all images in a batch.
[0053] (5) Training details: The AdamW optimizer was used, with a learning rate of 0.001, β of [0.9, 0.999], eps of 1e-8, and weight_decay of 0.01. The learning rate regulator was the Cosine AnnealingLR algorithm, with T_max = 50 and eta_min = 0.00001. Training was performed over 150 epochs.
[0054] (6) Evaluation metrics: IoU (Intersection over Union) and DSC (Dice similarity coefficient) are used to evaluate the segmentation accuracy of the designed model.
[0055] IoU and DSC can be calculated as follows:
[0056] 1) Calculate TP, FP, and FN. TP stands for true positive, which is a pixel predicted as positive and is actually positive; FP stands for false positive, which is a pixel predicted as positive but is actually negative; FN stands for false negative, which is a pixel predicted as negative but is actually positive.
[0057] 2) Calculate IoU: IoU = TP / (TP + FP + FN).
[0058] 3) Calculate DSC: DSC = 2 * TP / (2 * TP + FP + FN).
[0059] (7) Experimental Results
[0060] The table below compares the segmentation performance of the multi-scale dilated convolutional segmentation network model and the baseline model u-net on a dataset of 258 collected diaphragm thickness images:
[0061] Network Model IoU DSC U-net 0.7387 0.8357 Ours 0.7751 0.8626
[0062] S3: Calculate the diaphragm thickness of each first diaphragm segmentation mask image in the first diaphragm segmentation mask image frame sequence to obtain the diaphragm thickness sequence; determine the diaphragm thickening fraction based on the diaphragm thickness sequence.
[0063] When measuring diaphragm thickness, the M-mode is commonly used. The sampling line is placed perpendicular to a clearly defined pleura and peritoneum. The diaphragm thickness can be obtained by locating the end of the expiratory and inspiratory phases using the M-mode ultrasound image. At the 8th-10th intercostal space, the diaphragm ultrasound image shows a clear and distinct pleura and peritoneum in the central region, while the edges are obscured by the ribs, resulting in no obvious echoes. Therefore, the measurement of diaphragm thickness should be performed by fixing the measurement point in a region where the pleura and peritoneum are clearly visible. Specifically, in the middle region of the mask in the completed ultrasound diaphragm segmentation image sequence Masks, a suitable point is selected and fixed. A vertically downward straight line is drawn, and the number of pixels passing through the mask is calculated to obtain the diaphragm pixel thickness sequence DT(pix) = {dt1, dt2, ..., dtN}.
[0064] Resolution is defined as the actual distance represented by a unit pixel, i.e.: Resolution = Actual distance * Magnification / Number of pixels; Resolution can be calculated from actual images or obtained directly from the ultrasound scanner user manual. The diaphragm thickness sequence DT(pix) can be calculated from the obtained diaphragm pixel thickness sequence DT(pix) using the formula: DT = DT(pix) * Resolution.
[0065] Based on the obtained diaphragm thickness sequence DT, find the end of the inspiratory phase and the end of the expiratory phase of each respiratory cycle, which are the maximum and minimum values of the sequence. According to the formula: Diaphragm thickening fraction = (diaphragm thickness at the end of inspiration - diaphragm thickness at the end of expiration) / diaphragm thickness at the end of expiration * 100%, calculate the diaphragm thickening fraction. Calculate the average value of 2-3 respiratory cycles to obtain the final diaphragm thickening fraction DTF.
[0066] Therefore, as Figure 4 As shown, the specific process of step S3 is as follows:
[0067] (31) For each of the first diaphragm segmentation mask images, a first position point is selected on the upper or lower side of the diaphragm region of the first diaphragm segmentation mask image; the first position point is at a preset distance from the midline of the diaphragm region; the coordinates of the first position point of each of the first diaphragm segmentation mask images are the same.
[0068] (32) Draw a vertical line from the first position point toward the direction of the diaphragm region; the vertical line intersects with the upper and lower sides of the diaphragm region at points, which are recorded as the first intersection point.
[0069] (33) Determine the number of pixels between the two first intersection points, and then obtain the diaphragm pixel thickness in the first diaphragm segmentation mask image.
[0070] (34) Multiply the diaphragm pixel thickness of each of the first diaphragm segmentation mask images by the image resolution to obtain the diaphragm thickness sequence.
[0071] (35) Calculate the diaphragm thickening fraction for one respiratory cycle based on the diaphragm thickness at the end of inspiration and the diaphragm thickness at the end of expiration in the diaphragm thickness sequence.
[0072] (36) The diaphragm thickening fraction of multiple respiratory cycles is averaged to obtain the final diaphragm thickening fraction.
[0073] S4: Calculate the diaphragm position in each second diaphragm segmentation mask image in the second diaphragm segmentation mask image frame sequence to obtain the diaphragm position sequence; determine the diaphragm movement based on the diaphragm position sequence.
[0074] The dome-shaped diaphragm can be seen at the midclavicular line-anterior axillary line. At the mask of the image sequence Masks after the ultrasound diaphragm segmentation, select a suitable point and draw a straight line perpendicular to the dome-shaped diaphragm. Record the pixel positions where the line intersects the diaphragm to obtain the diaphragm pixel position sequence DP(pix)={dp1,dp2,...,dpN}.
[0075] The obtained diaphragm pixel position sequence DP(pix) can be used to calculate the diaphragm position sequence using the formula: DP = DP(pix) * resolution. Based on the obtained diaphragm position sequence DP, the end of the inspiratory phase and the end of the expiratory phase of each respiratory cycle are found, which are the maximum and minimum values of the sequence. The diaphragm movement is calculated using the formula: Diaphragm movement at the end of inspiration - Diaphragm movement at the end of expiration. The average value of 2-3 respiratory cycles is then calculated to obtain the final diaphragm movement DE.
[0076] Therefore, the specific process of step S4 is as follows:
[0077] (41) For each second diaphragm segmentation mask image, select a second position point on the upper or lower side of the diaphragm region of the second diaphragm segmentation mask image; the coordinates of the second position point of each second diaphragm segmentation mask image are the same.
[0078] (42) Draw a vertical line from the second position point toward the direction of the diaphragm region; the vertical line intersects with the upper or lower side of the diaphragm region at a point, which is recorded as the second intersection point.
[0079] (43) Determine the pixel position of the second intersection point to obtain the diaphragm pixel position of the second diaphragm segmentation mask image.
[0080] (44) Multiply the diaphragm pixel position of each second diaphragm segmentation mask image by the image resolution to obtain the diaphragm position sequence.
[0081] (45) Calculate the diaphragm movement in one respiratory cycle based on the diaphragm position at the end of inspiration and the diaphragm position at the end of expiration in the diaphragm position sequence.
[0082] (46) Take the average value of the diaphragm movement over multiple respiratory cycles to obtain the final diaphragm movement.
[0083] This invention designs an automatic segmentation algorithm for diaphragm ultrasound images based on a hybrid multi-scale module. A segmentation model with strong feature extraction capabilities is designed to achieve automatic segmentation of diaphragm ultrasound images. Based on the segmented diaphragm regions, researchers can freeze the images when obtaining diaphragm ultrasound images at the end of inspiration and expiration. The algorithm will automatically identify and segment the ultrasound diaphragm regions. After obtaining the segmented diaphragm images, the system will automatically calculate the diaphragm thickness / position based on pixels and resolution, obtaining the diaphragm thickness / position at the end of inspiration and expiration, respectively. Furthermore, the system can automatically calculate the diaphragm thickening fraction and diaphragm mobility.
[0084] This invention proposes a novel ultrasound diaphragm segmentation model: a multi-scale dilated convolutional segmentation network. Its main advantages are: 1) High segmentation accuracy: applicable to the segmentation of the anastomosis zone and abdominal diaphragm in B-mode and M-mode scans; 2) Automatic segmentation and measurement: capable of automatically identifying and segmenting the diaphragm region, and measuring the diaphragm thickness / position; 3) Capable of performing corresponding segmentation and automatic measurement based on the diaphragm ultrasound video, and finding the end of the expiratory phase and the end of the inspiratory phase, thereby further automatically calculating the diaphragm thickening fraction and diaphragm mobility.
[0085] Example 2
[0086] This embodiment provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the steps of the automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution in Embodiment 1.
[0087] Example 3
[0088] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of an automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution as described in Embodiment 1.
[0089] Example 4
[0090] This embodiment provides a computer program product, including a computer program that, when executed by a processor, implements the steps of an automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution as described in Embodiment 1.
[0091] Example 5
[0092] This embodiment provides a computer device, which may be a database, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores pending transactions. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements the automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilated convolution as described in Example 1.
[0093] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0094] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0095] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0096] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. An automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilated convolution, characterized in that, The method includes: Acquire ultrasound diaphragm videos and perform frame splitting on the ultrasound diaphragm videos to obtain an ultrasound diaphragm image frame sequence; the ultrasound diaphragm videos include a first ultrasound diaphragm video for calculating diaphragm thickness and a second ultrasound diaphragm video for calculating diaphragm movement. A multi-scale dilated convolutional segmentation network is used to segment the diaphragm region in each frame of the ultrasound diaphragm video in the first and second ultrasound diaphragm videos, resulting in a first diaphragm segmentation mask image frame sequence and a second diaphragm segmentation mask image frame sequence. The multi-scale dilated convolutional segmentation network is obtained by replacing the dual convolutional modules in each feature extraction stage of the encoder in the traditional u-net network with dilated convolutional modules of different scales, and setting a convolutional module between the encoder and the decoder. Calculate the diaphragm thickness in each first diaphragm segmentation mask image in the first diaphragm segmentation mask image frame sequence to obtain the diaphragm thickness sequence; determine the diaphragm thickening fraction based on the diaphragm thickness sequence; The diaphragm position is calculated in each second diaphragm segmentation mask image frame sequence to obtain the diaphragm position sequence; the diaphragm movement is determined based on the diaphragm position sequence.
2. The automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution according to claim 1, characterized in that, The number of channels in each feature extraction stage of the multi-scale dilated convolutional segmentation network is half the number of channels in the corresponding stage of the traditional u-net network.
3. The automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution according to claim 1, characterized in that, The dilated convolution module includes: a residual layer and a first convolutional layer, a first BatchNorm layer, a first ReLU layer, a dilated convolutional layer, a second BatchNorm layer, and a second ReLU layer connected in sequence; the input of the residual layer is the input of the first convolutional layer and the output of the second ReLU layer.
4. The automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution according to claim 1, characterized in that, The dilation factor of the dilated convolution module in different feature extraction stages of the multi-scale dilated convolution segmentation network is different.
5. The automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution according to claim 1, characterized in that, A multi-scale dilated convolutional segmentation network is used to segment the diaphragm region in each frame of the ultrasound diaphragm video in the first and second ultrasound diaphragm videos, resulting in a first diaphragm segmentation mask image frame sequence and a second diaphragm segmentation mask image frame sequence, specifically including: Perform a dilation convolution operation on the input image to obtain a first extracted feature image; the input image is each frame of ultrasound diaphragm image in the first ultrasound diaphragm video and the second ultrasound diaphragm video. The first extracted feature image is subjected to downsampling and dilation convolution operations in sequence to obtain the second extracted feature image; The second extracted feature image is subjected to downsampling and dilation convolution operations in sequence to obtain the third extracted feature image; The third extracted feature image is subjected to downsampling and dilation convolution operations in sequence to obtain the fourth extracted feature image; The fourth extracted feature image is subjected to downsampling and dilation convolution operations in sequence to obtain the fifth extracted feature image; Perform a convolution operation on the fifth extracted feature image to obtain the convolved image; An upsampling operation is performed on the convolved image to obtain a first upsampled image; a concatenation operation is performed on the first upsampled image and the fourth extracted feature image to obtain a first connected image; a double convolution operation is performed on the first connected image to obtain a first feature fusion image. An upsampling operation is performed on the first feature fusion image to obtain a second upsampling image; a concatenation operation is performed on the second upsampling image and the third extracted feature image to obtain a second connected image; a double convolution operation is performed on the second connected image to obtain a second feature fusion image. An upsampling operation is performed on the second feature fusion image to obtain a third upsampling image; a concatenation operation is performed on the third upsampling image and the second extracted feature image to obtain a third connected image; a double convolution operation is performed on the third connected image to obtain a third feature fusion image. An upsampling operation is performed on the third feature fusion image to obtain a fourth upsampling image; a concatenation operation is performed on the fourth upsampling image and the first extracted feature image to obtain a fourth connected image; a double convolution operation is performed on the fourth connected image to obtain a fourth feature fusion image. Perform a 1×1 convolution operation on the fourth feature fusion image to obtain the diaphragm segmentation mask image corresponding to the input image.
6. The automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution according to claim 1, characterized in that, Calculate the diaphragm thickness in each of the first diaphragm segmentation mask image frames in the first diaphragm segmentation mask image frame sequence to obtain the diaphragm thickness sequence; determine the diaphragm thickening fraction based on the diaphragm thickness sequence, specifically including: For each of the first diaphragm segmentation mask images, a first position point is selected on the upper or lower side of the diaphragm region of the first diaphragm segmentation mask image; the first position point is at a preset distance from the midline of the diaphragm region; the coordinates of the first position point are the same for each of the first diaphragm segmentation mask images; Draw a vertical line from the first position point toward the diaphragm region; the vertical line intersects the upper and lower sides of the diaphragm region at points, which are denoted as the first intersection point; By determining the number of pixels between the two first intersection points, the diaphragm pixel thickness in the first diaphragm segmentation mask image is obtained; Multiply the diaphragm pixel thickness of each of the first diaphragm segmentation mask images by the image resolution to obtain the diaphragm thickness sequence. The diaphragm thickening fraction for one respiratory cycle is calculated based on the end-inspiratory and end-expiratory diaphragm thicknesses in the diaphragm thickness sequence. The final diaphragm thickening score is obtained by averaging the diaphragm thickening scores over multiple respiratory cycles.
7. The automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution according to claim 1, characterized in that, Calculate the diaphragm position in each second diaphragm segmentation mask image frame sequence to obtain the diaphragm position sequence; determine the diaphragm mobility based on the diaphragm position sequence, specifically including: For each second diaphragm segmentation mask image, a second position point is selected on the upper or lower side of the diaphragm region of the second diaphragm segmentation mask image; the coordinates of the second position point are the same for each second diaphragm segmentation mask image. Starting from the second position point, draw a vertical line in the direction of the diaphragm region; the vertical line intersects the upper or lower side of the diaphragm region at a point, which is recorded as the second intersection point; Determine the pixel position of the second intersection point to obtain the diaphragm pixel position of the second diaphragm segmentation mask image; Multiply the diaphragm pixel position of each second diaphragm segmentation mask image by the image resolution to obtain the diaphragm position sequence; The diaphragm movement in one respiratory cycle is calculated based on the diaphragm position at the end of inspiration and the diaphragm position at the end of expiration in the diaphragm position sequence. The final diaphragmatic movement is obtained by averaging the diaphragmatic movement over multiple respiratory cycles.
8. A computer device comprising: The memory and processor contain a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilated convolution as described in any one of claims 1-7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution as described in any one of claims 1-7.
10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the automatic measurement method for ultrasound diaphragm parameters based on multi-scale dilatational convolution as described in any one of claims 1-7.