Small target accurate segmentation method for maternal-fetal ultrasonic image during delivery

Abstract

The invention discloses a small target accurate segmentation method based on an intra-delivery maternal-fetal ultrasound image, and the method comprises the following steps: enabling the obtained and extracted intra-delivery maternal-fetal ultrasound image to pass through a neural network which comprises a displacement feature extraction layer, a space and channel information modeling layer and a space gating up-sampling layer, and finally carrying out the segmentation of a small target under the condition that the overall segmentation precision is higher. High-precision segmentation of pubis symphysis is achieved, a doctor is assisted in measuring the parameter of delivery, and delivery progress evaluation is completed.

Description

Technical Field

[0001] This invention relates to the fields of translational medicine and artificial intelligence, and in particular to a method for precise segmentation of small targets in maternal and fetal ultrasound images during childbirth. Background Technology

[0002] In clinical practice, measuring labor parameters using intrapartum maternal-fetal ultrasound images has become routine. This process first requires accurate segmentation of the fetal head and pubic symphysis in the ultrasound images to assess labor progress. However, the pubic symphysis accounts for a small percentage of pixels in the image (approximately 20% of the maternal-fetal ultrasound) and is susceptible to edge loss and noise interference due to fetal movement and physiological changes. This results in traditional graphics and deep learning methods achieving significantly lower accuracy in pubic symphysis segmentation compared to fetal head segmentation. Therefore, a new method to improve pubic symphysis segmentation accuracy is urgently needed to reduce the gap between fetal head and pubic symphysis segmentation accuracy, thereby reducing errors in labor parameter measurement and providing more accurate decision support for obstetricians. Although small-object segmentation methods have made some progress, some problems still exist:

[0003] (1) Current data augmentation methods may increase computational complexity and the risk of overfitting.

[0004] (2) Post-processing methods are used to improve the segmentation accuracy of small targets by processing the output image of the neural network. However, the network cannot learn by adjusting the weights according to the post-processing output, which limits the further improvement of segmentation accuracy.

[0005] (3) While neural networks using feature pyramids and dilated convolutional structures help improve the segmentation accuracy of small targets, they do not solve the problem of losing subtle features in deep networks.

[0006] Furthermore, the number of doctors currently skilled in maternal-fetal ultrasound image segmentation and labor parameter measurement is limited, especially in impoverished areas. Therefore, to facilitate labor parameter measurement, an algorithm capable of simultaneously and accurately segmenting the fetal head and pubic symphysis is needed for better application in clinical practice.

[0007] Invention Content Information

[0008] The purpose of this invention is to overcome the shortcomings and deficiencies of existing technologies in intrapartum ultrasound image segmentation, and to provide a method for accurate segmentation of small targets in intrapartum maternal and fetal ultrasound images. This solves the problem that existing methods based on traditional image segmentation and deep learning segmentation have low accuracy in segmenting the pubic symphysis target, thus affecting obstetricians' measurement of delivery parameters.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a method for precise segmentation of small targets in maternal-fetal ultrasound images during childbirth, the method comprising the following steps:

[0010] (1) Acquire multiple intrapartum maternal-fetal ultrasound images containing two anatomical structures: the fetal head and the pubic symphysis.

[0011] (2) Each intrapartum ultrasound image of the mother and fetus obtained in step (1) is put into the trained neural network and the segmentation region of the fetal head and pubic symphysis is output.

[0012] Preferably, the neural network structure is as follows:

[0013] The first layer is the input layer, which takes as input maternal and fetal ultrasound images at birth, each with a size of 3*256*256.

[0014] The second layer is a convolutional layer that receives the input image. This layer uses 4x4 kernels, with 96 kernels and a stride of 4. It employs SAME padding and ultimately outputs a 96x64x64 matrix.

[0015] The third layer is the displacement feature extraction layer, which receives the output matrix from the second layer. First, the input matrix is ​​divided into 12 parts according to the channel dimension, with each part containing 8 channels. The divided features are then shifted by one pixel horizontally to the left, right, up, and down vertically to increase feature richness. Specifically, each channel of the feature is shifted proportionally: one-quarter of the channels are shifted one pixel to the left, one-quarter to the right, one-quarter upwards, and one-quarter downwards, while the last quarter remains unchanged. The shifted features are then residually concatenated with the original input to preserve the original feature information, resulting in a 96*64*64 matrix. This matrix is ​​then fed into a 1*1 convolutional layer with 192 kernels and a stride of 1. After passing through a Gaussian error linear activation function, it is again fed into a 1*1 convolutional layer with 96 kernels and a stride of 1, outputting a 96*64*64 matrix. Repeat the above operation 10 times, and finally output a matrix of size 96*64*64.

[0016] The fourth layer is a downsampling layer that receives the output matrix of the third layer. This layer uses 2*2 convolutional kernels, with 192 kernels and a stride of 2. This layer is filled using the SAME method, and the final output matrix is ​​192*32*32.

[0017] The fifth layer is the displacement-based feature extraction layer, which receives the output matrix from the fourth layer. First, the input matrix is ​​divided into 12 parts according to the channel dimension, with each part containing 16 channels. The divided features are then shifted by one pixel horizontally to the left, right, up, and down vertically to increase feature richness. Specifically, each channel of the feature is shifted proportionally: one-quarter of the channels are shifted one pixel to the left, one-quarter to the right, one-quarter upwards, and one-quarter downwards; the last quarter remains unchanged. The shifted features are then residually concatenated with the original input to preserve the original feature information, resulting in a matrix of size 192*32*32. The input is fed into a convolutional layer with a kernel size of 1*1, 384 kernels, and a stride of 1. After passing through a Gaussian error linear activation function, it is then fed into another convolutional layer with a kernel size of 1*1, 192 kernels, and a stride of 1, outputting a matrix of size 192*32*32. This process is repeated 18 times, finally outputting a matrix of size 192*32*32. The sixth layer is a downsampling layer that receives the output matrix from the fifth layer. This layer uses 2*2 kernels, 384 kernels, and a stride of 2. This layer uses SAME padding, ultimately outputting a matrix of size 384*16*16.

[0018] The seventh layer is the displacement-based feature extraction layer, receiving the output matrix from the sixth layer. First, the input matrix is ​​divided into 12 parts according to the channel dimension, each containing 32 channels. The divided features are then shifted by one pixel horizontally to the left, right, up, and down vertically to increase feature richness. Specifically, each channel of the feature is shifted proportionally: one-quarter of the channels are shifted one pixel to the left, one-quarter to the right, one-quarter upwards, and one-quarter downwards; the last quarter remains unchanged. The shifted features are then residually concatenated with the original input to preserve the original feature information, resulting in a matrix of size 384*16*16. The input is fed into a 1x1 convolutional layer with 768 kernels and a stride of 1. After passing through a Gaussian error linear activation function, it is then fed into another 1x1 convolutional layer with 384 kernels and a stride of 1, outputting a matrix of size 384x16x16. This process is repeated 36 times, finally outputting a matrix of size 384x16x16. The eighth layer is a downsampling layer that receives the output matrix from the seventh layer. This layer uses 2x2 convolutional kernels with 768 kernels and a stride of 2. This layer uses SAME padding, ultimately outputting a matrix of size 768x8x8.

[0019] The ninth layer is the displacement-based feature extraction layer, receiving the output matrix from the eighth layer. First, the input matrix is ​​divided into 12 parts according to the channel dimension, each containing 64 channels. The divided features are then shifted by one pixel horizontally to the left, right, up, and down vertically to increase feature richness. Specifically, each channel of the feature is shifted proportionally: one-quarter of the channels are shifted one pixel to the left, one-quarter to the right, one-quarter upwards, and one-quarter downwards, while the last quarter remains unchanged. The shifted features are then residually concatenated with the original input to preserve the original feature information, resulting in a 768*8*8 matrix. This matrix is ​​then fed into a 1*1 convolutional layer with 1536 kernels and a stride of 1, passing through a Gaussian error linear activation function, and then fed into another 1*1 convolutional layer with 768 kernels and a stride of 1, outputting a 768*8*8 matrix. Repeat the above operation 10 times, and finally output a matrix of size 768*8*8.

[0020] The tenth layer is a convolutional layer that receives the output of the third layer. First, the input matrix is ​​passed through a 16x16 convolutional layer with a kernel size of 96 and a stride of 16. This layer uses SAME padding to obtain a 96x4x4 matrix.

[0021] The eleventh layer is a convolutional layer that receives the output of the fifth layer. First, the input matrix is ​​passed through an 8x8 convolutional layer with 192 kernels and a stride of 8. This layer uses SAME padding to obtain a matrix of size 192x4x4.

[0022] The twelfth layer is a convolutional layer that receives the output from the seventh layer. First, the input matrix is ​​passed through a convolutional layer with a kernel size of 4x4, a kernel count of 384, and a stride of 4. This layer uses the SAME padding method to obtain a 384x4x4 matrix.

[0023] The thirteenth layer is a convolutional layer that receives the output from the ninth layer. First, the input matrix is ​​passed through a convolutional layer with a kernel size of 2x2, a kernel count of 768, and a stride of 2. This layer uses the SAME padding method to obtain a matrix of size 768x4x4.

[0024] The fourteenth layer is the concatenation layer, which receives the outputs from the tenth, eleventh, twelfth, and thirteenth layers. It concatenates the four input matrices together to obtain a 1440*4*4 matrix.

[0025] The fifteenth layer is the spatial and channel information modeling layer, receiving the outputs from the tenth, eleventh, twelfth, thirteenth, and fourteenth layers. First, all input matrices are convolved with 1x1 kernels (the number of kernels equal to the number of channels in the input matrix) and a stride of 1, outputting five matrices of the same size as the outputs from the tenth, eleventh, twelfth, thirteenth, and fourteenth layers, defined as q1, q2, q3, q4, and k, respectively. Then, the output matrix of the fourteenth layer is convolved with 1x1 kernels (the number of kernels equal to the number of channels in the input matrix) and a stride of 1, outputting a matrix of the same size as v. Next, matrices q1, q2, q3, q4, k, and v are convolved with 3x3 kernels (the number of kernels equal to the number of channels in the input matrix) and a stride of 1, adding spatial encoding to each group of feature maps. These convolutions help the model capture local features. All the convolved feature maps (q1, q2, q3, q4, k, and v) are rearranged into a shape suitable for matrix multiplication. Normalization is performed on q1, q2, q3, q4, k, and v to ensure computational stability. Attention weights are calculated by performing dot products of q1, q2, q3, q4, and k, respectively, and then standardized using the Softmax function. Finally, the attention weights are normalized to improve the model's generalization ability. The calculated attention weights are used to weight v, generating a weighted feature representation. The weighted features are averaged along the attention head dimension to reduce model complexity. The aggregated features are rearranged back to their original spatial dimensions, resulting in four output matrices. These four output matrices are then convolved with 1x1 kernels (the number of kernels equals the number of channels in the input matrix) with a stride of 1, and then added to the output matrices of layers 10, 11, 12, and 13, respectively, outputting matrices of sizes 96x4x4, 192x4x4, 382x4x4, and 768x4x4.

[0026] The sixteenth layer is the network's feedforward layer, receiving the output matrix from the fifteenth layer. It contains matrices of sizes 96*4*4, 192*4*4, 382*4*4, and 768*4*4. Taking one input matrix as an example, the 96*4*4 matrix is ​​passed through a 1*1 convolutional layer with the number of kernels equal to the number of channels multiplied by 5.3 (i.e., 510). This maps the feature channels of the input tensor to a higher-dimensional hidden feature space, generating two feature maps, resulting in two 510*4*4 matrices. Next, depthwise convolutions are performed on the two feature maps, using 3*3 and 5*5 kernels respectively. The number of kernels is equal to the original number of channels divided by 2, with a stride of 1 for both. SAME padding is used, and a linear rectified function is used as the activation function. Finally, the feature maps after depthwise convolution and activation are concatenated, connecting the two feature maps along the channel dimension to obtain a richer feature representation, resulting in a 510*4*4 matrix. The input feature map is then passed through a 1x1 convolutional layer with 96 kernels and a stride of 1. Finally, the projected features are input into the global spatial attention module. This module first reduces the spatial dimension of the feature map to 1 using adaptive average pooling, and then performs spatial attention calculation using a 1D convolutional kernel. The resulting attention weights are normalized using the sigmoid function. These normalized attention weights are then multiplied by the input feature map to achieve global feature weighting, and then residually connected to the original input to preserve the original feature information, resulting in a 96x4x4 matrix. The other three matrices are passed in the same way, ultimately outputting matrices of sizes 96x4x4, 192x4x4, 382x4x4, and 768x4x4. The seventeenth layer is the feature resolution restoration layer. It receives a 96*4*4 matrix from the output of the sixteenth layer, performs bilinear interpolation sampling to improve image resolution, and outputs a 96*64*64 matrix. Then, it passes through a convolutional layer with a 1*1 kernel, 96 kernels, a stride of 1, and uses a linear rectified function as the activation function. The result is added to the output of the third layer, and the output is a 96*4*4 matrix.

[0027] The eighteenth layer is the feature resolution restoration layer. It receives a 192*4*4 matrix from the output of the sixteenth layer, performs bilinear interpolation sampling to improve image resolution, and outputs a 192*32*32 matrix. Then, it passes through a convolutional layer with a 1*1 kernel, 192 kernels, a stride of 1, and uses a linear rectified function as the activation function. The result is added to the output of the fifth layer, and the output is a 192*32*32 matrix.

[0028] The nineteenth layer is the feature resolution restoration layer. It receives a 384*4*4 matrix from the output of the sixteenth layer, performs bilinear interpolation sampling to improve image resolution, and outputs a 384*16*16 matrix. Then, it passes through a convolutional layer with 1*1 kernels, 384 kernels, a stride of 1, and uses a linear rectified function as the activation function. The result is added to the output of the seventh layer, and the output is a 384*16*16 matrix.

[0029] The twentieth layer is the feature resolution restoration layer. It receives a 768*4*4 matrix from the output of the sixteenth layer, first performs bilinear interpolation sampling to improve image resolution, outputting a 768*8*8 matrix. Then, it passes through a convolutional layer with 1*1 kernels, 768 kernels, a stride of 1, and uses a linear rectified function as the activation function. This layer is then added to the output of the ninth layer, outputting a 768*8*8 matrix.

[0030] The 21st layer is an upsampling layer that receives the output of the 20th layer. First, the input matrix is ​​transformed to obtain a 64*768 matrix, and then linearly transformed to output a 256*384 matrix.

[0031] Layer 22 is a spatially gated upsampling layer that receives the outputs of layers 21 and 7. First, the output of layer 7 is dimensionally transformed to obtain a 256*384 matrix, which is then added to the output of layer 21. Next, the matrix is ​​dimensionally transformed again, outputting a 384*16*16 matrix, which is then fed into a layer normalization layer for standardization to reduce internal covariate shift. This is then fed into the spatial attention module, passing through a 1*1 convolutional layer with 384 kernels, a stride of 1, and a Gaussian error linear activation function. Finally, it is fed into the spatial gating unit. This unit models the input spatial information and generates a spatial attention mask or weights. The spatial gating unit comprises: a 5x5 convolutional layer with 384 kernels and 384 groups, padded using the SAME method; a 7x7 convolutional layer with 384 kernels and 384 groups, padded using the SAME method with a stride of 3; and a 1x1 convolutional layer with 384 kernels and a stride of 1. The output of the spatial gating unit is multiplied by the input to obtain a matrix of size 384x16x16, which is then passed through a 1x1 convolutional layer with 384 kernels and a stride of 1. Finally, the matrix is ​​added to the input of the spatial attention module. Finally, the perceptron layer consists of one 1x1 convolutional layer with 1536 kernels and a stride of 1; one 3x3 convolutional layer with 1536 kernels and a stride of 1, padded with SAME and passed through a Gaussian error linear activation function; and one 1x1 convolutional layer with 384 kernels and a stride of 1. A 1024x192 matrix is ​​then output through a linear transformation.

[0032] Layer 23 is a spatially gated upsampling layer that receives the outputs of layers 22 and 5. First, the output of layer 5 undergoes a dimensionality transformation, resulting in a 1024*192 matrix, which is then added to the output of layer 21. Next, the matrix undergoes a dimensionality transformation, outputting a 192*32*32 matrix, which is then fed into a layer normalization layer for standardization to reduce internal covariate shift. This matrix is ​​then fed into the spatial attention module, passing through a 1*1 convolutional layer with 192 kernels, a stride of 1, and a Gaussian error linear activation function. Finally, it is fed into the spatial gating unit. This unit models the spatial information of the input and generates a spatial attention mask or weights. The spatial gating unit comprises: a 5x5 convolutional layer with 192 kernels and 192 groups, padded using the SAME method; a 7x7 convolutional layer with 192 kernels and 192 groups, padded using the SAME method with a stride of 3; and a 1x1 convolutional layer with 192 kernels and a stride of 1. The output of the spatial gating unit is multiplied by the input to obtain a matrix of size 192x32x32, which is then passed through a 1x1 convolutional layer with 192 kernels and a stride of 1. Finally, this matrix is ​​added to the input of the spatial attention module. Finally, the perceptron layer consists of one 1x1 convolutional layer with 768 kernels and a stride of 1; one 3x3 convolutional layer with 768 kernels and a stride of 1, padded with SAME and activated by a Gaussian error linear activation function; and one 1x1 convolutional layer with 192 kernels and a stride of 1. A linear transformation then outputs a 4096x96 matrix.

[0033] Layer 24 is a spatially gated upsampling layer, receiving the outputs of layers 23 and 3. First, the output of layer 5 undergoes a dimensionality transformation, resulting in a 4096*96 matrix, which is then added to the output of layer 21. Next, the matrix undergoes another dimensionality transformation, outputting a 96*64*64 matrix, which is then fed into a layer normalization layer for standardization to reduce internal covariate shift. This is then fed into the spatial attention module, passing through a 1*1 convolutional layer with 96 kernels, a stride of 1, and a Gaussian error linear activation function. Finally, it is fed into the spatial gating unit. This unit models the input spatial information and generates a spatial attention mask or weights. The spatial gating unit comprises: a 5x5 convolutional layer with 96 kernels and 96 groups, padded using the SAME method; a 7x7 convolutional layer with 96 kernels and 96 groups, padded using the SAME method with a stride of 3; and a 1x1 convolutional layer with 96 kernels and a stride of 1. The output of the spatial gating unit is multiplied by the input to obtain a matrix of size 96x64x64, which is then passed through a 1x1 convolutional layer with 96 kernels and a stride of 1. Finally, the matrix is ​​added to the input of the spatial attention module. Finally, the perceptron layer consists of one 1x1 convolutional layer with 384 kernels and a stride of 1; one 3x3 convolutional layer with 384 kernels and a stride of 1, padded with SAME and activated by a Gaussian error linear activation function; and one 1x1 convolutional layer with 96 kernels and a stride of 1. A linear transformation then outputs a 4096x96 matrix.

[0034] The 25th layer is the output layer, receiving the output of the 24th layer. First, the input matrix is ​​transformed linearly into a 4096*1536 matrix, then dimensionally transformed into a 96*256*256 matrix. This matrix is ​​then passed through a 1*1 convolutional layer with 3 kernels and a stride of 1. The final output is a 3*256*256 segmentation result.

[0035] Preferably, the neural network is trained through the following steps:

[0036] (2-1) Obtain the maternal and fetal ultrasound image dataset during labor and divide it into training and test sets in an 8:2 ratio.

[0037] (2-2) Send the intrapartum maternal and fetal ultrasound image dataset obtained in step (2-1) to the ultrasound physician for manual annotation to obtain the annotated training set data.

[0038] (2-3) Data augmentation is performed on the training set of the labeled intrapartum maternal and fetal ultrasound image dataset obtained in step (2-2), including random image rotation, with a rotation range between -30 degrees and 30 degrees, random horizontal flipping, and pixel value changes, to obtain a data-augmented intrapartum maternal and fetal ultrasound image training set to improve the robustness of the network.

[0039] (2-4) The augmented intrapartum fetal ultrasound image training set obtained in step (2-3) is then standardized and normalized to obtain a preprocessed intrapartum fetal ultrasound image training set, ensuring the stability of the training neural network.

[0040] (2-5) Input the preprocessed intrapartum maternal and fetal ultrasound images from the training set obtained in step (2-4) into the neural network to obtain the corresponding inference output results. Input the inference output results and the intrapartum maternal and fetal ultrasound images annotated by the sonographer in step (2-2) into the segmentation Dice loss function of the neural network to obtain the loss value.

[0041] (2-6) The loss function of the neural network is optimized using the adaptive moment estimator and the loss value obtained in step (2-5) to iteratively update the parameters in the neural network.

[0042] (2-7) Repeat steps (2-5) and (2-6) above until the neural network converges to the optimal state, thus obtaining a trained neural network.

[0043] The present invention has the following advantages and effects compared with the prior art:

[0044] (1) This invention employs steps (1) to (2) to achieve high-precision segmentation of the pubic symphysis in maternal-fetal ultrasound images through a small-target accurate segmentation method, while also ensuring overall segmentation accuracy. Specifically, representative intrapartum maternal-fetal ultrasound images are randomly selected as a test set to test the neural network. The test set includes 403 maternal-fetal ultrasound images as evaluation criteria to test the detection results of the neural network. The segmentation accuracy of the pubic symphysis in intrapartum maternal-fetal ultrasound images is 88.47%, with an average surface distance of 1.49, while the overall segmentation accuracy reaches 94.94%, with an average surface distance of 1.95. This network can increase the capture of small target features and reduce the loss of small target features in deeper layers of the network, thereby improving the segmentation accuracy of the pubic symphysis while performing high-precision segmentation of maternal-fetal ultrasound images.

[0045] (2) Since the present invention applies a displacement extraction feature layer in step (2), the layer can reduce the interference of irrelevant pixels on the target pixel through displacement operation, thereby extracting more effective features, especially small target features, namely the pubic symphysis.

[0046] (3) Since the present invention applies a spatial information and channel information modeling layer in step (2), it can model the extracted features in spatial and channel dimensions, increase the difference between background pixels and target pixels, and highlight the pubic symphysis in maternal-fetal ultrasound.

[0047] (4) Since the present invention applies a spatial gated upsampling layer in step (3), it can use convolution kernels of different sizes to form spatial weights and perform upsampling, thereby enhancing the feature representation of the neural network. Attached Figure Description

[0048] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0049] Figure 1 This is a flowchart of a method for accurate segmentation of small targets in maternal and fetal ultrasound images during childbirth, as disclosed in this invention.

[0050] Figure 2 This is an example image of the results generated by the method for accurate segmentation of small targets in maternal and fetal ultrasound images during childbirth according to the present invention. The results include the segmentation results of the fetal head and pubic symphysis (both on one image). Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Example 1:

[0053] This embodiment discloses a method for accurate segmentation of small targets based on maternal-fetal ultrasound images during labor, mainly including the following steps:

[0054] S1. Acquire transperineal ultrasound images and label the fetal head and pubic symphysis;

[0055] In step S1, the data was collected by a radiologist. The equipment used was a Philips SONOS 5500B ultrasound machine for intrapartum maternal-fetal ultrasound image acquisition. Under the guidance of the radiologist, four students were tasked with labeling the fetal head and pubic symphysis. After completion, the images were reviewed by the doctor to confirm accuracy.

[0056] S2. Further screen the ultrasound images; among them, screen out maternal-fetal ultrasound images with roughly clear outlines of the fetal head and pubic symphysis to prepare for input into the neural network.

[0057] S3. Input the image obtained in S2 into the trained neural network.

[0058] Specifically, the neural network in S3 includes four displacement feature extraction layers, spatial and channel information modeling layers, and four spatially gated upsampling layers. The neural network structure is as follows:

[0059] The first layer is the input layer, which takes as input maternal and fetal ultrasound images at birth, each with a size of 3*256*256.

[0060] The second layer is a convolutional layer that receives the input image. This layer uses 4x4 kernels, with 96 kernels and a stride of 4. It employs SAME padding and ultimately outputs a 96x64x64 matrix.

[0061] The third layer is the displacement feature extraction layer, which receives the output matrix from the second layer. First, the input matrix is ​​divided into 12 parts according to the channel dimension, with each part containing 8 channels. The divided features are then shifted by one pixel horizontally to the left, right, up, and down vertically to increase feature richness. Specifically, each channel of the feature is shifted proportionally: one-quarter of the channels are shifted one pixel to the left, one-quarter to the right, one-quarter upwards, and one-quarter downwards, while the last quarter remains unchanged. The shifted features are then residually concatenated with the original input to preserve the original feature information, resulting in a 96*64*64 matrix. This matrix is ​​then fed into a 1*1 convolutional layer with 192 kernels and a stride of 1. After passing through a Gaussian error linear activation function, it is again fed into a 1*1 convolutional layer with 96 kernels and a stride of 1, outputting a 96*64*64 matrix. Repeat the above operation 10 times, and finally output a matrix of size 96*64*64.

[0062] The fourth layer is a downsampling layer that receives the output matrix of the third layer. This layer uses 2*2 convolutional kernels, with 192 kernels and a stride of 2. This layer is filled using the SAME method, and the final output matrix is ​​192*32*32.

[0063] The fifth layer is the displacement-based feature extraction layer, which receives the output matrix from the fourth layer. First, the input matrix is ​​divided into 12 parts according to the channel dimension, with each part containing 16 channels. The divided features are then shifted by one pixel horizontally to the left, right, up, and down vertically to increase feature richness. Specifically, each channel of the feature is shifted proportionally: one-quarter of the channels are shifted one pixel to the left, one-quarter to the right, one-quarter upwards, and one-quarter downwards; the last quarter remains unchanged. The shifted features are then residually concatenated with the original input to preserve the original feature information, resulting in a matrix of size 192*32*32. The input is fed into a convolutional layer with a kernel size of 1*1, 384 kernels, and a stride of 1. After passing through a Gaussian error linear activation function, it is then fed into another convolutional layer with a kernel size of 1*1, 192 kernels, and a stride of 1, outputting a matrix of size 192*32*32. This process is repeated 18 times, finally outputting a matrix of size 192*32*32. The sixth layer is a downsampling layer that receives the output matrix from the fifth layer. This layer uses 2*2 kernels, 384 kernels, and a stride of 2. This layer uses SAME padding, ultimately outputting a matrix of size 384*16*16.

[0064] The seventh layer is the displacement-based feature extraction layer, receiving the output matrix from the sixth layer. First, the input matrix is ​​divided into 12 parts according to the channel dimension, each containing 32 channels. The divided features are then shifted by one pixel horizontally to the left, right, up, and down vertically to increase feature richness. Specifically, each channel of the feature is shifted proportionally: one-quarter of the channels are shifted one pixel to the left, one-quarter to the right, one-quarter upwards, and one-quarter downwards; the last quarter remains unchanged. The shifted features are then residually concatenated with the original input to preserve the original feature information, resulting in a matrix of size 384*16*16. The input is fed into a 1x1 convolutional layer with 768 kernels and a stride of 1. After passing through a Gaussian error linear activation function, it is then fed into another 1x1 convolutional layer with 384 kernels and a stride of 1, outputting a matrix of size 384x16x16. This process is repeated 36 times, finally outputting a matrix of size 384x16x16. The eighth layer is a downsampling layer that receives the output matrix from the seventh layer. This layer uses 2x2 convolutional kernels with 768 kernels and a stride of 2. This layer uses SAME padding, ultimately outputting a matrix of size 768x8x8.

[0065] The ninth layer is the displacement-based feature extraction layer, receiving the output matrix from the eighth layer. First, the input matrix is ​​divided into 12 parts according to the channel dimension, each containing 64 channels. The divided features are then shifted by one pixel horizontally to the left, right, up, and down vertically to increase feature richness. Specifically, each channel of the feature is shifted proportionally: one-quarter of the channels are shifted one pixel to the left, one-quarter to the right, one-quarter upwards, and one-quarter downwards, while the last quarter remains unchanged. The shifted features are then residually concatenated with the original input to preserve the original feature information, resulting in a 768*8*8 matrix. This matrix is ​​then fed into a 1*1 convolutional layer with 1536 kernels and a stride of 1, passing through a Gaussian error linear activation function, and then fed into another 1*1 convolutional layer with 768 kernels and a stride of 1, outputting a 768*8*8 matrix. Repeat the above operation 10 times, and finally output a matrix of size 768*8*8.

[0066] The tenth layer is a convolutional layer that receives the output of the third layer. First, the input matrix is ​​passed through a 16x16 convolutional layer with a kernel size of 96 and a stride of 16. This layer uses SAME padding to obtain a 96x4x4 matrix.

[0067] The eleventh layer is a convolutional layer that receives the output of the fifth layer. First, the input matrix is ​​passed through an 8x8 convolutional layer with 192 kernels and a stride of 8. This layer uses SAME padding to obtain a matrix of size 192x4x4.

[0068] The twelfth layer is a convolutional layer that receives the output from the seventh layer. First, the input matrix is ​​passed through a convolutional layer with a kernel size of 4x4, a kernel count of 384, and a stride of 4. This layer uses the SAME padding method to obtain a 384x4x4 matrix.

[0069] The thirteenth layer is a convolutional layer that receives the output from the ninth layer. First, the input matrix is ​​passed through a convolutional layer with a kernel size of 2x2, a kernel count of 768, and a stride of 2. This layer uses the SAME padding method to obtain a matrix of size 768x4x4.

[0070] The fourteenth layer is the concatenation layer, which receives the outputs from the tenth, eleventh, twelfth, and thirteenth layers. It concatenates the four input matrices together to obtain a 1440*4*4 matrix.

[0071] The fifteenth layer is the spatial and channel information modeling layer, receiving the outputs from the tenth, eleventh, twelfth, thirteenth, and fourteenth layers. First, all input matrices are convolved with 1x1 kernels (the number of kernels equal to the number of channels in the input matrix) and a stride of 1, outputting five matrices of the same size as the outputs from the tenth, eleventh, twelfth, thirteenth, and fourteenth layers, defined as q1, q2, q3, q4, and k, respectively. Then, the output matrix of the fourteenth layer is convolved with 1x1 kernels (the number of kernels equal to the number of channels in the input matrix) and a stride of 1, outputting a matrix of the same size as v. Next, matrices q1, q2, q3, q4, k, and v are convolved with 3x3 kernels (the number of kernels equal to the number of channels in the input matrix) and a stride of 1, adding spatial encoding to each group of feature maps. These convolutions help the model capture local features. All the convolved feature maps (q1, q2, q3, q4, k, and v) are rearranged into a shape suitable for matrix multiplication. Normalization is performed on q1, q2, q3, q4, k, and v to ensure computational stability. Attention weights are calculated by performing dot products of q1, q2, q3, q4, and k, respectively, and then standardized using the Softmax function. Finally, the attention weights are normalized to improve the model's generalization ability. The calculated attention weights are used to weight v, generating a weighted feature representation. The weighted features are averaged along the attention head dimension to reduce model complexity. The aggregated features are rearranged back to their original spatial dimensions, resulting in four output matrices. These four output matrices are then convolved with 1x1 kernels (the number of kernels equals the number of channels in the input matrix) with a stride of 1, and then added to the output matrices of layers 10, 11, 12, and 13, respectively, outputting matrices of sizes 96x4x4, 192x4x4, 382x4x4, and 768x4x4.

[0072] The sixteenth layer is the network's feedforward layer, receiving the output matrix from the fifteenth layer. It contains matrices of sizes 96*4*4, 192*4*4, 382*4*4, and 768*4*4. Taking one input matrix as an example, the 96*4*4 matrix is ​​passed through a 1*1 convolutional layer with the number of kernels equal to the number of channels multiplied by 5.3 (i.e., 510). This maps the feature channels of the input tensor to a higher-dimensional hidden feature space, generating two feature maps, resulting in two 510*4*4 matrices. Next, depthwise convolutions are performed on the two feature maps, using 3*3 and 5*5 kernels respectively. The number of kernels is equal to the original number of channels divided by 2, with a stride of 1 for both. SAME padding is used, and a linear rectified function is used as the activation function. Finally, the feature maps after depthwise convolution and activation are concatenated, connecting the two feature maps along the channel dimension to obtain a richer feature representation, resulting in a 510*4*4 matrix. The input feature map is then passed through a 1x1 convolutional layer with 96 kernels and a stride of 1. Finally, the projected features are input into the global spatial attention module. This module first reduces the spatial dimension of the feature map to 1 using adaptive average pooling, and then performs spatial attention calculation using a 1D convolutional kernel. The resulting attention weights are normalized using the sigmoid function. These normalized attention weights are then multiplied by the input feature map to achieve global feature weighting, and then residually connected to the original input to preserve the original feature information, resulting in a 96x4x4 matrix. The other three matrices are passed in the same way, ultimately outputting matrices of sizes 96x4x4, 192x4x4, 382x4x4, and 768x4x4. The seventeenth layer is the feature resolution restoration layer. It receives a 96*4*4 matrix from the output of the sixteenth layer, performs bilinear interpolation sampling to improve image resolution, and outputs a 96*64*64 matrix. Then, it passes through a convolutional layer with a 1*1 kernel, 96 kernels, a stride of 1, and uses a linear rectified function as the activation function. The result is added to the output of the third layer, and the output is a 96*4*4 matrix.

[0073] The eighteenth layer is the feature resolution restoration layer. It receives a 192*4*4 matrix from the output of the sixteenth layer, performs bilinear interpolation sampling to improve image resolution, and outputs a 192*32*32 matrix. Then, it passes through a convolutional layer with a 1*1 kernel, 192 kernels, a stride of 1, and uses a linear rectified function as the activation function. The result is added to the output of the fifth layer, and the output is a 192*32*32 matrix.

[0074] The nineteenth layer is the feature resolution restoration layer. It receives a 384*4*4 matrix from the output of the sixteenth layer, performs bilinear interpolation sampling to improve image resolution, and outputs a 384*16*16 matrix. Then, it passes through a convolutional layer with 1*1 kernels, 384 kernels, a stride of 1, and uses a linear rectified function as the activation function. The result is added to the output of the seventh layer, and the output is a 384*16*16 matrix.

[0075] The twentieth layer is the feature resolution restoration layer. It receives a 768*4*4 matrix from the output of the sixteenth layer, first performs bilinear interpolation sampling to improve image resolution, outputting a 768*8*8 matrix. Then, it passes through a convolutional layer with 1*1 kernels, 768 kernels, a stride of 1, and uses a linear rectified function as the activation function. This layer is then added to the output of the ninth layer, outputting a 768*8*8 matrix.

[0076] The 21st layer is an upsampling layer that receives the output of the 20th layer. First, the input matrix is ​​transformed to obtain a 64*768 matrix, and then linearly transformed to output a 256*384 matrix.

[0077] Layer 22 is a spatially gated upsampling layer that receives the outputs of layers 21 and 7. First, the output of layer 7 is dimensionally transformed to obtain a 256*384 matrix, which is then added to the output of layer 21. Next, the matrix is ​​dimensionally transformed again, outputting a 384*16*16 matrix, which is then fed into a layer normalization layer for standardization to reduce internal covariate shift. This is then fed into the spatial attention module, passing through a 1*1 convolutional layer with 384 kernels, a stride of 1, and a Gaussian error linear activation function. Finally, it is fed into the spatial gating unit. This unit models the input spatial information and generates a spatial attention mask or weights. The spatial gating unit comprises: a 5x5 convolutional layer with 384 kernels and 384 groups, padded using the SAME method; a 7x7 convolutional layer with 384 kernels and 384 groups, padded using the SAME method with a stride of 3; and a 1x1 convolutional layer with 384 kernels and a stride of 1. The output of the spatial gating unit is multiplied by the input to obtain a matrix of size 384x16x16, which is then passed through a 1x1 convolutional layer with 384 kernels and a stride of 1. Finally, the matrix is ​​added to the input of the spatial attention module. Finally, the perceptron layer consists of one 1x1 convolutional layer with 1536 kernels and a stride of 1; one 3x3 convolutional layer with 1536 kernels and a stride of 1, padded with SAME and passed through a Gaussian error linear activation function; and one 1x1 convolutional layer with 384 kernels and a stride of 1. A 1024x192 matrix is ​​then output through a linear transformation.

[0078] Layer 23 is a spatially gated upsampling layer that receives the outputs of layers 22 and 5. First, the output of layer 5 undergoes a dimensionality transformation, resulting in a 1024*192 matrix, which is then added to the output of layer 21. Next, the matrix undergoes a dimensionality transformation, outputting a 192*32*32 matrix, which is then fed into a layer normalization layer for standardization to reduce internal covariate shift. This matrix is ​​then fed into the spatial attention module, passing through a 1*1 convolutional layer with 192 kernels, a stride of 1, and a Gaussian error linear activation function. Finally, it is fed into the spatial gating unit. This unit models the spatial information of the input and generates a spatial attention mask or weights. The spatial gating unit comprises: a 5x5 convolutional layer with 192 kernels and 192 groups, padded using the SAME method; a 7x7 convolutional layer with 192 kernels and 192 groups, padded using the SAME method with a stride of 3; and a 1x1 convolutional layer with 192 kernels and a stride of 1. The output of the spatial gating unit is multiplied by the input to obtain a matrix of size 192x32x32, which is then passed through a 1x1 convolutional layer with 192 kernels and a stride of 1. Finally, this matrix is ​​added to the input of the spatial attention module. Finally, the perceptron layer consists of one 1x1 convolutional layer with 768 kernels and a stride of 1; one 3x3 convolutional layer with 768 kernels and a stride of 1, padded with SAME and activated by a Gaussian error linear activation function; and one 1x1 convolutional layer with 192 kernels and a stride of 1. A linear transformation then outputs a 4096x96 matrix.

[0079] Layer 24 is a spatially gated upsampling layer, receiving the outputs of layers 23 and 3. First, the output of layer 5 undergoes a dimensionality transformation, resulting in a 4096*96 matrix, which is then added to the output of layer 21. Next, the matrix undergoes another dimensionality transformation, outputting a 96*64*64 matrix, which is then fed into a layer normalization layer for standardization to reduce internal covariate shift. This is then fed into the spatial attention module, passing through a 1*1 convolutional layer with 96 kernels, a stride of 1, and a Gaussian error linear activation function. Finally, it is fed into the spatial gating unit. This unit models the input spatial information and generates a spatial attention mask or weights. The spatial gating unit comprises: a 5x5 convolutional layer with 96 kernels and 96 groups, padded using the SAME method; a 7x7 convolutional layer with 96 kernels and 96 groups, padded using the SAME method with a stride of 3; and a 1x1 convolutional layer with 96 kernels and a stride of 1. The output of the spatial gating unit is multiplied by the input to obtain a matrix of size 96x64x64, which is then passed through a 1x1 convolutional layer with 96 kernels and a stride of 1. Finally, the matrix is ​​added to the input of the spatial attention module. Finally, the perceptron layer consists of one 1x1 convolutional layer with 384 kernels and a stride of 1; one 3x3 convolutional layer with 384 kernels and a stride of 1, padded with SAME and activated by a Gaussian error linear activation function; and one 1x1 convolutional layer with 96 kernels and a stride of 1. A linear transformation then outputs a 4096x96 matrix.

[0080] The 25th layer is the output layer, receiving the output of the 24th layer. First, the input matrix is ​​transformed linearly into a 4096*1536 matrix, then dimensionally transformed into a 96*256*256 matrix. This matrix is ​​then passed through a 1*1 convolutional layer with 3 kernels and a stride of 1. The final output is a 3*256*256 segmentation result.

[0081] Preferably, the neural network is trained through the following steps:

[0082] (2-1) Obtain the maternal and fetal ultrasound image dataset during labor and divide it into training and test sets in an 8:2 ratio.

[0083] (2-2) Send the intrapartum maternal and fetal ultrasound image dataset obtained in step (2-1) to the ultrasound physician for manual annotation to obtain the annotated training set data.

[0084] (2-3) Data augmentation is performed on the training set of the labeled intrapartum maternal and fetal ultrasound image dataset obtained in step (2-2), including random image rotation, with a rotation range between -30 degrees and 30 degrees, random horizontal flipping, and pixel value changes, to obtain a data-augmented intrapartum maternal and fetal ultrasound image training set to improve the robustness of the network.

[0085] (2-4) The augmented intrapartum fetal ultrasound image training set obtained in step (2-3) is then standardized and normalized to obtain a preprocessed intrapartum fetal ultrasound image training set, ensuring the stability of the training neural network.

[0086] (2-5) Input the preprocessed intrapartum maternal and fetal ultrasound images from the training set obtained in step (2-4) into the neural network to obtain the corresponding inference output results. Input the inference output results and the intrapartum maternal and fetal ultrasound images annotated by the sonographer in step (2-2) into the segmentation Dice loss function of the neural network to obtain the loss value.

[0087] (2-6) The loss function of the neural network is optimized using the adaptive moment estimator and the loss value obtained in step (2-5) to iteratively update the parameters in the neural network.

[0088] (2-7) Repeat steps (2-5) and (2-6) above until the neural network converges to the optimal state, thus obtaining a trained neural network.

[0089] S4. Output the segmentation results of the pubic symphysis and the fetal head.

[0090] All experiments in this invention were conducted within the PyTorch framework using an NVIDIA GeForce RTX 3090 graphics card. The AdamW optimizer was used in the neural network with an initial learning rate of 0.001 and weight decay of 0.00001. In the early stages of training, a larger learning rate leads to faster model convergence, while in the later stages, it can cause model oscillations and difficulty in convergence. To stabilize the convergence of the model's weight parameters, a cosine annealing decay strategy was used. The training batch size was 4. Due to the limited data volume, data augmentation techniques such as rotation, flipping, and cropping were used during training. To facilitate subsequent adjustments, the training loss value was recorded throughout the training process, and the optimal model was selected after training. Furthermore, this model does not require specific dimensions for the input ultrasound image; the input image is automatically reshaped to a 256×256 pixel size during neural network preprocessing.

[0091] In step S4, the segmentation accuracy is monitored based on the segmentation results, such as... Figure 2 The diagram shows a segmentation illustration of the fetal head and pubic symphysis. The Dice coefficient and mean surface distance (Asd) are important metrics for evaluating the segmentation results. The Dice coefficient is more sensitive to the internal filling of the segment, and a higher Dice coefficient is better. The mean surface distance (Asd) is more sensitive to the segmentation boundaries, and a lower mean surface distance (Asd) is better. The commonly used segmentation model UNet has an overall segmentation Dice coefficient of 91.57% and a mean surface distance (Asd) of 2.97; while the pubic symphysis segmentation Dice coefficient is 84.69% and the mean surface distance (Asd) is 2.07. In contrast, this small-target accurate segmentation method has an overall segmentation Dice coefficient of 94.94% and a mean surface distance (Asd) of 1.95; and a pubic symphysis segmentation Dice coefficient of 88.47% and a mean surface distance (Asd) of 1.49. It can be seen that the small-target accurate segmentation method used in this invention can not only improve the overall segmentation accuracy of maternal-fetal ultrasound images but also improve the segmentation accuracy of small targets (pubic symphysis).

[0092] In summary, this embodiment provides a method for accurate segmentation of small targets in maternal and fetal ultrasound images during labor, which can achieve high-precision segmentation of the fetal head and pubic symphysis and has strong reliability.

Claims

1. A method for accurate segmentation of small targets in maternal-fetal ultrasound images during childbirth, characterized in that, Includes the following steps: S1. Acquire transperineal ultrasound images and label the fetal head and pubic symphysis; S2. Further screen the ultrasound images and select those with roughly clear outlines of the fetal head and pubic symphysis. S3. Input the image obtained in S2 into the trained neural network, which includes a displacement extraction feature layer, a spatial and channel information extraction layer, and a spatial gated upsampling layer. S4. Output the segmentation results including the pubic symphysis and the fetal head.

2. The method for accurate segmentation of small targets in intrapartum maternal-fetal ultrasound images according to claim 1, characterized in that, In step S1, the image data comes from intrapartum maternal-fetal ultrasound images with different relative positions of the pubic symphysis and the fetal head.

3. A method for accurate segmentation of small targets in intrapartum maternal-fetal ultrasound images according to any one of claims 1-2, characterized in that, In step S3, the neural network model structure includes a displacement extraction feature layer, a spatial and channel information extraction layer, and a spatially gated upsampling layer, which enables the neural network to segment small targets with high precision and finally output accurate segmentation results of the fetal head and pubic symphysis.

4. The method for accurate segmentation of small targets in intrapartum maternal-fetal ultrasound images according to claim 3, characterized in that, The neural network encoder employs a displacement extraction feature layer, which reduces the influence of irrelevant background pixels on target pixels by performing up, down, left, and right displacement operations on the features of different channels, thereby improving the feature extraction of small targets in maternal and fetal ultrasound images.

5. The method for accurate segmentation of small targets in intrapartum maternal-fetal ultrasound images according to claim 3, characterized in that, The deep network structure of the neural network adopts spatial and channel information modeling. By modeling information in different dimensions, the semantic difference between the background and the target can be enhanced, the false alarms of small target pixels can be reduced, and the accuracy of small target segmentation in maternal and fetal ultrasound images can be improved.

6. The method for accurate segmentation of small targets in intrapartum maternal-fetal ultrasound images according to claim 3, characterized in that, The upsampling layer of the neural network structure employs spatial gating, which utilizes convolutions with different kernel sizes to form spatial weights, thereby better representing the features of the neural network and enhancing the segmentation accuracy of small targets.

Images